@brandtbucher@markshannon: In bytecodes.c I now get red wiggles on every use of a variable defined through a stack effect (since thePyObject *value; (etc.) is not in the instruction body any more). It's particularly annoying when it occurs in a macro likePy_DECREF(value) -- the wiggle shows onPy_DECREF. Any suggestions?

@markshannon I have some questions about howERROR_IF() should work. Your spec says "If anERROR_IF occurs, all values will be removed from the stack." It's easy enough to add aSTACK_SHRINK() call (see latest code in generated_cases.c.h), but I'm not sure about whose responsibility it should be toDECREF() those variables.In practice, in the dozen or so instructions I've converted so far, whenERROR_IF() is called the code has already calledDECREF(). Example definition:

        instr(UNARY_POSITIVE, (value -- res)) {            res = PyNumber_Positive(value);            Py_DECREF(value);            ERROR_IF(res == NULL, error);        }

This expands to the following:

        TARGET(UNARY_POSITIVE) {            PyObject *value = PEEK(1);            PyObject *res;            res = PyNumber_Positive(value);            Py_DECREF(value);            if (res == NULL) { STACK_SHRINK(1); goto error; }            POKE(1, res);            DISPATCH();        }

Shall we make this part of the spec forERROR_IF(), that you must call it after "consuming" all the inputs?

Perhaps we could makeinst a variadic macro that turns a list of stack items into a declaration list? Downside is that you need some marker to separate in/out parameters, and handling braces might get trickier.

On my phone now, but something like:

typedefPyObject*_dummy_stack_item;#defineinst_begin(NAME, ...)          \    case (NAME): {                     \        _dummy_stack_item __VA_ARGS__;#defineinst_end }inst_begin(BINARY_OP,lhs,rhs,_,res)// Implementation goes here...inst_end

Some notes after converting a few basic instructions (and failing to convert a few outliers).

• Converting "raw" instruction definitions (with PUSH()/POP() or custom stack operations) to streamlined DSL with input and output stack effects is a slow manual process that requires careful review (e.g. the mistake that Irit found).
• So far, every few instructions I converted required changes to the code generator.
• The code generator needs to be refactored to make future changes easier.
• I haven't even started to think about how to implement array and conditional stack effects or cache streams.
• The families are currently not read by the code generator. We can address this once we need them.
• For some reason I no longer see red wiggly underlines in bytecodes.c. (EDIT: After closing and reopening the file they are back.)
• There are some instructions that don't seem to fit in the DSL.
  • PUSH_NULL must wait until I've implemented types, since it pushes a NULL.
  • LIST_APPEND and SET_ADD dig up a stack entry that occurs 'oparg' deep.
  • Opcodes like BINARY_SUBSCR_ADAPTIVE are problematic since they have special exits (DISPATCH_SAME_OPARG and GO_TO_INSTRUCTION). We may have to rethink such exits.

At this point I think the way forward is to merge this and then iterate, leaving the hardest cases for last.

I have benchmark results (thanks@brandtbucher!). Bottom line, it's a wash.

We compared three commits:

• baseline (just before bytecodes.c &c created):2cfcaf5_io and_pyio inconsistenly raiseValueErrors on closed files #98999: RaiseValueError in_pyio on closed buffers (gh-98999: RaiseValueError in_pyio on closed buffers #99009)
• first improvement in bytecodes.c by@markshannon:f4adb97Enable specialization of iteration over generators,for gen():, and awaiting coroutines,await coro() #96793: Implement PEP 479 in bytecode. (GH-96793: Implement PEP 479 in bytecode. #99006)
• last commit in this PR:48c2a20 (HEAD -> stack-effect, gvanrossum/stack-effect) Add dummy variables for stack effects

The second and third both are 1% faster than the baseline. This suggests that there is no measurable effect from just this PR, or from the creation of super-instructions (which was merged into this commit from main, but not included in Mark's PEP 479 changes).
output.txt

I am going ahead with merging this.