René's URL Explorer Experiment

Title: gh-97933: inline list/dict/set comprehensions by carljm · Pull Request #101441 · python/cpython · GitHub

Open Graph Title: gh-97933: inline list/dict/set comprehensions by carljm · Pull Request #101441 · python/cpython

X Title: gh-97933: inline list/dict/set comprehensions by carljm · Pull Request #101441 · python/cpython

Description: Closes #97933. In builds with --enable-optimizations: ➜ ./python -m pyperf timeit -s 'l = [1, 2, 3, 4, 5]' '[x for x in l]' --compare-to ../mainopt/python /home/carljm/build/mainopt/python: ..................... 200 ns +- 3 ns /home/carljm/build/ic2opt/python: ..................... 120 ns +- 2 ns Mean +- std dev: [/home/carljm/build/mainopt/python] 200 ns +- 3 ns -> [/home/carljm/build/ic2opt/python] 120 ns +- 2 ns: 1.67x faster Credit to @vladima for authoring the original version of this code in Cinder 3.8. The compiler approach is different in this version (so as to support inlining all comprehensions, regardless of name clashes); some of the symbol table changes remain the same. A couple implementation notes: We need a new opcode LOAD_FAST_AND_CLEAR because we need to push/pop the value of possibly-undefined variables on the stack while clearing them. To do this with existing opcodes would require eliminating the invariant/assert that LOAD_FAST never loads NULL, and would add inefficient refcount operations and interpreter loop cycles. If a comprehension iteration variable is a cell inside the comprehension (i.e. the comprehension builds closures) we end up including that variable in both co_varnames and co_cellvars for the inlined-into outer function and using the non-offset (varnames) storage location for it. This is how function arguments that are also cells are currently handled, so we just piggyback on that handling. This makes the needed push/pop of the outer cell simpler as we can just use LOAD_FAST_AND_CLEAR/STORE_FAST as-is for it, and it will push/pop the cell itself. Otherwise we would need some new handling in the compiler for allowing push/pop of a cell in offset storage. This PR also adds a lot of new tests for comprehension scoping edge cases, and runs all new and existing scoping tests in module, function, and class scopes (without requiring duplication of the test code.) All comprehension scoping tests added in this diff also pass with comprehension inlining disabled (ensuring semantics did not change.) Issue: gh-97933

Open Graph Description: Closes #97933. In builds with --enable-optimizations: ➜ ./python -m pyperf timeit -s 'l = [1, 2, 3, 4, 5]' '[x for x in l]' --compare-to ../mainopt/python /home/carljm/build/mainopt...

X Description: Closes #97933. In builds with --enable-optimizations: ➜ ./python -m pyperf timeit -s 'l = [1, 2, 3, 4, 5]' '[x for x in l]' --compare-to ../mainopt/python /home/carl...

Opengraph URL: https://github.com/python/cpython/pull/101441

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width