Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
gh-97933: add opcode for more efficient comprehension execution#101310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
carljm commentedJan 25, 2023
@markshannon or@iritkatriel (or anyone else from faster-cpython team) -- if you could kick off a PyPerformance run with your fancy Github Action runner, I'd be much obliged! |
iritkatriel commentedJan 25, 2023
Done. |
iritkatriel commentedJan 25, 2023
Benchmark results are here:https://github.com/faster-cpython/benchmarking/blob/main/results/bm-20230125-3.12.0a4+-ee2ad56/bm-20230125-linux-x86_64-carljm-inlinecomp-3.12.0a4+-ee2ad56-vs-base.md (shows no difference overall). |
carljm commentedJan 25, 2023
I can't see the detailed results (private repo), but if there's no impact overall then I guess they aren't that interesting :) We must not have any benchmarks that make heavy use of comprehensions. |
markshannon commentedJan 25, 2023
All benchmarks:
Benchmark hidden because not significant (33): aiohttp, async_tree_none, async_tree_cpu_io_mixed, async_tree_io, chameleon, bench_mp_pool, coverage, crypto_pyaes, dask, deepcopy, deltablue, django_template, djangocms, dulwich_log, genshi_text, genshi_xml, hexiom, html5lib, meteor_contest, mypy, nbody, nqueens, pprint_safe_repr, pprint_pformat, scimark_lu, scimark_sor, sqlglot_parse, sqlglot_transpile, sqlite_synth, sympy_str, unpickle_list, xml_etree_parse, xml_etree_generate |
* main: (225 commits)pythongh-102056: Fix a few bugs in error handling of exception printing code (python#102078)pythongh-102011: use sys.exception() instead of sys.exc_info() in docs where possible (python#102012)pythongh-101566: Sync with zipp 3.14. (pythonGH-102018)pythonGH-99818: improve the documentation for zipfile.Path and Traversable (pythonGH-101589)pythongh-88233: zipfile: handle extras after a zip64 extra (pythonGH-96161)pythongh-101981: Apply HOMEBREW related environment variables (pythongh-102074)pythongh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (pythonGH-101912)pythongh-101819: Adapt _io types to heap types, batch 1 (pythonGH-101949)pythongh-101981: Build macOS as recommended by the devguide (pythonGH-102070)pythongh-97786: Fix compiler warnings in pytime.c (python#101826)pythongh-101578: Amend PyErr_{Set,Get}RaisedException docs (python#101962) Misc improvements to the float tutorial (pythonGH-102052)pythongh-85417: Clarify behaviour on branch cuts in cmath module (python#102046)pythongh-100425: Update tutorial docs related to sum() accuracy (FH-101854) Add missing 'is' to `cmath.log()` docstring (python#102049)pythongh-100210: Correct the comment link for unescaping HTML (python#100212)pythongh-97930: Also include subdirectory in makefile. (python#102030)pythongh-99735: Use required=True in argparse subparsers example (python#100927) Fix incorrectly documented attribute in csv docs (python#101250)pythonGH-84783: Make the slice object hashable (pythonGH-101264) ...
carljm commentedMar 8, 2023
Closing this for now in favor of#101441 May reopen this approach if PEP 709 (implemented by that PR) is rejected. |
Uh oh!
There was an error while loading.Please reload this page.
This avoids allocating a throwaway single-use function object every time we run a comprehension. Otherwise it shouldn't have any user-visible impact; the comprehension is still a separate code object and runs in its own frame, just as before. Tracebacks look the same, etc. We just have a new
COMPREHENSIONopcode that builds a frame directly from codeobject and optional closure and inline-calls it, without creating a function object.
In a micro-benchmark of comprehension execution time, this looks like it saves about 25%:
Currently this doesn't handle async comprehensions or generator expressions; those still create a function object and
CALLit. In principle I think they could be handled as well, but to keep the PR smaller I'll defer that to a second PR if this one is merged.Closes#97933.