
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2017-01-16 17:30 byvstinner, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| unicode_allow_kw.patch | vstinner,2017-01-16 20:00 | review | ||
| ac_more_fastcalls.patch | methane,2017-01-17 00:09 | review | ||
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 1886 | merged | vstinner,2017-05-31 14:39 | |
| Messages (20) | |||
|---|---|---|---|
| msg285574 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-01-16 17:30 | |
Changes27dc9a1c061e and01b06ca45f64 converted the str (Unicode) methods to Argument Clinic, cool! But str methods taking more than one argument use positional-only arguments.Currently, Argument Clinic doesn't use METH_FASTCALL for these methods, but METH_VARARGS.There are two options:* Allow passing arguments as keywoards: str.replace(old='a', new='b')* Enhance Argument Clinic to use also METH_FASTCALL for functions using positional-only functionsThe goal is to speedup method calls. Example with str.replace():$ ./python-patch -m perf timeit '"a".replace("x", "y")' --duplicate=100 --compare-to ./python-refpython-ref: ..................... 132 ns +- 1 nspython-patch: ..................... 102 ns +- 2 nsMedian +- std dev: [python-ref] 132 ns +- 1 ns -> [python-patch] 102 ns +- 2 ns: 1.30x faster (-23%) | |||
| msg285575 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-01-16 17:31 | |
Background: see also the old issue#17170 closed as rejected: "string method lookup is too slow". | |||
| msg285576 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-01-16 17:32 | |
The issue#29263 "Implement LOAD_METHOD/CALL_METHOD for C functions" should also optimize C methods even more. | |||
| msg285578 -(view) | Author: Yury Selivanov (yselivanov)*![]() | Date: 2017-01-16 17:54 | |
> * Allow passing arguments as keywoards: str.replace(old='a', new='b')I think Guido explicitly stated that he doesn't like the idea to always allow keyword arguments for all methods. I.e. `str.find('aaa')` just reads better than `str.find(needle='aaa')`. Essentially, the idea is that for most of the builtins that accept one or two arguments, positional-only parameters are better.> * Enhance Argument Clinic to use also METH_FASTCALL for functions using positional-only functionsSo this is the option to go with. | |||
| msg285581 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2017-01-16 18:52 | |
What is python-patch? | |||
| msg285584 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-01-16 20:00 | |
> What is python-patch?It's Python patched with attached unicode_allow_kw.patch: allow to pass parameters as keywords for Unicode methods. | |||
| msg285587 -(view) | Author: Inada Naoki (methane)*![]() | Date: 2017-01-16 23:24 | |
Parsing positional arguments for METH_KEYWORDS and METH_FASTCALL can be faster byhttp://bugs.python.org/issue29029 | |||
| msg285594 -(view) | Author: Inada Naoki (methane)*![]() | Date: 2017-01-17 00:09 | |
This patch makes AC produces more FASTCALL instead of VARARGS.When looking AC output, one downside is it produces many consts like:+ static const char * const _keywords[] = {"", "", "", "", "", NULL}; | |||
| msg285595 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-01-17 01:10 | |
ac_more_fastcalls.patch: Hum, I guess that your change on _PyStack_UnpackDict() is related to a bug in the function prototype. The function is unable to report failure if args is NULL. It changed the API in the change38ab8572fde2. | |||
| msg285596 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2017-01-17 01:17 | |
New changesetd07fd6e6d449 by Victor Stinner in branch 'default':Rename _PyArg_ParseStack to _PyArg_ParseStackAndKeywordshttps://hg.python.org/cpython/rev/d07fd6e6d449New changeset01c57ef1b651 by Victor Stinner in branch 'default':Add _PyArg_ParseStack() helper functionhttps://hg.python.org/cpython/rev/01c57ef1b651New changeset8bfec37ea86a by Victor Stinner in branch 'default':Add _PyArg_NoStackKeywords() helper functionhttps://hg.python.org/cpython/rev/8bfec37ea86aNew changeset38ab8572fde2 by Victor Stinner in branch 'default':_PyStack_UnpackDict() now returns -1 on errorhttps://hg.python.org/cpython/rev/38ab8572fde2New changesetde90f3d06bc9 by Victor Stinner in branch 'default':Argument Clinic: Use METH_FASTCALL for positionalshttps://hg.python.org/cpython/rev/de90f3d06bc9New changesetdea8922751a1 by Victor Stinner in branch 'default':Run Argument Clinic: METH_VARARGS=>METH_FASTCALLhttps://hg.python.org/cpython/rev/dea8922751a1 | |||
| msg285597 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-01-17 01:37 | |
Naoki> This patch makes AC produces more FASTCALL instead of VARARGS.Oh, funny, I wrote the same patch :-) (almost)> When looking AC output, one downside is it produces many consts like:> + static const char * const _keywords[] = {"", "", "", "", "", NULL};Yeah, I got the same result. I tried to hack a _PyArg_ParseStack() function which doesn't take keyword argments (no kwnames), but I lost my mind in parser_init().So I decided to simply rebase my old patches from my random CPython fastcall forks to add a simple _PyArg_ParseStack(). My implementation doesn't use the super-fast _PyArg_Parser object, but at least it allows to use METH_FASTCALL. Compared to what we had previously (METH_VARARGS), it is an enhancement :-)I prefer to move step by step, since each commit is already big enough. It's also easier for me to maintain my forks if I push more changes upstream.So there is still room for improvement in _PyArg_ParseStack(), but I suggest to defer further optimizations to a new issue. | |||
| msg285599 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2017-01-17 01:48 | |
New changeset3bf78c286daf by Victor Stinner in branch 'default':Add _PyArg_UnpackStack() function helperhttps://hg.python.org/cpython/rev/3bf78c286dafNew changeset905e902bd47e by Victor Stinner in branch 'default':Argument Clinic: Use METH_FASTCALL for boring positionalshttps://hg.python.org/cpython/rev/905e902bd47eNew changeset52acda52b353 by Victor Stinner in branch 'default':Run Argument Clinic: METH_VARARGS=>METH_FASTCALLhttps://hg.python.org/cpython/rev/52acda52b353 | |||
| msg285600 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2017-01-17 02:25 | |
New changesetd07fd6e6d449 by Victor Stinner in branch 'default':Rename _PyArg_ParseStack to _PyArg_ParseStackAndKeywordshttps://hg.python.org/cpython/rev/d07fd6e6d449New changeset01c57ef1b651 by Victor Stinner in branch 'default':Add _PyArg_ParseStack() helper functionhttps://hg.python.org/cpython/rev/01c57ef1b651New changeset8bfec37ea86a by Victor Stinner in branch 'default':Add _PyArg_NoStackKeywords() helper functionhttps://hg.python.org/cpython/rev/8bfec37ea86aNew changeset38ab8572fde2 by Victor Stinner in branch 'default':_PyStack_UnpackDict() now returns -1 on errorhttps://hg.python.org/cpython/rev/38ab8572fde2New changesetde90f3d06bc9 by Victor Stinner in branch 'default':Argument Clinic: Use METH_FASTCALL for positionalshttps://hg.python.org/cpython/rev/de90f3d06bc9New changesetdea8922751a1 by Victor Stinner in branch 'default':Run Argument Clinic: METH_VARARGS=>METH_FASTCALLhttps://hg.python.org/cpython/rev/dea8922751a1 | |||
| msg285614 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2017-01-17 06:24 | |
Oh, I have wrote almost the same patch before going to sleep yesteday! ;) But the building crashed (likely due to a bug in _PyStack_UnpackDict()) and it was too late to resolve this.I would prefer to rename "l" to "nargs" in PyArg_UnpackStack_impl and don't use upper case and namespace prefix in static functions. | |||
| msg286647 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-02-01 16:36 | |
Recently, I saw complains that I push changes too fast without reviews. This issue is a good example. So let me elaborate myself a little bit on these changes.I'm not really proud of the "Rename _PyArg_ParseStack to _PyArg_ParseStackAndKeywords" change. It should be called _PyArg_ParseStackAndKeywords() from the beginning, but when I designed FASTCALL, my plan was to only check arguments (positional and keyword arguments) in the callee, and remove checks in the caller. I expected that all functions will get positional + keyword arguments.I didn't know that not accepting keywords but only positional arguments like in str.replace(), was a deliberate language design choice.So yes, for functions accepting only positional arguments, _PyArg_ParseStack() + _PyArg_NoStackKeywords() makes sense.IMHO the implementation of the new _PyArg_ParseStack() and _PyArg_NoStackKeywords() functions was simple enough to avoid a review, they reuse existing code with minimum changes.In general for FASTCALL, I only add private functions. I consider that it's ok to push code quickly, since the private property allows us to change these functions anytime.Again, I consider that the "Argument Clinic: Use METH_FASTCALL for positionals" change is simple enough to avoid a review, I don't think that they are many ways to use FASTCALL for functions using positional-only parameters. I mean, I only see a single way to implement it. Again, if someone sees a new way to handle such parameters, we can change it anytime, Argument Clinic and FASTCALL are stil private.More generally, FASTCALL requires a lot of tiny changes everywhere.GitHub will allow to get reviews on long patch series. It will be easier to handle large changes. | |||
| msg286648 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2017-02-01 16:42 | |
New changeset758674087b12 by Victor Stinner in branch 'default':Issue#29286: Rename private PyArg_UnpackStack_impl() to unpack_stack()https://hg.python.org/cpython/rev/758674087b12 | |||
| msg286649 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-02-01 16:43 | |
Serhiy Storchaka: "Oh, I have wrote almost the same patch before going to sleep yesteday! ;) But the building crashed (likely due to a bug in _PyStack_UnpackDict()) and it was too late to resolve this."Oh, sorry that you wrote almost the same code. Well, at least it means that we agree on the design :-)Serhiy: "I would prefer to rename "l" to "nargs" in PyArg_UnpackStack_impl and don't use upper case and namespace prefix in static functions."Done. | |||
| msg286650 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-02-01 16:45 | |
str type still has a few methods using METH_VARARGS (slower than METH_FASTCALL):* count()* find()* index()* rfind()* rindex()* startswith()* endswith()* format()Maybe I will propose a change later to convert these methods to Argument Clinic, but I consider that the initial issue "Currently, Argument Clinic doesn't use METH_FASTCALL for these methods, but METH_VARARGS" is now fixed, so I close the issue.Thanks Naoki and Serhiy. | |||
| msg286653 -(view) | Author: Roundup Robot (python-dev)![]() | Date: | |
New changeset5910fd7231d34363798d2815be2f66909e638d1c by Victor Stinner in branch 'master':Issue#29286: Rename private PyArg_UnpackStack_impl() to unpack_stack()https://github.com/python/cpython/commit/5910fd7231d34363798d2815be2f66909e638d1c | |||
| msg295516 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-06-09 11:24 | |
New changesetf0ff849adc6b4a01f9d1f08d9ad0f1511ff84541 by Victor Stinner in branch '3.6':bpo-30524: Fix _PyStack_UnpackDict() (#1886)https://github.com/python/cpython/commit/f0ff849adc6b4a01f9d1f08d9ad0f1511ff84541 | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:42 | admin | set | github: 73472 |
| 2017-06-09 11:24:56 | vstinner | set | messages: +msg295516 |
| 2017-05-31 14:39:28 | vstinner | set | pull_requests: +pull_request1964 |
| 2017-02-01 17:00:21 | python-dev | set | stage: resolved |
| 2017-02-01 17:00:20 | python-dev | set | messages: +msg286653 |
| 2017-02-01 16:45:08 | vstinner | set | status: open -> closed resolution: fixed messages: +msg286650 |
| 2017-02-01 16:43:58 | vstinner | set | messages: +msg286649 |
| 2017-02-01 16:42:56 | python-dev | set | messages: +msg286648 |
| 2017-02-01 16:36:30 | vstinner | set | messages: +msg286647 |
| 2017-01-17 06:24:01 | serhiy.storchaka | set | messages: +msg285614 |
| 2017-01-17 02:25:18 | python-dev | set | messages: +msg285600 |
| 2017-01-17 01:48:49 | python-dev | set | messages: +msg285599 |
| 2017-01-17 01:37:53 | vstinner | set | messages: +msg285597 |
| 2017-01-17 01:17:09 | python-dev | set | nosy: +python-dev messages: +msg285596 |
| 2017-01-17 01:10:05 | vstinner | set | messages: +msg285595 |
| 2017-01-17 00:09:26 | methane | set | files: +ac_more_fastcalls.patch messages: +msg285594 |
| 2017-01-16 23:24:01 | methane | set | messages: +msg285587 |
| 2017-01-16 20:00:59 | vstinner | set | files: +unicode_allow_kw.patch keywords: +patch messages: +msg285584 |
| 2017-01-16 18:52:37 | serhiy.storchaka | set | messages: +msg285581 |
| 2017-01-16 17:54:46 | yselivanov | set | nosy: +yselivanov messages: +msg285578 |
| 2017-01-16 17:32:52 | vstinner | set | messages: +msg285576 |
| 2017-01-16 17:31:49 | vstinner | set | messages: +msg285575 |
| 2017-01-16 17:30:51 | vstinner | create | |