Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Initial attempt at support for Python 3.13.#264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
kpfleming wants to merge6 commits intoastral-sh:mainfromkpfleming:python-3.13

Conversation

@kpfleming
Copy link
Contributor

No description provided.

pawamoy and AllanDaemon reacted with rocket emoji
@indygreg
Copy link
Collaborator

Thank you for this PR!

I glanced at this and it seems to have all the components for 3.13 support. Let's see what CI says...

@kpfleming
Copy link
ContributorAuthor

Once the CI is happy I'll rebase this and squash it down to one commit, and them mark it ready for review.

@kpfleming
Copy link
ContributorAuthor

The builds are failing because the patches forconfigure.ac from 3.12 won't apply, which was expected. Those patches will need to be regenerated for 3.13.

zanieb reacted with thumbs up emoji

@kpfleming
Copy link
ContributorAuthor

A bit more progress this morning. A couple of new patches, disabling one (for apple-cross support), but a local Linux build still fails, just much much farther into the process.

@kpfleming
Copy link
ContributorAuthor

Well, I think I've reached the end of what I can reasonably accomplish at this point. The current build failure (inModules/_curses_panel.c) points to some macro not being defined, or a header not being included, but I've looked over all the files involved and I can't see what is causing this to happen. The changes in this file between 3.12 and 3.13 are not drastic, so I can only assume something has changed in the CPython build system itself.

@zanieb
Copy link
Member

zanieb commentedMay 22, 2024•
edited
Loading

Suspicious:python/cpython#111567

Edit: Perhaps not actually, while it usescurses,curses_panel has been aroundfor a long time and hasn't changed much due to the REPL changes.

@zanieb
Copy link
Member

Here's thebuild changes in the release notes

@zanieb
Copy link
Member

I presume the newly bundled mimalloc is going to require some sort of handling. Still not quite sure what's causing this particular failure though.

@indygreg
Copy link
Collaborator

./Modules/clinic/_curses_panel.c.h:169:36: error: implicit declaration of function '_Py_SINGLETON' [-Werror=implicit-function-declaration]

_Py_SINGLETON is defined bypycore_global_objects.h. This is probably a missing#include.

python-build-standalone is likely seeing this error because we statically compile extension modules. This configuration isn't well tested by upstream CPython. So it's likely that nobody has noticed yet.

Since this source file was generated by clinic, it is possibly a bug in the clinic code generator, which may want to emit the missing#include. We should file an upstream issue and get@vstinner's eyeballs on it.

zanieb reacted with heart emoji

@ofek
Copy link

I'm very excited to start testing free-threading support after this is fixed and merged!

@kpfleming
Copy link
ContributorAuthor

Issue opened:python/cpython#119661

ofek reacted with thumbs up emoji

@kpfleming
Copy link
ContributorAuthor

In the interests of saving the GitHub Actions runners from running jobs that will fail anyway, I'm running local builds now :-)

The patch from Victor for the clinic modules was successful, those build properly now. The next problem is:

cpython-3.13> clang -pthread  -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -fdebug-default-version=4 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw   -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include -fdebug-default-version=4 -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw  -fPIC -DPy_BUILD_CORE_BUILTIN -c ./Modules/syslogmodule.c -o Modules/syslogmodule.ocpython-3.13> ./Modules/readline.c:1305:21: error: incompatible function pointer types assigning to 'Function *' (aka 'int (*)(const char *, int)') from 'int (void)' [-Wincompatible-function-pointer-types]cpython-3.13>  1305 |     rl_startcpython-3.13> up_hook = on_startup_hook;cpython-3.13>       |                     ^ ~~~~~~~~~~~~~~~cpython-3.13> 1 error generated.cpython-3.13> Makefile:3392: recipe for target 'Modules/readline.o' failed

I suspect that support for building againstlibedit may have regressed in 3.13.

@kpfleming
Copy link
ContributorAuthor

Turns out that one was easy to fix... I'm continuing onwards :-)

ofek reacted with heart emoji

@kpfleming
Copy link
ContributorAuthor

OK, new status report: the actual CPython build succeeds, but then the process stops becauseTools/scripts/run_tests.py does not exist in 3.13. I'll clean up and push what I've got so far, but this one will be harder to resolve.

@zanieb
Copy link
Member

run_tests.py was removed inpython/cpython#110156 as part ofpython/cpython#110152. In brief, the suggestion is:

Just run./python -m test --slow-ci,make buildbottest ormake test instead. Python test runner (regrtest) now handles cross-compilation and HOSTRUNNER.

@zanieb
Copy link
Member

Here's a hacky patch that gets to the next failure :)zanieb@805c76f

@kpfleming
Copy link
ContributorAuthor

I've just updated to 3.13.0b2, and removed the patch that was only needed until that release.

zanieb reacted with thumbs up emoji

@zanieb
Copy link
Member

zanieb commentedJun 6, 2024•
edited
Loading

The next problem I'm encountering is during validation:

validating dist/cpython-3.13.0b1-armv7-unknown-linux-gnueabihf-lto-20240605T2025.tar.zstError: errors found  error: extra/unknown extension module: _interpchannels  error: extra/unknown extension module: _interpqueues  error: extra/unknown extension module: _interpreters  error: extra/unknown extension module: _suggestions  error: extra/unknown extension module: _sysconfig  error: extra/unknown extension module: _testexternalinspection  error: missing extension module: _crypt  error: missing extension module: _xxinterpchannels  error: missing extension module: _xxsubinterpreters  error: missing extension module: audioop  error: missing extension module: ossaudiodev  error: missing extension module: spwd  error: PYTHON.json referenced object file not in tar archive: python/build/Modules/_testinternalcapi/pytime.o  error: PYTHON.json referenced object file not in tar archive: python/build/Modules/_testinternalcapi/set.o  error: PYTHON.json referenced object file not in tar archive: python/build/Modules/_testinternalcapi/test_critical_sections.o  error: PYTHON.json referenced object file not in tar archive: python/build/Modules/_testinternalcapi/test_lock.o

which is confusing because at least some of these look correct inextension-modules.yml

Edit: Resolved now.

@kpfleming
Copy link
ContributorAuthor

I left a FIXME invalidation.rs because it will need to be updated to have the proper list. This is why :)

zanieb reacted with thumbs up emoji

@zanieb
Copy link
Member

Great I'll take a swing at updating that.

@zanieb
Copy link
Member

zanieb commentedJun 6, 2024•
edited
Loading

Now I have a spicy failure on just the PGO builds

2024-06-06T14:41:13.8509515Z cpython-3.13> LD_LIBRARY_PATH=/build/Python-3.13.0b1 ./python -m test --pgo --timeout=2024-06-06T14:41:14.4721772Z cpython-3.13> Using random seed: 36305332372024-06-06T14:41:14.4722931Z cpython-3.13> 0:00:00 load avg: 2.53 Run 44 tests sequentially2024-06-06T14:41:14.4723987Z cpython-3.13> 0:00:00 load avg: 2.53 [ 1/44] test_array2024-06-06T14:41:15.9231051Z cpython-3.13> 0:00:01 load avg: 2.41 [ 2/44] test_base642024-06-06T14:41:18.6740282Z cpython-3.13> 0:00:04 load avg: 2.41 [ 3/44] test_binascii2024-06-06T14:41:18.7698062Z cpython-3.13> 0:00:04 load avg: 2.41 [ 4/44] test_binop2024-06-06T14:41:18.8463920Z cpython-3.13> 0:00:04 load avg: 2.41 [ 5/44] test_bisect2024-06-06T14:41:18.9465060Z cpython-3.13> 0:00:04 load avg: 2.41 [ 6/44] test_bytes2024-06-06T14:41:19.0936489Z cpython-3.13> 0:00:04 load avg: 2.41 [ 7/44] test_bz2 -- test_bytes skipped2024-06-06T14:41:22.1080773Z cpython-3.13> 0:00:07 load avg: 2.30 [ 8/44] test_cmath2024-06-06T14:41:22.2968922Z cpython-3.13> 0:00:07 load avg: 2.30 [ 9/44] test_codecs2024-06-06T14:41:23.5730666Z cpython-3.13> 0:00:09 load avg: 2.30 [10/44] test_collections2024-06-06T14:41:24.3534783Z cpython-3.13> 0:00:09 load avg: 2.30 [11/44] test_complex2024-06-06T14:41:24.6535516Z cpython-3.13> 0:00:10 load avg: 2.30 [12/44] test_dataclasses2024-06-06T14:41:25.2374900Z cpython-3.13> 0:00:10 load avg: 2.30 [13/44] test_datetime2024-06-06T14:41:29.2557137Z cpython-3.13> 0:00:14 load avg: 2.19 [14/44] test_decimal2024-06-06T14:41:32.8490327Z cpython-3.13> 0:00:18 load avg: 2.10 [15/44] test_difflib2024-06-06T14:41:33.7806220Z cpython-3.13> 0:00:19 load avg: 2.10 [16/44] test_embed2024-06-06T14:41:45.9800425Z cpython-3.13> Fatal Python error: Segmentation fault2024-06-06T14:41:45.9801940Z cpython-3.13> 2024-06-06T14:41:45.9803753Z cpython-3.13> Current thread 0x00007f1ac0fbd740 (most recent call first):2024-06-06T14:41:45.9805390Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/test_embed.py", line 1620 in test_global_pathconfig2024-06-06T14:41:45.9806989Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/case.py", line 606 in _callTestMethod2024-06-06T14:41:45.9808342Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/case.py", line 651 in run2024-06-06T14:41:45.9811098Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/case.py", line 707 in __call__2024-06-06T14:41:45.9812381Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/suite.py", line 122 in run2024-06-06T14:41:45.9813849Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/suite.py", line 84 in __call__2024-06-06T14:41:45.9815235Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/suite.py", line 122 in run2024-06-06T14:41:45.9816564Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/unittest/suite.py", line 84 in __call__2024-06-06T14:41:45.9818271Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/testresult.py", line 146 in run2024-06-06T14:41:45.9819125Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 57 in2024-06-06T14:41:45.9819693Z cpython-3.13> _run_suite2024-06-06T14:41:45.9820311Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 37 in run_unittest2024-06-06T14:41:45.9820929Z cpython-3.13>   File2024-06-06T14:41:45.9821502Z cpython-3.13> "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 132 in test_func2024-06-06T14:41:45.9822423Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 88 in regrtest_runner2024-06-06T14:41:45.9823610Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 135 in _load_run_test2024-06-06T14:41:45.9824601Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 178 in _runtest_env_changed_exc2024-06-06T14:41:45.9825561Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 278 in _runtest2024-06-06T14:41:45.9826480Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/single.py", line 309 in run_single_test2024-06-06T14:41:45.9827378Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/main.py", line 355 in run_test2024-06-06T14:41:45.9828320Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/main.py", line 389 in run_tests_sequentially2024-06-06T14:41:45.9829253Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/main.py", line 533 in _run_tests2024-06-06T14:41:45.9830257Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/main.py", line 568 in run_tests2024-06-06T14:41:45.9831155Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/main.py", line 731 in main2024-06-06T14:41:45.9831996Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/libregrtest/main.py", line 739 in main2024-06-06T14:41:45.9832875Z cpython-3.13>   File "/build/Python-3.13.0b1/Lib/test/__main__.py", line 2 in <module>2024-06-06T14:41:45.9833528Z cpython-3.13>   File "<frozen runpy>", line 88 in _run_code2024-06-06T14:41:45.9834110Z cpython-3.13>   File "<frozen runpy>", line 198 in _run_module_as_main2024-06-06T14:41:45.9834656Z cpython-3.13> Extension modules: _testinternalcapi2024-06-06T14:41:45.9835059Z cpython-3.13> (total: 1)2024-06-06T14:41:46.1367157Z cpython-3.13> Segmentation fault (core dumped)2024-06-06T14:41:46.1368488Z cpython-3.13> Makefile:868: recipe for target 'profile-bolt-stamp' failed2024-06-06T14:41:46.1369458Z cpython-3.13> make[1]: Leaving directory '/build/Python-3.13.0b1'2024-06-06T14:41:46.1370196Z cpython-3.13> make[1]: *** [profile-bolt-stamp] Error 1392024-06-06T14:41:46.1380350Z cpython-3.13> make: *** [bolt-opt] Error 22024-06-06T14:41:46.1381212Z cpython-3.13> Makefile:899: recipe for target 'bolt-opt' failed2024-06-06T14:41:48.0010999Z Traceback (most recent call last):2024-06-06T14:41:48.0021028Z   File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 1219, in <module>2024-06-06T14:41:48.0021814Z     sys.exit(main())2024-06-06T14:41:48.0022074Z              ^^^^^^2024-06-06T14:41:48.0023383Z   File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 1201, in main2024-06-06T14:41:48.0024527Z     build_cpython(2024-06-06T14:41:48.0025830Z   File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 803, in build_cpython2024-06-06T14:41:48.0027150Z     build_env.run("build-cpython.sh", environment=env)2024-06-06T14:41:48.0028525Z   File "/home/runner/work/python-build-standalone/python-build-standalone/pythonbuild/buildenv.py", line 95, in run2024-06-06T14:41:48.0029426Z     container_exec(self.container, program, user=user, environment=environment)2024-06-06T14:41:48.0030512Z   File "/home/runner/work/python-build-standalone/python-build-standalone/pythonbuild/docker.py", line 140, in container_exec2024-06-06T14:41:48.0031434Z     raise Exception("exit code %d from %s" % (inspect_res["ExitCode"], command))2024-06-06T14:41:48.0032032Z Exception: exit code 2 from /build/build-cpython.sh2024-06-06T14:41:48.0307350Z make: *** [Makefile:325: /home/runner/work/python-build-standalone/python-build-standalone/build/cpython-3.13.0b1-x86_64-unknown-linux-gnu-pgo.tar] Error 12024-06-06T14:41:48.0444158Z ##[error]Process completed with exit code 2.

https://github.com/python/cpython/blob/2268289a47c6e3c9a220b53697f9480ec390466f/Lib/test/test_embed.py#L1620

@zanieb
Copy link
Member

And this error on the MUSL builds

2024-06-06T15:22:59.7323527Z cpython-3.13> checking for the platform triplet based on compiler characteristics...2024-06-06T15:22:59.7534322Z cpython-3.13> x86_64-linux-musl2024-06-06T15:22:59.7544473Z cpython-3.13> checking for multiarch...2024-06-06T15:22:59.7721531Z cpython-3.13> configure: error: internal configure error for the platform triplet, please file a bug report2024-06-06T15:23:03.4816270Z Traceback (most recent call last):2024-06-06T15:23:03.4828543Z   File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 1219, in <module>2024-06-06T15:23:03.4829764Z     sys.exit(main())2024-06-06T15:23:03.4830118Z              ^^^^^^2024-06-06T15:23:03.4831310Z   File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 1201, in main2024-06-06T15:23:03.4832339Z     build_cpython(2024-06-06T15:23:03.4833609Z   File "/home/runner/work/python-build-standalone/python-build-standalone/cpython-unix/build.py", line 803, in build_cpython2024-06-06T15:23:03.4834795Z     build_env.run("build-cpython.sh", environment=env)2024-06-06T15:23:03.4835741Z   File "/home/runner/work/python-build-standalone/python-build-standalone/pythonbuild/buildenv.py", line 95, in run2024-06-06T15:23:03.4836594Z     container_exec(self.container, program, user=user, environment=environment)2024-06-06T15:23:03.4837611Z   File "/home/runner/work/python-build-standalone/python-build-standalone/pythonbuild/docker.py", line 140, in container_exec2024-06-06T15:23:03.4838520Z     raise Exception("exit code %d from %s" % (inspect_res["ExitCode"], command))2024-06-06T15:23:03.4839101Z Exception: exit code 1 from /build/build-cpython.sh2024-06-06T15:23:03.5124987Z make: *** [Makefile:325: /home/runner/work/python-build-standalone/python-build-standalone/build/cpython-3.13.0b1-x86_64_v3-unknown-linux-musl-debug.tar] Error 12024-06-06T15:23:03.5261193Z ##[error]Process completed with exit code 2.

https://github.com/python/cpython/blob/78634cfa3dd4b542897835d5f097604dbeb0f3fd/configure.ac#L1103

@kpfleming
Copy link
ContributorAuthor

A little more progress... the branch is now building 3.13.0rc1, and it appears that the PGO and MUSL issues are all that is left on Linux. There are still problems on Windows and macOS which are quite beyond my ability to troubleshoot.

AllanDaemon reacted with heart emoji

@zanieb
Copy link
Member

zanieb commentedSep 5, 2024•
edited
Loading

I got Windows building over inzanieb#7 — I condensed that into a single commitzanieb@2110d43

henryiii reacted with hooray emoji

@zanieb
Copy link
Member

zanieb commentedSep 5, 2024•
edited
Loading

I also got the 3.13 musl builds working over inzanieb#6

Now it's really just the PGO problems (and some complexity around the macOS target version)

@zanieb
Copy link
Member

Withzanieb#6 (comment) I fixed the PGO build failures on Linux :)

AllanDaemon reacted with heart emoji

@zanieb
Copy link
Member

And now inzanieb#8 it looks like I've got the macOS builds passing.

Tomorrow I'll polish it and put up a new pull request.

charliermarsh reacted with heart emoji

@kpfleming
Copy link
ContributorAuthor

Fantastic!

@zaniebzanieb mentioned this pull requestSep 6, 2024
@kpfleming
Copy link
ContributorAuthor

Closing in favor of#319.

AllanDaemon reacted with hooray emoji

@kpflemingkpfleming deleted the python-3.13 branchSeptember 6, 2024 15:53
charliermarsh pushed a commit that referenced this pull requestSep 6, 2024
A continuation of the wonderful work by@kpfleming in#264Adds initial support for Python 3.13 using CPython 3.13.0rc1.There are a few caveats and interesting details:- BOLT is disabled. There's a segmentation fault in the tests with the BOLT instrumented binaries. The BOLT optimizations are not critical, so we'll follow up on this separately. [See more context](zanieb#6 (comment)).- `mpdecimal` is now built from source on Windows. We already did this in Unix builds, but in Windows we were still using the bundled library. The bundled library is no longer used upstream and it seemed prudent to switch though it will remain available until 3.15. [See more context](zanieb#6 (comment)).- Apple cross-compilation is not available. I have a patch that adds support, but need to test it and it's not needed for these builds. [See more context](zanieb@447fb86).- `run_tests.py` was removed upstream. We provide a compatibility script that calls the appropriate command still so that our distributions are stable. We may want to change how `run_tests.py` is declared in the distribution metadata though. [See more context](#319 (comment)) .
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@zaniebzaniebzanieb left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kpfleming@indygreg@zanieb@ofek
[8]ページ先頭
©2009-2025 Movatter.jp