Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-114099: Add configure and Makefile targets to support iOS compilation.#115390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
ned-deily merged 15 commits intopython:mainfromfreakboy3742:ios-configure-targets
Feb 26, 2024

Conversation

freakboy3742
Copy link
Contributor

@freakboy3742freakboy3742 commentedFeb 13, 2024
edited
Loading

Part of the PEP 730 work to add iOS support.

This PR adds the iOS targets to configure and Makefile to support compiling an iOS framework. It doesnot include the test harness and related targets that was included in#115063 - that will come in a follow up PR.

This PR slightly containsone deviation from PEP 730, as it uses the<cpu>-<abi> format for PLATFORM_TRIPLE and _PYTHON_HOST_PLATFORM, rather than<abi>-<cpu> (i.e., it usesarm64-iphoneos, notiphoneos-arm64). This was identified during the review of#115120 as an inconsistency with the ordering of existing platform triples. This is an almost entirely cosmetic change with no compatibility considerations, as the only impact is the final naming scheme ofsysconfig modules and thesys.implementation._multiarch attribute.

As of this PR, it is now possible tocompile an iOS framework; but the framework still isn't usable at runtime.

Summary of changes

  • Patchesconfig.sub to include 2 modifications required to support iOS/tvOS/watchOS builds (supporting the arm64_32 CPU identifier, and the-simulator suffix). These were submitted upstream as part of a patch in August last year, but that patch was only partially applied. The autoconf contribution process is opaque, and no feedback was given on why the other parts were rejected (there wasn't even feedback that they'd been partially accepted - I discovered that by accident). The extra pieces have been re-submitted upstream.
  • Modifies the internal target name for setting up a versioned macOS framework, and adds a set of targets for an unversioned iOS-style framework. See below for details on iOS framework structure.
  • Adds the flags to build iOS-compatible binary modules linked against the Python framework
  • Presumptively adds the arm64 device and simulator targets to the Tier 3 support list. Getting anx86_64 buildbot is proving difficult because of hardware availability, so I've omitted that target from Tier 3.This possibility was flagged in PEP 730.
  • Skips the automated detection of a couple of system functions where autoconf gets the answer wrong - autoconf only checks whether a symbol can be linked, and doesn't use header files; iOS headers raise warnings when some symbols are used. There are also methods that exist and can be compiled, but raise errors if used at runtime.
  • Automatically skips the configuration of ac_cv_file__dev_ptmx and ac_cv_file__dev_ptc, which are usually mandatory command line arguments for cross-platform builds. iOS is a reliable cross-build target, so we know these features aren't available.
  • Adds an iOS resources folder that contains template files and stub binaries. See below for a discussion about the stub binaries. Internal documentation for building the iOS framework has been added; user-facing docs will be added in a future PR.
  • Includes apyconfig.h file that can be used to switch between CPU-dependent pyconfig.h files.
  • Disables the standard library modules that aren't available on iOS.
  • Doesn't include an tvOS and watchOS targets or resources. These will be submitted independently.

Differences between macOS and iOS Frameworks

iOS frameworks are slightly different to macOS frameworks.

  • macOS frameworks are "multi-version", as they are designed to support having multiple framework versions in a single Framework folder. iOS frameworks can't be shared between apps, so they don't support "multi-version" format.
  • The metadata required by Info.plist is slightly different, and must be in the root of the framework, not a Resources subfolder.
  • The install name for the library in the framework must be based on an @rpath, rather than an absolute path, as the install path for the framework will be app-specific.
  • iOS frameworks can only include binary .dylib artefacts and headers; and the headers won't be copied into the final distribution binary. As a result, the standard library and support binaries cannot be distributed inside the framework - they must be distributed parallel to the Framework folder.

To accomodate these difference, the make install target for iOS will generate a file structure like the following:

(--enable-framework location)

  • bin
  • include symlink to -> Python.framework/Headers
  • lib
    • python3.X
      • ...
  • Python.framework
    • Headers
      • ...
    • Python (the dylib binary)
    • Info.plist

The task of copying the standard library into an appropriate location in the final app bundle is left as a task for the developer using the framework.

Stub binaries

Xcode doesn't expose explicit compilers for iOS/tvOS/watchOS; instead, it uses an xcrun script that resolves to a full compiler path (e.g., xcrun --sdk iphoneos clang to get the clang for an iPhone device). However, using this script poses 2 problems:

  • The output of xcrun includes paths that are then machine specific, resulting in a sysconfig module that cannot be shared between users
  • It results in CC/CPP/LD/AR definitions that include spaces. There is a lot of C ecosystem tooling (including distutils and setuptools) that assumes that you can split a command line at the first space to get the path to the compiler executable; this isn't the case when using xcrun.

To avoid these problems, the iOS/Resources/bin folder includes shell script wrappers around the tools needed by configure, named using the scheme that autoconf expects by default. These scripts are relocatable, and will always resolve to the appropriate local system paths. By including these scripts in the "installed" bin folder, the contents of the sysconfig module becomes useful for end-users to compile their own modules.

@mhsmith
Copy link
Member

I don't have access to resolve conversations in this repository, so I've marked them with a thumbs up instead. The only outstanding one, which we need guidance from the core team on, is whether the iOS files should go in a top-level directory as this PR currently does, or a subdirectory ofTools.

@erlend-aasland
Copy link
Contributor

[...] The only outstanding one, which we need guidance from the core team on, is whether the iOS files should go in a top-level directory as this PR currently does, or a subdirectory ofTools.

I personally would preferone top-level directory, like the currentMac/ dir. Windows stuff is split betweenPC/ andPCbuild, whichIMO is unfortunate. I haven't looked at how the WASM stuff is organised.

@freakboy3742
Copy link
ContributorAuthor

[...] The only outstanding one, which we need guidance from the core team on, is whether the iOS files should go in a top-level directory as this PR currently does, or a subdirectory ofTools.

I personally would preferone top-level directory, like the currentMac/ dir. Windows stuff is split betweenPC/ andPCbuild, whichIMO is unfortunate. I haven't looked at how the WASM stuff is organised.

That's good enough for me. I've modified the docs to use a folder inside iOS.

erlend-aasland reacted with thumbs up emoji

Copy link
Contributor

@erlend-aaslanderlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks to@mhsmith for the thorough reviews. Waiting thumbs-up from@ned-deily before landing.

Copy link
Member

@ned-deilyned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Overall, this is a very nice PR and a pleasure to review. I don't have time left tonight to actually test it; I will do so tomorrow. But I do have a few, relatively minor review comments here. Let me know what you think.

erlend-aasland reacted with hooray emoji
@@ -0,0 +1,2 @@
#!/bin/bash
xcrun --sdk iphoneos ar $@
Copy link
Member

@ned-deilyned-deilyFeb 22, 2024
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The use of these stub scripts for executables is a clever solution. As they stand, I think there is one potential shortcoming if I understand thexcrun documentation correctly (I haven't yet verified this): according to the man page, specifying--sdk toxcrun overrides anySDKROOT environment variable, something which I've had occasion to use when building and testing for multiple macOS versions. It's probably less of a deal on iOS but I think it would be nice to allow for this. If the behavior is as documented, perhaps the stub scripts could do a test for the presence of anSDKROOT env variable and, if defined, not specify--sdk iphoneos?

erlend-aasland reacted with thumbs up emoji
Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

That's an easy enough change to make; however, it does open a potential class of bugs.

As currently defined,arm64-apple-ios-clang will always be a clang install for ARM64 iOS devices. If we allow forSDKROOT as you describe, a user could easily think they're compiling an iOS build, but actually be using an iOS simulator SDK (or worse - a macOS or watchOS SDK).

This is especially problematic when the values are embedded insysconfig, as theCC value embedded in sysconfig will become dependent on the end-user's environment.

You can still switch between different Xcode installs withxcode-select for testing purposes; I'd argue this is likely more useful for testing purposes, as Xcode leans heavily to having a single iOS SDK per Xcode install.

My inclination is that being explicit with--sdk in these scripts and overridingSDKROOT is preferable on balance, but I'll defer to your judgement if you feel strongly otherwise. If wedo make this change, I'll add some docs clarifying how SDKROOT is interpreted, and clarifying that it's the user's responsibility to make sure SDKROOT matches their intended compilation target.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I see two drawbacks with relying solely onxcode-select. One,xcode-select -s has effect system-wide, and so would affect all other uses and users of Xcode on that system. Of course, there is another way to effect this on a process basis: set aDEVELOPER_DIR env variable. But, two, even with that, AFAICT there still would be no way to select a non-default iOS SDK within a particular Xcode instance and, while it may not be the default case at the moment, it would be better to not preclude supporting more than one SDK available in an Xcode instance. I think the primary value of supporting all this would be during development and testing of Python iOS frameworks themselves and, as such, you likely wouldn't want to use them in an actual release of a Python iOS framework but theremight be cases where you would. (And, the end-user of the framework, presumably someone developing an iOS app, would need to ensure they had a compatible developer environment anyway.)

So perhaps a slightly simpler way to address these concerns would be to define a new env variable that would allow overriding the SDK name in the stubs, something likeIOSSDK='iphoneos17.1' (I'm not hung up on the name) instead of the defaultiphoneos ?, and mentioning settingDEVELOPER_DIR to use a non-default Xcode installation:

xcrun --sdk ${IOSSDK:-iphoneos} clang -target arm64-apple-ios $@

Or we could just defer this until a demonstrated need arises but I think it is worth considering.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Passing in a specific SDK (or even better, a specific SDKversion) with a custom environment variable makes sense to me. I'll push an update shortly that usesIOS_SDK_VERSION so you can getiphoneosX.Y andiphonesimulatorX.Y from a single environment variable (since the base names of the SDKs are stable, and I can't think of a use case where you'd want a different version for the device vs simulator SDK in a single build), along with docs covering the new variable andDEVELOPER_DIR

ned-deily and erlend-aasland reacted with thumbs up emoji
@@ -0,0 +1,2 @@
#!/bin/bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Sorry, I didn't notice these earlier: all of the stubs should just use a shebang line of/bin/sh rather thanbash sincebash is officially deprecated on macOS, and there are no bash-isms used.

freakboy3742 and erlend-aasland reacted with thumbs up emoji
@freakboy3742
Copy link
ContributorAuthor

@ned-deily Updated with a README clarification and$(srcdir) cleanup.

Copy link
Member

@ned-deilyned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks good to me. Thanks again for the attention to detail. If any other comments come up, they can be resolved in a follow-on PR. Time to move on to the next phase.

erlend-aasland reacted with hooray emoji
@mhsmith
Copy link
Member

I'm attaching the script here just for future reference rather than suggesting including it in the PR. Perhaps it might be useful as a start for a buildbot or CI step.

I've already put together a more production-ready script for Android, based on the existing WASI build script from Tools/wasm/wasi.py, so I suggest iOS follows the same pattern.

The current version ishere, and I'll submit it in a PR as soon as the lower-level Android build fixes have been merged.

erlend-aasland reacted with thumbs up emoji

@freakboy3742freakboy3742 deleted the ios-configure-targets branchFebruary 26, 2024 22:20
erlend-aasland pushed a commit that referenced this pull requestFeb 29, 2024
This change is part of the work onPEP-738: Adding Android as a supported platform.* Remove the "1.0" suffix from libpython's filename on Android, which   would prevent Gradle from packaging it into an app. * Simplify the build command in the Makefile so that libpython always   gets given an SONAME with the `-Wl-h` argument, even if the SONAME is  identical to the actual filename.* Disable a number of functions on Android which can be compiled and   linked against, but always fail at runtime. As a result, the native  _multiprocessing module is no longer built for Android.*gh-115390 (bee7bb3) added some pre-determined results to the   configure script for things that can't be autodetected when  cross-compiling; this change adds Android to these where appropriate.* Add a couple more pre-determined results for Android, and making them   cover iOS as well. This means the --enable-ipv6 configure option will   no longer be required on either platform.
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull requestMar 4, 2024
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull requestMar 4, 2024
This change is part of the work on PEP-738: Adding Android as a supported platform.* Remove the "1.0" suffix from libpython's filename on Android, which   would prevent Gradle from packaging it into an app. * Simplify the build command in the Makefile so that libpython always   gets given an SONAME with the `-Wl-h` argument, even if the SONAME is  identical to the actual filename.* Disable a number of functions on Android which can be compiled and   linked against, but always fail at runtime. As a result, the native  _multiprocessing module is no longer built for Android.*pythongh-115390 (bee7bb3) added some pre-determined results to the   configure script for things that can't be autodetected when  cross-compiling; this change adds Android to these where appropriate.* Add a couple more pre-determined results for Android, and making them   cover iOS as well. This means the --enable-ipv6 configure option will   no longer be required on either platform.
adorilson pushed a commit to adorilson/cpython that referenced this pull requestMar 25, 2024
adorilson pushed a commit to adorilson/cpython that referenced this pull requestMar 25, 2024
This change is part of the work on PEP-738: Adding Android as a supported platform.* Remove the "1.0" suffix from libpython's filename on Android, which   would prevent Gradle from packaging it into an app. * Simplify the build command in the Makefile so that libpython always   gets given an SONAME with the `-Wl-h` argument, even if the SONAME is  identical to the actual filename.* Disable a number of functions on Android which can be compiled and   linked against, but always fail at runtime. As a result, the native  _multiprocessing module is no longer built for Android.*pythongh-115390 (bee7bb3) added some pre-determined results to the   configure script for things that can't be autodetected when  cross-compiling; this change adds Android to these where appropriate.* Add a couple more pre-determined results for Android, and making them   cover iOS as well. This means the --enable-ipv6 configure option will   no longer be required on either platform.
diegorusso pushed a commit to diegorusso/cpython that referenced this pull requestApr 17, 2024
diegorusso pushed a commit to diegorusso/cpython that referenced this pull requestApr 17, 2024
This change is part of the work on PEP-738: Adding Android as a supported platform.* Remove the "1.0" suffix from libpython's filename on Android, which   would prevent Gradle from packaging it into an app. * Simplify the build command in the Makefile so that libpython always   gets given an SONAME with the `-Wl-h` argument, even if the SONAME is  identical to the actual filename.* Disable a number of functions on Android which can be compiled and   linked against, but always fail at runtime. As a result, the native  _multiprocessing module is no longer built for Android.*pythongh-115390 (bee7bb3) added some pre-determined results to the   configure script for things that can't be autodetected when  cross-compiling; this change adds Android to these where appropriate.* Add a couple more pre-determined results for Android, and making them   cover iOS as well. This means the --enable-ipv6 configure option will   no longer be required on either platform.
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestAug 5, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestAug 5, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestAug 5, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestAug 5, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestSep 6, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestSep 9, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestSep 9, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestSep 9, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestSep 9, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestOct 9, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestDec 13, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestDec 13, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestDec 13, 2024
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull requestDec 13, 2024
LukasWoodtli pushed a commit to LukasWoodtli/cpython that referenced this pull requestJan 22, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@mhsmithmhsmithmhsmith left review comments

@ned-deilyned-deilyned-deily approved these changes

@erlend-aaslanderlend-aaslanderlend-aasland approved these changes

@corona10corona10Awaiting requested review from corona10corona10 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@freakboy3742@mhsmith@erlend-aasland@ned-deily

[8]ページ先頭

©2009-2025 Movatter.jp