This PR adds an iOS testbed project, plus the Makefile target to support running it from the command line. It is the last of the code that was originally submitted as#115063.

The bulk of the PR is an iOS Xcode project;open iOS/testbed/iOSTestbed.xcodeproj from the command line will open the project in Xcode. The project itself is a bare stub app with no UI, and an XCTest test suite that contains 1 test - "run the CPython test suite"; the XCTest passes or fails depending on the overall success or failure of the CPython test suite.

Most of the project files are as generated by the new project wizard; differences of note include:

iOS/testbed/iOSTestbed/main.m marks some signals as ignored that the CPython test suite exercises, but iOS doesn't like to be unhandled.
iOS/testbed/iOSTestbedTests/iOSTestbedTests.m is an Xcode test suite with a single test that initialises a CPython interpreter and runs thetest module. Command line options need to be passed in by modifying theargv array at the start of the file, as there's no real analog for a test suite command line (at least, not that I've found)
Build scripts have been added to post-process the standard library into "framework" form.
The build setting forEnable Testability has been set toYes; this prevents stripping of binaries in release configurations. This is required because an iOS app may notlink against symbols, but it willuse them.
The build setting forUser Script Sandboxing has been set toNo. This allows the post-processing scripts to modify and sign framework binaries.

There are 2 other changes:

An extra dependency was added to theframeworkinstallmobileheaders target; I discovered that parallelmake install builds would fail because it would try to correct the header install before actually installing the headers.
The configure script has been modified to use the iOS wrapper binaries rather than gcc as a default compiler when CC et al aren't defined. This makes configure easier to invoke (as you just need to have the path correct), and makes the failure mode more predictable (as an attempt to compile with gcc will get quite far along before failing). Manually specifying overrides for CC et al remains available as an option.

Running the test suite

As of this PR, it is possible to build CPython as a framework, and start the test suite.

The new Makefile target ismake testios. This target will run the test suite on an iPhone SE 3rd edition; this is a simulator that is reliably available, as it's a model that doesn't get updated all that often.

Running the testbed requires that an iOS framework build for a simulator target has been compiled and installed into the stub Python.xcframework folder that the testbed project contains. The minimum invocation for running the test suite is:

export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" ./configure --host=arm64-apple-ios12.0-simulator --build=arm64-apple-darwin --with-build-python=/path/to/python3.13 --enable-frameworkmake cleanmake -j4 allmake testios

(adjusting as necessary to point to a appropriate build-python). A cold start of the iOS simulator takes 2-3 minutes, and for most of that time, the simulator appears to be doing nothing. Don't be alarmed when the console output for the test run saysTesting started and then appears to do nothing for a long time. If you're running the test suite from inside Xcode, it's a lot faster once the simulator is warm.

However, if you invoke this target, the test suite will currently fail because it can't find binary modules at runtime. Correcting this will be the subject of the next PR.

Issue:Add support for iOS as a target platform #114099

Add iOS testbed, plus Makefile target to invoke it.

2ee1d5f

freakboy3742 requested review fromerlend-aasland andcorona10 ascode owners

February 26, 2024 06:18

bedevere-appbot added the awaiting review label

Feb 26, 2024

bedevere-appbot mentioned this pull request

Feb 26, 2024

Add support for iOS as a target platform#114099

Closed

freakboy3742 requested a review fromned-deily

February 26, 2024 06:18

sobolevn reviewed

Feb 26, 2024

View reviewed changes

Copy link

Member

sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Super excited for this feature 🎉
Thank you!

.gitignore OutdatedShow resolvedHide resolved

Makefile.pre.in Outdated

Comment on lines 2002 to 2003

		.PHONY: testiOS
		testiOS:

Copy link

Member

sobolevnFeb 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change

	.PHONY:testiOS
	testiOS:
	.PHONY:test-iOS
	test-iOS:

Copy link

ContributorAuthor

freakboy3742Feb 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

My thinking here was to be consistent withtestuniversal,cleantest,buildbottest et al. However, I'm happy to change this name if there's a general preference fortest-iOS overtestiOS.

Copy link

Member

mhsmithFeb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

All existing makefile targets are lower-case, even acronyms likemultissltest.

Copy link

ContributorAuthor

freakboy3742Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I guess that's true... although I suspect I'd twitch every time I need to spell it with capitalization that breaks the branding guide 😀

Copy link

ContributorAuthor

freakboy3742Feb 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I've updated this totestios

Makefile.pre.in OutdatedShow resolvedHide resolved

Make better use of wildcards.

90066ae

mhsmith reviewed

Feb 27, 2024

View reviewed changes

iOS/testbed/iOSTestbedTests/iOSTestbedTests.m Outdated

Comment on lines 77 to 79

		// Set the stdlib location for the Python interpreter
		path = [NSString stringWithFormat:@"%@/python/lib/python%@", resourcePath, py_version_string, nil];
		NSLog(@"Stdlib dir: %@", path);

Copy link

Member

mhsmithFeb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think this entire block is redundant, because this is already the default location that will be derived fromconfig.home. In which case, the whole chain that passes the Python version from the configure script can be removed as well.

Copy link

ContributorAuthor

freakboy3742Feb 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Agreed this block isn't needed. The testbed project still needs a version number, but I guess it's not critical that this version number be the same as the Python version, as it's not going to be a released app, and there are other details in the logs that identify the Python version, so I guess we can drop the configuration portion as well.

iOS/testbed/iOSTestbedTests/iOSTestbedTests.m Outdated

		// will be handled as if it were an argument to `python -m test`
		const char *argv[] = {
		"iOSTestbed", // argv[0] is the process that is running.
		"-uall,-gui,-curses", // Enable most resources; GUI and curses tests won't work on iOS

Copy link

Member

mhsmithFeb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The curses and tkinter tests will be skipped automatically due to their stdlib modules not being built, so there's no need to exclude them here.

freakboy3742 added2 commits

February 28, 2024 14:04

Simplify testing executable.

392c006

Remove capitalization from the testios target.

cdf5df9

ned-deily reviewed

Feb 28, 2024

View reviewed changes

Copy link

Member

ned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Overall, it looks pretty good to me. I like the adding of the default tool stub selections into configure. I might have more comments after the follow-on PRs to get a fully working import system and the test suite can run.

One thing that I would like you to give some thought to, if you haven't already, is to make it possible to runmake testiOS when the build directory is not the source directory. As a perhaps unwritten goal, we try to support multiple builds with different configure args from one source directory with the ultimate goal of a read-only source directory. Requiring the writable testbed directory to be in the source directory hinders those goals. Even something as simple as having themake testiOS rule copy theiOS/testbed directory from the source directory to the build directory would be fine. If that presents formidable obstacles, we wouldn't necessarily need to solve it immediately but I'd like your opinion on it.

Isolate testbed builds into the build directory.

6e768bd

Copy link

ContributorAuthor

freakboy3742 commentedFeb 29, 2024

One thing that I would like you to give some thought to, if you haven't already, is to make it possible to runmake testiOS when the build directory is not the source directory. As a perhaps unwritten goal, we try to support multiple builds with different configure args from one source directory with the ultimate goal of a read-only source directory.

That definitely makes sense as a goal; I think I've addressed that goal with my most recent commit.

To make this work, I've slightly modified the configure script. Previously,--enable-framework on iOSrequired the specification of an install location. In this version, a bare--enable-framework is now legal, and will create an "installed" version of the iOS framework iniOS/Frameworks/arm64-simulator (or similar). Specifying an explicit path will still work if you want.

Then, thetestios target hasinstall as a dependency; before running the test, it copies the testbed project into a timestamped copy in the./build folder, and copies the "installed" framework from wherever it was constructed into the build version of the testbed. The project is then executed as before.

This means each testbed run is completely isolated from past runs; it also means there will be multiple copies of the testbed project and the full "installed" iOS framework - but most of them will be in the build folder, so they're easily cleaned up.

I've retained the instructions to purge the "source" version of the testbed as part of the clean target, in case anyone wants to use the older approach of "in tree" testbed builds - my feeling is that an in-tree build will be easier to manage for day-to-day development than the copied version in the build folder.

Does that approach makes sense? Is there an alternative approach that I haven't considered?

Update README instructions for running the test suite.

0c720f2

ned-deily reviewed

Mar 1, 2024

View reviewed changes

Copy link

Member

ned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for addressing the previous comments. It's clear that, primarily due to the constraints of using an Xcode-provided simulator and of the necessary iOS framework structure, we are doing a bit of shoehorning here with a mixture of a Unix-style autoconf build and an Xcode project. But, if we can resolve the noted issues, I think this should be a reasonable compromise between the two styles.

Makefile.pre.in OutdatedShow resolvedHide resolved

configure.ac OutdatedShow resolvedHide resolved

Makefile.pre.inShow resolvedHide resolved

mhsmith reviewed

Mar 5, 2024

View reviewed changes

iOS/testbed/iOSTestbed/main.m Outdated

Comment on lines 14 to 16

		// iOS doesn't like uncaught signals.
		signal(SIGPIPE, SIG_IGN);
		signal(SIGXFSZ, SIG_IGN);

Copy link

Member

mhsmithMar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I encountered this on Android as well. It turns out Python normally ignores these signals itself, but this is disabled by default with an isolated config. This can be fixed by adding the following line to the initialization code:

config.install_signal_handlers = 1;

Then it should be possible to remove these manual ignores.

freakboy3742 added5 commits

March 6, 2024 07:17

Merge branch 'main' into ios-testbed

5e63e8e

Modify makefile targets to account for renamed frameworks and non-src…

f946473

…dir builds.

Use config flag to enable signal handling.

ccb2f14

Disable testexternalinspection tests on iOS.

04afafb

Switch some clean targets to distclean.

be8f9b1

Copy link

ContributorAuthor

freakboy3742 commentedMar 6, 2024

@ned-deily I've pushed an update that I think addresses all the issues you've flagged with theinstall dependency build location, and custom framework names; plus the signal configuration issue flagged by@mhsmith.

There's one additional piece - in#115774,@pablogsal added a new test module (_testexternalinspection.c) that was breaking the iOS build. AFAICT, iOS doesn't have alibproc.h, and the methods that are being appear to be Darwin kernel specific, but not exposed outside macOS. I've added extra pre-processing to make the test compile and pass.

Move to a positive macOS check, rather than a negative iOS check.

24e2749

Copy link

Member

pablogsal commentedMar 6, 2024

There's one additional piece - in#115774,@pablogsal added a new test module (_testexternalinspection.c) that was breaking the iOS build. AFAICT, iOS doesn't have alibproc.h, and the methods that are being appear to be Darwin kernel specific, but not exposed outside macOS. I've added extra pre-processing to make the test compile and pass.

You can skip it in iOS if you want or add something to configure so it's not compiled in iOS

ned-deily approved these changes

Mar 7, 2024

View reviewed changes

Copy link

Member

ned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for resolving the comments nicely! It looks good to me now, with the caveat that, as advertised, the tests don't yet run under the simulator pending the additional changes to make extension module loading work. Unless someone has an objection, I think we can merge this PR and move on to the next.

bedevere-appbot added awaiting merge and removed awaiting review labels

Mar 7, 2024

Copy link

ContributorAuthor

freakboy3742 commentedMar 7, 2024

Thanks for resolving the comments nicely! It looks good to me now, with the caveat that, as advertised, the tests don't yet run under the simulator pending the additional changes to make extension module loading work. Unless someone has an objection, I think we can merge this PR and move on to the next.

The good news is that the next PR in the chain adds the extension module loader, which means the test suite will run (although there are 4 failures for platform/sysconfig-related reasons). That PR is ready to go - I can push it as soon as this one lands. The PR after that one will resolve those 4 test failures.

ned-deily merged commitb33980a intopython:main

Mar 7, 2024

bedevere-appbot removed the awaiting merge label

Mar 7, 2024

freakboy3742 deleted the ios-testbed branch

March 7, 2024 04:25

Copy link

ContributorAuthor

freakboy3742 commentedMar 7, 2024

The good news is that the next PR in the chain adds the extension module loader, which means the test suite will run (although there are 4 failures for platform/sysconfig-related reasons). That PR is ready to go - I can push it as soon as this one lands. The PR after that one will resolve those 4 test failures.

Hrm... after rebasing with main, I'm getting a weird error I wasn't getting a couple of days ago. Investigating...