Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
Add CI job to build Python wheels for ppc64le architecture#29485
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thank you for opening your first PR into Matplotlib!
If you have not heard from us in a week or so, please leave a new comment below and that should bring it to our attention. Most of our reviewers are volunteers and sometimes things fall through the cracks.
You can also join uson gitter for real-time discussion.
For details on testing, writing docs, and our review process, please seethe developer guide
We strive to be a welcoming and open project. Please follow ourCode of Conduct.
I'm going to reference a similar discussion (regarding MUSL libc rather than CPU architecture, but the result is effectively the same, increasing the load of Wheels for the project to maintain):
Those will more fully delve into several maintainers viewpoints and thoughts on why we are hesitant to expand our collection of pre-built wheels. To be clear, hesitant does not mean "definitely no", but we may want some additional context, reasoning, and potentially even a level of commitment to having a person (not necessarily you) who we can tag in if problems arise with PPC builds, as none of our current maintainers regularly use such machines (to my knowledge). Put simply, building something that we do not have the means to properly test and maintain has strong potential to do more harm than good (even iflikely everything "just works" most of the time). It is a balancing act of providing pre-built versions for people vs the amount of maintainer time that is expended. While I will not make any hard claims, even just taking too long to build (as emulated systems often do) may be enough to put a heavy tilt towards not doing it. For end users on PPC systems, I wouldtend to lean towards using system package managers for pre-built matplotlib. I know we have previously received issue reports from e.g. debian packagers for problems with PPC builds. I am going to cycle it because it appears that despite having the tag added by a bot, the CIBW job was skipped, and I would like to have the data on how long the build takes to inform the decision (though speed is only one factor in the overall decision). I will note, however, that our style check is failing for the edited file, so even if we fall on the side of "add ppc", that will need to be brought in line (though wouldn't spend too much time on that until a decision is made). |
tacaswell commentedJan 22, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
conda-forge also build linux-ppc64le binaries which are likely a better option than using wheels. |
Given that most upstream communities rely on and consume wheels today rather than conda-forge packages, providing wheels is crucial for broader adoption and ease of installation. Wheels simplify installation by offering pre-built binaries that can be installed with pip without the need for additional build tools or compilers. This is particularly important on platforms like Power, where relying solely on conda might not be optimal or widely supported. Pre-built matplotlib wheels can significantly enhance environments such as CI/CD pipelines, Jupyter Notebook instances by cutting installation times and reducing the need for system compilers. For example: Having these wheels available—especially for platforms like Power—will greatly enhance usability and accessibility for a broad range of users, ensuring that installations are efficient and straightforward. |
We discussed our opinions for our policy on this front in our community call last week and have a consensus among those present that our policy should reflect the following:
Further, we would only consider Python's Tier 1 and 2 supported platforms as blocking for release, which would mean PPC would be considered non-blocking as it is Tier 3. SeePEP11. In the case of ppc64le, Pillow and Numpy do not currently upload PPC wheels to PyPI (though kiwisolver and contourpydo). Therefore a prerequisite for us to accept building a PPC wheel would be those two (and any others that I may have missed in my spot checking) gain wheels for ppc first, as they are our dependencies. Thus, those would be the places to start, though I'll admit you may face similar hesitation from those communities as well. Again, it is challenging to put out an official package without the means to properly test and debug it (testing can be mitigated with CI, but the feedback loop is slow compared to local dev, still... and if it has to be emulated it can be all that much worse) As you note, the primary benefit of providing wheels is that you do not have to have build dependencies (e.g. compilers in particular) on the target system, but if you still need that for our dependencies, then you aren't actually saving too much. |
I understand the dependency requirements, and we are currently working on enabling PPC64LE wheels for NumPy. We recognize that having all compiled dependencies available as wheels is a prerequisite, and we will continue coordinating with the relevant communities to address this. I’ll keep you updated as we make progress on NumPy and other dependencies. |
Why is this change necessary?
This change adds support for building Python wheels for the ppc64le architecture using cibuildwheel. This ensures users on ppc64le can access prebuilt wheels, reducing the complexity of local builds and increasing accessibility for this architecture.
What problem does it solve?
Users on ppc64le currently lack prebuilt wheels for supported Python versions.
This PR automates the wheel-building process for ppc64le and integrates it into the existing CI/CD workflow.
Reasoning for Implementation
Leveraged cibuildwheel for multi-architecture Python wheel builds.
Integrated docker/setup-qemu-action for platform emulation.
Ensured compatibility by using manylinux2014 images and validating wheels via twine and custom tests.
PR Checklist
-[YES] "closes #0000" is in the body of the PR description tolink the related issue
-[YES} New and changed code istested
[N/A] Plotting-related features are demonstrated in anexample
[N/A] New Features and API Changes are noted with adirective and release note
-[YES} Documentation complies withgeneral anddocstring guidelines
Notes for Reviewers
The new job build_ppc64le_wheels has been tested with QEMU-based emulation.
Artifacts for each Python version (3.10–3.13) are uploaded separately for easier validation.
CI workflow updates ensure minimal disruption to other processes.