Movatterモバイル変換

[0]ホーム
URL:

Skip to main content
HPy/

HPy - A better C API for Python

What is HPy?

HPy provides a new API for extending Python in C. In other words, you use#include <hpy.h> instead of#include <Python.h>.

What are the advantages of HPy?

  • Zero overhead on CPython: extensions written in HPy run at the samespeed as "normal" extensions.

  • Much faster on alternative implementations such asPyPy,GraalPy.

  • Universal binaries: extensions built for theHPy Universal ABI can beloaded unmodified on CPython, PyPy, GraalPython, etc.

  • A migration path for mixing legacy C-API calls with HPy API calls. Onceall the code is migrated, the extension can be compiled as a universal binarythat works on any CPython version, PyPy, or GraalPy.

  • Debug mode: indebug mode, you can easily identify common problems suchas memory leaks, invalid lifetime of objects, invalid usage of APIs. Haveyou ever forgot aPy_INCREF orPy_DECREF? The HPy debug mode can beactivated at runtime to detect these mistakes for you on universal binaries.

  • Nicer API: the standard Python/C API shows its age. HPy is designed toovercome some of its limitations, be more consistent, produce better qualityextensions and to make it harder to introduce bugs.

  • Evolvability: As nicely summarized inPEP 620 the standard Python/C APIexposes a lot of internal implementation details which makes it hard toevolve the C API. HPy doesn't have this problem because all internalimplementation details are hidden.

Current status

HPy is under active development.0.9.0 is the latest alpha release but wewill soon leave the _alpha_ state and are working hard towards a stable release.

We feel that the HPy ABI is stable enough now that we can fulfill our backwardsand forwards binary compatibility promises with the upcoming release, and thatthe API now covers enough use cases to migrate important packages (inparticular, checkout ournumpy port). We also provide porting guides andextensive documentation (in particular theAPI reference).

We are, of course, always open for design discussions and new reqiurements.

The Python/C API is huge. At the moment many popular functions areavailable. Is something missing for the port of your favorite extension?Please open an issue or even better a PR.

HPy-compatible extensions

The extensions that we are experimenting with include:

  • ultrajson-hpy: this was the first real-world module to be ported to HPy. It isa nice fit because it only exports functions (as opposed as custom types)and requires only a small number of API functions.

  • piconumpy: as the name suggests, this is a minimal numpy-like module whichdefines a custom type (similar tondarray but with many fewer features ofcourse).

  • numpy: one of our ambitious goals is to port numpy to HPy, and to use thisexperience to better understand how to design the API. This port is close topassing the test suite.

  • matplotlib: Since Matplotlib also has a dependency to NumPy, the migrationto universal mode is not fully finished. HPy provides the legacy compatibility APIsuch that we can still call legacy C API functions from HPy and successfullyrun the test suite.

  • kiwi-solver: A dependency of Matplotlib, it has been fully ported touniversal mode.

Benchmarks and more information about the Matplotlib and kiwi-solver ports canbe found in theGraalVM blog post. The benchmarks show that the HPy portshave little impact on CPython performance, while enabling close-to-cpythonperformance with GraalVM Python for the kiwi-solver complete HPy port.

Where we need help

  • Documentation: Our resources are very limited and we therefore concentrateon technical tasks. We already wrote a significant amount of documentation(seeDocumentation) but it is not complete yet. If this non-coding work issomething you can do, let us know.

  • Publicity: HPy is already well received among core developers of someimportant projects. For instance, we are actively talking to Python, NumPy,and Cython core developers. However, we need to get more package developers tobe interested in and even use HPy. We appreciate any help for advertising HPyand we are also happy to help preparing talks, demos, etc.

  • Tooling: This is also a topic we would need help. We are, for example,thinking of migration helper tools that do all the boilerplate work that canbe automated when migrating a package from C API to HPy API.

  • Packaging: There are several open questions we need to discuss and answer.

    • How to package a universal extension and how to put it on PyPI?

    • How best to package HPy itself

    • How does HPy integrate with setuptools, mesonpy, or other build systems

    • Upstreaming completed forks of the packages listed below, or upstreamingparts of the forks so the code does not suffer from bit-rot

  • Website and logo design: You don't like this website and the HPy logo?Well, we have expertise in virtual machines, compilers, low-level programming,etc, but zero expertise in web development or graphic design.PRs are welcome;)

More info

Recent blog posts

[8]ページ先頭
©2009-2026 Movatter.jp