As this mostly give a direct access to the PyLongObject - it almost as fast as using private stuff before.

Old code:

$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=10**2' 'mpz(x)'1000000 loops, best of 20: 232 nsec per loop$ python -m timeit -r11 -s 'from gmpy2 import mpz;x=10**100' 'mpz(x)'500000 loops, best of 11: 500 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=10**1000' 'mpz(x)'100000 loops, best of 20: 2.53 usec per loop

With proposed API:

$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=10**2' 'mpz(x)'1000000 loops, best of 20: 258 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=10**100' 'mpz(x)'500000 loops, best of 20: 528 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=10**1000' 'mpz(x)'100000 loops, best of 20: 2.56 usec per loop

But this is something I would like to avoid. This requires allocation of a temporary buffer and using memcpy. Can we offer awritable layout to use it's digits in the mpz_export directly?

Benchmarks for old code:

$ python -m timeit -r11 -s 'from gmpy2 import mpz;x=mpz(10**2)' 'int(x)'2000000 loops, best of 11: 111 nsec per loop$ python -m timeit -r11 -s 'from gmpy2 import mpz;x=mpz(10**100)' 'int(x)'500000 loops, best of 11: 475 nsec per loop$ python -m timeit -r11 -s 'from gmpy2 import mpz;x=mpz(10**1000)' 'int(x)'100000 loops, best of 11: 2.39 usec per loop

With new API:

$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=mpz(10**2)' 'int(x)'2000000 loops, best of 20: 111 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=mpz(10**100)' 'int(x)'500000 loops, best of 20: 578 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=mpz(10**1000)' 'int(x)'100000 loops, best of 20: 2.53 usec per loop

This requires allocation of a temporary buffer and using memcpy.

Right, PyLongObject has to manage its own memory.

Can we offer a writable layout to use it's digits in the mpz_export directly?

That sounds strange from the Python point of view and make the internals "less opaque". I would prefer to leak less implementation details.

Right, PyLongObject has to manage its own memory.

I'm not trying to change that. More complete proposal:vstinner#4

gmpy2 patch:https://github.com/skirpichev/gmpy/tree/trying-py-import-export-v2

New benchmarks:

$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=mpz(10**2)' 'int(x)'2000000 loops, best of 20: 111 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=mpz(10**100)' 'int(x)'500000 loops, best of 20: 509 nsec per loop$ python -m timeit -r20 -s 'from gmpy2 import mpz;x=mpz(10**1000)' 'int(x)'100000 loops, best of 20: 2.44 usec per loop

I would prefer to leak less implementation details.

I don't think this leak anything. It doesn't leak memory management details.PyLong_Import will just do allocation memory. Writting digits will be job for mpz_export, as before.

Without this, it seems - there are noticeable performance regression for integers of intermediate range. Something up to 20% vs 7% on my branch.

Edit: currently, proposedPyUnstable_Long_ReleaseImport() matchPyUnstable_Long_ReleaseExport(). Perhaps, it could be one function (say, PyUnstable_Long_ReleaseDigitArray()), but I unsure - maybe it puts some constraints on internals of the PyLongObject.

The docs say that we should check for NULL but agreed that we could update them.

Maybe bug is somewhere else? Newly added functions usually are explicitly say if they set an exception on error.

Another reason for using this API was "no-error" contract. Maybe we can specify that in docs as a CPython implementation detail? IIUC, we are free to change such things in new releases without deprecation period.

Note also, that gmpy2 benchmarks measure not just CPython side, but the whole conversion path (int->gmpy2.mpz in this case).

Lets see if we can restore this in a that way. It might be helpful for e.g. Sage, which doesn't support PyPy.

I would prefer to not add this note. It was controversial during PEP 757 design.

@serhiy-storchaka@encukou: What do you think? Would you be ok to declare that the PyLong_Export() function cannot fail if the argument is a Python int?

It was controversial during PEP 757 design.

It was proposed unconditionally, not as CPython's implementation detail.

I'm fine with it, as an implementation detail.

@serhiy-storchaka ?

This code is so close to the PyLong implementation, that I think we should use_PyLong_IsCompact() +_PyLong_CompactValue() to be sure that it matches the specification.

Why?

It's to make the API easier to use: the consumer doesn't have to both with ndigits==0 special case.

In fact it's assumed (see e.g._PyLong_New), that the digit array always has at least one digit (and it's initialized for 0 too). And also, if you pass ndigits==0 to the mpz_import() ascount parameter, then it just calls malloc(0).

But I think it's safe just drop this check. This condition is unreachible here, as0 handled in the!overflow case.

Why not allow 0?

@gpshead asked to reject this case:https://discuss.python.org/t/pep-757-c-api-to-import-export-python-integers/63895/74

Actually, he asked to document that case (i.e. when ndigits==0).

We don't think it's a good idea to overbloat docs with this edge case (different functions should be used for small integers). PEP has a dedicated section to discuss import for small integers, suggesting different functions. (In fact, the whole API is about import/export for big integers.)

I'm fine with it, as an implementation detail.