Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit7861dfd

Browse files
authored
gh-111140: Adds PyLong_AsNativeBytes and PyLong_FromNative[Unsigned]Bytes functions (GH-114886)
1 parenta82fbc1 commit7861dfd

File tree

14 files changed

+533
-26
lines changed

14 files changed

+533
-26
lines changed

‎Doc/c-api/long.rst

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,28 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
113113
retrieved from the resulting value using:c:func:`PyLong_AsVoidPtr`.
114114
115115
116+
..c:function:: PyObject*PyLong_FromNativeBytes(const void* buffer, size_t n_bytes, int endianness)
117+
118+
Create a Python integer from the value contained in the first *n_bytes* of
119+
*buffer*, interpreted as a two's-complement signed number.
120+
121+
*endianness* may be passed ``-1`` for the native endian that CPython was
122+
compiled with, or else ``0`` for big endian and ``1`` for little.
123+
124+
.. versionadded:: 3.13
125+
126+
127+
.. c:function:: PyObject* PyLong_FromUnsignedNativeBytes(const void* buffer, size_t n_bytes, int endianness)
128+
129+
Create a Python integer from the value contained in the first *n_bytes* of
130+
*buffer*, interpreted as an unsigned number.
131+
132+
*endianness* may be passed ``-1`` for the native endian that CPython was
133+
compiled with, or else ``0`` for big endian and ``1`` for little.
134+
135+
.. versionadded:: 3.13
136+
137+
116138
.. XXX alias PyLong_AS_LONG (for now)
117139
.. c:function:: long PyLong_AsLong(PyObject *obj)
118140
@@ -332,6 +354,50 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
332354
Returns ``NULL`` on error. Use:c:func:`PyErr_Occurred` to disambiguate.
333355
334356
357+
..c:function:: Py_ssize_tPyLong_AsNativeBytes(PyObject *pylong, void* buffer, Py_ssize_t n_bytes, int endianness)
358+
359+
Copy the Python integer value to a native *buffer* of size *n_bytes*::
360+
361+
int value;
362+
Py_ssize_t bytes = PyLong_CopyBits(v, &value,sizeof(value), -1);
363+
if (bytes <0) {
364+
// Error occurred
365+
return NULL;
366+
}
367+
elseif (bytes >sizeof(value)) {
368+
// Overflow occurred, but 'value' contains as much as could fit
369+
}
370+
371+
*endianness* may be passed ``-1`` for the native endian that CPython was
372+
compiled with, or ``0`` for big endian and ``1`` for little.
373+
374+
Return ``-1`` with an exception raised if *pylong* cannot be interpreted as
375+
an integer. Otherwise, return the size of the buffer required to store the
376+
value. If this is equal to or less than *n_bytes*, the entire value was
377+
copied.
378+
379+
Unless an exception is raised, all *n_bytes* of the buffer will be written
380+
with as much of the value as can fit. This allows the caller to ignore all
381+
non-negative results if the intent is to match the typical behavior of a
382+
C-style downcast.
383+
384+
Values are always copied as twos-complement, and sufficient size will be
385+
requested for a sign bit. For example, this may cause an value that fits into
386+
8 bytes when treated as unsigned to request 9 bytes, even though all eight
387+
bytes were copied into the buffer. What has been omitted is the zero sign
388+
bit, which is redundant when the intention is to treat the value as unsigned.
389+
390+
Passing *n_bytes* of zero will always return the requested buffer size.
391+
392+
..note::
393+
394+
When the value does not fit in the provided buffer, the requested size
395+
returned from the function may be larger than necessary. Passing 0 to this
396+
function is not an accurate way to determine the bit length of a value.
397+
398+
..versionadded::3.13
399+
400+
335401
..c:function::intPyUnstable_Long_IsCompact(const PyLongObject* op)
336402
337403
Return 1 if *op* is compact, 0 otherwise.

‎Doc/whatsnew/3.13.rst

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -587,6 +587,7 @@ Tier 2 IR by Mark Shannon and Guido van Rossum.
587587
Tier 2 optimizer by Ken Jin.)
588588

589589

590+
590591
Deprecated
591592
==========
592593

@@ -1526,6 +1527,11 @@ New Features
15261527

15271528
(Contributed by Victor Stinner and Petr Viktorin in:gh:`110850`.)
15281529

1530+
* Add:c:func:`PyLong_AsNativeBytes`,:c:func:`PyLong_FromNativeBytes` and
1531+
:c:func:`PyLong_FromUnsignedNativeBytes` functions to simplify converting
1532+
between native integer types and Python:class:`int` objects.
1533+
(Contributed by Steve Dower in:gh:`111140`.)
1534+
15291535

15301536
Porting to Python 3.13
15311537
----------------------
@@ -1585,7 +1591,6 @@ Porting to Python 3.13
15851591
platforms, the ``HAVE_STDDEF_H`` macro is only defined on Windows.
15861592
(Contributed by Victor Stinner in:gh:`108765`.)
15871593

1588-
15891594
Deprecated
15901595
----------
15911596

‎Include/cpython/longobject.h

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,40 @@
44

55
PyAPI_FUNC(PyObject*)PyLong_FromUnicodeObject(PyObject*u,intbase);
66

7+
/* PyLong_AsNativeBytes: Copy the integer value to a native variable.
8+
buffer points to the first byte of the variable.
9+
n_bytes is the number of bytes available in the buffer. Pass 0 to request
10+
the required size for the value.
11+
endianness is -1 for native endian, 0 for big endian or 1 for little.
12+
Big endian mode will write the most significant byte into the address
13+
directly referenced by buffer; little endian will write the least significant
14+
byte into that address.
15+
16+
If an exception is raised, returns a negative value.
17+
Otherwise, returns the number of bytes that are required to store the value.
18+
To check that the full value is represented, ensure that the return value is
19+
equal or less than n_bytes.
20+
All n_bytes are guaranteed to be written (unless an exception occurs), and
21+
so ignoring a positive return value is the equivalent of a downcast in C.
22+
In cases where the full value could not be represented, the returned value
23+
may be larger than necessary - this function is not an accurate way to
24+
calculate the bit length of an integer object.
25+
*/
26+
PyAPI_FUNC(Py_ssize_t)PyLong_AsNativeBytes(PyObject*v,void*buffer,
27+
Py_ssize_tn_bytes,intendianness);
28+
29+
/* PyLong_FromNativeBytes: Create an int value from a native integer
30+
n_bytes is the number of bytes to read from the buffer. Passing 0 will
31+
always produce the zero int.
32+
PyLong_FromUnsignedNativeBytes always produces a non-negative int.
33+
endianness is -1 for native endian, 0 for big endian or 1 for little.
34+
35+
Returns the int object, or NULL with an exception set. */
36+
PyAPI_FUNC(PyObject*)PyLong_FromNativeBytes(constvoid*buffer,size_tn_bytes,
37+
intendianness);
38+
PyAPI_FUNC(PyObject*)PyLong_FromUnsignedNativeBytes(constvoid*buffer,
39+
size_tn_bytes,intendianness);
40+
741
PyAPI_FUNC(int)PyUnstable_Long_IsCompact(constPyLongObject*op);
842
PyAPI_FUNC(Py_ssize_t)PyUnstable_Long_CompactValue(constPyLongObject*op);
943

@@ -50,7 +84,7 @@ PyAPI_FUNC(PyObject *) _PyLong_FromByteArray(
5084
*/
5185
PyAPI_FUNC(int)_PyLong_AsByteArray(PyLongObject*v,
5286
unsignedchar*bytes,size_tn,
53-
intlittle_endian,intis_signed);
87+
intlittle_endian,intis_signed,intwith_exceptions);
5488

5589
/* For use by the gcd function in mathmodule.c */
5690
PyAPI_FUNC(PyObject*)_PyLong_GCD(PyObject*,PyObject*);

‎Lib/test/test_capi/test_long.py

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
importunittest
22
importsys
3+
importtest.supportassupport
34

45
fromtest.supportimportimport_helper
56

@@ -423,6 +424,150 @@ def test_long_asvoidptr(self):
423424
self.assertRaises(OverflowError,asvoidptr,-2**1000)
424425
# CRASHES asvoidptr(NULL)
425426

427+
deftest_long_asnativebytes(self):
428+
importmath
429+
from_testcapiimport (
430+
pylong_asnativebytesasasnativebytes,
431+
SIZE_MAX,
432+
)
433+
434+
# Abbreviate sizeof(Py_ssize_t) to SZ because we use it a lot
435+
SZ=int(math.ceil(math.log(SIZE_MAX+1)/math.log(2))/8)
436+
MAX_SSIZE=2** (SZ*8-1)-1
437+
MAX_USIZE=2** (SZ*8)-1
438+
ifsupport.verbose:
439+
print(f"SIZEOF_SIZE={SZ}\n{MAX_SSIZE=:016X}\n{MAX_USIZE=:016X}")
440+
441+
# These tests check that the requested buffer size is correct
442+
forv,expectin [
443+
(0,SZ),
444+
(512,SZ),
445+
(-512,SZ),
446+
(MAX_SSIZE,SZ),
447+
(MAX_USIZE,SZ+1),
448+
(-MAX_SSIZE,SZ),
449+
(-MAX_USIZE,SZ+1),
450+
(2**255-1,32),
451+
(-(2**255-1),32),
452+
(2**256-1,33),
453+
(-(2**256-1),33),
454+
]:
455+
withself.subTest(f"sizeof-{v:X}"):
456+
buffer=bytearray(1)
457+
self.assertEqual(expect,asnativebytes(v,buffer,0,-1),
458+
"PyLong_AsNativeBytes(v, NULL, 0, -1)")
459+
# Also check via the __index__ path
460+
self.assertEqual(expect,asnativebytes(Index(v),buffer,0,-1),
461+
"PyLong_AsNativeBytes(Index(v), NULL, 0, -1)")
462+
463+
# We request as many bytes as `expect_be` contains, and always check
464+
# the result (both big and little endian). We check the return value
465+
# independently, since the buffer should always be filled correctly even
466+
# if we need more bytes
467+
forv,expect_be,expect_nin [
468+
(0,b'\x00',1),
469+
(0,b'\x00'*2,2),
470+
(0,b'\x00'*8,min(8,SZ)),
471+
(1,b'\x01',1),
472+
(1,b'\x00'*10+b'\x01',min(11,SZ)),
473+
(42,b'\x2a',1),
474+
(42,b'\x00'*10+b'\x2a',min(11,SZ)),
475+
(-1,b'\xff',1),
476+
(-1,b'\xff'*10,min(11,SZ)),
477+
(-42,b'\xd6',1),
478+
(-42,b'\xff'*10+b'\xd6',min(11,SZ)),
479+
# Extracts 255 into a single byte, but requests sizeof(Py_ssize_t)
480+
(255,b'\xff',SZ),
481+
(255,b'\x00\xff',2),
482+
(256,b'\x01\x00',2),
483+
# Extracts successfully (unsigned), but requests 9 bytes
484+
(2**63,b'\x80'+b'\x00'*7,9),
485+
# "Extracts", but requests 9 bytes
486+
(-2**63,b'\x80'+b'\x00'*7,9),
487+
(2**63,b'\x00\x80'+b'\x00'*7,9),
488+
(-2**63,b'\xff\x80'+b'\x00'*7,9),
489+
490+
(2**255-1,b'\x7f'+b'\xff'*31,32),
491+
(-(2**255-1),b'\x80'+b'\x00'*30+b'\x01',32),
492+
# Request extra bytes, but result says we only needed 32
493+
(-(2**255-1),b'\xff\x80'+b'\x00'*30+b'\x01',32),
494+
(-(2**255-1),b'\xff\xff\x80'+b'\x00'*30+b'\x01',32),
495+
496+
# Extracting 256 bits of integer will request 33 bytes, but still
497+
# copy as many bits as possible into the buffer. So we *can* copy
498+
# into a 32-byte buffer, though negative number may be unrecoverable
499+
(2**256-1,b'\xff'*32,33),
500+
(2**256-1,b'\x00'+b'\xff'*32,33),
501+
(-(2**256-1),b'\x00'*31+b'\x01',33),
502+
(-(2**256-1),b'\xff'+b'\x00'*31+b'\x01',33),
503+
(-(2**256-1),b'\xff\xff'+b'\x00'*31+b'\x01',33),
504+
505+
# The classic "Windows HRESULT as negative number" case
506+
# HRESULT hr;
507+
# PyLong_CopyBits(<-2147467259>, &hr, sizeof(HRESULT))
508+
# assert(hr == E_FAIL)
509+
(-2147467259,b'\x80\x00\x40\x05',4),
510+
]:
511+
withself.subTest(f"{v:X}-{len(expect_be)}bytes"):
512+
n=len(expect_be)
513+
buffer=bytearray(n)
514+
expect_le=expect_be[::-1]
515+
516+
self.assertEqual(expect_n,asnativebytes(v,buffer,n,0),
517+
f"PyLong_AsNativeBytes(v, buffer,{n}, <big>)")
518+
self.assertEqual(expect_be,buffer[:n],"<big>")
519+
self.assertEqual(expect_n,asnativebytes(v,buffer,n,1),
520+
f"PyLong_AsNativeBytes(v, buffer,{n}, <little>)")
521+
self.assertEqual(expect_le,buffer[:n],"<little>")
522+
523+
# Check a few error conditions. These are validated in code, but are
524+
# unspecified in docs, so if we make changes to the implementation, it's
525+
# fine to just update these tests rather than preserve the behaviour.
526+
withself.assertRaises(SystemError):
527+
asnativebytes(1,buffer,0,2)
528+
withself.assertRaises(TypeError):
529+
asnativebytes('not a number',buffer,0,-1)
530+
531+
deftest_long_fromnativebytes(self):
532+
importmath
533+
from_testcapiimport (
534+
pylong_fromnativebytesasfromnativebytes,
535+
SIZE_MAX,
536+
)
537+
538+
# Abbreviate sizeof(Py_ssize_t) to SZ because we use it a lot
539+
SZ=int(math.ceil(math.log(SIZE_MAX+1)/math.log(2))/8)
540+
MAX_SSIZE=2** (SZ*8-1)-1
541+
MAX_USIZE=2** (SZ*8)-1
542+
543+
forv_be,expect_s,expect_uin [
544+
(b'\x00',0,0),
545+
(b'\x01',1,1),
546+
(b'\xff',-1,255),
547+
(b'\x00\xff',255,255),
548+
(b'\xff\xff',-1,65535),
549+
]:
550+
withself.subTest(f"{expect_s}-{expect_u:X}-{len(v_be)}bytes"):
551+
n=len(v_be)
552+
v_le=v_be[::-1]
553+
554+
self.assertEqual(expect_s,fromnativebytes(v_be,n,0,1),
555+
f"PyLong_FromNativeBytes(buffer,{n}, <big>)")
556+
self.assertEqual(expect_s,fromnativebytes(v_le,n,1,1),
557+
f"PyLong_FromNativeBytes(buffer,{n}, <little>)")
558+
self.assertEqual(expect_u,fromnativebytes(v_be,n,0,0),
559+
f"PyLong_FromUnsignedNativeBytes(buffer,{n}, <big>)")
560+
self.assertEqual(expect_u,fromnativebytes(v_le,n,1,0),
561+
f"PyLong_FromUnsignedNativeBytes(buffer,{n}, <little>)")
562+
563+
# Check native endian when the result would be the same either
564+
# way and we can test it.
565+
ifv_be==v_le:
566+
self.assertEqual(expect_s,fromnativebytes(v_be,n,-1,1),
567+
f"PyLong_FromNativeBytes(buffer,{n}, <native>)")
568+
self.assertEqual(expect_u,fromnativebytes(v_be,n,-1,0),
569+
f"PyLong_FromUnsignedNativeBytes(buffer,{n}, <native>)")
570+
426571

427572
if__name__=="__main__":
428573
unittest.main()
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Adds:c:func:`PyLong_AsNativeBytes`,:c:func:`PyLong_FromNativeBytes` and
2+
:c:func:`PyLong_FromUnsignedNativeBytes` functions.

‎Modules/_io/textio.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2393,7 +2393,7 @@ textiowrapper_parse_cookie(cookie_type *cookie, PyObject *cookieObj)
23932393
return-1;
23942394

23952395
if (_PyLong_AsByteArray(cookieLong,buffer,sizeof(buffer),
2396-
PY_LITTLE_ENDIAN,0)<0) {
2396+
PY_LITTLE_ENDIAN,0,1)<0) {
23972397
Py_DECREF(cookieLong);
23982398
return-1;
23992399
}

‎Modules/_pickle.c

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2162,7 +2162,8 @@ save_long(PicklerObject *self, PyObject *obj)
21622162
pdata= (unsignedchar*)PyBytes_AS_STRING(repr);
21632163
i=_PyLong_AsByteArray((PyLongObject*)obj,
21642164
pdata,nbytes,
2165-
1/* little endian */ ,1/* signed */ );
2165+
1/* little endian */ ,1/* signed */ ,
2166+
1/* with exceptions */);
21662167
if (i<0)
21672168
gotoerror;
21682169
/* If the int is negative, this may be a byte more than

‎Modules/_randommodule.c

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -342,7 +342,8 @@ random_seed(RandomObject *self, PyObject *arg)
342342
res=_PyLong_AsByteArray((PyLongObject*)n,
343343
(unsignedchar*)key,keyused*4,
344344
PY_LITTLE_ENDIAN,
345-
0);/* unsigned */
345+
0,/* unsigned */
346+
1);/* with exceptions */
346347
if (res==-1) {
347348
gotoDone;
348349
}

‎Modules/_sqlite/util.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,7 @@ _pysqlite_long_as_int64(PyObject * py_val)
162162
sqlite_int64int64val;
163163
if (_PyLong_AsByteArray((PyLongObject*)py_val,
164164
(unsignedchar*)&int64val,sizeof(int64val),
165-
IS_LITTLE_ENDIAN,1/* signed */) >=0) {
165+
IS_LITTLE_ENDIAN,1/* signed */,0) >=0) {
166166
returnint64val;
167167
}
168168
}

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp