Bug report

Bug description:

TheBINSTRING opcode in Pickle has two arguments: a "4-byte little-endian signed int" length, and a string of that many bytes (see thecode comment forBINSTRING inpickletools). Since it is signed, this means that any provided value over0x7fffffff would be interpreted as a negative number. The Python pickle implementation specifically treats it as a signed 32-bit length and checks to see if the length is < 0:

However, the C pickle implementation runscalc_binsize(s, 4) forBINSTRING and returns aPy_ssize_t. SincePy_ssize_t data types are the same size as the compiler'ssize_t type (PEP0353), this means aPy_ssize_t is 64-bits long on 64-bit systems. Since thesize variable here is also aPy_ssize_t, that means the threshold for negative values is much higher.

This is all just the background to illustrate that becausesize is not anint, a pickle with theBINSTRING opcode using a length > 0x7fffffff will fail in the Python implementation (since it's negative), but deserialize just fine in the C implementation.

The following payload will demonstrate the difference:

payload:      b'T\x00\x00\x00\x80'+b'a'*0x80000000 + b'.'pickle:       FAILURE BINSTRING pickle has negative byte count_pickle.c:    aaaaaaaaaaaaaaaaaaa....pickletools:  pickletools failed string4 byte count < 0: -2147483648

The required payload does need to be 2GB in size which is very large, but not impossible on modern systems.

Note that theLONG4 opcode is in a similar situation, except the output forcalc_binint() is anint data type so this issue does not exist there.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-135321: Changing data type ofsize variable forBINSTRING in_pickle #135322
• [3.14] gh-135321: Always raise a correct exception for BINSTRING argument > 0x7fffffff in pickle (GH-135322) #135382
• [3.13] gh-135321: Always raise a correct exception for BINSTRING argument > 0x7fffffff in pickle (GH-135322) #135383