Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
Closed
Description
Bug report
Bug description:
I noticed that chainingstruct.unpack() andstruct.pack() for IEEE 754 Half Precision floats (e) is non-invertible fornan. E.g.:
importstructoriginal_bytes=b'\xff\xff'unpacked_float=struct.unpack('e',original_bytes)[0]# nanrepacked_bytes=struct.pack('e',unpacked_float)# b'\x00\xfe' != b'\xff\xff'
IEEEnans aren't unique, so this isn'tthat surprising... However I found it curious that the same behavior is not exhibited forfloat (f) ordouble (d) format, where every original bit pattern I tested could be recovered from the unpackednan object.
Is this by design?
Here's a quickpytest script that tests over a broad range ofnan/inf/-inf cases for each encoding format.
# /// script# requires-python = ">=3.11"# dependencies = ["pytest"]# ///importstructimportpytest# Floating Point Encodings Based on IEEE 754 per https://en.wikipedia.org/wiki/IEEE_754#Basic_and_interchange_formats# binary 16 (half precision) - 1 bit sign, 5 bit exponent, 11 bit significand# binary 32 (single precision) - 1 bit sign, 8 bit exponent, 23 bit significand# binary 64 (double precision) - 1 bit sign, 11 bit exponent, 52 bit significandMAX_TEST_CASES=100000# limit number of bit patterns being sampled so we aren't waiting too long@pytest.mark.parametrize(["precision_format","precision","exponent_bits"], [("f",32,8), ("d",64,11), ("e",16,5)])@pytest.mark.parametrize("sign_bit", [0,1])@pytest.mark.parametrize("endianness", ["little","big"])deftest_struct_floats(precision_format:str,precision:int,exponent_bits:int,sign_bit:int,endianness:str):significand_bits=precision-exponent_bits-1n_tests=min(MAX_TEST_CASES,2**significand_bits)significand_patterns= [significand_bits*"0",significand_bits*"1"]+ [bin(i+1)[2:]foriinrange(1,2**significand_bits,2**significand_bits//n_tests) ]foriinrange(n_tests):binary=str(sign_bit)+"1"*exponent_bits+significand_patterns[i]ifendianness=="big":format=">"+precision_formatelifendianness=="little":format="<"+precision_formatelse:raiseNotImplementedError()test_bytes=int(binary,base=2).to_bytes(precision//8,endianness)unpacked=struct.unpack(format,test_bytes)assertlen(unpacked)==1repacked=struct.pack(format,unpacked[0])assert (repacked==test_bytes ),f"struct pack/unpack was not invertible for format{format} with raw value:{test_bytes} -> unpacks to{unpacked[0]}, repacks to{repacked}"if__name__=="__main__":pytest.main([__file__])
CPython versions tested on:
3.13, 3.11, 3.12
Operating systems tested on:
Linux, Windows
Linked PRs
- gh-130317: fix PyFloat_Pack/Unpack[24] for NaN's with payload #130452
- gh-130317: Fix strict aliasing in PyFloat_Pack8() #133150
- gh-130317: Skip test_pack_unpack_roundtrip_for_nans() on x86 #133155
- gh-130317: fix test_pack_unpack_roundtrip() and add docs #133204
- gh-130317: Fix SNaN broken tests on HP PA RISC #140452
- [3.14] gh-130317: Fix SNaN broken tests on HP PA RISC (GH-140452) #140467
