Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
gh-112433: Add optional _align_ attribute to ctypes.Structure#113790
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
gh-112433: Add optional _align_ attribute to ctypes.Structure#113790
Uh oh!
There was an error while loading.Please reload this page.
Conversation
ghost commentedJan 7, 2024 • edited by ghost
Loading Uh oh!
There was an error while loading.Please reload this page.
edited by ghost
Uh oh!
There was an error while loading.Please reload this page.
All commit authors signed the Contributor License Agreement. |
Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It is an interesting idea.
But what should be the size and the alignment ofB in the following example?
classA(Structure):x:c_uint32classB(A):_align_=8y:c_uint32
Thanks for the review! My intuition would say "8 and 8". |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
!buildbot s390x |
bedevere-bot commentedJan 28, 2024
🤖 New build scheduled with the buildbot fleet by@serhiy-storchaka for commit2329fa3 🤖 The command will test the builders whose names match following regular expression: The builders matched are:
|
https://buildbot.python.org/all/#/builders/390/builds/1350/steps/5/logs/stdio |
In the process of making the tests run with both big and little endian versions of the fromctypesimport*classAlignedUnion(BigEndianUnion):_fields_= [ ("a",c_uint32), ("b",c_int32), ]classMain(BigEndianStructure):_fields_= [ ("first",c_uint32), ("union",AlignedUnion), ]m=Main() If I run this on my machine (little endian), I get the following error: This seems like a bug and is the reason the last few tests look the way they do... |
It was fixed not long time ago in#105102. |
monkeyman192 commentedJan 28, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Ok great! I know that as per the requirements for PR's it's required to make each commit separate and to not rebase, but is it fine to rebase the whole branch onto the current head so that I get the fix for this? I haven't contributed before and just want to make sure I'm following the right procedure... Edit: Oh I see you already merged the code into the branch, thanks! |
serhiy-storchaka commentedJan 28, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Don't rebase, just merge. I just merged |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Some questions and suggestions to tests. Please add more checks for offsets, alignments and sizes.
Interesting, how does setting_align_ in a Union subclass affect it?
| self.assertEqual(main.first, 7) | ||
| self.assertEqual(main.string.value, b'hello world!') | ||
| self.assertEqual( | ||
| bytes(main.string.__buffer__(inspect.BufferFlags.SIMPLE)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Maybe simplybytes(main.string)?
| self.assertEqual(main.string.value, b'hello world!') | ||
| self.assertEqual( | ||
| bytes(main.string.__buffer__(inspect.BufferFlags.SIMPLE)), | ||
| b'\x68\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64\x21\x00\x00\x00\x00' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
b'hello world!\0\0\0\0' could be more readable.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
| if ubase == LittleEndianUnion: | ||
| self.assertEqual(bytes(main.union.b), b"\3\0\0\0\4\0\0\0") | ||
| else: | ||
| self.assertEqual(bytes(main.union.b), b"\0\0\0\3\0\0\0\4") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It is simplydata[8:].
| class TestAlignedStructures(unittest.TestCase): | ||
| def test_aligned_string(self): | ||
| for base, data in ( | ||
| (LittleEndianStructure, bytearray( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
You can also usestruct.pack() to generate data. It may be more compact and more readable.
Thanks for the thorough review! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thank you for your response@monkeyman192. I have yet few suggestions for polishing test.
| self.assertEqual(Main.string.offset, 16) | ||
| self.assertEqual(Main.string.size, 16) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It is not affected by the_align_ setting, because its size was already 16. Maybe make thevalue field typec_char * 12? Then the_align_ setting will affect also the size ofAligned.
| ("y", c_uint32), | ||
| ("x", c_ubyte), | ||
| ("y", SomeBools), | ||
| ("z", c_uint32), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What if makez a byte and makeSomeBools containing only 2 fields? Then without the_align_ settingMain can be packed in just 4 bytes. The_align_ setting will made larger difference.
Or just set_align_ to 8. This would cover also the next test.
| ("y", SomeBoolsTooBig), | ||
| ("z", c_uint32), | ||
| ] | ||
| with self.assertRaises(ValueError) as ctx: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It is still not very informative test. If you want to test that the_align_ setting affects the size ofMain, you can do this more directly by checkingsizeof(Main). If you want to check also content, use the data of size 12.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I guess with this test I was just wanting to ensure that the appropriate error is raised when you have a Structure/Union with an alignment which will cause the size it expects to be bigger than what it is given. Rather than checking anything about the content itself.
| self.assertEqual(main.unsigned, 0xD6) | ||
| self.assertEqual(main.signed, -42) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Oh, it is difficult to test unions with different byte order. But your test will also pass if badding bytes are added beforeunsigned (i.e. ifMain.unsigned.offset is 7). I would use all different bytes in the input data, and tested:
self.assertEqual(main.unsigned,data[0])self.assertEqual(main.signed,data[0]-256)
Please add also checks for alignments, offsets and sizes.
| self.assertEqual(bytes(main.string), b'hello world!\0\0\0\0') | ||
| self.assertEqual(Main.string.offset, 16) | ||
| self.assertEqual(Main.string.size, 16) | ||
| self.assertEqual(alignment(main.string), 16) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Doesalignment(Main.string) work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It does not sinceMain.string is actypes.CField which has no alignment info.
I realised that I have neither documented nor tested the case of having both |
Uh oh!
There was an error while loading.Please reload this page.
Sorry for the ping@serhiy-storchaka just checking to see if there was anything else I needed to address with this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM.
!buildbot s390x Debian |
bedevere-bot commentedFeb 12, 2024
🤖 New build scheduled with the buildbot fleet by@serhiy-storchaka for commita7bc0fe 🤖 The command will test the builders whose names match following regular expression: The builders matched are:
|
Ah, I just realised that this was already in the |
Uh oh!
There was an error while loading.Please reload this page.
This PR adds an
_align_attribute toctypes.Structure's so that is it possible to set it manually.📚 Documentation preview 📚:https://cpython-previews--113790.org.readthedocs.build/