Generic bitfield packing and unpacking functions¶
Problem statement¶
When working with hardware, one has to choose between several approaches ofinterfacing with it.One can memory-map a pointer to a carefully craftedstructover the hardwaredevice’s memory region, and access its fields asstructmembers (potentiallydeclared as bitfields). But writing code this way would make it less portable,due to potential endianness mismatches between the CPU and the hardware device.Additionally, one has to pay close attention when translating registerdefinitions from the hardware documentation into bit field indices for thestructs. Also, some hardware (typically networking equipment) tends to groupits register fields in ways that violate any reasonable word boundaries(sometimes even 64 bit ones). This creates the inconvenience of having todefine “high” and “low” portions of register fields within the struct.A more robust alternative tostructfield definitions would be to extract therequired fields by shifting the appropriate number of bits. But this wouldstill not protect from endianness mismatches, except if all memory accesseswere performed byte-by-byte. Also the code can easily get cluttered, and thehigh-level idea might get lost among the many bit shifts required.Many drivers take the bit-shifting approach and then attempt to reduce theclutter with tailored macros, but more often than not these macros takeshortcuts that still prevent the code from being truly portable.
The solution¶
This API deals with 2 basic operations:
Packing a CPU-usable number into a memory buffer (with hardwareconstraints/quirks)
Unpacking a memory buffer (which has hardware constraints/quirks)into a CPU-usable number.
The API offers an abstraction over said hardware constraints and quirks,over CPU endianness and therefore between possible mismatches betweenthe two.
The basic unit of these API functions is the u64. From the CPU’sperspective, bit 63 always means bit offset 7 of byte 7, albeit onlylogically. The question is: where do we lay this bit out in memory?
The following examples cover the memory layout of a packed u64 field.The byte offsets in the packed buffer are always implicitly 0, 1, ... 7.What the examples show is where the logical bytes and bits sit.
Normally (no quirks), we would do it like this:
63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 327 6 5 431 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 03 2 1 0
That is, the MSByte (7) of the CPU-usable u64 sits at memory offset 0, and theLSByte (0) of the u64 sits at memory offset 7.This corresponds to what most folks would regard to as “big endian”, wherebit i corresponds to the number 2^i. This is also referred to in the codecomments as “logical” notation.
If QUIRK_MSB_ON_THE_RIGHT is set, we do it like this:
56 57 58 59 60 61 62 63 48 49 50 51 52 53 54 55 40 41 42 43 44 45 46 47 32 33 34 35 36 37 38 397 6 5 424 25 26 27 28 29 30 31 16 17 18 19 20 21 22 23 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 73 2 1 0
That is, QUIRK_MSB_ON_THE_RIGHT does not affect byte positioning, butinverts bit offsets inside a byte.
If QUIRK_LITTLE_ENDIAN is set, we do it like this:
39 38 37 36 35 34 33 32 47 46 45 44 43 42 41 40 55 54 53 52 51 50 49 48 63 62 61 60 59 58 57 564 5 6 77 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 23 22 21 20 19 18 17 16 31 30 29 28 27 26 25 240 1 2 3
Therefore, QUIRK_LITTLE_ENDIAN means that inside the memory region, everybyte from each 4-byte word is placed at its mirrored position compared tothe boundary of that word.
If QUIRK_MSB_ON_THE_RIGHT and QUIRK_LITTLE_ENDIAN are both set, we do itlike this:
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 634 5 6 70 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 310 1 2 3
If just QUIRK_LSW32_IS_FIRST is set, we do it like this:
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 03 2 1 063 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 327 6 5 4
In this case the 8 byte memory region is interpreted as follows: first4 bytes correspond to the least significant 4-byte word, next 4 bytes tothe more significant 4-byte word.
If QUIRK_LSW32_IS_FIRST and QUIRK_MSB_ON_THE_RIGHT are set, we do it likethis:
24 25 26 27 28 29 30 31 16 17 18 19 20 21 22 23 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 73 2 1 056 57 58 59 60 61 62 63 48 49 50 51 52 53 54 55 40 41 42 43 44 45 46 47 32 33 34 35 36 37 38 397 6 5 4
If QUIRK_LSW32_IS_FIRST and QUIRK_LITTLE_ENDIAN are set, it looks likethis:
7 6 5 4 3 2 1 0 15 14 13 12 11 10 9 8 23 22 21 20 19 18 17 16 31 30 29 28 27 26 25 240 1 2 339 38 37 36 35 34 33 32 47 46 45 44 43 42 41 40 55 54 53 52 51 50 49 48 63 62 61 60 59 58 57 564 5 6 7
If QUIRK_LSW32_IS_FIRST, QUIRK_LITTLE_ENDIAN and QUIRK_MSB_ON_THE_RIGHTare set, it looks like this:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 310 1 2 332 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 634 5 6 7
We always think of our offsets as if there were no quirk, and we translatethem afterwards, before accessing the memory region.
Note on buffer lengths not multiple of 4¶
To deal with memory layout quirks where groups of 4 bytes are laid out “littleendian” relative to each other, but “big endian” within the group itself, theconcept of groups of 4 bytes is intrinsic to the packing API (not to beconfused with the memory access, which is performed byte by byte, though).
With buffer lengths not multiple of 4, this means one group will be incomplete.Depending on the quirks, this may lead to discontinuities in the bit fieldsaccessible through the buffer. The packing API assumes discontinuities were notthe intention of the memory layout, so it avoids them by effectively logicallyshortening the most significant group of 4 octets to the number of octetsactually available.
Example with a 31 byte sized buffer given below. Physical buffer offsets areimplicit, and increase from left to right within a group, and from top tobottom within a column.
No quirks:
31 29 28 | Group 7 (most significant)27 26 25 24 | Group 623 22 21 20 | Group 519 18 17 16 | Group 415 14 13 12 | Group 311 10 9 8 | Group 2 7 6 5 4 | Group 1 3 2 1 0 | Group 0 (least significant)
QUIRK_LSW32_IS_FIRST:
3 2 1 0 | Group 0 (least significant) 7 6 5 4 | Group 111 10 9 8 | Group 215 14 13 12 | Group 319 18 17 16 | Group 423 22 21 20 | Group 527 26 25 24 | Group 630 29 28 | Group 7 (most significant)
QUIRK_LITTLE_ENDIAN:
30 28 29 | Group 7 (most significant)24 25 26 27 | Group 620 21 22 23 | Group 516 17 18 19 | Group 412 13 14 15 | Group 3 8 9 10 11 | Group 2 4 5 6 7 | Group 1 0 1 2 3 | Group 0 (least significant)
QUIRK_LITTLE_ENDIAN | QUIRK_LSW32_IS_FIRST:
0 1 2 3 | Group 0 (least significant) 4 5 6 7 | Group 1 8 9 10 11 | Group 212 13 14 15 | Group 316 17 18 19 | Group 420 21 22 23 | Group 524 25 26 27 | Group 628 29 30 | Group 7 (most significant)
Intended use¶
Drivers that opt to use this API first need to identify which of the above 3quirk combinations (for a total of 8) match what the hardware documentationdescribes.
There are 3 supported usage patterns, detailed below.
packing()¶
This API function is deprecated.
Thepacking() function returns an int-encoded error code, which protects theprogrammer against incorrect API use. The errors are not expected to occurduring runtime, therefore it is reasonable to wrappacking() into a customfunction which returns void and swallows those errors. Optionally it candump stack or print the error description.
voidmy_packing(void*buf,u64*val,intstartbit,intendbit,size_tlen,enumpacking_opop){interr;/* Adjust quirks accordingly */err=packing(buf,val,startbit,endbit,len,op,QUIRK_LSW32_IS_FIRST);if(likely(!err))return;if(err==-EINVAL){pr_err("Start bit (%d) expected to be larger than end (%d)\n",startbit,endbit);}elseif(err==-ERANGE){if((startbit-endbit+1)>64)pr_err("Field %d-%d too large for 64 bits!\n",startbit,endbit);elsepr_err("Cannot store %llx inside bits %d-%d (would truncate)\n",*val,startbit,endbit);}dump_stack();}
pack() and unpack()¶
These are const-correct variants ofpacking(), and eliminate the last “enumpacking_op op” argument.
Calling pack(...) is equivalent, and preferred, to calling packing(..., PACK).
Calling unpack(...) is equivalent, and preferred, to calling packing(..., UNPACK).
pack_fields() and unpack_fields()¶
The library exposes optimized functions for the scenario where there are manyfields represented in a buffer, and it encourages consumer drivers to avoidrepetitive calls topack() andunpack() for each field, but instead usepack_fields() andunpack_fields(), which reduces the code footprint.
These APIs use field definitions in arrays ofstructpacked_field_u8 orstructpacked_field_u16, allowing consumer drivers to minimize the sizeof these arrays according to their custom requirements.
Thepack_fields() andunpack_fields() API functions are actually macros whichautomatically select the appropriate function at compile time, based on thetype of the fields array passed in.
An additional benefit overpack() andunpack() is that sanity checks on thefield definitions are handled at compile time withBUILD_BUG_ON ratherthan only when the offending code is executed. These functions return void andwrapping them to handle unexpected errors is not necessary.
It is recommended, but not required, that you wrap your packed buffer into astructured type with a fixed size. This generally makes it easier for thecompiler to enforce that the correct size buffer is used.
Here is an example of how to use the fields APIs:
/* Ordering inside the unpacked structure is flexible and can be different * from the packed buffer. Here, it is optimized to reduce padding. */structdata{u64field3;u32field4;u16field1;u8field2;};#define SIZE 13typedefstruct__packed{u8buf[SIZE];}packed_buf_t;staticconststructpacked_field_u8fields[]={PACKED_FIELD(100,90,structdata,field1),PACKED_FIELD(90,87,structdata,field2),PACKED_FIELD(86,30,structdata,field3),PACKED_FIELD(29,0,structdata,field4),};voidunpack_your_data(constpacked_buf_t*buf,structdata*unpacked){BUILD_BUG_ON(sizeof(*buf)!=SIZE;unpack_fields(buf,sizeof(*buf),unpacked,fields,QUIRK_LITTLE_ENDIAN);}voidpack_your_data(conststructdata*unpacked,packed_buf_t*buf){BUILD_BUG_ON(sizeof(*buf)!=SIZE;pack_fields(buf,sizeof(*buf),unpacked,fields,QUIRK_LITTLE_ENDIAN);}