Background¶

Inline encryption hardware sits logically between memory and the disk, and canen/decrypt data as it goes in/out of the disk. Inline encryption hardware has afixed number of “keyslots” - slots into which encryption contexts (i.e. theencryption key, encryption algorithm, data unit size) can be programmed by thekernel at any time. Each request sent to the disk can be tagged with the indexof a keyslot (and also a data unit number to act as an encryption tweak), andthe inline encryption hardware will en/decrypt the data in the request with theencryption context programmed into that keyslot. This is very different fromfull disk encryption solutions like self encrypting drives/TCG OPAL/ATASecurity standards, since with inline encryption, any block on disk could beencrypted with any encryption context the kernel chooses.

Objective¶

We want to support inline encryption (IE) in the kernel.To allow for testing, we also want a crypto API fallback when actualIE hardware is absent. We also want IE to work with layered deviceslike dm and loopback (i.e. we want to be able to use the IE hardwareof the underlying devices if present, or else fall back to crypto APIen/decryption).

Constraints and notes¶

• IE hardware has a limited number of “keyslots” that can be programmedwith an encryption context (key, algorithm, data unit size, etc.) at any time.One can specify a keyslot in a data request made to the device, and thedevice will en/decrypt the data using the encryption context programmed intothat specified keyslot. When possible, we want to make multiple requests withthe same encryption context share the same keyslot.
• We need a way for upper layers like filesystems to specify an encryptioncontext to use for en/decrypting a struct bio, and a device driver (like UFS)needs to be able to use that encryption context when it processes the bio.
• We need a way for device drivers to expose their inline encryptioncapabilities in a unified way to the upper layers.

Design¶

We add astructbio_crypt_ctx tostructbio that canrepresent an encryption context, because we need to be able to pass thisencryption context from the upper layers (like the fs layer) to thedevice driver to act upon.

While IE hardware works on the notion of keyslots, the FS layer has noknowledge of keyslots - it simply wants to specify an encryption context touse while en/decrypting a bio.

We introduce a keyslot manager (KSM) that handles the translation fromencryption contexts specified by the FS to keyslots on the IE hardware.This KSM also serves as the way IE hardware can expose its capabilities toupper layers. The generic mode of operation is: each device driver that wantsto support IE will construct a KSM and set it up in its struct request_queue.Upper layers that want to use IE on this device can then use this KSM inthe device’s struct request_queue to translate an encryption context intoa keyslot. The presence of the KSM in the request queue shall be used to meanthat the device supports IE.

The KSM uses refcounts to track which keyslots are idle (either they have noencryption context programmed, or there are no in-flight struct biosreferencing that keyslot). When a new encryption context needs a keyslot, ittries to find a keyslot that has already been programmed with the sameencryption context, and if there is no such keyslot, it evicts the leastrecently used idle keyslot and programs the new encryption context into thatone. If no idle keyslots are available, then the caller will sleep until thereis at least one.

blk-mq changes, other block layer changes and blk-crypto-fallback¶

We add a pointer to abi_crypt_context andkeyslot tostructrequest. These will be referred to as thecryptofieldsfor the request. Thiskeyslot is the keyslot into which thebi_crypt_context has been programmed in the KSM of therequest_queuethat this request is being sent to.

We introduceblock/blk-crypto-fallback.c, which allows upper layers to remainblissfully unaware of whether or not real inline encryption hardware is presentunderneath. When a bio is submitted with a targetrequest_queue that doesn’tsupport the encryption context specified with the bio, the block layer willen/decrypt the bio with the blk-crypto-fallback.

If the bio is aWRITE bio, a bounce bio is allocated, and the data in the biois encrypted stored in the bounce bio - blk-mq will then proceed to process thebounce bio as if it were not encrypted at all (except when blk-integrity isconcerned).blk-crypto-fallback sets the bounce bio’sbi_end_io to aninternal function that cleans up the bounce bio and ends the original bio.

If the bio is aREAD bio, the bio’sbi_end_io (and alsobi_private)is saved and overwritten byblk-crypto-fallback tobio_crypto_fallback_decrypt_bio. The bio’sbi_crypt_context is alsooverwritten withNULL, so that to the rest of the stack, the bio looksas if it was a regular bio that never had an encryption context specified.bio_crypto_fallback_decrypt_bio will decrypt the bio, restore the originalbi_end_io (and alsobi_private) and end the bio again.

Regardless of whether real inline encryption hardware is used or theblk-crypto-fallback is used, the ciphertext written to disk (and hence theon-disk format of data) will be the same (assuming the hardware’s implementationof the algorithm being used adheres to spec and functions correctly).

If arequestqueue’s inline encryption hardware claimed to support theencryption context specified with a bio, then it will not be handled by theblk-crypto-fallback. We will eventually reach a point in blk-mq when astructrequest needs to be allocated for that bio. At that point,blk-mq tries to program the encryption context into therequest_queue’skeyslot_manager, and obtain a keyslot, which it stores in its newly addedkeyslot field. This keyslot is released when the request is completed.

When the first bio is added to a request,blk_crypto_rq_bio_prep is called,which sets the request’scrypt_ctx to a copy of the bio’sbi_crypt_context. bio_crypt_do_front_merge is called whenever a subsequentbio is merged to the front of the request, which updates thecrypt_ctx ofthe request so that it matches the newly merged bio’sbi_crypt_context. In particular, the request keeps a copy of thebi_crypt_context of the firstbio in its bio-list (blk-mq needs to be careful to maintain this invariantduring bio and request merges).

To make it possible for inline encryption to work with request queue basedlayered devices, when a request is cloned, itscryptofields are cloned aswell. When the cloned request is submitted, blk-mq programs thebi_crypt_context of the request into the clone’s request_queue’s keyslotmanager, and stores the returned keyslot in the clone’skeyslot.

API presented to users of the block layer¶

structblk_crypto_key represents a crypto key (the raw key, size of thekey, the crypto algorithm to use, the data unit size to use, and the number ofbytes required to represent data unit numbers that will be specified with thebi_crypt_context).

blk_crypto_init_key allows upper layers to initialize such ablk_crypto_key.

bio_crypt_set_ctx should be called on any bio that a user ofthe block layer wants en/decrypted via inline encryption (or theblk-crypto-fallback, if hardware support isn’t available for the desiredcrypto configuration). This function takes theblk_crypto_key and thedata unit number (DUN) to use when en/decrypting the bio.

blk_crypto_config_supported allows upper layers to query whether or not thean encryption context passed to request queue can be handled by blk-crypto(either by real inline encryption hardware, or by the blk-crypto-fallback).This is useful e.g. when blk-crypto-fallback is disabled, and the upper layerwants to use an algorithm that may not supported by hardware - this functionlets the upper layer know ahead of time that the algorithm isn’t supported,and the upper layer can fallback to something else if appropriate.

blk_crypto_start_using_key - Upper layers must call this function onblk_crypto_key and arequest_queue before using the key with any bioheaded for thatrequest_queue. This function ensures that either thehardware supports the key’s crypto settings, or the crypto API fallback hastransforms for the needed mode allocated and ready to go. Note that thisfunction may allocate anskcipher, and must not be called from the datapath, since allocatingskciphers from the data path can deadlock.

blk_crypto_evict_keymust be called by upper layers before ablk_crypto_key is freed. Further, itmust only be called only oncethere are no more in-flight requests that use thatblk_crypto_key.blk_crypto_evict_key will ensure that a key is removed from any keyslots ininline encryption hardware that the key might have been programmed into (or the blk-crypto-fallback).

API presented to device drivers¶

A :c:type:structblk_keyslot_manager should be set up by device drivers intherequest_queue of the device. The device driver needs to callblk_ksm_init on theblk_keyslot_manager, which specifying the number ofkeyslots supported by the hardware.

The device driver also needs to tell the KSM how to actually manipulate theIE hardware in the device to do things like programming the crypto key intothe IE hardware into a particular keyslot. All this is achieved through thestructblk_ksm_ll_ops field in the KSM that the device drivermust fill up after initing theblk_keyslot_manager.

The KSM also handles runtime power management for the device when applicable(e.g. when it wants to program a crypto key into the IE hardware, the devicemust be runtime powered on) - so the device driver must also set thedevfield in the ksm to point to thestruct device for the KSM to use for runtimepower management.

blk_ksm_reprogram_all_keys can be called by device drivers if the deviceneeds each and every of its keyslots to be reprogrammed with the key it“should have” at the point in time when the function is called. This is usefule.g. if a device loses all its keys on runtime power down/up.

blk_ksm_destroy should be called to free up all resources used by a keyslotmanager uponblk_ksm_init, once theblk_keyslot_manager is no longerneeded.

Layered Devices¶

Request queue based layered devices like dm-rq that wish to support IE need tocreate their own keyslot manager for their request queue, and expose whateverfunctionality they choose. When a layered device wants to pass a clone of thatrequest to anotherrequest_queue, blk-crypto will initialize and prepare theclone as necessary - seeblk_crypto_insert_cloned_request inblk-crypto.c.

Future Optimizations for layered devices¶

Creating a keyslot manager for a layered device uses up memory for eachkeyslot, and in general, a layered device merely passes the request on to a“child” device, so the keyslots in the layered device itself are completelyunused, and don’t need any refcounting or keyslot programming. We can insteaddefine a new type of KSM; the “passthrough KSM”, that layered devices can useto advertise an unlimited number of keyslots, and support for any encryptionalgorithms they choose, while not actually using any memory for each keyslot.Another use case for the “passthrough KSM” is for IE devices that do not have alimited number of keyslots.

Interaction between inline encryption and blk integrity¶

At the time of this patch, there is no real hardware that supports both thesefeatures. However, these features do interact with each other, and it’s notcompletely trivial to make them both work together properly. In particular,when a WRITE bio wants to use inline encryption on a device that supports bothfeatures, the bio will have an encryption context specified, after whichits integrity information is calculated (using the plaintext data, sincethe encryption will happen while data is being written), and the data andintegrity info is sent to the device. Obviously, the integrity info must beverified before the data is encrypted. After the data is encrypted, the devicemust not store the integrity info that it received with the plaintext datasince that might reveal information about the plaintext data. As such, it mustre-generate the integrity info from the ciphertext data and store that on diskinstead. Another issue with storing the integrity info of the plaintext data isthat it changes the on disk format depending on whether hardware inlineencryption support is present or the kernel crypto API fallback is used (sinceif the fallback is used, the device will receive the integrity info of theciphertext, not that of the plaintext).

Because there isn’t any real hardware yet, it seems prudent to assume thathardware implementations might not implement both features together correctly,and disallow the combination for now. Whenever a device supports integrity, thekernel will pretend that the device does not support hardware inline encryption(by essentially setting the keyslot manager in the request_queue of the deviceto NULL). When the crypto API fallback is enabled, this means that all bios withand encryption context will use the fallback, and IO will complete as usual.When the fallback is disabled, a bio with an encryption context will be failed.