Backport a fix from the v7 inline crypto patchset which ensures that the
block layer knows the number of DUN bytes the inline encryption hardware
supports, so that hardware isn't used when it shouldn't be.
(This unfortunately means introducing some increasing long argument
lists; this was all already fixed up in later versions of the patchset.)
To avoid breaking the KMI for drivers, don't add a dun_bytes argument to
keyslot_manager_create() but rather allow drivers to call
keyslot_manager_set_max_dun_bytes() to override the default. Also,
don't add dun_bytes as a new field in 'struct blk_crypto_key' but rather
pack it into the existing 'hash' field which is for block layer use.
Bug: 144046242
Bug: 153512828
Change-Id: I285f36557fb3eafc5f2f64727ef1740938b59dd7
Signed-off-by: Eric Biggers <ebiggers@google.com>
We need a way to tell which type of keys the inline crypto hardware
supports (standard, wrapped, or both), so that fallbacks can be used
when needed (either blk-crypto-fallback, or fscrypt fs-layer crypto).
We can't simply assume that
keyslot_mgmt_ll_ops::derive_raw_secret == NULL
means only standard keys are supported and that
keyslot_mgmt_ll_ops::derive_raw_secret != NULL
means that only wrapped keys are supported, because device-mapper
devices always implement this method. Also, hardware might support both
types of keys.
Therefore, add a field keyslot_manager::features which contains a
bitmask of flags which indicate the supported types of keys. Drivers
will need to fill this in. This patch makes the UFS standard crypto
code set BLK_CRYPTO_FEATURE_STANDARD_KEYS, but UFS variant drivers may
need to set BLK_CRYPTO_FEATURE_WRAPPED_KEYS instead.
Then, make keyslot_manager_crypto_mode_supported() take the key type
into account.
Bug: 137270441
Bug: 151100202
Test: 'atest vts_kernel_encryption_test' on Pixel 4 with the
inline crypto patches backported, and also on Cuttlefish.
Change-Id: Ied846c2767c1fd2f438792dcfd3649157e68b005
Signed-off-by: Eric Biggers <ebiggers@google.com>
The call to pm_runtime_get_sync() in ufshcd_program_key() can deadlock
because it waits for the UFS controller to be resumed, but it can itself
be reached while resuming the UFS controller via:
- ufshcd_runtime_resume()
- ufshcd_resume()
- ufshcd_reset_and_restore()
- ufshcd_host_reset_and_restore()
- ufshcd_hba_enable()
- ufshcd_hba_execute_hce()
- ufshcd_hba_start()
- ufshcd_crypto_enable()
- keyslot_manager_reprogram_all_keys()
- ufshcd_crypto_keyslot_program()
- ufshcd_program_key()
But pm_runtime_get_sync() *is* needed when evicting a key. Also, on
pre-4.20 kernels it's needed when programming a keyslot for a bio since
the block layer used to resume the device in a different place.
Thus, it's hard for drivers to know what to do in .keyslot_program() and
.keyslot_evict(). In old kernels it may even be impossible unless we
were to pass more information down from the keyslot_manager.
There's also another possible deadlock: keyslot programming and eviction
take ksm->lock for write and then resume the device, which may result in
ksm->lock being taken again via the above call stack. To fix this, we
should resume the device before taking ksm->lock.
Fix these problems by moving to a better design where the block layer
(namely, the keyslot manager) handles runtime power management instead
of drivers. This is analogous to the block layer's existing runtime
power management support (blk-pm), which handles resuming devices when
bios are submitted to them so that drivers don't need to handle it.
Test: Tested on coral with:
echo 5 > /sys/bus/platform/devices/1d84000.ufshc/rpm_lvl
sleep 30
touch /data && sync # hangs before this fix
Also verified via kvm-xfstests that blk-crypto-fallback continues
to work both with and without CONFIG_PM=y.
Bug: 137270441
Bug: 149368295
Change-Id: I6bc9fb81854afe7edf490d71796ee68a61f7cbc8
Signed-off-by: Eric Biggers <ebiggers@google.com>
Update the device-mapper core to support exposing the inline crypto
support of the underlying device(s) through the device-mapper device.
This works by creating a "passthrough keyslot manager" for the dm
device, which declares support for the set of (crypto_mode,
data_unit_size) combos which all the underlying devices support. When a
supported combo is used, the bio cloning code handles cloning the crypto
context to the bios for all the underlying devices. When an unsupported
combo is used, the blk-crypto fallback is used as usual.
Crypto support on each underlying device is ignored unless the
corresponding dm target opts into exposing it. This is needed because
for inline crypto to semantically operate on the original bio, the data
must not be transformed by the dm target. Thus, targets like dm-linear
can expose crypto support of the underlying device, but targets like
dm-crypt can't. (dm-crypt could use inline crypto itself, though.)
When a key is evicted from the dm device, it is evicted from all
underlying devices.
Bug: 137270441
Bug: 147814592
Change-Id: If28b574f2e28268db5eb9f325d4cf8f96cb63e3f
Signed-off-by: Eric Biggers <ebiggers@google.com>
The regular keyslot manager is designed for devices that have a small
number of keyslots that need to be programmed with keys ahead of time,
and bios that are sent to the device need to be tagged with a keyslot
index.
Some inline encryption hardware may not have any limitations on the
number of keyslot, and may instead allow each bio to be tagged with
a raw key, data unit number, etc. rather than a pre-programmed keyslot's
index. These devices don't need any sort of keyslot management, and it's
better for these devices not to have to allocate a regular keyslot
manager with some fixed number of keyslots. These devices can instead
set up a passthrough keyslot manager in their request queue, which
require less resources than regular keyslot managers, as they simply
do no-ops when trying to program keys into slots.
Separately, the device mapper may map over devices that have inline
encryption hardware, and it wants to pass the key along to the
underlying hardware. While the DM layer can expose inline encryption
capabilities by setting up a regular keyslot manager with some fixed
number of keyslots in the dm device's request queue, this only wastes
memory since the keys programmed into the dm device's request queue
will never be used. Instead, it's better to set up a passthrough
keyslot manager for dm devices.
Bug: 137270441
Bug: 147814592
Change-Id: I6d91e83e86a73b0d6066873c8a9117cf2c089234
Signed-off-by: Satya Tangirala <satyat@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Some inline encryption hardware supports protecting the keys in hardware
and only exposing wrapped keys to software. To use this capability,
userspace must provide a hardware-wrapped key rather than a raw key.
However, users of inline encryption in the kernel won't necessarily use
the user-specified key directly for inline encryption. E.g. with
fscrypt with IV_INO_LBLK_64 policies, each user-provided key is used to
derive a file contents encryption key, filenames encryption key, and key
identifier. Since inline encryption can only be used with file
contents, if the user were to provide a wrapped key there would
(naively) be no way to encrypt filenames or derive the key identifier.
This problem is solved by designing the hardware to internally use the
unwrapped key as input to a KDF from which multiple cryptographically
isolated keys can be derived, including both the inline crypto key (not
exposed to software) and a secret that *is* exposed to software.
Add a function to the keyslot manager to allow upper layers to request
this software secret from a hardware-wrapped key.
Bug: 147209885
Change-Id: Iffb05b297b7ba3f3e865e798e4bb73aef4e6ba19
Co-developed-by: Gaurav Kashyap <gaurkash@codeaurora.org>
Signed-off-by: Gaurav Kashyap <gaurkash@codeaurora.org>
Signed-off-by: Barani Muthukumaran <bmuthuku@codeaurora.org>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Currently, blk-crypto uses the algorithm to determine the size of keys.
However, some inline encryption hardware supports protecting keys from
software by wrapping the storage keys with an ephemeral key. Since
these wrapped keys are not of a fixed size, add the capability to
provide the key size when initializing a blk_crypto_key, and update the
keyslot manager to take size into account when comparing keys.
Bug: 147209885
Change-Id: I9bf26d06d18a2d671c51111b4896abe4df303988
Co-developed-by: Gaurav Kashyap <gaurkash@codeaurora.org>
Signed-off-by: Gaurav Kashyap <gaurkash@codeaurora.org>
Signed-off-by: Barani Muthukumaran <bmuthuku@codeaurora.org>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Changes v5 => v6:
- Blk-crypto's kernel crypto API fallback is no longer restricted to
8-byte DUNs. It's also now separately configurable from blk-crypto, and
can be disabled entirely, while still allowing the kernel to use inline
encryption hardware. Further, struct bio_crypt_ctx takes up less space,
and no longer contains the information needed by the crypto API
fallback - the fallback allocates the required memory when necessary.
- Blk-crypto now supports all file content encryption modes supported by
fscrypt.
- Fixed bio merging logic in blk-merge.c
- Fscrypt now supports inline encryption with the direct key policy, since
blk-crypto now has support for larger DUNs.
- Keyslot manager now uses a hashtable to lookup which keyslot contains
any particular key (thanks Eric!)
- Fscrypt support for inline encryption now handles filesystems with
multiple underlying block devices (thanks Eric!)
- Numerous cleanups
Bug: 137270441
Test: refer to I26376479ee38259b8c35732cb3a1d7e15f9b05a3
Change-Id: I13e2e327e0b4784b394cb1e7cf32a04856d95f01
Link: https://lore.kernel.org/linux-block/20191218145136.172774-1-satyat@google.com/
Signed-off-by: Satya Tangirala <satyat@google.com>
Inline Encryption hardware allows software to specify an encryption context
(an encryption key, crypto algorithm, data unit num, data unit size, etc.)
along with a data transfer request to a storage device, and the inline
encryption hardware will use that context to en/decrypt the data. The
inline encryption hardware is part of the storage device, and it
conceptually sits on the data path between system memory and the storage
device.
Inline Encryption hardware implementations often function around the
concept of "keyslots". These implementations often have a limited number
of "keyslots", each of which can hold an encryption context (we say that
an encryption context can be "programmed" into a keyslot). Requests made
to the storage device may have a keyslot associated with them, and the
inline encryption hardware will en/decrypt the data in the requests using
the encryption context programmed into that associated keyslot. As
keyslots are limited, and programming keys may be expensive in many
implementations, and multiple requests may use exactly the same encryption
contexts, we introduce a Keyslot Manager to efficiently manage keyslots.
The keyslot manager also functions as the interface that upper layers will
use to program keys into inline encryption hardware. For more information
on the Keyslot Manager, refer to documentation found in
block/keyslot-manager.c and linux/keyslot-manager.h.
Bug: 137270441
Test: tested as series; see I26aac0ac7845a9064f28bb1421eb2522828a6dec
Change-Id: I9a2dc72d61d5a3c64af379a97dd46155b41193eb
Signed-off-by: Satya Tangirala <satyat@google.com>
Link: https://patchwork.kernel.org/patch/11214713/
