LU-15420 sec: handle simple fscrypt changes for 5.15 kernels The patch covers the low impact changes to the Linux kernels fscrypt API. The changes are: For Linux kernel version 5.9-rc4 the following commits are: 8b10fe68985278de4926daa56ad6af701839e40a removed the inode parameter for the fscrypt function fscrypt_fname_alloc_buffer() 5b2a828b98ec1872799b1b4d82113c76a12d594f ended up exporting fscrypt_d_revalidate() and stopped stomping on the d_ops. c8c868abc91ff23f6f5c4444c419de7c277d77e1 changed fscrypt_set_test_dummy_encryption() take a 'const char * ac4acb1f4b2b6b7e8d913537cccec8789903e164 moved the fscrypt core from using fscrypt_context to using fscrypt_policy that is user forward facing. Lastly for Linux kernel version 5.10-rc4 the commit ec0caa974cd092549ab282deb8ec7ea73b36eba0 stopped exporting fscrypt_get_encryption_info(). Use fscrypt_prepare_readdir() in its place. 70fb2612aab62d47e03f82eaa7384a8d30ca175d renamed a field in struct fscrypt_name. Since Lustre can't use fscrypt_prepare_lookup we have to deal with this change. Remove sptlrpc_enc_pool_del_user() since its an empty function that waste cycles calling it. The other large change was the replacement of fscrypt_inherit_context which is described in Linux commit a992b20cd4ee360dbbe6f69339cb07146e4304d6. This change is very large since it expects the target inode to be available. Lustre uses fscrypt_inherit_context before the inode is available so this is a much more complex change that will be done in another patch. de3cdc6e75179a2324c23400b21483a1372c95e1 makes fscrypt_require_key private. You need to test the key's presence with fscrypt_has_encryption_key() instead but that key needs to be setup first by the new function fscrypt_prepare_new_inode(). Since this is the case we wait to introduce this change. Change-Id: I4bed7fef6e3302c0258c0f1563f4e180258d7a5a Signed-off-by: James Simmons <jsimmons@infradead.org> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49125 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Sebastien Buisson <sbuisson@ddn.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15003 sec: use enc pool for bounce pages Take pages from the enc pool so that they can be used for encryption, instead of letting llcrypt allocate a bounce page for every call to the encryption primitives. Pages are taken from the enc pool a whole array at a time. This requires modifying the llcrypt API, so that new functions llcrypt_encrypt_page() and llcrypt_decrypt_page() are exported. These functions take a destination page parameter. Until this change is pushed in upstream fscrypt, this performance optimization is not available when Lustre is built and run against the in-kernel fscrypt lib. Using enc pool for bounce pages is a worthwhile performance win. Here are performance penalties incurred by encryption, without this patch, and with this patch: ||=====================|=====================|| || Performance penalty | Performance penalty || || without patch | with patch || ||==========================================|=====================|| || Bandwidth – write | 30%-35% | 5%-10% large IOs || || | | 15% small IOs || ||------------------------------------------|---------------------|| || Bandwidth – read | 20% | less than 10% || ||------------------------------------------|---------------------|| || Metadata | N/A | 5% || || creat,stat,remove | | || ||==========================================|=====================|| Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: James Simmons <jsimmons@infradead.org> Change-Id: I3078d0a3349b3d24acc5e61ab53ac434b5f9d0e3 Reviewed-on: https://review.whamcloud.com/47149 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15922 sec: new connect flag for name encryption Introduce OBD_CONNECT2_ENCRYPT_NAME connection flag for compatibility with older versions that do not support name encryption. When server side does not have this flag, client side is forced to null encryption for file names. And client needs to use old xattr to store encryption context. Also update tests in sanity-sec to exercise name encryption only if server side supports it. Fixes: ed4a625d88 ("LU-13717 sec: filename encryption - digest support") Test-Parameters: serverversion=2.14 testlist=sanity-sec mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 fstype=ldiskfs Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I446a4caba8e45821d701628a14c96f03cb6c4525 Reviewed-on: https://review.whamcloud.com/47574 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13783 sec: support of native Ubuntu 20.04 HWE 5.8 kernel For Linux 5.5 kernel a patch to improve nokey names was landed that removed several variables that Lustre's llite and mdt layer were using. Rework the code to use other defines that exist. Second change for Ubuntu is several backports to handle a few variables changing across different kernel versions. One is the name change of DCACHE_ENCRYPTED_NAME and the other being is_chipertext_name. So the simpler approach Ubuntu took was to use fscrypt_prepare_lookup() and fscrypt_is_nokey_name() to work around these changes. Lastly the good news is for 5.12 the stomping of ll_d_ops no longer happens and the special revalidate_dentry fscrypto function is exported. Change-Id: I7f70fe9abddf34798e2e01b35099c9a032d92b91 Signed-off-by: James Simmons <jsimmons@infradead.org> Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Reviewed-on: https://review.whamcloud.com/46238 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Jian Yu <yujian@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15406 sec: fix in-kernel fscrypt support When using in-kernel fscrypt provided by Linux 5.4, the encryption context can be retrieved by calling the .get_context function defined in the struct fscrypt_operations of the super_block. llite needs to retrieve the encryption context explicitly in case of migration via volatile files. Fixes: 09c558d16f ("LU-14677 sec: migrate/extend/split on encrypted file") Fixes: fdbf2ffd41 ("LU-14677 sec: no encryption key migrate/extend/resync/split") Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Signed-off-by: James Simmons <jsimmons@infradead.org> Change-Id: I76dbd21f0dc95920519ea375c583bc378d7c9f53 Reviewed-on: https://review.whamcloud.com/45987 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13717 sec: missing defs in includes for client encryption Add a few missing definitions in lustre_crypto.h that are required in case Lustre client encryption is built against the in-kernel fscrypt library. Fixes: 028281ae19 ("LU-13717 sec: rework includes for client encryption") Test-Parameters: trivial Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I1965503554dcf660770d201444cfafd54aa84dce Reviewed-on: https://review.whamcloud.com/45221 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13717 sec: handle null algo for filename encryption Encrypted files created with Lustre 2.14 have clear text file names. With new code implementing filename encryption, newly created files will have cipher text names, unless they are in an encrypted directory created in Lustre 2.14. So we need to make sure llcrypt library can properly handle the "null" algorithm for client side filename encryption, which is basically a no-op. Handling this "null" algo for filename encryption will not be possible with the in-kernel fscrypt library, so modify the behaviour of configure to build with embedded llcrypt by default, and only build against in-kernel fscrypt if explicitly specified via --enable-crypto=in-kernel configure option. The objective is to urge users to convert their encrypted directories to the new fashion that encrypts filenames. However, with the new code some operations on encrypted files created with 2.14 might not be possible, like migrate, so expressly forbid migrate on files that use the "null" algorithm for client side filename encryption. Finally, we revert commit 11fcbfa9de4a5170abc2c5df2a6e4e02f0f84268 ("LU-12275 sec: force file name encryption policy to null") so that new encrypted directories will enforce filename encryption. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I393945adc9b720a56544b5da0669cb2848507457 Reviewed-on: https://review.whamcloud.com/43388 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-14677 sec: migrate/extend/split on encrypted file lfs migrate/extend/split makes use of volatile files to swap layouts. When operation is carried out on an encrypted file, the volatile file must be assigned the same encryption context as the original file, so that data moved/copied to different OSTs is identical to the original file's. Also update sanity-sec test_52 to exercise these commands. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I3878b5e9e6d3738dfee0ce0f89a3646e6a7b976f Reviewed-on: https://review.whamcloud.com/43878 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Bobi Jam <bobijam@hotmail.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13717 sec: rework includes for client encryption Simplify includes for crypto, by not repeating stubs in case HAVE_LUSTRE_CRYPTO is not defined. Expose encoding routines that are going to be used in the Lustre code (both client and server sides) with filename encryption. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I6c5853d6da7120edd2bec3a12494251d873151a8 Reviewed-on: https://review.whamcloud.com/43386 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-14306 sec: get rid of bad rss-counter state messages When doing O_DIRECT IOs on encrypted files, messages about bad rss-counter state can be seen in the console. The mm get confused because we twist the Lustre pages used for RPCs so that they are suitable for llcrypt API. In order to do this properly, the original mapping on these pages must be preserved outside of the encryption/decryption needs. Fixes: 728036f256 ("LU-12275 sec: O_DIRECT for encrypted file") Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I80ebcd3f96c51a3d158d7ef66f23b8da13904c52 Reviewed-on: https://review.whamcloud.com/41199 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: John L. Hammond <jhammond@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12275 sec: atomicity of encryption context getting/setting Encryption layer needs to set an encryption context on files and dirs that are encrypted. This context is stored as an extended attribute, that then needs to be fetched upon metadata ops like lookup, getattr, open, truncate, and layout. With this patch we send encryption context to the MDT along with create RPCs. This closes the insecure window between creation and setting of the encryption context, and saves a setxattr request. This patch also introduces a way to have the MDT return encryption context upon granted lock reply, making the encryption context retrieval atomic, and sparing the client an additional getxattr request. Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49" clientdistro=el8.1 fstype=ldiskfs mdscount=2 mdtcount=4 Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49" clientdistro=el8.1 fstype=zfs mdscount=2 mdtcount=4 Test-Parameters: clientversion=2.12 env=SANITY_EXCEPT="27M 56ra 151 156 802" Test-Parameters: serverversion=2.12 env=SANITY_EXCEPT="56oc 56od 165a 165b 165d 205b" Test-Parameters: serverversion=2.12 clientdistro=el8.1 env=SANITYN_EXCEPT=106,SANITY_EXCEPT="56oc 56od 165a 165b 165d 205b" Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I45599cdff13d5587103aff6edd699abcda6cb8f4 Reviewed-on: https://review.whamcloud.com/38430 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12275 sec: enable client side encryption Enable client side encryption. By default it is activated, letting user specifies actual encryption policy to use on a per-directory basis. It is possible to deactivate client side encryption by using the 'noencrypt' mount option. Also add the test dummy encryption mode option to ease testing. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I0e8d4db7ab8a77aba0600788cca9403f7c50f8a6 Reviewed-on: https://review.whamcloud.com/36143 Reviewed-by: John L. Hammond <jhammond@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>