LU-6142 lustre: Fix style issues for lustre_export.h This patch fixes issues reported by checkpatch for file lustre/include/lustre_export.h Test-Parameters: trivial Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Change-Id: I8a55aaad0702773ad83f4d7f7798d5509c086ba8 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53916 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Timothy Day <timday@amazon.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16314 llite: Migrate LASSERTF %p to %px This change covers lustre/ec through lustre/mgs and converts LASSERTF statements to explicitly use %px. Use %px to explicitly report the non-hashed pointer value messages printed when a kernel panic is imminent. When analyzing a crash dump the associated kernel address can be used to determine the system state that lead to the system crash. As crash dumps can and are provided by customers from production systems the use of the kernel command line parameter: no_hash_pointers is not always possible. Ref: Documentation/core-api/printk-formats.rst Test-Parameters: trivial HPE-bug-id: LUS-10945 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I708d9ef60c63f5b4006c7986599a2f39fc9e5fdf Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51213 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13805 obd: Reserve unaligned DIO connect flag Unaligned DIO generally requires only client changes, but an assert must be removed from ZFS servers for it to work correctly. This means we need a connect flag to recognize whether or not a server running ZFS can safely use unaligned DIO. All OSTs will present this flag - to keep things simple - but if the flag is not present, we'll still do unaligned DIO to ldiskfs OSTs. Actual implementation will be in another patch, this one just creates the flag itself. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I8b149cc54f4fb11e64182c65f2fbb01f8a3d3868 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51075 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Sebastien Buisson <sbuisson@ddn.com> Reviewed-by: Qian Yingjin <qian@ddn.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15671 mds: do not send OST_CREATE transno interop Send OST_CREATE RPCs from the MDS with no_resend and no_delay when communicating with an old OST that does not support the OBD_CONNECT2_REPLAY_RESEND. Likewise, the OST should not reply to the MDS RPC with rq_transno set, or this will trigger: osp_precreate_send() ASSERTION(req->rq_transno == 0) failed This can be avoided if the MDS is upgraded before the OSS, but will always be hit if OSS is upgraded first. After 2.20.53 the MDS/OSS assume that this is always true, since rolling upgrades are unsupported for larger version differences. Test-Parameters: testgroup=rolling-upgrade-oss Fixes: 63e17799a3 ("LU-8367 osp: enable replay for precreation request") Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com> Change-Id: I1ab601a2f55540dd75cf24838f7cdb7f823ed42c Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51056 Tested-by: Maloo <maloo@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16494 fileset: check fileset for operations by fid Some operations by FID, such as lfs rmfid, must be aware of subdirectory mount (fileset) so that they do not operate on files that are outside of the namespace currently mounted by the client. For lfs rmfid, we first proceed to a fid2path resolution. As fid2path is already fileset aware, it fails if a file or a link to a file is outside of the subdirectory mount. So we carry on with rmfid only for FIDs for which the file and all links do appear under the current fileset. This new behavior is enabled as soon as we detect a subdirectory mount is done (either directly or imposed by a nodemap fileset). This means the new behavior does not impact normal, whole-namespace client mount. sanity test_421h is added to exercise this new capability. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I47136ac0a3324b9afdd01b0f902abc37938bd361 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49696 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: jsimmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16205 sec: fid2path for encrypted files Add support of fid2path for encrypted files. Server side returns raw encrypted path name to client, which needs to process the returned string. This is done from top to bottom, by iteratively decrypting parent name and then doing a lookup on it, so that child can in turn be decrypted. For encrypted files that do not have their names encrypted, lookups can be skipped. Indeed, name decryption is a no-op in this case, which means it is not necessary to fetch the encryption key associated with the parent inode. Without the encryption key, lookups are skipped for the same reason. But names have to be encoded and/or digested. So server needs to insert FIDs of individual path components in the returned string. These FIDs are interpreted by the client to build encoded/digested names. Add sanity-sec test_63 to exercise this new capability. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I165bf2e5657037ae2e25c9378e4713537ea94bec Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48930 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: jsimmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15748 ofd: fix fallocate interop for older clients The logic for detecting older client fallocate was backward, and should be checking if the OBD_CONNECT_OLD_FALLOC patch was *not* present to detect old clients. Since the new server does not have OLD_FALLOC in the SUPPORTED flags, it will clear the flag from the export at connect time, so it needs to be saved in the export early on. Test-Parameters: testlist=sanityn env=ONLY=16,HONOR_EXCEPT=y Test-Parameters: serverversion=2.14 testlist=sanityn env=ONLY=16,HONOR_EXCEPT=y Fixes: 7905359296 ("LU-15748 osc: fallocate interop for 2.14 clients") Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: I327183025a8de6fd814a7c2929365497153ebbe5 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47548 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: Jian Yu <yujian@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 lnet: use large nids in struct lnet_event All nids, including those in process_id, are changed to to struct lnet_nid / struct lnet_processid. Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I799dbbc22f7cfe403f07eb22f4bfc4e4b5dc23ea Reviewed-on: https://review.whamcloud.com/43600 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-14487 lustre: remove references to Sun Trademark. "lustre" is no longer a Trademark of Sun Microsystems. There is no need to acknowledge the trademark is every file, so just remove all these claims. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I214670b39c5718f2b691193f268a64856e0cd743 Reviewed-on: https://review.whamcloud.com/41880 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org>
LU-10664 dom: non-blocking enqueue for DOM locks DOM lock enqueue waits for blocking locks on MDT due to ATOMIC flag, so MDT thread is blocked until lock is granted. When many clients attempt to write to shared file that may cause server thread starvation and lock contention. Switch to non-atomic lock enqueue for DOM locks. - switch IO lock to non-intent enqueue, so it doesn't consume server thread for a long time being blocked - on client take LVB from l_lvb_data updated by completion AST and update l_ost_lvb used by DoM - make glimpse performing similarly on MDT and OST, it uses one format with no intent buffer and return data in LVB buffer - introduce new connect flag 'dom_lvb' for compatibility reasons - on server handle glimpse for both old and new clients by filling either LVB reply buffer or mdt_body buffer - don't take RPC slot for a DOM enqueue like it is done for EXTENT locks, update ldlm_cli_enqueue_fini() to accept ldlm_enqueue_info as parameter - check that there is no atomic local lock issued with mandatory DOM bit, trybits should be used Test-Parameters: mdssizegb=20 testlist=dom-performance Test-Parameters: testlist=sanity-dom clientversion=2.12 Test-Parameters: testlist=sanity-dom serverversion=2.12 Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: I06573018126f0663a8f76dce07e126a1879cfba5 Reviewed-on: https://review.whamcloud.com/36903 Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10810 clio: SEEK_HOLE/SEEK_DATA on client side Patch introduces basic support for lseek SEEK_HOLE/SEEK_DATA parameters in lustre client. - introduce new IO type CIT_LSEEK in CLIO stack - LOV splits request to all stripes involved and merges results back. - OSC sends OST LSEEK RPC asynchronously - if target doesn't support LSEEK RPC then OSC assumes whole related object is data with virtual hole at the end - lseek restores released files assuming it is done prior the file copying. - tool is added to request needed lseek on file - basic tests are added in sanity, sanityn and sanity-hsm Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com> Change-Id: I0728329d4bce71c441de581a439cde1aa873fd46 Reviewed-on: https://review.whamcloud.com/39708 Reviewed-by: Oleg Drokin <green@whamcloud.com> Tested-by: Oleg Drokin <green@whamcloud.com>
LU-12275 sec: atomicity of encryption context getting/setting Encryption layer needs to set an encryption context on files and dirs that are encrypted. This context is stored as an extended attribute, that then needs to be fetched upon metadata ops like lookup, getattr, open, truncate, and layout. With this patch we send encryption context to the MDT along with create RPCs. This closes the insecure window between creation and setting of the encryption context, and saves a setxattr request. This patch also introduces a way to have the MDT return encryption context upon granted lock reply, making the encryption context retrieval atomic, and sparing the client an additional getxattr request. Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49" clientdistro=el8.1 fstype=ldiskfs mdscount=2 mdtcount=4 Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49" clientdistro=el8.1 fstype=zfs mdscount=2 mdtcount=4 Test-Parameters: clientversion=2.12 env=SANITY_EXCEPT="27M 56ra 151 156 802" Test-Parameters: serverversion=2.12 env=SANITY_EXCEPT="56oc 56od 165a 165b 165d 205b" Test-Parameters: serverversion=2.12 clientdistro=el8.1 env=SANITYN_EXCEPT=106,SANITY_EXCEPT="56oc 56od 165a 165b 165d 205b" Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I45599cdff13d5587103aff6edd699abcda6cb8f4 Reviewed-on: https://review.whamcloud.com/38430 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-8130 obd: convert obd_nid_hash to rhashtable Linux has a resizeable hashtable implementation in lib, so we should use that instead of having one in libcfs. This patch converts the struct obd_export obd_nid_hash to use rhashtable. In the process we gain lockless lookup which should improve performance. For the nid hash we use rhltable since the mapping can be many exports to a NID key. Signed-off-by: James Simmons <jsimmons@infradead.org> Change-Id: I45154ceb48336b20161f771d986d8fe7333b9849 Reviewed-on: https://review.whamcloud.com/33518 Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-6179 llite: remove LOCKAHEAD_OLD compatibility The OBD_CONNECT_LOCKAHEAD_OLD feature was added for a short time for compatibility with an implementation of LDLM lockahead that was later replaced by OBD_CONNECT_LOCKAHEAD2. Remove the compatibility code for this old implementation that has been disabled since the 2.13 release. Also remove other obsolete compatibility code dating back to 2.8.53. Test-Parameters: trivial Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: I4444eff180b2c6e2b27d260413f2debbb2ce7057 Reviewed-on: https://review.whamcloud.com/38109 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org>
LU-12722 target: disable recovery for local clients when client is running on a server node, then the local services can't rely on that client in the contex of recovery - such a client dies with the node, can't replay requests and states and then the restarting server has to wait till recovery expires which doesn't make any sense. so the servers should recogize local clients and exclude them from recovery (i.e. don't make them part of last_rcvd). for the purpose of local testing a special mount option "local_recov" has been added to {MDS|OST}_MOUNT_OPTS in tests/cfg/local.sh to save local testing when everyting is running within a single node. Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com> Change-Id: I4cb906c44c1192933f7d77dc782160e426e9efde Reviewed-on: https://review.whamcloud.com/36025 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Li Xi <lixi@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-8130 obd: convert obd uuid hash to rhashtable The rhashtable data type is a perfect fit for the export uuid hash table, so use that instead of cfs_hash (which will eventually be removed). As rhashtable supports lookups and insertions in atomic context, there is no need to drop a spinlock while inserting a new entry, which simplifies code quite a bit. Linux-commit: 4206c444e4a89dae9f67f08d9c29d58c37c960cd Change-Id: Icadf64d572982409008a1ef4d23eb0fe1e3c8cd0 Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-on: https://review.whamcloud.com/34429 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Shaun Tancheff <stancheff@cray.com> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12542 handle: move refcount into the lustre_handle. Most objects with a lustre_handle have a refcount. The exception is mdt_mfd which uses locking (med_open_lock) to manage its lifetime. The lustre_handles code currently needs a call-out to increment its refcount. To simplify things, move the refcount into the lustre_hanle (which will be largely ignored by mdt_mfd) and discard the call-out. To avoid warnings when refcount debugging is enabled the refcount of mdt_mfd is initialized to 1, and decremeneted after any class_handle2object() call which would have incremented it. In order to preserve the same debug messages, we store an object type name in the portals_handle_ops, and use that in a CDEBUG() when incrementing the ref count. Change-Id: I1920330b2aeffd4b865cb9b249997aa28b209c33 Signed-off-by: NeilBrown <neilb@suse.com> Reviewed-on: https://review.whamcloud.com/35794 Reviewed-by: Neil Brown <neilb@suse.de> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Shaun Tancheff <stancheff@cray.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11444 ptlrpc: resend may corrupt the data Late resend if arrives much later than another modification RPC which has been already handled on this slot, may be still applied and therefore overrides the last one Send RPCs from client in increasing order for each tag and check it on server to check late resend. A slot can be reused by a client after kill while the server continue to rely on it. Add flag for such obsolete requests, here we trust the client and perform xid check for all in progress requests. Cray-bug-id: LUS-6272, LUS-7277, LUS-7339 Signed-off-by: Andriy Skulysh <c17819@cray.com> Reviewed-by: Vitaly Fertman <c17818@cray.com> Reviewed-by: Alexander Boyko <c17825@cray.com> Reviewed-by: Andrew Perepechko <c17827@cray.com> Change-Id: I97806577cf979f49a75379ffc55947cc3dcd02b1 Reviewed-on: https://review.whamcloud.com/35114 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Alexandr Boyko <c17825@cray.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-9846 obd: Add overstriping CONNECT flag This patch reserves the OBD_CONNECT flag for overstriping, and also does some cleanup of OBD_CONNECT flags, putting them in the correct order and adding some missing ones in proc and the wire{test,check} checks. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I5d7c8f30d16cc2541d3202582fe55177022ccede Reviewed-on: https://review.whamcloud.com/34743 Tested-by: Jenkins Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
LU-8955 ptlrpc: manage SELinux policy info at connect time At connect time, compute SELinux policy info on client side, and send it over the wire. On server side, get SELinux policy info from nodemap and compare it with the one received from client. Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I9b4a206455f2c0b451f6b3ed7e3a85285592758e Reviewed-on: https://review.whamcloud.com/24422 Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Li Dongyang <dongyangli@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Tested-by: Jenkins Tested-by: Maloo <maloo@whamcloud.com>