LU-17317 sec: add srpc_serverctx proc file GSS srpc contexts for client connections can already be dumped via proc file <mdc,osc>.*.srpc_contexts. This patch adds a new proc file to dump server side GSS srpc contexts, e.g.: mgs.MGS.gss.srpc_serverctx mdt.testfs-MDT0000.gss.srpc_serverctx obdfilter.testfs-OST0000.gss.srpc_serverctx The GSS context information is dumped as YAML, with one line per context, like this: - 0000000013221bdf: { peer_nid: 192.168.56.206@tcp, uid: 0, ctxref: 1, expire: 1707934985, delta: 3401, flags: [uptodate, cached], seq: 0, win: 2048, key: 00000000, keyref: 0, hdl: "0x5ae1a771fd57043:0x65a64972fda4e200", mech: "krb5 (aes256-cts-hmac-sha1-96)" } Because of this new syntax, sanity-sec test_28 needs to be fixed. Test-Parameters: trivial Test-Parameters: kerberos=true testlist=sanity-krb5 Test-Parameters: testgroup=review-dne-selinux-ssk-part-2 Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: I37da9ffe6dd5884006b36271185a4d7155ead65b Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53376 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Aurelien Degremont <adegremont@nvidia.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17297 grant: move tgt_grant_sanity_check() calls Call tgt_grant_sanity_check() in ofd_obd_disconnect() and in mdt_obd_disconnect() after call to tgt_grant_discard(). Otherwise, sum of grants does not match to total grant counter which is reported as LustreError: ofd_obd_disconnect: tot_granted 0 != fo_tot_granted 8388608 This is because on stale export eviction class_disconnect_stale_exports() moves stale exports to separate list but does not update obd's grant counters. Test to illustrate the issue is included. HPE-bug-id: LUS-11469 Signed-off-by: Vladimir Saveliev <vladimir.saveliev@hpe.com> Change-Id: I0b4568b88a2fe7b50f4eac50b4b064d7afbc7a75 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53171 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-13307 nodemap: have nodemap_add_member support large NIDs Currently when mounting lustre using IPv6 address fails with Lustre: 27361:0:(nodemap_handler.c:395:nodemap_add_member()) lustre-MDT0000: error adding to nodemap, no valid NIDs found LustreError: 11-0: lustre-MDT0000-osp-MDT0003: operation mds_connect to node 0@lo failed: rc = -22 This was due to no nodemap being set so the ptlrpc layer was not seeing any new peers. Adding minimal support to nodemap allows mounting. Change-Id: If9cfe88ec92afc3f14788f3f3ded8387a1b5d8c7 Signed-off-by: James Simmons <jsimmons@infradead.org> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53135 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Sebastien Buisson <sbuisson@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17306 ofd: return error for reconnection During the cleanup orphan phase, reconnection leads to unsynchronized last id between MDT and OST. This means that MDT could assign non existing objects to a client for a file create operation. ofd_create_hdl()) capstor-OST0087: dropping old orphan cleanup request MDS LAST_ID [0x2540000400:0xb6941:0x0] (747841) is 352 behind OST LAST_ID [0x2540000400:0xb6aa1:0x0] (748193), trust the OST recovery-small 144c reproduce bug where MDT lost synchronization with OST. Fixes: 63e17799a3 ("LU-8367 osp: enable replay for precreation request") HPE-bug-id: LUS-11969 Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com> Change-Id: I22c3d3b3db2acc9ad8f1b978b234afe7d3eef51d Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53195 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17216 ofd: make enable_health_write tunable enable_health_write should be tunable rather than a compilation option. This allows us to test it more easily and gives admins the option to try it out without having to recompile their Lustre servers. It will still be disabled by default. Add sanity/70 test to run a simple check to ensure enable_health_write and health_check don't explode. It's not a thorough check. But it at least checks that the interfaces appear to work. Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I7b1832f8acf578b891386e28c5af760070a6862c Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52782 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-9859 libcfs: refactor libcfs initialization. Many lustre modules depend on libcfs having initialized properly, but do not explicit check that it did. When lustre is built as discrete modules, this does not cause a problem because if the libcfs module fails initialization, the other modules don't even get loaded. When lustre is compiled into the kernel, all module_init() routines get run, so they need to check the required initialization succeeded. This patch splits out the initialization of libcfs into a new libcfs_setup(), and has all modules call that. The misc_register() call is kept separate as it does not allocate any resources and if it fails, it fails hard - no point in retrying. Other set-up allocates resources and so is best delayed until they are needed, and can be worth retrying. Ideally, the initialization would happen at mount time (or similar) rather than at load time. Doing this requires each module to check dependencies when they are activated rather than when they are loaded. Achieving that is a much larger job that would have to progress in stages. For now, this change ensures that if some initialization in libcfs fails, other modules will fail-safe. Linux-commit: 64bf0b1a079d61e9e059b9dc7a58e064c7d994ae Change-Id: I6b5ecdba0defc6e033f78d8fc2b9be9e26c7f720 Signed-off-by: Mr. NeilBrown <neilb@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52700 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Timothy Day <timday@amazon.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16802 build: compatibility for 6.4 kernels linux kernel v6.3-rc4-32-g6eb203e1a868 iov_iter: remove iov_iter_iovec() Provide a replacement iov_iter_iovec() when one is not provided. linux kernel v6.3-rc4-34-g747b1f65d39a iov_iter: overlay struct iovec and ubuf/len This renames iov_iter member iov to __iov and provides the iov_iter() accessor. Define __iov as iov when __iov not present. Provide an iov_iter() for older kernels. linux kernel v6.3-rc1-13-g1aaba11da9aa driver core: class: remove module * from class_create() Provide an ll_class_create() to pass THIS_MODULE, or not, as needed by class_create(). Linux commit v6.2-rc1-20-gf861646a6562 quota: port to mnt_idmap Update osd_dquot_transfer to use mnt_idmap and fallback to user_ns, if needed, by dquot_transfer. Linux commit v6.3-rc7-2433-gcf64b9bce950 SUNRPC: return proper error from get_expiry() Updated get_expiry() requires a time64_t pointer to be passed to hold the expiry time. A non-zero return value indicates an error, nominally -EINVAL. Provide a wrapper for kernels that return a time64_t and return -EINVAL on error. Test-Parameters: trivial HPE-bug-id: LUS-11614 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I765d6257eec8b5a9bf1bd5947f03370eb9df1625 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50875 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com> Reviewed-by: xinliang <xinliang.liu@linaro.org>
LU-12998 mds: add no_create parameter to stop creates Add an target tunable parameter and mount option "no_create" to disable new *directory* creation on an MDT. This sends the flag OS_STATFS_NOCREATE to the clients, and the DNE MDT space balance will avoid selecting that MDT when creating a new subdirectory, without disabling access to existing files/dirs. This allows "soft disabling" an MDT in advance of storage upgrades to minimize new directories and files created on that MDT, reduce future migration, and/or backup/restore workload. As yet it does not totally disable *file* creation on the MDT, but it may be extended to do so in the future. This is analogous to the "no_precreate" option that was added on the OSTs, and "no_create" has been added to the OSTs for consistency ("no_precreate" is kept for compatibility for now). Test-Parameters: testlist=conf-sanity env=ONLY=112b,ONLY_REPEAT=50 Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com> Change-Id: I53cfb48ade2f844b18bfc630e7fcea6de9ce7057 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47124 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16796 ofd: Change struct ofd_seq to use refcount_t This patch changes struct ofd_seq to use refcount_t instead of atomic_t Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Change-Id: Ie149a6812671ea872e17d2881e52cf6096d147ff Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52722 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Timothy Day <timday@amazon.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15246 ptlrpc: per-device adaptive timeout parameters When a client is mounting multiple filesystems with different MGSes setting global parameters at_min, at_max, etc., then the settings from one filesystem's MGS config will also apply to RPCs sent for the OSC, MDC, and MGC devices on the other filesystem(s). Typically the settings of the last filesystem to mount on the client override the earlier values, and there is no way to separate them. Moving the parameters to be per-device values allows them to be set independently for each set of client devices, so that the client can interact properly with each set of servers. This allows e.g. different timeouts for local and remote mounts, or for flash and HDD filesystems that have different load and performance. Add per-device adaptive timeout parameters that can optionally replace global parameters of the same name: at_min -> *.<fsname>*.at_min at_max -> *.<fsname>*.at_max at_history -> *.<fsname>*.at_history ldlm_enqeue_min -> *.<fsname>*.ldlm_enqueue_min These parameters should always be set with fsname in the device name, rather than pure wildcard '*' device names, or it will be be same as the global parameters in the end (settings from one MGS will apply to devices on other filesystems). That is a bug in how "lctl set_param -P" works, but will be fixed separately. Signed-off-by: Lei Feng <flei@whamcloud.com> Change-Id: I5b04c9aa53a446fb5a78bfaff372b4f236c9eb8a Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45598 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-6142 lustre: use list_first/last_entry() for list heads This patch changes list_entry(foo.next, ...) to list_first_entry(&foo, ...) and list_entry(foo.prev, ...) to list_last_entry(&foo, ...) in cases where 'foo' is a list head - not a list member. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I22b1278f5b481ce3074db3e59d37d9148016aed5 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50828 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-8191 lustre: convert osp,osd,osc,ofd functions to static Static analysis shows that a number of functions could be made static. This patch declares several functions in osp, osd, osc, and ofd static. Also, fix a few minor style issues. Test-Parameters: trivial Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I3d7af7ec0fa2978bfdd0cb490f18f485a78f81f6 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51477 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: jsimmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15671 mds: do not send OST_CREATE transno interop Send OST_CREATE RPCs from the MDS with no_resend and no_delay when communicating with an old OST that does not support the OBD_CONNECT2_REPLAY_RESEND. Likewise, the OST should not reply to the MDS RPC with rq_transno set, or this will trigger: osp_precreate_send() ASSERTION(req->rq_transno == 0) failed This can be avoided if the MDS is upgraded before the OSS, but will always be hit if OSS is upgraded first. After 2.20.53 the MDS/OSS assume that this is always true, since rolling upgrades are unsupported for larger version differences. Test-Parameters: testgroup=rolling-upgrade-oss Fixes: 63e17799a3 ("LU-8367 osp: enable replay for precreation request") Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com> Change-Id: I1ab601a2f55540dd75cf24838f7cdb7f823ed42c Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51056 Tested-by: Maloo <maloo@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12610 misc: remove OBD_ -> CFS_ macros Remove OBD macros that are simply redefinitions of CFS macros. Test-Parameters: trivial Signed-off-by: Timothy Day <timday@amazon.com> Signed-off-by: Ben Evans <beevans@whamcloud.com> Change-Id: I15fe8aa22cb0203bed102a35361f4854ddaabecb Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50809 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Oleg Drokin <green@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-6142 obd: change lmd flags to bitmap Change lmd flags to an enum that is accessible with the Linux bitmap API. This lays the foundation for creating a match table for the server options for mounting. Change-Id: If7906a9a3ba177b67d0cfbaa276a00a6ba9b7b6d Signed-off-by: James Simmons <jsimmons@infradead.org> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49912 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16634 misc: standardize iocontrol param handling Validate uarg and karg early in iocontrol processing where needed. This needs kernel interop for 4.20+ kernels for access_ok(), but this can be checked by #ifdef and does not need an autoconf test. Fix incorrect definition of OBD_IOC_BARRIER to match reality. Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Reported-by: Tao Lyu <tao.lyu@epfl.ch> Change-Id: I1a0d2f839949debf346aa15c65b0f407e9ce7057 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50314 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 lustre: obd_connect and reconnect now use large nid The 'localdata' argument for o_connect and o_reconnect when called server-side is now a 'struct lnet_nid *' rather than an 'lnet_nid_t *'. Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I1ce72ec11a5d2463fb90ab2686410e2dd96118e2 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50097 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11912 tests: SEQ rollover fixes To avoid changeing SEQ after replay_barrier, we use force_new_seq when starting the test suites heavily using replay_barrier, e.g. replay-single. However when there are fewer OSTs, the default 16384 SEQ width could not last the entire test suite, SEQ rollover could still happen randomly after replay_barrier. To overcome this, change the default OSTSEQWIDTH to 65536, and divide by number of OSTs, so the SEQ width is larger with fewer OSTs. For 8 OSTs, the SEQ width is 16384, and make sure we don't go under it. Use force_new_seq_all for the test suites using replay_barrier on MDTs other than mds1. Add force_new_seq_all for replay-ost-single, which is using replay_barrier on OST. If SEQ rollover happens after that, the SEQ range update on ofd is lost due to replay_barrier, the next time when we try to allocate a new SEQ will end up with an old one. Use force_new_seq_all for the test cases(namely sanity-pfl/0b 0c 1c 16b sanity/27Cd) checking for number of stripes created with overstriping, to make sure we have enough objects in the precreate pool. Test-Parameters: trivial ostcount=4 testlist=replay-single Test-Parameters: ostcount=2 testlist=replay-single Test-Parameters: mdtcount=2 testlist=conf-sanity env=ONLY=122a,ONLY_REPEAT=10 Test-Parameters: testlist=sanity,sanity-pfl Test-Parameters: testlist=sanity-scrub,replay-single,obdfilter-survey,replay-ost-single,large-scale Fixes: 0ecb2a167c ("LU-11912 ofd: reduce LUSTRE_DATA_SEQ_MAX_WIDTH") Signed-off-by: Li Dongyang <dongyangli@ddn.com> Change-Id: Ic65111199f042405d6db8acb729b2cddf91138af Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50478 Tested-by: Maloo <maloo@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-8367 osp: remove unused fail_locs from sanity/27S,822 OBD_FAIL_OST_GET_LAST_FID and OBD_FAIL_OSP_GET_LAST_FID are not used anymore since sanity test_27S has been removed. This may lead to failures in interoperability testing. For example, 27S caused osp_precreate_cleanup_orphans to stuck due to not cleared opd_pre_recovering flag(normally, it should be set to 0 before calling osp_precreate_reserve). It was the reason of failure tests 27U, 27R, 39r and 51d in sanity.sh. The patch also removes OBD_FAIL_NET_ERROR_RPC and OBD_FAIL_OSP_PRECREATE_PAUSE that were used in sanity-822 "test precreate failure". This test also has been removed in 63e17799a3. Fixes: 63e17799a3 (LU-8367 osp: enable replay for precreation request) Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com> Change-Id: Ia78ce39adb9d59e476eb36e5e69954cc26353b27 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50543 Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-16350 osd-ldiskfs: no_llseek removed, dquot_transfer Linux commit v5.19-rc2-6-g868941b14441 fs: remove no_llseek With the removal of no_llseek, leaving .llseek set to NULL is functionally equivalent. Only provide no_llseek if it exists. Linux commit v5.19-rc3-6-g71e7b535b890 quota: port quota helpers mount ids dquot_transfer adds a user namespace argument. Provide an osd_dquot_transfer() wrapper to discard the additional argument for older kernels. Test-Parameters: trivial HPE-bug-id: LUS-11376 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: If3165aed0d7b827b90e26d9f0174137d087ce57a Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49266 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Yang Sheng <ys@whamcloud.com> Reviewed-by: xinliang <xinliang.liu@linaro.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>