Whamcloud - gitweb
James Simmons [Wed, 8 Feb 2023 13:04:45 +0000 (08:04 -0500)]
LU-15420 sec: handle simple fscrypt changes for 5.15 kernels
The patch covers the low impact changes to the Linux kernels
fscrypt API. The changes are:
For Linux kernel version 5.9-rc4 the following commits are:
8b10fe68985278de4926daa56ad6af701839e40a removed the inode
parameter for the fscrypt function fscrypt_fname_alloc_buffer()
5b2a828b98ec1872799b1b4d82113c76a12d594f ended up exporting
fscrypt_d_revalidate() and stopped stomping on the d_ops.
c8c868abc91ff23f6f5c4444c419de7c277d77e1 changed
fscrypt_set_test_dummy_encryption() take a 'const char *
ac4acb1f4b2b6b7e8d913537cccec8789903e164 moved the fscrypt core
from using fscrypt_context to using fscrypt_policy that is user
forward facing.
Lastly for Linux kernel version 5.10-rc4 the commit
ec0caa974cd092549ab282deb8ec7ea73b36eba0 stopped exporting
fscrypt_get_encryption_info(). Use fscrypt_prepare_readdir()
in its place.
70fb2612aab62d47e03f82eaa7384a8d30ca175d renamed a field in
struct fscrypt_name. Since Lustre can't use fscrypt_prepare_lookup
we have to deal with this change.
Remove sptlrpc_enc_pool_del_user() since its an empty function
that waste cycles calling it.
The other large change was the replacement of
fscrypt_inherit_context which is described in Linux commit
a992b20cd4ee360dbbe6f69339cb07146e4304d6. This change is very
large since it expects the target inode to be available. Lustre
uses fscrypt_inherit_context before the inode is available so
this is a much more complex change that will be done in another
patch.
de3cdc6e75179a2324c23400b21483a1372c95e1 makes fscrypt_require_key
private. You need to test the key's presence with
fscrypt_has_encryption_key() instead but that key needs to be
setup first by the new function fscrypt_prepare_new_inode().
Since this is the case we wait to introduce this change.
Change-Id: I4bed7fef6e3302c0258c0f1563f4e180258d7a5a
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49125
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sun, 12 Feb 2023 07:48:30 +0000 (23:48 -0800)]
LU-16101 tests: add sanity/27J to always_except
This patch adds sanity/27J to always_except for SLES15 SP4
and 5.16.0+ kernels before the issue introduced by upstream
commit
8c8387ee3f55
("mm: stop filemap_read() from grabbing a superfluous page")
is resolved.
Test-Parameters: trivial clientdistro=sles15sp4 testlist=sanity
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Iafde656530fcdc1de9265aacaa9266435c9d5c47
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49970
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Xing Huang <hxing@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Thu, 24 Nov 2022 21:51:58 +0000 (16:51 -0500)]
LU-14719 lod: ignore space check error in recovery
statfs may fail in recovery, ignore this error in
lod_trans_space_check().
Fix syntax error in replay-single 111g version check.
Fixes:
6aee406c84 ("LU-14719 lod: distributed transaction check space")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I6c7934ca242a639d996d0ab5a4d7648cec8a53de
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49249
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Etienne AUJAMES [Wed, 1 Dec 2021 23:22:33 +0000 (00:22 +0100)]
LU-15280 llog: fix processing of a wrapped catalog
Several issues were found with "lfs changelog --follow" for a wrapped
catalog (llog_cat_process() with startidx):
1/ incorrect lpcd_first_idx value for a wrapped catalog (startcat>0)
The first llog index to process is "lpcd_first_idx + 1". The startidx
represents the last record index processed for a llog plain. The
catalog index of this llog is startcat.
lpcd_first_idx of a catalog should be set to "startcat - 1"
e.g:
llog_cat_process(... startcat=10, startidx=101) means that the
processing will start with the llog plain at the index 10 of the
catalog. And the first record to process will be at index 102.
2/ startidx is not reset for an incorrect startcat index
startidx is relevant only for a startcat. So if the corresponding llog
plain is removed or if startcat is out of range, we need to reset
startidx.
This patch remove LLOG_CAT_FIRST, that was really confusing
(LU-14158). And update osp_sync_thread() with the
llog_cat_process() corrected behavior.
It modifies also llog_cat_retain_cb() to zap empty plain llog directly
in it (like for llog_cat_size_cb()), the current implementation is not
compatible with this patch.
The test "conf-sanity 135" verify "lfs changelog --follow" for a
wrapped changelog_catalog.
Test-Parameters: testlist=conf-sanity env=ONLY=135,ONLY_REPEAT=10
Test-Parameters: testlist=sanity env=ONLY=60a,ONLY_REPEAT=20
Test-Parameters: testlist=conf-sanity env=SLOW=yes,ONLY=106,ONLY_REPEAT=10
Fixes:
a4f049b9 ("LU-13102 llog: fix processing of a wrapped catalog")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Iaf46ddd4a6ec1e06cec0d17aa9bde766bd793abc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45708
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Li Dongyang [Tue, 23 Nov 2021 23:45:48 +0000 (10:45 +1100)]
LU-11912 fid: clean up OBIF_MAX_OID and IDIF_MAX_OID
Define the OBIF|IDIF_MAX_OID macros to 1ULL << OBIF|IDIF_MAX_BITS - 1
Clean up the callers and remove OBIF|IDIF_OID_MASK which are not used.
Test-Parameters: trivial
Change-Id: I9a679b930c73da5904b2eb4c74f785fc1d27a8a0
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45659
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 7 Feb 2023 09:29:24 +0000 (12:29 +0300)]
LU-16536 osp: don't cleanup ldlm in precleanup phase
instead do this in cleanup phase so that all OSPs have chance
to abort in-flight RPCs which can block MDT thread holding
LDLM locks.
Fixes:
226fd401f9 ("LU-7660 dne: support fs default stripe")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib3714b29c514a7fa938d47717dc36525654407d6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49925
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 19 Jan 2023 14:14:21 +0000 (17:14 +0300)]
LU-16493 tests: recovery-small/144b to wait longer
when ZFS is used as ZFS's primitives like osd_write()
and osd_declare_write() are still few times slower
compared to ldiskfs and this 144b creating wide-striped
files which causes lots of tiny writes to the last-used
file.
Test-Parameters: trivial testlist=recovery-small
Test-Parameters: testlist=recovery-small fstype=zfs env=ONLY=144b,ONLY_REPEAT=100
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib023c7c71a3bb486f9f9908e4cc03cc6e53ace7a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49693
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Patrick Farrell [Mon, 6 Feb 2023 23:22:24 +0000 (18:22 -0500)]
LU-16515 clio: Remove cl_page_size()
cl_page_size() is just a function which does:
1 << PAGE_SHIFT
and the kernel provides a macro for that - PAGE_SIZE.
Maybe it didn't when this function was added, but it sure
does now.
So, remove cl_page_size().
Test-parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6c27f6db7cfec5d9054aab95beccffe3c2da02bb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49918
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 12 Jan 2023 09:18:48 +0000 (10:18 +0100)]
LU-16532 sec: session key bad keyring
At initialization, the session key created for GSS context is linked
to a keyring from userpace, and then unlinked from kernelspace if it
is for root, as we want to share it across all root sessions.
Sometimes initialization fails (expired token, unresponsive server,
etc.) so the key cannot be unlinked. Survive this use case gracefully.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8ad2afa0e51e50640620e36211e5db1253d85e08
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49909
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 3 Feb 2023 08:17:16 +0000 (02:17 -0600)]
LU-16520 build: Move strscpy to libcfs common header
Ensure strscpy is available to lustre
Test-Parameters: trivial
Fixes:
0b406c91d17 ("LU-13642 lnet: modify lnet_inetdev to work with large NIDS")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0c3673c2aa7e6b61671521a8cabde8a364f7f6f8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49863
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 21 Jan 2023 01:45:05 +0000 (01:45 +0000)]
LU-16502 lutf: cleanup lutf_start.py, fix bugs
Remove code duplication by adding __check_env_var
method. Use os.environ.get to remove needlessly
verbose try-except blocks. Use __cfg_yaml more,
rather than passing this value explicitly
between methods.
Add check for environment variables, so LUTF
can fail gracefully if the environment is not
set correctly.
Update .gitignore to ignore __pycache__
Fix syntax error in python/infra/lutf.py
Test-Parameters: @lnet
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I16cf114ac4253d22a42399e2f2cb2fad49dd96cb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49767
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 21 Jan 2023 01:40:13 +0000 (01:40 +0000)]
LU-16502 lutf: fix bugs in bash scripts
Addressed some issues I found when running
LUTF. The "rm" fails without a file to remove.
A Lustre wiki led me to source the script, but this
will log out of the current shell. Adding a warning
against doing this.
Also, fix shellcheck errors and a few warnings in
LUTF related scripts. Added bash shebangs, since
shellcheck requires these to lint the scripts.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I501d58d25bfcd6564755485b9a1afa2277848b96
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49728
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Sebastien Buisson [Thu, 19 Jan 2023 17:07:27 +0000 (18:07 +0100)]
LU-16494 fileset: check fileset for operations by fid
Some operations by FID, such as lfs rmfid, must be aware of
subdirectory mount (fileset) so that they do not operate on files
that are outside of the namespace currently mounted by the client.
For lfs rmfid, we first proceed to a fid2path resolution. As fid2path
is already fileset aware, it fails if a file or a link to a file is
outside of the subdirectory mount. So we carry on with rmfid only
for FIDs for which the file and all links do appear under the
current fileset.
This new behavior is enabled as soon as we detect a subdirectory mount
is done (either directly or imposed by a nodemap fileset). This means
the new behavior does not impact normal, whole-namespace client mount.
sanity test_421h is added to exercise this new capability.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I47136ac0a3324b9afdd01b0f902abc37938bd361
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49696
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Akash B [Tue, 17 Jan 2023 15:51:19 +0000 (10:51 -0500)]
LU-16479 utils: Add option to manage degraded ZFS OST
Add new Lustre specific ZFS dataset user property to
control/manage degraded ZFS OSTs, also modify the existing
lustre/scripts/statechange-lustre.sh zedlet accordingly.
Extend the same to mkfs.lustre utility to add this property
by default when creating a new Lustre ZFS server.
HPE-bug-id: LUS-11447
Test-Parameters: trivial fstype=zfs testlist=sanity
Signed-off-by: Akash B <akash-b@hpe.com>
Change-Id: I7032538f507c9ad20d5b109b54e3c3bab8138458
Reviewed-by: Dipak Ghosh <dipak.ghosh@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49660
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Siddarth Raj <siddarth.raj@hpe.com>
Reviewed-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Andreas Dilger [Fri, 23 Dec 2022 00:28:14 +0000 (17:28 -0700)]
LU-16428 tests: cache is_project_quota_supported result
Rather than is_project_quota_supported() repeatedly checking if the
MDS supports project quota, check the MDS once after mount and then
cache it for the rest of the test run.
In resetquota() there is no need to check is_project_quota_supported()
each time, since this is only called with "-p" when project quota was
previously checked and is enabled. Also, there is no need to wait 1s
after every quota limit is reset, but rather only once at the end.
Test-Parameters: trivial testlist=sanity-quota mdtcount=4 mdscount=2
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id98240e2fcd2e862bb4305961a2946227e3ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49499
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Mon, 12 Dec 2022 04:42:11 +0000 (15:42 +1100)]
LU-16382 spec: use pkgconfig() as appropriate.
pkgconfig() is preferred over explicit dependencies on libfoo-devel.
For SUSE, this is particularly needed for system and with SLE15-SP4
the OBS automatically includes "systemd-mini", but that conflicts with
"system". Usig "pkgconfig(systemd) resolves this.
Also there is no need to have "Depends" for library packages. These
are determined automatically from the result of the build.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idfa4fdfd8bf060175b64a1991d3367024a368344
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49366
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Mr NeilBrown [Mon, 12 Dec 2022 04:25:31 +0000 (15:25 +1100)]
LU-16382 spec: Don't include Group: tags.
Fedora Project has deprecated group tags:
https://fedoraproject.org/wiki/RPMGroups
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_tags_and_sections
The groups tags currently used are not recognised by SUSE.
So remove all the Group: tags - except one.
The %kernel_module_package macro for SUSE requires that a group
be given.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6b7222200ea1a02319a703d64542dfb9780c048a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49364
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 7 Apr 2021 19:37:42 +0000 (12:37 -0700)]
LU-14224 misc: add firewalld service configuration
RHEL8 ships with restrictive firewalld rules out of the box.
This prevents servers and clients from connecting to each other.
Add a lustre.xml service file for firewalld, so that it is easy
to run a command like:
firewall-cmd --permanent --zone=public --add-service=lustre
to add the Lustre service ports with minimal difficulty.
It would be good if this was run automatically when the RPMs are
installed, or when mount.lustre is run, but it isn't clear what
is good/safe/correct in all cases. At least having the service
file will be a starting point to make this easier for admins.
It would be even better if the Lustre service rules were restricted
to accepting only new connections, and clients would only accept
requests from the MGS initially and then dynamically add ports for
servers as they are configured, but this is beyond my firewalld-fu.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9f49d4b0df1c9fb6b343df81f966d9110c300c1e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/41021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland <faaland1@llnl.gov>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Tue, 13 Oct 2020 14:12:23 +0000 (14:12 +0000)]
LU-14111 obdclass: count eviction per obd_device
Add a new 'obd_eviction_count' counter to obd_device which
is increased every time a client is evicted, which means
every time we call `class_fail_export()`.
Expose this counter through `lctl get_param *.*.eviction_count`
for every target.
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: I83b691662285cf2cd937187bffa54de6bd1f694c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/40528
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Andreas Dilger [Fri, 3 Feb 2023 10:14:39 +0000 (03:14 -0700)]
LU-16501 tgt: skip free inodes in OST weights
In lu_tgt_qos_weight_calc() calculate the target weight consistently
with how the per-OST and per-OSS penalty calculation is done in
ltd_qos_penalties_calc(). Otherwise, the QOS weighting calculations
combine two different units, which incorrectly weighs allocations on
OST with more free inodes over those with more free space.
Fixes:
d3090bb2b486 ("LU-11213 lod: share object alloc QoS code with LMV")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1ccc52d7ad5dc440ae48403ba129efd6a0a51c33
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49890
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:58:40 +0000 (16:58 -0500)]
LU-6142 lov: use list_for_each_entry in lov_obd.c
Using the *_entry macro simplifies the code slightly.
Change-Id: Ia50cd2cbaf9bac6c9873ef4af7e5cbfa2c8e660e
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49740
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 19 Jan 2023 19:06:36 +0000 (22:06 +0300)]
LU-14918 osd: don't declare similar zfs writes twice
in some cases (like overstriping) the same operations can be
declared multiple times (new llog records) and this lead to
huge number of credits and performance degradation. we can
avoid this checking for duplicate declarations.
notice each declare operation results in a allocation in ZFS.
the example for an overstriped file (2000 stripes over 4 OSTs),
declare ops before after
create: 2001 2
unlink: 10001 10
creation of 1K-stripe files (over 4 OSTs) is 2.5% faster.
removal of 1K-stripe files is 44% faster.
single-stripe file creation/removal does not degrade.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5d9e6d3a1574ccd7bf97fd3a67ab4fff0b6a352c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49701
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 7 Dec 2021 08:13:54 +0000 (11:13 +0300)]
LU-14918 osd: don't declare similar ldiskfs writes twice
in some cases (like overstriping) the same operations can be
declared multiple times (new llog records) and this lead to
huge number of credits and performance degradation. we can
avoid this checking for duplicate declarations.
As every declaration would need an allocation, limit the scope
of this checks to transaction likely to be large.
% of "large" transaction in sanity-benchmark, depending on threshold:
creates < 5 && writes < 5:
0.58% (mds1) and 2.97% (mds2)
create < 7 & writes < 7:
0.58% and 2.4%
create < 9 & writes < 9:
0.6% and 1.85%
create < 10 & write2 < 10:
0.0004% and 0.000001%
thus 10 creates or writes is selected as a threshold to enable this
logic.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I7c893fe3b95646b4b813b999bc832659dfcf03ad
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45765
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Vitaliy Kuznetsov [Wed, 8 Feb 2023 21:34:38 +0000 (00:34 +0300)]
LU-16454 mdt: Add a per-MDT "max_mod_rpcs_in_flight"
Value max_mod_rpcs_per_client doesn't define a static number of
slots for the per-client replies or anything, and the only
thing it is used for is to pass the limit to the client. For the
same reason, there also doesn't appear to be a particularly hard
limitation why the client cannot change and exceed the
server-provided parameter, except to avoid overloading the server
with too many RPCs at once, but that may also be true of the
current limit with a larger number of clients, no different than
"max_rpcs_in_flight".
This fix adds a tunable parameter "max_mod_rpcs_in_flight" per MDT
to lustre/mdt/mdt_lproc.c so that it can be set
with "lctl set_param" at runtime. The max_mod_rpcs_per_client global
setting is marked "deprecated" but is still used as the default
value when creating an MDT.
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I27cfcb68e1a534e80e6a2dbf2e1affc430803b49
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Oleg Drokin [Thu, 9 Feb 2023 17:38:57 +0000 (12:38 -0500)]
New tag 2.15.54
Change-Id: I592cabccefa9bbdf3d1d97fa313103b8b1b1eb3b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 1 Feb 2023 07:11:56 +0000 (23:11 -0800)]
LU-16500 utils: set default ost index for lfs migrate
Running "lfs migrate <file>" without any SETSTRIPE arguments
to balance space usage keeps the PFL file layout, but preserves
the OST selection exactly, which makes the migration virtually
useless for space balancing.
This patch fixes the above issue by clearing the specific
OST indices from the source layout before using the layout to
create the volatile file in lfs_migrate().
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I82e1dc0a11fdda7d555df994cf4e5f6e3dbdcb5c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49819
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Tue, 18 Jan 2022 13:55:01 +0000 (13:55 +0000)]
LU-930 ptlrpc: clarify AT error message
Clarify the error message related to passed deadline
for AT early replies. It was indicating that the system
was CPU bound which is most of the time wrong, as the issue
is rather communication failure delaying RPC traffic.
This could be confusing to people which will look for
CPU resource consumption where the network traffic is
more at cause.
Also try to use less cryptic keywords which makes only
sense to the feature developer, and not to admins.
Test-Parameters: trivial
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: Icdff8f4c6fb9905233f6b8ed1b961b2fd1127667
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49548
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 5 Dec 2022 18:59:02 +0000 (11:59 -0700)]
LU-16367 utils: clean up ldiskfs feature handling
Update the default ldiskfs features used by mkfs.lustre:
- enable large_dir on OSTs as well as MDTs
- remove obsolete handling of "ext3" filesystems
- clean up handling of other features that have become a bit messy
Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id717c3ba939ccf9b2de34e868d4415e88429ef39
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49316
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 27 Jan 2023 20:34:11 +0000 (12:34 -0800)]
LU-16221 kernel: new kernel [RHEL 9.1 5.14.0-162.12.1.el9_1]
This patch makes changes to support new RHEL 9.1 release
for Lustre client.
Test-Parameters: trivial clientdistro=el9.1 \
env=SANITY_EXCEPT="130 244a" testlist=sanity
Change-Id: I8af730f84c9ddf9dcb7e3ddfbd24a68173f51e8d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48938
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Shaun Tancheff [Mon, 30 Jan 2023 04:17:12 +0000 (22:17 -0600)]
LU-16510 build: fortified memcpy from linux 6.1
The fortified memcpy() from Linux v5.11-11104-ga28a6e860c6c
through v5.18-rc5-1405-g43213daed6d6 incorrectly reports
a false positive out of bounds check.
In function 'memcpy' ...
'__read_overflow2' declared with attribute error: detected
read beyond size of object passed as 2nd parameter
Test-Parameters: trivial
HPE-bug-id: LUS-11459
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I3a59d8b647833c05ff4b51e327ed8bce894141fe
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49811
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Shaun Tancheff [Thu, 19 Jan 2023 07:38:02 +0000 (01:38 -0600)]
LU-16292 llite: delete_from_page_cache not exported
Linux commit v5.16-rc4-44-g452e9e6992fe
filemap: Add filemap_remove_folio and __filemap_remove_folio
Directly removing a folio/page from the page cache is not
available.
Fallback to generic_error_remove_page for regular files,
and truncate_inode_pages_range as appropriate.
Test-Parameters: trivial
HPE-bug-id: LUS-11198
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I634e7d7719d497ce035a78b424be8e9e8c5a8104
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49069
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Aurelien Degremont [Mon, 26 Sep 2022 12:27:37 +0000 (12:27 +0000)]
LU-16188 mdt: fix incompatible HSM request handling
When the coordinator tries to send multiple hsm actions in
a single request, if one of the request fails incompat checks all the
requests are marked as STARTED but none of the requests are
sent to the agent.
Return EAGAIN from mdt_agent_hsm_send() so that the coordinator would
not mark the requests as STARTED. It would retry them later.
Add a sanity-hsm test.
Test-Parameters: trivial testlist=sanity-hsm
Change-Id: Id4fb858021be6dc6b0cbcf140c3f2051efce57ad
Signed-off-by: Jeya Ganesh Babu Jegatheesan <jeyaga@amazon.com>
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sun, 22 Jan 2023 17:43:29 +0000 (11:43 -0600)]
LU-16118 build: Workaround __write_overflow_field errors
Linux commit v5.17-rc3-1-gf68f2ff91512
fortify: Detect struct member overflows in memcpy() at compile-time
memcpy and memset of collections of struct members
will trigger:
error: call to ‘__write_overflow_field’ declared with attribute
warning: detected write beyond size of field (1st parameter);
maybe use struct_group()?
[-Werror] __write_overflow_field(p_size_field, size);
Test-Parameters: trivial
HPE-bug-id: LUS-11194
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iacd1ab03d1b90ce62b5d7b65e1cd518a5f7981f2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48364
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Sat, 21 Jan 2023 06:16:25 +0000 (00:16 -0600)]
LU-16354 ldiskfs: RHEL9.1 server support
ldiskfs patch series for RHEL9.1
Test-Parameters: trivial
HPE-bug-id: LUS-11332
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ia0757995ac7200eb50fadf5e106fe1d7b3dc0443
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49283
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Shaun Tancheff [Fri, 20 Jan 2023 15:27:15 +0000 (09:27 -0600)]
LU-16477 ldiskfs: Add ext4-enc-flag patch for RHEL9
Update ext4-enc-flag for linux 5.14 and include it
the 5.14 based RHEL9 and SUSE 15 SP4 ldiskfs series
Test-Parameters: trivial
HPE-bug-id: LUS-11442
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iaf4ba914fafe6a9e4ad58b74ae63343bb2918a44
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49635
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Aurelien Degremont [Thu, 7 Apr 2022 12:58:00 +0000 (12:58 +0000)]
LU-15728 llite: fix relatime support
relatime behavior is properly managed by VFS, however
Lustre also stores acmtime on OST objects and atime
updates for OST objects should honor relatime behavior.
This patch updates 'ci_noatime' feature which was introduced to
properly honor noatime option for OST objects, to also support
'relatime'.
file_is_noatime() code already comes from upstream touch_atime().
Add missing parts from touch_atime() to also support relatime.
It also forces atime to disk on MDD if ondisk atime is older than
ondisk mtime/ctime to match relatime (even if relatime is not enabled)
Add a new test for relatime feature.
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: I7a26f39841300a60c015944f9e544115b4446ead
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47017
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:51:18 +0000 (16:51 -0500)]
LU-6142 ldlm: minor list_entry improvements in ldlm_request.c
Small clarify improvements, and one local variable avoided.
Linux-commit:
cb830bef04f1bd80da7eca3d3edaea590f4b350b
Change-Id: I1a34849adca228a465a2b771fb0aa707a9283c7c
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49738
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:24:59 +0000 (16:24 -0500)]
LU-6142 ldlm: use list_for_each_entry in ldlm_lock.c
This makes some slightly-confusing code a bit clearer, and
avoids the need for 'tmp'.
Linux-commit:
557d001aa51fd6171d7a68dec21f8327fc824173
Change-Id: If9d070492e0016fa235fb38726f7c7a3b380d580
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49734
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 30 Jan 2023 21:46:37 +0000 (14:46 -0700)]
LU-12275 tests: skip new nodemap params on old MGS
Skip setting forbid_encryption and readonly_mount parameters on old
MGSes that do not support these options. Otherwise test_61 failures
are seen during interop testing. Running test_36 would also fail in
this case, except that it is already skipped due to encryption checks.
Test-Parameters: trivial testlist=sanity-sec
Fixes:
598c48707c ("LU-12275 tests: exercise file content encryption/decryption")
Fixes:
e7ce67de92 ("LU-15451 sec: read-only nodemap flag")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I94f2e2f609927fea618a3a22f103bd32ae3ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49828
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Fri, 20 Jan 2023 17:30:27 +0000 (12:30 -0500)]
LU-16412 llite: check read page past requested
Due to a kernel bug introduced in 5.12 in commit:
cbd59c48ae2bcadc4a7599c29cf32fd3f9b78251
("mm/filemap: use head pages in generic_file_buffered_read")
if the page immediately after the current read is in cache,
the kernel will try to read it.
This attempts to read a page past the end of requested
read from userspace, and so has not been safely locked by
Lustre.
For a page after the end of the current read, check wether
it is under the protection of a DLM lock. If so, we take a
reference on the DLM lock until the page read has finished
and then release the reference. If the page is not covered
by a DLM lock, then we are racing with the page being
removed from Lustre. In that case, we return
AOP_TRUNCATED_PAGE, which makes the kernel release its
reference on the page and retry the page read. This allows
the page to be removed from cache, so the kernel will not
find it and incorrectly attempt to read it again.
NB: Earlier versions of this description refer to stripe
boundaries, but the locking issue can occur whether or
not the page is on a stripe boundary, because dlmlocks
can cover part of a stripe. (This is rare, but is
allowed.)
Change-Id: Ib93bd0624fda0ed1c2b89f609d15208c86e21c29
Signed-off-by: Qian Yingjin <qian@ddn.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49723
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Tue, 10 Jan 2023 15:37:03 +0000 (08:37 -0700)]
LU-16457 tests: wait for remote sleep in sanity-pcc/101a
Wait longer for the remote sleep command to start on the agent node.
Test-Parameters: trivial testlist=sanity-pcc env=ONLY=101a,ONLY_REPEAT=200
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5dcbd6a7127b3e17aa658c87f5c75874432dc353
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49587
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 26 Jan 2023 07:34:25 +0000 (10:34 +0300)]
LU-16159 osp: destroy should not overtake writes
use transaction versioning for object destroy so that
destroy doesn't overtake writes, so writes don't hit
non-existing objects.
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Fixes:
b054fcd785 ("LU-16159 lod: cancel update llogs upon recovery abort")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iec2a5c72f27825820d36ebbe20d55fa303358982
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49787
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Andriy Skulysh [Mon, 21 Mar 2022 12:00:59 +0000 (14:00 +0200)]
LU-16431 mds: Close request is dropped during replay
MDS_CLOSE can have the same transno with SETATTR update.
But it still needs to be processed to close the file.
Change-Id: I44c8e10c5e30f2dca4fab4d49a74d147495640c2
HPE-bug-id: LUS-10838
Signed-off-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49506
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Thomas Bertschinger [Wed, 21 Dec 2022 16:52:50 +0000 (11:52 -0500)]
LU-16392 utils: use --list-commands for bash completion
The CLI utils lctl and lfs currently use a pseudo option
--non-existent-option to generate a list of completions. However, this
was broken when the help output for an invalid command was changed.
Using --list-commands instead means that the format of the help output
can be kept succinct.
However, currently there are 2 issues that make --list-commands
unsuitable.
First, --list-commands truncates long commands. This commit resolves
this by not truncating long commands, and removing the fixed-length
char buffer and writing directly to stdout so that the line length
can overflow slightly if needed.
Second, --list-commands recursively displays sub-commands. For
example, for `lctl`, it will display `pcc add`, `pcc del`, etc in
additon to just `pcc`. The bash completion tools would view these
as separate tokens and thus would inappropriately suggest `add`,
`del`, etc. as completions for `lctl`. This commit removes the
recursive behavior.
Removing the recursive behavior resolves an unrelated bug with the
recursion that can be observed for `lctl`, where a number of
top-level commands are skipped following recursion into a previous
sub-command, equal to the number of subcommands processed in the
recursive call. Specifically, the commands in the section "device
setup", e.g. `attach`, `detach`, were not displayed following the
recursive call into `pcc`.
Finally, this commit changes the command parser to recognize --help
and print the list of commands when this argument is seen.
Fixes:
bc69a8d058 ("LU-8621 utils: cmd help to stdout or short cmd error")
Signed-off-by: Thomas Bertschinger <bertschinger@lanl.gov>
Change-Id: Ib6e139402b9cd18e5a54b8fd3d6a2652d301e736
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49484
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Mon, 12 Dec 2022 04:20:08 +0000 (15:20 +1100)]
LU-16382 spec: Declare correct license
Lustre is primarily licensed under GPL-v2. Some files claim v2+,
others claim v2-only, but all are consistent with v2.
liblustreapi is LGPL2.1+
So make that explicit in lustre.spec. All 'kmp' packages are
GPL-v2-only, all the rest add "AND LGPL-2.1-or-later.
The Open Build Service complains that "GPL" is too vague.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4f10c50a39b5b48fed71b179bc888b0ae144444e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49363
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 14 Nov 2022 16:28:36 +0000 (17:28 +0100)]
LU-16310 sec: Lustre/HSM on enc file with enc key
Support for Lustre/HSM on encrypted files when the encryption key is
available requires similar attention as with file migration.
The volatile file used for HSM restore must have the same encryption
context as the Lustre file being restored, so that file content
remains accessible after the layout swap at the end of the restore
procedure.
Please note that using Lustre/HSM with the encryption key creates
clear text copies of encrypted files on the HSM backend storage.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I99cba202cd2c7c747bbe5c4ec7d9208c7f6baf4b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49153
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Sebastien Buisson [Thu, 3 Nov 2022 10:52:02 +0000 (11:52 +0100)]
LU-16205 sec: fid2path for encrypted files
Add support of fid2path for encrypted files. Server side returns raw
encrypted path name to client, which needs to process the returned
string. This is done from top to bottom, by iteratively decrypting
parent name and then doing a lookup on it, so that child can in turn
be decrypted.
For encrypted files that do not have their names encrypted, lookups
can be skipped. Indeed, name decryption is a no-op in this case, which
means it is not necessary to fetch the encryption key associated with
the parent inode.
Without the encryption key, lookups are skipped for the same reason.
But names have to be encoded and/or digested. So server needs to
insert FIDs of individual path components in the returned string.
These FIDs are interpreted by the client to build encoded/digested
names.
Add sanity-sec test_63 to exercise this new capability.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I165bf2e5657037ae2e25c9378e4713537ea94bec
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48930
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 17 Jun 2020 10:14:39 +0000 (04:14 -0600)]
LU-13703 utils: fix lfs_migrate with PFL arguments
Pass the '-c', '-S', and '--pool' options to "lfs migrate" when
they are part of a PFL component (after -E), rather than using
them to set the stripe_count and stripe_size of the whole file.
This precludes using '-A' and '-R' with explicitly specified PFL
file layouts, but that didn't make sense in the first place.
Fix the handling of "--pool <pool>" to use "-p <pool>" since
the script later only strips "-p " from the pool name.
Test-Parameters: trivial
Fixes:
60c5bc25025 ("LU-8235 scripts: pass unrecognized options to lfs migrate")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib7fb6e08d81dbae77e8348fc5f09837c612540e5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/39145
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 11 Nov 2020 21:53:22 +0000 (14:53 -0700)]
LU-13475 utils: disable lfs_migrate rsync and warning
The --rsync option is no longer enabled by default for fallback if
'lfs migrate' fails for some reason, and is mandatory for rsync usage.
The warning message and "-y" option of lfs_migrate is no longer needed
if rsync is not used, and is only shown if --rsync is used.
Remove the LFS_MIGRATE_RSYNC_MODE variable that was used for tests
and instead pass the "--rsync" option directly when needed.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I70b70d969f2dc8b4836c6c7692e6a73a0e2540e5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/40614
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:55:55 +0000 (16:55 -0500)]
LU-6142 ldlm: use list_for_each_entry in ldlm_resource.c
Having a stand-alone "list_entry()" call is often a sign
that something like "list_for_each_entry()" would
make the code clearer.
Linux-commit:
5eb50608ed0fa076d2783898055fb20934a3828c
Change-Id: I5abd6cc7ec0abd31acc55f5af58f440c4f7609a7
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49739
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:45:05 +0000 (16:45 -0500)]
LU-6142 ldlm: use list_first_entry in ldlm_lockd.c
This is only a small simplification, but it makes the code
a little clearer.
Linux-commit:
7378caf4fe5198ce572654c926437fba12fb2255
Change-Id: Ie65049e12a1b1bbe448baefc38a6657d831e0670
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49737
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:33:34 +0000 (16:33 -0500)]
LU-6142 lustre: obdclass: simplify cl_lock_fini()
Using list_first_entry_or_null() makes this (slightly)
simpler.
Linux-commit:
988b9ea9129bc24baf36ee421feb823285f234c4
Change-Id: Ic2fe2bb58b67781c8bc7b4e81cbf6b61dcaa56fb
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49736
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:29:57 +0000 (16:29 -0500)]
LU-6142 lov: simplfy lov_finish_set()
When deleting everything from a list, a while loop
is cleaner than list_for_each_safe().
Linux-commit:
dff162689a4061ff30d3a05f9d790e375c06ab8f
Change-Id: I90d98ebf14f461796d6f9d31a2c62de1520034cc
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49735
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Dean Luick [Thu, 19 Jan 2023 20:38:04 +0000 (21:38 +0100)]
LU-16349 o2iblnd: Fix key mismatch issue
If a pool memory region (mr) is mapped then unmapped without being
used, its key becomes out of sync with the RDMA subsystem.
At pool mr map time, the present code will create a local
invalidate work request (wr) using the mr's present key and then
change the mr's key. When the mr is first used after being mapped,
the local invalidate wr will invalidate the original mr key, and
then a fast register wr is used with the modified key. The fast
register will update the RDMA subsystem's key for the mr.
The error occurs when the mr is never used. The next time the mr
is mapped, a local invalidate wr will again be created, but this
time it will use the mr's modified key. The RDMA subsystem never
saw the original local invalidate, so now the RDMA subsystem's
key for the mr and o2iblnd's key for the mr are out of sync.
Fix the issue by tracking if the invalidate has been used.
Repurpose the boolean frd->frd_valid. Presently, frd_valid is
always false. Remove the code that used frd_valid to conditionally
split the invalidate from the fast register. Instead, use frd_valid
to indicate when a new invalidate needs to be generated. After a
post, evaluate if the invalidate was successfully used in the post.
These changes are only meaningful to the FRWR code path. The failure
has only been observed when using Omni-Path Architecture.
Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Change-Id: I532a11f10ae6a5917a4c054f37747d08eb4d6331
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49714
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Tue, 10 Jan 2023 04:55:47 +0000 (04:55 +0000)]
LU-13482 utils: bandwidth limit for lfs migrate
Add an option -W to control how much bandwidth
an lfs migrate job can consume. The migrate job
will periodically sleep to meet the bandwidth
restrictions.
This patch also adds a --stats option. The option
produces regular logs entries tracking the progress
of the migrate job. The logs are output in YAML
format. The frequency of the logs is controlled
by --stats-interval. This interval defaults to 5
seconds.
Also included are two tests, 56xh and 56xi. The
first verifies the functionality of the bandwidth
control. The second checks that the output is in
valid YAML and that the stats get printed without
using -W.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ic71cceb2434a737e3ad8bd325f719e37a70b0047
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Fri, 23 Dec 2022 12:26:27 +0000 (17:56 +0530)]
LU-6142 ldlm: Fix style issues for ldlm_extent.c
This patch fixes issues reported by checkpatch
for file lustre/ldlm/ldlm_extent.c
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I9cecd1f377f33f3d4129cddcd7b59c3a7c003e04
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49536
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Alex Zhuravlev [Thu, 2 Sep 2021 15:50:19 +0000 (18:50 +0300)]
LU-14980 lfsck: lock object in __lfsck_layout_update_pfid()
once the transaction has been started
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie43fe89009a123c88eb0e202ec961b52157e56c6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/44823
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 20 Jan 2023 06:22:18 +0000 (23:22 -0700)]
LU-14692 tests: restore sanity/312 to always_except
The sanity test_312 was incorrectly removed from ALWAYS_EXCEPT.
Fixes:
eaae465556 ("LU-14692 tests: allow FID_SEQ_NORMAL for MDT0000")
Test-Parameters: trivial testlist=sanity fstype=zfs
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6e8ed42561809b28fd6d5b4f7ee1104080ebe756
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49720
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Arshad Hussain [Thu, 19 Jan 2023 02:41:43 +0000 (08:11 +0530)]
LU-16492 tests: sanity/398b variable used without assignment
This patch initilizes 'before' variable with UNIX
timestamp. Variable 'before' was used without assigning
any value.
Test-Parameters: trivial testlist=sanity env=ONLY=398b
Fixes:
b4880f37582a ("LU-15483 tests: Improve test 398b")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Iba9e361735272d9c640a115f520ee7c60ac41239
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49687
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 18 Jan 2023 18:26:09 +0000 (11:26 -0700)]
LU-16425 tests: skip interop recovery-small/144a/144b
Skip recovery-small test_144a and test_144b for old MDS
missing the fix and for its corresponding test.
Fixes:
240938f7b1 ("LU-8367 tests: cleanup_orphans hang reproducer")
Fixes:
aa6250b741 ("LU-15724 tests: MDT failover hang reproducer")
Test-Parameters: trivial testlist=recovery-small env=ONLY=144 serverversion=2.14.0
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I77bfdf55d0218aa9e252f742cc90f1c61216d506
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49679
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sarah Liu <sarah@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Fri, 13 Jan 2023 19:33:15 +0000 (19:33 +0000)]
LU-15581 misc: update .gitignore files
Ignore the binary for check_iam utility
in lustre/utils.
Also, ignore more files for commit messages.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I5b11dc2d2f3761f778549a121ac940edeeb70980
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49632
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Timothy Day [Thu, 15 Dec 2022 06:19:01 +0000 (06:19 +0000)]
LU-14707 tests: Prefer #!/bin/bash
Change remaining #!/bin/sh to use bash.
Add a warning to the git-hook about using
sh in shebangs. Using bash allows scripts to
freely use bash-isms and lowers the risks
of bugs on Debian based platforms.
Also, change remaining callers to use bash
rather than sh.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I10f3e8f71435c38cfc1650dd13168d7ed5d3b31f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49479
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Qian Yingjin [Mon, 19 Dec 2022 06:57:39 +0000 (01:57 -0500)]
LU-16412 llite: check truncated page in ->readpage()
The page end offset calculation in filemap_get_read_batch() was
off by one. This bug was introduced in commit v5.11-10234-gcbd59c48ae
("mm/filemap: use head pages in generic_file_buffered_read")
When a read is submitted with end offset 1048575, it calculates
the end page index for read of 256 where it should be 255. This
results in the readpage() call for the page with index 256 is over
stripe boundary and may not be covered by a DLM extent lock.
This happens in a corner race case: filemap_get_read_batch()
batches the page with index 256 for read, but later this page is
removed from page cache due to the lock protected it being revoked,
but has a reference count due to the batch. This results in this
page in the read path is not covered by any DLM lock.
The solution is simple. We can check whether the page was
truncated and removed from page cache in ->readpage() by the
address_sapce pointer of the page. If it was truncated, return
AOP_TRUNCATED_PAGE to the upper caller. This will cause the
kernel to retry to batch pages and the truncated page will not
be added as it was already removed from page cache of the file.
Add sanityn/test_95 to verify it.
Test-Parameters: testlist=sanityn env=ONLY=95 clientdistro=ubuntu2204
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I192df92b1d1b79057055430cc81cb7cc760cc9ed
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Mon, 16 Jan 2023 02:21:09 +0000 (21:21 -0500)]
LU-15880 quota: fix insane grant quota
Fix the insane grant value in quota master/slave index,
the logs often contain the content similar to the following,
LustreError: 39815:0:(qmt_handler.c:527:qmt_dqacq0())
$$$ Release too much! uuid:work-MDT0000-lwp-MDT0002_UUID
release:
18446744070274413724 granted:
18446744070291193856,
total:
4118877744 qmt:work-QMT0000 pool:0-dt id:40212 enforced:1
hard:
128849018880 soft:
12884901888 granted:
4118877744 time:0
qunit:
16777216 edquot:0 may_rel:0 revoke:0 default:no
It could be caused by chgrp, which reserves quota before changing
GID for some file at MDT, then release the reserved quota after
the file GID has been changed on the corresponding OST, (this issue
is tracked at LU-5152 and LU-11303)
In some case, some quota could be released even the quota was not
reserved correctly, which cause the grant quota to be some negative
value, which is regarded as some insane big value because the type
of grant is "__u64", then the normal grant release will fail and
the grant field of some quota ID in the quota file (both at QMT and
QSD) contain insane value, but can't be reset correctly.
This patch resets the affected quota by clear the quota limits and
grant, and the grant will be reported by each QSD when the quota ID
is enforced again, then rebuild the grant at QMT.
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I083afa3b6648db5a1ccca0235667da022ff27e65
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48981
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Sun, 18 Jul 2021 00:01:25 +0000 (19:01 -0500)]
LU-16374 enc: align Base64 encoding with RFC 4648 base64url
Lustre encryption uses a Base64 encoding to encode no-key filenames
(the filenames that are presented to userspace when a directory is
listed without its encryption key).
Make this Base64 encoding compliant with RFC 4648 base64url. And use
'+' leading character to distringuish digested names.
This is adapted from kernel commit
ba47b515f594 fscrypt: align Base64 encoding with RFC 4648 base64url
To maintain compatibility with older clients, a new llite parameter
named 'filename_enc_use_old_base64' is introduced, set to 0 by
default. When 0, Lustre uses new-fashion base64 encoding. When set to
1, Lustre uses old-style base64 encoding.
To set this parameter globally for all clients, do on the MGS:
mgs# lctl set_param -P llite.*.filename_enc_use_old_base64={0,1}
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaa2256da7fb591d842b5bb7aa474b2ee6de9899d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49581
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sat, 14 Jan 2023 07:13:01 +0000 (01:13 -0600)]
LU-16476 ldiskfs: Fix old ea inode handling
ext4-old_ea_inodes_handling_fix.patch is applicable
to all linux version 4.18 and higher.
Apply it to all the current 5* series
Test-Parameters: trivial
HPE-bug-id: LUS-11441
Fixes:
8da23f070c ("LU-15544 ldiskfs: SUSE 15 SP4 kernel 5.14.21 SUSE")
Fixes:
1819f6006f ("LU-15801 ldiskfs: Server support for RHEL9")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I1d22ed9d505e1bb407d9388cac9c881b366b96a8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49634
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Qian Yingjin [Fri, 19 Aug 2022 02:32:36 +0000 (22:32 -0400)]
LU-16096 recovery: upgrade reply data after recovery finish
As the batched RPC protocol will change the disk format of the
client reply data "REPLY_DATA" for recovery, thus we need to
handle the compatibility during upgrade carefully for this new
format change of the reply data.
The solution is as follows:
When the client recovery has finished, the target truncates the
reply data file with zero size and rewrite the header to use the
new magic and reply data record size.
And then new reply data records will be written in the new format.
Enable the test case conf-sanity/32, 108 as the compatibility issue
is fixed.
This patch also fixes the usage of struct lsd_reply_data in
lustre/utils/lr_reader.c to support both struct versions.
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I26921d41915b8cad2d913e15f502f4543180c5c6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48261
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 20 Jan 2023 20:34:42 +0000 (13:34 -0700)]
LU-11170 tests: add debugging to sanity/415
Add a loop of renames without the concurrent 'touch' operation to
measure the test time, and then a second loop that has the 'touch'
so that we can see whether slow renames are because of COS (which
would make the test failure a kernel bug to be fixed), or because
the test is running in a VM and the server/disk is slow (which is
something to be fixed in the test (e.g. by making "slow" relative
to the non-touch baseline time).
Test-Parameters: trivial testlist=sanity env=ONLY=415,ONLY_REPEAT=120 mdscount=2 mdtcount=4
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic1a952be0b861005b46da3e673216e455f3ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49724
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Nikitas Angelinas [Fri, 6 Jan 2023 19:01:52 +0000 (21:01 +0200)]
LU-16464 osp: fix off-by-one errors in oxe_can_hold()
There are a couple of off-by-one errors when calculating the required
buffer size in oxe_can_hold(), which can cause the xattr entry to be
reallocated unnecessarily.
HPE-bug-id: LUS-11423
Fixes:
a1c5adf7f466 ("LU-14607 osp: separate buffer for large XATTR")
Change-Id: I486963066d7f8783ad64f1ea110fb73db0a8274b
Signed-off-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 11 Jan 2023 19:02:04 +0000 (12:02 -0700)]
LU-16456 tests: skip conf-sanity test_129/132 in interop
test_129 was added in commit v2_14_56-40-gcefabee52
test_132 was added in commit v2_14_56-96-ge26d7cc39
They should be skipped for older MDS versions.
Test-Parameters: trivial testlist=conf-sanity env=ONLY=122-133 serverversion=2.14.0
Fixes:
cefabee52 ("LU-15112 mgc: do not ignore target registration failure")
Fixes:
e26d7cc399 ("LU-14399 hsm: process hsm_actions in coordinator")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If1e276c816ecf2f30dc970f9b5afe85d722540e5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49601
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Sarah Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Tue, 3 Jan 2023 17:03:38 +0000 (10:03 -0700)]
LU-16461 kfilnd: Modify peer credits and RX buffers
It's desirable to lower peer credits because smaller values allow us
to cancel outstanding traffic to down peers faster (because there is
less traffic in flight). Testing shows that peer_credits 16 does not
perform any worse than our current default. Let's make 16 the
new default.
In addition, testing shows a benefit for further increasing the
default number of immediate receive buffers. Increase this to 8.
HPE-bug-id: LUS-11421
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I877fe408b276071f33a99c8b3b50d13f597aaa29
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49592
Reviewed-by: Ron Gredvig <ron.gredvig@hpe.com>
Reviewed-by: Ian Ziemba <ian.ziemba@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Tue, 1 Nov 2022 19:39:39 +0000 (13:39 -0600)]
LU-16451 kfilnd: Improve CQ error logging
Improve CQ error logging for send events by printing the errno from
the CQ event as well as the provider error. This should allow us to
better root cause TN failures.
Also remove an extra newline character.
HPE-bug-id: LUS-11314
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I79bbe0312a9124dd34285d43b6e83f9d897923c1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49589
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ron Gredvig <ron.gredvig@hpe.com>
Reviewed-by: Ian Ziemba <ian.ziemba@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Fri, 6 Jan 2023 20:56:13 +0000 (20:56 +0000)]
LU-13530 build: Add kernel version to depmod
The depmod commands in the postrm and
postinst scripts should use the kernel
version the package is built against.
Otherwise, depmod will use the current
kernel version - which might be different.
This patch also adds a line indicating that
the file has been modified.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I355420a85ea0ed301433816588758197795b5ede
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49573
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Thomas Stibor <thomas@stibor.net>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 5 Jan 2023 14:06:39 +0000 (15:06 +0100)]
LU-16445 sec: make nodemap root squash independent of map_mode
When the admin property is set to 0 on a nodemap, the root user must
be squashed, even if the map_mode property specifies to not map uids
or gids.
Enhance sanity-sec test_17 to exercise this use case.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1b41caa1ccc6e544ce9fac45b47d0c4c129221f7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49561
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Sat, 14 Jan 2023 17:56:14 +0000 (01:56 +0800)]
LU-16285 ldlm: send the cancel RPC asap
This patch try to send cancel RPC ASAP when bl_ast
received from server. The exist problem is that
lock could be added in regular queue before bl_ast
arrived since other reason. It will prevent lock
canceling in timely manner. The other problem is
that we collect many locks in one RPC to save
the network traffic. But this process could take
a long time when dirty pages flushing.
- The lock canceling will be processed even lock has
been added to bl queue while bl_ast arrived. Unless
the cancel RPC has been sent.
- Send the cancel RPC immediatly for bl_ast lock. Don't
try to add more locks in such case.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ie5efff3f1ed4e46448371185a0c08968233e7644
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49527
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sergey Cheremencev [Sat, 17 Dec 2022 21:42:10 +0000 (01:42 +0400)]
LU-16415 quota: enforce project quota for root
Patch adds an option to enforce project quotas for root.
It is disabled by default, to enable set
osd-ldiskfs.*.quota_slave.root_prj_enable to 1
at each target where you need this option.
Patch also adds sanity-quota_1j to test new feature.
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I978dc8442235149794f85110309f63bc560defdc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49460
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Sat, 26 Nov 2022 06:46:32 +0000 (23:46 -0700)]
LU-16367 misc: remove deprecated code
Remove code that is or will become deprecated in this release based
on the LUSTRE_VERSION_CODE checks.
Fixes:
53fa817657 ("LU-12514 llite: move client mounting from obdclass to llite")
Fixes:
3919a282ca ("LU-15106 ofd: quiet deprecated param warning")
Fixes:
115bba9ffb ("LU-11110 ofd: remove obdfilter.*.* symlinks in few releases")
Fixes:
73f15ad0f1 ("LU-9378 utils: split getstripe and find from lfs.1")
Fixes:
88d8f0f86b ("LU-11891 utils: getstripe use --mdt-index consistently")
Fixes:
78be823f33 ("LU-15218 quota: delete unused quota ID")
Fixes:
0c5fbd80f1 ("LU-5969 lustreapi: replace llapi_get_version()")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id59371084a102d6d8257c45b55d68077f2ce7057
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49338
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Mon, 28 Nov 2022 09:17:25 +0000 (12:17 +0300)]
LU-16345 ofd: ofd_commitrw_read() with non-existing object
a client can get evicted during OST_READ's bulk so it's LDLM
lock is cancelled and OST_DESTOY can remove the object.
ofd_commitrw_read() still needs to release the buffers and
ignore the object doesn't exist.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ibe9413de41c23b1b4f6d52e9b17a06590b3c0726
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49255
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Qian Yingjin [Wed, 23 Nov 2022 09:42:53 +0000 (04:42 -0500)]
LU-16338 readahead: add stats for read-ahead page count
This patch adds the stats for read-ahead page count:
lctl get_param llite.*.read_ahead_stats
llite.lustre-
ffff938b7849d000.read_ahead_stats=
snapshot_time 4011.
320890492 secs.nsecs
start_time 0.
000000000 secs.nsecs
elapsed_time 4011.
320890492 secs.nsecs
hits 4 samples [pages]
misses 1 samples [pages]
zero_size_window 4 samples [pages]
failed_to_reach_end 1 samples [pages]
failed_to_fast_read 1 samples [pages]
readahead_pages 1 samples [pages] 255 255 255
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iada06eb7d78ab28cfcc7167e49d25da252da4009
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49224
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andriy Skulysh [Fri, 3 Jun 2022 12:28:14 +0000 (15:28 +0300)]
LU-16333 osc: page fault in osc_release_bounce_pages()
pga[i] can be uninitialized. It happens after following
code path in osc_build_rpc():
OBD_SLAB_ALLOC_PTR_GFP(oa, osc_obdo_kmem, GFP_NOFS);
if (oa == NULL)
GOTO(out, rc = -ENOMEM);
Fixes:
a9ed5b149646 ("LU-12275 sec: encryption for write path")
HPE-bug-id: LUS-10991
Signed-off-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I7e21cb9ab69f0bce9c1bdb53a4b0ac7a673887cc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49210
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Deiter [Wed, 9 Nov 2022 17:06:37 +0000 (21:06 +0400)]
LU-15495 tests: fixed dbench test
* Using awk to get list shared libraries
* Fixed shellcheck warnings
Test-Parameters: trivial testlist=sanity \
clientdistro=el7.9 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=el8.6 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=el8.6 clientarch=aarch64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=el8.6 clientarch=ppc64le \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=el9.0 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=sles12sp5 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=sles15sp4 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=ubuntu2004 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial testlist=sanity \
clientdistro=ubuntu2204 clientarch=x86_64 \
env=SLOW=yes,ONLY=71
Test-Parameters: trivial env=SLOW=yes,ONLY=26 testlist=replay-dual
Test-Parameters: trivial env=SLOW=yes,ONLY=70b testlist=replay-single
Test-Parameters: trivial env=SLOW=yes,ONLY=8 testlist=sanity-pfl
Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes,ONLY=8 \
testlist=sanity-quota
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: Ic28bd67dcfb5ff24e65e33278ac867409a2c1cc6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49088
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Cyril Bordage [Tue, 25 Oct 2022 16:52:30 +0000 (18:52 +0200)]
LU-16267 lnet: fix missing error check in LUTF
In find_replace_file function, the file is opened with default
encoding option. If the file has a different encoding it will fail.
The solution is to use a try/except for UnicodeDecodeError and skip
bad encoded files.
Test-Parameters: @lnet
Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Change-Id: I9115d39414d31b628d550e8289b3193d13787288
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48951
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Mon, 17 Oct 2022 05:36:14 +0000 (13:36 +0800)]
LU-16228 utils: add lljobstat util
lljobstat util read datas from job_stats file(s),
parse, aggregate the data and list top jobs.
For example:
$ ./lljobstats -n 1 -c 3
---
timestamp:
1665984678
top_jobs:
- ll_sa_3508505.0: {ops: 64, ga: 64}
- touch.500: {ops: 6, op: 1, cl: 1, mn: 1, ga: 1, sa: 2}
- bash.0: {ops: 3, ga: 3}
...
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I0c4ac619496c184a5aebbaf8674f5090ab722d72
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48888
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Mon, 3 Oct 2022 07:51:16 +0000 (14:51 +0700)]
LU-16115 build: Linux 5.17 external module support
Linux commit v5.16-rc3-26-g129ab0d2d9f3
Added quotes around "$(CONFIG_CC_VERSION_TEXT)", however
.config stores CONFIG_CC_VERSION_TEXT with quotes thus
breaking the GNU make Makefile for external modules.
We need to supply a non-quoted value and override the
value in .config before it is used.
Test-Parameters: trivial
HPE-bug-id: LUS-11190
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I997b6987ef37a8c5b9d8f0984e81d9402a2ea705
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48360
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 7 Dec 2022 02:42:33 +0000 (20:42 -0600)]
LU-13485 ldiskfs: Parallel configure tests for ldiskfs
Transform the compile tests in ldiskfs to run in parallel
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I3a097ab5cd18b57e9311980d9aa708ed25f58464
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/38351
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sun, 15 Jan 2023 02:42:31 +0000 (20:42 -0600)]
LU-13485 kernel: Parallel core configure tests
Transform the compile tests in lustre-core to run in parallel
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I46fa852659eb4db86a12ec4ad3efddd0fdd3a655
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/38361
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Sun, 15 Jan 2023 01:57:02 +0000 (19:57 -0600)]
LU-13485 libcfs: Parallel configure tests for libcfs
Transform the compile tests in libcfs to run in parallel
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I76ab65558dd456dc08d6ef4a1985455ce1f17913
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/38349
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andrew Perepechko [Sun, 15 Jan 2023 16:55:58 +0000 (11:55 -0500)]
LU-16160 llite: SIGBUS is possible on a race with page reclaim
We can restart fault handling if page truncation happens
in parallel with the fault handler.
Change-Id: I6e60783e3334f87e799dc8b0e6e63d0bb358a236
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Signed-off-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49647
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andrew Perepechko [Mon, 16 Jan 2023 13:13:34 +0000 (08:13 -0500)]
LU-16480 lov: fiemap improperly handles fm_extent_count=0
FIEMAP calls with fm_extent_count=0 are supposed only to
return the number of extents.
lov_object_fiemap() attempts to initialize stripe_last
based on fiemap->fm_extents[0] which is not initialized
in userspace and not even allocated in kernelspace.
Eventually, the call exits with -EINVAL and "FIEMAP does
not init start entry" kernel log message.
Fixes:
409719608c ("LU-11848 lov: FIEMAP support for PFL and FLR file")
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Change-Id: I65e706b5dd5c8a6db90a539c2602af839b4da823
HPE-bug-id: LUS-11443
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49645
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
James Simmons [Thu, 12 Jan 2023 14:02:54 +0000 (09:02 -0500)]
LU-9680 utils: new llapi_param_display_value().
Currently the special YAML handling is done in lustre_cfg.c
for param handling. Other functionality internal to
liblustreapi.so will use this as well so move the handling
internal to liblustreapi.so. Currently we only make the new
llapi_param_display_value() function visible only to the
liblustreapi internal code. Later when we support /sys access
we can make this available for general use.
The "lctl get_param" and "lctl list_param" generally worked
for non-root users, but not for parameters under
/sys/kernel/debug due to permission changes in the kernel.
We still lacked proper non-root access for lctl get_param and
lctl list_param. Implement full lctl get_param functionality
for non-root users. Also make lctl list_param work for
non-root users. These changes will also work with any
parameters implemented with Netlink.
Change-Id: Ifd9aad16decb0803a336314d4dea38664ff41aa4
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49491
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Tue, 16 Aug 2022 07:57:47 +0000 (03:57 -0400)]
LU-14393 recovery: reply reconstruction for batched RPCs
Batched RPC can boost the metadata performance for Lustre
dramatically. However, it also increases the complexity of the
recovery, such as how to reconstruct the reply in case of the RPC
resend if the reply was lost.
In this patch, it adds a new field @lrd_batch_idx in the data
structure @lsd_reply_data to store each slot of the "reply_data"
file:
struct lsd_reply_data {
__u64 lrd_transno; /* transaction number */
__u64 lrd_xid; /* transmission id */
__u64 lrd_data; /* per-operation data */
__u32 lrd_result; /* request result */
__u32 lrd_client_gen; /* client generation */
__u32 lrd_batch_idx; /* index in a batched RPC */
__u32 lrd_padding[7]; /* unused fields */
};
When found that a batched RPC was a resend RPC request, and if
the index of the sub request in the batched RPC is smaller or
equal than @lrd_batch_idx in the reply data, it means that the sub
request has already executed, the server will reconstruct the
reply for this sub request; if the index is larger than
@lrd_batch_idx, the server will re-execute the sub reqeust in the
batched RPC.
Disable conf-sanity/32{a,b,c,d,e,f,g}, 108{a,b} temporarily until
the compatibility issue during upgrade for new reply data format
is fixed.
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Id48ecc263002cb783f5032642d05e1f3f6673837
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48228
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Mon, 1 Feb 2021 03:51:08 +0000 (11:51 +0800)]
LU-14393 protocol: basic batching processing framework
Batching processing can obtain boost performace. The larger the
batch size, the higher the latency for the entire batch. Although
the latency for the entire batch of operations is higher than the
latency of any single operation, the throughput of the batch of
operations is much high.
This patch implements the basic batching processing framework for
Lustre. It could be used for the future batching statahead and
WBC.
A batched RPC does not require that the opcodes of sub requests in
a batch are same. Each sub request has its own opcode. It allows
batching not only read-only requests but also multiple
modification updates with different opcodes, and even a mixed
workload which contains both read-only requests and modification
updates.
For the recovery, only the batched RPC contains a client XID,
there is no separate client XID for each sub-request. Although the
server will generate a transno for each update sub request, but
the transno only stores into the batched RPC (in @ptlrpc_body)
when the sub update request is finished. Thus the batched RPC only
stores the transno of the last sub update request. Only the
batched RPC contains the @ptlrpc_body message field. Each sub
request in a batched RPC does not contain @ptlrpc_body field.
A new field named @lrd_batch_idx is added in the client reply data
@lsd_reply_data. It indicates the sub request index in a batched
RPC. When the server finished a sub update request, it will update
@lrd_batch_idx accordingly.
When found that a batched RPC was a resend RPC, and if the index
of the sub request in the batched RPC is smaller or equal than
@lrd_batch_idx in the reply data, it means that the sub request has
already executed and committed, the server will reconstruct the
reply for this sub request; if the index is larger than
@lrd_batch_idx, the server will re-execute the sub request in the
batched RPC.
To simplify the reply/resend of the batched RPCs, the batch
processing stops at the first failure in the current design.
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Idaa814e82c968811bdda1c750b18c878b2c2ca67
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/41378
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Sat, 29 Oct 2022 22:30:17 +0000 (16:30 -0600)]
LU-16452 kfilnd: Check replay deadline before send
The LND timeout needs to account for the total time needed for bulk
operations to complete. On cassini, this can be ~120 seconds due to
the CXI retry-handler timeout on both the sender and target. i.e. LND
timeout is really the max round trip time, and (LND timeout)/2 is the
max one-way trip time.
When we replay a transaction we want to at least ensure we have enough
time to deliver the message to the receiver, as this gives us a
chance at still completing transactions. We should ensure that we
still have (LND timeout)/2 seconds remaining before posting a new
transaction.
Introduce kfilnd_transaction::tn_replay_deadline,
which is set to the transaction deadline minus (LND timeout)/2.
Check the replay deadline in kfilnd_tn_state_idle() before attempting
to post the transaction. If we've exceeded that deadline then fail
the transaction with -ETIMEDOUT and set a NETWORK_TIMEOUT health
status.
Modify the throttle check in kfilnd_tn_state_idle() to check
kfilnd_transaction::tn_replay_deadline instead of
kfilnd_transaction::deadline to determine when we should timeout
a transaction that is being throttled. Note, this check is switched
to using ktime_before() rather than ktime_after() since the case
is about checking whether we are currently before the deadline rather
than after it. The current code isn't wrong. It is just grammatically
awkward.
HPE-bug-id: LUS-11304
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I1911d51cee4acea20577e3fc45c99b8948b79523
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49593
Reviewed-by: Ron Gredvig <ron.gredvig@hpe.com>
Reviewed-by: Ian Ziemba <ian.ziemba@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Fri, 28 Oct 2022 22:27:17 +0000 (16:27 -0600)]
LU-16451 kfilnd: Throttle traffic to down peers
If a transaction fails with -EHOSTUNREACH then this suggests the
target is actually down. We want to avoid consuming resources on the
local NIC by trying to send messages to down peers, so we will
require a hello handshake before injecting other new messages to a
peer we suspect is down.
Introduce a new kfilnd_peer state, KP_STATE_DOWN. Peers in either
KP_STATE_UPTODATE or KP_STATE_STALE can transition to KP_STATE_DOWN.
We'll transition a peer to KP_STATE_DOWN if we fail a transaction with
them and the errno we get is either -EHOSTUNREACH or -ENOTCONN.
kfilnd_peer_down() transitions a peer to KP_STATE_DOWN as appropriate.
Similar to stale peers, if we continue to fail transactions with peers
that are down then we want to eventually purge them from the peer
cache. This logic in kfilnd_peer_stale() is moved to
kfilnd_peer_purge_old_peer(), and this new function is called by both
kfilnd_peer_stale() and kfilnd_peer_down().
Introduce kfilnd_peer_needs_throttle() that determines whether we
should queue a message for future replay pending a successful
handshake. Integrate this into kfilnd_tn_state_idle() so that we queue
messages for peers in KP_STATE_DOWN in addition to peers in
KP_STATE_NEW. Modify debug statements in this area to remove redundant
info and reflect that we can hit these conditions for down peers, not
just new peers.
Introduce kfilnd_peer_tn_failed() to interpret the errno for a
transaction failure and call kfilnd_peer_down() or kfilnd_peer_stale()
as appropriate. This function replaces all existing calls to
kfilnd_peer_stale(). kfilnd_peer_stale() is now a static function.
HPE-bug-id: LUS-11314
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I206075c3a1b2836715dc79b49b098dab51c6bb94
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49591
Reviewed-by: Ron Gredvig <ron.gredvig@hpe.com>
Reviewed-by: Ian Ziemba <ian.ziemba@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Tue, 25 Oct 2022 19:21:17 +0000 (13:21 -0600)]
LU-16450 kfilnd: Cancel TNs if handshake fails
When sending a message to a new peer a HELLO is sent first and the
original message waits for the handshake to complete. If the HELLO
fails to be sent then the original message will continue to wait for
the full LND timeout. When we retry the original message we should
check whether there is actually an outstanding HELLO. If not, then
this indicates the HELLO failed and we should cancel the TN.
HPE-bug-id: LUS-11310
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I4ed07964d5af0bcc3bdca33c1ea46fd436af2e98
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49590
Reviewed-by: Ron Gredvig <ron.gredvig@hpe.com>
Reviewed-by: Ian Ziemba <ian.ziemba@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Elena Gryaznova [Sun, 8 Jan 2023 18:05:19 +0000 (19:05 +0100)]
LU-16455 tests: recovery-small test_139() fix
mds device calculated before stop () can not be used
after stop() because of a device-mapper device is removed and
facet device is restored:
stop () ->
elif dm_flakey_supported $facet; then
if [[ -n ${!failover_host} &&
${!failover_host} != ${!host} ]]
dm_cleanup_dev $facet ->
unexport_dm_dev $facet
Without this fix test_139 fails on failover setup:
losetup: /dev/mapper/mds1_flakey: failed to set up loop device:
No such file or directory
To reproduce the failure just run:
sh llmountcleanup.sh
ONLY=139 sh recovery-small.sh
on failover setup where mds1_HOST != mds1failover_HOST
Fixes:
4597fa7d88 ("LU-13061 osp: check catlog FID after reading in")
Test-Parameters: trivial testlist="recovery-small" failover=true iscsi=1 \
env=ONLY=139,SLOW=yes mdssizegb=10 clientcount=4 osscount=2 mdscount=2 \
mdtcount=2 austeroptions=-R
Test-Parameters: trivial testlist="recovery-small" failover=true iscsi=1 \
env=FAILURE_MODE=HARD,ONLY=139,SLOW=yes mdssizegb=10 clientcount=4 \
osscount=2 mdscount=2 mdtcount=2 austeroptions=-R
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Reviewed-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
HPE-bug-id: LUS-10912
Change-Id: I67d98f633de4023a4430b55c6b4d308c7f17d988
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49579
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Thu, 5 Jan 2023 22:44:50 +0000 (15:44 -0700)]
LU-2771 ldlm: remove obsolete LDLM_FL_SERVER_LOCK
The LDLM_FL_SERVER_LOCK flag and accompanying accessor macros have
never been used since they were first introduced. Remove them.
It looks like this may have been duplicated by LDLM_FL_NS_SRV.
Test-Parameters: trivial
Fixes:
caa55aec4a ("LU-2771 dlmlock: compress out unused space")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iffc9b126334a327a9054f9acae86f4a0d03ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49563
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Thu, 5 Jan 2023 17:59:24 +0000 (12:59 -0500)]
LU-13642 lnet: modify lnet_inetdev to work with large NIDS
Change li_ipv6 field in struct lnet_inetdev to li_size which
now represents the size of the NID address. This will work
with the GUID of Inifiniband as well. Second change is
to store li_ipaddr always in network format. This will allow
direct comparsion between li_ipaddr and the nid_addr of
struct lnet_nid. We will ensure AF_IB will also be in the
same format as what will be stored in struct lnet_nid.
Implement setup with a NID address for the ko2iblnd LND driver.
Test-Parameters: trivial testlist=sanity-lnet
Change-Id: I7c27edb67263dd5bda4728c536aee266d38a4592
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49525
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Sat, 17 Dec 2022 13:06:16 +0000 (08:06 -0500)]
LU-16380 osd-ldiskfs: race in OI mapping
There is race in OI scrub thread and OI mapping entry insertion, which
may add an inconsistent OI mapping entry, but not started OI scrub
thread. This may lead to osd_fid_lookup() always returns -EINPROGRESS.
To avoid such race, osd_fid_lookup() returns -EINPROGRESS only when
OI mapping is inconsistent, and OI scrub thread is not running.
Fixes:
558784caad ("LU-15643 osd-ldiskfs: don't trigger scrub on irreparable FIDs")
Test-Parameters: mdscount=2 mdtcount=4 testlist=conf-sanity env=ONLY=108b,ONLY_REPEAT=50
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I05114b6a33940c210e9952f6e24f6c36fd7f76a2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49514
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Wed, 7 Dec 2022 02:53:25 +0000 (21:53 -0500)]
LU-16335 mdt: skip target check for rm_entry
For "lfs rm_entry", target may not exist, sanity check of it may fail
thus causes rm_entry fail.
Add sanity 832.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I824c7581af05c7494cf03c0c9bc999ca1abfec01
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49329
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log