Whamcloud - gitweb
Qian Yingjin [Fri, 13 Oct 2023 02:49:34 +0000 (22:49 -0400)]
LU-17190 osc: client-side high prio I/O under blocking AST
We found the following deadlock with parallel DIO:
T1: writer
Obtain DLM extent lock: L1=<PW, [0, EOF]>
T2: parallel DIO reader
- read 50M data, iosize=64M, max_pages_per_rpc=1024(4M)
max_rpcs_in_flight=8
ll_direct_IO_impl()
- use out all available RPC slots: number of read RPC in flight: 9
- OST side
->tgt_brw_read()
->tgt_brw_lock()
- Server side locking. Try to cancel the conflict lock L1.
T3: reader
- Take DLM lock ref on L1=<PW, [0, EOF]>.
->ll_readpage()
->ll_io_read_page()
->cl_io_submit_rw()
- wait fro RPC slots to send the read RPC to OST
...
deadlock:
- T2 => T3: T2 is waiting for T3 to release DLM lock L1;
- T3 => T2: T3 is waiting for T2 finished to free RPC slots;
To solve this problem, we introduce a client-side high priority
I/O handling mechanism where the extent lock protecting the I/O is
under blocking AST.
It implements as follows:
When received a lock blocking AST and the corresponding lock is in
use (reader and writer count is not zero), it checks whether there
are any I/O (osc_extent) used this lock is outstanding (i.e. wait
for RPC slot). If found, make this kind of I/Os with high
priority and put them into the HP list of the client. Thus the
client will force to send HP I/Os even the available RPC slots
are used out.
This makes I/O engine on OSC layer more efficent.
For normal I/Os, the client needs to iterate over the object list
and send I/O one by one. Moreover, the in-flight I/Os can not
exceed the @max_rpcs_in_flight.
High priority I/Os are put into HP list and will be handled more
quickly. This can avoid possible deadlock caused by parallel DIO
and the client can reponse the lock blocking AST more quickly.
Test-Parameters: testlist=sanity-pcc env=ONLY=99a,ONLY_REPEAT=100 clientdistro=el8.10
Test-Parameters: testlist=sanity-pcc env=ONLY=99a,ONLY_REPEAT=100 clientdistro=el9.3
Test-Parameters: testlist=sanity-pcc env=ONLY=99b,ONLY_REPEAT=100 clientdistro=el8.10
Test-Parameters: testlist=sanity-pcc env=ONLY=99b,ONLY_REPEAT=100 clientdistro=el9.3
Change-Id: I9afe032a79f40d55b800ddb13d8b8e9a3e10ba56
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56327
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Frederick Dilger [Fri, 30 Aug 2024 21:32:26 +0000 (15:32 -0600)]
LU-11048 docs: adding missing pool man pages
Adding lctl-pool_destroy.8, lctl-pool_list.8 and lctl-pool_remove.8.
Test-Parameters: trivial
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: I227a85b9703346a7bce4250670002c97c2d37cb8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56303
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Vladimir Saveliev [Tue, 27 Aug 2024 11:02:29 +0000 (14:02 +0300)]
LU-18172: lfsck: umount has to wait lfsck_stop
When called from umount lfsck_stop() should wait if lfsck is already
stopping. Otherwise, continuation of mdt_fini() or ofd_fini() leads to
various failures. Seen so far:
1.
osd_scrub_cleanup
LASSERT(dev->od_otable_it == NULL);
because lfsck_master_engine() has not reached yet
oit_iops->fini(env, oit_di)
osd_otable_it_fini
dev->od_otable_it = NULL;
2.
lfsck_find_mdt_idx_by_fid
rc = fld_server_lookup(env, ss->ss_server_fld...
BUG: unable to handle kernel NULL pointer dereference
because ss->ss_server_fld is NULL set on
mdt_fini
mdt_fld_fini
ss->ss_server_fld = NULL;
Test for umount while lfsck is stopping is added.
Test-Parameters: trivial testlist=sanity-lfsck env=ONLY=44
HPE-bug-id: LUS-12421
Signed-off-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Change-Id: I527c071d316ba3405f2199125fa7d018c98c403b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56165
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Emoly Liu [Tue, 3 Dec 2024 20:24:19 +0000 (04:24 +0800)]
LU-13720 lfs: resync file if needed before mirror_split()
If "lfs mirror delete" is called on a file to delete the only
non-stale mirror of the file, it makes sense from a usability
point of view to automatically resync the stale mirror(s) and
then remove the mirror requested by the user, rather than
returning an error to the user.
Also, sanity-flr.sh test_44a/b and test_211 are modified to verify
this patch. Man pages lfs-mirror-delete.1 and lfs-mirror-split.1
are updated to reflect this change.
Test-Parameters: testlist=sanity-flr env=ONLY="44,211"
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ice5ef893910e744d235dcb07f1049dfeede65858
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55978
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Maximilian Dilger [Tue, 6 Aug 2024 19:49:28 +0000 (13:49 -0600)]
LU-18109 utils: adding nodemap offset capability
nodemap offset: the ability to add an offset range to nodemaps.
When an offset is defined, idmaps can only be set inside of their
given range. This is defined as:
lctl nodemap_add_offset --name nodemap --offset 100000 --limit 99999
And removed as:
lctl nodemap_del_offset --name nodemap
Test sanity-sec 27ab is added to exercise this new feature.
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: Iba2116d21bc7de1ba03111b0313427301e4b0b50
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55943
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Fri, 2 Aug 2024 13:03:49 +0000 (15:03 +0200)]
LU-18087 enc: support encrypted names in changelogs
In order to support encrypted names in changelogs, we choose to have
the servers directly store the encoded+digested names in the
changelogs. This requires to add some knowledge of encrypted names on
server side, but that choice brings a number of benefits:
- as servers are storing changelog records, they have access to the
files' flags and can find out if files are encrypted or not. This
would not be possible on client side when reading changelogs,
because corresponding files might even no longer exist.
- no modifications are needed on client side, either in the kernel or
in userspace. As the API remains untouched, this is completely
transparent to applications that are consuming changelogs.
The file names retrieved from the changelogs are identical to the
names displayed when listing directories without the encryption key.
If names are not encrypted, they remain unchanged. If names are
encrypted, their digested+encoded form is presented.
Add sanity-sec test_73 to exercise this code.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: If0de4bfa210f9067a7a934ac74863a77b19482db
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55916
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sohei Koyama [Mon, 22 Apr 2024 00:55:17 +0000 (09:55 +0900)]
LU-928 misc: fix typos in comments
Fix typo obejct -> object.
Test-Parameters: trivial
Signed-off-by: Sohei Koyama <skoyama@ddn.com>
Change-Id: I4ab626ca188166e439191450b48e394243116bef
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54862
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Alex Zhuravlev [Mon, 1 Apr 2024 11:37:32 +0000 (14:37 +0300)]
LU-17668 ptlrpc: create env in few more threads
like evict thread, pinger, mount process. This avoids cases
where there is no lu_env found for a thread:
mdt_dom_lvbo_update()) ASSERTION( env ) failed
yet another case is when MDT destroys exports - we don't need to
re-allocate a new env (including thread infos for every layer) for
each export being destroyed.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iff7b757ad34063a3596a34998489cc3c635a02d2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54642
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Tue, 19 Apr 2022 00:32:47 +0000 (20:32 -0400)]
LU-10499 utils: add pcc pin/unpin commands
Store pin information in xattr trusted.pin.
Add lfs pcc pin/unpin commands to set/clear pcc pin flag.
A pcc-pinned file will not be removed from local storage
cache by lpcc_purge automatically.
Examples of xattr trusted.pin:
trusted.pin=[hsm: 2]
trusted.pin=[hsm: 2,hsm: 3]
EX-bug-id: EX-5113
Change-Id: I17ddf5ac8dc4eae48c0f6bdd0f2a19240474b0f5
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54460
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 20 Dec 2021 16:41:56 +0000 (17:41 +0100)]
LU-10499 sec: support of PCC-RO for encrypted files
In order to support PCC-RO for encrypted files, we decide to store
in PCC the ciphertext version of the Lustre files. We proceed to
decryption of PCC files only in the page cache, so cleartext is just
in memory. When a Lustre file is detached from PCC, or when the
encryption key is removed, we trash those PCC page cache pages.
As PCC files contain ciphertext, their sizes are aligned on
LUSTRE_ENCRYPTION_UNIT_SIZE instead of being lustre inode's clear
text size. In order to keep track of Lustre files' actual sizes,
we use a dedicated xattr on the PCC files. Its value is set at pcc
attach time, which is fine for PCC-RO.
Also add sanity-pcc test_21j to exercise this.
EX-5427 sec: fix pcc detach of encrypted file
In case of 'lfs pcc detach' of an encrypted file, its page cache
pages must be trashed as they might contain cipher text because
of the pcc file being detached.
Improve sanity-pcc test_21j to exercise this use case.
Was-Change-Id: Ice13c5b4205c074d9b46e2175d18f8743dbe9c58
EX-bug-id: EX-4182 EX-5427
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id7d96adb4eb4770c813a042acf7ed6c42224b9bf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54459
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Mon, 16 Dec 2024 16:55:24 +0000 (11:55 -0500)]
LU-14291 ofd: merge ost module into ofd
The ost module is very small and its very similar to the mds class
which is embedded in the mdt module. We can do the same thing with
the ost module. Merge it into the ofd module.
Test-Parameters: trivial
Change-Id: I1f7c44893d41ec8fc03113400a02b5076a07fbbe
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53993
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Cyril Bordage [Mon, 26 Aug 2024 12:05:41 +0000 (14:05 +0200)]
LU-16897 idl: add OBD_CONNECT2_SPARSE flag
This flag indicates when a client can handle sparse reads without
sending blank pages.
Test-Parameters: trivial
Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Change-Id: I5e510deaace86558990479351074ad8f47876212
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55404
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Thu, 12 Dec 2024 06:03:32 +0000 (01:03 -0500)]
LU-6142 contrib: update checkpatch.pl to 6.13-rc3
Update checkpatch.pl to the latest version. Notably, this version
drops the requirement for special comment style in net/. Lustre/LNET
will follow suit by also dropping this requirement.
Update the lustre-checkpatch.patch to match.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I6d9804965666e09196fb5f17b595914de1a5a109
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57388
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Tue, 10 Dec 2024 04:39:40 +0000 (23:39 -0500)]
LU-6142 obdclass: SPDX for OBD Class
Convert from verbose license text to SPDX.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ieaa62c9f4177abdeae18e55e5c8ce6dc638a8918
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57351
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Wed, 27 Nov 2024 09:11:13 +0000 (12:11 +0300)]
LU-17906 ptlrpc: reduce time for connection switch
If connection peer is not ready then reduce request
timeout for it to 1s, switching all imports connections
faster until some is ready
If connection status become ready after not-ready, try
it first to connect
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I87f2359f3a767ea9e52ce9da4cd5cf9b42b56320
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57234
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andrew Perepechko [Fri, 22 Nov 2024 17:05:36 +0000 (20:05 +0300)]
LU-18483 ldlm: serialize parallel lock res change requests
If parallel ldlm_lock_change_resource() calls race in a certain
way, the old resource can be released multiple times leading to
memory corruption.
The fix detects such race condtions after locking and restarts
the replace procedure if needed.
Change-Id: I742a8dde0ab4c63517c02c2e8e1bdfe402b331b4
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
HPE-bug-id: LUS-12212
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57110
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 2 Oct 2024 05:00:36 +0000 (12:00 +0700)]
LU-18224 o2ib: normalize in-kernel-ko2iblnd.ko name
in-kernel-ko2iblnd module should be named as ko2iblnd.
Also make it easier to identify in-kernel vs external
ko2ibldn driver with modinfo.
Module version should also include the ofed version provided
by ofed_info -n if it is available.
Test-Parameters: trivial
Fixes:
8b1d2a72f1 ("LU-16967 build: Add in-kernel-ko2iblnd driver")
HPE-bug-id: LUS-12525
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iab1c0502724a3bcf91ef7c62e7ae9ee23f2ccff6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56366
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Caleb Carlson <caleb.carlson@hpe.com>
Alexander Boyko [Thu, 26 Sep 2024 22:40:50 +0000 (18:40 -0400)]
LU-18218 llog: catalog lgh_lock refactoring
-call down_read for list processing, and don't use it for
single pointer reading.
-pass catalog flags to plain log.
llog_osd_prev_block() does not read from the end of file to
the beggining, and duplicates llog_osd_next_block.
tail->lrh_len was used without swabbing, it is an error.
Fixes llog_client error processing, according to llog_osd_next_block
EIO -> EBADR change.
It fixes test_135 race where only one record at llog. A cancel
happens before ENOSPC, a llog stays empty at catalog and could not be
deleted.
Fixes:
1a24dcdce121 ("LU-15938 lod: prevent endless retry in recovery thread")
HPE-bug-id: LUS-11970
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I353b3f291f2de65924f90650a400333a747a4f74
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56341
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sun, 8 Sep 2024 15:56:00 +0000 (11:56 -0400)]
LU-6142 libcfs: SPDX for libcfs headers
Convert from verbose license text to SPDX.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I84d166e5d37b1d29d968ca21683cc4047bde166d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56298
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Mon, 24 Jun 2024 06:18:51 +0000 (14:18 +0800)]
LU-17970 ldlm: add lock match flag LDLM_MATCH_SKIP_UNUSED
Add lock match flag LDLM_MATCH_SKIP_UNUSED to skip match unused lock.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I31d8f051b837a5b00cee6f2bc1ad9782acc62892
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55508
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Wed, 5 Jun 2024 13:46:27 +0000 (21:46 +0800)]
LU-17908 layout: preserve non-FLR state layout flags
lfsck and layout split/merge would set FLR state but ditch non-FLR
flags in error, this patch preserves them.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I7a8717756fba09ba10635bb3fbeed6afb4ed73a8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55319
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Tue, 27 Feb 2024 04:05:35 +0000 (15:05 +1100)]
LU-17538 fid: do not use allocation set for ofd
The sequence server on ofd is allocating new sequences
to osp, the lowater/hiwater allocation sets could
result in a gap of 2000 sequences after restaring the cluster.
Change-Id: I038a9b6add9977c58b58de3aa1f4d0926063bcb2
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54191
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Mon, 11 Nov 2024 16:54:33 +0000 (11:54 -0500)]
LU-13419 osc: Remove unnecessary spinlock assertion
This spinlock assertion notably increases the time spent in
getting grant with many threads to few OSCs, but adds
little value. Remove it.
Test-Parameters: trivial
Signed-off-by: Patrick Farrell <paf0187@gmail.com>
Change-Id: I596b9ac349a6dd1a633c6671d19d15bbf61b9fa2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56957
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Wed, 4 Dec 2024 09:37:16 +0000 (04:37 -0500)]
LU-12029 utils: remove max_sectors_kb tuning
Remove max_sectors_kb tuning and make the mount option print a warning
message that it is ignored.
Remove conf-sanity test_125 which is to verify max_sectors_kb tuning.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I5d11310589f70716df98101c62fd459dafbfddca
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57278
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Tue, 11 Jun 2024 09:17:02 +0000 (17:17 +0800)]
LU-17923 lod: update layout ver for mirror merge/split
In mirror extend/split, mdd_xattr_merge() and mdd_xattr_split() will
call mdd_object_pfid_replace() to update filter_fid for OST objects,
so we can update its layout version there.
This patch adds filter_fid::ff_layout_version in
lod_obj_stripe_replace_parent_fid_cb().
This patch adds ost_fid2_objpath() in test framework to derive OST
object path from its fid.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I68aca0e1f29c8e2800a28cfd3d630721ef869082
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55388
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sohei Koyama [Sun, 8 Dec 2024 15:09:35 +0000 (00:09 +0900)]
LU-18522 utils: fix -export-dynamic flag
When compiled with clang it fails with
`unknown argument: '-export-dynamic'`.
'-export-dynamic' is supported by gcc but not by clang.
Change to '-Wl,--export-dynamic' instead, which both supports.
-Wl, specifies the flags to be passed to the linker.
Signed-off-by: Sohei Koyama <skoyama@ddn.com>
Change-Id: I35830c0fec5387d532252478a402735ba2ad2dbf
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57334
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Sohei Koyama [Fri, 6 Dec 2024 04:12:39 +0000 (13:12 +0900)]
LU-18517 quota: fix LC_HAVE_DQUOT_QC_DQBLK check
When compiling with clang, the HAVE_DQUOT_QC_DQBLK check always
fails with "error: indirection of non-volatile null pointer will
be deleted, not trap".
Signed-off-by: Sohei Koyama <skoyama@ddn.com>
Change-Id: I8cbdfd70bef8164e40d19d68dc4127c6f40bd693
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57320
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Emoly Liu [Wed, 4 Dec 2024 21:38:40 +0000 (05:38 +0800)]
LU-16318 tests: wait longer for MDT destroys to complete
Wait longer for MDT destroys to complete in replay-ost-single.sh
test_6.
Test-Parameters: trivial testlist=replay-ost-single env=ONLY=6,ONLY_REPEAT=100
Test-Parameters: trivial testlist=replay-ost-single env=ONLY=6,ONLY_REPEAT=100
Test-Parameters: trivial testlist=replay-ost-single env=ONLY=6,ONLY_REPEAT=100
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I2668028e2ebecddb9167c4fed5963dda857446eb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57290
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 3 Dec 2024 11:10:26 +0000 (12:10 +0100)]
LU-18497 gss: carry out creds prepare as user
Instead of forking a child process that switches to user identity to
get the credentials cache name, just carry out the whole credentials
preparation as the user, and switch back to original uid/gid in order
to proceed to ioctls, eliminating the need to map memory between
processes.
Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I7807263d71cd3fc8a9934cc4c4da8b497b845e6f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57192
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Thu, 28 Nov 2024 02:49:03 +0000 (19:49 -0700)]
LU-17251 tests: increase parallel-scale OST seq_width
Increase seq_width earlier in parallel-scale so that OST SEQ rollover
does not happen during later testing (in particular test_rr_alloc).
Otherwise, the OST SEQ rollover can stall OST object preallocation
during the testing and provide skewed creation rates across OSTs.
Fix parallel-scale.sh so only tests which use MPI check MPI_USER.
Test-Parameters: trivial
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Test-Parameters: testlist=parallel-scale env=ONLY=statahead+rr_alloc
Fixes:
11d55259ab ("LU-17251 tests: try to fix test_rr_alloc again")
Fixes:
66d93ce3e4 ("LU-17251 test: improve parallel-scale rr_alloc test")
Fixes:
0ecb2a167c ("LU-11912 ofd: reduce LUSTRE_DATA_SEQ_MAX_WIDTH")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I79b146fff96dcc3c607e73bbb575c526d02540e5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57181
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 3 Oct 2019 20:40:36 +0000 (14:40 -0600)]
LU-930 misc: update .mailmap to fix bad commits
Fix the .mailmap file to fix the attribution of a few commits
that used the wrong authorship.
Test-Parameters: trivial
Build-Parameters: distro=el9.3 arch=x86_64
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Idd45f46fcefa7d9aaa9e2b6a63ccaa1b233ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57008
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Keguang Xu [Fri, 8 Nov 2024 01:35:00 +0000 (09:35 +0800)]
LU-18244 utils: add 'lfs mkdir -C -N' overstriping support
Adding support of "-N" in "lfs mkdir -C -N" which specify the
overstripe factor for each MDT. While meta-overstriping has already
been introduced (refer LU-12273), this change mainly aligns the
mkdir -C -N with the overstriping behavior OSTs.
- Util side, add range check [-5, -1] if overstriped enabled within
lfs_setdirstripe().
- Server side, in lod_object.c::lod_ah_init() where the decision of
striping configuration is made, we adjust the ldo_dir_stripe_count
value account for the specified overstripe factor.
- Macro LMV_OVERSTRIPE_COUNT_{MIN, MAX} added indicating the overstipe
count for each MDT, in -N format.
Signed-off-by: Keguang Xu <squalfof@gmail.com>
Change-Id: I99bdd543c1482798441d8753f4aeb2b7190060a8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56925
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Wed, 30 Oct 2024 06:52:34 +0000 (02:52 -0400)]
LU-18420 llite: Use enum mds_ibits_locks instead of __u64
This patch uses enum mds_ibits_locks instead of generic
__u64. This patch has no functional changes.
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I2d9ad5001977a532f2d7c3242ce2e40f38974c5e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56877
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Fri, 25 Oct 2024 19:20:16 +0000 (13:20 -0600)]
LU-8066 mdc: migrate simple proc entries to sysfs
Two simple proc entries dom_min_repsize and lsom can be moved
to sys/fs.
Change-Id: Ieb8af26b1936b8b3579ceb6bcc113c56444eec45
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56849
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Keguang Xu [Tue, 15 Oct 2024 06:28:58 +0000 (14:28 +0800)]
LU-17309 llog: set timestamp on llog objects at creation time
To facilitate debugging, this commit introduces the storage of the
creation time (ctime) for llog objects upon creation. Additionally,
the modification time (mtime) will be updated when the catalog is
modified.
- For llog creation, la_ctime/mtime/atime are set within
llog_osd_create().
- For update, there are 2 entries leading to the modification of
llog: 1) direct usage over plain-log?, dt_attr_set() is added with
la_mtime set in llog.c; 2) update through catalog, dt_attr_set() is
added in llog_cat.c
Performance Considerations:
- Catalog modifications are infrequent, minimizing the performance
impact.
- Updating the mtime incurs no additional overhead when new records
are added to an llog file, as the new inode size/ blocks must be
written regardless.
Signed-off-by: Keguang Xu <squalfof@gmail.com>
Change-Id: I13c260cc576e35a811a9d33cacfdd0bdf8492962
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56691
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Mon, 14 Oct 2024 15:31:34 +0000 (11:31 -0400)]
LU-8066 sysfs: migrate checksum_type to sys/fs tree
The checksum_type file is simple so the logical place is sys/fs
which is accessible to everyone. This more complex file is
one of thew few allowed in the general sysfs tree.
Change-Id: I6ce89416ef64058f911e18a3f743944f0a6a8f4e
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56667
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Fri, 11 Oct 2024 18:22:23 +0000 (14:22 -0400)]
LU-8066 sysfs: migrate pinger_recov to sys/fs tree
The pinger_recov file is simple so the logical place is sys/fs
which is accessible to everyone.
Change-Id: I1117c4bd54aa4941f3ff66acfca422cc83a54088
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56666
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Fri, 11 Oct 2024 14:24:00 +0000 (08:24 -0600)]
LU-8066 sysfs: migrate max_pages_per_rpc to sysfs tree
Both the osc and mdc layer place max_pages_per_rpc in the proc
tree. We can move it to the sysfs tree. This is a simple sysfs
file so no root only issues for this tunable.
Change-Id: Ib4c9bd5e3b6a56c489149779208a1461692538ac
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56665
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 6 Dec 2024 00:11:50 +0000 (07:11 +0700)]
LU-18351 obdclass: jobstat scaling
When there is a large number of jobstats (>100k) list walking in
seq_write does not scale and can trigger cpus soft lock-ups.
Prefer red black trees for easier lookup, removal and simplified
tree walk during seq_write()
Add an rbtree with a generated sequential id as jobs are added
to simplify iteration with restarts. The larger the amount of
data being written via seq_write() the more likely and frequent
a restart is needed to seek to new position which can be done
quickly with an rbtree.
Add an lru list to so periodic trim of jobs does not need to
walk the entire tree of jobs.
Unpatched:
jobs 50000 100000 200000 350000 500000
time 2.096s 3.672s 7.504s 11.875s 16.721s
Patched:
jobs 50000 100000 200000 350000 500000
time 1.366s 3.296s 6.763s 8.895s 11.965s
HPE-bug-id: LUS-12621
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Icca1365af5db761ed89ee9a8a97ce4ded65b8832
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56607
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Mon, 7 Oct 2024 11:48:28 +0000 (07:48 -0400)]
LU-18337 tests: Fix racer division by 0 error message
'file' under file_create.sh is set as /mnt/lustre/racer/M/N
Currently M/N is used directly as mathematical operation
to determine whether "--block" (default) is to be passed as
option to 'migrate'. This was causing the divide by zero error
when N is 0. This patch changes 'file' to use only the last
part (ie N) instead of M/N when doing the mod by 8 operation.
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I590250afb69607d906cf3f8422ca4bc97600eabb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <adeiter@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Elena Gryaznova [Fri, 27 Sep 2024 10:49:12 +0000 (13:49 +0300)]
LU-18274 tests: reboot the not up nodes only
It is reasonable to reboot only those nodes which are
not up after previous reboot attempt.
Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-12442
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I3e1181f10bdf8b7e74ab532bd2e77d092dcf1c9b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56515
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Mon, 23 Sep 2024 08:08:20 +0000 (16:08 +0800)]
LU-17692 llite: flock work with lockd
Change the flock to invoke lock copy API to work
with nfs server.
Fixes:
7f8af8f37e ("LU-17692 flock: get extra reference for lockd")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ic5b3daf825dfa616a394c47a76f0d08e87c5bc9a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56488
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sat, 21 Sep 2024 03:29:56 +0000 (10:29 +0700)]
LU-18250 ldiskfs: refresh Ubuntu 24.04 6.8.0-45-generic
A recent kernel update needs to use newer patches from 6.10:
ext4-prealloc.patch
ext4-delayed-iput.patch
Also add a kernel patch level check for Ubuntu 6.8 in to
select pick the newer series and fix a typo in the 5.11 check
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0fffb37fc29c1747d6aa7833111116095c55b666
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56445
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sun, 15 Sep 2024 10:18:25 +0000 (06:18 -0400)]
LU-16796 lnet: Change struct lmv_stripe_object to use kref
This patch changes struct lmv_stripe_object to use
kref instead of atomic_t
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I7df086a23f59eb352c0f07b96e30bfdf9bc96cad
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56363
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Etienne AUJAMES [Wed, 11 Sep 2024 18:31:37 +0000 (20:31 +0200)]
LU-18163 obdclass: add sysfs_memparse_total() string helper
Add sysfs_memparse_total() to parse '%' unit. The size returned is
computed based on a total size argument.
If the parsed size exceeds the total value (> 100% of total size),
the function returns -ERANGE.
The following string format are accepted: "1.0%", "0.1%", ".1%"
It adds some unit tests for sysfs_memparse_total()
in obd_init_checks().
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ic9390f2dbb64f0c2ca8756e78ba4100346e08f3f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56334
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sat, 24 Aug 2024 07:08:10 +0000 (03:08 -0400)]
LU-16796 lnet: Change struct lnet_ping_buffer to use kref
This patch changes struct lnet_ping_buffer to use
kref instead of atomic_t
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ic180e576c3b11afec687dcb3b55b857c730598f6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56177
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sat, 31 Aug 2024 03:40:43 +0000 (10:40 +0700)]
LU-17000 mdt: symbolic link integer overflow
CoverityID: 442376 ("Integer overflow")
Use ssize_t to avoid narrowing
CoverityID: 442114 ("Integer handling issues")
mdt_getattr_internal() a theoretical underflow when subtracting
from an unsigned int that could be 0
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I560c9cbcbfed907725b101ca90661fe8d0d6cf45
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56151
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Arshad Hussain [Fri, 23 Aug 2024 18:13:18 +0000 (14:13 -0400)]
LU-16796 lnet: Change struct srpc_client_rpc to use kref
This patch changes struct lov_stripe_md to use
kref instead of atomic_t
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I7dafb469aa6ad95e810f895654588a9a2114c7c6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56142
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Fri, 23 Aug 2024 19:02:26 +0000 (15:02 -0400)]
LU-16796 lfsck: Change struct lfsck_tgt_desc to use kref
This patch changes struct lfsck_tgt_desc to use
kref instead of atomic_t
Test-Parameters: trivial testlist=sanity-lfsck
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I1f9bf1889fcc6f0a4b4050c1a2be98476c5254dd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56141
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Fri, 23 Aug 2024 14:09:41 +0000 (10:09 -0400)]
LU-16796 lov: Change struct lov_stripe_md to use kref
This patch changes struct lov_stripe_md to use
kref instead of atomic_t
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I8c743d630c268a70ecd0d5c9907e2ce456cd60cc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56104
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 25 Nov 2024 11:39:23 +0000 (18:39 +0700)]
LU-18486 build: fl_flags removed in 6.8
Linux commit v6.7-rc4-79-gaf7628d6ec19
fs: convert error_remove_page to error_remove_folio
Provide an abstraction for fl_flags and c.flc_flags
Make mdt_dom_fiemap a static function.
Test-Parameters: trivial
Fixes:
3177b0dc5d1 ("LU-2525 ldlm: add asynchronous flocks")
Fixes:
5921e1571f2 ("LU-14510 dom: fiemap support for DoM files")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Icb659a8ab19c00454391cbb7df31c2bb7f9a85c4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57123
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Thu, 14 Dec 2023 06:28:30 +0000 (11:58 +0530)]
LU-16796 lnet: Change struct lnet_delay_rule to use kref
This patch changes struct lnet_delay_rule to use
kref instead of atomic_t
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I58bf5a27d779e20bc4734765980e83675ea53633
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53457
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Zarochentsev [Mon, 2 Dec 2024 16:47:57 +0000 (16:47 +0000)]
LU-17810 dne: dir restripe without fixed hash flag
Add an option to clear LMV_HASH_FIXED_FLAG
in dir re-striping, this restores an ability to convert a
striped dir with only one stripe into a plain dir.
HPE-bug-id: LUS-12321
Fixes:
4c2514f483 ("LU-14459 mdt: support fixed directory layout")
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: I7c119f435b19a446cb960cdc5a8ebec7e00ab0dc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57117
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alexander Boyko [Wed, 6 Nov 2024 03:20:36 +0000 (22:20 -0500)]
LU-18463 ptlrpc: removing cfs_flush_fput idle
mdtest performance degradations were observed for directory removes
of up to 28%. The cause is LU-16973.
The fix removes flushing from a ptlrpc path, and introduce a
work for osd ldiskfs. Let's flush files base on count. Also it
adds the sysfs file to set a descriptors count for flushing.
By default 5k files.
echo 1000 > /sys/fs/lustre/osd-ldiskfs/flush_descriptors_cnt
cat /sys/fs/lustre/osd-ldiskfs/flush_descriptors_cnt
1000
mdtest results with this patch (5 iterations Mean op/s)
base with a fix
MDT0
Directory creation 120447 111808
Directory stat 394309 388488
Directory removal 123516 169907
MDT1
Directory creation 123997 121788
Directory stat 403213 395777
Directory removal 116210 160593
HPE-bug-id: LUS-12373
Fixes:
2feb4a7bb0 ("LU-16973 ptlrpc: flush delayed file desc if idle")
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I26c73f09f7c8045a26a7876317e07b4cd28bcee3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57073
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sun, 22 Sep 2024 14:08:30 +0000 (10:08 -0400)]
LU-6142 mdd: Fix style issues for md_object.h
This patch fixes issues reported by checkpatch
for both files lustre/include/md_object.h
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Id28dc3451cb3c79335684b5aa2dd5f821eba4f36
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56478
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sun, 22 Sep 2024 14:04:48 +0000 (10:04 -0400)]
LU-6142 uapi: Fix style issues for lustre_user.h
This patch fixes issues reported by checkpatch
for both files lustre/include/uapi/linux/lustre/lustre_user.h
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I046ba401498c99e72915e504c715987fbf4267e7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sun, 22 Sep 2024 10:12:47 +0000 (06:12 -0400)]
LU-6142 gss: Fix style issues for lsupport.c
This patch fixes issues reported by checkpatch
for both files lustre/utils/gss/lsupport.c
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I1953e34226526ecde8129d419ee20dce0c9d3f15
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56455
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sun, 8 Sep 2024 16:31:57 +0000 (12:31 -0400)]
LU-6142 lnet: Fix style issues for list.h
This patch fixes issues reported by checkpatch
for both files libcfs/include/libcfs/util/list.h
Update inline doc to kernel-doc style
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ic8bef64aa24d079b2242ec9ac63ee7b38726f714
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56306
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sun, 8 Sep 2024 14:56:51 +0000 (10:56 -0400)]
LU-6142 libcfs: Fix style issues for libcfs_hash.h
This patch fixes issues reported by checkpatch
for file libcfs/include/libcfs/libcfs_hash.h
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I0826a6ea530b2e8547d52c4123455b6da3d02b46
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56302
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sat, 7 Sep 2024 13:02:36 +0000 (09:02 -0400)]
LU-6142 misc: Fix style issues for lustre_req_layout.h
This patch fixes issues reported by checkpatch
for file lustre/include/lustre_req_layout.h
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I7b2595244f8c4cf3a90a28d447612fe9ea9df6f5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56301
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Fri, 6 Sep 2024 15:33:41 +0000 (11:33 -0400)]
LU-6142 obd: Fix style issues for obd.h
This patch fixes issues reported by checkpatch
for file lustre/include/obd.h
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I98b0f4e3b5849d2c917c6236948770c94a497c41
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56299
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Wed, 11 Sep 2024 16:42:24 +0000 (10:42 -0600)]
LU-14288 nodemap: Use nidmasks for IPv6 NIDs
Using nidmasks for nodemap ranges allows us to specify more than one
IPv6 NID in a range. In range_create(), we construct a string
representing a nidmask based on the specified NID and netmask. This
is passed to cfs_parse_nidlist(), and the resulting nidlist is stored
in lu_nid_range::rn_nidlist. cfs_match_nid() is used in range_find()
and range_search() to locate appropriate ranges.
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=FORCE_LARGE_NID=true,LOAD_MODULES_REMOTE=true
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ib1f60eda6fbc9d0214b6c63dcc8656eab3977a8b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Fri, 29 Nov 2024 22:27:04 +0000 (17:27 -0500)]
LU-18500 mdc: rename mdc_get_unused
It doesn't 'get' unused - it cancels them.
Actually, it cancels unused conflicting locks, but that
made the name too long and hopefully it's clear from the
signature.
Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3f01f79a63069dfb019667ff4ca0d427e1699511
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57200
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Tue, 26 Nov 2024 08:07:48 +0000 (16:07 +0800)]
LU-18488 utils: make lrc_synced usage intuitive
llapi_resync_comp::lrc_synced is used against intuitiveness in the
mirror resync code, this patch corrects it.
Test-Parameters: testlist=sanity-flr,sanity-pfl,sanity-flr
Test-Parameters: testlist=sanity-pfl,sanity-flr,sanity-pfl
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I844dd3019b32b2a3746eb3b5b14c498ad7455270
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57137
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Wed, 20 Nov 2024 17:08:15 +0000 (20:08 +0300)]
LU-12706 tests: sanity-quota_4a fix
Don't set force_sync for OST0000 as we are
checking inode limit that is not affected
by the OST usage. Forced sync at OST0000
might take some time to be finished and break
the logic of a test case.
Createmany creates files in a directory on MDT0,
so don't sync all mdts, only MDT0.
Test-Parameters: testlist=sanity-quota env=ONLY=4a,ONLY_REPEAT=200
Fixes:
b86c88383f ("LU-12706 tests: sanity-quota 4a sync timeout fix")
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: If411429d8a1e968ebd554242db4b47b0c9421bf6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57088
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 8 Nov 2024 23:35:55 +0000 (16:35 -0700)]
LU-17165 tests: stable count in recovery-small/141
The lock cancellation and fetching the lock count on the OST
is racy and can randomly be "0" or "23" either before or after
the OST is restarted. Wait until the count has stabilized to
ensure the test can pass consistently.
Test-Parameters: trivial testlist=recovery-small
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib6d6fdeb721d6ff85a366e58c621ed7b883ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56945
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Etienne AUJAMES [Fri, 8 Nov 2024 15:47:12 +0000 (16:47 +0100)]
LU-17792 tests: fix replay-single 135 "not in REPLAY_LOCKS"
Fix test replay-single 135:
Error: 'import is not in REPLAY_LOCKS state
Wrong OST was selected for object creation due to ZFS replay-barrier:
it set the OSD to read-only and then the MDT QOS discard it.
Test-Parameters: trivial
Test-Parameters: fstype=zfs testlist=replay-single
Test-Parameters: fstype=zfs testlist=replay-single
Test-Parameters: fstype=ldiskfs testlist=replay-single
Test-Parameters: fstype=ldiskfs testlist=replay-single
Test-Parameters: fstype=zfs testlist=replay-single env=ONLY=135,ONLY_REPEAT=150
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ia5e1049c1fb75d99f669e38b2ac55ea6de116ddd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56935
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Alex Deiter <adeiter@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 8 Nov 2024 10:42:01 +0000 (17:42 +0700)]
LU-18433 llite: remove unused ci_target_is_zfs check
The check is not longer used and can safely removed.
HPE-bug-id: LUS-12597
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If3cb2a11b773f19f53de905be84ad32a92bb47c9
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56932
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Tue, 5 Nov 2024 21:01:14 +0000 (14:01 -0700)]
LU-17471 osd-ldiskfs: brw_stats symlink module option
Add a module option to symlink /proc/fs/lustre/osd-*/*/brw_stats
to /sys/kernel/debug/*/*/brw_stats for backward compatibility
for systems that need this for monitoring tools that depend on the
old /proc pathname instead of always creating it at startup.
Otherwise, "lctl get_param osd-ldiskfs.*.brw_stats" will display
the stats for both the original and symlinked parameter file,
which can confuse other monitoring tools.
To enable the /proc/.../brw_stats symlink, add the following line
into /etc/modprobe.d/lustre.conf before loading modules/mounting:
options osd_ldiskfs symlink_brw_stats=1
By default the symlink is not created, since newer tools should
be using the /sys/kernel/debug/.../brw_stats parameter file.
Remove sanity test_0f that is verifying the old parameter path,
since it is just too messy to get the interoperability correct.
Test-Parameters: trivial
Fixes:
47ccacd58d ("LU-17471 osd: add symlink for brw_stats")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib61630cf2bc89b84af438c811754a85e403ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56917
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Thu, 7 Nov 2024 17:00:46 +0000 (01:00 +0800)]
LU-13305 osc: rid of an invalid assertion
When osc_cache_truncate_start() race with ldlm lock blocking ast,
they would access the same extent, and the ext would has oe_hp
set by osc_cache_writeback_range() from the data flush caused by
ldlm lock canceling.
So the EASSERT(!ext->oe_hp && !ext->oe_urgent, ext) is invalid in
this case.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I15514c54af598f2a130c0ac0a3f356468b4cdfce
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56914
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Wed, 6 Nov 2024 18:26:57 +0000 (21:26 +0300)]
LU-18427 script: allow llog removal scripts on ZFS
Make both scripts working also for ZFS mounts
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I713548caa2f11af334c7bd10c07ecc81c387f5e1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56906
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Olaf Faaland <faaland1@llnl.gov>
Andreas Dilger [Mon, 4 Nov 2024 21:59:27 +0000 (14:59 -0700)]
LU-17769 tests: print subtest iter in test banner
Print the subtest iteration number/duration in the subtest banner
so that it is more visible across all of the nodes. Otherwise, it
is only printed via "echo" into the full test script run log, and
is not shown in most of the per-subtest logs in Maloo.
Test-Parameters: trivial testlist=sanity env=ONLY=1,ONLY_REPEAT=10
Fixes:
e16e3d46 ("LU-13169 tests: add ONLY_REPEAT parameter to repeat subtests")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic13196e9bc17a51a1af7a0b5aecaafe3c5349a4c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56883
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Fri, 1 Nov 2024 06:55:36 +0000 (17:55 +1100)]
LU-18392 tests: hold group locks in recovery-small/160
recovery-small/160 starts 10 threads and holding group locks
for 20s.
The osp destroy retries could actually happen on ofd after the 20s
delay, and make those rpc waiting for OST commit while increase
destroys_in_flight.
Make sure we hold on to the group locks while checking for
destroys_in_flight, and add additional check to make sure
the object destroy actually done after releasing the group lock.
Do not cast the transno to 32bit unsigned in osp_sync_interpret(),
which could produce a confusing transno in debug log.
Test-Parameters: trivial testlist=recovery-small env=ONLY=160,ONLY_REPEAT=100
Fixes:
27f787daa7 ("LU-15737 ofd: don't block destroys")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I2a6ac9700a3e79e9930cee905c1da73da948ba1a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56846
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Etienne AUJAMES [Fri, 25 Oct 2024 10:10:55 +0000 (12:10 +0200)]
LU-18401 utils: check_iam, support more iam records types
This patch fix check_iam to support different keysizes (the entries
offset was wrong for keysize other than fid size).
This adds callbacks to print different type of records for different
IAM files.
The type can be specified with the option "-t". If not, the type is
guess from the file name.
If the type is not supported, they keys and records will be dump in
hexadecimal format.
Here the following type of IAM files supported:
- oi.16.*: FID/inode mapping (already supported):
$ check_iam -r /tmp/oi.16.1
[0x200000401:0x448:0x0] 1157/
3251524137
[0x200000401:0x449:0x0] 1158/
3251524138
...
- lfsck_namespace: LFSCK namespace trace file:
$ check_iam -r /tmp/lfsck_namespace_01
[0:0x0:0x0] 0x0 ()
[0x200000401:0x65:0x0] 0x1 (CHECK_LINKEA)
- lfsck_layout: LFSCK layout trace file:
$ check_iam -r /tmp/lfsck_layout_01
{ parent: [0:0x0:0x0], comp_id: 0, ea_off: 0 } { cfid: [0:0x0:0x0],
ost_idx: 0 }
- nodemap: nodemap configuration:
$ check_iam -r /tmp/nodemap
{ id: 0x0, type: global(15) } { is_active: 1 }
{ id: 0x1, type: cluster(1), subtype: cluster } { name: toto, flag:
0xb8, flag2: 0x0, squash_uid: 65534, squash_gid: 65534, squash_projid:
65534}
{ id: 0x1, type: cluster(1), subtype: roles } { roles: 0xffffffc8}
{ id: 0x1, type: range(2) } { start_nid: 192.168.1.128@tcp,
end_nid: 192.168.1.130@tcp }
Update regression test: conf-sanity 134
Test-Parameters: trivial
Test-Parameters: testlist=conf-sanity
Test-Parameters: testlist=conf-sanity
Test-Parameters: testlist=conf-sanity env=ONLY=134,ONLY_REPEAT=20
Fixes:
99d1f12c7 ("LU-15581 utils: add check_iam util")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ic2cae2cbbf1c29daa661b1916d6321b6d87494dd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56790
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 14 Oct 2024 23:23:26 +0000 (17:23 -0600)]
LU-14520 ldlm: reduce ldlm_lock memory usage
Reduce the size of struct ldlm_lock so that it can fit into a 512-byte
slab allocation. The primary reduction in memory usage is from moving
struct l_ost_lvb into the union for IBITS locks where it is needed,
while it was previously part of the common strucutre. Add assertions
to verify that the l_ost_lvb field is only used for IBITS lock type.
Additionally, shrink some of the other fields in memory to bitfields
that only use the required bits, and pack them together to save space.
These are read-mostly fields for the lifetime of the lock and give
some space to add fields in the future before hitting 512 bytes again.
ldlm_lock BEFORE PATCHED
size: 536, members: 36 size: 496, members: 35
sum used: 536, holes: 0 sum used: 490, holes: 2, sum holes: 6
bit holes: 0 bit holes: 1, sum bit holes: 10
Also reduce struct ldlm_resource by a few bytes, so that 23 can fit
into a single 4096-byte slab instead of the previous 22 in one slab.
ldlm_resource BEFORE PATCHED
size: 184, members: 14 size: 176, members: 14
padding: 7, holes: 0 sum used: 171, holes: 1, sum holes: 5
bit holes: 0 bit holes: 1, sum bit holes: 4
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7f017a852228f8c9090e890f2766707fdbde2abd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56685
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sat, 9 Mar 2024 21:54:27 +0000 (14:54 -0700)]
LU-12597 tests: return comma-separated osts_nodes()
Start to return comma-separated OSTs list from osts_nodes(),
to avoid the redundant calling of comma_list() for each user.
Add the tgts_nodes() helper to print combined MDT and OST nodes to
avoid duplicate nodes vs. separate mdts_nodes() + osts_nodes().
Fix the few places that do not call comma_list() on osts_nodes()
output afterward, and code style in sanity-sec test_31.
Fix a minor bug in the error handling of sanity check_stats() where
it was calling "osts-nodes" instead of "osts_nodes".
Later patches will clean up all of the callers of osts_nodes,
but there are too many places it is used to do in a single patch.
Fixes:
f0324c5c2f ("LU-14992 tests: sanity/replay-vbr mkdir on MDT0")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7ae3278321e7552dc2afd5fbb9f48033af3ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56635
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Ronnie Sahlberg [Thu, 3 Oct 2024 03:31:45 +0000 (23:31 -0400)]
LU-4922 cfs_str2mask: Handle passing in a blank string better
Do not revert the flags to min_mask if passed an empty string
to the function.
Remove exclusion of the changelog_mask parameter from test_133g
as it no longer needs to be skipped.
The change initializes the new mask to the existing mask and only
re-sets it to use min_mask iff the first token in the string is a
'-' or '+'. This fixes a bug where if you were to pass an empty
string to this function it would reset the flags to min_mask.
With this change the flags will be left unchanged.
The only way curently for a user to invoke this function is via
the set_param param command.
That command does check for and will abort if the user tries to
specify "" before calling into cfs_str2mask() thus preventing the
issue in LU-4922 from triggering.
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: Ibe43509cbdec250e395e35648d399167026a1a14
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56619
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Ronnie Sahlberg [Thu, 19 Sep 2024 04:08:25 +0000 (00:08 -0400)]
LU-18231 obd: change obd_abort_mdt_recovery into OBDF_ABORT_MDT_RECOVERY
Change the use of the odb_abort_mdt_recovery bitfield into a bit in
the ofd_flags bitmap and use atomic bit set/clear/test
operations on it.
No logical changes to the code and no changes to locking.
Test-Parameters: trivial
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I827d8eb32b9dc79fb16d76f33bb7169abf228e8b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56417
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Ronnie Sahlberg [Thu, 19 Sep 2024 03:56:03 +0000 (23:56 -0400)]
LU-18231 obd: change obd_abort_recovery into OBDF_ABORT_RECOVERY
Change the use of the odb_abort_recovery bitfield into a bit in
the ofd_flags bitmap and use atomic bit set/clear/test
operations on it.
No logical changes to the code and no changes to locking.
Test-Parameters: trivial
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I1ff7f890a7ea67d2609ac3cb887a874e7ae6377b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56416
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Ronnie Sahlberg [Thu, 19 Sep 2024 03:45:24 +0000 (23:45 -0400)]
LU-18231 obd: change obd_recovering into OBDF_RECOVERING
Change the use of the odb_recovering bitfield into a bit in
the ofd_flags bitmap and use atomic bit set/clear/test
operations on it.
No logical changes to the code and no changes to locking.
Test-Parameters: trivial
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I031da297c4c6d0325851ddeac0768ea21d88e25f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56415
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Ronnie Sahlberg [Thu, 19 Sep 2024 02:48:46 +0000 (22:48 -0400)]
LU-18231 obd: change obd_set_up into OBDF_SET_UP
Change the use of the odb_set_up bitfield into a bit in
the ofd_flags bitmap and use atomic bit set/clear/test
operations on it.
No logical changes to the code and no changes to locking.
Test-Parameters: trivial
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I6b2b691e7cac733788fe239f9fc8ca5f8e749891
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56414
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Ronnie Sahlberg [Thu, 19 Sep 2024 02:38:43 +0000 (22:38 -0400)]
LU-18231 obd: change obd_attach into ODBF_ATTACH
Change the use of the odb_attach bitfield into a bit in
the ofd_flags bitmap and use atomic bit set/clear/test
operations on it.
No logical changes to the code and no changes to locking.
Test-Parameters: trivial
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I2f360f6074e2078fa20b770b5dc2abc173766c9f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56413
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Ronnie Sahlberg [Thu, 19 Sep 2024 02:24:42 +0000 (22:24 -0400)]
LU-18231 obd: Add a bitmap for the flags
We will use this bitmap and convert the current bitfields
into bits that are set/cleared/tested using atomic bit operations.
Signed-off-by: Ronnie Sahlberg <rsahlberg@whamcloud.com>
Change-Id: I59ee4621926d0bd79ed0187f6058b888e40315f4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56412
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Marc Vef [Wed, 18 Sep 2024 13:53:23 +0000 (15:53 +0200)]
LU-9936 utils: Improve ldiskfs_write_dd() writing mount data
Writing the mount data to a file used buffered stream I/O missing a
flush operation with incomplete error handling.
This patch replaces buffered stream I/O with the Unix I/O API and adds
error handling.
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: Ia7dfc186b1c6f581dee461bd277f07dcc2eabafa
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56408
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Marc Vef [Tue, 17 Sep 2024 00:08:58 +0000 (02:08 +0200)]
LU-10249 utils: Fix lustre_rsync program options
The lustre_rsync program options were incomplete and presented
incorrectly, e.g., the -r option.
This patch improves the program options. It adds a short and long
usage message in accordance to the Lustre documentation replacing the
previous usage message. Moreover, a check to specify the mdt device
was added as it was missing. Other small changes were added to improve
user interaction.
Signed-off-by: Marc Vef <mvef@whamcloud.com>
Change-Id: I871a218381b371e61f0b3a258a4773d3e755101e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56407
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Frederick Dilger [Thu, 12 Sep 2024 22:46:38 +0000 (16:46 -0600)]
LU-17501 libcfs: fix CPT NUMA core exclusion handling
The cfs_cpt_unset_node_core() function was only setting the offset
a single time for the first node, and using the same offset for all
other nodes.
cpu_pattern='C[0]' was only removing the first core on the first
NUMA node when it should be removing a core on every NUMA node.
It produced an output like:
cpu_partition_table=
0 : 1
1 : 2 3
2 : 4 5
when it should have been:
cpu_partition_table=
0 : 1
1 : 3
2 : 5
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: I22e1fe1aa73e413cf9c32562ec96f0cf4bd16fe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56347
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 6 Sep 2024 18:10:05 +0000 (12:10 -0600)]
LU-18210 utils: Fix issues with enhanced 'lfs check'
lfs check takes an optional path argument that should restrict its
output to the targets that are related to the specified file system.
To determine which MGC should be shown the NID embedded in the OBD
name is compared against a NID that is parsed from the mount command
line. There are a few problems with this code:
1. The NID is parsed from the mount command by looking for a ':', but
IPv6 NIDs can contain this character.
2. It doesn't handle case where multiple NIDs are listed on mount
command separated by commas. e.g. "1@kfi,2@kfi:3@kfi,4@kfi:/lustre"
will parse to "1@kfi,2@kfi" which is not a valid NID.
3. Even in the case where we have just two NIDs separated by ':', it
assumes that the first one will be the one we are connected to.
To resolve these issues get_root_path_slow() will now store the entire
NID string used at mount time (after it has been processed by
convert_hostnames()) in the struct root_cache::nid. do_target_check()
will then iterate over each NID in the string using a new nidstrings
function, cfs_nidstr_find_delimiter(). This function contains logic
that was formerly in convert_hostnames(). This should ensure that
each MGS NID is checked.
convert_hostnames() is updated to use cfs_nidstr_find_delimiter() as
well.
Fixes:
6301419509 ("LU-17367 utils: improved hostname and IPv6 NID support")
Fixes:
f5ca6853b8 ("LU-16076 utils: enhance 'lfs check' command")
Test-Parameters: testlist=sanityn env=ONLY=113,FORCE_LARGE_NID=true,LOAD_MODULES_REMOTE=true
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I478c66d77a2b241b910324210475d61b3786c986
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56286
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Maximilian Dilger <mdilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Sun, 29 Sep 2024 19:06:53 +0000 (15:06 -0400)]
LU-15420 llite: cleanup ll_lookup_it()
Newer kernels use fscrypt_prepare_new_inode() to enquire if the
inode is really encrypted. The current Lustre stack is not
setup for this. Much of the fscrypt handling happens at the
top of the basic VFS operation hooks. Rework the stack to
move this handling deeper into tha stack to the level of the
function ll_lookup_it(). This change also cleans up the code
greatly.
Change-Id: I803751759aa954403dd802f7277e870c1f9cd6da
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56233
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 6 Sep 2024 16:34:10 +0000 (23:34 +0700)]
LU-18184 llog: llog_process() with struct llog_process_data
BUG: KASAN: global-out-of-bounds in \
llog_process_or_fork+0xd0e/0xd60 [obdclass]
Calling llog_process 4th argument must be either NULL or
struct llog_process_data
Test-Parameters: trivial testlist=sanity env=ONLY="60a"
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iff4848365b5123df25d90811412a43307a115214
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56213
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Frederick Dilger [Thu, 29 Aug 2024 00:46:18 +0000 (18:46 -0600)]
LU-4315 doc: updating ls-tu man page style
Updating files to match the new code style for Lustre manual pages as
enforced by 'contrib/scripts/checkpatch-man.pl'.
This also includes other changes like removing < > or { } for singular
required arguments and placing [ ] around optional ones as well as
making all arguments CAPITAL and italicized, literal arguments are
bolded. Lines over 80 characters should be split at the natural line
end rather than the word that goes over the limit as fewer lines will
need to be modified when making changes if each sentence is on it's
own line.
Only using features that appear in groff 1.22.3 as this is the
available version is CentOS 8.
Checked files:
- lshowmount.8
- lustre.7
- lustreapi.7
- lustre_routes_config.8
- lustre_routes_conversion.8
- lustre_rsync.8
- mkfs.lustre.8
- mount.lustre.8
- mount.lustre_tgt.8
- nids.5
- plot-llstat.8
- routerstat.8
- tunefs.lustre.8
Test-Parameters: trivial
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: Ib932b582273ef567b48e504af2d5437389330dd3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56191
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Tue, 27 Aug 2024 11:36:22 +0000 (19:36 +0800)]
LU-18114 lctl: split "lctl net_delay_*" group into subcommands
Split "lctl net_delay_*" command group into subcommands, e.g.
"lctl net_delay_add" to "lctl net_delay add".
Also, sanity-lnet.sh is modified to verify this patch.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I64fa95155f93d8a5d844f9e14673acaea5f4ee60
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56164
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Tue, 27 Aug 2024 11:30:32 +0000 (19:30 +0800)]
LU-18114 lctl: split "lctl net_drop_*" group into subcommands
Split "lctl net_drop_*" command group into subcommands, e.g.
"lctl net_drop_add" to "lctl net_drop add".
Also, sanity-lnet.sh is modified to verify this patch.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I5094328435e5b93a8a7311f26ec77275211e0517
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56154
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Tue, 27 Aug 2024 11:26:08 +0000 (19:26 +0800)]
LU-18114 changelog: split "lctl changelog_*" group into subcommands
Split "lctl changelog_*" command group into subcommands, e.g.
"lctl changelog_register" to "lctl changelog register".
Also, test-framework is modified to verify this patch.
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ifd2c92ab3173507084342b7b575c2506e0e4bb9a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56133
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Frederick Dilger <fdilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Tue, 27 Aug 2024 11:22:02 +0000 (19:22 +0800)]
LU-18114 snapshot: split "lctl snapshot_*" group into subcommands
Split "lctl snapshot_*" command group into subcommands, e.g.
"lctl snapshot_create" to "lctl snapshot create".
Also, lsnapshot in test-framework.sh is modified to verify
this patch.
Test-Parameters: trivial fstype=zfs testlist=sanity-lsnapshot
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I30bf6f8a853f57bd1a14d5acb2b5aa4b3d49be71
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56120
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Frederick Dilger <fdilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Thu, 22 Aug 2024 03:07:06 +0000 (11:07 +0800)]
LU-18114 barrier: split "lctl barrier_*" group into subcommands
Split "lctl barrier_*" command group into subcommands, e.g.
"lctl barrier_freeze" to "lctl barrier freeze".
Also, sanity.sh test_801c is modified to verify this patch.
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ibccc0206bae6ef0fa8efd744b46c7f864aa42cbe
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56107
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Frederick Dilger <fdilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 20 Aug 2024 15:27:32 +0000 (17:27 +0200)]
LU-18158 sec: hint client in case of failed reint open
In case of failed file open, the Lustre client is already able to send
alternative supplementary groups to the server using a retry
mechanism. If the first attempt fails, then the failure reply from the
server includes a hint with a possible supplementary group and ACLs.
The client then retries with an alternative supplementary group,
selected using information hinted in the reply from the MDT.
The same mechanism can be implemented for all reint open cases.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic5e309cfbdb6d388397e3e251b2d007a612fd214
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/56098
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Frederick Dilger [Wed, 26 Jun 2024 22:16:56 +0000 (16:16 -0600)]
LU-17501 libcfs: allow CPT exclude list for cores
Allow a relative CPU core exclude list to be specified when
constructing the CPT distribution instead of having to specify all of
the CPU cores except the ones that should be avoided. The CPU core
numbers are the relative core numbers to each NUMA node, such that
for each node, the Mth to Nth cores are excluded. The exclude list
is set by specifying 'C' in /etc/modprobe.d/lustre.conf for the CPT
number, with either a comma-separated list or a range of CPU cores:
options libcfs cpu_pattern="N C[0-3]"
or:
options libcfs cpu_pattern="C[0,1,2,3]"
Both options are equivalent. This would exclude cores 0-3 on each NUMA
node from use by LNet and Lustre.
It isn't possible to specify both include and exclude lists at the
same time, but it doesn't really make sense to do this anyway.
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: Ief5f36e0bbc49865317cee199c91c9b4a350c418
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55544
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Wed, 19 Jun 2024 14:15:43 +0000 (16:15 +0200)]
LU-17961 sec: add server_upcall rbac role
The purpose of the new server_upcall rbac role is to control whether
clients use the server side defined identity upcall. When set, clients
do comply with the server side identity upcall. When not set, clients
are leveraging the special INTERNAL identity upcall, which means
servers trust supplementary groups as provided by the clients.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I01dcedad5da0e175aa7b8d187f2affd34d933e39
Was-Change-Id: I39a69904ce4709eacf6f08173d3cfe42e247b5bd
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55475
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 10 Feb 2022 16:14:02 +0000 (17:14 +0100)]
LU-17961 sec: support supplementary groups from client
The usual way to support more than 2 supplementary groups is to
resort to the server side's identity upcall. This identity upcall
retrieves all user's credentials, including all supplementary groups,
and stores them in cache. But this access to user's credentials from
server side is not always an option.
As an alternative to the server side's identity upcall, we implement
a retry mechanism for intent locking. The client can provide at most
2 supplementary groups in the request sent to the MDS, but sometimes
it does not know which ones are useful for credentials calculation on
server side. For instance in case of lookup, the client does not have
the child inode yet when it sends the intent lock request.
Hopefully, the server can hint at the useful groups, by putting in the
request reply the target inode's GID, and also its ACL. So, in case
the server replies -EACCES, we check the user's credentials against
those, and try again the intent lock request if we find a matching
supplementary group.
On server side, we add an INTERNAL dedicated, separate upcall cache.
This makes it distinct from the regular identity upcall cache that can
be defined to use any upcall including NONE, per an MDT side tuning.
It implements a particular behavior which does not involve an actual
upcall, but instead the cache is filled with supplementary groups read
from the client request, cumulatively at each request.
Dedicated mdt-side tunables are created to configure the entry expiry
time and the acquire expire time for INTERNAL, as well as a tunable to
flush the INTERNAL upcall cache.
A problem with the 'runas' utility was found during testing. If no
supplementary group is provided via the '-G' option, then it needs at
least to set the given GID as a supplementary group. Otherwise the
supplementary groups of the invoking user would be silently inherited.
For instance, if root user calls 'runas -u 500 -g 500 $CMD', we must
not execute $CMD with UID:GID 500:500 and supplementary group 0, as
it would make the user executing $CMD part of the superuser group.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4608bb766a70ca12a2142a0e2687813f3a4b9100
Was-Change-Id: I4b2298cf5c3b400e7a1436384653ce01a462a2e0
Was-Change-Id: I4bcc7e07a4f4886c5994d17cbef72ea09eb1be1d
Was-Change-Id: If55182ca29f37f2a783fdb88ba46512944a61c47
Was-Change-Id: I72cdfc6b76bfd9c2832a5d5e5f72c3aa45cf1efe
Was-Change-Id: Ie7088bdbfcae396602b59e2ab07fbfbbb14d96af
Was-Change-Id: I2af9c3964978c842dac8f70ad814adb529dff39f
Was-Change-Id: I0267182fbfa646de40ac62f832e89fbfd8477822
Was-Change-Id: Ifad125815318f07b332c6323feffbd216dad6144
Was-Change-Id: I29f1fa5744659e6095203e860267c6ed02268943
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55474
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Tue, 6 Jun 2023 11:47:05 +0000 (19:47 +0800)]
LU-10499 tests: disable metadata_csum_seed for pcc cache device
latest e2fsprogs enable metadata_csum_seed feature when mkfs.ext4
a device, which is used in pcc test as a cache device. Such a ext4
fs cannot be mounted on an old kernel of el7. So disable this
feature for pcc cache device if it is detected in sanity-pcc test.
EX-7596 tests: don't fail if metadata_csum_seed unset
Fix logic in the sanity-pcc cache filesystem setup. With mke2fs
1.47.0-wc1 it enabled metadata_csum_seed unconditionally, but it
caused problems on el7.9 kernels. In 1.47.0-wc2 it disabled that
feature, caused the check for removing the feature to fail.
Since metadata_csum_seed has been available since 1.44.2 it
shouldn't be a problem to force it off duing mke2fs.
Was-Change-Id: Ic04ab4043981dc9b5c32e01c4aa85be343e3f3f8
EX-6826 tests: wait before unmounting pcc device if busy
wait at most 10 seconds before unmounting pcc device
if it is busy.
Was-Change-Id: I77ec018d33d14af99bdc5d5c5c94c8fa0dafdb61
EX-bug-id: EX-7596 EX-6826
Test-Parameters: trivial testlist=sanity-pcc clientdistro=el8.10
Signed-off-by: Lei Feng <flei@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2ef3ff27b49fb479ec348b742cf614a43321813b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54486
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log