Whamcloud - gitweb
James Simmons [Mon, 8 Jul 2024 20:20:52 +0000 (16:20 -0400)]
LU-17460 lnet: properly handle net_device referencing
Most of the time LNet uses __in[6]_dev_get_xxx() which does no
reference changes. The one expection is the use of dev_get_by_index()
called in lnet_create_socket(). Replace dev_get_by_index() with
dev_get_by_index_rcu(). Also examined the code to make sure the
right type of locking was done. If we use rcu locking we should
use for_each_netdev_rcu() so update ksocknal_ip2index().
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
e4fa181abf1 ("LU-10391 lnet: allow creation of IPv6 socket.")
Fixes:
09c6e2b8722 ("LU-16836 lnet: ensure dev notification on lnd startup")
Change-Id: I0c496652553318bd0e47fa1e03d6e631fd8421bb
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55582
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sergey Cheremencev [Fri, 24 May 2024 16:28:47 +0000 (19:28 +0300)]
LU-15053 tests: sanity-quota_13 fix
Scope a case when there are any extra users
with quota limit and non zero usage besides
TSTUSR and TSTUSR2. This is possible when
tests are started with ENABLE_QUOTA=yes.
In a such case each user may have a lock between
OST and QMT depending. Take this into account
in sanity-quota_13. Fix with following failure:
sanity-quota test_13: @@@@@@ FAIL: 2 cached locks
Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota env=ONLY=13,ONLY_REPEAT=100
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Iaf48d0eb80eef0fc5ebc8246e8fac3f9c96563c0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55197
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Wed, 26 Jun 2024 17:43:35 +0000 (13:43 -0400)]
LU-17629 utils: support hostname with
lustre_lnet_parse_nid_range()
For a hostname it's possible it maps to multiple IPs. In
this case lnetctl commands that attempt to use the hostname
can resolve to the wrong IP address. Update the function
lustre_lnet_parse_nid_range() to work with hostnames and
properly resolve the correct IP address. Update both
lnetctl ping and lnetctl discover to work with
lnet_parse_nid_range().
Test-Parameters: trivial testlist=sanity-lnet
Change-Id: I670799edcb04a02380e96c289ba26854b057d978
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54894
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Wed, 10 Apr 2024 12:27:22 +0000 (15:27 +0300)]
LU-17722 tests: trim tmpfs from wait_delete_completed()
to release unused ram
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Idcd4d15e0f56184e1d1897f3a64d5b62baaf7edb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54720
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Deiter
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Qian Yingjin [Fri, 27 Aug 2021 03:46:13 +0000 (11:46 +0800)]
LU-10499 pcc: add stats for attach|detach|auto_attach
In this patch, we add stats for PCC attach, detach and
auto_attach.
With this feature, we verify that PCC can auto-attach the file
into PCC cache without having to re-fetch the data of the whole
file.
Add sanity-pcc test_44.
EX-bug-id: EX-3715
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ia0c1cd6b414998e72859aaf34c125b5a4e4e743c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54418
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Tue, 8 Jun 2021 09:54:49 +0000 (17:54 +0800)]
LU-10499 pcc: avoid to specify ID for every attach
In this patch, it avoids the need to specify "-i <attach_id>" for
every attach as in the very common case there is only a single
cache for that client.
If attach ID is not specified, it will select the first dataset
on the client as PCC backend.
And the new format of PCC state is as follows:
file: /mnt/lustre/f42.sanity-pcc, type: readonly, PCC_file:
/d42.sanity-pcc/0402/0x200000401:0x3:0x0, open_count: 0, flags: 0
EX-3752 pcc: show attaching state for PCC state output
When set llite.*.pcc_async_threshold=0, the client will do PCC
attach in asynchronous way.
When the file is large, attaching the file into PCC may take some
time.
In this patch, we improve that output of the PCC command
"lfs pcc state" to show that the file is in PCC attaching state
when the file is still in the phase of copying from Lustre OSTs
to PCC.
Was-Change-Id: I101d87638f5afac41fb4f55b4aaf95d938bc8ccd
EX-bug-id: EX-3292 EX-3752
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Icd23eda5dca4711f9bb7af940f6cef5ddb97ce69
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54414
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Qian Yingjin [Wed, 12 May 2021 03:43:28 +0000 (11:43 +0800)]
LU-10499 pcc: avoid dead lock for auto attach in PCC-RO
In this patch, It releases the pcc inode lock when calling
ll_layout_refresh() in @pcc_try_auto_attach() as it may cause the
following deadlock:
1. The client is writing or truncating a file in readonly mode.
At this time, it will send a write layout intent lock to clear
the readonly state on the layout on MDT.
2. A read process tries to auto attach the file with pcc inode
lock hold. During the pregress of auto attach, it will call
ll_layout_refresh(). The client-side enqueue request for a
layout lock returned a blocked lock, it will sleep and wait for
the lock being granted;
3. MDT will take EX layout lock to cancel all cached layout lock
on client to change the layout for clearing the PCC-RO state.
4. when the client handles the revocation of layout lock, it needs
to invalidate the PCC state which needs under the protection of
pcc inode lock.
EX-3191 pcc: add test for mmap | write | detach racer
This patch adds the mmap racer among: (write | read | mmap_cat |
detach | unlink): sanity-pcc/test_99.
Was-Change-Id: I5db160851a95937275fea6ae32f40dcd0fe69f46
EX-3478 pcc: avoid uninitialized pcc mutext lock in cleanup
Running racer concurrently crashed in the following way:
RIP: 0010:[...] [...] __list_add+0x1b/0xc0
__mutex_lock_slowpath+0xa6/0x1d0
mutex_lock+0x1f/0x2f
pcc_inode_free+0x1e/0x60 [lustre]
ll_clear_inode+0x64/0x6a0 [lustre]
ll_delete_inode+0x5d/0x220 [lustre]
evict+0xb4/0x180
iput+0xfc/0x190
ll_iget+0x156/0x350 [lustre]
ll_prep_inode+0x212/0x9b0 [lustre]
After analysis, we found that the mutex @lli_pcc_lock is not
initialized. The reason is that ll_lli_init() is not called to
initialize @lli.
When call pcc_inode_free(), it will call mutex_lock() on the
uniniitialized @lli_pcc_lock, thus crash the kernel.
In liblustreapi_pcc.c, it should set errno on error return.
Was-Change-Id: I612c79a5b8eb4fa9daeb9e446a457e95c666c04a
EX-3636 pcc: reset file mmaping for the file once mmaped
For a file once mmaped and cached on PCC, a new open will set the
mapping for the file handle of PCC copy (@file->f_mapping) with
the one of the Lustre file handle. When the file is detached from
PCC due to manual detach or layout lock shrinking, the normal I/O
(read/write) will auto-attach the file into PCC again during I/O
as the layout version is unchanged. However, it still needs to
reset the file mapping (@pcc_file->f_mapping) with the mapping of
the PCC copy. Otherwise it will cause panic as follows:
[ 935.516823] RIP: 0010:_raw_read_lock+0xa/0x20
[ 935.517077] ll_cl_find+0x19/0x60 [lustre]
[ 935.517098] ll_readpage+0x51/0x820 [lustre]
[ 935.517110] read_pages+0x122/0x190
[ 935.517119] __do_page_cache_readahead+0x1c1/0x1e0
[ 935.517131] ondemand_readahead+0x1f9/0x2c0
[ 935.517142] pagecache_get_page+0x30/0x2c0
[ 935.517165] generic_file_buffered_read+0x556/0xa00
[ 935.517189] pcc_try_auto_attach+0x3ac/0x400 [lustre]
[ 935.517552] pcc_io_init+0x146/0x560 [lustre]
[ 935.517906] pcc_file_read_iter+0x24d/0x2b0 [lustre]
[ 935.518259] ll_file_read_iter+0x74/0x2e0 [lustre]
[ 935.518604] new_sync_read+0x121/0x170
[ 935.518937] vfs_read+0x8a/0x140
This patch adds sanity-pcc test_98 to verify it.
I/O for a file previously opened before attach into PCC or once
opened while in ATTACHING state will fallback to Lustre OSTs.
For the later mmap() on the file, the mmap() I/O also needs to
fallback to Lustre OSTs and cannot read directly from local valid
cached PCC copy until all fallback file handles are closed as the
mapping of the PCC copy is replaced with the one of Lustre file
when mmapped a file.
Add sanity-pcc test_97 to verify it.
And we also forbid to auto attach the file which is still in
mmapped I/O.
EX-3636 pcc: auto attach should skip if already attached
When try to auto attach a file into PCC, if found that the file
had already attached into PCC, it should skip the auto attach
processing. Otherwise, it will result in wrong PCC inode refcount
when multiple threads try to auto attach a file at the same time.
For a file once mmapped into PCC and detached due to layout lock
shrinking or manual detach command, If found that file is still
valid cached (attach into PCC again by another thread), in the
@pcc_mmap_io_init(), it should set the mapping of PCC copy with
the one of Lustre file again.
Was-Change-Id: I5f049ca7d6db8708712e79e9ad459fc60b80f2be
LU-17964 pcc: set mapneg bit in all cases of normal I/O fallback
When a file is copying data from Lustre OSTs to the PCC copy, the
file is in PCC ATTACHING state. New opens and I/O on this file
will fallback to the normal I/O path (Lustre OSTs) before the
attach is finished. And the file handle will be set with fallback
and mapneg bit. Currently we only clear the fallback and mapneg
bit when the file handle is closed.
To support mmap() I/O, we replace the mapping of the PCC copy with
the one of the Lustre file. However, we can do that only if the
Lustre file has not any opened file handle with mapneg bit set.
Otherwise, we can not switch the mapping and the mmap() I/O will
also fallback to Lustre OSTs and use the mapping of the Lustre
file.
Once a mmap()ed file was detached from PCC backend due to the
manual detach command or the revocation of the LAYOUT ibit lock
(which protects the cache validity of PCC cache access), we should
reset the mapping of the PCC file accordingly and set fallback and
mapneg bits if the I/O is falling back into the normal path
(Lustre OSTs).
Was-Change-Id: Ibd152aaf724dcff48efbe022dc7f3e70848b4e0d
EX-bug-id: EX-3080 EX-3191 EX-3478 EX-3480 EX-3636
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I18890d19d03726a5991c923505e8c5363382fdc2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54390
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sat, 6 Jul 2024 01:54:39 +0000 (08:54 +0700)]
LU-16350 ldiskfs: Server support for linux v6.7 / Ubuntu 24.04
Exclude kunit tests [files matching *-test.c] from ldiskfs build
Updated patch series for Linux v6.7:
ext4-corrupted-inode-block-bitmaps-handling-patches.patch
ext4-ialloc-uid-gid-and-pass-owner-down.patch
Updated patch series for Linux v6.5:
ext4-data-in-dirent.patch
Change struct osd_it_ea_dirent.oied_name from zero length
to flexible array so strncmp works as expected.
Test-Parameters: trivial
HPE-bug-id: LUS-11376
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2b2325a5874a91096fbd63750096e459065668bc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54216
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Timothy Day [Mon, 8 Jul 2024 21:09:40 +0000 (21:09 +0000)]
LU-6142 misc: remove empty header
This header doesn't do anything.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia544c9629ea5c787390c393baa45c310126c14ea
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55666
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Wed, 3 Jul 2024 04:35:43 +0000 (04:35 +0000)]
LU-6142 osd-ldiskfs: SPDX for ldiskfs OSD
Convert from verbose license text to SPDX.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ie9a298fcf9999af1edfbf1acb58c23d7c83fbb7c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55613
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Wed, 3 Jul 2024 04:17:14 +0000 (04:17 +0000)]
LU-6142 osd-zfs: SPDX for ZFS OSD
Convert from verbose license text to SDPX.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I02da1c1772da64b88a4469a07d42522472f6ba72
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55612
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Thu, 27 Jun 2024 16:40:19 +0000 (10:40 -0600)]
LU-14810 lnet: Do not issue multiple PUSHes
PUSH ACK may be delayed in network. Meanwhile, some event could cause
peer to go through discovery again (e.g. config change or NI state
change). The discovery state machine doesn't consider whether there
is an outstanding PUSH so it may issue another one for the same peer.
When delayed ACK arrives it will then clear PUSH_SENT, so now
discovery doesn't know that there is an outstanding PUSH. If discovery
is stopped then it doesn't unlink the push MD and this can cause an
assert in lnet_assert_handler_unused() because the push event handler
is still in use.
Modify the discovery state machine to check for PUSH_SENT when
determining whether a peer needs a PUSH.
sanity-lnet test_304 can reproduce this issue under ipv6
configuration if modules are unloaded at the end of the test.
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ic3f7a8b44f85a18afb939fdbfa1f9bc5dc64d93d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55559
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Mon, 17 Jun 2024 05:39:25 +0000 (01:39 -0400)]
LU-6142 lnet: Fix style issues for timer.c
This patch fixes issues reported by checkpatch
for file lnet/selftest/timer.c
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ia672f0800389fc5fb1c323919b9345faba1b0347
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55447
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Wed, 5 Jun 2024 14:43:50 +0000 (14:43 +0000)]
LU-17893 tests: replay-dual/31 fails a lot
Add some debugging.
Test-Parameters: trivial testlist=replay-dual env=ONLY=31,ONLY_REPEAT=50
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ic417ee5d6a3d53ce0e7aa51708dc9d9317b1ce30
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55321
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Thu, 22 Feb 2024 07:45:37 +0000 (13:15 +0530)]
LU-6142 llite: Fix style issues for file.c
This patch fixes issues reported by checkpatch
for file lustre/llite/file.c
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I66d7a4adfad48d6de26be8d009f004efb90b6d23
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54139
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Fri, 21 Jun 2024 04:43:37 +0000 (12:43 +0800)]
LU-15491 mdt: rename lock inversion deadlock
MDT rewrite got rid of lock ordering by fid and replaced it
with parent/child ordering, but child to child lock inversion is
still possible with hardlinks as follows:
thread 1: mv dir1/10 dir2/10
thread 2: mv dir2/11 dir2/2
where dir1/10 is hardlink to dir4/2 and dir2/10 is hardlink to dir2/11
To solve this we enforce child ordering by fid in case of local rename
This should not create problems aynwhere else but rename since
the next closest candidate - link() does not delete the target
if it exists so it's not locked in that case.
Fixes:
d76cc65d5d68 ("LU-12125 mds: allow parallel regular file rename")
Change-Id: Idd3525e8b1a0de411766bdcaa480c60d3b5e491b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/46352
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Jian Yu [Thu, 20 Jun 2024 22:26:19 +0000 (15:26 -0700)]
LU-17404 kernel: update RHEL 9.4 [5.14.0-427.22.1.el9_4]
Update RHEL 9.4 kernel to 5.14.0-427.22.1.el9_4.
Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el9.4 serverdistro=el9.3 testlist=sanity
Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el9.4 serverdistro=el9.3 testlist=sanity
Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el9.3 serverdistro=el9.4 testlist=sanity
Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el9.3 serverdistro=el9.4 testlist=sanity
Test-Parameters: optional clientdistro=el9.4 serverdistro=el9.4 \
testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.4 serverdistro=el9.4 \
testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.4 serverdistro=el9.4 \
testgroup=full-part-3
Change-Id: Icb9aac22fbc4b7d34d13288e98fa4c28022db82e
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55487
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Mon, 1 Jul 2024 08:21:47 +0000 (10:21 +0200)]
LU-17978 tests: always except sanity-pcc test_33 on el8.9+
sanity-pcc test_33 is failing 100% of the time on rhel8.9 and el810
due to previous reported inconsistent LSOM problem in LU-17781,
so add it to ALWAYS_EXCEPT until a fix is found.
Test-Parameters: trivial clientdistro=el8.9 testlist=sanity-pcc
Test-Parameters: trivial clientdistro=el8.10 testlist=sanity-pcc
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ic8a48708e26776ff84201b040cbb9993fc1fe25a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55589
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Fri, 28 Jun 2024 16:56:41 +0000 (09:56 -0700)]
LU-17404 kernel: RHEL 9.4 server support
This patch makes changes to support RHEL 9.4 release
with kernel 5.14.0-427.20.1.el9_4 for Lustre server.
Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el9.4 serverdistro=el9.3 testlist=sanity
Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el9.4 serverdistro=el9.3 testlist=sanity
Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el9.3 serverdistro=el9.4 testlist=sanity
Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el9.3 serverdistro=el9.4 testlist=sanity
Test-Parameters: optional clientdistro=el9.4 serverdistro=el9.4 \
testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.4 serverdistro=el9.4 \
testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.4 serverdistro=el9.4 \
testgroup=full-part-3
Change-Id: I4741041c6b7f5604b13523a24060b6a2804a5ef2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54713
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Thu, 27 Jun 2024 20:08:49 +0000 (14:08 -0600)]
LU-17986 tests: Update sanity-lnet 111 for IPv6
Modify test_111 to use setup_router_test() so that it will use
IPv6 NIDs for the --gateway argument when that is what the test
environment has configured.
The test is also modified to check success/failure of the route add
command, and to verify that the correct number for routes were added.
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I8dea5874dd01eb7f35603bed46660cd6c6b85041
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55561
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 25 Jun 2024 23:07:30 +0000 (16:07 -0700)]
LU-17976 build: fix nla_strnid() -Werror=missing-prototypes
This patch explicitly defines nla_strnid() as a static function
to fix the following build failure:
lnet/lnet/api-ni.c:2929:1: error:
no previous prototype for 'nla_strnid' [-Werror=missing-prototypes]
2929 | nla_strnid(struct nlattr **attr, struct lnet_nid *nid, int *rem,
| ^~~~~~~~~~
It also fixes:
- lnet_fault_show_done()
- lnet_fault_show_start()
- lnet_fault_show_dump()
- __ll_dio_user_copy()
- ll_dio_user_copy_helper()
Change-Id: I0225794b3fac2f36aafadff783ca921fbc757edd
Fixes: f1c6623 ("LU-10391 lnet: Fault injection add/del ioctls to netlink")
Fixes: 1fa633c ("LU-17478 clio: parallelize unaligned DIO write copy")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55537
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Elena Gryaznova [Tue, 25 Jun 2024 10:08:27 +0000 (13:08 +0300)]
LU-17975 tests: execvp error on file write_append_truncate
test fails on sles15sp4 clients due to write_append_truncate
not found:
execvp error on file write_append_truncate
(No such file or directory)
HPE-bug-id: LUS-8427
Test-Parameters: trivial testlist=parallel-scale env=ONLY=write_append_truncate
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: I88a5a6c30510dff19b59d32eca2a90566a21f64e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55530
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 25 Jun 2024 03:00:20 +0000 (20:00 -0700)]
LU-17793 libcfs: fix objtool warning in lbug_with_loc()
After lbug_with_loc() was removed from the objtool
global_noreturns array in Linux commit v6.4-rc2-10-g34245659debd,
building Lustre hit the following warning:
libcfs/libcfs/fail.o: warning: objtool: __cfs_fail_check_set()
falls through to next function __cfs_fail_timeout_set()
This patch fixes the above warning by adding an unreachable
panic() at the end of lbug_with_loc() to terminate all of
the call paths in that function.
As a consequence of this change, we need to make the patch
fix more errors, such as:
lnet/lnet/api-ni.c: In function 'lnet_res_type2str':
libcfs/include/libcfs/libcfs_private.h:119:9:
error: this statement may fall through [-Werror=implicit-fallthrough=]
119 | lbug_with_loc(&msgdata); \
| ^~~~~~~~~~~~~~~~~~~~~~~
lnet/lnet/api-ni.c:1143:17: note: in expansion of macro 'LBUG'
1143 | LBUG();
| ^~~~
lnet/lnet/api-ni.c:1144:9: note: here
1144 | case LNET_COOKIE_TYPE_MD:
| ^~~~
and
lustre/obdclass/lprocfs_status.c: In function 'lprocfs_stats_lock':
lustre/obdclass/lprocfs_status.c:470:1:
error: control reaches end of non-void function [-Werror=return-type]
470 | }
| ^
Change-Id: I5574559619b4b6746f4e7da51f3213ede246a73b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55505
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Bruno Faccini <bfaccini@nvidia.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Sat, 22 Jun 2024 12:25:29 +0000 (08:25 -0400)]
LU-17969 tests: don't use hard coded mount path for quota testing
On setup for quota testing sanity-quota assumes the mount path
is always /mnt/lustre. This is not always true so update to
$MOUNT.
Fixes:
7e1fb1a296e ("LU-17179 tests: check the system is clean")
Test-Parameters: trivial testlist=sanity-quota
Change-Id: I2c1fbf5f8f38b4c508137b2c6e956d47031e9c12
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55504
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 6 Jun 2024 02:40:46 +0000 (09:40 +0700)]
LU-17904 build: fix typo in vvp_set_batch_dirty
Fix typo vvp_set_batch_dirty() when kallsyms_lookup_name()
is exported and account_page_dirtied is not.
HPE-bug-id: LUS-12374
Test-Parameters: trivial
Fixes:
b82eab822c0 ("LU-17081 build: Prefer folio_batch to pagevec")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I8b2e6884e74e384aba6e563bef30072175cc0efc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55301
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Shaun Tancheff [Fri, 28 Jun 2024 07:49:31 +0000 (14:49 +0700)]
LU-17903 build: enable fast path of vvp_set_batch_dirty
SUSE 15 SP6 6.4 kernel retains kallsyms_lookup_name so
the fast path of vvp_set_batch_dirty() can be enabled.
However the combination of kallsyms_lookup_name without
lock_page_memcg breaks some old assumptions
Prefer folio_memcg_lock to lock_page_memcg however
Linux commit v5.15-12272-g913ffbdd9985
mm: unexport folio_memcg_{,un}lock
folio_memcg_lock is also not exported so use
kallsyms_lookup_name to acquire the symbol
HPE-bug-id: LUS-12371
Test-Parameters: trivial
Fixes:
61e83a6f130 ("LU-16113 build: Fix configure tests for lock_page_memcg")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I8ac6b7bde8ee8964db5a801c2f3c4dfb2ef459f9
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55300
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Maximilian Dilger [Tue, 28 May 2024 17:55:27 +0000 (13:55 -0400)]
LU-10522 utils: new --mindepth for lfs find
Added [--mindepth | -d] option for 'lfs find' to print
only the results found after N levels. Similar usage to existing
mindepth found in find.
Signed-off-by: Maximilian Dilger <mdilger@whamcloud.com>
Change-Id: I7816e27355f6796edc8437f700342da1b7e564d0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55226
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Di Wang [Wed, 8 May 2024 23:20:40 +0000 (23:20 +0000)]
LU-17833 ptlrpc: Check lru_resize during connection
Since the parameter log processing might finish before connection
is established, so it should check if lru size has been disabled
by parameters log in ptlrpc_connect_set_flags().
OCI-bug-id: LFS-229
Signed-off-by: Di Wang <di.d.wang@oracle.com>
Change-Id: I246fcbcd17aa201f80b6950d8eff57489dc81645
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55060
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Patrick Farrell <patrick.farrell@oracle.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Fri, 28 Jun 2024 02:18:50 +0000 (12:18 +1000)]
LU-14712 ldiskfs: add bg_trimmed_threshold interface
Export interface bg_trimmed_threshold via sysfs, and only clear
BG_TRIMMED flag when there are enough blocks freed since
last fstrim(default 128).
Make sure we use cpu_to_le32() with EXT2_FLAGS_TRACK_TRIM.
Change-Id: I98d86f6d7335af53b8e74c747797b0dff3abb5d0
Test-Parameters: trivial
Fixes:
ad30edf910 ("LU-14712 ldiskfs: introduce EXT4_BG_TRIMMED to optimize fstrim")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Tue, 11 May 2021 03:38:36 +0000 (11:38 +0800)]
LU-10499 pcc: keep PCC copy when it is being attached
When detach a file from PCC backend via FID, if the file is being
attached, it should not purge the coresponding PCC copy from the
PCC backend. Just keep the PCC copy to finish the attach process.
EX-3144 pcc: revalidate the pointer after attach
In this patch, we also fix a bug during PCC open attach. It
refreshes @pcci pointer since the lock may be released in
@pcc_try_readonly_open_attach().
Was-Change-Id: I470358dfde525e08e7110e862b30b527e5db94fe
EX-bug-id: EX-3133 EX-3144
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I8a8f7c6986d51eaf9b2516e5dd5a6f21aa38b7db
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54389
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Fri, 19 Mar 2021 08:45:26 +0000 (16:45 +0800)]
LU-10499 pcc: don't reopen mountpoint for each cache file
When scanning and processing files in the PCC cache filesystem
(e.g. "llapi_pcc_scan_detach()" is looking for the Lustre
mountpoint and reopening it for every file processed.
This patch changed it to open the Lustre mountpoint only once,
then reuse the file handle for all of the later calls. The file
handle will be closed when finished the processing.
This patch also repaces to use llapi_fid_parse to get FID from
an given string.
EX-bug-id: EX-2861
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iad92c216262296096e30ca4a4c6b2765dfd3afaa
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54388
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Mon, 22 Mar 2021 09:16:15 +0000 (17:16 +0800)]
LU-10499 pcc: async attach in the background for PCC-RO file
In current PCC, it may have a long delay while the whole file is
being copied into the cache before it can be used. There is a
significant delay for the first file access if the file is large,
which wastes valuable computing time. Being able to shorten this
time to first access may help application efficiency.
In this patch, it adds an tuning parameter "async_threshold",
which means the size threshold to determine doing PCC-RO attach
asynchronously in the background.
When the file size is samller than the threshold, the PCC attach
during open() will be performed in synchronous way.
Otherwise, the client will start a dedicated kernel thread to
copy data from Lustre OSTs to the PCC copy in the background, but
reads could fall back to the normal Lustre I/O path from Lustre
OSTs until the file is fully cached.
This may double the reads to the Lustre filesystem initially if
the file is not read sequentially, but would avoid the high
latency for data access. This may be some cache sharing (avoiding
double reads) if the PCC copy and the application both shared
the filesystem cached pages on the client.
The tuning parameter "llite.*.pcc_async_threshold" is set with
256MiB by default.
EX-3880 pcc: add pcc_async_affinity for async PCC attach
This patch adds a tunable parameter "llite.*.pcc_async_affinity"
that enables or disables the CPT selection in PCC-RO asynchronous
attach for testing.
Was-Change-Id: I1473a7547555a2d6c615d37182b6cc359194aae0
EX-bug-id: EX-2873 EX-3880
Test-Parameters: clientcount=3 testlist=sanity-pcc,sanity-pcc,sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ia80992e9050cc6e4c7f61949fc4013dec303e150
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54379
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Sun, 6 Sep 2020 08:52:04 +0000 (16:52 +0800)]
LU-12358 pcc: add project quota support on PCC backend
Current PCC can enforce a quota limitation of the capacity usage
for each user and group to provide cache isolation. An admin
can specify the quota enforcement on the local PCC file system.
Users can perform PCC-cached I/O on files until they receive a
return value -ENOSPC of -EDQUOT, which means that they hit the
quota limit or that there is no free capacity left on the local
PCC backend fs during I/O or the attach process. At this time,
I/O will fall back to the normal I/O path.
This patch adds project quota on the PCC backend file system
along with user/group quota.
With this feature, it can have multiple PCC backends on a single
client with different caching rules, so we can define upfront
how much of the client FS can be used for each cache.
Test-Parameters: clientcount=3 testlist=sanity-pcc,sanity-pcc,sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ib93da953d4a3a7091f62094f8175bde91e819895
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/39831
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Thu, 25 Feb 2021 12:43:58 +0000 (20:43 +0800)]
LU-10499 pcc: get PCC state for a file without opening itself
Originally to get PCC state for a given file, the user needs to
open the file and then get the current PCC state of the file via
the file handle. After that, close the file.
If the file is met the predefined condition of auto prefetching
into PCC at the open time, "lfs pcc state" command on the file
will attach the file into PCC cache. This may be not the intention
of the user.
In this patch, we rework the "lfs pcc state" command. It always
open the parent directory, and then do the lookup by name/FID
without open the file itself to get the PCC state.
EX-5358 pcc: remove realpath() from lfs_pcc_state()
Before Ubuntu 20.04, realpath() executes lstat() for each
component of the path. If the file is still valid cached on PCC
device with the layout generation unchanged, the Lustre file will
be auto re-attach during the stat() call in the Lustre kernel.
This may result in misunderstanding for the operation "lfs pcc
state" on a file that has already detached but still valid cached
on PCC according to the unchanged layout generation.
This problem exposes on the newer Ubuntu 22.04 in which realpath()
executes readlink() for each component of the path instead of
lstat():
readlink("/mnt", 0x7fffd5760800, 1023) = -1
readlink("/mnt/lustre", 0x7fffd5760800, 1023) = -1
readlink("/mnt/lustre/sanity-pcc.f15", 0x7fffd5760800, 1023) = -1
Was-Change-Id: I50ae46a1e952a3faaf0d7a7293579e239156d6d3
LU-16030 pcc: enlarge PCC backend size for sanity-pcc script
In this patch, it removes realpath() from lfs_pcc_state() to avoid
this misunderstanding behavior for the command: $LFS pcc state.
And it also fixes the test scripts sanity-pcc: test_15, test_16,
test_27, test_39.
Was-Change-Id: Ib19f01ed054cb6c9eecceabea1f1da72dea0b113
EX-bug-id: EX-2455 EX-5358 LU-16030
Test-Parameters: clientcount=3 testlist=sanity-pcc,sanity-pcc,sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I310a7e73dc6c0f4318dc27df2e02ecf6559ee5b4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54372
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Qian Yingjin [Fri, 5 Feb 2021 03:48:26 +0000 (11:48 +0800)]
LU-10499 pcc: check first before set PCC-RO on a file
In this patch, MDT takes a CR layout lock against the file object
first to check whether the file is already PCC-RO cached. If so,
return immediately; Otherwise, take an EX lock on the file to
update the FLR PCC-RO state accordingly. By this check, it can
avoid heavy lock contention and unnecessary revocation of the
layout lock granted to the other clients when multiple processes
from many clients perform read-only attach on a shared file
simultaneously.
EX-bug-id: EX-2455
Test-Parameters: clientcount=3 testlist=sanity-pcc,sanity-pcc,sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: If59315abe444917f8a890b60a38c239b8ee045bf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54370
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Qian Yingjin [Wed, 30 Sep 2020 03:00:43 +0000 (11:00 +0800)]
LU-14003 pcc: rework PCC mmap implementation
In the old PCC mmap implementation, it replaces the vm_file with
the file of the PCC copy, and then call ->fault() or
->page_mkwrite() on the PCC copy, after that restore the vm_file
with the one of the Lustre file.
This design exists problem as a mmaped region (vma) could be
faulted concurrently with multiple children threads (each children
threads can clone the VM of the parent process). There is no any
atomic guarantee for the replacement and restore the vm_file during
calling ->fault() or ->page_mkwrite().
This patch reworks the mmap() implementation for PCC.
In the new design, PCC mmap replaces the inode mapping of the PCC
copy on the PCC backend filesystem with the one of the Lustre file.
By this way, the mmaped region (vma) will link into the mapping of
the Lustre inode not the mapping of the PCC copy.
It keeps using vm_file with the file handle of the PCC copy until
the PCC cached file is detached or unmmaped.
LU-14003 pcc: convert mapping pagecache for mmap
In the PCC mmap implementation, it will replace the mapping of
the PCC copy with the one of the Lustre file when do mmap() to
make the mmapped region (vma) link into the mapping of the
Lustre file not the mapping of the PCC copy.
At this time, in the old design the pagecache in the original
mapping of the PCC copy is simply dropped as the mapping of each
page is different after the replacement of the mapping.
This may have negative impact on the mmap performance.
The reason is that during PCC attach it will write the data from
Lustre into PCC copy in buffered I/O mode, these data will keep
in pagecache and managed by the mapping of the PCC copy if there
is enough system memory. Then for the latter mmap, the page fault
could directly read data from the pagecache to speed up the mmap
operation.
If drop these pagecahe due to the different mapping of each pages,
the page fault must read page from the disk and may result in bad
performance.
To make full use of these pagecache of the PCC copy, during mmap
call, it can first remove the page from the original mapping of
the PCC copy, and then convert and add it into the mapping of the
Lustre file. By this way, all pagecaches are converted and can be
reused for the latter page fault.
Was-Change-Id: I1591937543d7d31b8811ec62088accd0070d7d37
EX-8421 llite: disable kernel readahead for pcc mmap
Set ra_pages to 0 for PCC files when mmaped, because
otherwise this setting carries through to Lustre and will
cause crashes and possible inconsistencies. This happens
because the PCC file and Lustre file share a mapping, which
is a weird trick required to have mmap work on PCC.
Add a set of asserts which confirm kernel readahead is
disabled and wasn't used for mmap.
Was-Change-Id: I117042d68fac25158e8141c243acba698cf1930f
LU-17866 pcc: zero ra_pages explictly for a file after PCC mmap
To support mmap under PCC, we do some special magic with mmap to
allow Lustre and PCC to share the page mapping.
The mapping host (@mapping->host) for the Lustre file is replaced
with the PCC copy for mmap. This may result in the wrong setting
of @ra_pages for the Lustre file handle with the backing store of
the PCC copy in the kernel:
->do_dentry_open()->file_ra_state_init():
file_ra_state_init(struct file_ra_state *ra,
struct address_space *mapping)
{
ra->ra_pages = inode_to_bdi(mapping->host)->ra_pages;
ra->prev_pos = -1;
}
Setting readahead pages for a file handle is the last step of the
open() call and it is not under the control inside the Lustre file
system.
Thus, to avoid setting @ra_pages wrongly we set @ra_pages with
zero for Lustre file handle explictly in all read I/O path.
When invalidate a PCC copy, we will switch back the mapping
between Lustre and PCC. We also set mapping->a_ops back with
@ll_aops.
The readahead path in PCC backend may enter the ->readpage() in
Lustre. Then we check whethter the file handle is a Lustre file
handle. If not, it should be from mmap readahead I/O path of the
PCC copy and return error code directly in this case.
Was-Change-Id: Id1e4a9e47bb484e97053759e1743fd2fce040149
Test-Parameters: clientcount=3 testlist=sanity-pcc,sanity-pcc,sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Icc5019a691dfb04b5e1fdd580d83915cfe590158
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/40092
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Mon, 17 Jun 2024 10:10:05 +0000 (18:10 +0800)]
LU-17464 lod: use OBD_ALLOC_LARGE for ldo_comp_entries
The lod_object::ldo_comp_entries is allocated/free with _LARGE macros
so that it could be large enought to use vmalloc instead of kmalloc
for memory allocation. There are some places use OBD_ALLOC without
_LARGE to re-allocate memory which mismatch the assumption.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ie356ae875329af07c893586fa4b1485dbd17afe6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55449
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Arshad Hussain [Mon, 17 Jun 2024 08:29:19 +0000 (04:29 -0400)]
LU-6142 lnet: Fix style issues for console.[ch]
This patch fixes issues reported by checkpatch
for file lnet/selftest/console.[ch]
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I484b2ffee5d5add360055b424e23fdc97c5618ae
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55448
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Mon, 17 Jun 2024 18:09:06 +0000 (12:09 -0600)]
LU-16822 tests: Modify local_node to check for IPv6
Nodes may be configured with just IPv6 addresses, so local_node()
needs to look for both IPv4 and IPv6 addresses to determine if a given
host is local.
sanity-lnet/110 is re-written so that it does not rely on
local_addr_list(). Otherwise the test may attempt to configure an NI
using an invalid address. This test case can now execute on o2ib
configs.
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Id3471376dbb2089a44b00ed7cb9bc2256e5e7501
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55463
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 17 Jun 2024 14:29:33 +0000 (16:29 +0200)]
LU-17404 kernel: fix filemap_splice_read detection
On Centos 9 kernel 5.14, filemap_splice_read is in the header files,
but the symbol is not exported by the kernel.
So instead of trying to build a kernel module with a call to this
function, just use LB_CHECK_EXPORT on this symbol.
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1f55d0b41c46a992204c1cebc3f5c8c7dbc6128e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55454
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Srikanth Ramamurthy [Mon, 17 Jun 2024 17:57:40 +0000 (11:57 -0600)]
LU-16822 tests: Force IPv6 testing in mixed environment
When an interface has both IPv4 and v6 addresses LNet will, by
default, configure the NI using the v4 address. The '--large' option
to lnetctl lnet configure tells LNet to configure the NI using the
v6 address instead. This patch adds a test-framework environment
variable that, when set, passes the --large option to lnet configure.
This allows us to force testing of IPv6 when running in a mixed v4/v6
environment.
This patch implements ip_is_v6() which is needed by some of the
router tests when using IPv6 NIDs.
Some test cases are added to the except list:
- 230 requires lctl conn_list to be updated to work with large nids.
- 303, and 500 have been found to trip LU-17460 bug.
Change-Id: I8934a87bfd836779b167df39c5d09d97ff78debf
Test-Parameters: trivial
Signed-off-by: Srikanth Ramamurthy <srramamu@microsoft.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55422
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Oleg Drokin [Tue, 25 Jun 2024 03:34:26 +0000 (23:34 -0400)]
New release 2.15.64
Change-Id: I0d760b4b58bd24b72e781c52465c07417725cffe
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Olaf Weber [Mon, 17 Jun 2024 04:14:17 +0000 (00:14 -0400)]
LU-9119 lnet: whitespace cleanup for wirecheck.c
Clean up the whitespace use in lnet/utils/wirecheck.c
Test-Parameters: trivial
Signed-off-by: Olaf Weber <olaf@sgi.com>
Signed-off-by: Olaf Weber <olaf.weber@hpe.com>
Change-Id: I5c90d09fd694c8151f6f11f716c491ac3db79eb0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55446
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Frederick Dilger [Fri, 14 Jun 2024 03:29:10 +0000 (23:29 -0400)]
LU-17856 tests: ignore sanity stripe-count off-by-1
In some cases the MDS may not create all stripes on a file, if the
MDT-OST connection does not have precreated objects. This is OK,
so the tests should not fail the stripe-count check if trying to
create a fully-striped file and one of the stripes is missing.
parse_layout_param was modified to change the output value of
stripe count to be $OSTCOUNT if the stripe_count=$OSTCOUNT - 1.
Even if the stripe_count was meant to be $OSTCOUNT - 1 this
shouldn't fail any tests as both tested values will be modified.
Test-Parameters: trivial testlist=sanity env=ONLY=56xd-56xe,ONLY_REPEAT=100
Test-Parameters: trivial testlist=sanity env=ONLY=65n,ONLY_REPEAT=100
Test-Parameters: trivial testlist=sanity env=ONLY=184d,ONLY_REPEAT=100
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: Ie908a07d21b75e3ba60b7e6ca326675684ee2037
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55427
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Thu, 13 Jun 2024 00:07:07 +0000 (17:07 -0700)]
LU-17948 pcc: fix llapi_pcc_del() -Werror=enum-int-mismatch
gcc 13 does not allow mixing of enum and integer
types between function declaration and implementation.
This patch fixes the following build failures:
liblustreapi_pcc.c:755:5: error:
conflicting types for 'llapi_pcc_del' due to enum/integer mismatch;
have 'int(const char *, const char *, enum lu_pcc_cleanup_flags)'
[-Werror=enum-int-mismatch]
755 | int llapi_pcc_del(const char *mntpath, const char *pccpath,
| ^~~~~~~~~~~~~
liblustreapi_pcc.c:790:5: error:
conflicting types for 'llapi_pcc_clear' due to enum/integer mismatch;
have 'int(const char *, enum lu_pcc_cleanup_flags)' [-Werror=enum-int-mismatch]
790 | int llapi_pcc_clear(const char *mntpath, enum lu_pcc_cleanup_flags flags)
| ^~~~~~~~~~~~~~~
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: I2900b59a609410c6faab78d24f6176bc5c268e98
Fixes: 0d7d9ae ("LU-17657 build: gcc 13 stricter enum checking")
Fixes: c74878c ("LU-12373 pcc: uncache the pcc copies when remove a PCC backend")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55417
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Wed, 12 Jun 2024 23:32:53 +0000 (16:32 -0700)]
LU-17948 llite: replace i_mtime.tv_sec with inode_get_mtime_sec()
This patch replaces i_mtime.tv_sec with inode_get_mtime_sec() to
fix the following build failure:
lustre/llite/pcc.c:1691:32: error:
'struct inode' has no member named 'i_mtime'; did you mean '__i_mtime'?
1691 | item.pm_mtime = inode->i_mtime.tv_sec;
| ^~~~~~~
| __i_mtime
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Iaed264c32be3d48039c5350ebd306f4fc3ef5eb9
Fixes: 3835f4d ("LU-13881 pcc: comparator support for PCC rules")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55416
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Thu, 13 Jun 2024 11:44:46 +0000 (07:44 -0400)]
LU-17947 build: fix LASSERTF [-Werror=format=] failure
This patch fixes the following build failures:
libcfs/include/libcfs/libcfs_private.h:89:34:
error: format '%o' expects argument of type 'unsigned int',
but argument 4 has type 'long unsigned int' [-Werror=format=]
89 | "ASSERTION( %s ) failed: " fmt, #cond, \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
lustre/ptlrpc/wiretest.c:2718:9: note: in expansion of macro 'LASSERTF'
2718 | LASSERTF(MDS_FMODE_CLOSED == 000000000000UL, "found 0%.11oUL\n",
| ^~~~~~~~
Test-Parameters: trivial
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I97a895e6234721c34f681d0ee7ce91ead4dd30f8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55415
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 12 Jun 2024 21:38:10 +0000 (14:38 -0700)]
LU-17945 lnet: fix nla_extract_val() -Werror=missing-prototypes
This patch explicitly defines nla_extract_val() as a static function
to fix the following build failure:
lnet/lnet/api-ni.c:2888:1: error:
no previous prototype for 'nla_extract_val' [-Werror=missing-prototypes]
2888 | nla_extract_val(struct nlattr **attr, int *rem,
| ^~~~~~~~~~~~~~~
Test-Parameters: trivial testlist=sanity-lnet
Change-Id: Ieb11d25ea8fcd19b715e2decf958cfd9d920bcc8
Fixes: 629d80d ("LU-10003 lnet: migrate fail nid to Netlink")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55413
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 13 Jun 2024 09:19:04 +0000 (11:19 +0200)]
LU-17940 gss: get rid of root key sooner
The root key associated with a GSS context (gck_key) is used to pass
information between kernel and userspace during GSS context
negotiation.
Once the GSS context for root is up-to-date, the key is never used
again, although it has a permanent validity. And when the context
expires, the key is directly revoked and replaced with a new one to
serve the negotiation of a new root context.
So to avoid issues with keys staying in the root's kernel keyring and
being accidentally revoked, just get rid of the key associated with a
root context as soon as the negotiation process has finished.
Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4be773723b9046ed451684bd141d5ef2bc584bfb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55406
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Xinliang Liu [Wed, 12 Jun 2024 10:24:53 +0000 (10:24 +0000)]
LU-11671 tests: re-enable sanity test_45 for aarch64
This is fixed by patch https://review.whamcloud.com/54763
("LU-17733 tests: sanity test_45 fix dirty count").
Test-Parameters: trivial
Test-Parameters: testlist=sanity clientarch=aarch64 \
clientdistro=el9.3 env=ONLY=45,ONLY_REPEAT=100
Change-Id: I4716a0bee2689ffb33db8a81f1f33be6562b929e
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55403
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Arshad Hussain [Wed, 12 Jun 2024 09:35:17 +0000 (05:35 -0400)]
LU-17000 utils: Initialize 'idgot' time before using
In case there is an error reading the contents of permission
file. gettimeofday() is correctly not called on 'idgot'.
However, this means that 'idgot' timeval is left uninitialized.
This patch Initialize 'idgot' timeval to 0 so that in case as
above the value is printed as zero and not garbage.
Test-Parameters: trivial
CoverityID: 397122 ("Uninitialized scalar variable")
Fixes:
d5b26443 ("LU-16615 utils: add messages in l_getidentity")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ie3d5dff1f02ede83690472e60cc14c12ec5d978a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55402
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 10 May 2024 17:41:57 +0000 (11:41 -0600)]
LU-17935 kfilnd: Cleanup debug logging
Log messages that refer to a struct kfilnd_transaction should print
the pointer to the struct with "TN %p".
Assign kfilnd_transaction::msg_type in the kfilnd_process_rx_event
path so that debug messages show the correct message type.
HPE-bug-id: LUs-11325
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Iabe3bf245b64f1eb66c85259072491c723fb6119
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55398
Reviewed-by: Caleb Carlson <caleb.carlson@hpe.com>
Reviewed-by: Ron Gredvig <ron.gredvig@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Tue, 11 Jun 2024 10:40:26 +0000 (12:40 +0200)]
LU-17930 gss: node principal expectations
When a credentials cache exists for Kerberos, lgss_keyring looks into
it to find a valid entry. The cache's principal must match the
expected role for the GSS request being processed:
- LGSS_ROOT_CRED_MDT: expect "lustre_mds" principal;
- LGSS_ROOT_CRED_OST: expect "lustre_oss" principal;
- LGSS_ROOT_CRED_ROOT: expect "lustre_root" or "host" principal.
And there is the special case of the GSS request on the MGC, for which
by convention all 3 roles are applied at the same time.
Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4c46b03bb012c5f56bd26efdfaa6dab5fc7de31a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55392
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Tue, 11 Jun 2024 08:33:11 +0000 (10:33 +0200)]
LU-17929 ptlrpc: ptlrpc_request_alloc_pack() always returns an error code
Current code was always considering that when this function
returns NULL it meant ENOMEM error, but this is not always
true, especially when using GSS by example, or when
reconnecting from an IDLE state.
Also, instead of having every caller converting NULL to
ENOMEM, do that directly in the function when
appropriate.
Make ptlrpc_request_alloc_pack() return -errno in case
of error instead of a NULL pointer.
Thanks to that change, error code will be propagated up
and will help error reporting and debugging.
Took the opportunity to simplify related error path
for 2 HSM functions.
Also changed param.status to a signed data, as it can
store -errno.
Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Change-Id: Id2b873d5f0c5cb89db070f6db00269545e6c85e8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55391
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Fri, 7 Jun 2024 15:49:40 +0000 (11:49 -0400)]
LU-17919 tests: wait to resolve ENOSPC in sanity 398l
Test 398l does not wait to clear up the ENOSPC it induces,
so sometimes it causes 398m to fail with ENOSPC.
Wait for deletes to resolve this.
OCI-bug-id: LFS-288
Note on test-parameters - we can't 'REPEAT' a pair of
tests, it would run 398l over and over and then 398m,
which doesn't test what we need to test. So instead we
just create 5 sessions like this.
Test-Parameters: testlist=sanity envdefinitions=ONLY="398l 398m"
Test-Parameters: testlist=sanity envdefinitions=ONLY="398l 398m"
Test-Parameters: testlist=sanity envdefinitions=ONLY="398l 398m"
Test-Parameters: testlist=sanity envdefinitions=ONLY="398l 398m"
Test-Parameters: testlist=sanity envdefinitions=ONLY="398l 398m"
Test-Parameters: trivial
Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com>
Change-Id: I2fcc1069a0304bc6edfa576331b6255289b71b98
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55359
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Frederick Dilger [Thu, 6 Jun 2024 20:00:25 +0000 (16:00 -0400)]
LU-16491 utils: update getdirstripe yaml format
'lfs getdirstripe --yaml' now prints directory layout in yaml
format. getdirstripe now also prints the "self" FID whether the
directory is striped or not. "migrating" fields
(lmv_migrate_offset, lmv_migrate_hash) were not included because
of the additional code complexity required to add the two fields.
The migrating fields are stored in 'struct lmv_mds_md_v1' which
AFAIK isn't available though getdirstripe.
For 0 striped directories, lmv_objects: will now contain
information on the directory itself, this information
becomes redundant with -v, however it is useful when the
lmv_fid isn't being shown.
New YAML layout:
lmv_fid: 0x280000404:0x5:0x0
lmv_magic: 0xcd20cd0
lmv_stripe_count: 4
lmv_stripe_offset: 2
lmv_hash_type: crush
lmv_objects:
- l_mdt_idx: 2
l_fid: 0x280000400:0x4:0x0
- l_mdt_idx: 0
l_fid: 0x200000402:0x4:0x0
- l_mdt_idx: 1
l_fid: 0x240000402:0x2:0x0
- l_mdt_idx: 3
l_fid: 0x2c0000403:0x2:0x0
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: I03ddc24816484d11c8c70892831e9edc9da5455a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55346
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 15 Mar 2024 18:52:07 +0000 (12:52 -0600)]
LU-17855 lnet: Set peer NI down on lnet_notify
The LNet router peer health feature is intended to allow LNet routers
drop messages for peer NIs that it considers down/unreachable so that
resources can be freed to forward messages to peer NIs that are
up/reachable.
This feature was integrated with the LNet health feature under
LU-11300, and, as a result, routers only consider a peer NI
down/unreachable if two criteria are met:
1. The router hasn't received a message from the peer NI within the
LND's "peer_timeout" value (default 180 seconds).
2. The health value of the peer NI has been decremented or the cached
peer NI status is LNET_NI_STATUS_DOWN.
(1) is problematic because a lot of messages can be queued to a down
peer while we wait for the peer_timeout to expire. This can
introduce latency for messages being forwarded to other peers.
(2) is problematic because there are some cases where LNet health
will not be decremented (namely single-rail peers), and the cached
peer NI status can only be set to LNET_NI_STATUS_DOWN if the router
receives a discovery push from the peer. If the peer loses all
connectivity to the router then it is possible the router will never
consider it down.
To address the problems with (1) the requirement is dropped
completely.
To address the problems with (2), LNet routers will now decrement
health values of single-rail peers and lnet_notify() is modified to
set the peer NI status UP/DOWN according to the aliveness information
provided by the LND.
HPE-bug-id: LUS-12209
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I7823cc7ae73bcb0b6b52db8d4f84cff7b999d8c0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55342
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sat, 13 Apr 2024 02:39:01 +0000 (10:39 +0800)]
LU-17910 build: iokit should also be named per lustre_name
Update lustre-iokit to follow the ${lustre_name} scheme
HPE-bug-id: LUS-12250
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I236aaaac1acaf86f08aa584c6a7d5d3a3d75ff49
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 6 Jun 2024 02:17:15 +0000 (09:17 +0700)]
LU-17909 dkms: enable dkms with external lustre_name
Allow dkms package naming with --define 'lustre_name cray-lustre'
HPE-bug-id: LUS-12249
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I35ea6ed1017b691e5c0c105ff5c3f3a0028b2cbd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55327
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Caleb Carlson <caleb.carlson@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Wed, 5 Jun 2024 12:35:02 +0000 (14:35 +0200)]
LU-17907 enc: enc flag should not remove other flags
When updating inode flags, the lli_flags must be taken into account
so that they do not get lost. So provide helper functions for callers
of ll_update_inode_flags(), as an overlay to ll_inode_to_ext_flags().
And on server side, the mdd layer must fetch the existing flags when
setting LUSTRE_ENCRYPT_FL attr flag.
Fixes:
40d91eafe2 ("LU-12275 sec: atomicity of encryption context getting/setting")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I150f2d87cef112beab81d1d2030133671d4b7361
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55317
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 1 Jun 2024 04:37:13 +0000 (04:37 +0000)]
LU-17844 debug: purge the final LCONSOLE_ERROR_MSG()
Replace the remaining LCONSOLE_ERROR_MSG() with LCONSOLE_ERROR().
Remove the LCONSOLE_ERROR_MSG() macro.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I289ced87e5f77dd3eebcf94ee978e23b79e55698
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55282
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 1 Jun 2024 04:20:36 +0000 (04:20 +0000)]
LU-17844 llite: remove all LCONSOLE_ERROR_MSG()
These magic numbers aren't so magical anymore. Just
use LCONSOLE_ERROR().
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I6409f22f9707428d89cbbe1f92e1b2e11ce6f10a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55280
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Fri, 17 May 2024 00:26:26 +0000 (00:26 +0000)]
LU-17844 mgs: remove all LCONSOLE_ERROR_MSG()
These magic numbers aren't so magical anymore. Just
use LCONSOLE_ERROR().
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I933f7c7acbf85cddd1bdb95e9506cb0f37abdba5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55135
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Sebastien Buisson [Thu, 16 May 2024 09:58:24 +0000 (11:58 +0200)]
LU-17852 gss: do not use expired reverse gss contexts
On server side, a reverse context matches a gss context established
on client side. These reverse contexts have a expiration time, and are
replaced with fresh ones when they expire.
So get rid of expired reverse contexts when we find them in the
gsk_clist. And when we look up for a context, do not continue using
the current one if it is expired.
Add sanity-krb5 test_200 to check the expired reverse contexts.
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I11f2d8ab298073f9d5bedff187b67f2ca289ae47
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55127
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Tue, 21 May 2024 00:35:48 +0000 (10:35 +1000)]
LU-13577 tests: fix interop checking for intent_mkdir tests
Use the correct interop version for replay-single/137a|b|c
sanity/852, sanityn/116
Change-Id: I0b7fc03c542574dfb468a17719a6be81d738c5b3
Fixes:
668dfb53de ("LU-13577 wbc: reimplement mkdir() by using intent lock")
Test-Parameters: trivial
Test-Parameters: serverjob=lustre-master serverbuildno=4524 testlist=replay-single env=ONLY="137a 137b 137c"
Test-Parameters: serverjob=lustre-master serverbuildno=4524 testlist=sanity env=ONLY=852
Test-Parameters: serverjob=lustre-master serverbuildno=4524 testlist=sanityn env=ONLY=116
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55161
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 11 Jun 2024 06:25:06 +0000 (23:25 -0700)]
LU-17404 kernel: new kernel [RHEL 9.4 5.14.0-427.20.1.el9_4]
This patch makes changes to support new RHEL 9.4 release
for Lustre client.
Test-Parameters: trivial \
mdtcount=4 mdscount=2 clientdistro=el9.4 testlist=sanity
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-3
Change-Id: Ic292c01ad16dc06e8dee966c4a211896fea284c0
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54712
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Mon, 25 Mar 2024 21:14:39 +0000 (05:14 +0800)]
LU-11085 tests: add a test case for same range lock
Add a new case in ldlm_extent to test performance for
same range work with interval tree.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ia3ffe90263a2a7f01a3a44f8801a32fc789b5abc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54565
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Tue, 12 Mar 2024 17:20:05 +0000 (01:20 +0800)]
LU-17276 tests: performance test for same range lock
Lustre used a optimaized interval tree to manage the
extents. It only keeps one entry in tree for all of same
range locks. But upstream links all lock in tree. This
test want to compare the difference of performance
between two cases.
Test-Parameters: trivial testlist=performance-sanity
Test-Parameters: trivial testlist=performance-sanity
Test-Parameters: trivial testlist=performance-sanity
Test-Parameters: trivial testlist=performance-sanity
Test-Parameters: trivial testlist=performance-sanity
Test-Parameters: trivial testlist=performance-sanity
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I653ad1fc2cdc1012312d11af77b1e2c133b7c34e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54364
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Etienne AUJAMES [Mon, 5 Feb 2024 14:12:20 +0000 (15:12 +0100)]
LU-17480 o2iblnd: add a timeout for rdma_connect
For a RoCE network, if a RDMA connection request is sent to an
unreachable node, the CM can take >4min to return
CM_EVENT_UNREACHABLE.
This hangs lustre_rmmod if a Lustre router is down.
This patch track connection requests and apply a timeout of
lnd_timeout/4 (with a minimum of 5s) to destroy the hanging
connection.
Also, the patch decrease the timeout for
rdma_resolve_addr()/rdma_resolve_route() to 5s (like most of
the upstream drivers: sunrpc, smb).
The default timeouts should be:
lnd_timeout = (transaction_timeout - 1) / (retry_count + 1)
lnd_timeout = (150 - 1) / 3 = 49s
lnd_connreq_timeout = max(5, lnd_timeout / 4) = 12s
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I09e40ffaa75424c4acca1d0cf986e1ff9c6dc96b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53986
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Wed, 24 Apr 2024 22:26:04 +0000 (18:26 -0400)]
LU-17478 clio: parallelize unaligned DIO write copy
The data copying for unaligned/hybrid IO reads is already
parallel because it is done by the ptlrpc threads at the
end of the IO. That for the writes is not - it is done
by the submitting thread during IO submission.
This has a huge performance impact, limiting writes to
around 3.0 GiB/s when reads are at 12 GiB/s.
With the iov iter issue fixed, we can do this copy as
part of IO submission.
With this and the patch to use page pools for buffer
allocation (https://review.whamcloud.com/53670), the
maximum performance of hybrid IO is basically the same as
DIO, at least for current speeds.
This means hybrid reads and writes at 20 GiB/s with
current master + this and the pool patch.
Note this requires a funny workaround: If a user thread
calls fsync while a DIO write is in progress, the user
thread can pick that write up at the RPC layer and become
responsible for writing it out, even though that write
isn't in cache. (Because the write is waiting to be
picked up by a ptlrpcd thread.)
If that DIO write is unaligned, the user thread is unable
to do the memory copy. It's not an option to have the
thread ignore a ready RPC, so instead we spawn a kthread
to handle this case.
This only occurs when DIO is racing with fsync, so
performance doesn't really matter, and this cost is OK.
Signed-off-by: Patrick Farrell <patrick.farrell@oracle.com>
Change-Id: Ic8209e1fda97cda83e5b87baba48d15dd4dcc15f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53844
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Arshad Hussain [Mon, 22 Jan 2024 16:09:19 +0000 (21:39 +0530)]
LU-17000 utils: Fix negative argument passed
This patch fixes bunch of "Argument cannot be negative"
reported by Coverity.
CoverityID: 397713 ("Argument cannot be negative")
CoverityID: 397899 ("Argument cannot be negative")
Pass -rc to strerror() since rc is guaranteed to be negative.
CoverityID: 397769 ("Argument cannot be negative")
read() could return -1 on error. Check before using the value
CoverityID: 397780 ("Argument cannot be negative")
Use static allocation removing the need to call sysconf()
CoverityID: 403104 ("Argument cannot be negative")
On failure to open do not try to call close()
Fixes:
58d744e3 (LU-10092 pcc: Non-blocking PCC caching)
Fixes:
f172b116 (LU-10092 llite: Add persistent cache on client)
Fixes:
aed82919 (LU-16029 utils: add options to lr_reader to parse raw files)
Fixes:
86ba46c2 (LU-9680 obdclass: user netlink to collect devices information)
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Id51c6c184d30dce596e7ab948a6fd0768eed1503
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53796
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Shaun Tancheff [Wed, 1 Nov 2023 09:52:49 +0000 (04:52 -0500)]
LU-17246 build: Refine SUSE 15 SP3 ldiskfs version
SUSE linux-5.3.18-150300.59.60_7.0.4.15 should select
5.3.18-sles15sp3.series however a recent change maps
linux-5.3.18-150300.59.* to 5.3.18-sles15sp3-59.series
which does not apply.
Lustre should also consider the patch level [Ex: 60]
and map 59.60* to 5.3.18-sles15sp3.series while
61 and later should map to 5.3.18-sles15sp3-59.series
The sles15sp3*.series should also include
base/ext4-delayed-iput.patch
base/ext4-delayed-iput.patch
Test-Parameters: trivial
HPE-bug-id: LUS-11910
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I68d6ecfc94e9edf7c657e743fbdbdb58b39bf191
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52943
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Wed, 28 Feb 2024 16:00:26 +0000 (23:00 +0700)]
LU-17579 build: drop SUSE in-kernel ofed special handling
SUSE special case handling for mofed does not apply to in-kernel
ofed build.
Drop the complicated search and filter looking for Module.symvers
Test-Parameters: trivial
Fixes:
8b1d2a72f1 ("LU-16967 build: Add in-kernel-ko2iblnd driver")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I90c3de9020a26bdbc12c0ca859ada722b7c73c94
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54176
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Caleb Carlson <caleb.carlson@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 10 Jun 2024 18:25:38 +0000 (11:25 -0700)]
LU-16350 ldiskfs: Server support for LTS linux v6.6
Migrate upai ext4 headers into staging for ldiskfs
Updated patch series for Linux LTS v6.6.10
ext4-attach-jinode-in-writepages.patch
ext4-dont-check-before-replay.patch
ext4-mballoc-pa-free-mismatch.patch
ext4-pdirop.patch
ext4-prealloc.patch
ext4-corrupted-inode-block-bitmaps-handling-patches.patch
ext4-delayed-iput.patch
ext4-encdata.patch
ext4-ialloc-uid-gid-and-pass-owner-down.patch
ext4-mballoc-extra-checks.patch
Dropped:
ext4-add-periodic-superblock-update.patch
Test-Parameters: trivial
HPE-bug-id: LUS-11376
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2a0a5d4be1e724ed1936178ccc3f7a7e7a2672c7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52919
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Tue, 4 Jun 2024 23:18:34 +0000 (06:18 +0700)]
LU-17085 llite: safely duplicate iov_iter
When copying an iov_iter the underlying iovec/bvec also
needs to be duplicated.
Discard and xarray iters are not relevant here however the
memory allocated needs to be tracked and freed.
This borrows heavily from dup_iter()
The iter argument to ll_dio_user_copy() is no longer used
and can be removed.
Explicitly handle ubuf() cases to work with el9.4 back-port
of ITER_UBUF series.
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I9f68eaa14abc8915d543dba91eea598edbd9872d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52266
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <patrick.farrell@oracle.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Fri, 11 Aug 2023 05:32:31 +0000 (15:32 +1000)]
LU-14712 ldiskfs: introduce EXT4_BG_TRIMMED to optimize fstrim
Currently the flag indicating block group has done fstrim is not
persistent, and trim status will be lost after remount, as
a result fstrim can not skip the already trimmed groups, which
could be slow on very large devices.
This patch introduces a new block group flag EXT4_BG_TRIMMED,
we need 1 extra block group descriptor write after trimming each
block group.
When clearing the flag, the block group descriptor is journalled
already so no extra overhead.
Add a new super block flag EXT2_FLAGS_TRACK_TRIM, to indicate if
we should honour EXT4_BG_TRIMMED when doing fstrim.
The new super block flag can be turned on/off via tune2fs.
Change-Id: I7faaca4754b1726ad05d0aafe3e90e0e9f591617
Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51923
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Wed, 12 Jun 2024 05:56:51 +0000 (01:56 -0400)]
LU-6142 utils: Fix style issues for routerstat.c
This patch fixes issues reported by checkpatch
for file lnet/utils/routerstat.c
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I679fcf0530b1ce25c00fa35c05a904d5b0a4ecb8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55420
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Mon, 17 Jun 2024 17:20:33 +0000 (10:20 -0700)]
LU-17941 ofd: do not copy over filter_fid structure
When a bigger filter_fid has been writen on disk by newer server,
downgraded Lustre would read more data but we need store less to
fit smaller filter_fid structure.
Test-Parameters: serverdistro=el8.10
Fixes:
28c366cee6d ("LU-17218 ofd: improve filter_fid upgrade compatibility")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Idb5c8fffe4af22f35b64aa93e7efce7f9dd206d6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55408
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Tue, 11 Jun 2024 17:55:10 +0000 (11:55 -0600)]
LU-17440 tests: Fix sanity-lnet/test_226
test_226 is not deleting a route from the remote peer as intended.
This is because of two issues. The first issue is that the wrong
gateway NID is being provided to do_route_del(). The second issue is
that do_route_del() is not correctly quoting the remote command to
detect and delete routes. The correct NID is now specified and
the quoting in do_route_del() has been fixed. Some additional
output was also added so that we can verify via test output that it is
working correctly.
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
2b210f3905 ("LU-17440 lnet: prevent errorneous decref for asym route")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ie21a1bdb174cb65ae64134a727d69e1c95a4ddd5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55397
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bruno Faccini [Fri, 7 Jun 2024 09:22:44 +0000 (11:22 +0200)]
LU-17911 lustre: fix faked flexible arrays in getinfo_fid2path
faked (0-length) flexible arrays need some rework to comply with
new coding rules to stay Fortify feature compliant (see document
at https://people.kernel.org/kees/bounded-flexible-arrays-in-c).
With this particular getinfo_fid2path struct content, we ended-up
with generated code causing straight crash upon each call of
lmv_fid2path routine upon strlen() first call.
Signed-off-by: Bruno Faccini <bfaccini@nvidia.com>
Change-Id: Id6f594779ca0ae86f0c2842535abccbf4df688d3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55354
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 24 Feb 2024 18:05:14 +0000 (18:05 +0000)]
LU-6142 contrib: update checkpatch.pl to 6.10-rc2
checkpatch.pl hasn't been updated in around 7 years. To make sure
we don't waste our efforts, update the script while porting a few
Lustre specific changes.
This updates checkpatch.pl to 6.10-rc2 while recording the changes
in contrib/.
This is mostly a simple port of the old changes to a new version of
checkpatch.pl. It is intended to silence a few false positives and
generate a few Lustre specific warnings.
One feature has been added: the ability to differentiate between
user space code and kernel code when doing spell checks.
I tested this by running the updated script across the entire tree,
ensuring that nothing failed:
contrib: loc=192 errors/warnings=1
libcfs: loc=25117 errors/warnings=1624
lnet: loc=133290 errors/warnings=16381
lustre: loc=645087 errors/warnings=81360
Also, add a userspace_spelling.txt file by copying spelling.txt. We
start by simply duplicating the old spelling.txt file while removing
a number of entries.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia39c7f04d2407786f158d7f6f95968e182e99ebf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54153
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 24 Feb 2024 02:14:53 +0000 (02:14 +0000)]
LU-6142 contrib: Lustre checkpatch.pl diff
Record the changes made to vanilla checkpatch.pl.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I095d435c66425476be34483661137a1359f77911
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54154
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Sun, 16 Jun 2024 13:28:20 +0000 (09:28 -0400)]
LU-10391 lnet: Fault injection add/del ioctls to netlink
Convert the fault injection add/del ioctls to a netlink
implementaiton.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I20f38d4e7c0215a1b19772c6253c617174c0b00c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53732
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Wed, 12 Jun 2024 14:12:34 +0000 (10:12 -0400)]
LU-9680 lnet: Fault injection list ioctls to netlink
Convert the fault injection list ioctls to a netlink implementation.
sanity-lnet tests that use fault injection can now be enabled for
large NIDs.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ieaf9c01401fc0841c1e5805667531ba3455e8110
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53733
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 31 May 2024 15:58:30 +0000 (09:58 -0600)]
LU-17525 llite: unaligned DIO interop page alignment
Correctly size brw/ptlrpc bulk ops I/O between archs with differing
page sizes (ex: 64k client and 4k server).
Since the number of MDs need for a bulk are not stable until all the
pages are added we have two parts to interop calculation.
In the fist step if this is an unaligned dio bulk with a 64k offset
greater than 4k in size calculate the initial MD utilization
as if first partial page is 64k in length.
In the second step after bulk is sized and split across 1 or more
MDs, if the number of MDs:
- if the number of MDs is 1, clear the interop flag
- if the number of MDs is 3 or more, keep the interop flag
- if the number of MDs is 2 and the size with the 64k offset does
not exceed the LNET_MUT then collapse the extra MD.
This is done by assuming the first page is 64k length.
Additionally fixup OBD_CONNECT2_UNALIGNED_DIO and add a ZFS osd
heuristic check. No unaligned DIO should be performed with older
zfs-osd, however unaligned dio with unpatched ldiskfs servers is
allowed for most cases. Allow I/O that will trigger the page
size interop issue will get fail with -EINVAL, were previously
these i/o would fail sending with an error:
LNetError: 7386:0:(lib-ptl.c:189:lnet_try_match_md()) Matching packet
from 12345-10.240.22.81@tcp, match
1789613636069888 length 1044481
too big: 983041 left, 983041 allowed
triggering the MD to be resent, however the size calcuations reamin
unchanged resulting in a hang.
Test-Parameters: testlist=sanity clientarch=aarch64 clientdistro=el9.3
Test-Parameters: testlist=sanity clientarch=ppc64le clientdistro=el8.8 env=SANITY_EXCEPT="398c 411b"
Test-Parameters: testlist=sanity serverversion=2.15.4 serverdistro=el8.9 env=SANITY_EXCEPT="17n 24g 27R 56oc 56wc 162c 230b 230c 230x 230t 273c 300i"
Test-Parameters: testlist=sanity clientarch=aarch64 clientdistro=el9.3 serverversion=2.15.4 serverdistro=el8.9 env=SANITY_EXCEPT="27R 56oc 56wc 160a 160l 162c 230c 230m 230t 230x 273c 300i"
Test-Parameters: testlist=sanity clientarch=aarch64 clientdistro=el9.3 envdefinitions=ONLY="119e 119f 119g 119h 119i"
Test-Parameters: testlist=sanity clientarch=aarch64 clientdistro=el9.3 envdefinitions=ONLY="119m 119m 119n 119o 119p 119q"
Test-Parameters: testlist=sanity clientarch=x86_64 clientdistro=el9.3 envdefinitions=ONLY="119e 119f 119g 119h 119i"
Test-Parameters: testlist=sanity clientarch=x86_64 clientdistro=el9.3 envdefinitions=ONLY="119m 119m 119n 119o 119p 119q"
Fixes:
7194eb6431 ("LU-13805 clio: bounce buffer for unaligned DIO")
Fixes:
0e6e60b123 ("LU-13805 llite: Implement unaligned DIO connect flag")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ifb5152b7ebaba696e6f2cef3af43b0ecd5e53d94
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53997
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Patrick Farrell <patrick.farrell@oracle.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Thu, 18 Apr 2024 11:10:39 +0000 (21:10 +1000)]
LU-16692 tests: force_new_seq_all interop version checking
force_new_seq_all is still needed in those test suites
if testing against servers don't have v2_15_61-226-gf00d2467fc
Test-Parameters:trivial serverjob=lustre-master serverbuildno=4516 testlist=replay-single,replay-ost-single,replay-dual,recovery-small,replay-vbr,sanity-pfl
Change-Id: Iab963ac10308b56a60508774c1a63bcdfffdba85
Fixes:
9ef186b71b ("LU-16692 tests: remove force_new_seq from some test suites")
Fixes:
f00d2467fc ("LU-16692 osp: do not assert on seq got over network")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54840
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Zarochentsev [Thu, 30 May 2024 16:23:25 +0000 (16:23 +0000)]
LU-17888 osd-ldiskfs: osd_scrub_refresh_mapping deadlock
After copying a lustre special file (last_rcvd for example)
to a new inode, lustre mount hangs with the following stack trace:
[root@testbed ~]# cat /proc/pidof mount.lustre/stack
[<0>] rwsem_down_write_slowpath+0x32a/0x610
[<0>] osd_obj_update_entry.isra.22+0xb7/0x900 [osd_ldiskfs]
[<0>] osd_obj_spec_update+0x146/0x160 [osd_ldiskfs]
[<0>] osd_scrub_refresh_mapping+0x282/0x420 [osd_ldiskfs]
[<0>] osd_ios_scan_one+0x5df/0xe10 [osd_ldiskfs]
[<0>] osd_ios_root_fill+0x267/0x300 [osd_ldiskfs]
[<0>] call_filldir+0xb0/0x120 [ldiskfs]
[<0>] ldiskfs_readdir+0x7a7/0xac0 [ldiskfs]
[<0>] iterate_dir+0x13c/0x190
[<0>] osd_ios_general_scan+0x10e/0x250 [osd_ldiskfs]
[<0>] osd_initial_OI_scrub+0x72/0x920 [osd_ldiskfs]
[<0>] osd_scrub_setup+0x8ab/0x9e0 [osd_ldiskfs]
[<0>] osd_device_init0+0x447/0x810 [osd_ldiskfs]
[<0>] osd_device_alloc+0x186/0x220 [osd_ldiskfs]
[<0>] obd_setup+0x115/0x2d0 [obdclass]
[<0>] class_setup+0x57f/0x790 [obdclass]
[<0>] class_process_config+0x1104/0x2460 [obdclass]
[<0>] do_lcfg+0x21d/0x530 [obdclass]
[<0>] lustre_start_simple+0x77/0x1d0 [obdclass]
[<0>] osd_start+0x408/0x7f0 [obdclass]
[<0>] server_fill_super+0x382/0x10d0 [obdclass]
[<0>] lustre_fill_super+0x3a1/0x3f0 [lustre]
[<0>] mount_nodev+0x48/0xa0
[<0>] legacy_get_tree+0x27/0x40
[<0>] vfs_get_tree+0x25/0xb0
[<0>] do_mount+0x2e2/0x950
[<0>] ksys_mount+0xb6/0xd0
[<0>] __x64_sys_mount+0x21/0x30
[<0>] do_syscall_64+0x5b/0x1a0
[<0>] entry_SYSCALL_64_after_hwframe+0x65/0xca
root inode lock is attempted to be taken twice,
once in iterate_dir() and another attempt in
osd_obj_update_entry().
HPE-bug-id: LUS-12368
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: Idc5f9bd2a20d25dfb5eb4a044ddd00ff7eb4558b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55267
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Thu, 6 Jun 2024 21:43:09 +0000 (15:43 -0600)]
LU-17914 lnet: Fix erroneous net set error
lnetctl net set --health command reports a false error. This is
because in lnet_genl_parse_local_ni() the last value stored in rc is
from nla_strscpy() which can return a positive value.
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
fff650726b ("LU-13642 lnet: Allow dynamic IP specification")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I86211a1083af4b225076f966d2e0c7793589a33a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55344
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Thu, 6 Jun 2024 20:55:08 +0000 (14:55 -0600)]
LU-17913 iokit: Fix lst.sh host mode
The ssh command used to get primary NIDs of each test node uses a
single quote. This prevents the LCTL variable from being expanded
so the command fails. Use double quotes around this command and
make sure we return an error (non-zero) status when we are unable
to determine primary NIDs of test nodes.
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
c265e1c7b0 ("LU-16861 obdfilter: Exclude quotes when getting NIDs")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ic17d8a22baf93e205c6d7e12a0079f93222013e3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55343
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Thu, 6 Jun 2024 19:35:29 +0000 (13:35 -0600)]
LU-17894 lnet: Initialize common lnd params
This ensures the correct default values are used when the user does
not specify a value for them.
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
7e01787863 ("LU-12452 lnet: allow to set IP ToS value per-NI")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I201e9acbc3be8e27cd8957b722c8cea9a64de6c1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55341
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Thu, 6 Jun 2024 16:55:39 +0000 (10:55 -0600)]
LU-17895 lnet: Validate input for lnetctl import
Add validation of the input being parsed by lnetctl import when it is
expecting the global parameters. This resolves segmentation faults
when it encounters bad or missing keys and values.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I232daed773283ea89e9815bf6bd79526dfd5fd4f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55340
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Thu, 6 Jun 2024 11:12:00 +0000 (14:12 +0300)]
LU-15644 llog: don't report warning in no error case
Fix wrong check which includes rc == 0 valid case wronly
Fixes:
bd9839f7db (LU-15644 llog: don't replace llog error with -ENOTDIR)
Test-Parameters: trivial
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id6e7b2cd42b4769765c67d418552a13f048ea050
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55335
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Arshad Hussain [Thu, 6 Jun 2024 04:38:30 +0000 (00:38 -0400)]
LU-17000 utils: Call yaml_parser_scan before using 'token'
Function 'yaml_parser_scan' is used to initilize and get
the next token. Call yaml_parser_scan() before using token.
Test-Parameters: trivial testlist=sanity-lnet
CoverityID: 403112 ("Uninitialized scalar variable")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ieeb3d50a32690dc892a831fb60a9e85e4ec05113
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55330
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Wed, 5 Jun 2024 13:50:41 +0000 (15:50 +0200)]
LU-17000 gss: update init_channel initialization
Update 'init_channel' to match newer coding guidelines.
DDN-Bug-Id: EX-9705
Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Test-Parameters: testgroup=review-dne-part-2
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6539ade1a9d815664f6659a5c1ee25e7f1f7df0e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55322
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Frederick Dilger [Fri, 31 May 2024 04:28:27 +0000 (00:28 -0400)]
LU-15565 utils: updated lfs getstripe yaml format
For composite files 'lfs getstripe --yaml' used a key for each
component rather than an array. This hindered the use of the
output with common YAML tooling. The leading spaces meant users
needed to pre-process the output for seemingly no reason. Each
component also had the id harcoded into the name rather than
using a more flexible list ('components:').
'lfs setstripe --yaml YAML_FILE file|dir' is still compatible
with the previous YAML format incase there were stored files
being used.
The new YAML formatting has been check with results from
"https://zhwt.github.io/yaml-to-go/".
Modified the argument parsing in 'lfs_migrate' to be more
robust as it was failing tests 56[w[a-c],x[b,c]].
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: I50ff4ebd9413fb66f05647c11542f7ce9f1ba879
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55311
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Fri, 17 May 2024 09:40:23 +0000 (05:40 -0400)]
LU-17897 lfsck: don't assert on orphan existence
lfsck_namespace_create_orphan_dir() is called in several cases,
and orphan may exist in some cases, change assertion to check.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I28563aa60d0f345616fd30cd0899495e7c1ef8f0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55302
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Mon, 3 Jun 2024 07:37:33 +0000 (03:37 -0400)]
LU-17000 lnet: Initilize lnet_nidlist local variable
Initialize vairbale lnet_nidlist under
lustre_lnet_modify_peer(). This is due to a case
where "nids" is false lustre_lnet_mod_peer_nidlist()
is called with lnet_nidlist uninitialized.
Test-Parameters: trivial testlist=sanity-lnet
CoverityID 397609: ("Uninitialized scalar variable")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I07de6a637c4567daed4ce7b35c54a1eed4d9bd2f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55290
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 24 May 2024 17:32:17 +0000 (11:32 -0600)]
LU-17892 lnet: Fix export with empty nets
lnetctl export --backup should not print an error when there haven't
been any NIs added.
Test-Parameters: trivial
Fixes:
8f8f6e2f36 ("LU-10003 lnet: use Netlink to support old and new NI APIs.")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Id9e916d2d70d5dc01442e24449cb787c5a6a7e1d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55279
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Frederick Dilger [Mon, 27 May 2024 21:38:51 +0000 (17:38 -0400)]
LU-17416 utils: option for lctl get_param to skip links
Added new --links and --no-links options for 'lctl get_param' and
'lctl list_param' to avoid following symlinks. Useful when combined
with a command like "lctl get_param -R '*'" which can dump a lot of
duplicate data due to symlinks under lov.*.target_obds and
lmv.*.target_obds pointing back to their respective osc.* and mdc.*
trees. By default --links is enabled to for lctl get_param to
continue to operate as it did before this patch.
Additionally, long options have been added for all previous options
in {list, get, set}_param to update command options to current
standards. This will also facilitate adding new options in the future
as well as code maintenance and readability.
Signed-off-by: Frederick Dilger <fdilger@whamcloud.com>
Change-Id: I24115835f5045623f78fa2045dc3e0ce0b795316
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55236
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Mon, 27 May 2024 22:49:24 +0000 (01:49 +0300)]
LU-12706 tests: sanity-quota 4a sync timeout fix
Don't sync all OSTs in a system - this might take
too much time. Instead, set striping only on OST0000
and sync only MDTs and OST0000. This fix is against
the following failure:
FAIL: Passed grace time 20,
1566910527,
1566910563
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I525e6c73c6d14a126a2bde7d92bc28f11f3c78c8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55216
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log