Whamcloud - gitweb
Alex Zhuravlev [Tue, 9 Nov 2021 08:03:32 +0000 (11:03 +0300)]
LU-13494 osp: wait for import invalidation
osp_update_fini() should wait till racing ptlrpc_import_invalidate()
(running in a dedicated invalidation thread) is complete as the both
threads access ou_update and osp_update_fini() release the structure.
the change also fixes kernel's warning on scheduling while atomic:
old osp_update_fini() took the spinlock to protect ou_list, but
given new wait_event() we don't need to protect the list anymore.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iea40d3be8b1b3079b9fe8bdd015cc3392027b64d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45499
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Mon, 25 Oct 2021 17:27:15 +0000 (20:27 +0300)]
LU-15162 osd: improve OI lookup concurrency
replace inode->i_mutex with i_rwsem in osd_obj_map_lookup()
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Id8df20e00ae254ea4dcf4b10415e1927fac6bd44
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45353
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Courrier Guillaume [Thu, 1 Jul 2021 09:35:19 +0000 (11:35 +0200)]
LU-14834: list the UUIDs of stale clients during recovery
Add a new entry in debugfs for MDT and OSD to display the uuid of clients
yet to be reconnected during a recovery.
For example:
$ lctl get_param obdfilter.lustre-OST0000.recovery_stale_clients
obdfilter.lustre-OST0000.stale_clients=
9a7ab21d-207c-4680-b9bf-
b5873fd05540
This will display, during the recovery, the UUIDs of clients
that are expected to be connected.
Signed-off-by: Courrier Guillaume <guillaume.courrier@cea.fr>
Change-Id: Ib8c0b500adc9098e3cfb1998df06757a7d31b7b9
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/44196
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Mon, 14 Jun 2021 01:31:33 +0000 (11:31 +1000)]
LU-14760 llite: restart clio for AIO if necessary
If the clio needs to be restarted from where it left off,
do it for AIO as well, so we don't end up with short IO.
Limit thr number of retries to 1000, to avoid potential
issues if the loop is stuck forever.
Change-Id: Iccca31b032b01b940656864bfff22a821ff5061d
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/43995
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Hongchao Zhang [Sun, 31 May 2020 21:00:01 +0000 (05:00 +0800)]
LU-13654 target: print info of evicted clients to syslog
During recovery, the information of the evicted clients could be
useful for administrator to evaluate the affect of the recovery,
the debug log could be not easy to get in this case, then it is
better to dump it to syslog.
Change-Id: Ib32e10b1bbf12fe65be7862a018144827201e58a
Test-Parameters: trivial
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/38878
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Fri, 10 Dec 2021 11:44:09 +0000 (22:44 +1100)]
LU-14692 osp: deprecate IDIF sequence for MDT0000
Always return true for IDIF seq osp_fid_end_seq
so osp precreate will rollover to a new seq in
the FID_SEQ_NORMAL range for MDT0000.
Remove conf-sanity test_122b:
Check OST sequence wouldn't change when IDIF 32bit overflows
Change-Id: I85a0e38266331c96d971d68ec353949ccac3fc21
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45822
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Bobi Jam [Thu, 2 Mar 2023 09:39:01 +0000 (17:39 +0800)]
LU-16612 llite: protect cp_state with vmpage lock
cl_page_make_ready() calls cl_page_io_start() without vmpage lock
protection, and that could mess up cl_page's cp_state/cp_owner.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Id0df7e14246aa561494a9b6e581cebc55241c4b9
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50180
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Mon, 20 Feb 2023 17:27:52 +0000 (20:27 +0300)]
LU-16501 lod: add qos_ost_weights to debugfs
The patch adds files qos_ost_weights and qos_mdt_weights
at lod directory in debugfs. File qos_ost_weights would be
also added for each OST pool in a new directory lod/pool.
lod.<fsname>-MDT*-mdtlov.qos_mdt_weights
lod.<fsname>-MDT*-mdtlov.qos_ost_weights
lod.<fsname>-MDT*-mdtlov.pool.<pool>.qos_ost_weights
These files provide target and server weights, penalties and other
data needed to debug QOS allocator imbalance issues in YAML:
- { ost_idx: 0, tgt_weight: 1137680, tgt_penalty: 0,
tgt_penalty_per_obj: 115544, tgt_avail: 1137680,
tgt_last_used:
1677104866, svr_nid: 192.168.100.31@tcp,
svr_bavail: 2070560, svr_iavail: 1, svr_penalty: 0,
svr_penalty_per_obj: 52572, svr_last_used:
1677104866 }
Writing to qos_ost_weights/qos_mdt_weights would cause
resetting of tgt_weight, tgt_penaly and svr_penalty.
The patch also adds sanity_205f to check YAML output.
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I27e3f5abeb2f31b1c445658be035ec7e76c1572e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50074
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Wed, 15 Feb 2023 09:09:53 +0000 (17:09 +0800)]
LU-16263 lov: continue fsync on other OST objs even on -ENOENT
When fsync races with truncate, we'd continue to other OST object's
fsync even some stripe fsync returns -ENOENT, so that on client it
could potentially discard caching pages by calling
osc_io_fsync_start()->osc_cache_writebase_range().
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Test-Parameters: testlist=sanity ostcount=4 env=ONLY="273b 273c",ONLY_REPEAT=120
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I457ba80063086e310df55aaa22778b51a6ea211e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50005
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Qian Yingjin [Mon, 20 Feb 2023 03:11:54 +0000 (22:11 -0500)]
LU-16579 llite: fix the wrong beyond read end calculation
During the test, we found a dead loop in the read path which
retruns AOP_TRUNCATED_PAGE(0x8001) endless.
The reason is that the calculation of the ending beyond offset is
wrong: (iter->count + iocb->ki_pos).
The ending beyond offset was supposed to be not changed during
the read I/O loop for each page in buffered I/O mode.
However, @iter->count is decreased with read bytes when finished
the read of each page: @iter->count -= read_bytes.
In this patch, we store the ending beyond page index in
@lcc->lcc_end_index before call @generic_file_read_iter into a
loop for each read page and solve this bug.
Fixes:
2f8f38effa ("LU-16412 llite: check read page past requested")
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I5bb7ab82e5e2de8b9bd911798fb8ae65fc7c91af
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50065
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Tue, 7 Mar 2023 01:11:01 +0000 (18:11 -0700)]
LU-15542 tests: skip interop conf-sanity test_61b
Skip large_xattr test for old MDS without fix for large_xattr.
Minor cleanup to code style.
Test-Parameters: trivial
Test-Parameters: testlist=conf-sanity env=ONLY="61" serverversion=2.15.0
Fixes:
716de353b7 ("LU-15542 osd-ldiskfs: exclude EA inode from processing")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ieb30568b617177a9986a139b289ba1ced63ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50222
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sarah Liu <sarah@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andrew Perepechko [Sun, 11 Dec 2022 11:22:49 +0000 (14:22 +0300)]
LU-16381 ofd: fix soft_sync_limit lprocfs write handler
soft_sync_limit_store() must return the size of the input buffer
in the success case, otherwise the writer may loop forever.
Change-Id: I2d926eaba7062c495a56c6ee32e1b82e08df63ce
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49362
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Thu, 2 Sep 2021 15:21:38 +0000 (18:21 +0300)]
LU-14980 osd: check for locks in osd_trans_start()
since LU-10048 we shouldn't be starting a transaction with
object (osd) locks held.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia7d1de9351a23f8e0de52f3d5d0948f1e65529e7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/44822
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Zhuravlev [Thu, 2 Sep 2021 16:17:03 +0000 (19:17 +0300)]
LU-14980 mdd: mdd_layout_swap() to follow tx-lock rule
i.e. start transaction first, then take local (osd) locks
on the objects involved.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9a4add277f0911fa02d9b214e996c441d0952f9c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/44824
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Thu, 2 Sep 2021 18:24:04 +0000 (21:24 +0300)]
LU-14980 mdd: unlock object before changelog
we can't hold the object (osd) lock over transaction start
due to the locking rules. and we don't need the object
to be locked as only the fid is used at that point.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iaeb4645bfc9271d21d3644398c4c83f8e9b7aa04
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/44828
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Arshad Hussain [Fri, 3 Mar 2023 09:10:01 +0000 (14:40 +0530)]
LU-16614 lfs: Fix memory leak
Fix memory leak while running "lfs check all"
Before patch:
=============
$ valgrind --leak-check=full lfs check all
<snip>
==93768== LEAK SUMMARY:
==93768== definitely lost: 56 bytes in 2 blocks
==93768== indirectly lost: 282 bytes in 5 blocks
==93768== possibly lost: 0 bytes in 0 blocks
==93768== still reachable: 0 bytes in 0 blocks
==93768== suppressed: 0 bytes in 0 blocks
<snip>
After patch:
=============
$ valgrind --leak-check=full lfs check all
<snip>
HEAP SUMMARY:
in use at exit: 0 bytes in 0 blocks
total heap usage: 98 allocs, 98 frees, 294,999 bytes allocated
All heap blocks were freed -- no leaks are possible
<snip>
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ib39d1f89a9c6937668f2a7d515606c7df86ac1df
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50196
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Wed, 1 Mar 2023 20:55:05 +0000 (14:55 -0600)]
LU-16607 tests: Use real IPs for router tests
Use real IPs for some of the router tests in sanity-lnet. By this we
mean use IPs that are on the same network as the interface that is
configured for LNet. This decreases the time it takes to clean up
from some test cases because socklnd is able to tear down connections
more easily.
For example, running all of sanity-lnet in a single node config takes
approximately 912 seconds without this patch. With this patch it takes
190 seconds.
validate_gateway_nids() is also enhanced to check that each gateway
nid that is expected to be configured is present in the actual
yaml configuration.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I962c6a52c411f27ed7f0258e1840073da12f31f0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50175
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Tue, 7 Feb 2023 06:39:01 +0000 (00:39 -0600)]
LU-16535 ldiskfs: Add SUSE 15 SP5 server support
SUSE 15 SP5 server support needs an updated
ext4-mballoc-pa-free-mismatch.patch
for Linux v5.18 and later, as linux/genhd.h was removed
Test-Parameters: trivial
HPE-bug-id: LUS-11471
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4c54edda99e9871389d560de58c415fc8c0ff0a2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49923
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Tue, 7 Feb 2023 08:18:36 +0000 (02:18 -0600)]
LU-16534 build: Prefer timer_delete[_sync]
Linux commit v6.1-rc1-7-g9a5a30568697
timers: Get rid of del_singleshot_timer_sync()
Linux commit v6.1-rc1-11-g9b13df3fb64e
timers: Rename del_timer_sync() to timer_delete_sync()
Linux commit v6.1-rc1-12-gbb663f0f3c39
timers: Rename del_timer() to timer_delete()
Prefer timer_delete_sync() to del_singleshot_timer_sync()
Prefer timer_delete_sync() to del_timer_sync()
Prefer del_timer() to timer_delete()
Provide del_timer and del_timer_sync when
timer_delete[_sync] is not available
Test-Parameters: trivial
HPE-bug-id: LUS-11470
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4c946c315a83482dd0bd69e5e89f0302a67bf81c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49922
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaly Fertman [Thu, 2 Feb 2023 20:20:04 +0000 (23:20 +0300)]
LU-16527 llite: dir layout inheritance fixes
fixes for some minor problems:
- it may happen that the depth is not set on a dir, do not consider
depth == 0 as a real depth while checking if the root default is
applicable;
- setdirstripe util implicitely sets max_inherit to 3 for non-striped
dir when -i option is given but -c is not; at the same time 3 is the
default for striped dirs only;
- getdirstripe shows inherited default layouts with max_inherit==0,
whereas it has no sense anymore; the same for an explicitely set
default layout on a dir/root with max_inherit==0;
- getdirstripe hides max_inherit_rr when stripe_offset != -1 as it has
no sense and reset to 0, however it leads to user confusion;
HPE-bug-id: LUS-11090
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I65daaac76533fad44a88e1c5a8aad4467c9f7682
Reviewed-on: https://es-gerrit.dev.cray.com/161035
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49882
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaly Fertman [Thu, 18 Aug 2022 15:40:03 +0000 (18:40 +0300)]
LU-16526 lmv: module params for default dir layout
add module parameters to let the default ROOT layout to be flexible
enough to be disabled now and set to something arbitrary in future.
HPE-bug-id: LUS-11140
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I66cae2e0aa5be9c56552bcf0e8810f3f1b944836
Reviewed-on: https://es-gerrit.dev.cray.com/161058
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49878
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Wed, 22 Jun 2022 12:59:17 +0000 (18:29 +0530)]
LU-15626 tests: Fix "error" reported by shellcheck (5/5)
This patch fixes "error" issues reported by shellcheck
for file lustre/tests/test-framework.sh. This patch also
moves spaces to tabs.
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ic0d1046577dacf242787841b64319a6e206bae22
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49439
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Arshad Hussain [Wed, 22 Jun 2022 11:28:54 +0000 (16:58 +0530)]
LU-15626 tests: Fix "error" reported by shellcheck (2/5)
This patch fixes "error" issues reported by shellcheck
for file lustre/tests/test-framework.sh. This patch also
moves spaces to tabs.
Change-Id: Ib2bb4e864ef0032f2906bf7e0f9ad0542a8411b3
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49436
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Sergey Cheremencev [Fri, 24 Jun 2022 20:38:29 +0000 (23:38 +0300)]
LU-16341 quota: fix panic in qmt_site_recalc_cb
The panic occurred due to empty qit_lqes array after
qmt_pool_lqes_lookup_spec. Sometimes it is possible if
global lqe is not enforced. Return -ENOENT from
qmt_pool_lqes_lookup_spec if no lqes have been added.
It fixes following panic:
BUG: unable to handle kernel NULL pointer dereference at
00000000000000f8
...
RIP: 0010:qmt_site_recalc_cb+0x2ec/0x780 [lquota]
...
[
ffffa5564118fda0] cfs_hash_for_each_tight at
ffffffffc0c72c81 [libcfs]
[
ffffa5564118fe08] qmt_pool_recalc at
ffffffffc142dec7 [lquota]
[
ffffa5564118ff10] kthread at
ffffffffb45043a6
[
ffffa5564118ff50] ret_from_fork at
ffffffffb4e00255
Add test sanity-quota_14 that reproduces above panic
without the fix.
HPE-bug-id: LUS-11007
Change-Id: Ie51396269fae7ed84379bef5fc964cce789eba7c
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://es-gerrit.dev.cray.com/160828
Tested-by: Jenkins Build User <nssreleng@cray.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49241
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Mon, 27 Jun 2022 07:47:22 +0000 (03:47 -0400)]
LU-15971 uapi: add DMV_IMP_INHERIT connect flag
Add OBD_CONNECT2_DMV_IMP_INHERIT for implicit default LMV inherit.
Test-Parameters: trivial
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I49c217952df65461567c236790a49211a66a33d3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47788
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaliy Kuznetsov [Fri, 3 Mar 2023 13:17:52 +0000 (16:17 +0300)]
LU-5170 utils: Continue on error when multiple files requested
For lfs commands that accept multiple file arguments, processing
continues even if any of the files generates an error, instead
of immediately aborting. This follows the Unix convention of
attempting to process all requested files. Any relevant error
messages are reported as they are encountered.
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I1bcfc18d17c76505796b8e367224d01f48731d9f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/30663
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Deiter [Mon, 6 Mar 2023 13:59:46 +0000 (13:59 +0000)]
LU-16621 enc: file names encryption when using secure boot
Secure boot activates lockdown mode in the Linux kernel.
And debugfs is restricted when the kernel is locked down.
This patch moves file names encryption from debugfs to sysfs.
Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: I434714941ffac2a4694cabd33f613aef70933678
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50219
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Mon, 13 Feb 2023 19:38:23 +0000 (22:38 +0300)]
LU-16501 tgt: add qos debug
Add several debug lines for QOS allocator.
Patch also changes S_CLASS subsystem to S_LOV in
lu_tgt_desc_tgt.c thus it can be enabled to capture
only QOS debugging.
Test-Parameters: trivial
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I2dd53022d7f199e7c521bbf78acc4a8bf4abca51
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49977
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Wed, 1 Mar 2023 19:27:21 +0000 (13:27 -0600)]
LU-16606 lnet: lnet_parse_route uses wrong loop var
When looping over the gateways list, we're referencing the wrong
loop variable to get the gateway nid (ltb instead of ltb2).
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
3b76020810 ("LU-6142 lnet: use list_first_entry() in lnet/lnet subdirectory.")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Idbcc5e211fc8fd49831ba572805b60be511d0ffd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50173
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wei Liu [Tue, 7 Feb 2023 21:25:53 +0000 (13:25 -0800)]
LU-9341 tests: skip interop sanity test_27U
Skip sanity test_27U for MDS less than 2.15.51
append pool was added by LU-9341 and test_27U was
added by LU-15727
Fixes:
e2ac6e1eaa ("LU-9341 lod: Add special O_APPEND striping")
Fixes:
0396310692 ("LU-15727 lod: honor append_pool with default composite layouts")
Test-Parameters: trivial testlist=sanity env=ONLY=27U serverversion=2.14.0
Change-Id: I5c9d2ebf4e1e660fa60d845c9894409f0e96e01f
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49935
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Tue, 28 Feb 2023 18:30:32 +0000 (22:30 +0400)]
LU-15129 tests: sanity-quota_75_dom fix
oflag=sync causes dd to write page by page instead
of sending several pages in RPC. Furthermore, when
granted space becomes closer to soft_limit(i.e. over
9MB if soft_limit is 10MB), OST can not preacquire
space anymore. Also OST could acquire only requested
amount of space - see qmt_alloc_expand. Thus OST has
to send quota acquire request at MDT for each BRW request
from the client. Sometimes 20 seconds is not enough to write
10MB. Change oflag=sync with conv=fsync to reduce the number
of RPCs between the client and OST and between QSDs and QMT.
One fsync at close should help to avoid timeout failure.
Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota env=ONLY=75,ONLY_REPEAT=50
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Iad363fdc8a0984861055c295ea9cc3f23110fd9f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50164
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaly Fertman [Tue, 28 Feb 2023 01:45:15 +0000 (04:45 +0300)]
LU-16285 ldlm: BL_AST lock cancel still can be batched
The previous patch makes BLAST locks to be cancelled separately.
However the main problem is flushing the data under the other batched
locks, thus still possible to batch it with those with no data.
Could be optimized for not yet CANCELLING locks only, otherwise it is
already in the l_bl_ast list.
Fixes:
b65374d9 ("LU-16285 ldlm: send the cancel RPC asap")
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Change-Id: Ie4a7c7f3e0f5462290f72af7c3b2ff410a31f5e7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50158
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Tue, 28 Feb 2023 01:38:22 +0000 (09:38 +0800)]
LU-16555 obdclass: quote jobid with ':' char
quote jobid with ':' char even though ':' is printed literally.
Fixes:
12604ac4de (LU-16555 obdclass: print more special chars for jobid)
Test-Parameters: trivial
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I8bf642180b00f068921598c3a0ea30b4f5b99c1e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50150
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Serguei Smirnov [Mon, 27 Feb 2023 23:41:19 +0000 (15:41 -0800)]
LU-14668 lnet: add 'force' option to lnetctl peer del
Add --force option to 'lnetctl peer del' command.
If the peer has primary NID locked, this option allows
for the peer to be deleted manually:
lnetctl peer del --prim_nid <nid> --force
Add --prim_lock option to 'lnetctl peer add' command.
If specified, the primary NID of the peer is locked
such that it is going to be the NID used to identify
the peer in communications with Lustre layer.
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ia6001856cfbce7b0c3288cff9b244b569d259647
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50149
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 24 Feb 2023 19:15:40 +0000 (12:15 -0700)]
LU-16591 build: use vendor tag for LUSTRE_FIX
Improve the regexp for the LUSTRE_FIX (fourth) component of the
LUSTRE_VERSION parsing so that if there is an additional component
in the version like "2.15.2-abc123" the "123" will be reported as
the fourth component of the version in obd_connect_data.ocd_version.
If there is already a fix version specified (e.g. 2.15.2.4-abc123)
then it will continue to be used for LUSTRE_FIX instead of the extra
"-abcNN" value.
The build version is shown in "{mdt,obdfilter}.*.exports.*.export"
param on servers for connected clients, and in "{mdc,osc}.*.import"
param on clients for connected servers. Displaying the full version
improves debuggability of remote peers to know their specific build
instead of showing the first three digits and always ".0" at the end.
Since ocd_version is a numeric field it is not possible to include
the "abc" part of the version string on the peer nodes.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I96b38f99b2522c5ea3f3b3e2ddd7cd64f1ce7057
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50136
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Mon, 9 May 2022 01:34:05 +0000 (11:34 +1000)]
LU-10360 ldlm: remove client_import_find_conn()
This function hasn't been used since Commit
37be05eca3f4 ("LU-10360
mgc: Use IR for client->MDS/OST connections").
So remove it.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idd68c2de1aba914e9017e9e8c10fbbe869ea5b26
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50000
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Timothy Day [Sun, 19 Feb 2023 21:21:09 +0000 (21:21 +0000)]
LU-16518 ptlrpc: fix clang build errors
Fixed bugs which cause errors on Clang.
The majority of changes involve adding
defines for the 'ptlrpc_nrs_ctl' enum.
This avoids having to explicitly cast
enums from one type to another.
An unused variable 'req' was removed from
'nrs_tbf_req_get'. A 'strlcpy' in
'sptlrpc_process_config' was copying the
wrong number of bytes. Another variable,
'rc' in 'sptlrpc_lproc_init', seemed to
be neglected unintentionally; this was also
fixed.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If994c625199b392198f944f9cd21bbf2142bce69
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49859
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Wed, 22 Dec 2021 12:26:49 +0000 (07:26 -0500)]
LU-14824 test: sanity 413a/b unlink timeout v2
Unlinking remote/striped directories is slow on ZFS system, limit
total directory number for 1-stripe directory test in 413a/b on ZFS
system, and don't test striped directory to avoid timeout.
Also limit total stripe object count to avoid timeout.
Use fallocate to fill the MDTs if it is supported. Add a new helper
function check_fallocate() that just determines if fallocate support
is available on the OST without trying to change the mode. This is
cached across calls to avoid repeated SSH calls to get information
that is the same for the entire test run.
Test-Parameters: trivial
Test-Parameters: mdscount=2 mdtcount=4 fstype=ldiskfs testlist=sanity env=ONLY="413a 413b",ONLY_REPEAT=50
Test-Parameters: mdscount=2 mdtcount=4 fstype=zfs testlist=sanity env=ONLY="413a 413b",ONLY_REPEAT=50
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ie116e6df5aee3877ed9f093f58e7bd71f63ebbe5
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49799
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Wed, 22 Jun 2022 09:57:53 +0000 (15:27 +0530)]
LU-15626 tests: Fix "error" reported by shellcheck (1/5)
This patch fixes "error" issues reported by shellcheck
for file lustre/tests/test-framework.sh. This patch also
moves spaces to tabs.
Change-Id: I2ef9e79856f3c86d71e5078c78ae309d48a9d71f
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49435
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Cyril Bordage [Thu, 29 Sep 2022 15:19:19 +0000 (17:19 +0200)]
LU-15944 lnet: remove crash with UDSP
The following sequence of commands caused a crash:
# lnetctl udsp add --dst tcp --prio 1
# lnetctl discover 192.168.122.60@tcp
Pointer to lnet_peer_net in udsp_info is checked before used.
Comments about syntax of "lnetctl udsp" command were updated.
Test-Parameters: trivial testlist=lnet-selftest,sanity-lnet
Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Change-Id: Ie3ae40b184e22627655e7f3813c5d16d38a6cfb8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48801
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Tue, 21 Feb 2023 16:37:56 +0000 (10:37 -0600)]
LU-16575 lnet: memory leak in copy_ioc_udsp_descr
copy_ioc_udsp_descr() doesn't correctly handle the case where a
net number was not specified. In this case, there isn't any net
number range that needs to be copied into the udsp descriptor.
Test-Parameters: trivial testlist=sanity-lnet env=ONLY=400
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ica900e2b0ec816237a8303d1d9e07cc1f6c5a652
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50081
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 11 Oct 2022 11:59:48 +0000 (14:59 +0300)]
LU-14958 kernel: use rhashtable for revoke records in jbd2
resizable hashtable should improve journal replay time when
the latter has got million of revoke records. notice that
rhashtable is used during replay only as removal with list_del()
is less expensive and it's used a lot during regular processing.
before:
1048576 records - 95 seconds
2097152 records - 580 seconds
after:
1048576 records - 2 seconds
2097152 records - 3 seconds
4194304 records - 7 seconds
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9a9e3801223fa9e36cbf6d2ef5ddbad5dff3e19d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45122
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Alex Zhuravlev [Tue, 14 Sep 2021 09:50:24 +0000 (12:50 +0300)]
LU-15004 osd: remove wrong assertion on oo_dn
which is invalid since LU-14531 landing. instead we just
check that the object does exist and return an error otherwise.
all this is serialized against object destroy.
Fixes:
51350e9b738d("LU-14531 osd: serialize access to object vs object destroy")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Icff39987f4f8d5a8227c6e3b829b58979c1b1941
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/44911
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Amir Shehata [Thu, 6 May 2021 06:02:22 +0000 (23:02 -0700)]
LU-14668 lnet: don't delete peer created by Lustre
Peers created by Lustre have their primary NIDs locked.
If that peer is deleted, it'll confuse lustre. So when manually
deleting a peer using:
lnetctl peer del --prim_nid ...
We must continue to preserve the primary NID. Therefore we delete
all the constituent NIDs, but keep the primary NID. We then
flag the peer for rediscovery.
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I34eef9b0049435a01fde87dc8263dd50f631c551
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/43565
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Chris Horn [Tue, 25 May 2021 16:17:49 +0000 (11:17 -0500)]
LU-14668 lnet: Peers added via kernel API should be permanent
The LNetAddPeer() API allows Lustre to predefine the Peer for LNet.
Originally these peers would be temporary and potentially re-created
via discovery. Instead, let's make these peers permanent. This allows
Lustre to dictate the primary NID of the peer. LNet makes sure this
primary NID is not changed afterwards.
Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I3f54c04719c9e0374176682af08183f0c93ef737
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/43788
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Amir Shehata [Wed, 5 May 2021 18:35:06 +0000 (11:35 -0700)]
LU-14668 lnet: Lock primary NID logic
If a peer is created by Lustre make sure to lock that peer's
primary NID. This peer can be discovered in the background.
There is no need to block until discovery is complete, as Lustre
can continue on with the primary NID it provided.
Discovery will populate the peer with other interfaces the peer has
but will not change the peer's primary NID. It can also delete
peer's NIDs which Lustre told it about (not the Primary NID).
If a peer has been manually discovered via
lnetctl discover <nid>
command, then make sure to delete the manually discovered
peer and recreate it with the Lustre NID information
provided for us.
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I8fc8a69caccca047e3085bb33d026a3f09fb359b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50106
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Shaun Tancheff [Thu, 2 Feb 2023 03:47:15 +0000 (21:47 -0600)]
LU-13743 build: Explicitly require encryption support
Linux commit v5.18-rc5-17-gb1241c8eb977
ext4: move ext4 crypto code to its own file crypto.c
Update the ldiskfs Makefile to exclude crypto.c when
CONFIG_FS_ENCRYPTION is not enabled.
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ic8a40f3d395286bb52ed20693fd7cc4755b10556
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/39243
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Shaun Tancheff [Fri, 27 Jan 2023 07:08:23 +0000 (01:08 -0600)]
LU-16509 lnet: quash memcpy WARN_ONCE false positive
Linux v6.1-rc1-4-g6f7630b1b5bc
fortify: Capture __bos() results in const temp var
In lnet_peer_push_event() the memcpy triggers a WARN_ONCE
due to the flexible array at the end of
struct lnet_ping_info contained in struct lnet_ping_buffer
Use unsafe_memcpy() to avoid this false positive warning.
Test-Parameters: trivial
HPE-bug-id: LUS-11455
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4aa8f38678cd1522004d98b58a3f440d8a38589c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49801
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Etienne AUJAMES [Wed, 22 Feb 2023 10:37:49 +0000 (11:37 +0100)]
LU-16571 utils: fix parallel "lfs migrate -b" on hard links
Multiple blocking "lfs migrate" on the same file can exhaust "ost"
service threads of an OSS CPT.
llapi_get_data_version(...,LL_DV_RD_FLUSH) causes the OSS server to
take a server-side extent lock PR to force clients with write lock to
update the data version of the object.
migrate_block() (lfs.c) checks the file data version is check with
LL_DV_RD_FLUSH before taking the group lock.
So "ofd_getattr_hdl()" server side lock will conflict with the lfs
instance that has the group lock.
Each attempt to get server-side extent lock will take an "ost" service
thread slot waiting the group lock to be released.
If all threads of the "ost" servive are exhausted on a CPT, the OSS
can not handle requests from the client and it will get queued inside
the NRS policy. This causes the lfs process with the group lock to
hang (pread needs "ost" service to get sizes of objects).
This patch check the file data version inside the group lock without
LL_DV_RD_FLUSH. This flag is not needed, the client already has an
extent group lock on all the OST objects.
Add the regression test sanity 56xj.
Test-Parameters: testlist=sanity env=ONLY=56xj,ONLY_REPEAT=20
Test-Parameters: testlist=sanity env=ONLY=56
Test-Parameters: testlist=sanity env=ONLY=56
Test-Parameters: testlist=sanity env=ONLY=56
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I0bacd372dd6f36a4ac776133dff45dc836c7c7f7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50113
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Deiter [Tue, 21 Feb 2023 22:27:47 +0000 (02:27 +0400)]
LU-16585 build: remove python2 dependencies
Fixed packaging issue casued by scripts and control files.
Test-Parameters: trivial
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: I6c9b24bf811269928494af17c15627902e5fe27b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50084
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Mon, 13 Feb 2023 09:07:45 +0000 (12:07 +0300)]
LU-16557 client: -o network needs add_conn processing
Mount option -o network restricts client import to use
only selected network. It processes connection UUID/NIDs
during 'setup' config command handling but skips any
'add_conn' command if its UUID has no mention about that
network. Meahwhile connection UUID is just a name and may
have many NIDs configured including those on restricted
network which are skipped as well. Therefore client import
configuration misses failover NIDs on restricted network.
Patch makes import to save restricted network information
after 'setup' command processing, so it is applied to any
client_import_add_conn() call. The 'add_conn' command is
always processed now and its NIDs will be filtered in the
same way as for 'setup'.
Test 31 in sanity-sec.sh is extended to check imports
failover_nids has all and only NIDs on restricted network
Test-Parameters: env=ONLY=31 testlist=sanity-sec
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id70ebd836f061f154e3779b07b52f1baea9a1776
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49986
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Fri, 17 Feb 2023 03:38:29 +0000 (22:38 -0500)]
LU-16568 lfs: call Parser_exit() at lfs/lctl exit
Call Parser_exit() before lfs and lctl cleanly exit
to free memory allocated in Parser_init() via strdup().
Test-Parameters: trivial fstype=zfs testlist=sanity
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I1bc86d4b17f62a545e51fb3e479b2576e6362c42
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50039
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Timothy Day [Wed, 8 Feb 2023 17:27:16 +0000 (17:27 +0000)]
LU-16553 utils: cleanup lfs options
The enums for lfs long options should start
after the char range. This is supported
by getopt_long and used by lustre/test/statx.c.
I didn't see this issue in any of the other
uses of getopt_long, so this is the only place
this fix needs to be made.
Also, enable stats when stats_interval is used.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I9144b2d68291811edf95b9d912fbbb8fe0266392
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49992
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Mon, 6 Feb 2023 20:02:15 +0000 (20:02 +0000)]
LU-16518 misc: use fixed hash code
There is a configure check to avoid using
broken hashing code. All calls to 'hash_*'
are replace by the 'cfs_hash_*' equivalents,
to make use of this check.
Two functions which hash then apply a mask
are removed. The calls are replaced with
'cfs_hash_*' and manually applying the mask.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia4b27cb6fb1329b9df45c00f748a8d22178b0654
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49916
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Xinliang Liu [Mon, 21 Nov 2022 03:36:38 +0000 (03:36 +0000)]
LU-16481 build: add server support for openEuler
openEuer uses dnf as rpm pkg manager, it is somewhat like RHEL/Fedora.
The current openEuler LTS 22.03 kernel is based on Linux 5.10.0.
Ldiskfs patches based on ldiskfs-5.10.0-ml.series, different patches
compared with ldiskfs-5.10.0-ml.series are:
oe2203/ext4-misc.patch
oe2203/ext4-pdirop.patch
use due to openEuler kernel backport new bugfixes and
based on ldiskfs-5.14.21-sles15sp4.series
linux-5.16/ext4-inode-version.patch
ubuntu20.04.3/ext4-simple-blockalloc.patch
linux-5.14/ext4-xattr-disable-credits-check.patch
use due to openEuler kernel backport new bugfixes.
This patch also fixes lbuild that no need a kernel config file for
patchless-server build. And add patched-server build needs an series
patches checking.
Test notes
----------
This patch is tested with below lbuild cmd:
../lustre-release/contrib/lbuild/lbuild --ccache
--kerneldir=/home/openeuler/kernel-src-rpm/
--kernelrpm=/home/openeuler/kernel-src-rpm/
--lustre=/home/openeuler/lustre-release/lustre-2.15.54_1_xxx.tar.gz
--patchless-server --disable-zfs
Note that, due to zfs openEuler build support patches[1] haven't been
backported to the stable release branch zfs-2.1-release and tag 2.1.5,
current lbuild doesn't support zfs rpms build for openEuler you need
to build zfs rpms in the zfs source code individually with cmd 'make
rpms'.
And until the openEuler gcc issue[2] is fixed, or you need to apply
Lustre rpm spec patch[3].
Until the openEuler kernel symbols providing issue[4] is fixed, or you
need to install kmod rpms with cmd 'sudo rpm -ivh --nodeps
./*.aarch64.rpm '
[1] https://github.com/openzfs/zfs/pulls?q=is%3Apr+is%3Aclosed+openeuler
[2] https://gitee.com/openeuler/gcc/issues/I5XMD0
[3] diff lustre.spec.in
...
-%define optflags -g -O2 -Werror
+%define optflags -g -O2 -Werror -Wno-stringop-overflow
[4] https://gitee.com/src-openeuler/kernel/issues/I6DQDX
Test-Parameters: trivial
Change-Id: Ie00e7d37ba3965e409b924109085a675bf3f7f4f
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49652
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Wed, 29 Jun 2022 15:51:47 +0000 (11:51 -0400)]
LU-15971 llite: match lock in corresponding namespace
For remote object, LOOKUP lock is on parent MDT, so lmv_lock_match()
iterates all MDT namespaces to match locks. This is needed in places
where only LOOKUP ibit is matched, and the lock namespace is unknown.
Test-Parameters: mdscount=2 mdtcount=4 testlist=sanityn env=ONLY=109,ONLY_REPEAT=50
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I32767f63f92a4df825ddacbf435bda8cfcfbbdd7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47843
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 16 Feb 2023 16:21:50 +0000 (09:21 -0700)]
LU-12275 sec: remove bio functions in fscrypt compat
Remove libcfs/llibcfs/crypto/bio.c since direct block device access
is not needed for client builds, and the use of stuct bio on the
client adds unnecessary complexity to portability.
Test-Parameters: trivial
Fixes:
a813e8187 ("LU-12275 sec: add llcrypt as file encryption library")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I97642dfd85053b9ea4196374f2002ffb6a2540e5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50023
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaliy Kuznetsov [Sat, 18 Feb 2023 08:28:37 +0000 (11:28 +0300)]
LU-16558 mdt: Fix max limit for "max_mod_rpcs_in_flight"
This minor fix fixes a bug with the definition of the maximum
limit for the mdt_max_mod_rpcs_in_flight parameter when it is
changed.
Fixes:
f16c31ccd9 ("LU-16454 mdt: Add a per-MDT "max_mod_rpcs_in_flight")
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I5576da1dbcaa0b4202af4b02023a46991f443a4b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50010
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Wed, 15 Feb 2023 10:42:25 +0000 (10:42 +0000)]
LU-15728 mdd: fix sanity-56oc failure
sanity 56oc starts failing since relatime patch was landed.
'relatime' patch introduced an atime behavior change. It was forcing
atime uptime to disk on MDD if ondisk atime is older than ondisk
mtime/ctime to match relatime (even if relatime was not enabled).
This was an optimization, trying to have a slightly better atime value
cheaply. This is unfortunately causing regression in sanity-56oc.
Let's remove it for now until we understand that better.
Fixes: c10c6ee ("LU-15728 llite: fix relatime support")
Test-Parameters: testlist=sanity env=ONLY=56oc,ONLY_REPEAT=70
Change-Id: Ieed4d4c7523c26cfc5bc230986d96b2acf152dee
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50009
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Wed, 15 Feb 2023 03:56:36 +0000 (11:56 +0800)]
LU-16555 obdclass: print more special chars for jobid
Print more YAML compatible special chars for jobid.
Currently they are any of ".@-_:/".
Test-Parameters: trivial
Fixes:
338381574b ("LU-11407 tgt: cleanup job_stats output printing")
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Ic3272b73c95b76ad3171cc7a368a18f804b9aa3e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49998
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Mon, 13 Feb 2023 21:15:31 +0000 (15:15 -0600)]
LU-16551 tests: Ensure all peer credits used in MR
sanity-lnet test_254 needs to ensure that all peer credits are
consumed. Because of the raciness of the round robin code in LNet,
we cannot rely on just issuing the appropriate number of pings.
Instead we should use the --source argument to lnetctl ping to ensure
that we send the correct number of pings over each interface.
To simplify matters, only perform this test, and the other tests that
call setup_health_test(), in non-routed configurations.
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
52db11cdce ("LU-16303 lnet: Drop LNet message if deadline exceeded")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I05a7ffec37d16c14711fe696232708f927357b1c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49979
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sat, 11 Feb 2023 00:09:13 +0000 (16:09 -0800)]
LU-16544 kernel: kernel update RHEL 7.9 [3.10.0-1160.83.1.el7]
Update RHEL 7.9 kernel to 3.10.0-1160.83.1.el7.
Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I8f2e67c27e80e7ac852fc367f0bd83f2c91016ce
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49969
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 10 Feb 2023 23:48:01 +0000 (15:48 -0800)]
LU-16486 kernel: kernel update RHEL8.7 [4.18.0-425.10.1.el8_7]
Update RHEL8.7 kernel to 4.18.0-425.10.1.el8_7.
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Change-Id: I5759d0cb06a1148689ed9b8c947cb6516ab3aca1
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49683
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Mon, 12 Dec 2022 04:47:31 +0000 (15:47 +1100)]
LU-16382 spec: improve Summary and description.
Summary should not repeat the name of the package, and should avoid
being overly long. It should start with CAPS and not end with period.
Description should be more detailed than the summary.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ifcdb881f649200e1857c707fb2c589579e4035a4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49367
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
James Simmons [Tue, 7 Feb 2023 15:08:13 +0000 (10:08 -0500)]
LU-16463 llite: replace lld_nfs_dentry flag with opencache handling
The lld_nfs_dentry flag was created for the case of caching the
open lock (opencache) when fetching fhandles for NFSv3. This same
path is used by the fhandle APIs. This lighter open changes key
behaviors since the open lock is always cached which we don't
want. Lustre introduced a way to modify caching the open lock
based on the number of opens done on a file within a certain
span of time. We can replace lld_nfs_dentry flag with the
new open lock caching. This way for fhandle handling we match
the open lock caching behavior of a normal file open.
In the case of NFS this code path will always be called with the
internal kernel thread 'nfsd'. If we are called by this kernel
thread set the open threshold to zero which means always cache the
open lock. Once Lustre is only supported on Linux kernels above
5.5 we can remove this special NFSv3 work around.
Change-Id: Iba27f7ad4579fdd1f34e1e35c2cbd547e15f129a
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49237
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Thu, 2 Feb 2023 01:39:03 +0000 (09:39 +0800)]
LU-16523 lprocfs: adjust the format of rename_stats
Adjust the format of rename_stats to a more human-friendly YAML.
Fixes:
bedb797c5d ("LU-16110 lprocfs: make job_stats and rename_stats valid YAML")
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I20e6d07c974e907bb2e30412dd1899f845de2021
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49869
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
James Simmons [Wed, 8 Feb 2023 13:04:45 +0000 (08:04 -0500)]
LU-15420 sec: handle simple fscrypt changes for 5.15 kernels
The patch covers the low impact changes to the Linux kernels
fscrypt API. The changes are:
For Linux kernel version 5.9-rc4 the following commits are:
8b10fe68985278de4926daa56ad6af701839e40a removed the inode
parameter for the fscrypt function fscrypt_fname_alloc_buffer()
5b2a828b98ec1872799b1b4d82113c76a12d594f ended up exporting
fscrypt_d_revalidate() and stopped stomping on the d_ops.
c8c868abc91ff23f6f5c4444c419de7c277d77e1 changed
fscrypt_set_test_dummy_encryption() take a 'const char *
ac4acb1f4b2b6b7e8d913537cccec8789903e164 moved the fscrypt core
from using fscrypt_context to using fscrypt_policy that is user
forward facing.
Lastly for Linux kernel version 5.10-rc4 the commit
ec0caa974cd092549ab282deb8ec7ea73b36eba0 stopped exporting
fscrypt_get_encryption_info(). Use fscrypt_prepare_readdir()
in its place.
70fb2612aab62d47e03f82eaa7384a8d30ca175d renamed a field in
struct fscrypt_name. Since Lustre can't use fscrypt_prepare_lookup
we have to deal with this change.
Remove sptlrpc_enc_pool_del_user() since its an empty function
that waste cycles calling it.
The other large change was the replacement of
fscrypt_inherit_context which is described in Linux commit
a992b20cd4ee360dbbe6f69339cb07146e4304d6. This change is very
large since it expects the target inode to be available. Lustre
uses fscrypt_inherit_context before the inode is available so
this is a much more complex change that will be done in another
patch.
de3cdc6e75179a2324c23400b21483a1372c95e1 makes fscrypt_require_key
private. You need to test the key's presence with
fscrypt_has_encryption_key() instead but that key needs to be
setup first by the new function fscrypt_prepare_new_inode().
Since this is the case we wait to introduce this change.
Change-Id: I4bed7fef6e3302c0258c0f1563f4e180258d7a5a
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49125
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sun, 12 Feb 2023 07:48:30 +0000 (23:48 -0800)]
LU-16101 tests: add sanity/27J to always_except
This patch adds sanity/27J to always_except for SLES15 SP4
and 5.16.0+ kernels before the issue introduced by upstream
commit
8c8387ee3f55
("mm: stop filemap_read() from grabbing a superfluous page")
is resolved.
Test-Parameters: trivial clientdistro=sles15sp4 testlist=sanity
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Iafde656530fcdc1de9265aacaa9266435c9d5c47
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49970
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Xing Huang <hxing@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Thu, 24 Nov 2022 21:51:58 +0000 (16:51 -0500)]
LU-14719 lod: ignore space check error in recovery
statfs may fail in recovery, ignore this error in
lod_trans_space_check().
Fix syntax error in replay-single 111g version check.
Fixes:
6aee406c84 ("LU-14719 lod: distributed transaction check space")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I6c7934ca242a639d996d0ab5a4d7648cec8a53de
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49249
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Etienne AUJAMES [Wed, 1 Dec 2021 23:22:33 +0000 (00:22 +0100)]
LU-15280 llog: fix processing of a wrapped catalog
Several issues were found with "lfs changelog --follow" for a wrapped
catalog (llog_cat_process() with startidx):
1/ incorrect lpcd_first_idx value for a wrapped catalog (startcat>0)
The first llog index to process is "lpcd_first_idx + 1". The startidx
represents the last record index processed for a llog plain. The
catalog index of this llog is startcat.
lpcd_first_idx of a catalog should be set to "startcat - 1"
e.g:
llog_cat_process(... startcat=10, startidx=101) means that the
processing will start with the llog plain at the index 10 of the
catalog. And the first record to process will be at index 102.
2/ startidx is not reset for an incorrect startcat index
startidx is relevant only for a startcat. So if the corresponding llog
plain is removed or if startcat is out of range, we need to reset
startidx.
This patch remove LLOG_CAT_FIRST, that was really confusing
(LU-14158). And update osp_sync_thread() with the
llog_cat_process() corrected behavior.
It modifies also llog_cat_retain_cb() to zap empty plain llog directly
in it (like for llog_cat_size_cb()), the current implementation is not
compatible with this patch.
The test "conf-sanity 135" verify "lfs changelog --follow" for a
wrapped changelog_catalog.
Test-Parameters: testlist=conf-sanity env=ONLY=135,ONLY_REPEAT=10
Test-Parameters: testlist=sanity env=ONLY=60a,ONLY_REPEAT=20
Test-Parameters: testlist=conf-sanity env=SLOW=yes,ONLY=106,ONLY_REPEAT=10
Fixes:
a4f049b9 ("LU-13102 llog: fix processing of a wrapped catalog")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Iaf46ddd4a6ec1e06cec0d17aa9bde766bd793abc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45708
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Li Dongyang [Tue, 23 Nov 2021 23:45:48 +0000 (10:45 +1100)]
LU-11912 fid: clean up OBIF_MAX_OID and IDIF_MAX_OID
Define the OBIF|IDIF_MAX_OID macros to 1ULL << OBIF|IDIF_MAX_BITS - 1
Clean up the callers and remove OBIF|IDIF_OID_MASK which are not used.
Test-Parameters: trivial
Change-Id: I9a679b930c73da5904b2eb4c74f785fc1d27a8a0
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45659
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 7 Feb 2023 09:29:24 +0000 (12:29 +0300)]
LU-16536 osp: don't cleanup ldlm in precleanup phase
instead do this in cleanup phase so that all OSPs have chance
to abort in-flight RPCs which can block MDT thread holding
LDLM locks.
Fixes:
226fd401f9 ("LU-7660 dne: support fs default stripe")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib3714b29c514a7fa938d47717dc36525654407d6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49925
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 19 Jan 2023 14:14:21 +0000 (17:14 +0300)]
LU-16493 tests: recovery-small/144b to wait longer
when ZFS is used as ZFS's primitives like osd_write()
and osd_declare_write() are still few times slower
compared to ldiskfs and this 144b creating wide-striped
files which causes lots of tiny writes to the last-used
file.
Test-Parameters: trivial testlist=recovery-small
Test-Parameters: testlist=recovery-small fstype=zfs env=ONLY=144b,ONLY_REPEAT=100
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib023c7c71a3bb486f9f9908e4cc03cc6e53ace7a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49693
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Patrick Farrell [Mon, 6 Feb 2023 23:22:24 +0000 (18:22 -0500)]
LU-16515 clio: Remove cl_page_size()
cl_page_size() is just a function which does:
1 << PAGE_SHIFT
and the kernel provides a macro for that - PAGE_SIZE.
Maybe it didn't when this function was added, but it sure
does now.
So, remove cl_page_size().
Test-parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6c27f6db7cfec5d9054aab95beccffe3c2da02bb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49918
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 12 Jan 2023 09:18:48 +0000 (10:18 +0100)]
LU-16532 sec: session key bad keyring
At initialization, the session key created for GSS context is linked
to a keyring from userpace, and then unlinked from kernelspace if it
is for root, as we want to share it across all root sessions.
Sometimes initialization fails (expired token, unresponsive server,
etc.) so the key cannot be unlinked. Survive this use case gracefully.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8ad2afa0e51e50640620e36211e5db1253d85e08
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49909
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 3 Feb 2023 08:17:16 +0000 (02:17 -0600)]
LU-16520 build: Move strscpy to libcfs common header
Ensure strscpy is available to lustre
Test-Parameters: trivial
Fixes:
0b406c91d17 ("LU-13642 lnet: modify lnet_inetdev to work with large NIDS")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0c3673c2aa7e6b61671521a8cabde8a364f7f6f8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49863
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 21 Jan 2023 01:45:05 +0000 (01:45 +0000)]
LU-16502 lutf: cleanup lutf_start.py, fix bugs
Remove code duplication by adding __check_env_var
method. Use os.environ.get to remove needlessly
verbose try-except blocks. Use __cfg_yaml more,
rather than passing this value explicitly
between methods.
Add check for environment variables, so LUTF
can fail gracefully if the environment is not
set correctly.
Update .gitignore to ignore __pycache__
Fix syntax error in python/infra/lutf.py
Test-Parameters: @lnet
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I16cf114ac4253d22a42399e2f2cb2fad49dd96cb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49767
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 21 Jan 2023 01:40:13 +0000 (01:40 +0000)]
LU-16502 lutf: fix bugs in bash scripts
Addressed some issues I found when running
LUTF. The "rm" fails without a file to remove.
A Lustre wiki led me to source the script, but this
will log out of the current shell. Adding a warning
against doing this.
Also, fix shellcheck errors and a few warnings in
LUTF related scripts. Added bash shebangs, since
shellcheck requires these to lint the scripts.
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I501d58d25bfcd6564755485b9a1afa2277848b96
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49728
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Sebastien Buisson [Thu, 19 Jan 2023 17:07:27 +0000 (18:07 +0100)]
LU-16494 fileset: check fileset for operations by fid
Some operations by FID, such as lfs rmfid, must be aware of
subdirectory mount (fileset) so that they do not operate on files
that are outside of the namespace currently mounted by the client.
For lfs rmfid, we first proceed to a fid2path resolution. As fid2path
is already fileset aware, it fails if a file or a link to a file is
outside of the subdirectory mount. So we carry on with rmfid only
for FIDs for which the file and all links do appear under the
current fileset.
This new behavior is enabled as soon as we detect a subdirectory mount
is done (either directly or imposed by a nodemap fileset). This means
the new behavior does not impact normal, whole-namespace client mount.
sanity test_421h is added to exercise this new capability.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I47136ac0a3324b9afdd01b0f902abc37938bd361
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49696
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Akash B [Tue, 17 Jan 2023 15:51:19 +0000 (10:51 -0500)]
LU-16479 utils: Add option to manage degraded ZFS OST
Add new Lustre specific ZFS dataset user property to
control/manage degraded ZFS OSTs, also modify the existing
lustre/scripts/statechange-lustre.sh zedlet accordingly.
Extend the same to mkfs.lustre utility to add this property
by default when creating a new Lustre ZFS server.
HPE-bug-id: LUS-11447
Test-Parameters: trivial fstype=zfs testlist=sanity
Signed-off-by: Akash B <akash-b@hpe.com>
Change-Id: I7032538f507c9ad20d5b109b54e3c3bab8138458
Reviewed-by: Dipak Ghosh <dipak.ghosh@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49660
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Siddarth Raj <siddarth.raj@hpe.com>
Reviewed-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Andreas Dilger [Fri, 23 Dec 2022 00:28:14 +0000 (17:28 -0700)]
LU-16428 tests: cache is_project_quota_supported result
Rather than is_project_quota_supported() repeatedly checking if the
MDS supports project quota, check the MDS once after mount and then
cache it for the rest of the test run.
In resetquota() there is no need to check is_project_quota_supported()
each time, since this is only called with "-p" when project quota was
previously checked and is enabled. Also, there is no need to wait 1s
after every quota limit is reset, but rather only once at the end.
Test-Parameters: trivial testlist=sanity-quota mdtcount=4 mdscount=2
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id98240e2fcd2e862bb4305961a2946227e3ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49499
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Mon, 12 Dec 2022 04:42:11 +0000 (15:42 +1100)]
LU-16382 spec: use pkgconfig() as appropriate.
pkgconfig() is preferred over explicit dependencies on libfoo-devel.
For SUSE, this is particularly needed for system and with SLE15-SP4
the OBS automatically includes "systemd-mini", but that conflicts with
"system". Usig "pkgconfig(systemd) resolves this.
Also there is no need to have "Depends" for library packages. These
are determined automatically from the result of the build.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idfa4fdfd8bf060175b64a1991d3367024a368344
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49366
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Mr NeilBrown [Mon, 12 Dec 2022 04:25:31 +0000 (15:25 +1100)]
LU-16382 spec: Don't include Group: tags.
Fedora Project has deprecated group tags:
https://fedoraproject.org/wiki/RPMGroups
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_tags_and_sections
The groups tags currently used are not recognised by SUSE.
So remove all the Group: tags - except one.
The %kernel_module_package macro for SUSE requires that a group
be given.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6b7222200ea1a02319a703d64542dfb9780c048a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49364
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 7 Apr 2021 19:37:42 +0000 (12:37 -0700)]
LU-14224 misc: add firewalld service configuration
RHEL8 ships with restrictive firewalld rules out of the box.
This prevents servers and clients from connecting to each other.
Add a lustre.xml service file for firewalld, so that it is easy
to run a command like:
firewall-cmd --permanent --zone=public --add-service=lustre
to add the Lustre service ports with minimal difficulty.
It would be good if this was run automatically when the RPMs are
installed, or when mount.lustre is run, but it isn't clear what
is good/safe/correct in all cases. At least having the service
file will be a starting point to make this easier for admins.
It would be even better if the Lustre service rules were restricted
to accepting only new connections, and clients would only accept
requests from the MGS initially and then dynamically add ports for
servers as they are configured, but this is beyond my firewalld-fu.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9f49d4b0df1c9fb6b343df81f966d9110c300c1e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/41021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland <faaland1@llnl.gov>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Tue, 13 Oct 2020 14:12:23 +0000 (14:12 +0000)]
LU-14111 obdclass: count eviction per obd_device
Add a new 'obd_eviction_count' counter to obd_device which
is increased every time a client is evicted, which means
every time we call `class_fail_export()`.
Expose this counter through `lctl get_param *.*.eviction_count`
for every target.
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: I83b691662285cf2cd937187bffa54de6bd1f694c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/40528
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Andreas Dilger [Fri, 3 Feb 2023 10:14:39 +0000 (03:14 -0700)]
LU-16501 tgt: skip free inodes in OST weights
In lu_tgt_qos_weight_calc() calculate the target weight consistently
with how the per-OST and per-OSS penalty calculation is done in
ltd_qos_penalties_calc(). Otherwise, the QOS weighting calculations
combine two different units, which incorrectly weighs allocations on
OST with more free inodes over those with more free space.
Fixes:
d3090bb2b486 ("LU-11213 lod: share object alloc QoS code with LMV")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1ccc52d7ad5dc440ae48403ba129efd6a0a51c33
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49890
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr. NeilBrown [Mon, 23 Jan 2023 21:58:40 +0000 (16:58 -0500)]
LU-6142 lov: use list_for_each_entry in lov_obd.c
Using the *_entry macro simplifies the code slightly.
Change-Id: Ia50cd2cbaf9bac6c9873ef4af7e5cbfa2c8e660e
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49740
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 19 Jan 2023 19:06:36 +0000 (22:06 +0300)]
LU-14918 osd: don't declare similar zfs writes twice
in some cases (like overstriping) the same operations can be
declared multiple times (new llog records) and this lead to
huge number of credits and performance degradation. we can
avoid this checking for duplicate declarations.
notice each declare operation results in a allocation in ZFS.
the example for an overstriped file (2000 stripes over 4 OSTs),
declare ops before after
create: 2001 2
unlink: 10001 10
creation of 1K-stripe files (over 4 OSTs) is 2.5% faster.
removal of 1K-stripe files is 44% faster.
single-stripe file creation/removal does not degrade.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5d9e6d3a1574ccd7bf97fd3a67ab4fff0b6a352c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49701
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 7 Dec 2021 08:13:54 +0000 (11:13 +0300)]
LU-14918 osd: don't declare similar ldiskfs writes twice
in some cases (like overstriping) the same operations can be
declared multiple times (new llog records) and this lead to
huge number of credits and performance degradation. we can
avoid this checking for duplicate declarations.
As every declaration would need an allocation, limit the scope
of this checks to transaction likely to be large.
% of "large" transaction in sanity-benchmark, depending on threshold:
creates < 5 && writes < 5:
0.58% (mds1) and 2.97% (mds2)
create < 7 & writes < 7:
0.58% and 2.4%
create < 9 & writes < 9:
0.6% and 1.85%
create < 10 & write2 < 10:
0.0004% and 0.000001%
thus 10 creates or writes is selected as a threshold to enable this
logic.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I7c893fe3b95646b4b813b999bc832659dfcf03ad
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45765
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Vitaliy Kuznetsov [Wed, 8 Feb 2023 21:34:38 +0000 (00:34 +0300)]
LU-16454 mdt: Add a per-MDT "max_mod_rpcs_in_flight"
Value max_mod_rpcs_per_client doesn't define a static number of
slots for the per-client replies or anything, and the only
thing it is used for is to pass the limit to the client. For the
same reason, there also doesn't appear to be a particularly hard
limitation why the client cannot change and exceed the
server-provided parameter, except to avoid overloading the server
with too many RPCs at once, but that may also be true of the
current limit with a larger number of clients, no different than
"max_rpcs_in_flight".
This fix adds a tunable parameter "max_mod_rpcs_in_flight" per MDT
to lustre/mdt/mdt_lproc.c so that it can be set
with "lctl set_param" at runtime. The max_mod_rpcs_per_client global
setting is marked "deprecated" but is still used as the default
value when creating an MDT.
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I27cfcb68e1a534e80e6a2dbf2e1affc430803b49
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Oleg Drokin [Thu, 9 Feb 2023 17:38:57 +0000 (12:38 -0500)]
New tag 2.15.54
Change-Id: I592cabccefa9bbdf3d1d97fa313103b8b1b1eb3b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 1 Feb 2023 07:11:56 +0000 (23:11 -0800)]
LU-16500 utils: set default ost index for lfs migrate
Running "lfs migrate <file>" without any SETSTRIPE arguments
to balance space usage keeps the PFL file layout, but preserves
the OST selection exactly, which makes the migration virtually
useless for space balancing.
This patch fixes the above issue by clearing the specific
OST indices from the source layout before using the layout to
create the volatile file in lfs_migrate().
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I82e1dc0a11fdda7d555df994cf4e5f6e3dbdcb5c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49819
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Tue, 18 Jan 2022 13:55:01 +0000 (13:55 +0000)]
LU-930 ptlrpc: clarify AT error message
Clarify the error message related to passed deadline
for AT early replies. It was indicating that the system
was CPU bound which is most of the time wrong, as the issue
is rather communication failure delaying RPC traffic.
This could be confusing to people which will look for
CPU resource consumption where the network traffic is
more at cause.
Also try to use less cryptic keywords which makes only
sense to the feature developer, and not to admins.
Test-Parameters: trivial
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: Icdff8f4c6fb9905233f6b8ed1b961b2fd1127667
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49548
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 5 Dec 2022 18:59:02 +0000 (11:59 -0700)]
LU-16367 utils: clean up ldiskfs feature handling
Update the default ldiskfs features used by mkfs.lustre:
- enable large_dir on OSTs as well as MDTs
- remove obsolete handling of "ext3" filesystems
- clean up handling of other features that have become a bit messy
Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id717c3ba939ccf9b2de34e868d4415e88429ef39
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49316
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 27 Jan 2023 20:34:11 +0000 (12:34 -0800)]
LU-16221 kernel: new kernel [RHEL 9.1 5.14.0-162.12.1.el9_1]
This patch makes changes to support new RHEL 9.1 release
for Lustre client.
Test-Parameters: trivial clientdistro=el9.1 \
env=SANITY_EXCEPT="130 244a" testlist=sanity
Change-Id: I8af730f84c9ddf9dcb7e3ddfbd24a68173f51e8d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48938
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Shaun Tancheff [Mon, 30 Jan 2023 04:17:12 +0000 (22:17 -0600)]
LU-16510 build: fortified memcpy from linux 6.1
The fortified memcpy() from Linux v5.11-11104-ga28a6e860c6c
through v5.18-rc5-1405-g43213daed6d6 incorrectly reports
a false positive out of bounds check.
In function 'memcpy' ...
'__read_overflow2' declared with attribute error: detected
read beyond size of object passed as 2nd parameter
Test-Parameters: trivial
HPE-bug-id: LUS-11459
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I3a59d8b647833c05ff4b51e327ed8bce894141fe
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49811
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Shaun Tancheff [Thu, 19 Jan 2023 07:38:02 +0000 (01:38 -0600)]
LU-16292 llite: delete_from_page_cache not exported
Linux commit v5.16-rc4-44-g452e9e6992fe
filemap: Add filemap_remove_folio and __filemap_remove_folio
Directly removing a folio/page from the page cache is not
available.
Fallback to generic_error_remove_page for regular files,
and truncate_inode_pages_range as appropriate.
Test-Parameters: trivial
HPE-bug-id: LUS-11198
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I634e7d7719d497ce035a78b424be8e9e8c5a8104
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49069
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Aurelien Degremont [Mon, 26 Sep 2022 12:27:37 +0000 (12:27 +0000)]
LU-16188 mdt: fix incompatible HSM request handling
When the coordinator tries to send multiple hsm actions in
a single request, if one of the request fails incompat checks all the
requests are marked as STARTED but none of the requests are
sent to the agent.
Return EAGAIN from mdt_agent_hsm_send() so that the coordinator would
not mark the requests as STARTED. It would retry them later.
Add a sanity-hsm test.
Test-Parameters: trivial testlist=sanity-hsm
Change-Id: Id4fb858021be6dc6b0cbcf140c3f2051efce57ad
Signed-off-by: Jeya Ganesh Babu Jegatheesan <jeyaga@amazon.com>
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sun, 22 Jan 2023 17:43:29 +0000 (11:43 -0600)]
LU-16118 build: Workaround __write_overflow_field errors
Linux commit v5.17-rc3-1-gf68f2ff91512
fortify: Detect struct member overflows in memcpy() at compile-time
memcpy and memset of collections of struct members
will trigger:
error: call to ‘__write_overflow_field’ declared with attribute
warning: detected write beyond size of field (1st parameter);
maybe use struct_group()?
[-Werror] __write_overflow_field(p_size_field, size);
Test-Parameters: trivial
HPE-bug-id: LUS-11194
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iacd1ab03d1b90ce62b5d7b65e1cd518a5f7981f2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48364
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Sat, 21 Jan 2023 06:16:25 +0000 (00:16 -0600)]
LU-16354 ldiskfs: RHEL9.1 server support
ldiskfs patch series for RHEL9.1
Test-Parameters: trivial
HPE-bug-id: LUS-11332
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ia0757995ac7200eb50fadf5e106fe1d7b3dc0443
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49283
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log