Whamcloud - gitweb
fs/lustre-release.git
10 months agoLU-16831 lfs: limit stripe count for component size 43/51143/7
Bobi Jam [Wed, 21 Jun 2023 06:33:21 +0000 (14:33 +0800)]
LU-16831 lfs: limit stripe count for component size

If stripe count is larger than component_size/stripe_size, some
allocated OST objects are created but inaccessible. This patch
reduces the number of stripes in that case to avoid this.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I117ed8a7696c6c6adcdd0c2c6531a958cc53bd51
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51143
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-15969 llite: add support for ->fileattr_get/set 07/51107/4
Mr NeilBrown [Wed, 7 Dec 2022 07:55:57 +0000 (18:55 +1100)]
LU-15969 llite: add support for  ->fileattr_get/set

From Linux 5.13, FS_IOC_SETFLAGS and GETFLAGS aren't passed down to
the filesystem, we need ->fileattr_get/set inode_operations instead.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Change-Id: Ib3ffba3529ea32b702ad80abd4b9e4e3ad90b412
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51107
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-16824 ldiskfs: add support for openEuler 22.03 LTS SP1 78/50978/3
Xinliang Liu [Wed, 10 May 2023 10:08:38 +0000 (10:08 +0000)]
LU-16824 ldiskfs: add support for openEuler 22.03 LTS SP1

Add openEuler 22.03 LTS SP1 config target file.
Fix tiny conflicts for patch ext4-delayed-iput.patch and
ext4-data-in-dirent.patch.
Add missing patch ext4-encdata.patch.
Add build required pkg kernel-debugsource for ldiskfs build.

Change-Id: I68314c9df17ce991a5e46f2ed4746ce1703b1587
Test-Parameters: trivial
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50978
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16793 build: Enable compile tests to require <module>.ko 49/50849/2
Shaun Tancheff [Tue, 2 May 2023 10:29:44 +0000 (05:29 -0500)]
LU-16793 build: Enable compile tests to require <module>.ko

Currently the build tests only demand a kernel api test
create an object (.o).

Cases that have a missing symbol export, directly or
indirectly, will generate an object file and fail to
generate a kernel module (.ko).

Enable tests to select the stricter criteria.

Test-Parameters: trivial
Fixes: cc5594df3e ("LU-16759 o2ib: MOFED 5.5+ ib_dma_virt_map_sg")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iae481f1287023ea6c2432d147c497fa0a55fd689
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50849
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-9859 llite: simplify parsing in pcc_conds_parse() 39/50839/6
Mr NeilBrown [Tue, 24 Nov 2020 06:16:38 +0000 (17:16 +1100)]
LU-9859 llite: simplify parsing in pcc_conds_parse()

If we duplicate the string to be parsed we can use standard parsing
tools like strsep() and strcmp().

Test-Parameters: trivial testlist=sanity-pcc
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia0d0fb4fbc8d85f1e47e6085392e0f84b000b8a8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50839
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-9859 ptlrpc: simplify nrs_tbf_jobid_list_parse() 38/50838/8
Mr NeilBrown [Tue, 24 Nov 2020 05:52:12 +0000 (16:52 +1100)]
LU-9859 ptlrpc: simplify nrs_tbf_jobid_list_parse()

If we duplicate the string passed to nrs_tbf_jobid_list_parse(), we
can parse using standard mutating tools.

Test-Parameters: trivial
Test-Parameters: testlist=sanityn env=ONLY=77
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Id04390c2ed0a26a0311a8bbc784979eb18f4d19d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50838
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-9859 ptlrpc: simplify nrs_tbf_check_id_value and callers 36/50836/4
Mr NeilBrown [Tue, 24 Nov 2020 05:17:33 +0000 (16:17 +1100)]
LU-9859 ptlrpc: simplify nrs_tbf_check_id_value and callers

The string passed down to nrs_tbf_check_id_value() is always "token"
in nrs_tbf_id_parse(), which is in 'buffer' in nrs_tbf_parse_cmd() and
is modified in-place there and in
ptlrpc_lprocfs_nrs_tbf_rule_seq_write().

So it must be safe to modify the string in nrs_tbf_check_id_value()
too.  So change that function to parse it using the primitives
commonly used in the kernel such as strsep and strim.

Change all callers to use "char *" rather than "struct cfs_lstr".

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie67dba002530e4bdfb3c3601a15dc49904f1adcf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50836
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-9859 ptlrpc: simplifying expression parsing in nrs_tbf 35/50835/13
Mr NeilBrown [Tue, 24 Nov 2020 04:53:46 +0000 (15:53 +1100)]
LU-9859 ptlrpc: simplifying expression parsing in nrs_tbf

The standard approach to parsing in the kernel is to modify strings as
needed, such as to nul-terminate substrings.

Lustre tends to pass around lengths instead, which means that various
kernel functions such as kstrtoNN() or strsep() or even strcmp()
cannot be used.

We can simplify code in nrs_tbf if we kstrdup() strings before parsing
them, and then use standard functions.

cfs_gettok() strips spaces while finding the token.  With this patch,
stripping of spaces is left to the final stage where an expression
(a={b}) is being parsed.  It might arrive with arbitrary space such as
" a ={ b  }  ".

A test in sanityn has some spaces added in various places to ensure
that are parsed correctly as an earlier version of this patch got some
of that wrong.

The list parsed in nrs_tbf_id_list_parse() can have multiple separator
(spaces) between elements, which contrasts with expressions which only
have a single '=" or "&" etc.

So strsep() might return an empty token between two consecutive
spaces.  This is not necessarily an error - it is only an error if
*all* tokens are empty.  So we add a "list_empty()" test at the end.

Test-Parameters: trivial
Test-Parameters: testlist=sanityn
Test-Parameters: testlist=sanity
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Id4fb399773e49e4869ca5ebf93fe63c864d82287
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50835
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-9859 lnet: move ioctl device to lnet 33/50833/10
Mr NeilBrown [Tue, 24 Nov 2020 00:42:44 +0000 (11:42 +1100)]
LU-9859 lnet: move ioctl device to lnet

The misc device "/dev/lnet" is currently managed in libcfs code,
despite that fact that it is named "lnet" and almost all ioctl
handlers are in lnet code.

So move the management of the device to lnet code, leaving just
the minimal amount in libcfs:
case IOC_LIBCFS_CLEAR_DEBUG:
case IOC_LIBCFS_MARK_DEBUG:

Also rename various parts of the interface from libcfs_ioctl* to
lnet_ioctl*.
ioctl names, data structures, and include files are left unchanged for
now.

Note that the return value from LNetCtl() was previously passed
through notifier_from_errno() and notifier_to_errno().  This had the
effect of turning any positive value to zero.  We need to preserve
that and not return positive results.  PING would return a positive
result.  lnd->lnd_ctl probably doesn't, but due to the difficulty of
auditing, it is safer to always force the result to non-positive.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9cf158f1f9d8b03687d85ba40bd88f1f0ab8e4b8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50833
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16786 utils: Replace open call to WANT_FD 64/50764/11
Arshad Hussain [Wed, 26 Apr 2023 05:51:15 +0000 (11:21 +0530)]
LU-16786 utils: Replace open call to WANT_FD

Replace open call to WANT_FD with newly added API
llapi_root_path_open() which was added under LU-16427

Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I97d55321cf32e40eaf7d6284c47f313199a6c406
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50764
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
10 months agoLU-16275 tests: Modify killall of replay-dual 25 77/48977/5
Kevin Zhao [Fri, 28 Oct 2022 06:55:52 +0000 (14:55 +0800)]
LU-16275 tests: Modify killall of replay-dual 25

The in-tree test will have rename the multiop to lt-multiop,
which will not be killed with killall. So it's better to add
regex to killall for kill all the process with multiop and
lt-multiop

Test-Parameters: trivial testlist=replay-dual env=ONLY=25
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
Change-Id: Ic9b064a6bb0d944eedb5dc019ba5d4d05c98eeae
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48977
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-16298 ldiskfs: Periodically write ldiskfs superblock 40/51340/11
Vitaliy Kuznetsov [Tue, 11 Jul 2023 08:12:25 +0000 (14:12 +0600)]
LU-16298 ldiskfs: Periodically write ldiskfs superblock

This patch introduces a mechanism to periodically check and update
the superblock within the ext4 file system. The main purpose of this
patch is to keep the disk superblock up to date. The update will be
performed if more than one hour has passed since the last update,
and if more than 16MB of data have been written to disk.

This check and update is performed within the
ext4_journal_commit_callback function, ensuring that the superblock
is written while the disk is active, rather than based on a timer
that may trigger during disk idle periods.

Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I06eb9624b663a6ca6b15c6af2373b82f1bb63de6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51340
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16847 ldiskfs: ->fiemap replaced with ldiskfs_map_blocks. 46/51146/8
Alexey Lyashkov [Fri, 26 May 2023 10:15:36 +0000 (13:15 +0300)]
LU-16847 ldiskfs: ->fiemap replaced with ldiskfs_map_blocks.

lets avoid hacks with data copy from fiemap buffer to kernel
space by using an ldiskfs_map_blocks directly.
It might code lots clear.

Fixes: 5cd5a49c7213 ("LU-16321 osd: Allow fiemap on kernel buffers")
HPe-bug-id: LUS-11645
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I1c527ce653b8943801e5bf56fd172a5f05e22dfc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51146
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-6142 lustre: use list_first/last_entry() for list heads 28/50828/5
Mr NeilBrown [Wed, 6 Nov 2019 22:47:35 +0000 (09:47 +1100)]
LU-6142 lustre: use list_first/last_entry() for list heads

This patch changes
    list_entry(foo.next, ...)
to
    list_first_entry(&foo, ...)
and
    list_entry(foo.prev, ...)
to
    list_last_entry(&foo, ...)

in cases where 'foo' is a list head - not a list member.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I22b1278f5b481ce3074db3e59d37d9148016aed5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50828
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-9806 obdclass: wait for all exports to go 47/50147/11
Alex Zhuravlev [Mon, 27 Feb 2023 18:40:34 +0000 (21:40 +0300)]
LU-9806 obdclass: wait for all exports to go

obd_zombie_export_add() removes an export from the stale list
and then schedules a job to destroy that export. in this short
window ofd_fini()/mdt_fini() can find obd_linked_exports list
empty and no work in zombie work queue. then the obd is being
removed and concurrent export destroy may find the obd in a
unexpected state:
LustreError: 11166:0:(tgt_lastrcvd.c:469:tgt_client_free())
ASSERTION( lut && lut->lut_client_bitmap ) failed

use obd_stale_export_num counter to block in obd_zombie_barrier.

move atomic_inc() from class_unlink_export to obd_export_zombie_add()
as self-exports are not added to the stale list. I

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I62ed019f86becd3c66f5fcdf991f13cd47466e5e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50147
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8853 nodemap: fix up lctl nodemap_info 68/23868/4
Kit Westneat [Fri, 18 Nov 2016 21:10:55 +0000 (16:10 -0500)]
LU-8853 nodemap: fix up lctl nodemap_info

When nodemap_info was moved over to using param_display, some of the
functionality got lost. This patch modifies nodemap_info to use
GET_PARAM instead of LIST_PARAM when a nodemap is specified.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I85df8e1cb43e002f4a112b9b671725862210dbec
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/23868
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
10 months agoLU-8582 tests: skip sanity/905 for old OSTs 68/51568/3
Andreas Dilger [Wed, 5 Jul 2023 00:58:07 +0000 (18:58 -0600)]
LU-8582 tests: skip sanity/905 for old OSTs

The fail_loc used in sanity test_905 does not exist in older OSTs.
Skip this subtest for older OSTs.

Fixes: 566edb8c43 ("LU-8582 target: send error reply on wrong opcode")
Test-Parameters: trivial testlist=sanity serverversion=2.12.9 env=ONLY=905
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8fa50ec0f66afd9f24d562e0be57a416c04d8ba8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51568
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sarah Liu <sarah@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16101 tests: skip sanity/27J for more kernels 67/51567/4
Andreas Dilger [Tue, 4 Jul 2023 23:08:03 +0000 (17:08 -0600)]
LU-16101 tests: skip sanity/27J for more kernels

This is a bug in the kernel that is not present in older kernels
before commit v5.11-10234-gcbd59c48ae2b (5.12), and is fixed with
commit v6.2-rc4-61-g5956592ce337 (6.2).

Move this from ALWAYS_EXCEPT (bug that needs to be fixed) to skip
(test that is known to fail in some configs but has been fixed).

Fixes: af6f49698a18 ("LU-16101 tests: add sanity/27J to always_except")
Test-Parameters: trivial testlist=sanity clientdistro=el9.2 env=ONLY=27J
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8ec0a6d25a90e05672b039cd6c2b2fbf8a3ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16937 utils: avoid lctl shmget() if not needed 26/51526/2
Andreas Dilger [Fri, 30 Jun 2023 19:41:23 +0000 (13:41 -0600)]
LU-16937 utils: avoid lctl shmget() if not needed

lctl is dynamically allocating an IPC shared memory segment
during every startup, even though it is only needed for a
small number of uncommon debug commands:

    shmget(IPC_PRIVATE, 65680, 0600)        = 196641
    shmat(196641, NULL, 0)                  = 0x7f752b1c5000
    shmctl(196641, IPC_RMID, NULL)          = 0

This setup can be moved to sub-commands that actually need it.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I41c790ce7cba2d9c48c1ec06eb23eb94aa548242
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51526
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16868 tests: add check for replace_nid 24/51524/5
Alex Deiter [Fri, 30 Jun 2023 18:02:49 +0000 (22:02 +0400)]
LU-16868 tests: add check for replace_nid

Added check for replace_nid operations and return an
error to prevent module reload errors and timeout
when unmounting targets.

Test-Parameters: trivial
Test-Parameters: testlist=conf-sanity env=ONLY=32a serverdistro=el7.9
Test-Parameters: testlist=conf-sanity env=ONLY=32a serverdistro=el8.7
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Change-Id: I29a5de826ac8f0040dd671e502d30bac4a082c43
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51524
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16934 kernel: update RHEL 8.8 [4.18.0-477.15.1.el8_8] 17/51517/2
Jian Yu [Fri, 30 Jun 2023 10:59:31 +0000 (18:59 +0800)]
LU-16934 kernel: update RHEL 8.8 [4.18.0-477.15.1.el8_8]

Update RHEL 8.8 kernel to 4.18.0-477.15.1.el8_8.

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity

Change-Id: I66365dce63065a0a07958a182a3c705e9948d424
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51517
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-14301 client: use EOPNOTSUPP instead of ENOTSUPP 11/51511/3
Andreas Dilger [Fri, 30 Jun 2023 00:26:44 +0000 (18:26 -0600)]
LU-14301 client: use EOPNOTSUPP instead of ENOTSUPP

Don't return NFS-specific error code ENOTSUPP back to userspace,
instead use EOPNOTSUPP.  ENOTSUPP does not print a useful error
message from strerror() if it is hit by an application.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iabd07b31069737e8ee7ca2382fd8cff6143ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51511
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-15626 tests: Remove variables not used 99/51499/6
Arshad Hussain [Wed, 28 Jun 2023 08:06:43 +0000 (13:36 +0530)]
LU-15626 tests: Remove variables not used

This patch removes variables which were
defined but not used. This was reported
by shellcheck.

This patch also replaces unicode double
quotes wherever applicable

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I996532de119806e20552e9bbe54b15615ea2c2e0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51499
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16929 tests: Fix syntax error under ha.sh 82/51482/2
Arshad Hussain [Wed, 28 Jun 2023 09:41:03 +0000 (15:11 +0530)]
LU-16929 tests: Fix syntax error under ha.sh

Fix bash syntax error under ha.sh/ha_lfsck_repaired

Test-Parameters: trivial
Fixes: 1a7c352e9 ("LU-11504 tests: trigger lfsck after/during failover/failback")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ic31099a9438e1174013843156147cbb3bd98366a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51482
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Elena <elena.gryaznova@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 ptlrpc: add missing headers 80/51480/2
Timothy Day [Fri, 23 Jun 2023 20:49:08 +0000 (20:49 +0000)]
LU-8191 ptlrpc: add missing headers

Missing headers for several c-files in ptlrpc
cause functions to be incorrectly marked as only
being used within their respective c-files. This
patch adds those missing headers. It also addresses
a couple minor style issues.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Idd8fa747a671079aba2b691ef23cc7564e5e2430
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51480
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 lustre: convert mdc,mdd,mdt,mgc functions to static 78/51478/2
Timothy Day [Fri, 23 Jun 2023 20:52:17 +0000 (20:52 +0000)]
LU-8191 lustre: convert mdc,mdd,mdt,mgc functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in mdc, mdd, mdt, and mgc static.

Also, remove mgs_client_add() since it was unused, and
move a declaration from a c-file to the proper header file.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia23f62465c27c83a9a0260bb45e8c8b710491558
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51478
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 lustre: convert osp,osd,osc,ofd functions to static 77/51477/2
Timothy Day [Fri, 23 Jun 2023 20:51:00 +0000 (20:51 +0000)]
LU-8191 lustre: convert osp,osd,osc,ofd functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in osp, osd, osc, and ofd static.

Also, fix a few minor style issues.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I3d7af7ec0fa2978bfdd0cb490f18f485a78f81f6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 lustre: convert ec,fid,ldlm,quota functions to static 76/51476/4
Timothy Day [Fri, 23 Jun 2023 20:47:26 +0000 (20:47 +0000)]
LU-8191 lustre: convert ec,fid,ldlm,quota functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in ec, fid, ldlm, and quota static.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ic64bdf0d802fd4c963b7b7d3a654575ebde5c07d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51476
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 target: convert functions to static 75/51475/2
Timothy Day [Fri, 23 Jun 2023 20:41:36 +0000 (20:41 +0000)]
LU-8191 target: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in target static.

Also, remove an unused function tgt_obd_log_cancel(),
and add some headers where they were missing.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I1823df3562cb181b275788560166c92b63483637
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51475
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 llite: convert functions to static 41/51441/2
Timothy Day [Fri, 23 Jun 2023 20:46:29 +0000 (20:46 +0000)]
LU-8191 llite: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in llite static.

Also, conserve more * in comments.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Iafa3bb84de158e31b27b7784243bc15e78187f10
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51441
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 obdclass: add static and remove functions 40/51440/3
Timothy Day [Fri, 23 Jun 2023 20:40:35 +0000 (20:40 +0000)]
LU-8191 obdclass: add static and remove functions

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in obdclass static.

There are a few functions which are never called
anywhere. These are removed. Additionally, there
is some debugging code (added 15 years ago) that
has also been removed.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I5f1d438c9663e62789d26093ec9bdd5d76a3060a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51440
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 lnet: remove unused, fix non-static functions 36/51436/2
Timothy Day [Fri, 23 Jun 2023 20:45:23 +0000 (20:45 +0000)]
LU-8191 lnet: remove unused, fix non-static functions

lnet_selftest_structure_assertion() and
lnet_net_is_pref_rtr_locked() are never called.
This patch removes both functions.

Static analysis shows that a number of functions
could be made static. This patch also declares
several functions in lnet static.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ie1b49c5652553715cd9f96b56090d33a95e3b438
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51436
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16904 test: Add sanity-compr.sh to run sanity and sanityn with PFL layout 71/51371/6
Andreas Dilger [Tue, 20 Jun 2023 18:57:56 +0000 (12:57 -0600)]
LU-16904 test: Add sanity-compr.sh to run sanity and sanityn with PFL layout

Add sanity-compr.sh to run sanity and sanityn with PFL layout
Also fix sanity subtests problem of 56ba,57b,65e,65g,65n,204e

Test-Parameters: trivial testlist=sanity-compr

Signed-off-by: Wei Liu <sarah@whamcloud.com>
Change-Id: Iefdc7757697629eb5c57d7694456249d62a2049e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51371
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16667 build: kernel_cap_t contains u64 21/50421/9
Shaun Tancheff [Sun, 7 May 2023 02:11:27 +0000 (21:11 -0500)]
LU-16667 build: kernel_cap_t contains u64

linux kernel v6.2-13111-gf122a08b197d
  capability: just use a 'u64' instead of a 'u32[2]' array

Add configure test for kernel_cap_t as u64 and provide
and accessor for the least significant 32 bits.

As of linux commit v3.6-10973-g607ca46e97a1 lustre implicitly
started to ignore some capabilities, see:
   include/uapi/linux/capability.h

The last capability flag was added by:
   linux commit v5.8-rc5-1-g124ea650d307

The capabilities the Lustre currently ignores are:
 - CAP_MAC_OVERRIDE
 - CAP_MAC_ADMIN
 - CAP_SYSLOG
 - CAP_WAKE_ALARM
 - CAP_BLOCK_SUSPEND
 - CAP_AUDIT_READ
 - CAP_PERFMON
 - CAP_BPF
 - CAP_CHECKPOINT_RESTORE

None of which appear to be important to Lustre operations
and should be fine to continue ignore.

Test-Parameters: trivial
HPE-bug-id: LUS-11557
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I48ad7b1a34fff378c260dc73ea91b22aaa0d7469
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50421
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-13805 llite: Rename ldp_aio to sdio 70/50170/19
Patrick Farrell [Wed, 1 Mar 2023 17:54:11 +0000 (12:54 -0500)]
LU-13805 llite: Rename ldp_aio to sdio

ldp_aio is a weird name for a 'sub dio' struct - rename it
to sdio.

Test-parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie6f75e420e4bf4069af36c802b51063f02981613
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50170
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
10 months agoLU-13805 llite: Convert allocate/get to use pvec 68/50168/21
Patrick Farrell [Wed, 1 Mar 2023 16:01:38 +0000 (11:01 -0500)]
LU-13805 llite: Convert allocate/get to use pvec

ll_allocate_dio_buffer and ll_get_user_pages both basically
work on the ll_dio_pages pvec, so this patch converts them
to do so explicitly - this makes it more obvious what
they're doing and their connection to the rest of the DIO
code.  This makes them less generic, but they have no other
users and seem unlikely to acquire any.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If643d7684d89e0e0c81ee9d13f0d94f84ed87a56
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50168
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
10 months agoLU-13805 tests: add debug to aiocp 89/49989/22
Patrick Farrell [Tue, 14 Feb 2023 18:26:23 +0000 (13:26 -0500)]
LU-13805 tests: add debug to aiocp

Improve debug in aiocp.c.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I154036992a61b64b1753b15e47e64c01b630a5cb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49989
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
10 months agoLU-16634 build: remove obsolete checkpatch strings 96/51596/2
Andreas Dilger [Thu, 6 Jul 2023 19:03:34 +0000 (13:03 -0600)]
LU-16634 build: remove obsolete checkpatch strings

Remove obsolete checkpatch.pl spelling.txt warnings.  These will
not normally be hit, but the mti_flags warning is occasionally a
false positive due to a duplicate struct name (which was originally
the reason for those fields to be renamed).

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4ee16eb62cf0ed944ac604d181ce58beead8501d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51596
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-15431 llite: skip fast reads if layout is invalid 82/46282/39
Alex Zhuravlev [Mon, 24 Jan 2022 13:10:03 +0000 (16:10 +0300)]
LU-15431 llite: skip fast reads if layout is invalid

don't let fast reads from the pagecache if the layout
is not valid.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie4357a184faf9a5d0e33804270d3cb0cb7e67bb7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/46282
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16637 llite: call truncate_inode_pages() in inode lock 57/50857/22
Bobi Jam [Thu, 4 May 2023 02:39:29 +0000 (10:39 +0800)]
LU-16637 llite: call truncate_inode_pages() in inode lock

In some cases vvp_prune()->truncate_inode_pages() is get called
without IO context, we need protect it with inode lock as well.

So we add ll_inode_info::lli_inode_lock_owner and set it according to
vfs lock rules (Documentation/filesystems/Locking or
Documentation/filesystems/locking.rst), so before calling
truncate_inode_pages(), we'd lock the inode if it's not locked in
vfs.

Fixes: ef9be34478 ("LU-16637 llite: call truncate_inode_pages() under inode lock")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I84d7d999a49325810062a9a7337e184d35467820
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50857
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16927 tests: improve sanity-quota 69/51469/4
Arshad Hussain [Tue, 27 Jun 2023 07:03:12 +0000 (12:33 +0530)]
LU-16927 tests: improve sanity-quota

DD variable already includes "if=/dev/zero"
and "bs=1M". This patch removes double
declaration of 'bs' and/or 'if' when called
with DD

This patch also changes direct 'dd' calls
to use $DD whenever applicable

Also for sanity-quota/52 fix error path variable
name.

Test-Parameters: trivial testlist=sanity-quota
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I9006c406f3b794ecdc37a451451538e5202a006a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51469
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 tests: convert functions to static 33/51433/3
Timothy Day [Fri, 23 Jun 2023 20:39:12 +0000 (20:39 +0000)]
LU-8191 tests: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various test helpers static.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I065fb4398ed1670ce6ad58cf946054f6bd1ec282
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16925 osd-ldiskfs: Remove unused bio_integrity_enabled 32/51432/2
Shaun Tancheff [Sat, 24 Jun 2023 05:08:48 +0000 (12:08 +0700)]
LU-16925 osd-ldiskfs: Remove unused bio_integrity_enabled

bio_integrity_enabled() is not used in lustre.

Remove the configure check and the associated code.

Test-Parameters: trivial
HPE-bug-id: LUS-10118
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I9d07333b91210a2f6545945cf48293179a71258e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51432
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16922 kernel: update RHEL 9.2 [5.14.0-284.18.1.el9_2] 10/51410/2
Jian Yu [Thu, 22 Jun 2023 06:56:51 +0000 (14:56 +0800)]
LU-16922 kernel: update RHEL 9.2 [5.14.0-284.18.1.el9_2]

Update RHEL 9.2 kernel to 5.14.0-284.18.1.el9_2.

Test-Parameters: trivial env=SANITY_EXCEPT=27J fstype=ldiskfs \
clientdistro=el9.2 serverdistro=el9.2 testlist=sanity

Test-Parameters: trivial env=SANITY_EXCEPT=27J fstype=zfs \
clientdistro=el9.2 serverdistro=el9.2 testlist=sanity

Change-Id: Ifa8f13200550e5f473b7d7d641155e349c453c03
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51410
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 mdt: convert functions to static 45/51345/2
Timothy Day [Sun, 18 Jun 2023 04:26:48 +0000 (04:26 +0000)]
LU-8191 mdt: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in mdt_coordinator.c static.

 mdt_coordinator.c:2145:9: warning: Should this function be static?
 ssize_t loop_period_show(struct kobject *kobj,
         ^

Further patches will follow to clean up the
remaining non-static functions in other subsystems.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I0350b0d5c88c0a8d1f1748d1d429cdf90afb96b7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51345
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16548 lnet: Fixing missing gnilnd define CURRENT_LND_VERSION 42/51342/3
Frank Sehr [Fri, 16 Jun 2023 19:36:33 +0000 (12:36 -0700)]
LU-16548 lnet: Fixing missing gnilnd define CURRENT_LND_VERSION

Added missing define CURRENT_LND_VERSION for gni.
Declared kgnilnd_tunables_setup.

Test-Parameters: trivial
Signed-off-by: Frank Sehr <fsehr@whamcloud.com>
Change-Id: Ia327dcbdaa518a24a60e32b1dcb37c5b1d0dc78e
Fixes: 56097c4904 ("LU-16548 lnet: report actual timeout used by lnd")
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51342
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
10 months agoLU-16723 parser: fix help hanging 39/51339/2
Timothy Day [Fri, 16 Jun 2023 04:58:42 +0000 (04:58 +0000)]
LU-16723 parser: fix help hanging

Running a command such as 'lctl pcc help v' will
hang indefinitely. This is due to a bug in find_cmd,
return pointers from two different arrays.

This is fixed by setting top_level to be the
concatenation of the override_cmdlist and the
regular cmds.

Also, rather than recursing forever, give up after
reaching an arbitrary depth.

Improve several help messages so that users will
have a better idea why their commands aren't
working.

Fixes: 21080400f9 ("LU-16723 libcfs: refactor parser to be simpler")
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ib22bc71e5952b1beb868bbe37bc8f6b08c94ff72
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51339
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16890 obd: OBD_FREE_PRE() to ignore NULL pointers 32/51332/7
Arshad Hussain [Thu, 15 Jun 2023 11:10:54 +0000 (16:40 +0530)]
LU-16890 obd: OBD_FREE_PRE() to ignore NULL pointers

This patch modifies OBD_FREE_PRE() to not LASSERT
when null pointer is passed as the kfree() function
accepts NULL as no-op.

This is first set of patch that modifies the definitions.
Subsequent set will modify the callers to accept this
case.

As as example:

This caller will now be change to:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
char *test;
OBD_ALLOC(test, 32);
...
OBD_FREE(test, 32);

Previously the caller used to be:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
char *test;
OBD_ALLOC(test, 32);
...
if (test)
OBD_FREE(test, 32);

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I735c8210e30f58da19ede4c87c07186108b35b99
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51332
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16899 gnilnd: Use libcfs_nidstr and fix typo 30/51330/3
Shaun Tancheff [Sun, 18 Jun 2023 02:39:00 +0000 (09:39 +0700)]
LU-16899 gnilnd: Use libcfs_nidstr and fix typo

CDEBUG() in kgnilnd_peer_notify() should use libcfs_nidstr()

kgnilnd_finish_connect() has a typo in lnet_notify() where
peer_nid was intended.

Test-Parameters: trivial
Fixes: 4a88236f40 ("LU-10391 lnet: change lnet_notify() to take struct lnet_nid")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If45a28b654e27aa34b655fefaea142dc740fa46f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51330
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16898 osd-ldiskfs: do not return dr_error from past RPC 20/51320/7
Andrew Perepechko [Wed, 14 Jun 2023 16:33:43 +0000 (19:33 +0300)]
LU-16898 osd-ldiskfs: do not return dr_error from past RPC

dr_error was cleared in osd_init_iobuf() only before handling new
read/write RPCs, so a later non-read/write RPC handled by that thread
would return the stale dr_error value from the last read/write RPC.

Always clear dr_error in osd_trans_stop->osd_fini_iobuf() after it
is checked, so that it cannot affect later RPCs.

Change-Id: Idbeab67edc66b58e9869b67640693c7f1dd9d6f2
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
HPE-bug-id: LUS-11682
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51320
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
10 months agoLU-16518 obd: fix style and clang error 11/51311/3
Timothy Day [Wed, 14 Jun 2023 02:07:52 +0000 (02:07 +0000)]
LU-16518 obd: fix style and clang error

Tabify the remaining code in this file and clean
up some of the comments. Add SPDX text.

Remove a function which is never used.

Fix the style of and inline cl_io_invariant.

Conserve a significant number of * so that they
can be repurposed in other comments.

Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I6b1c5c700d7e6f13c8c57726db5da6595b9c060a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51311
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16796 libcfs: Remove reference to LASSERT_ATOMIC_GT 89/51189/4
Arshad Hussain [Thu, 1 Jun 2023 07:33:01 +0000 (13:03 +0530)]
LU-16796 libcfs: Remove reference to LASSERT_ATOMIC_GT

This patch removes all reference to LASSERT_ATOMIC_GT macro.

Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I7978ceb495c3e03153843439109d48d47bba1e2a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51189
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
10 months agoLU-16846 nrs: Fix console messages 21/51121/6
Etienne AUJAMES [Wed, 24 May 2023 12:35:29 +0000 (14:35 +0200)]
LU-16846 nrs: Fix console messages

Fix format of console messages and missing end-of-line.

CERROR("%s.%d NRS: ....", service_name, cpt, ...);

Test-Parameters: trivial
Fixes: c098c09 ("LU-14976 nrs: change nrs policies at run time")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ib447673c69bcc853ebd1479463ca79bd5aa59964
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51121
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16842 fsx: tolerate delete last non-stale mirror error 90/51090/3
Bobi Jam [Tue, 23 May 2023 03:11:37 +0000 (11:11 +0800)]
LU-16842 fsx: tolerate delete last non-stale mirror error

fsx mirror split test could try to delete the last non-stale mirror
of a file and that's a tolerable error scenario. The fsx FLR test
randomly choose a mirror operation and this situation could happen.

Test-Parameters: trivial testlist=sanity-flr env=ONLY=70a
Fixes: 04ab0cc869c (LU-14156 utils: mirror split to check for last in-sync early)
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I80c294da80740b21e00ae72a092fd8883ec7d60e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51090
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-12019 build: Recognize Debian Kernel and set KMP dir 66/51066/6
Thomas Stibor [Tue, 13 Jun 2023 17:34:39 +0000 (13:34 -0400)]
LU-12019 build: Recognize Debian Kernel and set KMP dir

Recognize Debian kernel and make sure kernel module package (KMP)
directory matches with KMP_MODDIR of Ubuntu and the Debian building
package system.

Change-Id: Ia3570500ed538c5d3c7a002eafddfc715efbf580
Test-Parameters: trivial clientdistro=ubuntu2204
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51066
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Thomas Stibor <thomas@stibor.net>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16805 llite: improve readpage debug 92/50892/2
Patrick Farrell [Mon, 8 May 2023 21:44:11 +0000 (17:44 -0400)]
LU-16805 llite: improve readpage debug

LU-16412 (which is a workaround for a kernel bug) added a
debug message in ll_readpage(), but this message is printed
every time rather than only when the kernel bug is hit.

Let's fix this.

Fixes: 209afbe28b "LU-16412 llite: check truncated page in ->readpage()"
Test-parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ice02178eb9c07e03b58fb4e2d64ed3ea878cf137
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50892
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-16697 llite: Set BDI_CAP_* flags for lustre 97/50497/8
Shaun Tancheff [Sat, 1 Apr 2023 08:41:16 +0000 (03:41 -0500)]
LU-16697 llite: Set BDI_CAP_* flags for lustre

Lustre should set the BDI_CAP_* flags and the s_iflags
to indicate support for write back and cgroup write back

HPE-bug-id: LUS-11553
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I49ce07fce8a9d153b9a71d8a0ba28b799354fc7f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50497
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
10 months agoLU-16691 ldiskfs: limit length of per-inode prealloc list 81/50481/15
Alex Zhuravlev [Fri, 31 Mar 2023 05:41:07 +0000 (08:41 +0300)]
LU-16691 ldiskfs: limit length of per-inode prealloc list

In the scenario of writing sparse files, the per-inode prealloc list may
be very long, resulting in high overhead for ext4_mb_use_preallocated().
To circumvent this problem, we limit the maximum length of per-inode
prealloc list to 512 and allow users to modify it.

After patching, we observed that the sys ratio of cpu has dropped, and
the system throughput has increased significantly. We created a process
to write the sparse file, and the running time of the process on the
fixed kernel was significantly reduced, as follows:

Running time on unfixed kernel:
    # time taskset 0x01 ./sparse /data1/sparce.dat
    real    0m2.051s
    user    0m0.008s
    sys     0m2.026s

Running time on fixed kernel:
    # time taskset 0x01 ./sparse /data1/sparce.dat
    real    0m0.471s
    user    0m0.004s
    sys     0m0.395s

Link: https://lore.kernel.org/r/d7a98178-056b-6db5-6bce-4ead23f4a257@gmail.com
Signed-off-by: Chunguang Xu <brookxu@tencent.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5e4ea3acfc07f6e69890690211bf6a34c1230979
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50481
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
10 months agoLU-16651 llite: hold invalidate_lock when invalidate cache pages 71/50371/4
Qian Yingjin [Tue, 21 Mar 2023 08:53:00 +0000 (04:53 -0400)]
LU-16651 llite: hold invalidate_lock when invalidate cache pages

The newer kernel (such as Ubuntu 2204) introduces a new member:
invalidate_lock in the structure @address_space.
The filesystem must exclusively acquire invalidate_lock before
invalidating page cache in truncate / hole punch (and thus calling
into ->invalidatepage) to block races between page cache
invalidation and page cache filling functions (fault, read, ...)

However, current Lustre client does not hold this lock when remove
pages from page cache caused by the revocation of the extent DLM
lock protecting them.
If a client has two overlapped PR DLM extent locks, i.e:
- L1 = <PR, [1M, 4M - 1]
- L2 = <PR, [3M, 5M - 1]
A reader process holds L1 and reads data in range [3M, 4M - 1].
L2 is being revoken due to the conflict access.
Then the page read-in by the reader may be invalidated and deleted
from page cache by the revocation of L2 (in lock blocking AST).

The older kernel will check each page after read whether it was
invalidated and deleted from page cache. If so, it will retry the
page read.

In the newer kernel, it removes this check and retry.
Instead, it introduces a new rw_semaphore in the address_space -
invalidate_lock - that holding the shared lock to protect adding
of pages to page cache for page faults / reads / readahead, and
the exclusive lock to protect invalidating pages, removing them
from page cache for truncate / hole punch.

Thus, in this patch it holds exclusive invalidate_lock in newer
kernels when remove pages from page cache caused by the revocation
of a extent DLM lock protecting them. Otherwsie, it will result in
-EIO error or partial reads in the new added test case sanity/833.

Test-parameters: clientdistro=ubuntu2204 testlist=sanity env=ONLY=833,ONLY_REPEAT=10
Change-Id: If3a27002b89636b9fd4d7b5ea50afa9aeac5d121
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50371
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
10 months agoLU-16594 build: get_random_u32_below, get_acl with dentry 93/50193/7
Shaun Tancheff [Sun, 2 Apr 2023 09:15:58 +0000 (04:15 -0500)]
LU-16594 build: get_random_u32_below, get_acl with dentry

Linux commit v6.1-13825-g3c202d14a9d7
  prandom: remove prandom_u32_max()

Use get_random_u32_below() and provide a replacement
when get_random_u32_below is not available.

Linux commit v6.1-rc1-2-g138060ba92b3
  fs: pass dentry to set acl method
Linux commit v6.1-rc1-4-g7420332a6ff4
  fs: add new get acl method

get_acl() and set_acl() have new signatures

Test-Parameters: trivial
HPE-bug-id: LUS-11556
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I1de02f86fd2719fc75de4f014f51d73736d83c33
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50193
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-13340 lustre: Support large nids in LCFG_ADD_UUID 96/50096/9
Mr NeilBrown [Thu, 12 May 2022 04:39:16 +0000 (14:39 +1000)]
LU-13340 lustre: Support large nids in LCFG_ADD_UUID

struct lustre_cfg contains lcfg_nid which is used only for
LCFG_ADD_UUID.  This is not sufficient for large nids.

The LCFG_ADD_UUID config record has room for 4 arbitrary strings only
one of which is used ("node").  So we can use the second string to
store a larger nid.

Specifically: if lcfg_nid is zero, then the second config string
(named "nid") will store the nid in string format.  When a nid with
4-byte address is needed, that will always be stored in lcfg_nid,
never in the config string.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: If08477df677f26e0ff450803e79dde707becde0f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50096
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
10 months agoLU-16518 build: llvm/clang support 63/50063/16
Timothy Day [Sat, 18 Feb 2023 03:49:34 +0000 (03:49 +0000)]
LU-16518 build: llvm/clang support

Other projects, notably Linux, have build support for LLVM and
Clang via special environment variables. This is implemented
for Lustre, in the style of:

https://www.kernel.org/doc/html/latest/kbuild/llvm.html

Instances in which GCC is explicitly called are replaced by the
use of $CC. The proper environment variables as passed to make
invocations as needed.

All checks which influence global compiler and toolchain settings
are collected in 'config/lustre-toolchain.m4'.

A configure option is added to disable the strict error flags that
are passes to the C compiler by default. CFLAGS and EXTRA_CFLAGS
are made to work in the typical way. Having fine grained control
over compiler options makes experimenting with Clang smoother.

Some compile checks in 'lustre-core.m4' have been improved by using
unused variables and explicitly setting the compile flag to be used
during the test.

This also sets the execute bit on autogen.sh.

Tested with:
Linux (mainline) - 5.15.94
openZFS - 2.1.99
Lustre (latest master) - 2.15.55
CentOS - 8.5
Clang (default on CentOS) - 12.0.1

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia8654c22fa8fca7bfb96c545ac144a1d3737fa00
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50063
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-13343 gss: no sec flavor on loopback connection 04/46704/10
Sebastien Buisson [Fri, 4 Mar 2022 15:45:59 +0000 (16:45 +0100)]
LU-13343 gss: no sec flavor on loopback connection

When using a local client, i.e. a client mounted on a server node,
there is no benefit from a security standpoint to enforce an SSK or
KRB flavor, since the data does not go over the network.
So force the 'null' security flavor for connections on 0@lo,
independently of the currently defined srpc flavor.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: If25d69bb1e67735cb0544ca954e49175f7471248
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/46704
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-12511 build: include firewalld files for native Linux client 79/51379/3
James Simmons [Tue, 20 Jun 2023 13:36:27 +0000 (09:36 -0400)]
LU-12511 build: include firewalld files for native Linux client

Build rpms for the native Linux client fails due to xml files for
firewalld not being packaged. These files are useful for the
native client support so package them in that case.

Test-Parameters: trivial
Fixes: 9cb4b10c87d2 ("LU-14224 misc: add firewalld service configuration")
Change-Id: Id2887cef2c9b5e5d27fca3f77589775a31ee94b1
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51379
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
10 months agoLU-12511 llite: use mapping_set_error instead of opencoded set_bit 72/51372/4
Michal Hocko [Tue, 20 Jun 2023 13:34:09 +0000 (09:34 -0400)]
LU-12511 llite: use mapping_set_error instead of opencoded set_bit

The mapping_set_error() helper sets the correct AS_ flag for the mapping
so there is no reason to open code it.  Use the helper directly.

[akpm@linux-foundation.org: be honest about conversion from -ENXIO to -EIO]
Link: http://lkml.kernel.org/r/20160912111608.2588-2-mhocko@kernel.org
Linux-commit: 5114a97a8bce7f4ead29a32b67dee85438699b9e

Change-Id: I153bc04d4745a20013820ba81572cadb37ab8f39
Signed-off-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51372
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16847 ldiskfs: refactor t10 code. 45/51145/6
Alexey Lyashkov [Thu, 25 May 2023 07:47:38 +0000 (10:47 +0300)]
LU-16847 ldiskfs: refactor t10 code.

use a t10 private structure at each time to make testing easy.
use a slab to allocate to do it fast and memory leak will found early,
let move a t10 code into own file.

HPe-bug-id: LUS-11645
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: Iaf1cb9fd63db2af866003f4b1f81a5e2c3b8f540
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51145
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16847 ldiskfs: do not copy ldiskfs_chunk_trans_blocks 38/51138/7
Alexey Lyashkov [Thu, 25 May 2023 07:45:29 +0000 (10:45 +0300)]
LU-16847 ldiskfs: do not copy ldiskfs_chunk_trans_blocks

Do not make a copy of ldiskfs_chunk_trans_blocks() function.
Instead, export existing function from the ldiskfs module.

HPe-bug-id: LUS-11645
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: Ic5d4a8e82b0284241b0e8e2a167271a6dc6fc297
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51138
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-16804 tests: rename 'complete' to 'complete_test' 83/51383/6
Andreas Dilger [Tue, 20 Jun 2023 18:57:56 +0000 (12:57 -0600)]
LU-16804 tests: rename 'complete' to 'complete_test'

The test-framework.sh "complete" function conflicts with "complete"
exported from bash_completion, and this causes lustre-initialization
to fail in some configurations now that the lustre test config
is loaded earlier during test-framework.sh init_test_env() setup.

Rename "complete" to "complete_test" to avoid this conflict.

Test-Parameters: trivial
Fixes: fdbb2bc849 ("LU-16804 tests: load CONFIG at beginning of init_test_env")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic72d8d5cc4a65feec6bfb2a76ac5f9b9d78e3f75
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51383
Tested-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-8191 ptlrpc: convert functions to static 53/51353/2
Timothy Day [Mon, 19 Jun 2023 03:44:02 +0000 (03:44 +0000)]
LU-8191 ptlrpc: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in ptlrpc static.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If0d92f7f4e625c146833f360806ae80b8914cc20
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51353
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16905 tests: Fix 'timeout' value under sanity-quota/18 37/51337/5
Arshad Hussain [Thu, 15 Jun 2023 22:42:09 +0000 (04:12 +0530)]
LU-16905 tests: Fix 'timeout' value under sanity-quota/18

Presently timeout is picked up via sysctl command
under sanity-quota/18. Which is picking up timeout
from incorrect location. This patch fixes the issue
by reading via $LCTL directly.

sysctl output:
timeout=$(sysctl -n lustre.timeout)
sysctl: cannot stat /proc/sys/lustre/timeout: No such file or directory

Test-Parameters: trivial fstype=zfs testlist=sanity-quota env=ONLY=18
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I4604d31c615fa65bf79598cc09f05d9c7c7abf1b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51337
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
10 months agoLU-16894 lod: fix stripe_count limit in lod_qos_set_pool 14/51314/5
Etienne AUJAMES [Wed, 14 Jun 2023 08:02:01 +0000 (10:02 +0200)]
LU-16894 lod: fix stripe_count limit in lod_qos_set_pool

For a conflicting pool name and stripe offset parameter, the MDS
should not set the pool on the file layout in favor of the stripe
offset (LU-15658). But lod_qos_set_pool() still limit the
stripe_count to the number of OSTs in the specified pool.

Let's fix this.

Fixes: 06dd5a4 ("LU-15658 lod: ost list and pool name conflict")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ic47f9aadd8feea01367e526aaf0ea41a69ade9fa
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51314
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16778 tests: sanity-quota_75 fix 58/51158/4
Sergey Cheremencev [Tue, 30 May 2023 08:14:47 +0000 (11:14 +0300)]
LU-16778 tests: sanity-quota_75 fix

Change conf=fsync to oflag=direct to avoid
cache write.

Test-Parameters: trivial testlist=sanity-quota env=ONLY=75,ONLY_REPEAT=100
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Iff04bac63f772dc2d0d0ad765d210b2539fbe33e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51158
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16836 lnet: ensure dev notification on lnd startup 57/51057/6
Serguei Smirnov [Fri, 19 May 2023 02:12:19 +0000 (19:12 -0700)]
LU-16836 lnet: ensure dev notification on lnd startup

Look up device and link state on lnd startup so that
the initial NI state may be set properly.

Reduce code duplication by adding lnet_set_link_fatal_state() and
lnet_get_link_status() functions which are shared across LNDs.
LND-specific versions of these are removed.

This fixes the issue with adding LNet NI using an interface with
cable unplugged which results in the NI state initialized as "up".

Fixes: da230373bd ("LU-16563 lnet: use discovered ni status")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I16084092cc21a4e42dfef4624adfbf57eb4fdecb
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51057
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16709 lnet: fix locking multiple NIDs of the MR peer 30/50530/15
Serguei Smirnov [Tue, 4 Apr 2023 21:02:51 +0000 (14:02 -0700)]
LU-16709 lnet: fix locking multiple NIDs of the MR peer

If Lustre identifies the same peer with multiple NIDs,
as a result of peer discovery it is possible that
the discovered peer is found to contain a NID which is locked
as primary by a different existing peer record.
In this case it is safe to merge the peer records,
but the NID which got locked the earliest should be
kept as primary.

This allows for the first of the two locked NIDs
to stay primary as intended for the purpose of communicating
with Lustre even if peer discovery succeeded
using a different NID of MR peer.

Fixes: aacb16191a ("LU-14668 lnet: Lock primary NID logic")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Iec9f8b70053fe24cddee552358500dfad0234b7f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50530
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-16518 misc: fix clang build errors 32/50332/4
Timothy Day [Wed, 14 Jun 2023 02:19:41 +0000 (02:19 +0000)]
LU-16518 misc: fix clang build errors

Fix several format security errors by explicitly giving
the format to the affected functions.

A write test in badareaio attempts to write more than
2,147,479,552 bytes. The write will never write that
much, so reduce the size of the write to make the test
useful.

Explicitly cast ll_nfs_get_name_filldir as a filldir_t
and NR_WRITEBACK as a zone_stat_item. This silences
some implicit cast errors. These casts can likely be
removed when older kernel support is dropped.

Refactor some code to avoid strncat, which was being
used incorrectly anyway.

Adjust some variables to use more appropriate types.

Inline some functions which are only sometimes used.

Remove a LASSERTF that will never trigger, since u32
is always smaller than IDIF_MAX_OID.

Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I3962611de7d012e544636248353c072c9f9c9830
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50332
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
10 months agoLU-12031 mdt: explicit data version of DoM files 39/47139/7
Mikhail Pershin [Mon, 25 Apr 2022 06:13:53 +0000 (09:13 +0300)]
LU-12031 mdt: explicit data version of DoM files

Use EA to store 'data_version' for DoM files explicitly.

Unlike OST objects the 'inode_version' of DoM file is changed
by metadata operations as well and that leads to problems
during HSM operations, e.g. writing HSM EA with file data
version inside causes DoM object version update making this
HSM EA version obsoleted, also any metadata update on
restored file makes it dirty and prevents second release.

DoM files have now explicitly updated 'data_version' in
addition to ordinary 'inode_version'. The 'data_version'
is updated along with 'inode_version' upon write/truncate and
fallocate operations and is stored as 'trusted.dataver' EA.
Layout swap procedure is updated to move data version between
files being swept along with HSM attributes.
If DoM file is migrated to RAID0 file then 'dataver' EA is
deleted.

Corresponding test 1f is added to sanity-hsm.sh and
207j to sanity.sh.

Test-Parameters: clientversion=2.12.4 testlist=sanity-hsm
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I4689c56394c7323d32cd6f7dd86f58beb6e53353
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47139
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
10 months agoLU-11912 tests: consume precreated objects in parallel 92/51292/4
Andreas Dilger [Tue, 13 Jun 2023 07:02:22 +0000 (01:02 -0600)]
LU-11912 tests: consume precreated objects in parallel

Run the force_new_seq_all() file creations to run in parallel, since
this can take a significant amount of time when there are multiple
MDTs and OSTs (up to 1000s for 4x MDTs and 8x OSTs).

Test-Parameters: trivial testlist=replay-dual mdscount=2 mdtcount=4
Test-Parameters: testlist=replay-ost-single mdscount=2 mdtcount=4
Test-Parameters: testlist=replay-single mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-pfl env=ONLY=0+1+16+27 mdscount=2 mdtcount=4
Fixes: 2fdb1f8d01b9f ("LU-11912 tests: SEQ rollover fixes")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I849370586fe320d1f7df069f0b83980449658d97
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51292
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
10 months agoLU-15934 tests: add a test case for update llog 08/51208/6
Yang Sheng [Sat, 3 Jun 2023 18:47:30 +0000 (02:47 +0800)]
LU-15934 tests: add a test case for update llog

A test case to simulate the update llog corruption
situation. It replaces the catalog file with a
random data. The recovery of mdt will be blocked
if without the fixing patch.

Fixes: 814691bcff ("LU-15934 lod: renew the update llog")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I0ade8d0ff33ddc06b622e5e67cf4b4775dfff129
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51208
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
10 months agoLU-13805 osc: Add debug 88/49988/22
Patrick Farrell [Tue, 14 Feb 2023 18:24:15 +0000 (13:24 -0500)]
LU-13805 osc: Add debug

This adds some minor additional debug for unaligned IO.

The purpose here is just to shorten the length of the
main patch by pulling out supporting bits.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ia1e749788b66f1d6d3440eb99ff3e86c893fd3f3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49988
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
10 months agoLU-13805 clio: Trivial DIO cleanups 87/49987/22
Patrick Farrell [Sat, 11 Feb 2023 18:12:14 +0000 (13:12 -0500)]
LU-13805 clio: Trivial DIO cleanups

This is some minor DIO refactoring and an additional debug
message discovered while working on this.  Extremely minor.

test-parameters: trivial

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ica2a7340ac02d3ba0d8f65662ff4b39026078e81
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49987
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
10 months agoLU-13805 osc: Don't include lock for srvlock 67/50067/16
Patrick Farrell [Mon, 20 Feb 2023 05:19:40 +0000 (00:19 -0500)]
LU-13805 osc: Don't include lock for srvlock

When doing server side locking, it doesn't make sense to do
the 'search for covering lock and send it to the server'
step when building an RPC, because we will not use that
lock.

This can disguise issues on the client, because prolonging
a lock is supposed to let a client avoid eviction if it is
still doing IO under the lock, but we are not.  This can
result in delaying an eviction which should be occurring
because the client can't give the lock back.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6957925bf2d8b7be2340469337906a94a758953d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50067
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
10 months agoLU-15404 ldiskfs: fix truncate during setxattr for el7.9 35/51335/3
Andreas Dilger [Thu, 15 Jun 2023 19:32:05 +0000 (13:32 -0600)]
LU-15404 ldiskfs: fix truncate during setxattr for el7.9

Backport the ext4-delayed-iput.patch to rhel7.9 kernels so the
delayed osd-ldiskfs truncate can use s_misc_wq consistently.

This moves the call to the final iput in a separate thread.
This way, setxattr transactions will never be split into two.
Since the setxattr code adds xattr inodes with nlink=0 into the
orphan list, old xattr inodes will be properly cleaned up in
any case.

Test-Parameters: trivial
Fixes: e239a14001 ("LU-15404 ldiskfs: truncate during setxattr leads to kernel panic")
Change-Id: Idd70befa6a83818ece06daccf9bb6256813ebbe5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51335
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16399 tests: add subtests setup/cleanup records 82/49582/45
Alex Deiter [Mon, 9 Jan 2023 16:19:26 +0000 (20:19 +0400)]
LU-16399 tests: add subtests setup/cleanup records

* Added setup/cleanup records for subtests

Change-Id: Icb203a864fa8785e423a073b4ee0f02ea3d3ac77
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49582
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-12610 cfs: add unlikely to CFS_ macros 91/51291/2
Timothy Day [Tue, 13 Jun 2023 03:33:37 +0000 (03:33 +0000)]
LU-12610 cfs: add unlikely to CFS_ macros

Fix the (hopefully) last few OBD_ users to use
CFS_ macros instead.

Add an 'unlikely()' to CFS_ macros. Some of the
OBD_ macros included this hint. Once those macros
are removed, the hint will be lost. Add it to the
CFS_ macros instead.

The libcfs_fail.h only has a couple style issues
left. Just fix them in this patch.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ie06533b8b408cacf6f6fe2d29a1a8e727ca4280b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51291
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16888 gss: fix ptlrpc_gss automatic loading 64/51264/4
Sebastien Buisson [Fri, 9 Jun 2023 12:50:25 +0000 (14:50 +0200)]
LU-16888 gss: fix ptlrpc_gss automatic loading

ptlrpc_gss kernel module is automatically loaded when a GSS security
flavor is enforced. Loading success is recorded in a static variable
in the ptlrpc module, which prevents further reloading in case
ptlrpc_gss is unloaded while keeping ptlrpc loaded.
Get rid of this static variable as it is not required in order to
avoid calling request_module("ptlrpc_gss") when not needed. Indeed,
once loaded, the static array policies[] has an entry at the
SPTLRPC_POLICY_GSS index, indicating that the ptlrpc_gss module is
loaded.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9bb100a202fe9c3fc455a2ffba6ee6398e19b9bf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51264
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16887 scrub: delete OI when inode missing 63/51263/3
Alexander Boyko [Thu, 1 Jun 2023 14:19:40 +0000 (10:19 -0400)]
LU-16887 scrub: delete OI when inode missing

osd_iget_check() function have no ability to check
OI when osd_iget() returns error, because inode is
lost during error. Let's return old logic.

Scrub doesn't check consistency between OI and inode
for items from inconsistent list. When OI points to
worng inode, OI record should be deleted.

Fixes: 716de353b ("LU-15542 osd-ldiskfs: exclude EA inode from processing")
HPE-bug-id: LUS-11540, LUS-11585
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ic1618db1c8ee24bb307a9cf3f5ca98441a739b7f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51263
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16807 llite: make lsm_sem nested for ll_update_dir_depth 92/51192/4
James Simmons [Fri, 9 Jun 2023 12:09:45 +0000 (08:09 -0400)]
LU-16807 llite: make lsm_sem nested for ll_update_dir_depth

Lockdep is reporting:

chmod/16751 is trying to acquire lock:
 (&lli->lli_lsm_sem){++++}-{3:3}, at: ll_update_dir_depth+0x8b/0x280

but task is already holding lock:
 (&lli->lli_lsm_sem){++++}-{3:3}, at: ll_update_dir_depth+0x7b/0x280

 other info that might help us debug this:
 Possible unsafe locking scenario:

        CPU0
        ----
 lock(&lli->lli_lsm_sem);
 lock(&lli->lli_lsm_sem);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

Lockdep sees acquiring more than one lock of the same lock class
as a potential dead lock. The execption is if the locks are used
for objects that belong to a hierarchy. For our case of the
lsm_sem we do have a hierarchy since a lsm for a child directory
is related to the parent directory lsm that is being protected
as well. Create new lock classes for the lsm_sem with proper
ordering.

Change-Id: I06c890d16816b83492cbeadabde3515ee0233424
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51192
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
11 months agoLU-15458 tests: double delay for sync/async test 83/51183/2
Patrick Farrell [Wed, 31 May 2023 16:28:14 +0000 (12:28 -0400)]
LU-15458 tests: double delay for sync/async test

This test verifies that an async request takes less than
half of the 'delay' time, but occasionally the async
request hits a delay on the file open due to some VM issue
on the MDT.  This causes a spurious failure.

This is already pretty rare, so the simplest thing is to
double the delay time, which should make it extremely rare.

Setting trivial so it just runs sanity.

test-parameters: trivial

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3046303fcc4e10364de9f673fab2142c8cecff64
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51183
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16751 conf: remove outdated config files 50/51150/4
Timothy Day [Fri, 26 May 2023 13:33:17 +0000 (13:33 +0000)]
LU-16751 conf: remove outdated config files

Remove the example modules.conf. This config file isn't really
specific to Lustre, and it very outdated regardless.

The remaining conf files (lustre.dtd, lustre2ldif.xsl, top.ldif)
were added ~15 years ago. They don't appear to account for the
changes in Lustre in the interim. It would be better to remove
them.

Also, update .gitignore.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia96eaac0a57c4b65c98d4b7931c96b344d7abd56
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51150
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16694 tests: remove deprecated sanity/62 97/51097/2
Timothy Day [Tue, 23 May 2023 14:11:24 +0000 (14:11 +0000)]
LU-16694 tests: remove deprecated sanity/62

This test was disabled many years ago. It doesn't seem useful
enough to reenable it. Hence, remove it.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I1a27a1dc271f76358136efb81e793c81e971f037
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51097
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16574 udsp: lnetctl udsp improvements 87/51087/2
Chris Horn [Tue, 9 May 2023 15:58:15 +0000 (09:58 -0600)]
LU-16574 udsp: lnetctl udsp improvements

lnet_udsp_del_policy() did not previously return non-zero, but its
single caller would check for a non-zero and call
lnet_udsp_apply_policies(). This code is removed.
lnet_udsp_del_policy() will now return non-zero but only in the case
where there is no matching policy index. In this case the policies
are not modified and thus we needn't re-apply them.

Modify some error checking for lnetctl udsp commands to provide better
error messages.

Correct typos in lustre_lnet_add_udsp() error messages.

Correct lustre_lnet_del_udsp()'s handling of errno.

Update help text of udsp commands.

Use parse_long() in jt_show_udsp() to parse the idx argument.

Sanity check priority and idx arguments for udsp add/del rather than
silently modifying them when the user passes in bad values.

Implement 'lnetctl udsp del --all' to provide easy way for admin to
delete all configured policies (this is equivalent to
'lnetctl udsp del --idx -1', but it is more user friendly). The
udsp del command now requires either --all or --idx be specified.

Test-Parameters: trivial
HPE-bug-id: LUS-11490
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ie5e91d8ac1c810473768566593e993e47070e14c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51087
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16808 lfs: return invalid=-1 if no projid retrieved 16/50916/8
Bobi Jam [Wed, 10 May 2023 15:15:24 +0000 (23:15 +0800)]
LU-16808 lfs: return invalid=-1 if no projid retrieved

lfs find --printf "%LP" returns -1 as invalid projid if a
special file is found while it does not contain projid.

Fix bug where "lfs find --printf %LP" always printed "0".

Fixes: 6b8e97b76c ("LU-10378 utils: add formatted printf to lfs find")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic1b474145212e0ce091f97281440816d9a437a4f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50916
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Thomas Bertschinger <bertschinger@lanl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
11 months agoLU-16808 lfs: lfs find --printf won't hang on special files 05/50905/6
Bobi Jam [Wed, 10 May 2023 05:11:47 +0000 (13:11 +0800)]
LU-16808 lfs: lfs find --printf won't hang on special files

Add O_NONBLOCK flag on opening special files lest it hang.

Since '--printf' requires gather_all, and in check_projid case and
gather_all case, it open the special file and try to get project id
from it, we shouldn't error out on only gather_all case.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic2afff7bbccf73ff94c64c62799f9b37b18d10a1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50905
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Thomas Bertschinger <bertschinger@lanl.gov>
11 months agoLU-3080 misc: remove checks for pre-3.10 kernels 14/50814/3
Andreas Dilger [Mon, 1 May 2023 03:05:15 +0000 (21:05 -0600)]
LU-3080 misc: remove checks for pre-3.10 kernels

The oldest supported client kernel is el7.9 3.10.x, and even that
is old, so no need to keep checks for 2.6.x kernel versions around.

Test-Parameters: trivial testlist=replay-dual
Test-Parameters: testlist=replay-single
Test-Parameters: testlist=sanity-flr env=ONLY=50d
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0db781c048334e6ef6df102b100d29e13c64fd25
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50814
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16733 tests: wait for recovery done 83/50683/5
Lei Feng [Wed, 19 Apr 2023 03:37:09 +0000 (11:37 +0800)]
LU-16733 tests: wait for recovery done

wait for recovery done in recovery-small.sh/test_153.

Test-Parameters: trivial testlist=recovery-small
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I163b15e1c38f28eea7051c1886d94a79428039ef
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50683
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16747 llapi: fix race in get_root_path_slow() 82/50682/6
Etienne AUJAMES [Tue, 18 Apr 2023 13:34:23 +0000 (15:34 +0200)]
LU-16747 llapi: fix race in get_root_path_slow()

The patch bdf7788d ("LU-8585 llapi: use open_by_handle_at in
llapi_open_by_fid") caches the Lustre root fd to avoid re-openning
it each time an ioctl() is needed on the fs.

For now, only 1 entry is stored. If a llapi call is performed on
another mountpoint, llapi needs to close the old root fd and open a
new one.

A race condition exists at startup, when root_cached.fd is not
initialized yet. Several threads try to determine root information at
the same time (in get_root_path_slow()). Those threads will close(),
open() and update different "root_cached.fd".
The usage of a closed root fd will return EBADFD (e.g: in
llapi_open_by_fid(), llapi_hsm_request() or llapi_fid2path()).

This patch checks if the fs is the same before updating the root
entry. If so, the root entry (and cached root fd) will not be changed.

Add the regresion test sanityn 85 (llapi_root_test).

Test-Parameters: trivial testlist=sanityn env=ONLY=85,ONLY_REPEAT=20
Test-Parameters: testlist=sanityn
Test-Parameters: testlist=sanity
Fixes: bdf7788d ("LU-8585 llapi: use open_by_handle_at in llapi_open_by_fid")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I681aac7d5715022e700cdb092db94deaa6bf6a8f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50682
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Guillaume Courrier <guillaume.courrier@cea.fr>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 months agoLU-16168 tests: fix get_slave_nr in sanity-quota 59/50659/4
Sergey Cheremencev [Mon, 17 Apr 2023 14:44:22 +0000 (17:44 +0300)]
LU-16168 tests: fix get_slave_nr in sanity-quota

Redirect output of wait_update_facet in get_slave_nr
to /dev/null/. Otherwise if wait_update_cond prints
something while waiting, get_slave_nr returns this
output together with slave number causing test_68
to fail.

Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota fstype=zfs
Fixes: 83dd308db5 ("LU-15460 test: wait quota pool to be prepared")
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I813ed31db864897372c7eb6aab4e1f5a4b955f49
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50659
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
11 months agoLU-15873 osp: don't sync on RO device 97/48097/9
Wang Shilong [Thu, 3 Jun 2021 02:39:35 +0000 (10:39 +0800)]
LU-15873 osp: don't sync on RO device

If lower layer device is mounted as RO, call syncing
will trigger warning of transaction start.

Change-Id: I31be379932f145093285cd87de1fc35a1b5cc305
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48097
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
11 months agoLU-16837 csdc: reserve connect bits for compressed layout 08/51108/5
Bobi Jam [Wed, 24 May 2023 00:25:05 +0000 (08:25 +0800)]
LU-16837 csdc: reserve connect bits for compressed layout

Add connect data bit for compressed layout (OBD_CONNECT2_COMPRESS)
and another connect data bit to be used (OBD_CONNECT2_LARGE_NID).

Also reserve obd_connect_data::ocd_compr_type which is a bitmask of
supported compression type to be negotiated between client and MDS.

Test-Parameters: trivial
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I21029c6c3e8a7e690ecc8d489bbb95aec3ab1fa8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51108
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
11 months agoLU-6142 utils: use list_first/list_entry() on list heads 30/50830/7
Mr NeilBrown [Wed, 6 Nov 2019 22:49:03 +0000 (09:49 +1100)]
LU-6142 utils: use list_first/list_entry() on list heads

This patch changes
   list_entry(foo.next, ...)
to
   list_first_entry(&foo, ...)
and
   list_entry(foo.prev, ...)
to
   list_last_entry(&foo, ...)

in cases where 'foo' is a list head - not a list member.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9daaaed044af596f6407801259cfb672150bfc34
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50830
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>