Whamcloud - gitweb
fs/lustre-release.git
14 months agoLU-14692 tests: allow FID_SEQ_NORMAL for MDT0000 54/49754/2
Li Dongyang [Tue, 25 Jan 2022 00:53:33 +0000 (11:53 +1100)]
LU-14692 tests: allow FID_SEQ_NORMAL for MDT0000

Fix the tests asssuming objects created for MDT0000
always have a seq number of 0, to prepare for
deprecating IDIF sequence.

Fix sanity test_312 on ZFS to properly identify which
OST the object was created on, and re-enable it.

Lustre-change: https://review.whamcloud.com/46293
Lustre-commit: eaae4655567b16260237764dadb7ab57df8b0edd

Test-Parameters: testlist=sanity env=ONLY="39r 312"
Test-Parameters: testlist=sanity-scrub env=ONLY=19
Test-Parameters: testlist=sanity-sec env=ONLY=37
Change-Id: I4bffabe25a6f84cdba760aabea1da3429715a283
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49754
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-16101 tests: add sanity/27J to always_except 71/49971/5
Jian Yu [Sun, 12 Feb 2023 07:56:54 +0000 (23:56 -0800)]
LU-16101 tests: add sanity/27J to always_except

This patch adds sanity/27J to always_except for SLES15 SP4
and 5.16.0+ kernels before the issue introduced by upstream
commit 8c8387ee3f55
("mm: stop filemap_read() from grabbing a superfluous page")
is resolved.

Lustre-change: https://review.whamcloud.com/49970
Lustre-commit: 63dd644747f4eab20d640b4d87060e56c20bc37f

Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT=101j testlist=sanity

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Iafde656530fcdc1de9265aacaa9266435c9d5c47
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49971
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Xing Huang <hxing@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-14598 tests: skip conf-sanity test_122b in interop 86/49586/4
Andreas Dilger [Mon, 9 Jan 2023 23:27:46 +0000 (16:27 -0700)]
LU-14598 tests: skip conf-sanity test_122b in interop

Code was fixed in 2.15.0.

Lustre-change: https://review.whamcloud.com/49583
Lustre-commit: a9f83af896f2ce20e9ee430ad371b707c0c140cc

Test-Parameters: trivial testlist=conf-sanity env=ONLY=122 serverversion=2.14.0
Fixes: 747fed818b ("LU-14598 ofd: fix for IDIF sequence at ofd_preprw_write")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6d9480f4b43706b597df6bd74c65959776cf2b5b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49586
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Sarah Liu <sarah@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
14 months agoLU-16187 tests: Fix is_project_quota_supported() 49/49449/2
Arshad Hussain [Mon, 26 Sep 2022 09:31:41 +0000 (15:01 +0530)]
LU-16187 tests: Fix is_project_quota_supported()

is_project_quota_supported() is called from sanity-quota.sh
to verify if the ldiskfs FS $ENABLE_PROJECT_QUOTAS is true
and to verify if current version of lfs command supports
'project'.  To do this it calls 'lfs --help' which is
not supported. This patch moves 'lfs --help' call to
'lfs --list-commands' call to verify if the present
version of lfs supports 'project'

Lustre-change: https://review.whamcloud.com/48654
Lustre-commit: d4848d779bb8716c6df2fe5438fbe00997f87f3d

Test-Parameters: trivial testlist=sanity-quota
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Iba7e6696d3fa9e980088f448ae72b07a4b47f4f2
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49449
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-16655 scrub: upgrade scrub_file from 2.12 format 80/50480/2
Alexander Zarochentsev [Tue, 28 Mar 2023 16:00:09 +0000 (19:00 +0300)]
LU-16655 scrub: upgrade scrub_file from 2.12 format

Scrub_file->sf_oi_count has different offsets in Lustre-2.10,
Lustre-2.12, and Lustre-2.15 due to unintended format changes.
Lustre-2.15 reads sf_oi_count from offset of sf_success_count
and may initialize incorrect number of OI files, and not be
able to do FID lookups for existing filesystem objects.

Fixes: a114f6b8c5 ("LU-13344 servers: change request timeouts to s32")
Fixes: 4c2f028a95 ("LU-9019 osd-ldiskfs: migrate to 64 bit time")
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: Id7c8bd555229405d604456c48447f01fd121aca9
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50480
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
15 months agoLU-16221 kernel: new kernel [RHEL 9.1 5.14.0-162.18.1.el9_1] 24/49124/9
Jian Yu [Thu, 9 Mar 2023 06:10:43 +0000 (22:10 -0800)]
LU-16221 kernel: new kernel [RHEL 9.1 5.14.0-162.18.1.el9_1]

This patch makes changes to support new RHEL 9.1 release
for Lustre client.

Lustre-change: https://review.whamcloud.com/48938
Lustre-commit: a05d02ea0e43bc656b0c25b8cd821323857e6cc2

Test-Parameters: trivial clientdistro=el9.1 \
env=SANITY_EXCEPT=101j testlist=sanity

Test-Parameters: trivial serverdistro=el8.7 clientdistro=el9.1 \
env=SANITY_EXCEPT=101j testlist=sanity

Change-Id: I8af730f84c9ddf9dcb7e3ddfbd24a68173f51e8d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49124
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
15 months agoLU-16510 build: fortified memcpy from linux 6.1 15/49815/5
Shaun Tancheff [Thu, 9 Feb 2023 03:50:05 +0000 (19:50 -0800)]
LU-16510 build: fortified memcpy from linux 6.1

The fortified memcpy() from Linux v5.11-11104-ga28a6e860c6c
through v5.18-rc5-1405-g43213daed6d6 incorrectly reports
a false positive out of bounds check.

In function 'memcpy' ...
  '__read_overflow2' declared with attribute error: detected
   read beyond size of object passed as 2nd parameter

Lustre-change: https://review.whamcloud.com/49811
Lustre-commit: 919b93b951d4a9aa0400b9c882a1f68b79d8f118

Test-Parameters: trivial
HPE-bug-id: LUS-11459
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I3a59d8b647833c05ff4b51e327ed8bce894141fe
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49815
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-13485 libcfs: Parallel configure tests for libcfs 92/49092/7
Shaun Tancheff [Thu, 9 Mar 2023 05:41:15 +0000 (21:41 -0800)]
LU-13485 libcfs: Parallel configure tests for libcfs

Transform the compile tests in libcfs to run in parallel

Lustre-change: https://review.whamcloud.com/38349
Lustre-commit: 182fa9be075f5866aba2f37fbc3434cd0292ac0e

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I76ab65558dd456dc08d6ef4a1985455ce1f17913
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49092
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-12275 sec: remove bio functions in fscrypt compat 40/50140/3
Andreas Dilger [Wed, 8 Mar 2023 20:36:46 +0000 (12:36 -0800)]
LU-12275 sec: remove bio functions in fscrypt compat

Remove libcfs/llibcfs/crypto/bio.c since direct block device access
is not needed for client builds, and the use of stuct bio on the
client adds unnecessary complexity to portability.

Lustre-change: https://review.whamcloud.com/50023
Lustre-commit: d328818a456daf30c20c8df0aa0be9dd2a2b6a9e

Test-Parameters: trivial
Fixes: a813e8187 ("LU-12275 sec: add llcrypt as file encryption library")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I97642dfd85053b9ea4196374f2002ffb6a2540e5
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50140
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16210 llite: replace selinux_is_enabled() 30/49630/9
Etienne AUJAMES [Fri, 10 Feb 2023 01:22:56 +0000 (17:22 -0800)]
LU-16210 llite: replace selinux_is_enabled()

selinux_is_enabled() was removed from kernel 5.1.
The commit 39e5bfa add the kernel support by assuming SELinux to be
enabled if the function selinux_is_enabled() does not exist.

This has performances impacts: on older kernel (e.g: Centos7) getxattr
RPCs was not send for "security.selinux" if selinux was disabled.
Utilities like "ls -l" always try to get "security.selinux".
See the LU-549 for more information.

This patch uses security_inode_listsecurity() when mounting the
client to know if a LSM module (selinux) required a xattr to store
file contexts. If a xattr is returned we store it and use it for in
request security context.

For getxattr/setxattr we use the stored LSM's xattr to filter xattr
security contexts like security.selinux. If xattr does not match the
stored xattr name we returned -EOPNOTSUPP to userspace.

It adds also the s_security check for security_inode_notifysecctx() to
avoid calling this function if selinux is disabled (as in
nfs_setsecurity()).

For "Enforcing SELinux Policy Check" functionnality, the selinux check
have been moved in l_getsepol: -ENODEV is returned if selinux is
disabled.

Add a regresion test "sanity test_434" for this use case.

*Note:*
This patch detects that selinux is disabled without explicitly
disabled it in kernel cmdline. This is recommended for RHEL >= 8.5.

*Performances:*
Tests with "strace -c ls -l" with 100000 files on root in a multi VMs
env (on Rocky 9). FS is remount for each tests (cache is cleaned) and
selinux is disabled.
 __________________ ___________ _________
| Total time %     | lgetxattr | statx   |
|__________________|___________|_________|
|Without the patch:|    29%    |   51%   |
|__________________|___________|_________|
|With the patch:   |    0%     |   87%   |
|__________________|___________|_________|
"ls -l" uses lgetxattr to get "security.selinux".

Linux-commit: 3d252529480c68bfd6a6774652df7c8968b28e41

Lustre-change: https://review.whamcloud.com/48875
Lustre-commit: 1d8faaf6caf4acaf0e2d4943b51c024a96c80624

Fixes: 39e5bfa ("LU-12355 llite: include file linux/selinux.h removed")
Fixes: 9bcac0b ("LU-549 llite: Improve statfs performance if selinux is disabled")
Test-Parameters: clientselinux=false clientdistro=el7.9 testlist=sanity env=ONLY=434,ONLY_REPEAT=20
Test-Parameters: clientselinux=false clientdistro=el8.5 testlist=sanity env=ONLY=434,ONLY_REPEAT=20
Test-Parameters: clientselinux=false clientdistro=el8.6 testlist=sanity env=ONLY=434,ONLY_REPEAT=20
Test-Parameters: clientselinux clientdistro=el8.6 testlist=sanity-selinux
Test-Parameters: clientselinux clientdistro=el8.6 testlist=sanity-selinux
Test-Parameters: clientselinux clientdistro=el7.9 testlist=sanity-selinux
Test-Parameters: clientselinux clientdistro=el7.9 testlist=sanity-selinux
Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: I4dac87ac0341b45a1c2fef836cdce0361017b3f5
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-14645 tests: test lfs setdirstripe with '/$' 50/49650/4
Jian Yu [Wed, 8 Mar 2023 20:24:18 +0000 (12:24 -0800)]
LU-14645 tests: test lfs setdirstripe with '/$'

This patch improves one of the lfs setdirstripe tests to
verify that dir name ending with '/' also works.

Lustre-change: https://review.whamcloud.com/49463
Lustre-commit: 4b9a39d3ed58a664a2498911ca1d3c9073c13bd3

Test-Parameters: trivial mdscount=2 mdtcount=4 \
env=ONLY=24B testlist=sanity

Change-Id: I237d5a9ebad42cc0569aa1db487d0df147372316
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49650
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-15833 llapi: don't use realpath in llapi_search_fsname() 49/49649/4
Etienne AUJAMES [Fri, 10 Feb 2023 01:17:58 +0000 (17:17 -0800)]
LU-15833 llapi: don't use realpath in llapi_search_fsname()

This patch use st_dev value to dertermine the fsname in
llapi_search_fsname().
The main purpose of this is to limit the number of lstat()
(realpath()) in this function.

get_root_path() is modified to search a mountpoint by dev.
And the last results of get_root_path() is cached to avoid reading
/proc/mount for each call.

A new api function llapi_search_rootpath_by_dev() is added to get
the path of Lustre mountpoint using the specified device value.

**Testing:**

*Environement:*
VMs: 1 client, 1 MDS (2MDT), 1 OSS (2 OST)
Lustre tree: test{001..100}/test{001..100}/test{01..10}/file{01..05}
(500000 files + 110100 folders)
OS: Centos 7 (no statx)
Lustre: 2.15.50_15_g1116739

*Tests*
cd <rootfs>
strace lfs getstripe -r .
echo 3 > /proc/sys/vm/drop_caches
/usr/bin/time lfs getstripe -r . (2 iterations)

*Results*
times (s):

                 ______________________________
                | user | system | real | real% |
 _______________|______|________|______|_______|
|without patch: | 6.18 | 57.3   | 427  | 0%    |
|_______________|______|________|______|_______|
|with patch:    | 2.88 | 47.3   | 404  |-5.45% |
|_______________|______|________|______|_______|

strace (only significant changes are displayed):
(*stat = lstat + stat + fstat)
                 _____________________________________________
                | *stat  | mmap   | open   | read   | all     |
 _______________|________|________|________|________|_________|
|without patch: | 760545 | 110142 | 330379 | 330325 | 4742658 |
|_______________|________|________|________|________|_________|
|with patch:    | 440484 | 0      | 220277 | 19     | 3541739 |
|_______________|________|________|________|________|_________|

-25.32% syscalls after patching.

Lustre-change: https://review.whamcloud.com/47258
Lustre-commit: 4fd7d5585d33240a658f57bf7399da4415a7eb6c

Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: I3812d922d5b1d194d52132cba95d11820424c5d7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49649
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
15 months agoLU-11695 som: disabling xattr cache for LSOM on client 52/49952/3
Qian Yingjin [Wed, 8 Mar 2023 20:16:53 +0000 (12:16 -0800)]
LU-11695 som: disabling xattr cache for LSOM on client

To obtain uptodate LSOM data, currently a client needs to set
llite.*.xattr_cache =0 to disable the xattr cache on client
completely. This leads that other kinds of xattr can not be cached
on the client too.
This patch introduces a heavy-weight solution to disable caching
only for LSOM xattr data ("trusted.som") on client.

Lustre-change: https://review.whamcloud.com/33711
Lustre-commit: 192902851d73ec246af92a2ff7be8f23b08c4343

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iab5ef3030b05ac09184d01f2a3a8ed92ff1cf26b
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49952
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16292 llite: delete_from_page_cache not exported 21/49121/6
Shaun Tancheff [Wed, 8 Mar 2023 20:13:19 +0000 (12:13 -0800)]
LU-16292 llite: delete_from_page_cache not exported

Linux commit v5.16-rc4-44-g452e9e6992fe
filemap: Add filemap_remove_folio and __filemap_remove_folio

Directly removing a folio/page from the page cache is not
available.

Fallback to generic_error_remove_page for regular files,
and truncate_inode_pages_range as appropriate.

Lustre-change: https://review.whamcloud.com/49069
Lustre-commit: 738e69d4b97d28ef037fe50f4146aabead9a2528

Test-Parameters: trivial
HPE-bug-id: LUS-11198
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I634e7d7719d497ce035a78b424be8e9e8c5a8104
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49121
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
15 months agoLU-16118 build: Workaround __write_overflow_field errors 18/49118/6
Shaun Tancheff [Wed, 8 Mar 2023 20:08:27 +0000 (12:08 -0800)]
LU-16118 build: Workaround __write_overflow_field errors

Linux commit v5.17-rc3-1-gf68f2ff91512
   fortify: Detect struct member overflows in memcpy() at compile-time

memcpy and memset of collections of struct members
will trigger:

error: call to ‘__write_overflow_field’ declared with attribute
   warning: detected write beyond size of field (1st parameter);
   maybe use struct_group()?
   [-Werror] __write_overflow_field(p_size_field, size);

Lustre-change: https://review.whamcloud.com/48364
Lustre-commit: a3a51806ef361f55421a1bc07f64c78730ae50d5

Test-Parameters: trivial
HPE-bug-id: LUS-11194
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iacd1ab03d1b90ce62b5d7b65e1cd518a5f7981f2
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49118
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
15 months agoLU-16113 build: Fix configure tests for lock_page_memcg 10/49810/3
Shaun Tancheff [Wed, 8 Mar 2023 19:59:01 +0000 (11:59 -0800)]
LU-16113 build: Fix configure tests for lock_page_memcg

Linux commit v5.15-12273-gab2f9d2d3626
   mm: unexport {,un}lock_page_memcg

Fails when lock_page_memcg exists but is not exported.

Adjust usage of [un]lock_page_memcg() to vvp_[un]lock_page_memcg() and
define the mapping accordingly to avoid the compile error.

Lustre-change: https://review.whamcloud.com/49144
Lustre-commit: 61e83a6f13064aa9ae0d0e8ce8244493bad1f1bd

Test-Parameters: trivial
HPE-bug-id: LUS-11189
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I18029d078a00a0b21a14721bcdf953939b4118a1
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49810
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16342 mdt: not copy pool_name to quotactl in reply 06/49806/3
Sergey Cheremencev [Wed, 8 Mar 2023 19:55:06 +0000 (11:55 -0800)]
LU-16342 mdt: not copy pool_name to quotactl in reply

Don not copy pool_name in mdt reply to avoid out-of-bounds:
BUG: KASAN: slab-out-of-bounds in mdt_quotactl+0x13ff/0x1430 [mdt]

Lustre-change: https://review.whamcloud.com/49242
Lustre-commit: 7e5f927458544bd2681027bfc3df6136d059121f

HPE-bug-id: LUS-10579
Change-Id: I34c4cd8aaccd938c95005dca06644e02132def34
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://es-gerrit.dev.cray.com/160899
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49806
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
15 months agoLU-16115 build: Linux 5.17 external module support 14/49114/4
Shaun Tancheff [Wed, 8 Mar 2023 19:51:08 +0000 (11:51 -0800)]
LU-16115 build: Linux 5.17 external module support

Linux commit v5.16-rc3-26-g129ab0d2d9f3

Added quotes around "$(CONFIG_CC_VERSION_TEXT)", however
.config stores CONFIG_CC_VERSION_TEXT with quotes thus
breaking the GNU make Makefile for external modules.

We need to supply a non-quoted value and override the
value in .config before it is used.

Lustre-change: https://review.whamcloud.com/48360
Lustre-commit: 945ab61176213e082dfd6b3a2a35aa4687e31129

Test-Parameters: trivial
HPE-bug-id: LUS-11190
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I997b6987ef37a8c5b9d8f0984e81d9402a2ea705
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49114
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-13485 build: Parallel build configure cache fixes 05/49805/3
Shaun Tancheff [Wed, 8 Mar 2023 19:46:11 +0000 (11:46 -0800)]
LU-13485 build: Parallel build configure cache fixes

This fixes the infrastructure for parallel builds when cached
enabled results are enabled, the critical fix being proper
usage of AC_CACHE_CHECK in the LB2_LINUX_TEST_RESULT macro.

Lustre-change: https://review.whamcloud.com/49149
Lustre-commit: 24c9f1f4c56d266656572e7bd97197d690d49369

Test-Parameters: trivial
Fixes: b0209c2d4d ("LU-13485 build: Enable 2 stage configure tests")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I94d7ff291dfd4f2dc5e218acc811329b986f8fbf
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49805
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16302 llite: Use alloc_inode_sb() to allocate inodes 22/49122/4
Shaun Tancheff [Wed, 8 Mar 2023 19:41:01 +0000 (11:41 -0800)]
LU-16302 llite: Use alloc_inode_sb() to allocate inodes

linux-commit: v5.17-49-g8b9f3ac5b01d
  fs: introduce alloc_inode_sb() to allocate filesystems specific
      inode

Filesystems are expected to use alloc_inode_sb to allocate inodes
for proper lru handling.

Lustre-change: https://review.whamcloud.com/49070
Lustre-commit: a883fec55694491a26b664c2cff7fb904f72b7af

Test-Parameters: trivial
HPE-bug-id: LUS-11332
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ie6f091a01df33738ed2ef6f7fef9c1f9c1a51e03
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49122
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16121 llite: invalidate_folio and dirty_folio 20/49120/3
Shaun Tancheff [Fri, 27 Jan 2023 18:35:47 +0000 (10:35 -0800)]
LU-16121 llite: invalidate_folio and dirty_folio

linux commit v5.17-rc4-10-g128d1f8241d6
   fs: Add invalidate_folio() aops method

A struct folio is often analogous to a struct page however
a struct folio can represent (contain) multiple pages.

linux commit v5.17-rc4-38-g6f31a5a261db
   fs: Add aops->dirty_folio

__set_page_dirty_nobuffers() is replaced with filemap_dirty_folio()

Lustre-change: https://review.whamcloud.com/48366
Lustre-commit: f1604e680ba2b919c0c52c76c75a7e0dc27ec3c5

Test-Parameters: trivial
HPE-bug-id: LUS-11197
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iefe67615b333e066c49c4b884dad5bea3b3ae226
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49120
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16117 build: Avoid excessive modpost warnings 17/49117/3
Shaun Tancheff [Fri, 27 Jan 2023 18:25:25 +0000 (10:25 -0800)]
LU-16117 build: Avoid excessive modpost warnings

To avoid modpost warnings about duplicate symbols do not add
the LINUX_OBJ kernel symbols to the KBUILD_EXTRA_SYMBOLS list

Lustre-change: https://review.whamcloud.com/48362
Lustre-commit: 445f85de2baddbaa8a84a709b98a9d0b8ed4231a

Test-Parameters: trivial
HPE-bug-id: LUS-11192
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I85fc90661efcb66e4aa39c9bd3393dbe4f7ba5eb
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49117
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16116 build: Configure tests for rhltable, bitmap_alloc... 16/49116/3
Shaun Tancheff [Fri, 27 Jan 2023 18:20:26 +0000 (10:20 -0800)]
LU-16116 build: Configure tests for rhltable, bitmap_alloc...

rhel8.6 with kernel 5.18 breaks a couple of compile tests

struct rhltable test fails with:
... error: ‘hlt’ is used uninitialized in this function
    [-Werror=uninitialized]

rdma_wr() test failes with:
... error: assignment discards ‘const’ qualifier from pointer
    target type [-Werror=discarded-qualifiers]
   wr = rdma_wr(NULL);

nla_strdup() test fails due to unused variable 'tmp'

Lustre-change: https://review.whamcloud.com/48361
Lustre-commit: 009faf132dc47dcfb604b95c93211da7d2c644e2

Test-Parameters: trivial
HPE-bug-id: LUS-11191
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ib2b1d223ac809cea157158fe35fd2535b04367df
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49116
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16118 build: Use pde_data() when available 15/49115/3
Shaun Tancheff [Fri, 27 Jan 2023 18:13:38 +0000 (10:13 -0800)]
LU-16118 build: Use pde_data() when available

Linux commit v5.16-11573-g6dfbbae14a7b
   introduce pde_data() and
Linux commit v5.16-11574-g359745d78351
   remove PDE_DATA()

Use PDE_DATA() when pde_data is not available.

Lustre-change: https://review.whamcloud.com/48363
Lustre-commit: d54e8e95de88e13f193bd44eb43662e75b2091c1

Test-Parameters: trivial
HPE-bug-id: LUS-11193
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ida570462acd466a251adc81a14bc1fbf35d96b00
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49115
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16120 build: Add support for kobj_type default_groups 86/49786/3
Shaun Tancheff [Thu, 26 Jan 2023 17:37:41 +0000 (09:37 -0800)]
LU-16120 build: Add support for kobj_type default_groups

Linux commit v5.1-rc3-29-gaa30f47cf666
  kobject: Add support for default attribute groups to kobj_type

Linux commit v5.18-rc1-2-gcdb4f26a63c3
  kobject: kobj_type: remove default_attrs

Switch to using kobj_type default_groups when it is available.
Provide support for default_attrs for older kernels.

Lustre-change: https://review.whamcloud.com/48365
Lustre-commit: 62e9d055d9516ec6a2fe2f0962b23d478a45dad0

HPE-bug-id: LUS-11196
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I43b03c67c22307293a2abc444aa1a73889ca09ee
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49786
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16359 build: RHEL use Module.symvers during find-provides 85/49785/3
Shaun Tancheff [Thu, 26 Jan 2023 17:34:03 +0000 (09:34 -0800)]
LU-16359 build: RHEL use Module.symvers during find-provides

find-provides fails to find module versions on newer
kernels.

The generated Module.symvers is always generated and
correct. Install it to the well known location BUILDROOT
use it to generate provides and ignore it for installation.

Create a new find-provides and find-provides.ksyms for
lustre based on the one provided by the redhat-rpm-config
package using Module.symvers to supply the symbol versions
instead of extracting symbol versions from the .ko files.

Lustre-change: https://review.whamcloud.com/49302
Lustre-commit: bb951b90268bb013b55166e1c0bcae1c2a66b165

Test-Parameters: trivial
HPE-bug-id: LUS-11383
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I01c3b3692e6a2a6be86a6930eaead9df75147f90
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49785
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16291 build: make kobj_type constant 23/49123/3
Jian Yu [Thu, 26 Jan 2023 06:54:37 +0000 (22:54 -0800)]
LU-16291 build: make kobj_type constant

Kernel v5.16-rc2-28-gee6d3dd4ed48:
commit ee6d3dd4ed48ab24b74bab3c3977b8218518247d
driver core: make kobj_type constant.

This patch makes struct kobj_type constant to fix
the following build failure against kernel 5.16:

lustre/obdclass/obd_config.c: In function 'class_modify_config':
lustre/obdclass/obd_config.c:1639:13: error: assignment discards
'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
1639 |         typ = get_ktype(kobj);
     |             ^

Lustre-change: https://review.whamcloud.com/49043
Lustre-commit: d1dbf26afd6676e02a2a00e635b9ad1fe14cf68e

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I19e0d1f4e3cf97f6871e038487cda9294ac1f67b
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49123
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16114 build: Update security_dentry_init_security args 29/49629/3
Shaun Tancheff [Thu, 26 Jan 2023 06:28:33 +0000 (22:28 -0800)]
LU-16114 build: Update security_dentry_init_security args

Linux commit v5.15-rc1-20-g15bf32398ad4
   security: Return xattr name from security_dentry_init_security()

Adjust security_dentry_init_security() calls accordingly

Lustre-change: https://review.whamcloud.com/48359
Lustre-commit: 88bccc4fa4dd7310560f588c730eefedf423c515

Test-Parameters: trivial
HPE-bug-id: LUS-11188
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I42d3307f7fe0d2412381363f60ac5b3df2d5891a
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49629
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
15 months agoLU-16112 build: ki_complete removed unused argument 84/49784/2
Shaun Tancheff [Thu, 26 Jan 2023 06:24:24 +0000 (22:24 -0800)]
LU-16112 build: ki_complete removed unused argument

Linux commit v5.15-rc6-145-g6b19b766e8f0
   fs: get rid of the res2 iocb->ki_complete argument

Prior to 4.1 Linux provided an aio_complete(iocb, res, res2)
which propagated res2 to io_event.res2. This functionality
migrated to iocb->ki_complete().

Provide a wrapper around iocb->ki_complete() to use
aio_complete() or iocb->ki_complete() as appropriate.

Lustre-change: https://review.whamcloud.com/48357
Lustre-commit: c13eccf71ddee4bc8bc8cbee8f927bdba8768735

HPE-bug-id: LUS-11187
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I11d1ee61528d4d89e2a316fd71066824b202dac7
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49784
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-16111 build: Fix include of stddef.h 05/49105/3
Shaun Tancheff [Thu, 26 Jan 2023 05:23:43 +0000 (21:23 -0800)]
LU-16111 build: Fix include of stddef.h

In kernel builds include the linux/stddef.h

Lustre-change: https://review.whamcloud.com/48367
Lustre-commit: 95f7ef609467a9534b0b1639eba0ef5a2c02f21f

Test-Parameters: trivial
HPE-bug-id: LUS-11185
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0db81e01fadd01445515f96b3d04a2ec51f43044
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49105
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
15 months agoLU-16335 build: remove _GNU_SOURCE dependency in lustre_user.h 76/49776/2
Lai Siyao [Thu, 26 Jan 2023 03:12:08 +0000 (19:12 -0800)]
LU-16335 build: remove _GNU_SOURCE dependency in lustre_user.h

The lustre_user.h header uses the non-standard strchrnul() function
in userspace.  This will always leads to LC_IOC_REMOVE_ENTRY configure
check to fail, and in the end "lfs rm_entry" always returns -ENOTSUP.

Implement an alternative approach to avoid external dependencies on
the lustre_user.h header.  Also, LC_IOC_REMOVE_ENTRY is itself
unnecessary, the code can check for LL_IOC_REMOVE_ENTRY directly.

Replace the NFS-specific -ENOTSUP error return code with -EOPNOTSUPP.

Fix the compile test_400[ab] checks to not use "-std=c99" to verify
that the uapi headers are usable without this dependency.

Lustre-change: https://review.whamcloud.com/49328
Lustre-commit: efc5c8d4de60d394344506f7cfb188eaf04a4bac

Fixes: b59835f8b6 ("LU-13903 utils: have liblustreapi support Linux client")
Fixes: 7a7309fa84 ("LU-13274 uapi: make lustre UAPI headers C99 compliant")
Fixes: 6331eadbd6 ("LU-15420 uapi: avoid gcc-11 -Werror=stringop-overread")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If42743a2148c317b8a9b701ceb5d08bac5149f5f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49776
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
15 months agoLU-9680 utils: fix Netlink / YAML library handling 91/49091/4
James Simmons [Wed, 25 Jan 2023 02:20:14 +0000 (18:20 -0800)]
LU-9680 utils: fix Netlink / YAML library handling

Testing the implementation of Netlink with lustre with early
code revealed some userland lnetconfig library bugs. Several bugs
fixed are:

1) First issue was that the YAML parser and emitter can share an
   netlink socket. This means the netlink callbacks will expect
   that same void argument passed in. We were in the case of error
   handling expecting an struct yaml_netlink_output but all other
   callbacks were using struct yaml_netlink_input. This mismatch
   can cause the application to segfault. So move all netlink
   callback handling to use just the yaml_parser. The yaml
   emittter now is used to just send Netlink packets to the
   kernel.

   Also fix the Netlink ext_ack error message handling.

2) In my board testing I found various bugs related to the
   paring of the YAML to create Netlink packet to send to the
   kernel. This patch resolves all the known issues. Most
   related to the complex layering of sequences, mappings and
   flows in a YAML block.

3) Fix up nla_strlcpy autoconf test which always fails with
   Oleg's special setup.

4) Add a Netlink protocol version YAML function.

Lustre-change: https://review.whamcloud.com/47802
Lustre-commit: 70149f4ea89aa9f5c002b63f56d0c98afcafbad9

Test-Parameters: trivial
Change-Id: I7e7c755ceaa969dffff8c6f771c2ac048dc55720
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Joe Atzinger <joseph.atzinger@microsoft.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49091
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-15393 tests: check QoS hang with OST failover 97/49097/3
Alexander Boyko [Tue, 24 Jan 2023 19:29:14 +0000 (11:29 -0800)]
LU-15393 tests: check QoS hang with OST failover

Patch adds recovery small test 152, to reproduce situation
where MDT object allocation sleeps on OST failover at
lod_ost_alloc_rr under lq_rw_sem read. And all other creation threads
hang at lod_ost_alloc_qos at down_write(lq_rw_sem).

Lustre-change: https://review.whamcloud.com/47715
Lustre-commit: 52057d85eaef8c7b5262f0718629fabff919ff1d

HPE-bug-id: LUS-10388
Test-Parameters: trivial testlist=recovery-small env=ONLY=152
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I7b9c5a5c9870a559e673a5fd253dcaea40d9fe63
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49097
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-15393 lod: skip qos for qos_threshold_rr=100 96/49096/3
Alexander Boyko [Tue, 24 Jan 2023 19:23:43 +0000 (11:23 -0800)]
LU-15393 lod: skip qos for qos_threshold_rr=100

Current implementation of qos allocation is called for
every statfs update. It takes lq_rw_sem for write and
recalculate penalties, even whith setting qos_threshold_rr=100.
Which means always use rr allocation. Let's skip unnecessary
locking and calculation for 100% round robin allocation.

Lustre-change: https://review.whamcloud.com/46388
Lustre-commit: 2f23140d5c1396fd0b247bd7f9c249f6e24096b7

HPE-bug-id: LUS-10388
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I2fcc272d00a988ca4ba0f745b1d5809d65b28654
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49096
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-15393 lod: use killable semaphore for creation path 95/49095/3
Alexander Boyko [Tue, 24 Jan 2023 19:13:32 +0000 (11:13 -0800)]
LU-15393 lod: use killable semaphore for creation path

lod_ost_alloc_qos() function sleeps during ost failover, but object
allocation could use different OSTs. The patch changes
down_write call to down_write_killable and adds timer for a
wakeup.

The main idea of this fix is next, when OST is lost during
lod_ost_alloc_rr() and MDT does not have precreated objects for it
lod_ost_alloc_rr()->..->lod_qos_declare_object_on() would sleep while
holding a lq_rw_sem for read. Any creation thread would stuck at
lod_ost_alloc_qos() waiting lq_rw_sem for write, after statfs update.
Whith a fix sleep is limited and allocation would going through
lod_ost_alloc_rr(). For read lq_rw_sem is shared and stripe allocation
would skip OST without objects.

lod_ost_alloc_rr() refills OST pool with a lq_rw_sem for write, when
lq_rr.lqr_flags has LQ_DIRTY. This should happen only when OST is
added/removed. No need to set LQ_DIRTY for lq_rr when statfs get
error, this flag does not affect any change for pool list at
lod_qos_calc_rr().

Change behaviour for lod_check_and_reserve_ost(), it  would sleep
during object allocation for speed 2 only.

Lustre-change: https://review.whamcloud.com/45921
Lustre-commit: f46782b4c7dcaacd0046ebad3e3d84c2bb0367d4

HPE-bug-id: LUS-10388
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I4768c4cf7d2f9f02f0a9e0dfb6d15e02932cb5fe
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49095
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
16 months agoLU-16523 lprocfs: adjust the format of rename_stats 70/49870/3
Lei Feng [Thu, 2 Feb 2023 01:39:03 +0000 (09:39 +0800)]
LU-16523 lprocfs: adjust the format of rename_stats

Adjust the format of rename_stats to a more human-friendly YAML.

Lustre-change: https://review.whamcloud.com/49869
Lustre-commit: 73b5d7db7e8d3ede42524fc447fb30fa05ea7a3f
Fixes: bedb797c5d ("LU-16110 lprocfs: make job_stats and rename_stats valid YAML")
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I20e6d07c974e907bb2e30412dd1899f845de2021
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49870
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
16 months agoLU-16149 lnet: Discovery queue and deletion race 72/49772/2
Chris Horn [Mon, 12 Sep 2022 21:09:38 +0000 (15:09 -0600)]
LU-16149 lnet: Discovery queue and deletion race

lnet_peer_deletion() can race with another thread calling
lnet_peer_queue_for_discovery.

Discovery thread:
 - Calls lnet_peer_deletion():
 - LNET_PEER_DISCOVERING bit is cleared from lnet_peer::lp_state
 - releases lnet_peer::lp_lock

Another thread:
 - Acquires lnet_net_lock/EX
 - Calls lnet_peer_queue_for_discovery()
 - Takes lnet_peer::lp_lock
 - Sets LNET_PEER_DISCOVERING bit
 - Releases lnet_peer::lp_lock
 - Sees lnet_peer::lp_dc_list is not empty, so it does not add peer
   to dc request queue
 - lnet_peer_queue_for_discovery() returns, lnet_net_lock/EX releases

Discovery thread:
 - Acquires lnet_net_lock/EX
 - Deletes peer from ln_dc_working list
 - performs the peer deletion

At this point, the peer is not on any discovery list, and it has
LNET_PEER_DISCOVERING bit set. This peer is now stranded, and any
messages on the peer's lnet_peer::lp_dc_pendq are likewise stranded.

To solve this, we modify lnet_peer_deletion() so that it waits to
clear the LNET_PEER_DISCOVERING bit until it has completed deleting
the peer and re-acquired the lnet_peer::lp_lock. This ensures we
cannot race with any other thread that may add the
LNET_PEER_DISCOVERING bit back to the peer. We also avoid deleting
the peer from the ln_dc_working list in lnet_peer_deletion(). This is
already done by lnet_peer_discovery_complete().

There is another window where the LNET_PEER_DISCOVERING bit can be
added when the discovery thread drops the lp_lock just before
acquiring the net_lock/EX and calling lnet_peer_discovery_complete().
Have lnet_peer_discovery_complete() clear LNET_PEER_DISCOVERING to
deal with this (it already does this for the case where discovery hit
an error). Also move the deletion of lp_dc_list to after we clear the
DISCOVERING bit. This is to mirror the behavior of
lnet_peer_queue_for_discovery() which sets the DISCOVERING bit and
then manipulates the lp_dc_list.

Also tweak the logic in lnet_peer_deletion() to call
lnet_peer_del_locked() in order to avoid extra calls to
lnet_net_lock()/lnet_net_unlock().

Lustre-change: https://review.whamcloud.com/48532/
Lustre-commit: a966b624ac76e34e8ed28c6980c3f58cb441eeb0

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-11237
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ifcfef1d49f216af4ddfcdaf928024e8ee3952555
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
17 months agoNew release 2.15.2 2.15.2 v2_15_2
Oleg Drokin [Wed, 11 Jan 2023 20:30:07 +0000 (15:30 -0500)]
New release 2.15.2

Change-Id: I1e955e3b98123cc7080994aa4c28b274921ec86e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
17 months agoNew RC 2.15.2-RC2 origin/b2_15 2.15.2-RC2 v2_15_2-RC2
Oleg Drokin [Tue, 3 Jan 2023 22:05:00 +0000 (17:05 -0500)]
New RC 2.15.2-RC2

Change-Id: I956f27aaf856f29cf84aea9c940c7d56ff31387f

17 months agoLU-16433 llite: check vvp_account_page_dirtied 20/49520/4
Jian Yu [Thu, 29 Dec 2022 08:24:13 +0000 (00:24 -0800)]
LU-16433 llite: check vvp_account_page_dirtied

This patch removes duplicated codes from vvp_set_pagevec_dirty()
and check vvp_account_page_dirtied to determine if falling back
to call __set_page_dirty_nobuffers().

HAVE_ACCOUNT_PAGE_DIRTIED_EXPORT also needs to be checked because
vvp_account_page_dirtied is not defined if account_page_dirtied
is exported.

Lustre-change: https://review.whamcloud.com/49512
Lustre-commit: 61c4c2b5e5d7d919149921d913322586ba5ebcab
Test-Parameters: trivial clientdistro=el8.6 testlist=sanity

Test-Parameters: trivial clientdistro=el8.7 testlist=sanity

Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity

Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity

Change-Id: I272033d7494a157145224b1b8ce999a80958aa6c
Fixes: 4bf090b811 ("LU-15959 kernel: new kernel [SLES15 SP4 5.14.21-150400.24.18.1]")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49520
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
18 months agoNew RC 2.15.2 RC1 2.15.2-RC1 v2_15_2-RC1
Oleg Drokin [Sun, 11 Dec 2022 14:44:04 +0000 (09:44 -0500)]
New RC 2.15.2 RC1

Change-Id: Ib47fbd7b9f0cc67f82cb1a247ac781ba13f649b9
Signed-off-by: Oleg Drokin <green@whamcloud.com>
18 months agoLU-16231 misc: fix stats snapshot_time to use wallclock 20/49220/4
Andreas Dilger [Tue, 11 Oct 2022 09:09:08 +0000 (03:09 -0600)]
LU-16231 misc: fix stats snapshot_time to use wallclock

The timestamps reported during stats collection inadvertently changed
from being POSIX epoch timestamps to elapsed-from-boot timestamps.

While some collection tools ignore these timestamps, or only use the
delta between successive reads, having uniform timestaps in stats
files simplifies stats correlation between different servers.

Revert the snapshot_time back to showing wallclock time.

Some "init" times were not initialized when stats were allocated or
cleared, do this for all stats shown by lprocfs_stats_header().

Rename struct osc_device fields from od_ to osc_ to avoid confusion
with struct osd_device. Having two od_stats was especially confusing.

Add a test case to verify snapshot_time, start_time, elapsed_time.

Lustre-change: https://review.whamcloud.com/48821
Lustre-commit: e42efe35eec7b9725f7f4fff86aaee04093366b0

Test-Parameters: testlist=sanity env=ONLY=127a,ONLY_REPEAT=100
Fixes: ea2cd3af7b ("LU-11407 obdclass: add start time to stats files")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I421c3b0301c2566b48c2fc6fe7bb8b54ec48ca5d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49220
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
18 months agoLU-16110 lprocfs: make job_stats and rename_stats valid YAML 15/49215/6
Lei Feng [Fri, 2 Sep 2022 07:05:22 +0000 (15:05 +0800)]
LU-16110 lprocfs: make job_stats and rename_stats valid YAML

Adjust the format of job_stats and rename_stats to make
them valid YAML.

Lustre-change: https://review.whamcloud.com/48417
Lustre-commit: e96cb6ff1fea7a2bc62a6c0786fb0e07cbfda81a
Test-Parameters: testlist=sanity env=ONLY=907
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Idca36621241e97ff87f8ab0448f3c5604057a460
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49215
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
18 months agoLU-15706 llog: deal with "SKIP" pool llog records correctly 30/49230/3
Emoly Liu [Mon, 25 Apr 2022 02:56:55 +0000 (10:56 +0800)]
LU-15706 llog: deal with "SKIP" pool llog records correctly

If the requested "start" llog records by one ioctl are just the
useful ones "protected" by one marker, they will don't know their
cfg_flags(e.g. "SKIP" or not), and then will be mis-labeled in
class_config_yaml_output().
To fix this issue, this patch does the following changes:
- In kernel space, remember the marker cfg_flags for 10 records
  earlier than the requested "start" in function llog_print_cb(),
  so that the "start" can be output with its correct cfg_flags.
- In user space, since the pool_new/add record will be marked as
  "SKIP" if its corresponding pool_destroy/remove record is logged
  later in mgs_pool_cmd(), these "SKIP" records won't be printed
  by function jt_llog_print_iter(), so lpd_ost_num doesn't need to
  be decreased in callback function llog_search_pool_cb() as well,
  otherwise, it will cause "lctl pool_destroy" error.

Lustre-change: https://review.whamcloud.com/46951
Lustre-commit: 474f2670d63b66a77ee3acb72b18bc7b5afbec84

Test-Parameters: standalonemgs=true testlist=ost-pools
Test-Parameters: standalonemgs=true testlist=conf-sanity env=ONLY=123,HONOR_EXCEPT=y
Fixes: 2a5b50d20717 ("LU-15142 lctl: fixes for set_param -P and llog_print")
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ie45127ac8b80a75eaeb7158559c690da52eef103
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49230
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
18 months agoLU-16183 test: sanity-hsm/70 should detect python 47/49247/3
Minh Diep [Mon, 3 Oct 2022 18:22:47 +0000 (11:22 -0700)]
LU-16183 test: sanity-hsm/70 should detect python

Check for python2 and python3 explicitly, since the
generic python command does not exist in newer distros.

Lustre-change: https://review.whamcloud.com/48737
Lustre-commit: c24a38b583cad8e3a92c1d74e8e56e80f07c67b5

Test-Parameters: env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=sles15sp3 testlist=sanity-hsm
Test-Parameters: env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=el7.9 testlist=sanity-hsm

Change-Id: I35bbe15fd298341870ad4f1ab5976e82ccc84667
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Charlie Olmstead <charlie@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49247
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
18 months agoLU-15460 test: wait quota pool to be prepared 50/49250/2
Hongchao Zhang [Fri, 25 Mar 2022 02:34:56 +0000 (10:34 +0800)]
LU-15460 test: wait quota pool to be prepared

When one OST pool was created, the corresponding quota pool
could need more time to prepare after it was created, then
fail to check the info of the quota pool

Lustre-change: https://review.whamcloud.com/46853
Lustre-commit: 83dd308db59a9b95c1db5efa12ea5af22e9c8346

Test-Parameters: trivial
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: Ibea33403639087f27e438d71c0e87fea5367bc3e
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49250
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
18 months agoLU-16295 kernel: kernel update RHEL 7.9 [3.10.0-1160.80.1.el7] 91/49191/3
Jian Yu [Fri, 18 Nov 2022 19:56:49 +0000 (11:56 -0800)]
LU-16295 kernel: kernel update RHEL 7.9 [3.10.0-1160.80.1.el7]

Update RHEL 7.9 kernel to 3.10.0-1160.80.1.el7.

Lustre-change: https://review.whamcloud.com/49045
Lustre-commit: 32dcf69007b74917ec0b6434ad30d00a8fafcb2e

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I50a0ee572d24ddc73f8af6dc32ef701c260e45b7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49191
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
18 months agoLU-16317 build: dkms build requires flex, bison and libmount-devel 72/49272/2
Jian Yu [Tue, 29 Nov 2022 17:01:45 +0000 (09:01 -0800)]
LU-16317 build: dkms build requires flex, bison and libmount-devel

This patch fixes lustre.spec.in and lustre-dkms.spec.in to add
requires for flex, bison, libmount and libmount-devel. The last
two have already been added into lustre.spec.in.

Lustre-change: https://review.whamcloud.com/49183
Lustre-commit: c74c630ff7596317d1b500fd385fca271b31708c

Test-Parameters: trivial

Fixes: 121a79651f ("LU-15967 build: configure script does not check for required build tools")
Fixes: f21b944127 ("LU-15940 build: add a required dependency for libmount")

Change-Id: I9923fc7eb09f974e8c38c3664138486a424e16d7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
19 months agoLU-15305 obdclass: fix race in class_del_profile 78/49178/4
Li Dongyang [Fri, 7 Oct 2022 12:09:10 +0000 (23:09 +1100)]
LU-15305 obdclass: fix race in class_del_profile

Move profile lookup and remove from lustre_profile_list
into the same critical section, otherwise we could race with
class_del_profiles or another class_del_profile.

Do not create duplicate mount opts in the client config,
otherwise we will add duplicate lustre_profile to
lustre_profile_list for a single mount.

Lustre-change: https://review.whamcloud.com/48802
Lustre-commit: 83d3f42118579d7fb7c3002533c047badcf41e0d

Change-Id: I648aa206716213b064d045f546516b219337e0ed
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49178
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
19 months agoLU-16258 llite: Explicitly support .splice_write 58/49058/3
Shaun Tancheff [Fri, 21 Oct 2022 04:54:49 +0000 (23:54 -0500)]
LU-16258 llite: Explicitly support .splice_write

Linux commit v5.9-rc1-6-g36e2c7421f02
  fs: don't allow splice read/write without explicit ops

Lustre supports splice_write and previously provide handlers
for splice_read.
Explicitly use iter_file_splice_write, if it exists.

Lustre-change: https://review.whamcloud.com/48928
Lustre-commit: c619b6d6a54235cc0e34a65cf5916a632f4011c3

HPE-bug-id: LUS-11259
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I858688fc9b4dd370b6018c3b134f01e580477b25
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49058
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
19 months agoLU-16294 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.28.1] 03/49103/3
Jian Yu [Thu, 10 Nov 2022 18:38:57 +0000 (10:38 -0800)]
LU-16294 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.28.1]

Update SLES15 SP4 kernel to 5.14.21-150400.24.28.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/49046
Lustre-commit: 165bf5acdfebe001af8e15f64cd1842972a434b2
Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I651894274a09b6240f321e787736d298c5dc41ce
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49103
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
19 months agoLU-16269 kernel: kernel update RHEL8.6 [4.18.0-372.32.1.el8_6] 32/49032/5
Jian Yu [Tue, 22 Nov 2022 17:52:23 +0000 (09:52 -0800)]
LU-16269 kernel: kernel update RHEL8.6 [4.18.0-372.32.1.el8_6]

Update RHEL8.6 kernel to 4.18.0-372.32.1.el8_6.

Lustre-change: https://review.whamcloud.com/48969
Lustre-commit: 705edb250d62dca84f97f7e97eb468d554611c20

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Change-Id: I5576180ddf10ed2b0a5e2ef85b58fef993de65a4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49032
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
19 months agoLU-16293 kernel: kernel update RHEL9.0 [5.14.0-70.30.1.el9_0] 54/49054/5
Jian Yu [Tue, 22 Nov 2022 18:00:41 +0000 (10:00 -0800)]
LU-16293 kernel: kernel update RHEL9.0 [5.14.0-70.30.1.el9_0]

Update RHEL9.0 kernel to 5.14.0-70.30.1.el9_0 for Lustre client.

Lustre-change: https://review.whamcloud.com/49044
Lustre-commit: 212826c28e369a10cc3afd1306d49eca71b1b9d2

Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Ide942f88242c80af1e103b226b65cfbce94bfb57
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49054
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
19 months agoLU-16304 kernel: kernel update RHEL8.7 [4.18.0-425.3.1.el8] 56/49156/2
Jian Yu [Tue, 15 Nov 2022 00:39:43 +0000 (16:39 -0800)]
LU-16304 kernel: kernel update RHEL8.7 [4.18.0-425.3.1.el8]

Update RHEL8.7 kernel to 4.18.0-425.3.1.el8.

Lustre-change: https://review.whamcloud.com/49080
Lustre-commit: TBD (from 8900b469b4d521361d31ca96fed23c49a141fe93)

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I13e6d83ada1ec0c4da92f307bf56db5281c41892
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49156
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
19 months agoLU-15986 ptlrpc: protect rq_repmsg in ptlrpc_req_drop_rs() 57/49157/2
Lei Feng [Thu, 30 Jun 2022 02:46:31 +0000 (10:46 +0800)]
LU-15986 ptlrpc: protect rq_repmsg in ptlrpc_req_drop_rs()

There is a race condition that: on server side, one thread sent
reply message and is deleting the reply message, another is
searching for existing request and print some debug information
in _debug_req() if there is a duplicated request. They both operate on
req->rq_repmsg but it is not protected in ptlrpc_req_drop_rs().
So we protected it with req->rq_early_free_lock.

Lustre-change: https://review.whamcloud.com/47839
Lustre-commit: aaef545cff2dd958418ec9fb364d4bbe1408edb9

Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Ied55427ee15c3ef84bdd2d579844eba398dbf010
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49157
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
19 months agoLU-16222 kernel: RHEL 8.7 client and server support 93/48993/3
Jian Yu [Mon, 31 Oct 2022 19:13:52 +0000 (12:13 -0700)]
LU-16222 kernel: RHEL 8.7 client and server support

This patch makes changes to support RHEL 8.7 release
with kernel 4.18.0-423.el8 for Lustre client and server.

Lustre-change: https://review.whamcloud.com/48879
Lustre-commit: 293844d132b79a1d256ed4200d5dbd8bb790bfb4

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity

Change-Id: Ie97ff67c9a5fbd46bc145ab559665dcbc630b4a0
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Co-Authored-By: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48993
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
19 months agoLU-15757 llite: check s_root ll_md_blocking_ast() 06/48406/3
Alex Zhuravlev [Tue, 19 Apr 2022 12:01:14 +0000 (15:01 +0300)]
LU-15757 llite: check s_root ll_md_blocking_ast()

ll_md_blocking_ast() can be called in the context of import
invalidation which in turn caused by umount. this way
ll_md_blocking_ast() and umount can race and ll_md_blocking_ast()
can found sb->s_root NULL which should be checked before
calling into is_root_inode().

Lustre-change: https://review.whamcloud.com/47086/
Lustre-commit: 0095c0d05ca80a2494710e3b4afb1d1e4b5cdcfe

Change-Id: I61c1d29a7de3084ad1dfd0c216cee628418b7038
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48406
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
19 months agoLU-16249 sec: krb5_decrypt_bulk calls decryption primitive 08/48908/2
Sebastien Buisson [Tue, 18 Oct 2022 15:19:01 +0000 (17:19 +0200)]
LU-16249 sec: krb5_decrypt_bulk calls decryption primitive

krb5_decrypt_bulk() was mistakenly calling an encryption primitive
instead of a decryption primitive for the confounder.

Lustre-change: https://review.whamcloud.com/48907
Lustre-commit: TBD (851f3915659941db00a0cda58867e68139e5e0d1)

Test-Parameters: trivial
Fixes: 0a65279121 ("LU-13344 gss: Update crypto to use sync_skcipher")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9251172644ed6baa3bb06a59dbe7c1bab401d817
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48908
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
20 months agoLU-16173 kernel: kernel update SLES15 SP3 [5.3.18-150300.59.93.1] 59/48859/3
Jian Yu [Thu, 13 Oct 2022 17:47:23 +0000 (10:47 -0700)]
LU-16173 kernel: kernel update SLES15 SP3 [5.3.18-150300.59.93.1]

Update SLES15 SP3 kernel to 5.3.18-150300.59.93.1 for Lustre client.

Test-Parameters: trivial clientdistro=sles15sp3 \
testlist=sanity

Lustre-change: https://review.whamcloud.com/48601
Lustre-commit: c3467db7e7d0652c09bdcef26e2b708ab51cba9e

Change-Id: I1e0afe6974567d13680dbb0d463fbbd873ef2e5f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48859
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-16050 build: replace ofed_info with dpkg/rpm 92/48592/2
Jian Yu [Mon, 19 Sep 2022 16:53:54 +0000 (09:53 -0700)]
LU-16050 build: replace ofed_info with dpkg/rpm

After installing MLNX_OFED by running mlnxofedinstall command,
mlnx-ofed-kernel-modules package is not listed by ofed_info,
which causes Lustre configure fail as follows:

checking whether to use Compat RDMA... /usr/bin/ofed_info
dpkg-query: error: --listfiles needs at least one package name argument

This patch fixes the above issue by replacing ofed_info with
"dpkg -l" and "rpm -qa" commands to find OFED package.

Lustre-change: https://review.whamcloud.com/48047
Lustre-commit: 3a7930e63c15b0fbe51ac73db81a1186939115bb

Test-Parameters: trivial
Fixes: ec03c9628cae ("LU-15417 build: find the new path for MOFED 5.5")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Ia3c2d6bf10e147ca2761221741eff6f93008556c
Reviewed-by: Gaurang Tapase <gtapase@ddn.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48592
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-15795 lbuild: enable KABI 10/48610/3
Minh Diep [Tue, 20 Sep 2022 18:32:02 +0000 (11:32 -0700)]
LU-15795 lbuild: enable KABI

Enable build kabi and clean up kmodtool patch

Lustre-change: https://review.whamcloud.com/47507
Lustre-commit: 0a639a32cf32423e826420e24726912d5a39f13e

Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.5 serverdistro=el8.5
Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.6 serverdistro=el8.6

Change-Id: I16d54af0004c4ddc1cc5e6acca81e4aa89a1a1c1
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48610
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-16233 build: Add always target for SUSE15 SP3 LTSS 58/48858/3
Shaun Tancheff [Thu, 13 Oct 2022 17:37:54 +0000 (10:37 -0700)]
LU-16233 build: Add always target for SUSE15 SP3 LTSS

SUSE 15 SP3 LTSS kernel version 5.3.18-150300.59.93
(and later) breaks lustre build tests which expect
conftest.i to be generated.

Lustre-change: https://review.whamcloud.com/48833
Lustre-commit: 3493db6a675d6bcbda3b52b41e627cdc983d6c0c

HPE-bug-id: LUS-11286
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If23e9b31b537878a43075ffff62a99906f47fd9a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48858
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-16082 ldiskfs: old-style EA inode fix for el8.5/el8.6 11/48611/2
Andreas Dilger [Tue, 20 Sep 2022 18:54:08 +0000 (11:54 -0700)]
LU-16082 ldiskfs: old-style EA inode fix for el8.5/el8.6

Add the rhel8/ext4-old_ea_inodes_handling_fix.patch to the ldiskfs
series for el8.5 and el8.6 kernels.

Lustre-change: https://review.whamcloud.com/48496
Lustre-commit: ba9845274c8ea5c55f57b7fa0e839f18d76031ea

Test-Parameters: trivial testlist=sanity serverdistro=el8.6
Fixes: 76c3fa96dc30 ("LU-16082 ldiskfs: old-style EA inode handling fix")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifb66a0b7d78e5153d7897bee45fbf1d0e58fbc5c
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48611
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-16174 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.21.2] 88/48688/3
Jian Yu [Wed, 28 Sep 2022 06:56:20 +0000 (23:56 -0700)]
LU-16174 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.21.2]

Update SLES15 SP4 kernel to 5.14.21-150400.24.21.2 for Lustre client.

Lustre-change: https://review.whamcloud.com/48604
Lustre-commit: 5d31fdaf681a8d9c39d197531f010dcd97bf604c

Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity

Change-Id: Ia68e1c960c79f40d0f725b0f440cd562b820a19f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48688
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-16177 kernel: kernel update RHEL9.0 [5.14.0-70.26.1.el9_0] 86/48686/4
Jian Yu [Wed, 28 Sep 2022 06:40:07 +0000 (23:40 -0700)]
LU-16177 kernel: kernel update RHEL9.0 [5.14.0-70.26.1.el9_0]

Update RHEL9.0 kernel to 5.14.0-70.26.1.el9_0 for Lustre client.

Lustre-change: https://review.whamcloud.com/48676
Lustre-commit: 8a2bd9661d07043e61366c77f21c55612c431cbc

Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity

Change-Id: I9da2ccdf419d6490fdba80199eda69f4f19361be
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48686
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
20 months agoLU-16138 kernel: preserve RHEL8.x server kABI for block integrity 09/48609/2
Jian Yu [Tue, 20 Sep 2022 18:29:43 +0000 (11:29 -0700)]
LU-16138 kernel: preserve RHEL8.x server kABI for block integrity

Currently there are two kernel patches supporting SCSI T10-PI feature
left in the RHEL8.x series:

- block-integrity-allow-optional-integrity-functions-rhel8.patch
- block-pass-bio-into-integrity_processing_fn-rhel8.patch

The changes in the patches modified "struct bio_integrity_payload"
and "struct blk_integrity_iter", which caused kABI breakage.

This patch fixes the patches to preserve kABI by using
RH-supplied compatibility macros.

Lustre-change: https://review.whamcloud.com/48608
Lustre-commit: ec555c17c0254662c5886188dd2f7c0447fd4aef

Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.5 serverdistro=el8.5
Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.6 serverdistro=el8.6

Change-Id: If547e1cd4ae4ff1affd315bbfefaeeff4f1dea81
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48609
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
21 months agoLU-15724 tests: MDT failover hang reproducer 49/48549/2
Alexander Boyko [Wed, 14 Sep 2022 20:02:59 +0000 (13:02 -0700)]
LU-15724 tests: MDT failover hang reproducer

The patch adds recovery-small 144a test to reproduce
MDT failover hang when precreate threads are blocked on objects.

LustreError: 0-0: Forced cleanup waiting for mdt-kjcf05-MDT0001_UUID
namespace with 46 resources in use, (rc=-110)

Lustre-change: https://review.whamcloud.com/47006
Lustre-commit: aa6250b7412e7baf6760fe4010a81f4f22187127

Test-Parameters: trivial testlist=recovery-small env=ONLY=144a
HPE-bug-id: LUS-10750
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I2743a1b5c8911d6982b527f7e7b7bbbaf310cd04
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/48549
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15724 osp: wakeup all precreate threads 48/48548/2
Alexander Boyko [Wed, 14 Sep 2022 19:59:41 +0000 (12:59 -0700)]
LU-15724 osp: wakeup all precreate threads

Number of threads could sleep at osp_precreate_reserve() and
wait objects from OST. When MDT stops Lustre should wakeup
all threads. When opd_pre_recovering is set any wakeup of
opd_pre_user_waitq is useless. Failover of MDT does not produce
disconnect event, only inactive, so osp_precreate_cleanup_orphans()
can not be awakened.

LustreError: 0-0: Forced cleanup waiting for mdt-kjcf05-MDT0001_UUID
namespace with 46 resources in use, (rc=-110)

 schedule_timeout at ffffffff8e551cd3
 osp_precreate_reserve at ffffffffc17d2d83 [osp]
 osp_declare_create at ffffffffc17c7eb9 [osp]
 lod_sub_declare_create at ffffffffc156415b [lod]
 lod_qos_declare_object_on at ffffffffc155bf42 [lod]
 lod_ost_alloc_rr.constprop.23 at ffffffffc155db2f [lod]
 lod_qos_prep_create at ffffffffc15630a6 [lod]
 lod_declare_instantiate_components at ffffffffc154b237 [lod]

Lustre-change: https://review.whamcloud.com/47005
Lustre-commit: e55fc043679cdfadfff6874ef78e2e0128ec37ac

HPE-bug-id: LUS-10750
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: If0164cfbecb1e358d9857421cb234559dc8cecbc
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/48548
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16000 utils: align updatelog parameters in llog_reader 52/48552/2
Etienne AUJAMES [Wed, 14 Sep 2022 20:21:23 +0000 (13:21 -0700)]
LU-16000 utils: align updatelog parameters in llog_reader

Parameters in update log records are aligned on 64bits. llog_reader
do not aligned these parameters: if a parameters size is not mutiple
of 8, the next parameter size will be read incorrectly.

Lustre-change: https://review.whamcloud.com/47913
Lustre-commit: 6d74b759634355e7f6647ccaefef519a1ff208e2

Test-Parameters: trivial
Fixes: 9962d6f ("LU-14617 utils: llog_reader updatelog support")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: I6871614ab4ea79d59c3c3b4644b377de395bad56
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48552
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16106 lnet: allow direct messages regardless of peer NI status 29/48529/3
Serguei Smirnov [Mon, 12 Sep 2022 20:50:07 +0000 (13:50 -0700)]
LU-16106 lnet: allow direct messages regardless of peer NI status

If check_routers_before_use is enabled, the router needs to
be pinged before it is used, which is not possible because
its NIs are assumed to be down at start-up. Don't prevent
discovery of the router in this case.

This change allows non-routed traffic to peer NIs with "down"
status.

Lustre-commit: 4c6d1491d1df078a4477b3fcfeddadf3d1fd7239
Lustre-change: https://review.whamcloud.com/c/48355

Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I36fa60e37ef4f47c82c69855c9b0b80bad8a36f4
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/48529
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15911 enc: null encrypted names is embedded llcrypt only 21/48521/2
Sebastien Buisson [Fri, 3 Jun 2022 09:16:35 +0000 (11:16 +0200)]
LU-15911 enc: null encrypted names is embedded llcrypt only

enable_filename_encryption tunable only makes sense when Lustre client
is built against embedded llcrypt. When built against in-kernel
fscrypt, this tunable is silently ignored, as fscrypt always carries
out file name encryption.

So have the enable_filename_encryption tunable only when Lustre client
is built against embedded llcrypt. Also fix sanity-sec test_54 so that
it works for in-kernel fscrypt.

Lustre-change: https://review.whamcloud.com/47520
Lustre-commit: 4887aff7f10423674939682a2fd56f0490224e6e

Fixes: e68d496ada ("LU-15858 sec: reinstate null encryption for file names")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ibe52feb670a00c9f421907ecd438bcccc62856f0
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/48521
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11407 tgt: cleanup job_stats output printing 39/48539/2
Andreas Dilger [Tue, 13 Sep 2022 18:30:05 +0000 (11:30 -0700)]
LU-11407 tgt: cleanup job_stats output printing

Escape non-printable and other special characters in the JobID
name, which may be passed from the client environment, to avoid
breaking YAML format parsing.  We can't use the kernel "%*pE"
escape format, because that doesn't have any option to escape
printable characters like quotes or regular spaces.

Lustre-change: https://review.whamcloud.com/37764
Lustre-commit: 338381574b643da064e90e44d75be85d1be3a93c

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If6a0bc4276fae03305f94e8c85d8f109913ebbe5
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/48539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16059 build: Installation of dkms server builds 12/48212/3
Shaun Tancheff [Mon, 19 Sep 2022 16:42:09 +0000 (09:42 -0700)]
LU-16059 build: Installation of dkms server builds

The linux-zfs-dkms package is passing the wrong paths
for zfs [and spl] causing the dkms build to fail.

ZFS_VERSION is not parsed correctly from 'dkms status'.

The splver and zfsver check can match against the wrong
package(s).

lustre-zfs-dkms provides: kmod-lustre-osd-zfs, and
                          lustre-osd-zfs-mount
lustre-ldiskfs-dkms provides: kmod-lustre-osd-ldiskfs and
                              lustre-osd-ldiskfs-mount

In the case of multiple zfs versions installed, build lustre
osd against the highest version number.

Lustre-change: https://review.whamcloud.com/48083
Lustre-commit: c3dc67b2c5bf1974d792b3701d932bd04c756bd8

HPE-bug-id: LUS-11113
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ic154ca045427bf26cb7e6a44b8c467675e987aad
Reviewed-on: https://review.whamcloud.com/48212
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15793 kernel: kernel update RHEL8.5 [4.18.0-348.23.1.el8_5] 92/47392/8
Jian Yu [Tue, 13 Sep 2022 05:36:56 +0000 (22:36 -0700)]
LU-15793 kernel: kernel update RHEL8.5 [4.18.0-348.23.1.el8_5]

Update RHEL8.5 kernel to 4.18.0-348.23.1.el8_5.

Lustre-change: https://review.whamcloud.com/47175
Lustre-commit: 597665de6ee4e77b259a82f126d95cb2d60bb260

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.5 serverdistro=el8.5 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.5 serverdistro=el8.5 testlist=sanity

Change-Id: I9724e304c2d32bb0ac2dcf4138438c79d4abca0f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47392
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16131 build: Do not depend on libmount during --enable-dist 46/48446/3
Shaun Tancheff [Wed, 7 Sep 2022 04:27:04 +0000 (21:27 -0700)]
LU-16131 build: Do not depend on libmount during --enable-dist

Defer the libmount requirement when using --enable-dist to
generate the lustre-src.rpm.

This allows mock and/or yum build-deps to resolve resolve
dependencies and pickup the libmount requirement without changing
the existing minimal build.

Lustre-change: https://review.whamcloud.com/48407
Lustre-commit: 819c8b169325045ae8bac9c4f38a58c75e22d099

Test-Parameters: trivial
HPE-bug-id: LUS-11091
Fixes: f21b944127 ("LU-15940 build: add a required dependency for libmount")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I20a7a097f9b651b6ea5519f79efda6c96b6f2199
Reviewed-on: https://review.whamcloud.com/48446
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15811 llite: Refactor DIO/AIO free code 56/48456/2
Patrick Farrell [Wed, 7 Sep 2022 17:38:37 +0000 (10:38 -0700)]
LU-15811 llite: Refactor DIO/AIO free code

Refactor the DIO/AIO free code and add some asserts.

This removes a potential use-after-free in the freeing
code.

Lustre-change: https://review.whamcloud.com/48115
Lustre-commit: f1c8ac1156ebea2b8e94a75f056c403973bb9010

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I335b18fc7a28fc426a25675e2449d3d192cba596
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/48456
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15811 llite: Unify range unlock 55/48455/2
Patrick Farrell [Wed, 7 Sep 2022 17:32:32 +0000 (10:32 -0700)]
LU-15811 llite: Unify range unlock

Correct parallel_dio condition and unify range unlock code
block.

Lustre-change: https://review.whamcloud.com/48000
Lustre-commit: 36c34af60767bd5da5e9a66df0297e1dc7fa1fb1

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ib66e8def571054df5117c279e238894bc3b58bce
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/48455
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15811 llite: Rework upper/lower DIO/AIO 54/48454/2
Patrick Farrell [Wed, 7 Sep 2022 17:27:07 +0000 (10:27 -0700)]
LU-15811 llite: Rework upper/lower DIO/AIO

One of the patches for LU-13799,
"Implement lower/upper aio"
(https://review.whamcloud.com/44209/) created a
complicated setup where the cl_dio_aio struct was used
both for the top level DIO or AIO and for the lower level
sub I/Os (corresponding to stripes).

This is quite complicated and hard to follow, so this
rewrites these two uses to be separate structs.  This
incidentally fixes at least one possible memory leak, but
is mostly a cleanup.

Lustre-change: https://review.whamcloud.com/47187
Lustre-commit: 51c18539338f1a23f3fa725470b5d36941b15c46

Fixes: 46ff761371 "LU-13799 Implement lower/upper aio"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ide4a2b84f48624ee97dfb57fe80d201fbb7fe8d0
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/48454
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16075 kernel: kernel update RHEL8.6 [4.18.0-372.19.1.el8_6] 15/48315/5
Jian Yu [Tue, 23 Aug 2022 23:20:44 +0000 (16:20 -0700)]
LU-16075 kernel: kernel update RHEL8.6 [4.18.0-372.19.1.el8_6]

Update RHEL8.6 kernel to 4.18.0-372.19.1.el8_6.

Lustre-change: https://review.whamcloud.com/48116
Lustre-commit: 872308001bf43ae3f190ceea11850f2f4eb8b7b3

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Change-Id: I8e0fbdab54d36512c4c4cbdbc97c580994ebcbd3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48315
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16058 build: proc_ops check fails with SUBARCH undefined 23/48423/2
Shaun Tancheff [Fri, 2 Sep 2022 18:37:33 +0000 (11:37 -0700)]
LU-16058 build: proc_ops check fails with SUBARCH undefined

During configure with config.cache enabled SUBARCH may not
be defined.

Move the definition to a location that must be traversed.

Lustre-change: https://review.whamcloud.com/48101
Lustre-commit: 8a7aa8d5902fcffe6c8b56d0c92ee36380a26e88

Test-Parameters: trivial
Fixes: a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0a7b4de3ecccd41b1c55e8b2df29039517e0c416
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48423
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16090 build: Module.symvers lookup by flavor on SUSE 28/48328/3
Shaun Tancheff [Wed, 24 Aug 2022 19:26:13 +0000 (12:26 -0700)]
LU-16090 build: Module.symvers lookup by flavor on SUSE

When multiple kernel flavors are found we need to select only
the Module.symvers for the flavor that is being built.

Lustre-change: https://review.whamcloud.com/48195
Lustre-commit: f3a9921ae4f9c3e48328f2c682e0c7e61221e0d3

HPE-bug-id: LUS-11149
Test-Parameters: trivial
Fixes: 1f4aaefe1aae ("LU-15962 build: add in-kernel Module.symvers to symbol path")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I1c9af91108534d3a67f816077756fded4cd0b653
Reviewed-on: https://review.whamcloud.com/48328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16082 ldiskfs: old-style EA inode handling fix 12/48412/2
Alexander Zarochentsev [Thu, 1 Sep 2022 17:14:10 +0000 (10:14 -0700)]
LU-16082 ldiskfs: old-style EA inode handling fix

The upstream version of EA inodes support coming
with RHEL8 (linux kernel 4.18+) have a slightly different
implementation of EA inodes support and also have a
compatibility code to recognize old-style Lustre-only EAs.
Unfortunately the compatibility code is broken and makes
old xattr data unaccessible due to a wrong hash value check.

Lustre-change: https://review.whamcloud.com/48174
Lustre-commit: 76c3fa96dc30f21e95d80f9119972d7358975258

HPE-bug-id: LUS-11133
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: Icd6f93d4ebb33dcd03b58f9eb364905c18ae81dc
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48412
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16056 libcfs: restore umask handling in kernel threads 14/48414/2
Andreas Dilger [Thu, 1 Sep 2022 17:31:48 +0000 (10:31 -0700)]
LU-16056 libcfs: restore umask handling in kernel threads

This reverts commit 9013eb2bb5 which incorrectly assumes that Lustre
service threads do not modify umask.  A quick grep shows that umask
is modified in osd-ldiskfs __osd_create().

If some other thread sharing the same fs context is modifying umask
in an incompatible way (which includes all Lustre threads after
this patch) then it will occasionally break created file access
permissions for Lustre.

Lustre-change: https://review.whamcloud.com/48233
Lustre-commit: c92bdd97d99cc755a187987f3d5963adeb3ea475

Change-Id: I589b72e4286dc84f4e3f1a0c54fe31aa988e6c18
Fixes: 9013eb2bb5 (LU-9859 libcfs: don't call unshare_fs_struct()")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/48414
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15003 sec: use enc pool for bounce pages 11/48411/2
Sebastien Buisson [Fri, 25 Mar 2022 08:24:32 +0000 (09:24 +0100)]
LU-15003 sec: use enc pool for bounce pages

Take pages from the enc pool so that they can be used for
encryption, instead of letting llcrypt allocate a bounce page
for every call to the encryption primitives.
Pages are taken from the enc pool a whole array at a time.

This requires modifying the llcrypt API, so that new functions
llcrypt_encrypt_page() and llcrypt_decrypt_page() are exported.
These functions take a destination page parameter.
Until this change is pushed in upstream fscrypt, this performance
optimization is not available when Lustre is built and run against
the in-kernel fscrypt lib.

Using enc pool for bounce pages is a worthwhile performance win. Here
are performance penalties incurred by encryption, without this patch,
and with this patch:

                     ||=====================|=====================||
                     || Performance penalty | Performance penalty ||
                     ||    without patch    |     with patch      ||
||==========================================|=====================||
|| Bandwidth – write |        30%-35%       |   5%-10% large IOs  ||
||                   |                      |    15% small IOs    ||
||------------------------------------------|---------------------||
|| Bandwidth – read  |         20%          |    less than 10%    ||
||------------------------------------------|---------------------||
||      Metadata     |         N/A          |         5%          ||
|| creat,stat,remove |                      |                     ||
||==========================================|=====================||

Lustre-change: https://review.whamcloud.com/47149
Lustre-commit: f3fe144b8572e9e75bb55076e29057227476ebf5

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
Change-Id: I3078d0a3349b3d24acc5e61ab53ac434b5f9d0e3
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48411
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16085 llite: fix stat attributes_mask 04/48404/3
Sebastien Buisson [Fri, 12 Aug 2022 07:59:02 +0000 (09:59 +0200)]
LU-16085 llite: fix stat attributes_mask

Fix stat attributes_mask to return STATX_ATTR_ENCRYPTED whenever it is
possible. Also fix sanityn test_106c to expect at least the 0x30 flag
for attributes_mask.

Lustre-change: https://review.whamcloud.com/48208
Lustre-commit: 0e48653c27eacad29dbff1589da771ad4f5d1014
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
LU-16085 tests: fix sanityn test_106c

Fix sanityn test_106c after modification introduced when fixing
stat attributes_mask.

Lustre-change: https://review.whamcloud.com/48435
Lustre-commit: TBD (12ab7defee65a82b495dab9ba2da03dcc7d55799)

Test-Parameters: trivial testlist=sanityn env=ONLY=106c
Fixes: 0e48653c27 ("LU-16085 llite: fix stat attributes_mask")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Icd16beff058c42d77e9b04ad1a287ec2ac04dfed
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/48404
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-9243 gss: fix GSS struct definition badness 64/47664/2
Sebastien Buisson [Thu, 17 Feb 2022 15:40:20 +0000 (16:40 +0100)]
LU-9243 gss: fix GSS struct definition badness

struct lgssd_ioctl_param should not be defined in multiple places. So
move it to a new header file lgss.h that can be included from kernel
space and user space.

struct gss_header, struct gss_rep_header, struct gss_err_header and
struct gss_wire_ctx are going on the wire, so they need to be moved to
lustre_idl.h, and be wire-checked.

Lustre-change: https://review.whamcloud.com/46543
Lustre-commit: 7dfbc71350c982be20d75e7d7853038809f4e850

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I97c4a8322e6bb7627c6dff5f068931278f4567d7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47664
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16089 kernel: kernel update RHEL 7.9 [3.10.0-1160.76.1.el7] 70/48270/3
Jian Yu [Sat, 20 Aug 2022 00:07:07 +0000 (17:07 -0700)]
LU-16089 kernel: kernel update RHEL 7.9 [3.10.0-1160.76.1.el7]

Update RHEL 7.9 kernel to 3.10.0-1160.76.1.el7.

Lustre-change: https://review.whamcloud.com/48202
Lustre-commit: TBD (from 109d26b15b4d24d11bcb76fef00bf06dd7d5e975)

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I97d087a5d5bb27996a5c0caf382c011928c651b4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48270
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-15508 gss: protect from arbitrary write to init channel 68/47668/2
Sebastien Buisson [Fri, 4 Feb 2022 14:53:35 +0000 (15:53 +0100)]
LU-15508 gss: protect from arbitrary write to init channel

In case some arbitrary data was written to the gss init channel,
directly return -EINVAL. This protects against unsolicited
authentication requests, and avoids having a dandling entry
in the auth init cache.

Lustre-change: https://review.whamcloud.com/46461
Lustre-commit: f06807d409c82dd85c208c79c8072402e77f9bac

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iadde630012e4ded83f9609fbb3e10b2e092deb57
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-on: https://review.whamcloud.com/47668
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16045 enc: force use of new enc xattr on new servers 36/48036/3
Sebastien Buisson [Mon, 25 Jul 2022 14:39:56 +0000 (16:39 +0200)]
LU-16045 enc: force use of new enc xattr on new servers

When an older client uses encryption with a newer server, the client
wants to see the encryption context in security.c xattr. But
internally on server side, we force use of newer encryption.c xattr
for consistency purpose. When required, the encryption context is put
in the request to the client as usual, which interprets it as desired.

Lustre-change: https://review.whamcloud.com/48035
Lustre-commit: afa4c31087f21fef8d64f572e08a7bec77bc7325

Fixes: 4231fab66e ("LU-13717 sec: make client encryption compatible with ext4")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I667e123bdff912acc270666e8c74ebda6f0534e7
Reviewed-on: https://review.whamcloud.com/48036
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-16012 sec: fix detection of SELinux enforcement 93/48193/3
Sebastien Buisson [Wed, 27 Jul 2022 12:39:26 +0000 (12:39 +0000)]
LU-16012 sec: fix detection of SELinux enforcement

On newer distros (e.g. RHEL 9.0), on which selinux_is_enabled() does
not exist anymore, the only way to find out if SELinux is enforced
when initializing the security context is to fetch the length of the
security attribute name. If it is 0, we conclude SELinux is disabled.

Lustre-change: https://review.whamcloud.com/48049
Lustre-commit: 155cbc22ba4f758cf9eec415f36f940ca2b23de9

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ifcdcb8ffbb7f9ad50d16d7d3317e94d0d212fa42
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48193
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-15959 kernel: new kernel [SLES15 SP4 5.14.21-150400.24.18.1] 24/47924/15
Jian Yu [Tue, 16 Aug 2022 01:04:25 +0000 (18:04 -0700)]
LU-15959 kernel: new kernel [SLES15 SP4 5.14.21-150400.24.18.1]

This patch makes changes to support new SLES15 SP4 release
with kernel 5.14.21-150400.24.18.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/47696
Lustre-commit: TBD (from 4bf090b81119d02ed5baa59c6857e2c88a746736)

Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity
Test-Parameters: trivial clientdistro=sles15sp3

Change-Id: I0bf548835578163767d2f6a2a5e5bd2b33154871
Co-Authored-By: Minh Diep <mdiep@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47924
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
22 months agoLU-15855 enc: enc-unaware clients get ENOKEY if file not found 90/47790/2
Sebastien Buisson [Fri, 13 May 2022 15:03:22 +0000 (17:03 +0200)]
LU-15855 enc: enc-unaware clients get ENOKEY if file not found

To reduce issues with applications running on clients without keys
or without fscrypt support that check for the existence of a file in
an encrypted directory, return -ENOKEY instead of -ENOENT.
For encryption-unaware clients, this is done on server side in the
mdt layer, by checking if clients have the OBD_CONNECT2_ENCRYPT
connection flag.
For clients without the key, this is done in llite when the searched
filename is not in encoded form.

Lustre-change: https://review.whamcloud.com/47349
Lustre-commit: 00898697f998c095e6dab3dd27e347cb1150ee8c

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9a3b7af3a856b7fc7222c61a308ad23168869d57
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47790
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-15625 sec: fix set_param -P for root squash 92/47792/2
Sebastien Buisson [Tue, 8 Mar 2022 11:07:47 +0000 (12:07 +0100)]
LU-15625 sec: fix set_param -P for root squash

lctl set_param -P needs to be fixed for root squash in order
to properly propagate to clients.
On server (MDT) side, the paramaters are mdt.*.*squash*.
On client side, the parameters are llite.*.*squash*.
So if the setting on MGS side is done as
set_param -P *.*.root_squash and
*.*.nosquash_nids, it works fine. But if it is done as
set_param -P mdt.*.root_squash, this setting is not applied to
client side.
The solution is to have the settings translated into
*.*.root_squash and *.*.nosquash_nids to make both sides happy.

Also, to ensure client vs. server consistency, prevent
set_param -P llite.*.root_squash.

Lustre-change: https://review.whamcloud.com/46739
Lustre-commit: 7b8449e8f9ed310376a191ec8dfa2f1cb6233377

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8984c9d9024d7a053e548ed15d1321b281d9940f
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47792
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-15896 gss: support OpenSSLv3 84/48184/2
Sebastien Buisson [Mon, 13 Jun 2022 12:41:11 +0000 (14:41 +0200)]
LU-15896 gss: support OpenSSLv3

Lustre GSS code makes use of some OpenSSL API that has been
deprecated in v3, namely all the functions in the DH_* family.
So replace them with their EVP_PKEY_* counterparts if Lustre is
built on a system with OpenSSLv3.

Lustre-change: https://review.whamcloud.com/47717
Lustre-commit: 615691a531a80b75c4dd054dbb86d0bdbf4cf808

Fixes: ee60c14360 ("LU-15896 gss: ignore OpenSSLv3 deprecated API")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I78a4ca18b25aca3c34fe84e41413a33caddc01b6
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/48184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-15888 build: Debian dkms-debs requires ed and libkeyutils 67/47967/2
Shaun Tancheff [Mon, 18 Jul 2022 16:58:52 +0000 (09:58 -0700)]
LU-15888 build: Debian dkms-debs requires ed and libkeyutils

dkms install/build needs dependencies on libmount-dev,
libkeyutils1, and libkeyutils-dev

Debian does not install the 'ed' package by default.
Without the 'ed' package the version is not correctly added
to the changelog and parsed to the package names.

Debian does not have linux-image or linux-headers psuedo
packages so require the arch specific ones, ex:
   linux-image | linux-image-amd64 | linux-image-arm64
and:
   linux-headers | linux-headers-amd64 | linux-headers-arm64
respectively.

o2ib fails to find Debian in-kernel Module.symvers and
should check $LINUX_OBJ/Module.symvers before failing.

Lustre-change: https://review.whamcloud.com/47455
Lustre-commit: 7dc6e1128a030c7e12eb23ea41935ed6fe77ce1f

HPE-bug-id: LUS-10984
Test-Parameters: trivial
Fixes: 85a6eebeca1 ("LU-15652 build: On Debian detect -common kernel headers")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I82e2689f3af4b9ce106ee3ab6b4109d2709c8872
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47967
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-15993 ofd: don't leak pages if nodemap fails 03/48003/2
Alex Zhuravlev [Wed, 20 Jul 2022 19:10:23 +0000 (12:10 -0700)]
LU-15993 ofd: don't leak pages if nodemap fails

ofd_commitrw() shouldn't exit w/o calling ofd_commitrw_write(),
otherwise the pages taken in ofd_preprw() are leaked.

same in mdt_obd_commitrw()

Lustre-change: https://review.whamcloud.com/47873
Lustre-commit: ee4b50278e4471b7c64e619e13699c560502b81e

Fixes: bbfdc7c167 ("LU-14739 quota: fix quota with root squash enabled")

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Icd60c7ab80c5a7b65603d7da0d2e83872dc6b97f
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48003
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-13813 tests: fix stack_trap in conf-sanity test 110/111 21/48021/3
Jian Yu [Sat, 23 Jul 2022 06:49:40 +0000 (23:49 -0700)]
LU-13813 tests: fix stack_trap in conf-sanity test 110/111

This patch fixes stack_trap in conf-sanity test 110 and 111
to restore test environment.

Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=el8.5 serverdistro=el8.5 testlist=conf-sanity

Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
fstype=zfs \
clientdistro=el8.5 serverdistro=el8.5 testlist=conf-sanity

Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
mdscount=2 mdtcount=4 \
clientdistro=el8.5 serverdistro=el8.5 testlist=conf-sanity

Change-Id: I540d96e8ad2c4990e7da18fe22256b44e9a19c72
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-16037 build: remove quotes from %{mkconf_options} 17/48117/3
Jian Yu [Thu, 4 Aug 2022 06:37:19 +0000 (23:37 -0700)]
LU-16037 build: remove quotes from %{mkconf_options}

This patch fixes lustre-dkms.spec.in to remove quotes
from %{mkconf_options} passed to dkms.mkconf, so as to
resolve the following build issue:

dkms.conf: Error! Directive 'DEST_MODULE_LOCATION'
does not begin with '/kernel', '/updates', or '/extra'
in record #0.

Lustre-change: https://review.whamcloud.com/48044
Lustre-commit: 33efefc496159f7d0caed0fa85d8f92603060ae2

Test-Parameters: trivial

Change-Id: I0b365d7a96cb632680bc2321e87b28a3bf076e47
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48117
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-16048 build: Update ZFS version to 2.1.5 46/48046/2
Jian Yu [Tue, 26 Jul 2022 23:37:25 +0000 (16:37 -0700)]
LU-16048 build: Update ZFS version to 2.1.5

Update ZFS version to 2.1.5. The changes are listed in:
https://github.com/openzfs/zfs/releases/tag/zfs-2.1.5

Change-Id: I9f25aafe889f87fb80677e59dbe4679932d8b920
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48046
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-15983 lnet: Define KFILND network type 87/47987/3
Chris Horn [Wed, 29 Jun 2022 00:24:32 +0000 (19:24 -0500)]
LU-15983 lnet: Define KFILND network type

Define the KFILND network type. This reserves the network type number
for future implementation and allows creation of kfi peers and
adding routes to kfi peers.

Lustre-change: https://review.whamcloud.com/47830
Lustre-commit: 5fea36c952373c9a235be7bf57eb2e516fcb36b2

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-11060
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I9111645f1290c8af4937d1b2689a068df81922a4
Reviewed-on: https://review.whamcloud.com/47987
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>