Whamcloud - gitweb
Shaun Tancheff [Wed, 8 Mar 2023 19:59:01 +0000 (11:59 -0800)]
LU-16113 build: Fix configure tests for lock_page_memcg
Linux commit v5.15-12273-gab2f9d2d3626
mm: unexport {,un}lock_page_memcg
Fails when lock_page_memcg exists but is not exported.
Adjust usage of [un]lock_page_memcg() to vvp_[un]lock_page_memcg() and
define the mapping accordingly to avoid the compile error.
Lustre-change: https://review.whamcloud.com/49144
Lustre-commit:
61e83a6f13064aa9ae0d0e8ce8244493bad1f1bd
Test-Parameters: trivial
HPE-bug-id: LUS-11189
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I18029d078a00a0b21a14721bcdf953939b4118a1
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49810
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Wed, 8 Mar 2023 19:55:06 +0000 (11:55 -0800)]
LU-16342 mdt: not copy pool_name to quotactl in reply
Don not copy pool_name in mdt reply to avoid out-of-bounds:
BUG: KASAN: slab-out-of-bounds in mdt_quotactl+0x13ff/0x1430 [mdt]
Lustre-change: https://review.whamcloud.com/49242
Lustre-commit:
7e5f927458544bd2681027bfc3df6136d059121f
HPE-bug-id: LUS-10579
Change-Id: I34c4cd8aaccd938c95005dca06644e02132def34
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://es-gerrit.dev.cray.com/160899
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49806
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Shaun Tancheff [Wed, 8 Mar 2023 19:51:08 +0000 (11:51 -0800)]
LU-16115 build: Linux 5.17 external module support
Linux commit v5.16-rc3-26-g129ab0d2d9f3
Added quotes around "$(CONFIG_CC_VERSION_TEXT)", however
.config stores CONFIG_CC_VERSION_TEXT with quotes thus
breaking the GNU make Makefile for external modules.
We need to supply a non-quoted value and override the
value in .config before it is used.
Lustre-change: https://review.whamcloud.com/48360
Lustre-commit:
945ab61176213e082dfd6b3a2a35aa4687e31129
Test-Parameters: trivial
HPE-bug-id: LUS-11190
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I997b6987ef37a8c5b9d8f0984e81d9402a2ea705
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49114
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 8 Mar 2023 19:46:11 +0000 (11:46 -0800)]
LU-13485 build: Parallel build configure cache fixes
This fixes the infrastructure for parallel builds when cached
enabled results are enabled, the critical fix being proper
usage of AC_CACHE_CHECK in the LB2_LINUX_TEST_RESULT macro.
Lustre-change: https://review.whamcloud.com/49149
Lustre-commit:
24c9f1f4c56d266656572e7bd97197d690d49369
Test-Parameters: trivial
Fixes:
b0209c2d4d ("LU-13485 build: Enable 2 stage configure tests")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I94d7ff291dfd4f2dc5e218acc811329b986f8fbf
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49805
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 8 Mar 2023 19:41:01 +0000 (11:41 -0800)]
LU-16302 llite: Use alloc_inode_sb() to allocate inodes
linux-commit: v5.17-49-g8b9f3ac5b01d
fs: introduce alloc_inode_sb() to allocate filesystems specific
inode
Filesystems are expected to use alloc_inode_sb to allocate inodes
for proper lru handling.
Lustre-change: https://review.whamcloud.com/49070
Lustre-commit:
a883fec55694491a26b664c2cff7fb904f72b7af
Test-Parameters: trivial
HPE-bug-id: LUS-11332
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ie6f091a01df33738ed2ef6f7fef9c1f9c1a51e03
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49122
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 27 Jan 2023 18:35:47 +0000 (10:35 -0800)]
LU-16121 llite: invalidate_folio and dirty_folio
linux commit v5.17-rc4-10-g128d1f8241d6
fs: Add invalidate_folio() aops method
A struct folio is often analogous to a struct page however
a struct folio can represent (contain) multiple pages.
linux commit v5.17-rc4-38-g6f31a5a261db
fs: Add aops->dirty_folio
__set_page_dirty_nobuffers() is replaced with filemap_dirty_folio()
Lustre-change: https://review.whamcloud.com/48366
Lustre-commit:
f1604e680ba2b919c0c52c76c75a7e0dc27ec3c5
Test-Parameters: trivial
HPE-bug-id: LUS-11197
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iefe67615b333e066c49c4b884dad5bea3b3ae226
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49120
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 27 Jan 2023 18:25:25 +0000 (10:25 -0800)]
LU-16117 build: Avoid excessive modpost warnings
To avoid modpost warnings about duplicate symbols do not add
the LINUX_OBJ kernel symbols to the KBUILD_EXTRA_SYMBOLS list
Lustre-change: https://review.whamcloud.com/48362
Lustre-commit:
445f85de2baddbaa8a84a709b98a9d0b8ed4231a
Test-Parameters: trivial
HPE-bug-id: LUS-11192
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I85fc90661efcb66e4aa39c9bd3393dbe4f7ba5eb
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49117
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 27 Jan 2023 18:20:26 +0000 (10:20 -0800)]
LU-16116 build: Configure tests for rhltable, bitmap_alloc...
rhel8.6 with kernel 5.18 breaks a couple of compile tests
struct rhltable test fails with:
... error: ‘hlt’ is used uninitialized in this function
[-Werror=uninitialized]
rdma_wr() test failes with:
... error: assignment discards ‘const’ qualifier from pointer
target type [-Werror=discarded-qualifiers]
wr = rdma_wr(NULL);
nla_strdup() test fails due to unused variable 'tmp'
Lustre-change: https://review.whamcloud.com/48361
Lustre-commit:
009faf132dc47dcfb604b95c93211da7d2c644e2
Test-Parameters: trivial
HPE-bug-id: LUS-11191
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ib2b1d223ac809cea157158fe35fd2535b04367df
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49116
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 27 Jan 2023 18:13:38 +0000 (10:13 -0800)]
LU-16118 build: Use pde_data() when available
Linux commit v5.16-11573-g6dfbbae14a7b
introduce pde_data() and
Linux commit v5.16-11574-g359745d78351
remove PDE_DATA()
Use PDE_DATA() when pde_data is not available.
Lustre-change: https://review.whamcloud.com/48363
Lustre-commit:
d54e8e95de88e13f193bd44eb43662e75b2091c1
Test-Parameters: trivial
HPE-bug-id: LUS-11193
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ida570462acd466a251adc81a14bc1fbf35d96b00
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49115
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 26 Jan 2023 17:37:41 +0000 (09:37 -0800)]
LU-16120 build: Add support for kobj_type default_groups
Linux commit v5.1-rc3-29-gaa30f47cf666
kobject: Add support for default attribute groups to kobj_type
Linux commit v5.18-rc1-2-gcdb4f26a63c3
kobject: kobj_type: remove default_attrs
Switch to using kobj_type default_groups when it is available.
Provide support for default_attrs for older kernels.
Lustre-change: https://review.whamcloud.com/48365
Lustre-commit:
62e9d055d9516ec6a2fe2f0962b23d478a45dad0
HPE-bug-id: LUS-11196
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I43b03c67c22307293a2abc444aa1a73889ca09ee
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49786
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 26 Jan 2023 17:34:03 +0000 (09:34 -0800)]
LU-16359 build: RHEL use Module.symvers during find-provides
find-provides fails to find module versions on newer
kernels.
The generated Module.symvers is always generated and
correct. Install it to the well known location BUILDROOT
use it to generate provides and ignore it for installation.
Create a new find-provides and find-provides.ksyms for
lustre based on the one provided by the redhat-rpm-config
package using Module.symvers to supply the symbol versions
instead of extracting symbol versions from the .ko files.
Lustre-change: https://review.whamcloud.com/49302
Lustre-commit:
bb951b90268bb013b55166e1c0bcae1c2a66b165
Test-Parameters: trivial
HPE-bug-id: LUS-11383
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I01c3b3692e6a2a6be86a6930eaead9df75147f90
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49785
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Thu, 26 Jan 2023 06:54:37 +0000 (22:54 -0800)]
LU-16291 build: make kobj_type constant
Kernel v5.16-rc2-28-gee6d3dd4ed48:
commit
ee6d3dd4ed48ab24b74bab3c3977b8218518247d
driver core: make kobj_type constant.
This patch makes struct kobj_type constant to fix
the following build failure against kernel 5.16:
lustre/obdclass/obd_config.c: In function 'class_modify_config':
lustre/obdclass/obd_config.c:1639:13: error: assignment discards
'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
1639 | typ = get_ktype(kobj);
| ^
Lustre-change: https://review.whamcloud.com/49043
Lustre-commit:
d1dbf26afd6676e02a2a00e635b9ad1fe14cf68e
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I19e0d1f4e3cf97f6871e038487cda9294ac1f67b
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49123
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 26 Jan 2023 06:28:33 +0000 (22:28 -0800)]
LU-16114 build: Update security_dentry_init_security args
Linux commit v5.15-rc1-20-g15bf32398ad4
security: Return xattr name from security_dentry_init_security()
Adjust security_dentry_init_security() calls accordingly
Lustre-change: https://review.whamcloud.com/48359
Lustre-commit:
88bccc4fa4dd7310560f588c730eefedf423c515
Test-Parameters: trivial
HPE-bug-id: LUS-11188
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I42d3307f7fe0d2412381363f60ac5b3df2d5891a
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49629
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Thu, 26 Jan 2023 06:24:24 +0000 (22:24 -0800)]
LU-16112 build: ki_complete removed unused argument
Linux commit v5.15-rc6-145-g6b19b766e8f0
fs: get rid of the res2 iocb->ki_complete argument
Prior to 4.1 Linux provided an aio_complete(iocb, res, res2)
which propagated res2 to io_event.res2. This functionality
migrated to iocb->ki_complete().
Provide a wrapper around iocb->ki_complete() to use
aio_complete() or iocb->ki_complete() as appropriate.
Lustre-change: https://review.whamcloud.com/48357
Lustre-commit:
c13eccf71ddee4bc8bc8cbee8f927bdba8768735
HPE-bug-id: LUS-11187
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I11d1ee61528d4d89e2a316fd71066824b202dac7
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49784
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 26 Jan 2023 05:23:43 +0000 (21:23 -0800)]
LU-16111 build: Fix include of stddef.h
In kernel builds include the linux/stddef.h
Lustre-change: https://review.whamcloud.com/48367
Lustre-commit:
95f7ef609467a9534b0b1639eba0ef5a2c02f21f
Test-Parameters: trivial
HPE-bug-id: LUS-11185
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0db81e01fadd01445515f96b3d04a2ec51f43044
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49105
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Lai Siyao [Thu, 26 Jan 2023 03:12:08 +0000 (19:12 -0800)]
LU-16335 build: remove _GNU_SOURCE dependency in lustre_user.h
The lustre_user.h header uses the non-standard strchrnul() function
in userspace. This will always leads to LC_IOC_REMOVE_ENTRY configure
check to fail, and in the end "lfs rm_entry" always returns -ENOTSUP.
Implement an alternative approach to avoid external dependencies on
the lustre_user.h header. Also, LC_IOC_REMOVE_ENTRY is itself
unnecessary, the code can check for LL_IOC_REMOVE_ENTRY directly.
Replace the NFS-specific -ENOTSUP error return code with -EOPNOTSUPP.
Fix the compile test_400[ab] checks to not use "-std=c99" to verify
that the uapi headers are usable without this dependency.
Lustre-change: https://review.whamcloud.com/49328
Lustre-commit:
efc5c8d4de60d394344506f7cfb188eaf04a4bac
Fixes:
b59835f8b6 ("LU-13903 utils: have liblustreapi support Linux client")
Fixes:
7a7309fa84 ("LU-13274 uapi: make lustre UAPI headers C99 compliant")
Fixes:
6331eadbd6 ("LU-15420 uapi: avoid gcc-11 -Werror=stringop-overread")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If42743a2148c317b8a9b701ceb5d08bac5149f5f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49776
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
James Simmons [Wed, 25 Jan 2023 02:20:14 +0000 (18:20 -0800)]
LU-9680 utils: fix Netlink / YAML library handling
Testing the implementation of Netlink with lustre with early
code revealed some userland lnetconfig library bugs. Several bugs
fixed are:
1) First issue was that the YAML parser and emitter can share an
netlink socket. This means the netlink callbacks will expect
that same void argument passed in. We were in the case of error
handling expecting an struct yaml_netlink_output but all other
callbacks were using struct yaml_netlink_input. This mismatch
can cause the application to segfault. So move all netlink
callback handling to use just the yaml_parser. The yaml
emittter now is used to just send Netlink packets to the
kernel.
Also fix the Netlink ext_ack error message handling.
2) In my board testing I found various bugs related to the
paring of the YAML to create Netlink packet to send to the
kernel. This patch resolves all the known issues. Most
related to the complex layering of sequences, mappings and
flows in a YAML block.
3) Fix up nla_strlcpy autoconf test which always fails with
Oleg's special setup.
4) Add a Netlink protocol version YAML function.
Lustre-change: https://review.whamcloud.com/47802
Lustre-commit:
70149f4ea89aa9f5c002b63f56d0c98afcafbad9
Test-Parameters: trivial
Change-Id: I7e7c755ceaa969dffff8c6f771c2ac048dc55720
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Joe Atzinger <joseph.atzinger@microsoft.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49091
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Boyko [Tue, 24 Jan 2023 19:29:14 +0000 (11:29 -0800)]
LU-15393 tests: check QoS hang with OST failover
Patch adds recovery small test 152, to reproduce situation
where MDT object allocation sleeps on OST failover at
lod_ost_alloc_rr under lq_rw_sem read. And all other creation threads
hang at lod_ost_alloc_qos at down_write(lq_rw_sem).
Lustre-change: https://review.whamcloud.com/47715
Lustre-commit:
52057d85eaef8c7b5262f0718629fabff919ff1d
HPE-bug-id: LUS-10388
Test-Parameters: trivial testlist=recovery-small env=ONLY=152
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I7b9c5a5c9870a559e673a5fd253dcaea40d9fe63
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49097
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Boyko [Tue, 24 Jan 2023 19:23:43 +0000 (11:23 -0800)]
LU-15393 lod: skip qos for qos_threshold_rr=100
Current implementation of qos allocation is called for
every statfs update. It takes lq_rw_sem for write and
recalculate penalties, even whith setting qos_threshold_rr=100.
Which means always use rr allocation. Let's skip unnecessary
locking and calculation for 100% round robin allocation.
Lustre-change: https://review.whamcloud.com/46388
Lustre-commit:
2f23140d5c1396fd0b247bd7f9c249f6e24096b7
HPE-bug-id: LUS-10388
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I2fcc272d00a988ca4ba0f745b1d5809d65b28654
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49096
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Boyko [Tue, 24 Jan 2023 19:13:32 +0000 (11:13 -0800)]
LU-15393 lod: use killable semaphore for creation path
lod_ost_alloc_qos() function sleeps during ost failover, but object
allocation could use different OSTs. The patch changes
down_write call to down_write_killable and adds timer for a
wakeup.
The main idea of this fix is next, when OST is lost during
lod_ost_alloc_rr() and MDT does not have precreated objects for it
lod_ost_alloc_rr()->..->lod_qos_declare_object_on() would sleep while
holding a lq_rw_sem for read. Any creation thread would stuck at
lod_ost_alloc_qos() waiting lq_rw_sem for write, after statfs update.
Whith a fix sleep is limited and allocation would going through
lod_ost_alloc_rr(). For read lq_rw_sem is shared and stripe allocation
would skip OST without objects.
lod_ost_alloc_rr() refills OST pool with a lq_rw_sem for write, when
lq_rr.lqr_flags has LQ_DIRTY. This should happen only when OST is
added/removed. No need to set LQ_DIRTY for lq_rr when statfs get
error, this flag does not affect any change for pool list at
lod_qos_calc_rr().
Change behaviour for lod_check_and_reserve_ost(), it would sleep
during object allocation for speed 2 only.
Lustre-change: https://review.whamcloud.com/45921
Lustre-commit:
f46782b4c7dcaacd0046ebad3e3d84c2bb0367d4
HPE-bug-id: LUS-10388
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I4768c4cf7d2f9f02f0a9e0dfb6d15e02932cb5fe
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49095
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Thu, 2 Feb 2023 01:39:03 +0000 (09:39 +0800)]
LU-16523 lprocfs: adjust the format of rename_stats
Adjust the format of rename_stats to a more human-friendly YAML.
Lustre-change: https://review.whamcloud.com/49869
Lustre-commit:
73b5d7db7e8d3ede42524fc447fb30fa05ea7a3f
Fixes:
bedb797c5d ("LU-16110 lprocfs: make job_stats and rename_stats valid YAML")
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I20e6d07c974e907bb2e30412dd1899f845de2021
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49870
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Mon, 12 Sep 2022 21:09:38 +0000 (15:09 -0600)]
LU-16149 lnet: Discovery queue and deletion race
lnet_peer_deletion() can race with another thread calling
lnet_peer_queue_for_discovery.
Discovery thread:
- Calls lnet_peer_deletion():
- LNET_PEER_DISCOVERING bit is cleared from lnet_peer::lp_state
- releases lnet_peer::lp_lock
Another thread:
- Acquires lnet_net_lock/EX
- Calls lnet_peer_queue_for_discovery()
- Takes lnet_peer::lp_lock
- Sets LNET_PEER_DISCOVERING bit
- Releases lnet_peer::lp_lock
- Sees lnet_peer::lp_dc_list is not empty, so it does not add peer
to dc request queue
- lnet_peer_queue_for_discovery() returns, lnet_net_lock/EX releases
Discovery thread:
- Acquires lnet_net_lock/EX
- Deletes peer from ln_dc_working list
- performs the peer deletion
At this point, the peer is not on any discovery list, and it has
LNET_PEER_DISCOVERING bit set. This peer is now stranded, and any
messages on the peer's lnet_peer::lp_dc_pendq are likewise stranded.
To solve this, we modify lnet_peer_deletion() so that it waits to
clear the LNET_PEER_DISCOVERING bit until it has completed deleting
the peer and re-acquired the lnet_peer::lp_lock. This ensures we
cannot race with any other thread that may add the
LNET_PEER_DISCOVERING bit back to the peer. We also avoid deleting
the peer from the ln_dc_working list in lnet_peer_deletion(). This is
already done by lnet_peer_discovery_complete().
There is another window where the LNET_PEER_DISCOVERING bit can be
added when the discovery thread drops the lp_lock just before
acquiring the net_lock/EX and calling lnet_peer_discovery_complete().
Have lnet_peer_discovery_complete() clear LNET_PEER_DISCOVERING to
deal with this (it already does this for the case where discovery hit
an error). Also move the deletion of lp_dc_list to after we clear the
DISCOVERING bit. This is to mirror the behavior of
lnet_peer_queue_for_discovery() which sets the DISCOVERING bit and
then manipulates the lp_dc_list.
Also tweak the logic in lnet_peer_deletion() to call
lnet_peer_del_locked() in order to avoid extra calls to
lnet_net_lock()/lnet_net_unlock().
Lustre-change: https://review.whamcloud.com/48532/
Lustre-commit:
a966b624ac76e34e8ed28c6980c3f58cb441eeb0
Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-11237
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ifcfef1d49f216af4ddfcdaf928024e8ee3952555
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Wed, 11 Jan 2023 20:30:07 +0000 (15:30 -0500)]
New release 2.15.2
Change-Id: I1e955e3b98123cc7080994aa4c28b274921ec86e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Tue, 3 Jan 2023 22:05:00 +0000 (17:05 -0500)]
New RC 2.15.2-RC2
Change-Id: I956f27aaf856f29cf84aea9c940c7d56ff31387f
Jian Yu [Thu, 29 Dec 2022 08:24:13 +0000 (00:24 -0800)]
LU-16433 llite: check vvp_account_page_dirtied
This patch removes duplicated codes from vvp_set_pagevec_dirty()
and check vvp_account_page_dirtied to determine if falling back
to call __set_page_dirty_nobuffers().
HAVE_ACCOUNT_PAGE_DIRTIED_EXPORT also needs to be checked because
vvp_account_page_dirtied is not defined if account_page_dirtied
is exported.
Lustre-change: https://review.whamcloud.com/49512
Lustre-commit:
61c4c2b5e5d7d919149921d913322586ba5ebcab
Test-Parameters: trivial clientdistro=el8.6 testlist=sanity
Test-Parameters: trivial clientdistro=el8.7 testlist=sanity
Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity
Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity
Change-Id: I272033d7494a157145224b1b8ce999a80958aa6c
Fixes:
4bf090b811 ("LU-15959 kernel: new kernel [SLES15 SP4 5.14.21-150400.24.18.1]")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49520
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Oleg Drokin [Sun, 11 Dec 2022 14:44:04 +0000 (09:44 -0500)]
New RC 2.15.2 RC1
Change-Id: Ib47fbd7b9f0cc67f82cb1a247ac781ba13f649b9
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Tue, 11 Oct 2022 09:09:08 +0000 (03:09 -0600)]
LU-16231 misc: fix stats snapshot_time to use wallclock
The timestamps reported during stats collection inadvertently changed
from being POSIX epoch timestamps to elapsed-from-boot timestamps.
While some collection tools ignore these timestamps, or only use the
delta between successive reads, having uniform timestaps in stats
files simplifies stats correlation between different servers.
Revert the snapshot_time back to showing wallclock time.
Some "init" times were not initialized when stats were allocated or
cleared, do this for all stats shown by lprocfs_stats_header().
Rename struct osc_device fields from od_ to osc_ to avoid confusion
with struct osd_device. Having two od_stats was especially confusing.
Add a test case to verify snapshot_time, start_time, elapsed_time.
Lustre-change: https://review.whamcloud.com/48821
Lustre-commit:
e42efe35eec7b9725f7f4fff86aaee04093366b0
Test-Parameters: testlist=sanity env=ONLY=127a,ONLY_REPEAT=100
Fixes:
ea2cd3af7b ("LU-11407 obdclass: add start time to stats files")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I421c3b0301c2566b48c2fc6fe7bb8b54ec48ca5d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49220
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lei Feng [Fri, 2 Sep 2022 07:05:22 +0000 (15:05 +0800)]
LU-16110 lprocfs: make job_stats and rename_stats valid YAML
Adjust the format of job_stats and rename_stats to make
them valid YAML.
Lustre-change: https://review.whamcloud.com/48417
Lustre-commit:
e96cb6ff1fea7a2bc62a6c0786fb0e07cbfda81a
Test-Parameters: testlist=sanity env=ONLY=907
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Idca36621241e97ff87f8ab0448f3c5604057a460
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49215
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Emoly Liu [Mon, 25 Apr 2022 02:56:55 +0000 (10:56 +0800)]
LU-15706 llog: deal with "SKIP" pool llog records correctly
If the requested "start" llog records by one ioctl are just the
useful ones "protected" by one marker, they will don't know their
cfg_flags(e.g. "SKIP" or not), and then will be mis-labeled in
class_config_yaml_output().
To fix this issue, this patch does the following changes:
- In kernel space, remember the marker cfg_flags for 10 records
earlier than the requested "start" in function llog_print_cb(),
so that the "start" can be output with its correct cfg_flags.
- In user space, since the pool_new/add record will be marked as
"SKIP" if its corresponding pool_destroy/remove record is logged
later in mgs_pool_cmd(), these "SKIP" records won't be printed
by function jt_llog_print_iter(), so lpd_ost_num doesn't need to
be decreased in callback function llog_search_pool_cb() as well,
otherwise, it will cause "lctl pool_destroy" error.
Lustre-change: https://review.whamcloud.com/46951
Lustre-commit:
474f2670d63b66a77ee3acb72b18bc7b5afbec84
Test-Parameters: standalonemgs=true testlist=ost-pools
Test-Parameters: standalonemgs=true testlist=conf-sanity env=ONLY=123,HONOR_EXCEPT=y
Fixes:
2a5b50d20717 ("LU-15142 lctl: fixes for set_param -P and llog_print")
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ie45127ac8b80a75eaeb7158559c690da52eef103
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49230
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Minh Diep [Mon, 3 Oct 2022 18:22:47 +0000 (11:22 -0700)]
LU-16183 test: sanity-hsm/70 should detect python
Check for python2 and python3 explicitly, since the
generic python command does not exist in newer distros.
Lustre-change: https://review.whamcloud.com/48737
Lustre-commit:
c24a38b583cad8e3a92c1d74e8e56e80f07c67b5
Test-Parameters: env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=sles15sp3 testlist=sanity-hsm
Test-Parameters: env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=el7.9 testlist=sanity-hsm
Change-Id: I35bbe15fd298341870ad4f1ab5976e82ccc84667
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Charlie Olmstead <charlie@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49247
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Fri, 25 Mar 2022 02:34:56 +0000 (10:34 +0800)]
LU-15460 test: wait quota pool to be prepared
When one OST pool was created, the corresponding quota pool
could need more time to prepare after it was created, then
fail to check the info of the quota pool
Lustre-change: https://review.whamcloud.com/46853
Lustre-commit:
83dd308db59a9b95c1db5efa12ea5af22e9c8346
Test-Parameters: trivial
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: Ibea33403639087f27e438d71c0e87fea5367bc3e
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49250
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 18 Nov 2022 19:56:49 +0000 (11:56 -0800)]
LU-16295 kernel: kernel update RHEL 7.9 [3.10.0-1160.80.1.el7]
Update RHEL 7.9 kernel to 3.10.0-1160.80.1.el7.
Lustre-change: https://review.whamcloud.com/49045
Lustre-commit:
32dcf69007b74917ec0b6434ad30d00a8fafcb2e
Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I50a0ee572d24ddc73f8af6dc32ef701c260e45b7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49191
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Tue, 29 Nov 2022 17:01:45 +0000 (09:01 -0800)]
LU-16317 build: dkms build requires flex, bison and libmount-devel
This patch fixes lustre.spec.in and lustre-dkms.spec.in to add
requires for flex, bison, libmount and libmount-devel. The last
two have already been added into lustre.spec.in.
Lustre-change: https://review.whamcloud.com/49183
Lustre-commit:
c74c630ff7596317d1b500fd385fca271b31708c
Test-Parameters: trivial
Fixes:
121a79651f ("LU-15967 build: configure script does not check for required build tools")
Fixes:
f21b944127 ("LU-15940 build: add a required dependency for libmount")
Change-Id: I9923fc7eb09f974e8c38c3664138486a424e16d7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Fri, 7 Oct 2022 12:09:10 +0000 (23:09 +1100)]
LU-15305 obdclass: fix race in class_del_profile
Move profile lookup and remove from lustre_profile_list
into the same critical section, otherwise we could race with
class_del_profiles or another class_del_profile.
Do not create duplicate mount opts in the client config,
otherwise we will add duplicate lustre_profile to
lustre_profile_list for a single mount.
Lustre-change: https://review.whamcloud.com/48802
Lustre-commit:
83d3f42118579d7fb7c3002533c047badcf41e0d
Change-Id: I648aa206716213b064d045f546516b219337e0ed
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49178
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 21 Oct 2022 04:54:49 +0000 (23:54 -0500)]
LU-16258 llite: Explicitly support .splice_write
Linux commit v5.9-rc1-6-g36e2c7421f02
fs: don't allow splice read/write without explicit ops
Lustre supports splice_write and previously provide handlers
for splice_read.
Explicitly use iter_file_splice_write, if it exists.
Lustre-change: https://review.whamcloud.com/48928
Lustre-commit:
c619b6d6a54235cc0e34a65cf5916a632f4011c3
HPE-bug-id: LUS-11259
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I858688fc9b4dd370b6018c3b134f01e580477b25
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49058
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Thu, 10 Nov 2022 18:38:57 +0000 (10:38 -0800)]
LU-16294 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.28.1]
Update SLES15 SP4 kernel to 5.14.21-150400.24.28.1 for Lustre client.
Lustre-change: https://review.whamcloud.com/49046
Lustre-commit:
165bf5acdfebe001af8e15f64cd1842972a434b2
Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I651894274a09b6240f321e787736d298c5dc41ce
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49103
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Tue, 22 Nov 2022 17:52:23 +0000 (09:52 -0800)]
LU-16269 kernel: kernel update RHEL8.6 [4.18.0-372.32.1.el8_6]
Update RHEL8.6 kernel to 4.18.0-372.32.1.el8_6.
Lustre-change: https://review.whamcloud.com/48969
Lustre-commit:
705edb250d62dca84f97f7e97eb468d554611c20
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity
Change-Id: I5576180ddf10ed2b0a5e2ef85b58fef993de65a4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49032
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 22 Nov 2022 18:00:41 +0000 (10:00 -0800)]
LU-16293 kernel: kernel update RHEL9.0 [5.14.0-70.30.1.el9_0]
Update RHEL9.0 kernel to 5.14.0-70.30.1.el9_0 for Lustre client.
Lustre-change: https://review.whamcloud.com/49044
Lustre-commit:
212826c28e369a10cc3afd1306d49eca71b1b9d2
Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Ide942f88242c80af1e103b226b65cfbce94bfb57
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49054
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 15 Nov 2022 00:39:43 +0000 (16:39 -0800)]
LU-16304 kernel: kernel update RHEL8.7 [4.18.0-425.3.1.el8]
Update RHEL8.7 kernel to 4.18.0-425.3.1.el8.
Lustre-change: https://review.whamcloud.com/49080
Lustre-commit: TBD (from
8900b469b4d521361d31ca96fed23c49a141fe93)
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I13e6d83ada1ec0c4da92f307bf56db5281c41892
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49156
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lei Feng [Thu, 30 Jun 2022 02:46:31 +0000 (10:46 +0800)]
LU-15986 ptlrpc: protect rq_repmsg in ptlrpc_req_drop_rs()
There is a race condition that: on server side, one thread sent
reply message and is deleting the reply message, another is
searching for existing request and print some debug information
in _debug_req() if there is a duplicated request. They both operate on
req->rq_repmsg but it is not protected in ptlrpc_req_drop_rs().
So we protected it with req->rq_early_free_lock.
Lustre-change: https://review.whamcloud.com/47839
Lustre-commit:
aaef545cff2dd958418ec9fb364d4bbe1408edb9
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Ied55427ee15c3ef84bdd2d579844eba398dbf010
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49157
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Mon, 31 Oct 2022 19:13:52 +0000 (12:13 -0700)]
LU-16222 kernel: RHEL 8.7 client and server support
This patch makes changes to support RHEL 8.7 release
with kernel 4.18.0-423.el8 for Lustre client and server.
Lustre-change: https://review.whamcloud.com/48879
Lustre-commit:
293844d132b79a1d256ed4200d5dbd8bb790bfb4
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Change-Id: Ie97ff67c9a5fbd46bc145ab559665dcbc630b4a0
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Co-Authored-By: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48993
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Alex Zhuravlev [Tue, 19 Apr 2022 12:01:14 +0000 (15:01 +0300)]
LU-15757 llite: check s_root ll_md_blocking_ast()
ll_md_blocking_ast() can be called in the context of import
invalidation which in turn caused by umount. this way
ll_md_blocking_ast() and umount can race and ll_md_blocking_ast()
can found sb->s_root NULL which should be checked before
calling into is_root_inode().
Lustre-change: https://review.whamcloud.com/47086/
Lustre-commit:
0095c0d05ca80a2494710e3b4afb1d1e4b5cdcfe
Change-Id: I61c1d29a7de3084ad1dfd0c216cee628418b7038
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48406
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Tue, 18 Oct 2022 15:19:01 +0000 (17:19 +0200)]
LU-16249 sec: krb5_decrypt_bulk calls decryption primitive
krb5_decrypt_bulk() was mistakenly calling an encryption primitive
instead of a decryption primitive for the confounder.
Lustre-change: https://review.whamcloud.com/48907
Lustre-commit: TBD (
851f3915659941db00a0cda58867e68139e5e0d1)
Test-Parameters: trivial
Fixes:
0a65279121 ("LU-13344 gss: Update crypto to use sync_skcipher")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9251172644ed6baa3bb06a59dbe7c1bab401d817
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48908
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Jian Yu [Thu, 13 Oct 2022 17:47:23 +0000 (10:47 -0700)]
LU-16173 kernel: kernel update SLES15 SP3 [5.3.18-150300.59.93.1]
Update SLES15 SP3 kernel to 5.3.18-150300.59.93.1 for Lustre client.
Test-Parameters: trivial clientdistro=sles15sp3 \
testlist=sanity
Lustre-change: https://review.whamcloud.com/48601
Lustre-commit:
c3467db7e7d0652c09bdcef26e2b708ab51cba9e
Change-Id: I1e0afe6974567d13680dbb0d463fbbd873ef2e5f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48859
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Mon, 19 Sep 2022 16:53:54 +0000 (09:53 -0700)]
LU-16050 build: replace ofed_info with dpkg/rpm
After installing MLNX_OFED by running mlnxofedinstall command,
mlnx-ofed-kernel-modules package is not listed by ofed_info,
which causes Lustre configure fail as follows:
checking whether to use Compat RDMA... /usr/bin/ofed_info
dpkg-query: error: --listfiles needs at least one package name argument
This patch fixes the above issue by replacing ofed_info with
"dpkg -l" and "rpm -qa" commands to find OFED package.
Lustre-change: https://review.whamcloud.com/48047
Lustre-commit:
3a7930e63c15b0fbe51ac73db81a1186939115bb
Test-Parameters: trivial
Fixes:
ec03c9628cae ("LU-15417 build: find the new path for MOFED 5.5")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Ia3c2d6bf10e147ca2761221741eff6f93008556c
Reviewed-by: Gaurang Tapase <gtapase@ddn.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48592
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Minh Diep [Tue, 20 Sep 2022 18:32:02 +0000 (11:32 -0700)]
LU-15795 lbuild: enable KABI
Enable build kabi and clean up kmodtool patch
Lustre-change: https://review.whamcloud.com/47507
Lustre-commit:
0a639a32cf32423e826420e24726912d5a39f13e
Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.5 serverdistro=el8.5
Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.6 serverdistro=el8.6
Change-Id: I16d54af0004c4ddc1cc5e6acca81e4aa89a1a1c1
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48610
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Thu, 13 Oct 2022 17:37:54 +0000 (10:37 -0700)]
LU-16233 build: Add always target for SUSE15 SP3 LTSS
SUSE 15 SP3 LTSS kernel version 5.3.18-150300.59.93
(and later) breaks lustre build tests which expect
conftest.i to be generated.
Lustre-change: https://review.whamcloud.com/48833
Lustre-commit:
3493db6a675d6bcbda3b52b41e627cdc983d6c0c
HPE-bug-id: LUS-11286
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If23e9b31b537878a43075ffff62a99906f47fd9a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48858
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Tue, 20 Sep 2022 18:54:08 +0000 (11:54 -0700)]
LU-16082 ldiskfs: old-style EA inode fix for el8.5/el8.6
Add the rhel8/ext4-old_ea_inodes_handling_fix.patch to the ldiskfs
series for el8.5 and el8.6 kernels.
Lustre-change: https://review.whamcloud.com/48496
Lustre-commit:
ba9845274c8ea5c55f57b7fa0e839f18d76031ea
Test-Parameters: trivial testlist=sanity serverdistro=el8.6
Fixes:
76c3fa96dc30 ("LU-16082 ldiskfs: old-style EA inode handling fix")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifb66a0b7d78e5153d7897bee45fbf1d0e58fbc5c
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48611
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Wed, 28 Sep 2022 06:56:20 +0000 (23:56 -0700)]
LU-16174 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.21.2]
Update SLES15 SP4 kernel to 5.14.21-150400.24.21.2 for Lustre client.
Lustre-change: https://review.whamcloud.com/48604
Lustre-commit:
5d31fdaf681a8d9c39d197531f010dcd97bf604c
Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity
Change-Id: Ia68e1c960c79f40d0f725b0f440cd562b820a19f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48688
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Wed, 28 Sep 2022 06:40:07 +0000 (23:40 -0700)]
LU-16177 kernel: kernel update RHEL9.0 [5.14.0-70.26.1.el9_0]
Update RHEL9.0 kernel to 5.14.0-70.26.1.el9_0 for Lustre client.
Lustre-change: https://review.whamcloud.com/48676
Lustre-commit:
8a2bd9661d07043e61366c77f21c55612c431cbc
Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity
Change-Id: I9da2ccdf419d6490fdba80199eda69f4f19361be
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48686
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Tue, 20 Sep 2022 18:29:43 +0000 (11:29 -0700)]
LU-16138 kernel: preserve RHEL8.x server kABI for block integrity
Currently there are two kernel patches supporting SCSI T10-PI feature
left in the RHEL8.x series:
- block-integrity-allow-optional-integrity-functions-rhel8.patch
- block-pass-bio-into-integrity_processing_fn-rhel8.patch
The changes in the patches modified "struct bio_integrity_payload"
and "struct blk_integrity_iter", which caused kABI breakage.
This patch fixes the patches to preserve kABI by using
RH-supplied compatibility macros.
Lustre-change: https://review.whamcloud.com/48608
Lustre-commit:
ec555c17c0254662c5886188dd2f7c0447fd4aef
Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.5 serverdistro=el8.5
Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.6 serverdistro=el8.6
Change-Id: If547e1cd4ae4ff1affd315bbfefaeeff4f1dea81
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48609
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alexander Boyko [Wed, 14 Sep 2022 20:02:59 +0000 (13:02 -0700)]
LU-15724 tests: MDT failover hang reproducer
The patch adds recovery-small 144a test to reproduce
MDT failover hang when precreate threads are blocked on objects.
LustreError: 0-0: Forced cleanup waiting for mdt-kjcf05-MDT0001_UUID
namespace with 46 resources in use, (rc=-110)
Lustre-change: https://review.whamcloud.com/47006
Lustre-commit:
aa6250b7412e7baf6760fe4010a81f4f22187127
Test-Parameters: trivial testlist=recovery-small env=ONLY=144a
HPE-bug-id: LUS-10750
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I2743a1b5c8911d6982b527f7e7b7bbbaf310cd04
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/48549
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Boyko [Wed, 14 Sep 2022 19:59:41 +0000 (12:59 -0700)]
LU-15724 osp: wakeup all precreate threads
Number of threads could sleep at osp_precreate_reserve() and
wait objects from OST. When MDT stops Lustre should wakeup
all threads. When opd_pre_recovering is set any wakeup of
opd_pre_user_waitq is useless. Failover of MDT does not produce
disconnect event, only inactive, so osp_precreate_cleanup_orphans()
can not be awakened.
LustreError: 0-0: Forced cleanup waiting for mdt-kjcf05-MDT0001_UUID
namespace with 46 resources in use, (rc=-110)
schedule_timeout at
ffffffff8e551cd3
osp_precreate_reserve at
ffffffffc17d2d83 [osp]
osp_declare_create at
ffffffffc17c7eb9 [osp]
lod_sub_declare_create at
ffffffffc156415b [lod]
lod_qos_declare_object_on at
ffffffffc155bf42 [lod]
lod_ost_alloc_rr.constprop.23 at
ffffffffc155db2f [lod]
lod_qos_prep_create at
ffffffffc15630a6 [lod]
lod_declare_instantiate_components at
ffffffffc154b237 [lod]
Lustre-change: https://review.whamcloud.com/47005
Lustre-commit:
e55fc043679cdfadfff6874ef78e2e0128ec37ac
HPE-bug-id: LUS-10750
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: If0164cfbecb1e358d9857421cb234559dc8cecbc
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/48548
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Etienne AUJAMES [Wed, 14 Sep 2022 20:21:23 +0000 (13:21 -0700)]
LU-16000 utils: align updatelog parameters in llog_reader
Parameters in update log records are aligned on 64bits. llog_reader
do not aligned these parameters: if a parameters size is not mutiple
of 8, the next parameter size will be read incorrectly.
Lustre-change: https://review.whamcloud.com/47913
Lustre-commit:
6d74b759634355e7f6647ccaefef519a1ff208e2
Test-Parameters: trivial
Fixes: 9962d6f ("LU-14617 utils: llog_reader updatelog support")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: I6871614ab4ea79d59c3c3b4644b377de395bad56
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48552
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Serguei Smirnov [Mon, 12 Sep 2022 20:50:07 +0000 (13:50 -0700)]
LU-16106 lnet: allow direct messages regardless of peer NI status
If check_routers_before_use is enabled, the router needs to
be pinged before it is used, which is not possible because
its NIs are assumed to be down at start-up. Don't prevent
discovery of the router in this case.
This change allows non-routed traffic to peer NIs with "down"
status.
Lustre-commit:
4c6d1491d1df078a4477b3fcfeddadf3d1fd7239
Lustre-change: https://review.whamcloud.com/c/48355
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I36fa60e37ef4f47c82c69855c9b0b80bad8a36f4
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/48529
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 3 Jun 2022 09:16:35 +0000 (11:16 +0200)]
LU-15911 enc: null encrypted names is embedded llcrypt only
enable_filename_encryption tunable only makes sense when Lustre client
is built against embedded llcrypt. When built against in-kernel
fscrypt, this tunable is silently ignored, as fscrypt always carries
out file name encryption.
So have the enable_filename_encryption tunable only when Lustre client
is built against embedded llcrypt. Also fix sanity-sec test_54 so that
it works for in-kernel fscrypt.
Lustre-change: https://review.whamcloud.com/47520
Lustre-commit:
4887aff7f10423674939682a2fd56f0490224e6e
Fixes:
e68d496ada ("LU-15858 sec: reinstate null encryption for file names")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ibe52feb670a00c9f421907ecd438bcccc62856f0
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/48521
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Tue, 13 Sep 2022 18:30:05 +0000 (11:30 -0700)]
LU-11407 tgt: cleanup job_stats output printing
Escape non-printable and other special characters in the JobID
name, which may be passed from the client environment, to avoid
breaking YAML format parsing. We can't use the kernel "%*pE"
escape format, because that doesn't have any option to escape
printable characters like quotes or regular spaces.
Lustre-change: https://review.whamcloud.com/37764
Lustre-commit:
338381574b643da064e90e44d75be85d1be3a93c
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If6a0bc4276fae03305f94e8c85d8f109913ebbe5
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/48539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 19 Sep 2022 16:42:09 +0000 (09:42 -0700)]
LU-16059 build: Installation of dkms server builds
The linux-zfs-dkms package is passing the wrong paths
for zfs [and spl] causing the dkms build to fail.
ZFS_VERSION is not parsed correctly from 'dkms status'.
The splver and zfsver check can match against the wrong
package(s).
lustre-zfs-dkms provides: kmod-lustre-osd-zfs, and
lustre-osd-zfs-mount
lustre-ldiskfs-dkms provides: kmod-lustre-osd-ldiskfs and
lustre-osd-ldiskfs-mount
In the case of multiple zfs versions installed, build lustre
osd against the highest version number.
Lustre-change: https://review.whamcloud.com/48083
Lustre-commit:
c3dc67b2c5bf1974d792b3701d932bd04c756bd8
HPE-bug-id: LUS-11113
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ic154ca045427bf26cb7e6a44b8c467675e987aad
Reviewed-on: https://review.whamcloud.com/48212
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 13 Sep 2022 05:36:56 +0000 (22:36 -0700)]
LU-15793 kernel: kernel update RHEL8.5 [4.18.0-348.23.1.el8_5]
Update RHEL8.5 kernel to 4.18.0-348.23.1.el8_5.
Lustre-change: https://review.whamcloud.com/47175
Lustre-commit:
597665de6ee4e77b259a82f126d95cb2d60bb260
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.5 serverdistro=el8.5 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.5 serverdistro=el8.5 testlist=sanity
Change-Id: I9724e304c2d32bb0ac2dcf4138438c79d4abca0f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47392
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 7 Sep 2022 04:27:04 +0000 (21:27 -0700)]
LU-16131 build: Do not depend on libmount during --enable-dist
Defer the libmount requirement when using --enable-dist to
generate the lustre-src.rpm.
This allows mock and/or yum build-deps to resolve resolve
dependencies and pickup the libmount requirement without changing
the existing minimal build.
Lustre-change: https://review.whamcloud.com/48407
Lustre-commit:
819c8b169325045ae8bac9c4f38a58c75e22d099
Test-Parameters: trivial
HPE-bug-id: LUS-11091
Fixes:
f21b944127 ("LU-15940 build: add a required dependency for libmount")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I20a7a097f9b651b6ea5519f79efda6c96b6f2199
Reviewed-on: https://review.whamcloud.com/48446
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Wed, 7 Sep 2022 17:38:37 +0000 (10:38 -0700)]
LU-15811 llite: Refactor DIO/AIO free code
Refactor the DIO/AIO free code and add some asserts.
This removes a potential use-after-free in the freeing
code.
Lustre-change: https://review.whamcloud.com/48115
Lustre-commit:
f1c8ac1156ebea2b8e94a75f056c403973bb9010
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I335b18fc7a28fc426a25675e2449d3d192cba596
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/48456
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Wed, 7 Sep 2022 17:32:32 +0000 (10:32 -0700)]
LU-15811 llite: Unify range unlock
Correct parallel_dio condition and unify range unlock code
block.
Lustre-change: https://review.whamcloud.com/48000
Lustre-commit:
36c34af60767bd5da5e9a66df0297e1dc7fa1fb1
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ib66e8def571054df5117c279e238894bc3b58bce
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/48455
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Wed, 7 Sep 2022 17:27:07 +0000 (10:27 -0700)]
LU-15811 llite: Rework upper/lower DIO/AIO
One of the patches for LU-13799,
"Implement lower/upper aio"
(https://review.whamcloud.com/44209/) created a
complicated setup where the cl_dio_aio struct was used
both for the top level DIO or AIO and for the lower level
sub I/Os (corresponding to stripes).
This is quite complicated and hard to follow, so this
rewrites these two uses to be separate structs. This
incidentally fixes at least one possible memory leak, but
is mostly a cleanup.
Lustre-change: https://review.whamcloud.com/47187
Lustre-commit:
51c18539338f1a23f3fa725470b5d36941b15c46
Fixes:
46ff761371 "LU-13799 Implement lower/upper aio"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ide4a2b84f48624ee97dfb57fe80d201fbb7fe8d0
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/48454
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 23 Aug 2022 23:20:44 +0000 (16:20 -0700)]
LU-16075 kernel: kernel update RHEL8.6 [4.18.0-372.19.1.el8_6]
Update RHEL8.6 kernel to 4.18.0-372.19.1.el8_6.
Lustre-change: https://review.whamcloud.com/48116
Lustre-commit:
872308001bf43ae3f190ceea11850f2f4eb8b7b3
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity
Change-Id: I8e0fbdab54d36512c4c4cbdbc97c580994ebcbd3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48315
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 2 Sep 2022 18:37:33 +0000 (11:37 -0700)]
LU-16058 build: proc_ops check fails with SUBARCH undefined
During configure with config.cache enabled SUBARCH may not
be defined.
Move the definition to a location that must be traversed.
Lustre-change: https://review.whamcloud.com/48101
Lustre-commit:
8a7aa8d5902fcffe6c8b56d0c92ee36380a26e88
Test-Parameters: trivial
Fixes:
a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0a7b4de3ecccd41b1c55e8b2df29039517e0c416
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48423
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 24 Aug 2022 19:26:13 +0000 (12:26 -0700)]
LU-16090 build: Module.symvers lookup by flavor on SUSE
When multiple kernel flavors are found we need to select only
the Module.symvers for the flavor that is being built.
Lustre-change: https://review.whamcloud.com/48195
Lustre-commit:
f3a9921ae4f9c3e48328f2c682e0c7e61221e0d3
HPE-bug-id: LUS-11149
Test-Parameters: trivial
Fixes:
1f4aaefe1aae ("LU-15962 build: add in-kernel Module.symvers to symbol path")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I1c9af91108534d3a67f816077756fded4cd0b653
Reviewed-on: https://review.whamcloud.com/48328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Zarochentsev [Thu, 1 Sep 2022 17:14:10 +0000 (10:14 -0700)]
LU-16082 ldiskfs: old-style EA inode handling fix
The upstream version of EA inodes support coming
with RHEL8 (linux kernel 4.18+) have a slightly different
implementation of EA inodes support and also have a
compatibility code to recognize old-style Lustre-only EAs.
Unfortunately the compatibility code is broken and makes
old xattr data unaccessible due to a wrong hash value check.
Lustre-change: https://review.whamcloud.com/48174
Lustre-commit:
76c3fa96dc30f21e95d80f9119972d7358975258
HPE-bug-id: LUS-11133
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: Icd6f93d4ebb33dcd03b58f9eb364905c18ae81dc
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48412
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 1 Sep 2022 17:31:48 +0000 (10:31 -0700)]
LU-16056 libcfs: restore umask handling in kernel threads
This reverts commit
9013eb2bb5 which incorrectly assumes that Lustre
service threads do not modify umask. A quick grep shows that umask
is modified in osd-ldiskfs __osd_create().
If some other thread sharing the same fs context is modifying umask
in an incompatible way (which includes all Lustre threads after
this patch) then it will occasionally break created file access
permissions for Lustre.
Lustre-change: https://review.whamcloud.com/48233
Lustre-commit:
c92bdd97d99cc755a187987f3d5963adeb3ea475
Change-Id: I589b72e4286dc84f4e3f1a0c54fe31aa988e6c18
Fixes:
9013eb2bb5 (LU-9859 libcfs: don't call unshare_fs_struct()")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/48414
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 25 Mar 2022 08:24:32 +0000 (09:24 +0100)]
LU-15003 sec: use enc pool for bounce pages
Take pages from the enc pool so that they can be used for
encryption, instead of letting llcrypt allocate a bounce page
for every call to the encryption primitives.
Pages are taken from the enc pool a whole array at a time.
This requires modifying the llcrypt API, so that new functions
llcrypt_encrypt_page() and llcrypt_decrypt_page() are exported.
These functions take a destination page parameter.
Until this change is pushed in upstream fscrypt, this performance
optimization is not available when Lustre is built and run against
the in-kernel fscrypt lib.
Using enc pool for bounce pages is a worthwhile performance win. Here
are performance penalties incurred by encryption, without this patch,
and with this patch:
||=====================|=====================||
|| Performance penalty | Performance penalty ||
|| without patch | with patch ||
||==========================================|=====================||
|| Bandwidth – write | 30%-35% | 5%-10% large IOs ||
|| | | 15% small IOs ||
||------------------------------------------|---------------------||
|| Bandwidth – read | 20% | less than 10% ||
||------------------------------------------|---------------------||
|| Metadata | N/A | 5% ||
|| creat,stat,remove | | ||
||==========================================|=====================||
Lustre-change: https://review.whamcloud.com/47149
Lustre-commit:
f3fe144b8572e9e75bb55076e29057227476ebf5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
Change-Id: I3078d0a3349b3d24acc5e61ab53ac434b5f9d0e3
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48411
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 12 Aug 2022 07:59:02 +0000 (09:59 +0200)]
LU-16085 llite: fix stat attributes_mask
Fix stat attributes_mask to return STATX_ATTR_ENCRYPTED whenever it is
possible. Also fix sanityn test_106c to expect at least the 0x30 flag
for attributes_mask.
Lustre-change: https://review.whamcloud.com/48208
Lustre-commit:
0e48653c27eacad29dbff1589da771ad4f5d1014
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
LU-16085 tests: fix sanityn test_106c
Fix sanityn test_106c after modification introduced when fixing
stat attributes_mask.
Lustre-change: https://review.whamcloud.com/48435
Lustre-commit: TBD (
12ab7defee65a82b495dab9ba2da03dcc7d55799)
Test-Parameters: trivial testlist=sanityn env=ONLY=106c
Fixes:
0e48653c27 ("LU-16085 llite: fix stat attributes_mask")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Icd16beff058c42d77e9b04ad1a287ec2ac04dfed
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/48404
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 17 Feb 2022 15:40:20 +0000 (16:40 +0100)]
LU-9243 gss: fix GSS struct definition badness
struct lgssd_ioctl_param should not be defined in multiple places. So
move it to a new header file lgss.h that can be included from kernel
space and user space.
struct gss_header, struct gss_rep_header, struct gss_err_header and
struct gss_wire_ctx are going on the wire, so they need to be moved to
lustre_idl.h, and be wire-checked.
Lustre-change: https://review.whamcloud.com/46543
Lustre-commit:
7dfbc71350c982be20d75e7d7853038809f4e850
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I97c4a8322e6bb7627c6dff5f068931278f4567d7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47664
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sat, 20 Aug 2022 00:07:07 +0000 (17:07 -0700)]
LU-16089 kernel: kernel update RHEL 7.9 [3.10.0-1160.76.1.el7]
Update RHEL 7.9 kernel to 3.10.0-1160.76.1.el7.
Lustre-change: https://review.whamcloud.com/48202
Lustre-commit: TBD (from
109d26b15b4d24d11bcb76fef00bf06dd7d5e975)
Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9
Change-Id: I97d087a5d5bb27996a5c0caf382c011928c651b4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48270
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 4 Feb 2022 14:53:35 +0000 (15:53 +0100)]
LU-15508 gss: protect from arbitrary write to init channel
In case some arbitrary data was written to the gss init channel,
directly return -EINVAL. This protects against unsolicited
authentication requests, and avoids having a dandling entry
in the auth init cache.
Lustre-change: https://review.whamcloud.com/46461
Lustre-commit:
f06807d409c82dd85c208c79c8072402e77f9bac
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iadde630012e4ded83f9609fbb3e10b2e092deb57
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-on: https://review.whamcloud.com/47668
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 25 Jul 2022 14:39:56 +0000 (16:39 +0200)]
LU-16045 enc: force use of new enc xattr on new servers
When an older client uses encryption with a newer server, the client
wants to see the encryption context in security.c xattr. But
internally on server side, we force use of newer encryption.c xattr
for consistency purpose. When required, the encryption context is put
in the request to the client as usual, which interprets it as desired.
Lustre-change: https://review.whamcloud.com/48035
Lustre-commit:
afa4c31087f21fef8d64f572e08a7bec77bc7325
Fixes:
4231fab66e ("LU-13717 sec: make client encryption compatible with ext4")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I667e123bdff912acc270666e8c74ebda6f0534e7
Reviewed-on: https://review.whamcloud.com/48036
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Wed, 27 Jul 2022 12:39:26 +0000 (12:39 +0000)]
LU-16012 sec: fix detection of SELinux enforcement
On newer distros (e.g. RHEL 9.0), on which selinux_is_enabled() does
not exist anymore, the only way to find out if SELinux is enforced
when initializing the security context is to fetch the length of the
security attribute name. If it is 0, we conclude SELinux is disabled.
Lustre-change: https://review.whamcloud.com/48049
Lustre-commit:
155cbc22ba4f758cf9eec415f36f940ca2b23de9
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ifcdcb8ffbb7f9ad50d16d7d3317e94d0d212fa42
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48193
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 16 Aug 2022 01:04:25 +0000 (18:04 -0700)]
LU-15959 kernel: new kernel [SLES15 SP4 5.14.21-150400.24.18.1]
This patch makes changes to support new SLES15 SP4 release
with kernel 5.14.21-150400.24.18.1 for Lustre client.
Lustre-change: https://review.whamcloud.com/47696
Lustre-commit: TBD (from
4bf090b81119d02ed5baa59c6857e2c88a746736)
Test-Parameters: trivial clientdistro=sles15sp4 \
env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity
Test-Parameters: trivial clientdistro=sles15sp3
Change-Id: I0bf548835578163767d2f6a2a5e5bd2b33154871
Co-Authored-By: Minh Diep <mdiep@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47924
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Fri, 13 May 2022 15:03:22 +0000 (17:03 +0200)]
LU-15855 enc: enc-unaware clients get ENOKEY if file not found
To reduce issues with applications running on clients without keys
or without fscrypt support that check for the existence of a file in
an encrypted directory, return -ENOKEY instead of -ENOENT.
For encryption-unaware clients, this is done on server side in the
mdt layer, by checking if clients have the OBD_CONNECT2_ENCRYPT
connection flag.
For clients without the key, this is done in llite when the searched
filename is not in encoded form.
Lustre-change: https://review.whamcloud.com/47349
Lustre-commit:
00898697f998c095e6dab3dd27e347cb1150ee8c
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9a3b7af3a856b7fc7222c61a308ad23168869d57
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47790
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 8 Mar 2022 11:07:47 +0000 (12:07 +0100)]
LU-15625 sec: fix set_param -P for root squash
lctl set_param -P needs to be fixed for root squash in order
to properly propagate to clients.
On server (MDT) side, the paramaters are mdt.*.*squash*.
On client side, the parameters are llite.*.*squash*.
So if the setting on MGS side is done as
set_param -P *.*.root_squash and
*.*.nosquash_nids, it works fine. But if it is done as
set_param -P mdt.*.root_squash, this setting is not applied to
client side.
The solution is to have the settings translated into
*.*.root_squash and *.*.nosquash_nids to make both sides happy.
Also, to ensure client vs. server consistency, prevent
set_param -P llite.*.root_squash.
Lustre-change: https://review.whamcloud.com/46739
Lustre-commit:
7b8449e8f9ed310376a191ec8dfa2f1cb6233377
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8984c9d9024d7a053e548ed15d1321b281d9940f
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47792
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 13 Jun 2022 12:41:11 +0000 (14:41 +0200)]
LU-15896 gss: support OpenSSLv3
Lustre GSS code makes use of some OpenSSL API that has been
deprecated in v3, namely all the functions in the DH_* family.
So replace them with their EVP_PKEY_* counterparts if Lustre is
built on a system with OpenSSLv3.
Lustre-change: https://review.whamcloud.com/47717
Lustre-commit:
615691a531a80b75c4dd054dbb86d0bdbf4cf808
Fixes:
ee60c14360 ("LU-15896 gss: ignore OpenSSLv3 deprecated API")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I78a4ca18b25aca3c34fe84e41413a33caddc01b6
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/48184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 18 Jul 2022 16:58:52 +0000 (09:58 -0700)]
LU-15888 build: Debian dkms-debs requires ed and libkeyutils
dkms install/build needs dependencies on libmount-dev,
libkeyutils1, and libkeyutils-dev
Debian does not install the 'ed' package by default.
Without the 'ed' package the version is not correctly added
to the changelog and parsed to the package names.
Debian does not have linux-image or linux-headers psuedo
packages so require the arch specific ones, ex:
linux-image | linux-image-amd64 | linux-image-arm64
and:
linux-headers | linux-headers-amd64 | linux-headers-arm64
respectively.
o2ib fails to find Debian in-kernel Module.symvers and
should check $LINUX_OBJ/Module.symvers before failing.
Lustre-change: https://review.whamcloud.com/47455
Lustre-commit:
7dc6e1128a030c7e12eb23ea41935ed6fe77ce1f
HPE-bug-id: LUS-10984
Test-Parameters: trivial
Fixes:
85a6eebeca1 ("LU-15652 build: On Debian detect -common kernel headers")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I82e2689f3af4b9ce106ee3ab6b4109d2709c8872
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47967
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Wed, 20 Jul 2022 19:10:23 +0000 (12:10 -0700)]
LU-15993 ofd: don't leak pages if nodemap fails
ofd_commitrw() shouldn't exit w/o calling ofd_commitrw_write(),
otherwise the pages taken in ofd_preprw() are leaked.
same in mdt_obd_commitrw()
Lustre-change: https://review.whamcloud.com/47873
Lustre-commit:
ee4b50278e4471b7c64e619e13699c560502b81e
Fixes:
bbfdc7c167 ("LU-14739 quota: fix quota with root squash enabled")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Icd60c7ab80c5a7b65603d7da0d2e83872dc6b97f
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48003
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sat, 23 Jul 2022 06:49:40 +0000 (23:49 -0700)]
LU-13813 tests: fix stack_trap in conf-sanity test 110/111
This patch fixes stack_trap in conf-sanity test 110 and 111
to restore test environment.
Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=el8.5 serverdistro=el8.5 testlist=conf-sanity
Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
fstype=zfs \
clientdistro=el8.5 serverdistro=el8.5 testlist=conf-sanity
Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
mdscount=2 mdtcount=4 \
clientdistro=el8.5 serverdistro=el8.5 testlist=conf-sanity
Change-Id: I540d96e8ad2c4990e7da18fe22256b44e9a19c72
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Thu, 4 Aug 2022 06:37:19 +0000 (23:37 -0700)]
LU-16037 build: remove quotes from %{mkconf_options}
This patch fixes lustre-dkms.spec.in to remove quotes
from %{mkconf_options} passed to dkms.mkconf, so as to
resolve the following build issue:
dkms.conf: Error! Directive 'DEST_MODULE_LOCATION'
does not begin with '/kernel', '/updates', or '/extra'
in record #0.
Lustre-change: https://review.whamcloud.com/48044
Lustre-commit:
33efefc496159f7d0caed0fa85d8f92603060ae2
Test-Parameters: trivial
Change-Id: I0b365d7a96cb632680bc2321e87b28a3bf076e47
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48117
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 26 Jul 2022 23:37:25 +0000 (16:37 -0700)]
LU-16048 build: Update ZFS version to 2.1.5
Update ZFS version to 2.1.5. The changes are listed in:
https://github.com/openzfs/zfs/releases/tag/zfs-2.1.5
Change-Id: I9f25aafe889f87fb80677e59dbe4679932d8b920
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48046
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Wed, 29 Jun 2022 00:24:32 +0000 (19:24 -0500)]
LU-15983 lnet: Define KFILND network type
Define the KFILND network type. This reserves the network type number
for future implementation and allows creation of kfi peers and
adding routes to kfi peers.
Lustre-change: https://review.whamcloud.com/47830
Lustre-commit:
5fea36c952373c9a235be7bf57eb2e516fcb36b2
Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-11060
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I9111645f1290c8af4937d1b2689a068df81922a4
Reviewed-on: https://review.whamcloud.com/47987
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 28 Jun 2022 22:05:33 +0000 (15:05 -0700)]
LU-15779 ofd: don't hold read lock over bulk
as this can block all operations on OST:
1) ofd_preprw_read() takes a shared object lock and initiates BULK
2) OUT needs an exclusive object lock on the same object
3) ofd_commitrw_write() starts transaction and now has to wait
for OUT to get and release that exclusive object lock (step 2)
4) number of threads can get stuck waiting for ofd_commit_write()
to stop it's transaction
this patch drops a shared object lock before BULK transfer.
at the moment it's not clear how such read would race with
object removal on ZFS - this should be investigated.
Lustre-change: https://review.whamcloud.com/47126
Lustre-commit:
98ba50819024b908453b62fd095647442929a61f
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I131493abd90283e9ca897f904e00c25d26e3d8d3
Reviewed-on: https://review.whamcloud.com/47825
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Mon, 15 Aug 2022 16:03:00 +0000 (09:03 -0700)]
LU-15874 kernel: new kernel [RHEL 9.0 5.14.0-70.22.1.el9_0]
This patch makes changes to support new RHEL 9.0 release
for Lustre client.
fix lbuild to include modified find-requires.ksyms
Lustre-change: https://review.whamcloud.com/47847
Lustre-commit:
bbe5e9818053e43ebf97e2d3fa240917bfbd8336
Test-Parameters: trivial clientdistro=el9.0 \
env=SANITY_EXCEPT="101j 130 244a" testlist=sanity
Change-Id: Ib7fdf9d3946df626759d395b5000b375391da344
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47857
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Wed, 10 Aug 2022 17:50:02 +0000 (13:50 -0400)]
New release 2.15.1
Change-Id: If811b50be420ba9576057c9273f541ccd5fe1679
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Sat, 16 Jul 2022 23:30:36 +0000 (19:30 -0400)]
New rc 2.15.1-RC1
Change-Id: I0abcf7295d077ff3e364082da768940bbf28216d
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 29 Jun 2022 21:37:30 +0000 (14:37 -0700)]
LU-15521 spec: fix more bare words error with rpm 4.16
This patch fixes more bare words errors and extra tokens
warnings with rpm 4.16.
Lustre-change: https://review.whamcloud.com/47833
Lustre-commit:
81530fe44c6b8d8819b8e70ea99ef7014c23a05b
Test-Parameters: trivial
Change-Id: Ic40b5763d1cb362d5aa77b06e9a5768b2abbc708
Fixes:
597a6bf9e085 ("LU-15521 spec: fix bare words error with rpm 4.16")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47835
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 9 Jun 2022 08:00:52 +0000 (10:00 +0200)]
LU-15922 sec: new connect flag for name encryption
Introduce OBD_CONNECT2_ENCRYPT_NAME connection flag for compatibility
with older versions that do not support name encryption.
When server side does not have this flag, client side is forced to
null encryption for file names. And client needs to use old xattr to
store encryption context.
Also update tests in sanity-sec to exercise name encryption only if
server side supports it.
Lustre-change: https://review.whamcloud.com/47574
Lustre-commit:
71d63602c57e6d0cb723ae02ea767c6c170e5997
Fixes:
ed4a625d88 ("LU-13717 sec: filename encryption - digest support")
Test-Parameters: serverversion=2.14 testlist=sanity-sec mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 fstype=ldiskfs
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I446a4caba8e45821d701628a14c96f03cb6c4525
Reviewed-on: https://review.whamcloud.com/47852
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 8 Jul 2022 18:08:26 +0000 (11:08 -0700)]
LU-15991 kernel: kernel update RHEL7.9 [3.10.0-1160.71.1.el7]
Update RHEL7.9 kernel to 3.10.0-1160.71.1.el7.
Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9
Change-Id: I89215145ea8da2925e5c8c01cdf963ba8a087877
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47920
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Thu, 23 Jun 2022 06:36:19 +0000 (16:36 +1000)]
LU-13189 osd-zfs: add project id for old objects without ZFS_PROJID
After project quota zpool upgrade, the ZFS_PROJID
flag could still be missing on some old objects.
We used to check for this and return ENXIO in
osd_declare_attr_set(), however the check is changed
by "LU-12309 osd-zfs: Support disabled project quotas".
Now if the target project id is the default
project id 0, we will pass the check in
osd_declare_attr_set() and trigger the assert in
osd_attr_set() later.
Instead of returning ENXIO, we could adjust the
attribute layout of the old objects to accommodate
project id.
Also add back the logic from "LU-14740 quota: reject
invalid project id on server side", which got removed
by "LU-14927 quota: move qsd_transfer to lquota module",
due to using GPL symbols.
Lustre-change: https://review.whamcloud.com/47709
Lustre-commit:
ec79791a7cda5b66649200b16a70167d86059e65
Change-Id: Ib62fdd2a0e07f15ae12daf564273a249a54dd8ea
Fixes:
291e7196d3 ("LU-12309 osd-zfs: Support disabled project quotas")
Fixes:
d2e8208e22 ("LU-14927 quota: move qsd_transfer to lquota module")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/47846
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 28 Jun 2022 07:25:47 +0000 (00:25 -0700)]
LU-15956 gss: allow build without ssk
The GSS part of Lustre should be able to build without SSK, in case
some SSK requirements are not met at configure time.
Lustre-change: https://review.whamcloud.com/47681
Lustre-commit:
2e061e88a44db70e4446be567243a38a76ed5370
Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ieacbcb5db77fcc12cc13579785e640857ce7fb02
Reviewed-on: https://review.whamcloud.com/47781
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 28 Jun 2022 07:36:04 +0000 (00:36 -0700)]
LU-15973 build: remove AC_DEFINE(__state, state, ...)
RHEL 8.6 build failed with MLNX_OFED 5.6-2.0.9.0 as follows:
error: 'struct task_struct' has no member named 'state'; did you mean '__state'?
#define __state state
^~~~~
The failure was introduced by
commit
bb7c82f13e7a01891edcdf0626c6fb91a240e56e and a proper
way to resolve the original issue was in
commit
c04adbcd76725a360f411f09c63df785bf7db426. So, let's
remove the improper way AC_DEFINE(__state, state, ...).
Lustre-change: https://review.whamcloud.com/47799
Lustre-commit:
f4c65e01a2e58b2bae770770590494d85ef636f7
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity
Change-Id: Icbc897cf5870352311262af8bd59ec24ea9d7301
Fixes:
bb7c82f13e7 ("LU-15795 kernel: RHEL 8.6 server support")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47807
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 24 Jun 2022 07:02:31 +0000 (00:02 -0700)]
LU-15962 build: add in-kernel Module.symvers to symbol path
After building Lustre with in-kernel OFED, installing
ko2iblnd module hit the following errors:
ko2iblnd: disagrees about version of symbol __ib_alloc_pd
ko2iblnd: Unknown symbol __ib_alloc_pd (err -22)
ko2iblnd: disagrees about version of symbol rdma_resolve_addr
ko2iblnd: Unknown symbol rdma_resolve_addr (err -22)
Those exported symbols are contained in in-kernel Module.symvers,
which should be added to the symbol path KBUILD_EXTRA_SYMBOLS.
Lustre-change: https://review.whamcloud.com/47699
Lustre-commit:
1f4aaefe1aae8eb5c75fff446f4dc377e968677f
Test-Parameters: trivial
Change-Id: Ic30caa7079af00a452ea24e7e982a856874af702
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47760
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 24 Jun 2022 06:59:02 +0000 (23:59 -0700)]
LU-15940 build: add a required dependency for libmount
The Lustre client utilities (mount/umount) have
an optional dependency on libmount to update utab.
However, libmount has been introduced in util-linux
since 2.18 in 2010. There is no need to make the
dependency as optional.
Lustre-change: https://review.whamcloud.com/47619
Lustre-commit:
f21b9441275bd0d8b0d56cf8441aa67bede37ddc
Test-Parameters: trivial clientdistro=ubuntu2004
Test-Parameters: trivial clientdistro=el8.5
Change-Id: I4b965a5ce6cb6fc5d2061a53c44ef9b709ebab49
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47759
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 24 Jun 2022 06:44:51 +0000 (23:44 -0700)]
LU-15652 build: On Debian detect -common kernel headers
Check for a matching /usr/src/linux-headers-<ver>-common/
and update the --with-linux argument accordingly.
Also move LC_GLIBC_SUPPORT_COPY_FILE_RANGE outside
of utils as this also breaks the dkms build on Debian
with 'static' follows 'non static' declairation of
copy_file_range.
Lustre-change: https://review.whamcloud.com/46836
Lustre-commit:
85a6eebeca1e1efb5bb9c3e737ab1978f8aca741
HPE-bug-id: LUS-10826
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I6e4f0b27eba6c5b07cda14f064e57aa9c93ae3cc
Reviewed-on: https://review.whamcloud.com/47758
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 24 Jun 2022 06:40:24 +0000 (23:40 -0700)]
LU-15703 ldiskfs: Disable unused fast commit buffer
Linux commit v5.9-rc7-39-g6866d7b3f2bb
ext4 / jbd2: add fast commit initialization
Disable journal fast commit buffer via a mount option because it is
not used by lustre's ldiskfs since it will break recovery.
Linux commit v5.10-rc2-9-gede7dc7fa0af
jbd2: rename j_maxlen to j_total_len and add
jbd2_journal_max_txn_bufs
Change osd_transaction_size to use jbd2_journal_get_max_txn_bufs
and provide a jbd2_journal_get_max_txn_bufs when it is not
provided.
Lustre-change: https://review.whamcloud.com/46943
Lustre-commit:
ed80c3f64317a87ed10919cedbf2d65b36abcdcb
Test-Parameters: trivial
HPE-bug-id: LUS-10858
Fixes:
c93a3e5b15 ("LU-14195 ldiskfs: update patches for Linux 5.10")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I9bffc3559a8bbce9d4c1c2b6692cb8518f3f991a
Reviewed-on: https://review.whamcloud.com/47757
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Fri, 24 Jun 2022 06:36:31 +0000 (23:36 -0700)]
LU-15933 libcfs: fix configure check for iov_iter member
Kernel commit v5.13-rc1-12-g8cd54c1c8480 changed iov_iter member
->type to ->iter_type, add check at configure to cover this change.
Lustre-change: https://review.whamcloud.com/47601
Lustre-commit:
ae53ef4a6ab0491c01b723322f45c931e22f6140
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8da26e1f6c6242a3260b458770ec34a65aecaf7d
Reviewed-on: https://review.whamcloud.com/47756
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log