Whamcloud - gitweb
Patrick Farrell [Thu, 29 Jun 2023 17:52:01 +0000 (13:52 -0400)]
EX-6265 llite: allow mmap reads of compressed files
mmap reads of compressed files work, so we should only
block writes.
We cannot block the actual fault operations because that
will cause the application to get a SIGBUS, so we check
the file open mode when we go to create the memory mapping.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51506
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Zhuravlev [Thu, 22 Jun 2023 08:36:51 +0000 (11:36 +0300)]
EX-6265 llite: disable mmap on compressed files
disable mmap(2) on compressed files until well tested.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51413
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Deiter [Mon, 9 Jan 2023 16:19:26 +0000 (20:19 +0400)]
LU-16399 tests: add subtests setup/cleanup records
* Added setup/cleanup records for subtests
Lustre-change: https://review.whamcloud.com/49582
Lustre-commit:
d0ae0079d747d05f74f733fb594d8edb512f8b16
Change-Id: Icb203a864fa8785e423a073b4ee0f02ea3d3ac77
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51406
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Fri, 19 May 2023 02:12:19 +0000 (19:12 -0700)]
LU-16836 lnet: ensure dev notification on lnd startup
Look up device and link state on lnd startup so that
the initial NI state may be set properly.
Reduce code duplication by adding lnet_set_link_fatal_state() and
lnet_get_link_status() functions which are shared across LNDs.
LND-specific versions of these are removed.
This fixes the issue with adding LNet NI using an interface with
cable unplugged which results in the NI state initialized as "up".
Lustre-change: https://review.whamcloud.com/51057
Lustre-commit:
09c6e2b872287c847d15620788f6cf50b3a9f30b
Fixes:
c4df48116d ("LU-16563 lnet: use discovered ni status")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I16084092cc21a4e42dfef4624adfbf57eb4fdecb
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51310
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Wed, 28 Jun 2023 09:36:19 +0000 (17:36 +0800)]
EX-7509 tests: enable neterror logging in sanity-benchmark/iozone
This patch enables LNet error logging in sanity-benchmark test_iozone()
to gather network errors while connection issue occurs.
Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=el8.6 serverdistro=el7.9 testlist=sanity-benchmark
Change-Id: I398779abc95525fe5579fc7505e6e6221c32bf90
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51483
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Tue, 21 Jun 2022 12:06:22 +0000 (20:06 +0800)]
LU-8582 target: send error reply on wrong opcode
Unknown opcode does not necessarily means insane client. A new client
might send RPCs with new opcodes to an old server. The client might
desperately stuck there waiting for a reply. So, send an error back
when RPC has a wrong opcode.
This patch returns the EOPNOTSUPP to client instead of block. ENOTSUPP
is not used here since strerror() does not understand ENOTSUPP.
OBD_FAIL_OST_OPCODE=0x253 is added for fault injection test of opcode.
To test whether an invalid opcode is handled properly on OST, use the
following command:
lctl set_param fail_val=${opcode} fail_loc=0x253
Lustre-change: https://review.whamcloud.com/47761
Lustre-commit:
03c1ddf19c83891683e1726f240a2449941e8b22
Change-Id: I46ca62bc532b92368e06a4f883b102c7186c453c
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51513
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
John L. Hammond [Wed, 20 Jan 2021 15:22:54 +0000 (09:22 -0600)]
LU-14301 lustre: add ENOTSUPP to spelling.txt
Add a spelling check for ENOTSUPP to suggest use of EOPNOTSUPP
instead. Note:
ENOTSUPP (524) and defined only in the kernel errno.h and is a NFSv3
specific errno. If ENOTSUPP is returned to userspace then strerror()
will print "Unknown error 524".
EOPNOTSUPP (95) is defined in kernel and userspace errno.h.
ENOTSUP is defined in userspace errno.h as an alias for EOPNOTSUPP.
Lustre-change: https://review.whamcloud.com/41280
Lustre-commit:
e00733f0f87659c936039a58ea738cfb070638bc
Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I13b0389c9ec0853f43d8ab4a8f6538eb24c8a2ad
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51512
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Sebastien Buisson [Wed, 28 Jun 2023 15:05:18 +0000 (17:05 +0200)]
EX-7683 man: document supported compression types
Update lfs-setstripe to mention the currently supported compression
types: gzip, lz4, lz4fast and lzo.
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51485
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 28 Jun 2023 09:34:22 +0000 (09:34 +0000)]
EX-7775 utils: fix cp_comp_type size
cp_comp_type should be 8 bits, as llch_compr_type and all associated
variables are declared as u8.
So remove useless cp_comp_enabled and fix code to test for compressed
component with cp_comp_type against LL_COMPR_TYPE_NONE.
And update LL_COMPR_TYPE_MAX value to 255 to avoid conflicts with
future compression types.
Fixes:
f43b9ce9af ("EX-6127 osc: osc brw request compression")
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia15868ac0ac003b62942540a57f782226ae8c141
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51481
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Tue, 27 Jun 2023 14:42:20 +0000 (16:42 +0200)]
EX-7775 utils: fix LL_COMPR_TYPE wire checks
Fix LL_COMPR_TYPE* wire checks to avoid duplication.
The value of LL_COMPR_TYPE_UNCHANGED is also declared as 15 so that
is does not conflict with other potential real compression types in
the future.
Fixes:
67d4601737 ("EX-6249 csdc: set compress component for file")
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iab9830f09f0778e1e1f3b1ea4c9878ce1017de8d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51473
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Tue, 20 Jun 2023 15:49:14 +0000 (15:49 +0000)]
EX-7683 utils: update handling of compr level for lz4
The way lz4 compression level or acceleration factor is handled needs
to be adapted in order to match what is provided by the lz4 userspace
tool:
- any level between 0 and 2 is interpreted as the default lz4
acceleration factor of 1;
- any level from 3 and up to 16 is interpreted as a compression level
for internal lz4hc. Increasing the compression level trades CPU
time for improved compression ratio;
- acceleration factor can be specified for the lz4fast compression
type, from 1 to 26. This acceleration factor trades compression
ratio for faster speed.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51380
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 21 Jun 2023 12:00:07 +0000 (12:00 +0000)]
EX-7715 utils: error for invalid compression level
Not all compression types accept a compression level (e.g. lzo in its
kernel implementation). For those, return an error if a compression
level is provided on 'lfs setstripe' command-line.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51394
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alena Nikitenko [Wed, 28 Jun 2023 13:28:44 +0000 (16:28 +0300)]
EX-6931 tests: add version check in sanity test_904
The patch that added hide virtual projid xattr functionality
to b_es6_0 branch was submitted under a tag of 2.14.0-ddn55.
In case of interop testing with older servers test 904 fails.
At the same time the rest of the test should still run on
older servers, so I moved parts of the code related to hide
virtual projid xattr functionality under version check
condition.
Sanity test 904 was modified.
Fixes:
8acf413647 ("LU-15548 osd-ldiskfs: hide virtual projid xattr")
Test-Parameters: trivial testlist=sanity env=ONLY="904"
Test-Parameters: testlist=sanity env=ONLY="904" serverversion=2.14.0-ddn54
Signed-off-by: Alena Nikitenko <anikitenko@ddn.com>
Change-Id: Ie3bc67cb8a7a83954f3b2048f009f84ab77bf53d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51484
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Thu, 29 Jun 2023 15:01:35 +0000 (23:01 +0800)]
EX-7752 test: hot-pools test_59: not expected flag 'stale'
Make the 1st mirror preferable for write for the test purpose.
Test-Parameters: trivial env=ONLY=59 testlist=hot-pools
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I06c33badfc10ff9743df83a6e421e27afc9b6dbd
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51502
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 24 Jun 2023 18:15:33 +0000 (12:15 -0600)]
RM-620 build: New tag 2.14.0-ddn91
New tag 2.14.0-ddn91
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8634b3650bf43aadaf303e48b9ebe2931e6af6ab
Jian Yu [Fri, 23 Jun 2023 03:10:35 +0000 (11:10 +0800)]
LU-16923 kernel: update RHEL 8.8 [4.18.0-477.13.1.el8_8]
Update RHEL 8.8 kernel to 4.18.0-477.13.1.el8_8.
Lustre-change: https://review.whamcloud.com/51411
Lustre-commit: TBD (from
a04c2ff950a2a6b65e55184113b1513fc9bf0058)
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Change-Id: I23c6439aedd6f8e9473ddb629ff7e01c50d9c8fc
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51420
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Fri, 23 Jun 2023 22:21:28 +0000 (01:21 +0300)]
EX-7680 llite: skip hole/data lseek() on compressed file
Doesn't execute real lseek() on compressed file with
SEEK_HOLE/SEEK_DATA origin but consider file always
non-sparsed and do generic lseek() only
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Thu, 18 May 2023 21:41:47 +0000 (15:41 -0600)]
LU-15671 mds: do not send OST_CREATE transno interop
Send OST_CREATE RPCs from the MDS with no_resend and no_delay
when communicating with an old OST that does not support the
OBD_CONNECT2_REPLAY_RESEND. Likewise, the OST should not reply
to the MDS RPC with rq_transno set, or this will trigger:
osp_precreate_send() ASSERTION(req->rq_transno == 0) failed
This can be avoided if the MDS is upgraded before the OSS, but
will always be hit if OSS is upgraded first.
After 2.20.53 the MDS/OSS assume that this is always true, since
rolling upgrades are unsupported for larger version differences.
Lustre-change: https://review.whamcloud.com/51056
Lustre-commit:
9ee1281060d0a00a9c5d715a9a6d9b99c27123ff
Test-Parameters: testgroup=rolling-upgrade-oss
Fixes:
63e17799a3 ("LU-8367 osp: enable replay for precreation request")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I1ab601a2f55540dd75cf24838f7cdb7f823ed42c
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51425
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Artem Blagodarenko [Wed, 14 Jun 2023 21:10:28 +0000 (22:10 +0100)]
EX-6127 llite: getting stripe info optimization
ll_lov_getstripe_ea_info() is expensive call and should be avoided
if possible. Let's use cached chunk size rather than get it from
stripe info every time.
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51321
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Patrick Farrell [Tue, 13 Jun 2023 17:26:09 +0000 (13:26 -0400)]
EX-7601 llite: restrict overwrites during preview
EX-7601 is an issue where when modifying a compressed file
we do not correctly read-up and re-write existing
compressed data.
To avoid this, we can only allow writes which are not
aligned to compression chunk size when they are not
overwriting existing data, ie, when they are extending the
file.
This returns EINVAL for all writes to compressed files
which are not either chunk aligned or extending the file.
This should prevent users from hitting the data corruption
issue but still allows some basic usage. This is intended
just for the preview period.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51259
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Patrick Farrell [Fri, 2 Jun 2023 02:24:15 +0000 (22:24 -0400)]
EX-6127 llite: DIO fallback on compressed files
Fully supporting direct I/O on compressed files is tricky
because we cannot pull the full chunk in to the page cache
(because there is no page cache for DIO).
So instead we fall back to buffered I/O for DIO on
compressed files.
This patch adds the check and a test for this.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I8224ef9b8ad1d912d8a11eccad37d3dff8dd8498
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51200
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Artem Blagodarenko [Fri, 19 May 2023 09:27:36 +0000 (10:27 +0100)]
EX-7528 lov: fallocate is not allowed for compressed files
Client Side Data Compression allocates blocks after a compression.
It is impossible to preallocate blocks for the whole file, so
fallocate should be disabled in case of compression.
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51059
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Artem Blagodarenko [Wed, 30 Nov 2022 14:54:57 +0000 (14:54 +0000)]
EX-6127 osc: osc brw request compression
This patch adds client-side compression/decompression.
The client-side data compression project (CSDC) reduces storage and network
utilization by leveraging the more plentiful memory and CPU resources on the
local client. Data is sent compressed over the network, saved directly to
storage on the server side, and decompressed back on the client side.
Uncompressed data is kept in client page cache, all while being functionally
transparent to the end user and application.
As an example, a test file is compressed and decompressed.
The resulting file is compared with the original one.
The test case shows 2.5x compression ratio:
356K /mnt/lustre/d460.sanity/sanity.sh
884K /tmp/cmp-46ofie/decompressed_sanity.sh
Compression should read whole chunk even if offset
and size differ.
Let's modify readahead to force reading data from the
offset and size multiple to the chunk size.
Test-Parameters: testlist=sanity env=ONLY=460
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49511
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Wed, 24 May 2023 00:25:05 +0000 (08:25 +0800)]
LU-16837 csdc: reserve connect bits for compressed layout
Add connect data bit for compressed layout (OBD_CONNECT2_COMPRESS)
and another connect data bit to be used (OBD_CONNECT2_LARGE_NID).
Also reserve obd_connect_data::ocd_compr_type which is a bitmask of
supported compression type to be negotiated between client and MDS.
Lustre-change: https://review.whamcloud.com/51108
Lustre-commit:
83189aef3b23f18cb8c1deae34994a00f8582039
Test-Parameters: trivial
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I21029c6c3e8a7e690ecc8d489bbb95aec3ab1fa8
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51409
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 28 May 2021 08:49:19 +0000 (02:49 -0600)]
LU-1904 idl: add checks for OBD_CONNECT flags
Make it harder to accidentally declare OBD_CONNECT flags without
properly defining their names. Otherwise, this can cause serious
compatibility problems if two features are using the same flag.
Add the definition lines into spelling.txt so there is *always*
a warning generated, since this always needs proper attention.
Make it clear whom to contact when reserving a new feature flag.
Lustre-change: https://review.whamcloud.com/48053
Lustre-commit:
d851381ea6947244842ae6f138cd0bfd399b7ef4
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9a5e2c97c40c39ea57d20979d4b130854edc785a
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51408
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 23 Jun 2023 08:01:51 +0000 (02:01 -0600)]
RM-620 build: New tag 2.14.0-ddn90
New tag 2.14.0-ddn90
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8a73e4233db5fc835eb33849e0508204417e7429
Bobi Jam [Thu, 8 Jun 2023 03:42:46 +0000 (11:42 +0800)]
EX-7593 csdc: don't set compression layout when disabled
When llite_enable_compression is disabled
(lfs set_param llite.*.enable_compression=0), we should check
it before sending it to MDS lest we get a file with compressed
component which we cannot handle.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ib1e2123ffdb239c3e1401d682ae9c2c49e3f4a6f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51250
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 21 Jun 2023 11:07:36 +0000 (11:07 +0000)]
EX-7714 utils: logical AND for 'lfs find' compression exprs
All search expressions provided to 'lfs find' must be combined as a
logical AND. Fix newly added options for compression support, so that
they comply with this logical AND.
Fixes:
093bd2f343 ("EX-6856 utils: add 'lfs find' support for compressed file")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I3b28cd87c1d304df6d04753b413d46f5abcfe16e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51393
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sergey Cheremencev [Tue, 30 May 2023 08:14:47 +0000 (11:14 +0300)]
LU-16778 tests: sanity-quota_75 fix
Change conf=fsync to oflag=direct to avoid
cache write.
Lustre-change: https://review.whamcloud.com/51158
Lustre-commit: TBD (from
b1c5e39335820602abeecbb91a3afb86879f84f2)
Test-Parameters: trivial testlist=sanity-quota env=ONLY=75,ONLY_REPEAT=100
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Iff04bac63f772dc2d0d0ad765d210b2539fbe33e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51407
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Thu, 22 Jun 2023 14:56:23 +0000 (22:56 +0800)]
LU-16922 kernel: update RHEL 9.2 [5.14.0-284.18.1.el9_2]
Update RHEL 9.2 kernel to 5.14.0-284.18.1.el9_2 for Lustre client.
Lustre-change: https://review.whamcloud.com/51410
Lustre-commit: TBD (from
6111eb7a418fb4396e0451d0613af43870951f72)
Test-Parameters: trivial env=SANITY_EXCEPT=27J clientdistro=el9.2 testlist=sanity
Test-Parameters: trivial env=SANITY_EXCEPT=27J clientdistro=el9.2 serverdistro=el8.8 testlist=sanity
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Ifa8f13200550e5f473b7d7d641155e349c453c03
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51416
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Fri, 23 Jun 2023 01:05:20 +0000 (19:05 -0600)]
EX-7596 tests: don't fail if metadata_csum_seed unset
Fix logic in the sanity-pcc cache filesystem setup. With mke2fs
1.47.0-wc1 it enabled metadata_csum_seed unconditionally, but it
caused problems on el7.9 kernels. In 1.47.0-wc2 it disabled that
feature, caused the check for removing the feature to fail.
Since metadata_csum_seed has been available since 1.44.2 it
shouldn't be a problem to force it off duing mke2fs.
Test-Parameters: trivial testlist=sanity-pcc env=ONLY=1
Fixes:
7d35dd13273e7 ("EX-7596 tests: disable metadata_csum_seed for pcc cache device")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic04ab4043981dc9b5c32e01c4aa85be343e3f3f8
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51417
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Sebastien Buisson [Fri, 9 Jun 2023 12:50:25 +0000 (14:50 +0200)]
LU-16888 gss: fix ptlrpc_gss automatic loading
ptlrpc_gss kernel module is automatically loaded when a GSS security
flavor is enforced. Loading success is recorded in a static variable
in the ptlrpc module, which prevents further reloading in case
ptlrpc_gss is unloaded while keeping ptlrpc loaded.
Get rid of this static variable as it is not required in order to
avoid calling request_module("ptlrpc_gss") when not needed. Indeed,
once loaded, the static array policies[] has an entry at the
SPTLRPC_POLICY_GSS index, indicating that the ptlrpc_gss module is
loaded.
Lustre-change: https://review.whamcloud.com/51264
Lustre-commit:
b80d6defb7b018250ef4fafccff7c980aed6a444
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9bb100a202fe9c3fc455a2ffba6ee6398e19b9bf
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51374
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sergey Cheremencev [Mon, 19 Jun 2023 14:55:09 +0000 (18:55 +0400)]
LU-16772 quota: access lqe_glbl_data only under mutex
Hold mutex to protect lqe_glbl_data against freeing in
qmt_lvbo_update, qmt_allock_lock_array and qmt_setup_id_desc.
This patch should help against below(LU-14434) and similar panics.
BUG: unable to handle kernel NULL pointer dereference at
0000000000000000
RIP: 0010:qmt_id_lock_cb+0x69/0x100 [lquota]
qmt_id_lock_cb+0x69/0x100 [lquota]
qmt_glimpse_lock.isra.19+0x27e/0xfb0 [lquota]
qmt_reba_thread+0x5da/0x9b0 [lquota]
kthread+0x112/0x130
ret_from_fork+0x35/0x40
It is the 2nd part of
50ff4d1da63e8. The latest patchset of
https://review.whamcloud.com/c/fs/lustre-release/+/50748/ has extra
changes, but occasionally has been landed without inspection.
So it's not simple porting from upstream.
Fixes:
50ff4d1da63 ("LU-16772 quota: protect lqe_glbl_data in qmt_site_recalc_cb")
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I5753ace94a739c6df70dd8ea3bde828e2b5ed812
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51368
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Tue, 20 Jun 2023 18:57:56 +0000 (12:57 -0600)]
LU-16804 tests: rename 'complete' to 'complete_test'
The test-framework.sh "complete" function conflicts with "complete"
exported from bash_completion, and this causes lustre-initialization
to fail in some configurations now that the lustre test config
is loaded earlier during test-framework.sh init_test_env() setup.
Rename "complete" to "complete_test" to avoid this conflict.
Lustre-change: https://review.whamcloud.com/51383
Lustre-commit: TBD (from
9bf8d1944995e7bf627a602aa1d0523d810c84b6)
Test-Parameters: trivial
Fixes:
fdbb2bc849 ("LU-16804 tests: load CONFIG at beginning of init_test_env")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic72d8d5cc4a65feec6bfb2a76ac5f9b9d78e3f75
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51389
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Charlie Olmstead <charlie@whamcloud.com>
Nathan Rutman [Wed, 22 Feb 2023 22:34:09 +0000 (14:34 -0800)]
LU-16587 utils: give lfs migrate a larger buffer
lfs migrate is slow because it mostly uses a small 1MB buffer. Bigify.
[root@kjlmo4n00 16G]# time lfs migrate -S 1M -p flash 16G.1
real 0m25.341s
[root@kjlmo4n00 16G]# time /root/tools/lfs_nzr migrate -S 1M -p flash 16G.1
real 0m6.526s
Lustre-change: https://review.whamcloud.com/50118
Lustre-commit:
23224e03dc30c89dd449de5a7fe99b0bd3aca495
Signed-off-by: Nathan Rutman <nathan.rutman@hpe.com>
Change-Id: I850ca475fcd0efe2d71d26e4d1544f462c60252a
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51352
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Fri, 12 May 2023 03:24:35 +0000 (11:24 +0800)]
EX-6510 csdc: prefer uncompressed mirror for write
When writing to mirrored files with both compressed and uncompressed
mirrors, prefer the uncompressed components to write, and that is
better for performance, more compatible with older clients, and better
fits the model of compressing files after initial write.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I62a117d5cc3d34e2c0c96d1a9ade8eef0a2d1291
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50974
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Artem Blagodarenko [Tue, 11 Apr 2023 14:04:49 +0000 (15:04 +0100)]
EX-6261 ptlrpc: extend sec bulk functionality
Client Side Data Compression(CSDC) needs buffers pool for efficient
work. Encryption used ptlrpc sec bulk, but it works with pages.
This patch extends sec bulk functionality to allocate different
size buffers. Memory shrinking and other usefull features
should still work as expected.
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I929b4dfdcb0e8197f3804629b000af0d4bd6f2a0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 17 Jun 2023 05:47:21 +0000 (23:47 -0600)]
RM-620 build: New tag 2.14.0-ddn89
New tag 2.14.0-ddn89
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id5516daccbef1da2f342afd79e912bc91bc9da13
Alex Zhuravlev [Thu, 30 Sep 2021 12:16:57 +0000 (15:16 +0300)]
LU-15046 osp: precreate thread vs connect race
lcs_exp (required for fid client) was initialized in osp_obd_connect()
which races with osp_precreate_thread(). the latter can get stuck if
lcs_exp is not initialized and then the whole precreation logic is
blocked until remount. instead the precreation thread can just wait
preliminary until lcs_exp is initialized properly.
Lustre-change: https://review.whamcloud.com/45099
Lustre-commit:
7e0a2b073701e23f6c941d249e034abe1043ccd6
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I7a42bf4b17ce5d46bc25bd548d81eb55f168804b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51308
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Fri, 26 May 2023 17:42:23 +0000 (10:42 -0700)]
LU-16850 socklnd: remove ksnr_myiface from ksock_conn_cb
Drop ksnr_myiface: it is no longer needed since socklnd
TCP bonding got removed. There's one interface per
connection cb per peer_ni, and it can be accessed as
net->ksnn_interface.ksni_index.
Fix setting of ksni_nroutes accordingly. Duplication of
interface index in conn_cb and ksnn_interface was causing
the assertion
ASSERTION( net->ksnn_interface.ksni_nroutes == 0 )
in ksocknal_shutdown() to fail if the corresponding
device is deregistered before lnd shutdown.
Modify test_214 of sanity-lnet to create connections so that
the scenario of socklnd shutdown with NI on a deregistered
interface is covered.
Lustre-change: https://review.whamcloud.com/51148
Lustre-commit:
f6be07c457385cfacd9b802e4cade9f6f6ab7d6f
Fixes:
a7ee03d7ca4185 ("LU-16378 lnet: handles unregister/register events)
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I4de164c9e64aa770164a8320b9460fadce49aa06
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51326
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr NeilBrown [Thu, 15 Sep 2022 05:32:05 +0000 (15:32 +1000)]
LU-13641 socklnd: remove remnants of tcp bonding
->ksnp_n_passive_ips is now always zero, so remove it and all uses of
it. ->ksnp_passive_ips is gone too, as is ksocknal_ip2iface().
Lustre-change: https://review.whamcloud.com/48568
Lustre-commit:
3630e1eaf9db562a1de707762cd649db815459c8
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5de6d027c545087c961673d8704f68c4f3dd5076
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51325
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Nathaniel Clark [Mon, 5 Jun 2023 13:49:29 +0000 (09:49 -0400)]
EX-4130 lipe: Remove old hotpool scripts
Hotpool scripts now live in EMF repo.
The scrips in lipe/ are unmaintained and non-functional.
Change-Id: I2f288f7f68886015ddb9a85da4e437c9f03c3928
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51215
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Artur Novik <anovik@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Mon, 18 Oct 2021 11:26:40 +0000 (13:26 +0200)]
LU-15047 gss: gss integrity check with multi-rail
With multi-rail, a primary NID is used as node identifier, but LNet
decides which NID is actually used for sending/receiving data, on a
per request basis.
For the integrity check mechanism implemented as part of GSS, the
primary NID must be used in order to compute HMAC with the correct
key, independently of the actual NID for the current request.
Lustre-change: https://review.whamcloud.com/45277
Lustre-commit:
c8301a65c5672a1d081669343466746df983eabc
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I2bf3974d3aa0e8365a9413dca56c69ee3734c12b
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51274
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Thomas Bertschinger [Sun, 7 May 2023 17:34:42 +0000 (13:34 -0400)]
LU-7495 utils: add --links option for lfs find
This adds a "--links" option for lfs find to filter files and
directories by the number of hard links. It also adds a printf format
'%n' to print the number of links for a file.
This commit also fixes '-l' as a short option for '--lazy' which was
added in
11aa7f8704c490b011f60f234c3ac9929ce76948 but the short option
did not work.
Lustre-change: https://review.whamcloud.com/50886
Lustre-commit:
f759d6386d5d0edb95d683d97ca8d84c80080c1c
Signed-off-by: Thomas Bertschinger <bertschinger@lanl.gov>
Change-Id: I5d15bc290df8e8a08402f8d5cfa0a7139791b0a4
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Anjus George <georgea@ornl.gov>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51327
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Thu, 4 May 2023 23:10:50 +0000 (01:10 +0200)]
LU-16799 tests: fix sanity-krb5
sanity-krb5.sh needs to be fixed in several ways.
It cannot assume that the Kerberos credentials cache is FILE, and
expect ccache files to be under /tmp/krb5cc_*.
The lsvcgssd daemon must be launched with -vvv flags for easier
debugging.
Keyring needs to be cleared appropriately after using 'lfs flushctx'.
Lustre-change: https://review.whamcloud.com/50864
Lustre-commit:
f8f8b3c574e95cb7272310bba19f97fe68cd9b11
Test-Parameters: trivial testlist=sanity-krb5 kerberos=true
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I31ca8d2d97e137c7ba9fa478d5432aeedb5135a8
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51265
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Mon, 28 Dec 2020 20:56:12 +0000 (15:56 -0500)]
LU-12514 llite: move client mounting from obdclass to llite
Mounting a lustre client is currently handled
in obdclass, using services from llite.
This requires obdclass to load the llite module
and set up inter-module linkage.
The purpose of this was for common code to support both
client and server mounts. This isn't really a good idea
and need to go. For lustre servers we already use a
separate filesystem type.
So move the mounting code from obdclass/obd_mount to llite/super25
and remove the inter-module linkages.
Add some EXPORT_SYMBOL() so that llite can access some helpers
that remain in obdclass.
Linux-commit:
a989830c88149511ee840356d9c1b34304bac576
Lustre-change: https://review.whamcloud.com/37693
Lustre-commit:
53fa81765750e38f7879ed5092fd729c1bdc8a0f
Change-Id: Ia33bd55a042f90b178156c745a8072b516f00568
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51315
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Thu, 15 Jun 2023 07:45:08 +0000 (10:45 +0300)]
LU-16425 tests: skip interop recovery-small/144a/144b
Skip recovery-small test_144a and test_144b for old MDS
missing the fix and for its corresponding test.
Lustre-change: https://review.whamcloud.com/49679
Lustre-commit:
64faf832a6128cc55c0f3ffa0595d9715d3bdd25
Fixes:
240938f7b1 ("LU-8367 tests: cleanup_orphans hang reproducer")
Fixes:
aa6250b741 ("LU-15724 tests: MDT failover hang reproducer")
Test-Parameters: trivial testlist=recovery-small env=ONLY=144 serverversion=2.14.0
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I77bfdf55d0218aa9e252f742cc90f1c61216d506
Reviewed-by: Sarah Liu <sarah@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Etienne AUJAMES [Mon, 19 Sep 2022 10:23:47 +0000 (12:23 +0200)]
LU-16167 obdclass: fix lctl llog_print with skipped records
The 2a5b50d ignores the skipped records in configuration llog.
But if ioctl OBD_IOC_LLOG_PRINT return 0 record to display,
jt_llog_print_iter() will stop the processing and ignore the
non-skipped records at the end of the llog.
This patch returns to user space if the last index processed
(by llog_print_cb) is the last of llog file. If true,
jt_llog_print_iter() stops the processing.
Add regression test "conf-sanity test_123ai" for this issue.
Lustre-change: https://review.whamcloud.com/48586
Lustre-commit:
c6da54aa7546440339265c644538d3d109e46bde
Fixes: 2a5b50d ("LU-15142 lctl: fixes for set_param -P and llog_print")
Test-Parameters: testlist=conf-sanity env=ONLY=123ai,SLOW=yes,ONLY_REPEAT=10
Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: I78395268c57555e4fd2a4048ccf5b6132ca2877f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander <alexander.boyko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51316
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Mon, 15 May 2023 20:41:08 +0000 (13:41 -0700)]
LU-16619 build: Ubuntu jammy 5.19 client support
Ubuntu 5.19 kernel removed lsmcontext_init() and changed
security_dentry_init_security to require struct context *
Linux kernel linux-hwe-5.19
LSM: Removed scaffolding function lsmcontext_init
Linux kernel linux-hwe-5.19
LSM: security_dentry_init_security with struct lsmcontext
Lustre-change: https://review.whamcloud.com/50210
Lustre-commit: TBD (from
d7001a6c68c334d15d3daeb932b8456b101153d2)
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ib6479a2cd20df5e565ae6203e05df2afa3f3de31
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51002
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Shaun Tancheff [Wed, 17 May 2023 09:09:01 +0000 (02:09 -0700)]
LU-16327 llite: read_folio, release_folio, filler_t
Linux commit v5.18-rc5-221-gb7446e7cf15f
fs: Remove aop flags parameter from grab_cache_page_write_begin()
flags have been dropped from write_begin() and
grab_cache_page_write_begin()
Linux commit v5.18-rc5-241-g5efe7448a142
fs: Introduce aops->read_folio
Provide a ll_read_folio handler around ll_readpage
Linux commit v5.18-rc5-280-ge9b5b23e957e
fs: Change the type of filler_t
Affects read_cache_page, provides a wrapper for read_cache_page
and wrappers for filler functions
Linux commit v5.18-rc5-282-gfa29000b6b26
fs: Add aops->release_folio
Provide an ll_release_folio function based on ll_releasepage
Lustre-change: https://review.whamcloud.com/49199
Lustre-commit:
133ed0cf6f0c84d2b5b84e1de3ff2c54b1fb902d
Test-Parameters: trivial
HPE-bug-id: LUS-11357
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ibd4ec1133c80cd0eb8400c4cd07b50e421dd35c5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50977
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Shaun Tancheff [Fri, 12 May 2023 08:02:08 +0000 (01:02 -0700)]
LU-13485 kernel: Parallel core configure tests
Transform the compile tests in lustre-core to run in parallel
Lustre-change: https://review.whamcloud.com/38361
Lustre-commit:
a346cf6cf22c285e05177a5a5e70a7d8e5bd9fa8
LU-16586 build: Remove old check for linux_selinux_enabled
LC_SRC_HAS_LINUX_SELINUX_ENABLED is used and not defined, it
should be removed.
LC_PROG_LINUX_SRC and LC_PROG_LINUX_RESULTS are defined twice,
remove the unused ones.
Lustre-change: https://review.whamcloud.com/50111
Lustre-commit:
2bdff20f97707cf05118e822adf202ccb209e3a5
Fixes:
a346cf6cf22 ("LU-13485 kernel: Parallel core configure tests")
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I46fa852659eb4db86a12ec4ad3efddd0fdd3a655
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50976
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Mon, 17 Apr 2023 11:13:03 +0000 (21:13 +1000)]
LU-16745 mdc: md_open_data should keep ref on close_req
md_open_data should keep a ref on mod_close_req,
otherwise the mod_close_req could be freed before
we try to access mod_close_req via md_open_data.
Lustre-change: https://review.whamcloud.com/50656
Lustre-commit:
ca716f763f89af192ab86678ee9d14f49c80cae6
Change-Id: I621f7db389854326db298d99957a0bce43024b6e
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51198
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Wed, 19 Apr 2023 07:20:33 +0000 (10:20 +0300)]
LU-15123 tests: check quota reintegration after recovery
4th step of quota reintegration (reconciliation) waits for recovery
completion. So the tests (like sanity-quota/7a) should wait for
recovery completion before checking reintegration results.
Lustre-change: https://review.whamcloud.com/50688
Lustre-commit:
4432b6e2824775e292f96e202d6fc0db231bc749
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Id0aa5db01658621103d94ad6dafe91b2960b3a33
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51232
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Alexey Lyashkov [Fri, 29 Jan 2021 09:29:27 +0000 (12:29 +0300)]
LU-14277 lod: statfs should not block a create
lod_qos_statfs_update() need a guarantee that targets
isn't changed, so it doesn't need to take a QoS mutex.
Lustre-change: https://review.whamcloud.com/41497
Lustre-commit:
6cdb08a0c598b20a4b0ce27f7fe4763b0ada3118
HPE-bug-id: LUS-2106
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: Id9f0ea2fa006bee601d05e14b2e515fcf8248249
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51246
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Thu, 18 May 2023 19:20:41 +0000 (12:20 -0700)]
LU-16755 kernel: update RHEL 8.8 [4.18.0-477.10.1.el8_8]
Update RHEL 8.8 kernel to 4.18.0-477.10.1.el8_8.
Lustre-change: https://review.whamcloud.com/51051
Lustre-commit: TBD (from
266266a8874cab2df8e9b5cc909d578dc239a1b1)
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Change-Id: I6d7703512f9c5a8b686f06e94f32f0e51c9b2001
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51054
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Wed, 17 May 2023 00:12:13 +0000 (17:12 -0700)]
LU-16756 kernel: new kernel [RHEL 9.2 5.14.0-284.11.1.el9_2]
This patch makes changes to support new RHEL 9.2 release
for Lustre client.
Lustre-change: https://review.whamcloud.com/50745
Lustre-commit:
dd390cd315f505456e8e8a29e16a0f24b6878e27
Test-Parameters: trivial env=SANITY_EXCEPT=27J clientdistro=el9.2 testlist=sanity
Test-Parameters: trivial env=SANITY_EXCEPT=27J clientdistro=el9.2 serverdistro=el8.7 testlist=sanity
Change-Id: I4886bbf30d6d6a93c4adbfb68871e9d91f5b64de
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50964
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Fri, 12 May 2023 16:45:49 +0000 (09:45 -0700)]
LU-16755 kernel: RHEL 8.8 client and server support
This patch makes changes to support RHEL 8.8 release
with kernel 4.18.0-477.el8 for Lustre client and server.
Lustre-change: https://review.whamcloud.com/50708
Lustre-commit:
fccc55866bc7da0b4908130d3f16bd355ffb7dc6
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Change-Id: Ie47f131e0340a601c8a5d748ecf9b1b73d4baa1f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50963
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Wed, 31 May 2023 05:34:27 +0000 (22:34 -0700)]
LU-16517 build: pass extra configure options to "make debs"
While running "make debs", the configure command in debian/rules
ignores some user defined configure options. This patch fixes
the issue by adding the detection of the extra options into
debian/rules.
Lustre-change: https://review.whamcloud.com/50464
Lustre-commit:
3989529f22f5c54a98e445674b4b3cc443a3af5f
Test-Parameters: trivial clientdistro=ubuntu2204
Change-Id: Ia9db4e05abf33834cb3c853f4f0829dadc8d7400
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51174
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
James Simmons [Wed, 31 May 2023 05:23:00 +0000 (22:23 -0700)]
LU-6142 obdclass: resolve lu_ref checkpatch issues
Fix up all the checkpatch issues reported for the code handling
lu_ref. Also change USE_LU_REF to CONFIG_LUSTRE_DEBUG_LU_REF
which will match what will be upstream.
Lustre-change: https://review.whamcloud.com/44088
Lustre-commit:
5c98de856618f301035c7579e61fffa10cc726cc
Change-Id: I100e2679fc04c97eb67e4d44c4f6a6b530da6fa8
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51173
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr NeilBrown [Wed, 31 May 2023 05:19:28 +0000 (22:19 -0700)]
LU-6142 lustre: change all 'struct seq_operations' to const
All 'struct seq_operations' are now 'static const'.
Lustre-change: https://review.whamcloud.com/39396
Lustre-commit:
c5b90540731f13c37e6ad62c194ebef6725608a7
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia0f6a2863835ec02fb7d283a099eea489f65ab3a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51172
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Tue, 22 Mar 2022 01:12:23 +0000 (12:12 +1100)]
LU-15668 osd-ldiskfs: fix osd_bio_private double free
In osd_do_bio(), if the IO is fragmented and bio_alloc()
fails to allocate new bio, the bio_private still holds
osd_bio_private for last bio and will be double freed
in osd_do_bio() and dio_integrity_complete_routine().
Lustre-change: https://review.whamcloud.com/46879
Lustre-commit:
82d44b71f16bea216e04f7797ed032a783301d66
Test-Parameters: trivial
Change-Id: I42eaf95a85ec99a60359122054efb5beb0fb6104
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51187
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Aurelien Degremont [Wed, 12 Apr 2023 09:44:37 +0000 (11:44 +0200)]
LU-16829 gss: quiet noisy warnings
GSS code has plenty of debugging messages as warnings.
Update the code to make them just debugging messages and
nothing more.
Lustre-change: https://review.whamcloud.com/50613
Lustre-commit:
2247cf23ce55398ea4bebc1f8093275534736a0a
Test-Parameters: trivial
Change-Id: I53c471d758b0309ae10bab000211fa0381cabf87
Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51190
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Thu, 27 Apr 2023 09:38:24 +0000 (17:38 +0800)]
EX-6511 csdc: prefer uncompressed mirror for read
When accessing a mirrored file with both compressed and uncompressed
components, choose uncompressed components for read.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I043b27bd891c039901075a08c76630f8f0f9f182
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50791
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Mon, 17 Apr 2023 08:17:05 +0000 (16:17 +0800)]
EX-7327 utils: add warning for 'lfs setstripe' compression
Add warning message to indicate that file compression is a preview
feature, not for production use.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1d5f9bdb11e33e8dbd65c6048d62e612020af3a3
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Bobi Jam [Mon, 6 Feb 2023 10:36:52 +0000 (18:36 +0800)]
EX-6856 utils: add 'lfs find' support for compressed file
* Add "--comp-flags=[^]compress" to locate file with/without
compressed components.
* Add "--comp-flags=[^]nocompr" to locate file with/without setting
component compress preference.
* Add "[!] --layout=compress" to locate file with/without compressed
components.
* Add "[!] --compress-type=<compress-type>" to locate compressed file
with/without specified compress algorithm.
* Add "[!] --compress-level=[+-]<compress-level>" to locate compressed
file with/without specified compress level.
* Add "[!] --compress-chunk=[+-]<compress-chunk>" to locate compressed
file with/without specified compress chunk in KiB.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic36946738c3463fd862aeca4ee2e2c2ed85eff84
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49930
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Bobi Jam [Tue, 6 Dec 2022 13:49:31 +0000 (21:49 +0800)]
EX-6249 csdc: set compress component for file
* 'lfs setstripe' to support compress component
--compress|-Z <type>[:<level>]
Set component compression algorithm <type> and compress <level>;
--compress-chunk=<size>
Set compress data chunk size in KiB used by the compression
algorithm, the value will be adjusted to power-of-two multiples of
the base 64KiB.
Example:
$ lfs setstripe -Eeof -Z lz4:5 --compress-chunk=512 <file>
* 'lfs getstripe' to show compress component parameters
Display component's compression parameters if possible.
--compress-type|--compr-type
Print only the compress type if possible.
--compress-level|--compr-level
Print only the compress level if possible.
--compress-chunk|--compr-chunk
Print only the compress chunk size in KiB if possible.
Example:
$ lfs getstripe <file>
lcme_compr_type: lz4
lcme_compr_lvl: 5
lcme_compr_chunk_kb: 512
lmm_pattern: raid0,compress
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ife0382469cbc5099e0c6dc96534bb169ddeff61e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49509
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Bobi Jam [Mon, 14 Nov 2022 08:25:05 +0000 (16:25 +0800)]
EX-6245 csdc: define compress component layout
Define a layout containing compress component with a compress
pattern so that old client cannot process it.
Add a client parameter to enable file compression support.
lctl get_param llite.*.enable_compression
lctl set_param llite.*.enable_compression={0,1}
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: If88d446fc7425644ea9484004ffa4e6d2a05f7e6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49508
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Lai Siyao [Fri, 28 Apr 2023 09:22:03 +0000 (05:22 -0400)]
LU-16717 mdt: resume dir migration with bad_type
LFSCK may set hash type to "none,bad_type" upon migration failure,
set it back to "fnv_1a_64,migrating,bad_type,fixed" to allow
migration resumption. fnv_1a_64 is set because it's the default hash
type, and now that we don't know the hash type in the original
migration command, just try with it.
LFSCK just add "bad_type" flag on such directory, so that such
migration can always be resumed in the future.
Add sanity 230z.
Lustre-change: https://review.whamcloud.com/50797
Lustre-commit:
151650e468ab423e831c30d635ea380e0434a122
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I19606aefcb9115e6724843785aea89a1c380e23f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51230
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Lai Siyao [Sun, 23 Apr 2023 08:09:02 +0000 (04:09 -0400)]
LU-16717 mdt: treat unknown hash type as sane type
Directory migration failure may leave directory hash type as
LMV_HASH_TYPE_UNKNOWN|LMV_HASH_FLAG_BAD_TYPE, which should be treated
as sane hash type on existing directories, otherwise such directories
can't be unlinked.
Add sanity 230y.
Lustre-change: https://review.whamcloud.com/50796
Lustre-commit:
05cdb71ba6813570123613993f3cfcf74fc83561
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ieffc0808d1db989d0bf9723f05cddb06f349e208
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51202
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lei Feng [Tue, 6 Jun 2023 11:47:05 +0000 (19:47 +0800)]
EX-7596 tests: disable metadata_csum_seed for pcc cache device
latest e2fsprogs enable metadata_csum_seed feature when mkfs.ext4
a device, which is used in pcc test as a cache device. Such a ext4
fs cannot be mounted on an old kernel of el7. So disable this
feature for pcc cache device if it is detected in sanity-pcc test.
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc clientdistro=el7.9
Change-Id: I2ef3ff27b49fb479ec348b742cf614a43321813b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51234
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Fri, 26 May 2023 10:56:48 +0000 (11:56 +0100)]
RM-620 build: New tag 2.14.0-ddn88
New tag 2.14.0-ddn88
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib22ed479bd8d4f50b327b4287dd4c68cb1530db5
Wei Liu [Thu, 18 May 2023 15:58:17 +0000 (08:58 -0700)]
EX-7506 test: change ALWAYS_EXCEPT with function always_except
Change ALWAYS_EXCEPT with function always_except in replay-single.sh
Test-Parameters: trivial testlist=replay-single
Change-Id: I6fb4c350b1aa6f8c38920bf52e8ae42b64bdcd64
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51086
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Tue, 23 May 2023 03:14:25 +0000 (11:14 +0800)]
LU-16842 fsx: tolerate delete last non-stale mirror error
fsx mirror split test could try to delete the last non-stale mirror
of a file and that's a tolerable error scenario. The fsx FLR test
randomly choose a mirror operation and this situation could happen.
Lustre-change: https://review.whamcloud.com/51090
Lustre-commit: TBD (from
9bc22dcefbe3a7ac66a3e8f2ad042f1e018d45a9)
Fixes:
04ab0cc869c (LU-14156 utils: mirror split to check for last in-sync early)
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I80c294da80740b21e00ae72a092fd8883ec7d60e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51092
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Thu, 9 Mar 2023 23:00:46 +0000 (15:00 -0800)]
LU-14668 tests: verify state of peer added with '--lock_prim'
Add peer state verification to sanity-lnet test_26:
check that peer state has corresponding bit set for a peer
created with '--lock_prim' option.
Lustre-change: https://review.whamcloud.com/50249
Lustre-commit:
9b6fcfa334b153e52caec16d4cfd180306826a3a
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
54a28697c ("LU-14668 lnet: add 'force' option to lnetctl peer del")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Id5fde036907f9dd19a21e8e6611a070321310f0e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50972
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Tue, 28 Feb 2023 23:02:20 +0000 (15:02 -0800)]
LU-14668 lnet: add 'lock_prim_nid" lnet module parameter
Add 'lock_prim_nid' lnet module parameter to allow control
of how Lustre peer primary NID is selected.
If set to 1 (default), the NID specified by Lustre when
calling LNet API is designated as primary for the peer,
allowing for non-blocking discovery in the background.
If set to 0, peer discovery is blocking until complete
and the NID listed first in discovery response is designated
as primary.
Lustre-change: https://review.whamcloud.com/50159
Lustre-commit:
fc7a0d6013b46ebc17cdfdccc04a5d1d92c6af24
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I6ed1cb0c637f4aa7a7340a6f01819ba9a85858f4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50971
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Mon, 27 Feb 2023 23:41:19 +0000 (15:41 -0800)]
LU-14668 lnet: add 'force' option to lnetctl peer del
Add --force option to 'lnetctl peer del' command.
If the peer has primary NID locked, this option allows
for the peer to be deleted manually:
lnetctl peer del --prim_nid <nid> --force
Add --prim_lock option to 'lnetctl peer add' command.
If specified, the primary NID of the peer is locked
such that it is going to be the NID used to identify
the peer in communications with Lustre layer.
Lustre-change: https://review.whamcloud.com/50149
Lustre-commit:
f1b2d8d60c593a670b36006bcf9b040549d8c13a
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ia6001856cfbce7b0c3288cff9b244b569d259647
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50970
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Oleg Drokin [Tue, 4 Jun 2019 04:41:40 +0000 (00:41 -0400)]
LU-11623 mdt: return UPDATE|PERM on open
This patch includes the following changes:
* try lock UPDATE|PERM ibits on open to speed up subsequent stat.
* open returns PR lock to client by default, because CR mode
UPDATE|PERM ibits don't make sense since all modifications take
PW lock.
* don't lock UPDATE|PERM ibits for PCC attach, otherwise these ibits
revoke will cause file detach.
* update sanity-pcc 13a to make it fail on single client test if
anything went wrong.
* update sanity-lfsck 31d because previously CR UPDATE lock is
fetched, thus the test pass by mistake.
This should help common workloads with open followed by a stat
or other such operation.
Benchmark results:
This patch can significantly improve open-create + stat on the same
client.
This patch in combination with two others:
https://review.whamcloud.com/32157
https://review.whamcloud.com/33584
Improves the 'stat' side of open-create + stat by >10x.
Without patches (master branch commit 26a7abe):
mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d
/cache1/out/ -F -C -T -v -w 32k
Operation Max Min Mean
--------- --- --- ----
File creation : 3838.205 3838.204 3838.204
File stat : 33459.289 33459.249 33459.271
File read : 0.000 0.000 0.000
File removal : 0.000 0.000 0.000
Tree creation : 3146.841 3146.841 3146.841
Tree removal : 0.000 0.000 0.000
With the three patches:
mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d
/cache1/out/ -F -C -T -v -w 32k
SUMMARY rate: (of 1 iterations)
Operation Max Min Mean
--------- --- --- ----
File creation : 3822.440 3822.439 3822.440
File stat : 350620.140 350615.980 350617.193
File read : 0.000 0.000 0.000
File removal : 0.000 0.000 0.000
Tree creation : 2076.727 2076.727 2076.727
Tree removal : 0.000 0.000 0.000
Note 33K stats/second vs 350K stats/second.
ls -l time of the mdtest directory is also reduced from 23.5 seconds
to 5.8 seconds.
Lustre-change: https://review.whamcloud.com/33585
Lustre-commit:
4b44204930cf4d312425dd95e51720fe5d3e801b
Change-Id: Ib3410629c190de6f74246a4a92f8216537fa2b95
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Qian Yingjiin <qian@ddn.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51071
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Stephane Thiell [Tue, 9 Feb 2021 06:47:31 +0000 (22:47 -0800)]
LU-7668 utils: add lctl del_ost
Add helper command:
lctl del_ost [--dryrun] --target fsname-OSTxxxx
Permanently disable an OST by altering the client and MDT llog
catalogs on MGS. The command finds all catalog records related to the
specified OST and cancel them. A --dryrun option is provided so that
the system administrator can see which records would have been
cancelled, but without actually cancelling them.
Lustre-change: https://review.whamcloud.com/41449
Lustre-commit:
1121816c4a4e1bb2ef097c4a9802362181c43800
LU-7668 tests: skip conf-sanity test_33a for old MGS
Skip del_ost test for MGS versions that do not have this command.
Lustre-change: https://review.whamcloud.com/50221
Lustre-commit:
ba5346b050fb395844252a706d4dba2ef0e0d8dc
Signed-off-by: Stephane Thiell <sthiell@stanford.edu>
Change-Id: I58c4f10fa0f7164a40231e807698eb224cccf062
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50923
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Thu, 17 Nov 2022 07:31:45 +0000 (08:31 +0100)]
EX-6275 lustre: add lz4 and lz4hc kernel modules
lz4 and lz4hc kernel modules implement compression according to the
lz4 and lz4hc algorithms respectively, through the kernel Crypto API.
lz4 module provides 2 cipher drivers under the generic name 'lz4':
* lz4-lustre-generic of type compression
* lz4-lustre-scomp of type scomp
lz4hc module provides 2 cipher drivers under the generic name 'lz4hc':
* lz4hc-lustre-generic of type compression
* lz4hc-lustre-scomp of type scomp
lz4 and lz4hc kernel module sources are copied from linux v6.1-rc5,
and renamed to llz4.c and llz4hc.c respectively to avoid name
collisions. Use of vmalloc has been changed to kvmalloc since it is
faster in most cases. They implement the Crypto API interface, and
rely on the lz4/lz4hc kernel library for compression implementation.
They have been modified to grok a compression acceleration/level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The lz4/lz4hc library sources are also copied from linux v6.1-rc5 and
built statically, so lz4_compress, lz4_decompress and lz4hc_compress
sources have been "de-modulified", and EXPORT_SYMBOLs removed. Headers
have also been copied from linux v6.1-rc5 for consistency, and source
files modified to include the copied headers instead of the system
headers.
All aforementioned sources are located in the lustre/lz4 directory.
The lz4/lz4hc modules are built only if the kernel does not provide
them. This is verified by checking if the CONFIG_CRYPTO_LZ4 and
CONFIG_CRYPTO_LZ4HC kernel config options are defined.
This patch provides unit testing of several compression modules, such
as lz4, lz4hc, lzo and gzip. This is done via a new test kernel module
kcompr.ko, and new sanity test_84. File lustre/tests/kernel/kcompr.c
contains examples of how to call the compression/decompression
routines.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I74ce95ff18194c6052d291588d7e8c79596a7f23
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49208
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Deiter [Mon, 7 Nov 2022 17:47:21 +0000 (21:47 +0400)]
LU-14294 tests: fixed NFS configuration issue
* Used the systemctl command to manage system services
* Used the same order of parameters to setup and cleanup NFS
* Used tab for indentation
Lustre-change: https://review.whamcloud.com/49062
Lustre-commit:
1a8fe55b17ac2bc2195aaba446467ccdac67b564
Test-Parameters: trivial clientdistro=el7.9 \
testlist=parallel-scale-nfsv3,parallel-scale-nfsv4
Test-Parameters: clientdistro=el8.7 \
testlist=parallel-scale-nfsv3,parallel-scale-nfsv4
Test-Parameters: clientdistro=el9.0 \
testlist=parallel-scale-nfsv3,parallel-scale-nfsv4
Test-Parameters: clientdistro=sles15sp4 \
testlist=parallel-scale-nfsv3,parallel-scale-nfsv4
Test-Parameters: clientdistro=ubuntu2004 \
testlist=parallel-scale-nfsv3,parallel-scale-nfsv4
Test-Parameters: clientdistro=ubuntu2204 \
testlist=parallel-scale-nfsv3,parallel-scale-nfsv4
Change-Id: I6b087035ac7524aa99c0facad48f8c3fb7444cbc
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50889
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Dongyang [Tue, 23 Nov 2021 23:45:48 +0000 (10:45 +1100)]
LU-11912 fid: clean up OBIF_MAX_OID and IDIF_MAX_OID
Define the OBIF|IDIF_MAX_OID macros to 1ULL << OBIF|IDIF_MAX_BITS - 1
Clean up the callers and remove OBIF|IDIF_OID_MASK which are not used.
Lustre-change: https://review.whamcloud.com/45659
Lustre-commit:
bb2f0dac868cf1321277bc3d7d6fc71f016d921b
Test-Parameters: trivial
Change-Id: I9a679b930c73da5904b2eb4c74f785fc1d27a8a0
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50759
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Tue, 22 Mar 2022 13:08:35 +0000 (08:08 -0500)]
LU-15670 clio: Disable lockless for DIO with O_APPEND
Lockless O_DIRECT with O_APPEND can allow interleaved / racy
appends from concurrent I/O.
Disable lockless I/O when O_APPEND is set
Lustre-change: https://review.whamcloud.com/46890
Lustre-commit:
649d638467c0375797cd59ab7c9ac4113e6c682e
HPE-bug-id: LUS-9776
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I5c56f92c90e631c295f56e5958985f516e1990f8
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49666
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lei Feng [Fri, 19 May 2023 05:01:06 +0000 (13:01 +0800)]
EX-7544 lipe: set lpcc interval to 30 by default
set the default value of interval lpcc.conf and lpcc_purge
to 30 seconds everywhere.
Test-Parameters: trivial testlist=sanity-pcc
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Iaee812bdc0d3c4d04549092e5c8aa78b0a2f1dbd
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51058
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Tue, 22 Nov 2022 05:40:03 +0000 (22:40 -0700)]
LU-16331 utils: fix 'lfs find -O <uuid>' with gaps
Fix the UUID parsing in llapi_get_target_uuids() so that the OST
UUIDs are put into the right slots when there is a gap in numbering.
Otherwise, "lfs find -O <uuid>" will not be able to find the given
UUID if it is the first OST after the gap.
Add test case for 'lfs find' and 'lfs getstripe' with large/sparse
OST indices in conf-sanity test_81.
Lustre-change: https://review.whamcloud.com/49207
Lustre-commit:
05334b90a5d3ddd6c8eabc3683fd487f47df6e35
Test-Parameters: trivial testlist=conf-sanity env=ONLY="81-82"
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia0581f85f016c202514148114924509118a0f792
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51076
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Charlie Olmstead [Tue, 23 May 2023 15:18:53 +0000 (09:18 -0600)]
EX-7553 - reduce b_es6_0 failover-part-3 duration
Reduced recovery-mds-scale duration to 12 hours
Test-Parameters: trivial
Test-Parameters: austeroptions="-R" clientcount=4 clientdistro=el7.9 \
env=SLOW=yes failover=true iscsi=1 mdscount=2 \
mdssizegb=10 osscount=2 ostsizegb=2 serverdistro=el7.9 \
testgroup=failover-part-3
Change-Id: I791044863b5c52f44a0db36e816581b9d4655287
Signed-off-by: Charlie Olmstead <charlie@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51099
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Tue, 25 Jan 2022 00:53:33 +0000 (11:53 +1100)]
LU-14692 tests: allow FID_SEQ_NORMAL for MDT0000
Fix the tests asssuming objects created for MDT0000
always have a seq number of 0, to prepare for
deprecating IDIF sequence.
Fix sanity test_312 on ZFS to properly identify which
OST the object was created on, but do not re-enable it.
Lustre-change: https://review.whamcloud.com/46293
Lustre-commit:
eaae4655567b16260237764dadb7ab57df8b0edd
Lustre-change: https://review.whamcloud.com/49720
Lustre-commit:
8767d2e44110fc19e624e963d5ebc788409339d3
Test-Parameters: trivial
Test-Parameters: testlist=sanity-scrub env=ONLY=19
Test-Parameters: testlist=sanity-sec env=ONLY=37
Change-Id: I4bffabe25a6f84cdba760aabea1da3429715a283
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50756
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alexander Boyko [Thu, 2 Dec 2021 09:43:54 +0000 (04:43 -0500)]
LU-15252 mdt: reduce contention at mdt_lsom_update
mot_som_mutex serialize all close requests with lsom updates for
a same mdt_object. For a massive open/read/close single shared
file load, it leads to high load avarage cause many threads sleep
on mutex.
This patch introduces a cached lsom size, and uses a mutex at update
part only. Close requests with lsom size less or equal to cached size
would not take a mutex at all.
Test results MPI open/flock/funlock/close SSF
10 iterations 10 node 100 thread each, 1000 file ops per thread
close time secs master patch MDT load avarage master patch
avg 0.142 0.086 47.05 38.89
max 0.164 0.129 49.39 44.77
min 0.097 0.041 44.44 34.7
Lustre-change: https://review.whamcloud.com/45709
Lustre-commit:
c8b7afe4970415f8dae84f5e20661f8a3b3681a0
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I807b468b128295df9391b0467e74d4f10240662e
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51030
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Dongyang [Wed, 10 May 2023 00:18:20 +0000 (10:18 +1000)]
EX-6841 tests: do not truncate over s_maxbytes
sanity-lipe-scan3/104 truncates file to MAX_LFS_FILESIZE
but it should not go over s_maxbytes.
Fixes:
c05dbbbbca ("EX-4015 lipe: add lipe_scan3")
Test-Parameters: trivial testlist=sanity-lipe-scan3 env=ONLY=104
Change-Id: I4a14befb0dfc34b2611052850b77a0841ff853aa
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50899
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Hongchao Zhang [Tue, 18 Apr 2023 18:58:26 +0000 (02:58 +0800)]
LU-16529 test: verify the grant > 0 before test
In the test 84 in sanity-quota, the grant should be larger than 0
after the dd complete, this patch adds check for it and forces the
quota to be re-integrated during the check.
Lustre-change: https://review.whamcloud.com/50799
Lustre-commit:
fdffbd3d7842a2f99e5fcdcb5b6e6766949f6333
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I9cddcf0c4afc12a11f3535792262ebb35a1e480e
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51094
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Etienne AUJAMES [Tue, 28 Mar 2023 19:46:24 +0000 (21:46 +0200)]
LU-16674 obdclass: optimize job_stats reads
This patch has 2 objectives:
1/ limit the lock time on ojs_list (list of job stats)
"lctl get_param mdt.*.job_stats" can not dump job_stats in a single
read (seq_file buffer is limited to 4k). So, several reads are needed
to dump the full job list.
For each read, we have to find the job entry corresponding to the file
offset. For now, we walk ojs_list from the beginning to get this
entry.
This patch saved the last known entry and the corresponding offset to
start the next read from here.
2/ avoid the lock contention when reading job_stats
This patch replaces the read lock on ojs_lock by RCU locking, this
enables userspace processes reading the job_stats not to interfere
with the kernel target threads.
Add the stress test sanity 205g to check for possible races.
Add stack_trap in sanity test 205a and 205e to restore jobid_name and
jobid_var.
* Performance *
The following command is used to capture records:
$ time grep -c job_id /proc/fs/lustre/mdt/lustrefs-MDT0000/job_stats
- job_stats dump with no fs activity
Here are results after ending sanity test 205g with slow mode and
job_cleanup_interval=300s.
___________________________________
| nbr of job | time | rate |
_____________|____________|______|_______________|
|without patch| 14749 | 1.3s | 11345 jobid/s |
|_____________|____________|______|_______________|
|with patch | 22209 | 0.6s | 37015 jobid/s |
|_____________|____________|______|_______________|
|diff % | +43% | -54% | +226% |
|_____________|____________|______|_______________|
- job_stats dump with fs activity
Here are results before ending sanity test 205g with slow mode and
job_cleanup_interval=300s.
___________________________________
| nbr of job | time | rate |
_____________|____________|______|_______________|
|without patch| 14849 | 2.3s | 6428 jobid/s |
|_____________|____________|______|_______________|
|with patch | 22776 | 1.2s | 18823 jobid/s |
|_____________|____________|______|_______________|
|diff % | +53% | -47% | +192% |
|_____________|____________|______|_______________|
Lustre-change: https://review.whamcloud.com/50459
Lustre-commit:
c6890a955f89508db46fd8ffbf22b05b145976cd
Test-Parameters: testlist=sanity env=SLOW=yes,ONLY=205g,ONLY_REPEAT=10
Test-Parameters: testlist=sanity env=ONLY=205g serverversion=2.14.0
Test-Parameters: testlist=sanity env=SLOW=yes,ONLY=205
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: Ic4cd90965720af76eff0ed4e00ca897518bfbc66
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51110
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Dongyang [Wed, 5 Apr 2023 02:54:13 +0000 (12:54 +1000)]
LU-16712 cksum: fix generating T10PI guard tags for partialbrw
To get better performance, we allocate a page as the buffer for
T10PI guard tags, we fill the buffer going over every page for brw,
when the buffer is considered full, we use
cfs_crypto_hash_update_page() to update the hash and reuse the buffer.
It doesn't work when there's a page in the brw gets clipped, and the
checksum sector size is 512. For a page with PAGE_SIZE of 4096,
and offset at 1024, we will end up with 6 guard tags, and won't have
enough space in the very end of the buffer for a full brw page, which
needs 8.
Work out the number of guard tags for each page, update the
checksum hash and reuse the buffer when needed.
Lustre-change: https://review.whamcloud.com/50540
Lustre-commit:
3999627447c01eebd96c14cc5cf8bba93f89a66b
Change-Id: Ic591e63b24534f2a42b670669520895cb35a9546
Fixes:
b1e7be00cb ("LU-10472 osc: add T10PI support for RPC checksum")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51079
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Fri, 12 May 2023 19:10:04 +0000 (12:10 -0700)]
LU-12353 ldiskfs: add ext4-dquot-commit-speedup patch to more series
Add ext4-dquot-commit-speedup.patch to RHEL 8.x ldiskfs patch series.
Lustre-change: https://review.whamcloud.com/50853
Lustre-commit: TBD (from
06da805983c298f0957decfdb1d08cf7c39fd99b)
Test-Parameters: trivial clientdistro=el8.7 serverdistro=el8.7 testlist=sanity
Change-Id: Ib0ac325bde442b4eafedde9ba44984b02d5ea061
Fixes:
dad25f258e50 ("LU-12353 ldiskfs: speedup quota journalling")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50984
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 13 May 2023 00:01:58 +0000 (18:01 -0600)]
RM-620 build: New tag 2.14.0-ddn87
New tag 2.14.0-ddn87
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6450c71a5e3ada3e0f63670867420a57c88bde5d
Patrick Farrell [Fri, 12 May 2023 14:22:58 +0000 (10:22 -0400)]
LU-16649 llite: EIO is possible on a race with page reclaim
We must clear the 'uptodate' page flag when we delete a
page from Lustre, or stale reads can occur. However,
generic_file_buffered_read requires any pages returned from
readpage() be uptodate.
So, we must retry reading if page truncation happens in
parallel with the read.
This implements the same fix as:
https://review.whamcloud.com/49647
b4da788a819f82d35b685d6ee7f02809c05ca005
did for the mmap path.
Lustre-change: https://review.whamcloud.com/50344
Lustre-commit:
1d98e5c32b41e19bb1247958e666bb66e69dbc4c
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iae0d1eb343f25a0176135347e54c309056c2613a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50346
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Patrick Farrell [Fri, 12 May 2023 14:22:09 +0000 (10:22 -0400)]
Revert "LU-14541 llite: Check vmpage in releasepage"
This reverts commit
c524079f4f59a39b99467d9868ee4aafdcf033e9,
because it breaks releasepage for Lustre and does not
completely fix the data consistency issue in LU-14541.
Breaking releasepage matters because it prevents direct I/O
from working if there is page cache data present, and
because it causes similar issues with GDS, which must be
able to flush page cache pages before doing I/O.
With patches:
"LU-16160 llite: SIGBUS is possible on a race with page reclaim"/
d9c23a7934747eb19e23470b30806482a1aa60f8
and
"LU-14541 llite: Check for page deletion after fault"/
19678e30147f50f813e72e8216cfb0453fe0ca6e
LU-14541 is fully resolved, so we can revert this patch.
Lustre-change: https://review.whamcloud.com/49654
Lustre-commit:
e3cfb688ed7116a57b2c7f89a3e4f28291a0b69f
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I613bdb4f27161ffc3638d1d8ea38827af5a7bd47
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50304
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Qian Yingjin [Wed, 26 Apr 2023 09:18:18 +0000 (05:18 -0400)]
EX-7395 pcc: use llapi_open_by_fid to check pinned files
When check whether a file was pinned in PCC backend, it reported:
"cannot read or parse pin xattr of file
'/lustre/fsr/.lustre/fid/[0x780001b83:0x2138:0x0]'.: No such file
or directory (2)"
The failure reason is that open by FID is not configured for
subdirectory mounts.
In this patch, we use llapi_open_by_fid (which supports for
subdirectory mount) to open the file to solve this error.
Change-Id: If0120d745418836cfdf449a795b6f524c40f9c27
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50770
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Mon, 17 Apr 2023 08:34:12 +0000 (02:34 -0600)]
LU-11170 tests: don't fail sanity/415 in VM
Don't fail sanity test_415 when running in a VM due to variable
runtimes for the tests.
A proper solution would be to examine the logs to determine if
the renames are blocked or just all slow due to VM contention.
Lustre-change: https://review.whamcloud.com/50654
Lustre-commit:
73a7b1c2a3f0114618db7781adb56974ed682f24
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1a5d0f601705c9ec8559e760c4ec27c7f83ebbe5
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Vikentsi Lapa <vlapa@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50938
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Thu, 20 Apr 2023 22:13:54 +0000 (16:13 -0600)]
LU-10733 tests: increase conf-sanity/106 OST size
conf-sanity test_106 is trying to create ~64k files, but OST0000
only has about 48k objects in this case, so the file creates are
failing during the test. This makes the test somewhat unreliable
and hitting errors not related to what was originally intended
(llog wrap handling).
Increase the OSTSIZE for this test to handle the number of objects
needed by the test so it can run more reliably.
Lustre-change: https://review.whamcloud.com/50732
Lustre-commit:
334d780617561c66c91697fb1681ce24b5379387
Test-Parameters: trivial testlist=conf-sanity env=ONLY=106
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie33825801172ea565d9d1d5fb81595d2cad65677
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50714
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Thu, 6 Apr 2023 07:00:16 +0000 (00:00 -0700)]
LU-16650 kernel: update RHEL 7.9 [3.10.0-1160.88.1.el7]
Update RHEL 7.9 kernel to 3.10.0-1160.88.1.el7.
Lustre-change: https://review.whamcloud.com/50553
Lustre-commit:
bd0d79456b91db58a75eeb717c7805d78d8a9a1a
Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9
Change-Id: I4119595943940cca94d1853b59c94a02fed8cb71
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50556
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Wed, 26 Apr 2023 09:26:29 +0000 (12:26 +0300)]
LU-14699 tests: fix server version to check
tests 160g and 160s check Lustre version wrongly, patch
corrects these checks to use correct version
Test-Parameters: trivial
Fixes:
735136ead955 (LU-14699 mdd: proactive changelog garbage collection)
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ib6a78a55672629686b59f0227e81d78f28bb81ac
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50957
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log