Whamcloud - gitweb
Alex Zhuravlev [Wed, 17 Jun 2020 14:05:28 +0000 (17:05 +0300)]
LU-13668 mdt: change lock mode for lease
make it PW so that lfs getstripe and open-for-read do not
interrupt replication.
Lustre-change: https://review.whamcloud.com/38964
Lustre-commit: TBD (from
a1738536459d9ea87b4d89e54cde624487fbc53d)
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I20f4bbbc4e7bf9055333aba1b8cca80aa899c664
Reviewed-on: https://review.whamcloud.com/41840
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Wed, 3 Mar 2021 01:41:12 +0000 (17:41 -0800)]
LU-14395 kernel: kernel update RHEL7.9 [3.10.0-1160.15.2.el7]
Update RHEL7.9 kernel to 3.10.0-1160.15.2.el7.
Change debuginfo download location since debuginfo.centos.org
does not provide kernel-debuginfo-common anymore.
The patch also reverts the following fix from RHEL 7.9 kernel
since version 3.10.0-1160.8.1.el7:
- [kernel] timer: Fix lockup in __run_timers() caused by
large jiffies/timer_jiffies delta (Waiman Long) [1849716]
The above fix caused Hard LOCKUP kernel panic.
Test-Parameters: clientdistro=el7.9 serverdistro=el7.9
Change-Id: Icdd9e8bf4bd595dece266f6c5a9b0de344781a93
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41902
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Tue, 2 Mar 2021 07:11:55 +0000 (18:11 +1100)]
EX-2616 kernel: add missing kernel patches back to rhel7.9
The patches were added for rhel7.7 but we still need them
for rhel7.9
Test-Parameters: serverdistro=el7.9
Change-Id: If84e08220e984019dbc71ea47c1202db7e5e70ac
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41912
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
John L. Hammond [Fri, 16 Oct 2020 21:36:53 +0000 (16:36 -0500)]
EX-1951 osc: workaround osc aio crash
This this avoids an aio related use after free on the object
associated to the page in osc_page_delete().
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: If2ad7d673bb2cce364982544f097d57ca28ccbe9
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41843
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sun, 28 Feb 2021 23:51:14 +0000 (15:51 -0800)]
LU-14180 utils: verify setstripe comp_end is valid
Verify that the "lfs setstripe -E <component_end>" value is valid.
Otherwise, if "-S" is not specified at the same time, then an
invalid file layout can be created and the file cannot be deleted
normally, only via "lfs rmdif <FID>".
Allow values < 4096 (e.g. '64' or '128' which would all be invalid
anyway) to be interpreted as KiB units.
Update usage messages and man pages to match.
Lustre-change: https://review.whamcloud.com/41239
Lustre-commit:
83e38bba6237f838c9a5d7d36b258cf6dd28bd13
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I47fe7729ffd447c1c1cc098e5117e456263ebbe5
Reviewed-on: https://review.whamcloud.com/41790
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 4 Apr 2018 01:15:27 +0000 (09:15 +0800)]
RM-844 lbuild: add support for building kmod-lustre-o2ib-ofed
With this patch, we will be able to build ofed
and mlnx lustre RPMS at the same time.
Introduce extra option --enable-o2ib_kernel
to lbuild enable ofed RPMS, in default, it won't be built.
This is a quick walkaround solution that we could
support both MLNX and OFED KMOD RPMS, for the long
term, we should handle this more gracefully
Test-Parameters: trivial
Change-Id: I581535d2e13b5d412a9b6e057294eb146f4ef086
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41875
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Hongchao Zhang [Fri, 25 Sep 2020 10:27:04 +0000 (18:27 +0800)]
EX-1428 kernel: contain update related to snapshot
Adding extra fields in "struct jbd2_journal_handle" and
"struct journal_head", which are used by snapshot into the
4-byte hole at the end of struct jbd2_journal_handle so
that they do not increase the structure size and memory
usage for this common allocation.
Test-Parameters: trivial
Change-Id: I6bab8ae1cfcab39632cb1397ffd7991f72e00fb4
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41882
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Sun, 1 Apr 2018 23:11:22 +0000 (07:11 +0800)]
RM-844 lbuild: add support for building kmod-lustre-o2ib-mlnx RPM
This patch try to add a new kmod RPM which is based
on MLNX_OFA Driver.
Test-Parameters: trivial
Change-Id: Ibea5b4bd43a62316d77e4b8de735b05dec3ccdf8
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41874
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Wed, 3 Mar 2021 09:26:42 +0000 (17:26 +0800)]
LU-13970 tests: skip sanity test_427 on SLES12
For SLES12 sanity.sh test_427 cannot determine the number of
objects in the lustre_inode_cache slab, so skip this old distro.
Test-Parameters: trivial testlist=sanity env=ONLY=427 clientdistro=sles12sp5
Fixes:
acbc5b5bb2b5 ("LU-13970 llite: add option to disable Lustre inode cache")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia6f2d57969d83d763ed91251dfaaf9929c3ebbe5
Reviewed-on: https://review.whamcloud.com/41851
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mikhail Pershin [Fri, 12 Feb 2021 07:16:24 +0000 (10:16 +0300)]
LU-14430 mdd: fix inheritance of big default ACLs
If the number of default ACLs in directory is more than 31, then
mdd_acl_init() fails to inherit them for a newly created file.
This limitation is caused by using a fixed-size def_acl_buf buffer
in the mdd_create()->mdd_acl_init() call chain. Instead, the
default ACL buffer should be increased when it is needed.
Patch adds check for -ERANGE after mdd_acl_init(), reallocates
default ACL buffer with required size and calls mdd_acl_init()
again. Thus big default ACL are processed as expected.
Lustre-commit:
f3d03bc38a3afdef83635d578ee0b2ffdd985685
Lustre-change: https://review.whamcloud.com/41494
Fixes:
6350af100c20 ("LU-3437 mdd: Fix ACL/def_ACL during object creation")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I700da90c09f824955fcb8dc7ca0bc2f581f916a0
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41861
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Fri, 5 Mar 2021 15:59:24 +0000 (23:59 +0800)]
LU-14375 kernel: kernel update SLES15 SP2 [5.3.18-24.46.1]
Update SLES15 SP2 kernel to 5.3.18-24.46.1 for Lustre client.
Test-Parameters: trivial \
env=SANITY_EXCEPT="100 130 136 817" \
clientdistro=sles15sp2 serverdistro=el7.9 \
testlist=sanity
Change-Id: I45ec236657ea4f54d7a08a8f0af8397cb161c4bf
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41901
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Fri, 5 Mar 2021 15:57:07 +0000 (23:57 +0800)]
LU-14376 kernel: kernel update SLES12 SP5 [4.12.14-122.57.1]
Update SLES12 SP5 kernel to 4.12.14-122.57.1 for Lustre client.
Test-Parameters: trivial clientdistro=sles12sp5 \
env=SANITY_EXCEPT="56oc 817" testlist=sanity
Change-Id: I1ad5feb6f63cbaa948226fcb4248a2a767b67ce3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41900
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Fri, 5 Mar 2021 15:50:30 +0000 (23:50 +0800)]
LU-13659 kernel: kernel update SLES12 SP4 [4.12.14-95.54.1]
Update SLES12 SP4 kernel to 4.12.14-95.54.1 for Lustre client.
Test-Parameters: trivial clientdistro=sles12sp4 \
envdefinitions=LNET_SELFTEST_EXCEPT=smoke,SANITY_EXCEPT="103a 817"
Change-Id: If7b9143bec6d9c526bd65e96a771c83f2530e608
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41898
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Minh Diep [Thu, 11 Feb 2021 05:12:58 +0000 (21:12 -0800)]
EX-2596 build: add ddn tag to build MOFED
We need to add ddn tag to MOFED kmod build
to indicate that kmod-mlnx-ofed_kernel is different
from the stock version
Test-Parameters: trivial
Change-Id: Ibc8b95c228afa921cedbc30d74196754e0f0cc24
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41879
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 20 Jun 2018 04:31:46 +0000 (12:31 +0800)]
RM-849 lbuild: add mlnx fix patches
we might need apply some mlnx patches and maintain
them by ourselves before official release include them.
[Updates: now we might not need mlnx 4.3, but let's keep the ability...]
Change-Id: Id357d581a602153db65c6c00d6475d01d4761c04
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41878
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Gu Zheng [Mon, 4 Sep 2017 10:32:23 +0000 (06:32 -0400)]
RM-790 : fix Kernel string mismatch for ppc64le in RPMs
The macro in the kernel RPMs for ppc64 provided
by Redhat describing the kernel version supported,
mismatch the content of the macro of the Lustre kernel
client modules for ppc64le describing the required
kernel. This leads to unresolved dependency errors
during installation of the Lustre client modules
although the kernel versions match actually.
While Redhat uses the convention:
kernel = <major-number>-<minor-release-number>.el6
the following string is used in Lustre client RPM
for the required kernel version:
kernel = <major-number>-<minor-release-number>.el6.ppc64le
The patch will remove the trailing string '.ppc64le' in
macro 'krequires' defined in the lustre specfile
describing the software dependeny.
AUTOTEST:ONLY:sanity
[Shilong added autotest args]
Change-Id: Id1d9fd92928a567048d5ad6e9d0c872b5666b4e6
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41877
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
John L. Hammond [Wed, 3 Feb 2021 20:19:05 +0000 (14:19 -0600)]
LU-14398 hsm: use llapi_fid2path_at() in the copytool
In lhsmtool_posix.c and liblustreapi_hsm.c, convert several uses of
uses of llapi_fid2path() to llapi_fid2path_at().
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ice64d02010b4260287be4d4e26c6b75b178bc81b
Reviewed-on: https://review.whamcloud.com/41865
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
John L. Hammond [Wed, 3 Feb 2021 20:17:11 +0000 (14:17 -0600)]
LU-14398 lfs: use llapi_fid2path_at() in lfs_fid2path()
Use llapi_fid2path_at() in lfs_fid2path(). This avoids resolving and
opening the mount point for each FID argument passed. Make the -c,
--cur, --current option actually print the link. Add a more
descriptive long option name for this (--print-link). Update the
lfs-fid2path manpgae accordingly.
Lustre-change: https://review.whamcloud.com/41407
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: If851e4ce95f87d3188b644eb4a345ba3cfca530d
Reviewed-on: https://review.whamcloud.com/41864
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
John L. Hammond [Wed, 3 Feb 2021 19:06:16 +0000 (13:06 -0600)]
LU-14398 llapi: add llapi_fid2path_at()
Add llapi_fid2path_at() which works like llapi_fid2path() takes an
open FD on the moint point instead of a 'fsname or dirirectory path'
and a const struct lu_fid * instead of a const char *.
Lustre-commit:
c45558bf560cf43d440af5679b86ba7e4d2542f3
Lustre-change: https://review.whamcloud.com/41406
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I76234bc28de231587b65c5d866954441e0893aac
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/41863
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
John L. Hammond [Wed, 3 Feb 2021 18:33:26 +0000 (12:33 -0600)]
LU-14398 llapi: simplify llapi_fid2path()
Simplify llapi_fid2path(). Remove the fid_is_sane() check. Remove the
call to root_ioctl() and use get_root_path() directly.
Lustre-commit:
4cfe77df6f2499effa1644e6ad5a594abb11be23
Lustre-change: https://review.whamcloud.com/41405
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ib70b8c9e239c77da8b46408de8341fc8aaf4d1c3
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41862
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Tue, 16 Feb 2021 12:40:05 +0000 (23:40 +1100)]
LU-14436 tgt: only use T10PI guard when doing full sector read
The T10PI guard was generated on full sectors, if we
do we partial read and still use the guard, the rpc
checksum won't match.
Lustre-commit:
f44413717eaf5cb938d9c9b2b62d312f064d282a
Lustre-change: https://review.whamcloud.com/41677
Test-Parameters: trivial
Change-Id: I40d481d703a46b9711021a162208b86a956bd8d1
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41860
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Tue, 16 Feb 2021 08:58:25 +0000 (17:58 +0900)]
LU-14435 doc: include lfs-flushctx manpage inside packages
lfs manpage redirects to lfs-flushctx(1), so it has to be
included in the Lustre packages.
Lustre-change: https://review.whamcloud.com/41676
Lustre-commit:
ece23db121d94c8194fada3cf0d0d1f9d9beeed7
Test-Parameters: trivial
Fixes:
c246a9ba04 ("LU-14263 gss: unlink revoked key")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I5c55f8a74eb6dac20fa85b6ea0663ad701341006
Reviewed-on: https://review.whamcloud.com/41859
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Thu, 18 Feb 2021 11:03:31 +0000 (20:03 +0900)]
LU-14444 gss: handle empty reqmsg in sptlrpc_req_ctx_switch
In sptlrpc_req_ctx_switch(), everything is already there to handle
the case of a ptlrpc_request that has an empty rq_reqmsg.
But assertions were left over at the beginning of the function, so
just remove them from here.
Lustre-change: https://review.whamcloud.com/41685
Lustre-commit:
dfe87b089b662663ba125769866c98e803f89a8c
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6ae1f8b9da9600d3b57b9efc9018c2461114f2fe
Reviewed-on: https://review.whamcloud.com/41858
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Fri, 19 Feb 2021 20:50:54 +0000 (23:50 +0300)]
LU-14455 mdt: fix DoM lock prolong logic
- don't stop at the first found lock if it is not PW or EX lock
- add LCK_GROUP lock as valid mode of lock to check
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: If947f8565008953cc34146b6f0ac1e0f0a038bb5
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41857
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Yang Sheng [Wed, 4 Nov 2020 18:36:43 +0000 (02:36 +0800)]
LU-13514 tests: replace nid in conf-sanity test_32
Need replace_nid for test_32a. Else the mdc cannot
be initialzed and prevent client mounting hung.
Lustre-change: https://review.whamcloud.com/40537
Lustre-commit:
327c8b77694bb0796f168df26e0c543d9610691e
Test-Parameters: trivial
Test-Parameters: env=ONLY=32a,ONLY_REPEAT=20 fstype=ldiskfs testlist=conf-sanity
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I651f5728ad4ff96a309ed599490c9dd6ed9c5274
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41856
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Trung Nguyen [Mon, 9 Nov 2020 06:46:44 +0000 (23:46 -0700)]
EX-2010 scsi: requeue aborted commands instead of retry
If the underlying SCSI command returns an abort, rather than retry
it quickly in a loop, which can finish within a few milliseconds,
requeue it with delay so that the hardware has a chance to recover.
The command requeue will take several seconds each time and allows
more chance for the problem to be resolved at the SCSI layer instead
of returning an error to the filesystem and causing server failover.
Test-Parameters: trivial testlist=sanity
Signed-off-by: Trung Nguyen <trunguyen@ddn.com>
Change-Id: Ibdf1b3a52dd0a1b388c7f5f97aa7a51620138845
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-on: https://review.whamcloud.com/41852
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lai Siyao [Fri, 18 Sep 2020 09:53:17 +0000 (17:53 +0800)]
LU-13970 llite: add option to disable Lustre inode cache
A tunable option is added to disable Lustre inode cache:
"llite.*.inode_cache=0" (default =1)
When it's turned off, ll_drop_inode() always returns 1, then the last
iput() will release inode.
Add sanity 427.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I0642bdc694dc365a05395c3fae98131e1e7723c6
Reviewed-on: https://review.whamcloud.com/41850
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Wed, 25 Nov 2020 16:05:05 +0000 (19:05 +0300)]
LU-13397 lfs: mirror resync to keep sparseness
Use SEEK_HOLE/SEEK_DATA in llapi_mirror_resync_many() to
copy just data chunks between components. Holes at the last
component are done with truncate(), holes in other components
are done with fallocate(FALLOC_FL_PUNCH_HOLE). In case of any
punch() error the hole is just copied via read(), i.e. as zeroes
Currently fallocate(FALLOC_FL_PUNCH_HOLE) is not supported yet,
so resync preserves sparseness only for last components
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id249739c5cd2d1c8a998da3341d326de1a8b8d32
Reviewed-on: https://review.whamcloud.com/41849
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Mon, 23 Nov 2020 11:06:12 +0000 (14:06 +0300)]
LU-13397 lfs: mirror extend/copy keeps sparseness
- make ll_lseek() to work under group lock and on designated
mirror
- enhance lfs mirror copy functions migrate_copy_data() and
llapi_mirror_copy_many() with lseek() to find holes and copy
only data chunks.
Both 'migrate' and 'copy' lfs functionality rewrite designated
mirror fully, so holes are not punched in destination file, but
truncate is called first to make sure old data is erased.
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic4a8768b816c921acd7f0adb3311138caac05a7c
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41848
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 17 Oct 2020 07:02:12 +0000 (01:02 -0600)]
DDN-675 virtio: fix high-order kmalloc() failures
virtio_scsi needs high-order (64KB) allocations for large SCSI
requests in atomic context. The __GFP_HIGH mask is intended for
such uses and should not have been removed from the mask.
Add a memory pool for these allocations in case of failure under
fragmentation to allow the IO to complete. The default number
of items in the mempool is 16, but it can be tuned at module load
time via a parameter in /etc/modprobe.d/lustre.conf:
options virtio_ring vring_desc_pool_sz=N
This avoids the need for excessive memory reservation that was
previously set with "vm.min_free_kbytes=1048576" and similar.
The __GFP_NOWARN flag is added to avoid scary stack dumps, and
instead a brief error message is printed periodically so that
it is possible to track whether there are still allocation
issues, but not so verbose as to cause undue alarm.
ll_ost_io01_053: page allocation failure: order:4, mode:0x104000
CPU: 5 PID: 946 Comm: ll_ost_io01_053 3.10.0-862.9.1.el7_lustre.ddn1
Call Trace:
__alloc_pages_nodemask+0x9b4/0xbb0
alloc_pages_current+0x98/0x110
__get_free_pages+0xe/0x40
kmalloc_order_trace+0x2e/0xa0
__kmalloc+0x211/0x230
virtqueue_add+0x1c4/0x4d0 [virtio_ring]
virtqueue_add_sgs+0x87/0xa0 [virtio_ring]
virtscsi_add_cmd+0x17a/0x270 [virtio_scsi]
virtscsi_kick_cmd+0x38/0xa0 [virtio_scsi]
virtscsi_queuecommand+0x15d/0x340 [virtio_scsi]
virtscsi_queuecommand_multi+0x6e/0xe0 [virtio_scsi]
scsi_dispatch_cmd+0xb0/0x240
scsi_queue_rq+0x5a5/0x6f0
blk_mq_dispatch_rq_list+0x96/0x640
blk_mq_do_dispatch_ctx+0xe0/0x160
blk_mq_sched_dispatch_requests+0x138/0x1c0
__blk_mq_run_hw_queue+0xa2/0xb0
__blk_mq_delay_run_hw_queue+0x9d/0xb0
blk_mq_run_hw_queue+0x14/0x20
blk_mq_sched_insert_requests+0x64/0x80
blk_mq_flush_plug_list+0x19c/0x200
blk_flush_plug_list+0xce/0x230
blk_finish_plug+0x14/0x40
osd_do_bio.isra.25+0x651/0x8d0 [osd_ldiskfs]
osd_write_commit+0x3fc/0x8d0 [osd_ldiskfs]
ofd_commitrw_write+0xffe/0x1c90 [ofd]
ofd_commitrw+0x4c9/0xae0 [ofd]
obd_commitrw+0x2f3/0x336 [ptlrpc]
tgt_brw_write+0xffd/0x17d0 [ptlrpc]
tgt_request_handle+0x92a/0x1370 [ptlrpc]
ptlrpc_server_handle_request+0x23b/0xaa0 [ptlrpc]
ptlrpc_main+0xa92/0x1e40 [ptlrpc]
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I35d5ed6d0d83648e1b7f625a4f3c4c8a333ebbe5
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41845
Tested-by: jenkins <devops@whamcloud.com>
Andreas Dilger [Tue, 14 Jul 2020 19:58:06 +0000 (13:58 -0600)]
LU-12029 utils: disable max_sectors_kb autotuning
Disable automatic max_sectors_kb tuning via mount.lustre by default.
This conflicts in EXA with tune_devices.sh that has SFA-specific
tuning parameters from the tune_devices.sh script.
Disable l_tunedisk in /etc/udev/rules.d/99-lustre-server.rules by
default for EXA so that it does not conflict with the EXA script.
Allow shrinking max_sectors_kb if explicitly set via mount option.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I58cf548d08f8680ec5d6ffd00e936a5d903ebbe5
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41839
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 8 Apr 2020 13:04:13 +0000 (21:04 +0800)]
EX-994 kernel: backport a virtio bug fix to lustre kernel
backport following commit to RHEL7 kernel, as the bug
is frequently hit on heavy testing:
scsi: virtio: Reduce BUG if total_sg > virtqueue size to WARN.
If using indirect descriptors, you can make the total_sg as large as you
want. If not, BUG is too serious because the function later returns
-ENOSPC.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Linux-commit:
44ed8089e991a60d614abe0ee4b9057a28b364e4
Change-Id: I237b5cfc4215093346224c8d0ba8a69541bf7694
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38230
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41837
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 1 Nov 2019 08:03:07 +0000 (16:03 +0800)]
EX-687 build: fix to generate lustre version on ubuntu
Problem is dash doesn't support >& operaton and will report
as Syntax error: Bad fd number.
Test-parameters: trivial
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Iafc95756c4981798fe68dd4c51e2d6418335b7dd
Reviewed-on: https://review.whamcloud.com/41831
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Wed, 9 Oct 2019 15:23:21 +0000 (23:23 +0800)]
RM-620: add ddn tags to RPMs
The RPMs should have formats like following:
2.12.2_ddn0_334_g460f956
Change-Id: Ic7dd5290b4e9b439c8513150e3813ddeb23f1dfd
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41830
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 10 Apr 2020 10:25:11 +0000 (18:25 +0800)]
DDN-941 test: relax sanity 450 memory checking
For debugging purpose, it could be helpful to output
memory usage.
Memory checking somehow not working, before figuring it
out, let's only output it rather than error to make
we don't block other patches landing.
Test-Parameters: trivial
Change-Id: I76dcac18824f2fe3b2f8d1a86abe3fd6cba40a3e
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/41836
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 4 Mar 2020 02:34:46 +0000 (10:34 +0800)]
EX-852 ldiskfs: adjust default block allocation threshold
Use default 25% 15% might be too aggressive, especially
with DDN OST will be 1PB soon, let's adjust it to 15% and 10%,
also limit the max space to 15TB,10TB
Change-Id: I472160fc37fa8d119f13084f00bb22ea55c5ca18
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41835
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Fri, 20 Dec 2019 02:21:41 +0000 (10:21 +0800)]
EX-752 kernel: backport two patches for E2EDI
From
40423bfb35bb6057bcefe93b738b5a9411e037a2 Mon Sep 17 00:00:00 2001
From: Jianchao Wang <jianchao.w.wang@oracle.com>
Date: Tue, 12 Feb 2019 09:56:25 +0800
blk-mq: insert rq with DONTPREP to hctx dispatch list
when requeue
When requeue, if RQF_DONTPREP, rq has contained some driver
specific data, so insert it to hctx dispatch list to avoid any
merge. Take scsi as example, here is the trace event log (no
io scheduler, because RQF_STARTED would prevent merging),
kworker/0:1H-339 [000] ...1 2037.209289: block_rq_insert: 8,0 R 4096 () 32768 + 8 [kworker/0:1H]
scsi_inert_test-1987 [000] .... 2037.220465: block_bio_queue: 8,0 R 32776 + 8 [scsi_inert_test]
scsi_inert_test-1987 [000] ...2 2037.220466: block_bio_backmerge: 8,0 R 32776 + 8 [scsi_inert_test]
kworker/0:1H-339 [000] .... 2047.220913: block_rq_issue: 8,0 R 8192 () 32768 + 16 [kworker/0:1H]
scsi_inert_test-1996 [000] ..s1 2047.221007: block_rq_complete: 8,0 R () 32768 + 8 [0]
scsi_inert_test-1996 [000] .Ns1 2047.221045: block_rq_requeue: 8,0 R () 32776 + 8 [0]
kworker/0:1H-339 [000] ...1 2047.221054: block_rq_insert: 8,0 R 4096 () 32776 + 8 [kworker/0:1H]
kworker/0:1H-339 [000] ...1 2047.221056: block_rq_issue: 8,0 R 4096 () 32776 + 8 [kworker/0:1H]
scsi_inert_test-1986 [000] ..s1 2047.221119: block_rq_complete: 8,0 R () 32776 + 8 [0]
(32768 + 8) was requeued by scsi_queue_insert and had RQF_DONTPREP.
Then it was merged with (32776 + 8) and issued. Due to RQF_DONTPREP,
the sdb only contained the part of (32768 + 8), then only that part
was completed. The lucky thing was that scsi_io_completion detected
it and requeued the remaining part. So we didn't get corrupted data.
However, the requeue of (32776 + 8) is not expected.
Suggested-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Jianchao Wang <jianchao.w.wang@oracle.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
[gedwards@ddn.com: s/RQF_DONTPREP/REQ_DONTPREP/]
From
e0c0d06892051e17ee58e041259f339d0c194804 Mon Sep 17 00:00:00 2001
From: "Martin K. Petersen" <martin.petersen@oracle.com>
Date: Fri, 26 Sep 2014 19:20:06 -0400
block: Don't merge requests if integrity flags differ
We'd occasionally merge requests with conflicting integrity flags.
Introduce a merge helper which checks that the requests have compatible
integrity payloads.
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
[gedwards@ddn.com: only compare INTEGRITY bi_flags since we don't have bip_flags yet]
Change-Id: Iece6117afa7b16c86e682e8f44b702ac79ea3609
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/41833
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Zhuravlev [Wed, 16 Oct 2019 14:38:12 +0000 (17:38 +0300)]
EX-603 ioctl: LL_IOC_FID2MDTIDX on server mount point
add LL_IOC_FID2MDTIDX ioctl support on server's mount point.
this way lpurge can lookup MDS# by FID w/o Lustre client, so
lpurge won't need Lustre client on OSTs.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If3c8c96e75573b812688686a331a38250826cd05
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/41832
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 4 Mar 2020 01:35:12 +0000 (09:35 +0800)]
RM-633 ldiskfs: add loadbitmaps to load block bitmaps for rhel7.7
Change-Id: I02bffab5eb809b2d8945562ad9b42f04929df380
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/41829
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Wed, 9 Oct 2019 15:26:07 +0000 (23:26 +0800)]
RM-633 ldiskfs: add loadbitmaps to load block bitmaps
Loading bitmaps help performaces for DDN SFA 4K sector
platform. small read during write will drop whole
throughoutput.
Change-Id: Icc02c5f16c181417ab3833ffd612cf6c1287eeb5
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41828
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Sat, 12 May 2018 06:46:16 +0000 (14:46 +0800)]
lbuild: seperate kernel tag from lustre version
Bumped Kernel tags using different tags like Lustre.
As Lustre changed more frequently than Kernel changes.
With this change, we only bumped DDN kernel tag
if there is really some kernel changes here.
Change-Id: Icdebe62fe1ef5de2096ae0f568494db394dc04f7
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/41827
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Fri, 26 Feb 2021 17:54:17 +0000 (09:54 -0800)]
EX-2674 kernel: add kernel config file for RHEL 8.3 aarch64
This patch adds the kernel config file to support
RHEL 8.3 aarch64 server.
Test-Parameters: trivial \
clientdistro=el8.3 serverdistro=el8.3 \
clientarch=aarch64 serverarch=aarch64 \
testlist=sanity
Change-Id: Id63b4ed0622701bb4fb4ab00e38ecb7e0c6ab7ec
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41780
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Minh Diep [Wed, 27 Jan 2021 19:58:40 +0000 (11:58 -0800)]
EX-2439 build: Add opa-src option to lbuild
Create an option to pass in the opa tarball directly,
since we can not download directly from Intel site
anymore
Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/41334
Lustre-commit:
3a1a94db3c84c94ad99b8437f482aebfeb6b6246
Change-Id: I3fcad6ee7eff35d26ad7659b6148b0df67c48ad0
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Oleg Drokin [Fri, 19 Feb 2021 19:28:17 +0000 (14:28 -0500)]
New release 2.14.0
Change-Id: I2eb99af8fbeaab80b6614e427b77949b1225b406
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 8 Feb 2021 11:56:25 +0000 (04:56 -0700)]
LU-14345 misc: update e2fsprogs to 1.45.6.wc5
Update Changelog to reference new e2fsprogs release.
4aea203f LU-5949 e2fsck: call delete_inode() properly
8725134d LU-5949 e2fsck: simplify inode badness handling
71b74579 LU-14345 e2fsck: fix check of directories over 4GB
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie0845eeed410f6f9f8ef985342fc19d160aa8cb0
Reviewed-on: https://review.whamcloud.com/41433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Sat, 13 Feb 2021 00:52:28 +0000 (19:52 -0500)]
New RC 2.14.0-RC3
Change-Id: I594b5c6d0da7f067bef69fa7a7027374d4434dd8
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Fri, 12 Feb 2021 15:55:42 +0000 (10:55 -0500)]
LU-14424 Revert "LU-9679 osc: simplify osc_extent_find()"
It looks like there are performance regressions atttributed to this patch.
This reverts commit
80e21cce3dd6748fd760786cafe9c26d502fd74f.
Change-Id: I55e0abd50573dd82a9d216f9c3b01483f99c3223
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41498
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Oleg Drokin [Mon, 8 Feb 2021 22:13:56 +0000 (17:13 -0500)]
New release candidate 2.14.0 RC2
Change-Id: Iad3d71e7dcf96173d192717ef4fef3f0dc12b051
Signed-off-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Tue, 26 Jan 2021 01:15:49 +0000 (18:15 -0700)]
LU-13751 tests: remove read of changelog sanity 160j
sanity test 160j tries to read the changelog after one of two
client mounts is unmounted. In this case, we can fail to read
the changelog and get a "Cannot send after transport endpoint
shutdown" error.
The intention of sanity test 160j is to check that
there is no LBUG due to missed obd device. So, do not try to
read from the changelog after file system unmount.
Test-Parameters: trivial testlist=sanity env=ONLY=160j,ONLY_REPEAT=200
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I1746a422b25d546b9aae38ae8438d9c08bce8827
Reviewed-on: https://review.whamcloud.com/41317
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Sebastien Buisson [Mon, 25 Jan 2021 08:24:19 +0000 (17:24 +0900)]
LU-14355 ptlrpc: do not output error when imp_sec is freed
There is a race condition on client reconnect when the import is being
destroyed. Some outstanding client bound requests are being processed
when the imp_sec has already been freed.
Ensure to output the error message in import_sec_validate_get() only
if import is not already in the zombie work queue.
Fixes:
135fea8fa9 ("LU-4423 obdclass: use workqueue for zombie management")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4b431128e04f11b1e3ee7de47090af87538c3558
Reviewed-on: https://review.whamcloud.com/41310
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Fri, 29 Jan 2021 20:51:43 +0000 (04:51 +0800)]
LU-14299 test: sleep to enable quota acquire again
sanity-quota test_61 fails with incorrect quota exceeded
errors because quota acquire will be disabled for 5 seconds
after edquot flag is set. The test should introduce some
delay between the test of over quota and normal one.
Test-Parameters: trivial fstype=zfs testlist=sanity-quota env=ONLY=61,ONLY_REPEAT=20
Fixes:
530881fe4ee20 ("LU-7816 quota: add default quota setting support")
Change-Id: I8040ba960f32cf01cb7cee3a77c06ad4bd732f0e
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41389
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 14 Sep 2020 23:07:17 +0000 (17:07 -0600)]
LU-13449 tests: fix recovery-small test_140b check
The recovery timer is printed in MM:SS format, but the current test
is unhappy if the SS part is printed as "08" or "09" since that is
interpreted by bash as an invalid octal number. Also, the current
check does not handle the case if recovery is longer than a minute.
Change the code to convert MM:SS back to seconds for the comparison.
Test-Parameters: trivial testlist=recovery-small env=ONLY=140
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie1dc77a88bb0e8fd5025f2b5ca57d4a61d3ebbe5
Reviewed-on: https://review.whamcloud.com/39909
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 5 Feb 2021 20:13:10 +0000 (13:13 -0700)]
LU-14316 llite: quiet spurious ioctl warning
Calling "lfs setstripe" prints a suprious warning about using the old
ioctl(LL_IOC_LOV_GETSTRIPE) when that is not actually the case.
Remove the ioctl warning for now and deal with related issues later.
Fixes:
364ec95f3688 ("LU-9367 llite: restore ll_file_getstripe in ll_lov_setstripe")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I20f5a7adb60a30fce27e49827bd46229e2ce7057
Reviewed-on: https://review.whamcloud.com/41427
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Mon, 26 Aug 2019 22:45:40 +0000 (16:45 -0600)]
LU-11607 tests: replace lustre_version/fstype in sanity-scrub
The routine get_lustre_env() is available to all Lustre test
suites and sets an environment variable for the Lustre
version and file system types of servers.
In sanity-scrub, sanity-hsm and sanity-lfsck, replace calls
to lustre_version_code() and facet_fstype() for all server
types with definitions from get_lustre_env().
Clean up around any modifications by removing calls to
return after skip() or skip_env().
Fixes:
c8790ae52393 (LU-1538 tests: standardize test script init - dne-part-2)
Fixes:
c54b6ca2bdb5 (LU-13718 tests: add LU numbers to skipped tests)
Test-Parameters: trivial fstype=ldiskfs testlist=sanity-scrub,sanity-hsm,sanity-lfsck
Test-Parameters: fstype=zfs testlist=sanity-scrub,sanity-hsm,sanity-lfsck
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I5aa2898f6efef127cf2d7f4a2f08838f503c51ab
Reviewed-on: https://review.whamcloud.com/35929
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Vikentsi Lapa <vlapa@whamcloud.com>
Andreas Dilger [Sun, 31 Jan 2021 07:20:47 +0000 (00:20 -0700)]
LU-14389 lov: avoid NULL dereference in cleanup
Running racer concurrently with file migration crashes easily
when the layout changes for a file in an unexpected way:
lov_init_composite() lustre-clilov: DOM entries with different sizes
lov_layout_change() lustre-clilov: cannot apply new layout on
[0x200000402:0x3e6a:0x0] : rc = -22
BUG: unable to handle kernel NULL pointer dereference at 0x00000014
IP: [<
ffffffffa08baef4>] lov_delete_composite+0x104/0x540 [lov]
Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
CPU: 1 PID: 20227 Comm: ln
Avoid the NULL dereference if the entry is not fully initialized
during cleanup.
Test-Parameters: testlist=racer env=DURATION=3600
Fixes:
61a002cd863 ("LU-13602 flr: skip unknown FLR component types")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8fe17f1b49ca2bccc7a285febe47032d023ebbe5
Reviewed-on: https://review.whamcloud.com/41398
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Sat, 9 Jan 2021 10:28:43 +0000 (18:28 +0800)]
LU-13669 llite: make readahead aware of hints
Calling madvise(MADV_SEQUENTIAL) and madvise(MADV_RANDOM) sets the
VM_SEQ_READ and VM_RAND_READ hints in vma->vm_flags. These should
be used to guide the Lustre readahead for better performance.
Disable the kernel readahead for mmap() pages and use the llite
readahead instead. There was also a bug in ll_fault0() that would
set both VM_SEQ_READ and VM_RAND_READ at the same time, which was
confusing the detection of the VM_SEQ_READ case, since VM_RAND_READ
was being checked first.
This changes the readahead for mmap from submitting mostly 4KB RPCs
to a large number of 1MB RPCs for the application profiled:
llite.*.read_ahead_stats before patched
------------------------ ------ -------
hits 2408 135924 samples [pages]
misses 34160 2384 samples [pages]
osc.*.rpc_stats read before read patched
--------------- ------------- --------------
pages per rpc rpcs % cum% rpcs % cum%
1: 6542 95 95 351 55 55
2: 224 3 99 76 12 67
4: 32 0 99 28 4 72
8: 2 0 99 9 1 73
16: 25 0 99 32 5 78
32: 0 0 99 8 1 80
64: 0 0 99 5 0 80
128: 0 0 99 15 2 83
256: 2 0 99 102 16 99
512: 0 0 99 0 0 99
1024: 1 0 100 3 0 100
Readahead hit rate improved from 6% to 98%, and 4KB RPCs dropped from
95% to 55% and 1MB+ RPCs increased from 0% to 16% (79% of all pages).
Add debug to ll_file_mmap(), ll_fault() and ll_fault_io_init() to
allow tracing VMA state functions for future IO optimizations.
Fixes:
62ef9c949753 ("add 2.6.27 kernel support")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I4bbb028db05b21ae01dafe6a7bea398e9b74d8a4
Reviewed-on: https://review.whamcloud.com/41228
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andrew Perepechko [Mon, 9 Dec 2019 17:13:50 +0000 (20:13 +0300)]
LU-10958 ofd: data corruption due to RPC reordering
Without read-only cache, it is possible that a client
resends a BRW RPC, receives a reply from the original
BRW RPC, modifies the same data and sends a new BRW
RPC, however, because of RPC reordering stale data
gets to disk.
Let's use range locking to protect against this race.
Change-Id: I35cbf95594601eacfc5f108b14e4c447962b0bbf
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-5578,LUS-8943
Reviewed-on: https://review.whamcloud.com/32281
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Mon, 1 Feb 2021 19:11:10 +0000 (14:11 -0500)]
New release candidate 2.14.0-RC1
Change-Id: I54bced5067f605bae67faffce46d89383dc69a39
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 31 Jan 2020 21:09:14 +0000 (14:09 -0700)]
LU-13184 tests: wait for OST startup in test_112
Wait for OST0000 to finish mounting and connecting to the MDS before
forcing to create any files there. Otherwise, intermittent failures
are seen because the OST has no objects and returns -ERANGE:
lfs setstripe: 'f112.conf-sanity.0': Numerical result out of range
problem creating f112.conf-sanity.0 on OST0000
Test-Parameters: trivial testlist=conf-sanity envdefinitions=ONLY=112,ONLY_REPEAT=100
Fixes:
416e67222b76 ("LU-12036 ofd: add 'no_precreate' mount option")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If2d547a42c51a7028803ec25680931a7593ebbe5
Reviewed-on: https://review.whamcloud.com/37393
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Mon, 18 Jan 2021 22:11:45 +0000 (01:11 +0300)]
LU-14307 quota: fix NULL pointer dereference
Fix NULL pointer dereference at 0x20 in
qmt_trans_start_with_slv->lquota_lqe_debug.
HPE-bug-id: LUS-9662
Change-Id: Iead0df053ae0dcb7453c1910a4b4b7a3728da829
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/41264
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sat, 23 Jan 2021 20:47:29 +0000 (12:47 -0800)]
LU-14356 utils: change position of the -lext2fs option
This patch changes the position of the -lext2fs option
in the gcc command line so as to resolve the following
issue:
mount_osd_ldiskfs.so: undefined symbol: unix_io_manager
Change-Id: I9ceaca867697c132b8d4a7800169101a024d17b8
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41305
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Tue, 15 Dec 2020 15:34:34 +0000 (09:34 -0600)]
LU-13578 test: use a single read() in sanity test_39r
In sanity test_39r() ensure that the we only call read() once on the
file ot update it's atime. Otherwise the file atime may be greater
than the OST object atime due to a final read() done by dd which
returns no bytes and does not generate a BRW RPC to the OST. Even
though it returns 0 bytes, it requested a non-zero number of bytes and
is therefore required to update the file access time.
Fixes:
7c9ce8aac ("LU-13383 ofd: lazy atime update")
Test-Parameters: trivial testlist=sanity env=ONLY=39r,ONLY_REPEAT=400
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1dd4e93ac150b34f4fc943fe25b92bf9119b0461
Reviewed-on: https://review.whamcloud.com/40973
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Daniel Ahlin [Tue, 8 Dec 2020 23:30:34 +0000 (00:30 +0100)]
LU-14198 build: Use pax tar format for build of dist-targets
The tar ustar format used during build of dist-* targets (which is
used when building e.g. rpms and debs) has several limitations and
will prevent e.g. users with uid > 2097151 (2^21) from building
these targets (or from passing the autoconf test for tar).
This commit changes format from ustar to POSIX.1-2001/pax which
will remove this and several other limits (with path-length being
one relevant example, see LU-12078).
Signed-off-by: Daniel Ahlin <ahlin@google.com>
Change-Id: Ic66ca696ede2e359a04c179c6d630baacaa9bcb1
Reviewed-on: https://review.whamcloud.com/40915
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Andreas Dilger [Mon, 25 Jan 2021 23:18:09 +0000 (16:18 -0700)]
LU-14286 osd-ldiskfs: enable fallocate by default
Enable fallocate on ldiskfs OSTs by default now that the known
problems have been resolved. The default mode=0 is the standard
"allocate unwritten extents" behavior used by ext4. This is by
far the fastest for space allocation, but requires the unwritten
extents to be split and/or zeroed when they are overwritten.
The OST fallocate mode=1 can also be set to use "zeroed extents",
which may be handled by "WRITE SAME", "TRIM zeroes data", or
other low-level functionality in the underlying block device.
This is somewhat slower at fallocate() time (especially for very
large allocations), but still avoids sending any data over the
network, avoids runtime overhead from managing the extents. There
is not yet an FALLOC_FL_* flag to request this behavior from the
client on a per-file basis.
If problems are hit in the field, fallocate can also be disabled
with mode=-1 at runtime or persistently.
lctl set_param [-P] osd-ldiskfs.*.fallocate_zero_blocks=<mode>
Ensure that all of the tests which currently use fallocate() are
enabling it for test runs, even if the default changes again.
Fixes:
4f18e08099e5 ("LU-14286 osd-ldiskfs: fallocate with unwritten extents")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iefa71c525597d54fc82a3d6de27a50d4d2ce7057
Reviewed-on: https://review.whamcloud.com/41315
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Tue, 19 Jan 2021 04:06:06 +0000 (12:06 +0800)]
LU-14337 lov: return stripe_count=1 instead of 0 for DoM files
Return stripe_count=1 instead of 0 for DoM files to avoid
divide-by-zero for older userspace that calls this ioctl,
e.g. lustre ADIO driver.
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I43c5e01bdee834f9a05a669a3e6f3d5cd926cb87
Reviewed-on: https://review.whamcloud.com/41265
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 14 Jan 2021 16:01:21 +0000 (09:01 -0700)]
LU-14283 osc: avoid crash if ocd reset
Avoid divide-by-zero if OSC obd_connect_data is not fully initialized.
cl_ocd_grant_param is only set after cl_max_extent_pages is OK to use.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibcee30b46e24ca3d4c2b571b27f3c0bb43f4bf71
Reviewed-on: https://review.whamcloud.com/41225
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Mon, 18 Jan 2021 17:46:05 +0000 (01:46 +0800)]
LU-14339 obdclass: add option %H for jobid
Add a option %H to avoid jobid too long in some cases.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Iaf70da5de25fd321a21e6e6cd7f7d211dca1adf3
Reviewed-on: https://review.whamcloud.com/41262
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Sebastien Buisson [Mon, 11 Jan 2021 09:36:06 +0000 (09:36 +0000)]
LU-14306 sec: get rid of bad rss-counter state messages
When doing O_DIRECT IOs on encrypted files, messages about bad
rss-counter state can be seen in the console. The mm get confused
because we twist the Lustre pages used for RPCs so that they are
suitable for llcrypt API.
In order to do this properly, the original mapping on these pages
must be preserved outside of the encryption/decryption needs.
Fixes:
728036f256 ("LU-12275 sec: O_DIRECT for encrypted file")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I80ebcd3f96c51a3d158d7ef66f23b8da13904c52
Reviewed-on: https://review.whamcloud.com/41199
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Tue, 12 Jan 2021 04:38:31 +0000 (21:38 -0700)]
LU-14286 osd-ldiskfs: fallocate with unwritten extents
The osd_fallocate() code should typically be allocating unwritten
extents with LDISKFS_GET_BLOCKS_CREATE_UNWRIT_EXT instead of actually
zeroing the blocks on disk with LDISKFS_GET_BLOCKS_CREATE_ZERO.
Writing zeroes during fallocate() is typically slower initially, and
is causing timeouts in sanity test_150e, which is trying to fill up
all OSTs to 90%. In some cases, zeroing the underlying blocks can
use the underlying storage support for efficient zeroing (WRITE_SAME),
so it may be faster for later use than uninitialized extents that have
to be converted to initialized extents by (possibly) splitting them
into smaller extents and/or zero filling them when they are paritally
being overwritten.
Add a tunable parameter osd-ldiskfs.*.fallocate_zero_blocks to allow
selecting this behavior at runtime. The default is -1, to disable
fallocate completely (return -EOPNOTSUPP) due to current bugs.
Test-Parameters: testlist=sanityn env=ONLY=16,ONLY_REPEAT=10
Fixes:
72617588ac8c ("LU-14286 osd-ldiskfs: fallocate() should zero new blocks")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ida3692c487fdc8918863fc5c99459caaba17d92e
Reviewed-on: https://review.whamcloud.com/41204
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Tue, 19 Jan 2021 14:21:57 +0000 (17:21 +0300)]
LU-14312 ldlm: don't change GROUP lock GID on client
GROUP lock GID is part of inodebits policy and is passed
to the server from client in policy li_gid field.
Meanwhile the ldlm_ibits_policy_wire_to_local() is used on
client also when server reply or completion AST is processed,
so client original GID can be overwritten by server value.
This is not problem if both server and client have the same
Lustre version but if server is older then it can have garbage
in li_gid field and client lock policy is updated with it.
Considering that GROUP lock GID is never changed and server should
not do that, the solution is to ignore returned li_gid from server
and never update original GID of GROUP lock on client from server
response.
Test-Parameters: testlist=sanity serverversion=2.12.6 env=ONLY=272b
Test-Parameters: testlist=sanity serverversion=2.13.0 serverdistro=el7.7 env=ONLY=272b
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I9a82f4b3513fd93d63b92a9527cb7b89c635e61b
Reviewed-on: https://review.whamcloud.com/41268
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
John L. Hammond [Tue, 29 Dec 2020 02:10:31 +0000 (21:10 -0500)]
LU-14034 hsm: add PID file handling to lhsmtool_posix
Add pid-file handling to lhsmtool_posix to prevent accidentally
running concurrent instances of the copytool. (Multiple instances are
still allowed if you do not use PID files or use separate files.)
Use the PID file to avoid needing libtool when stopping, continuing,
or killing the copytool from test scripts.
Remove unnecessary libtool usage from test scripts.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I4f012a88c58f3a86a731df3b7d35ff32db047c2d
Reviewed-on: https://review.whamcloud.com/40253
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Fri, 15 Jan 2021 05:33:17 +0000 (13:33 +0800)]
LU-14326 osc: correctly update size/kms for fallocate
* fallocate chose oa->o_size for falloc_offset and o->o_blocks for
falloc_end, but forgot to change attr->cat_size and attr->cat_kms
to use sa_attr.lvb_size to update osc's lvb and kms if it expands
the file's size.
Other setattr IO uses @size (sa_falloc_offset in fallocate case) to
update the lvb and kms.
* lock request extent for fallocate should be
[sa_falloc_offset, sa_falloc_end)
* calculate sa_attr.lvb_size correctly for osc objects
(lov_io_sub_inherit())
Test-Parameters: testlist=sanityn env=ONLY=16,COUNT=50000,ONLY_REPEAT=10
Test-Parameters: testlist=sanity-benchmark env=ONLY=fsx,COUNT=50000,ONLY_REPEAT=10
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Fixes:
48457868a02a ("LU-3606 fallocate: Implement fallocate preallocate operation")
Change-Id: I7dbed3bc6899a3db53284c8aac3cb9476e7958f5
Reviewed-on: https://review.whamcloud.com/41272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Sun, 10 Jan 2021 15:50:44 +0000 (23:50 +0800)]
LU-14303 tests: parallel-scale test rr_alloc fails
Correct the parameter for DNE environment. Else the
test case will fail on 'No such file or directory'.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ib94c5b17a3b49153ac229bfc4dfcee39bd9f60d4
Reviewed-on: https://review.whamcloud.com/41192
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Serguei Smirnov [Mon, 18 Jan 2021 19:20:47 +0000 (11:20 -0800)]
LU-14334 lnet: update changelog
Updated changelog to indicate changes in OFED/MOFED support
as well as new features added in this version of LNet.
Test-Parameters: trivial
Change-Id: I264f6566324da42fd51a8e159d172cbf0ae1a28b
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41263
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Sat, 16 Jan 2021 12:46:13 +0000 (20:46 +0800)]
LU-14283 obdclass: connect vs disconnect race
There might be a possible race if setup (connect)
and cleanup (disconnect) are tangled together(similar
comments in osc_disconnect()):
Thread1: Thread2:
connecting class_cleanup
ptlrpc_connect_interpret
obd->obd_setup = 0
obd_import_event
if (obd->obd_set_up)
osc_init_grant() /*skipped*/
ptlrpc_activate_import..
And If RPC was waked up and send out before
class_disconnect_exports(), It might hit divide zero crash
in osc_announce_cached() because @cl_max_extent_pages is zero.
The problem is we clear @obd_setup too early, It should be cleared
when OBD is really shutdown.
Fixes: 45900a ("LU-4134 obdclass: obd_device improvement")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I898b6f53602c05221a3154a61615a0e270167ac6
Reviewed-on: https://review.whamcloud.com/41256
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Mon, 27 Jul 2020 03:56:22 +0000 (11:56 +0800)]
LU-13602 flr: skip unknown FLR component types
Currently, in lov_init_composite() it will quit with error when
reading an unknown LOV pattern.
Since FLR will be used for upcoming new features, like PCC-RO,
an old client should be able to read the old format of the
component types, ignore and skip the new types of FLR component.
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ica3fe98203d44b52cf25b085c34c83b1a4702464
Reviewed-on: https://review.whamcloud.com/39513
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sat, 9 Jan 2021 07:23:39 +0000 (00:23 -0700)]
LU-12125 llite: send file mode with rename RPC
In preparation for parallel rename operations, send renamed file mode
to the MDS in order to allow the rename locking to be more efficient.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4691f30d151a8ff81e443d24109234341b3ebbe5
Reviewed-on: https://review.whamcloud.com/41184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Wed, 13 Jan 2021 15:48:54 +0000 (18:48 +0300)]
LU-14286 osd-ldiskfs: don't read unwritten blocks
which were allocated using fallocate(2), instead fill
the pages with zeroes.
Add a test to verify that fallocated blocks read as zeroes.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9c0c90f93fd33f26f834144e225b2643cf9fffb7
Reviewed-on: https://review.whamcloud.com/41216
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mikhail Pershin [Tue, 15 Dec 2020 13:56:21 +0000 (16:56 +0300)]
LU-14194 cksum: add lprocfs checksum support in MDC/MDT
Add missed support for checksum parameters in MDC and MDT
Handle T10-PI parameters in MDT similar to OFD, move all
functionality to target code and unify its usage in both
targets
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I7d397067304e028bf597d5c3ab16250731ccba9d
Reviewed-on: https://review.whamcloud.com/40971
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Andreas Dilger [Sun, 13 Dec 2020 11:57:23 +0000 (04:57 -0700)]
LU-13676 tools: find unique stacks from sysrq-t
Enhance the crash-find-unique-traces.awk script to be more useful:
- allow parsing output from sysrq-t
- skip leading fields from input (e.g. timestamps, syslog header)
- sort messages to avoid duplicates
- handle some variance in kernel function names
- print thread names as well as PIDs for all matching threads
- print number of threads matching a particular stack trace
- warn if parts of the stack traces were lost due to buffer overflow
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Icfd70fbf255f413ca1aab0b94c62754e4c00628d
Reviewed-on: https://review.whamcloud.com/40956
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Mon, 11 Jan 2021 03:46:42 +0000 (19:46 -0800)]
LU-14317 ldiskfs: ‘%llu’ mismatch with type ‘long int’ on arm64
This patch fixes build issues like this one on arm64:
lustre-release/ldiskfs/mballoc.c: In function ‘ldiskfs_mb_release’:
lustre-release/ldiskfs/mballoc.c:3180:4: warning:
format ‘%llu’ expects argument of type ‘long long unsigned int’,
but argument 4 has type ‘long int’ [-Wformat=]
"mballoc: (%llu, %llu, %llu) useless c(0,1,2) loops",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Change-Id: I07018302fae8da3b71c0cc649753bdcccdc0d34f
Test-Parameters: trivial
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41191
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Sebastien Buisson [Fri, 18 Dec 2020 14:49:11 +0000 (23:49 +0900)]
LU-14263 gss: unlink revoked key
When a GSS context is destroyed, it is unbound from its key, marking
the key as revoked.
The key also needs to be unlinked from the session keyring. This way,
a subsequent context initialization will manage to create a new valid
key and link it to the keyring.
Similarly, add a new '-r' flag to 'lfs flushctx', in order to reap the
revoked keys from the keyring when flushing the GSS context.
Test-Parameters: trivial
Test-Parameters: clientdistro=el7.9 testgroup=review-dne-ssk
Test-Parameters: clientdistro=el8.3 testgroup=review-dne-ssk
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ida4b4ea53202c1f40ad93816fb4ec96fec2bf8bc
Reviewed-on: https://review.whamcloud.com/41047
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Tue, 12 Jan 2021 17:56:32 +0000 (10:56 -0700)]
LU-14286 tests: reduce fallocate space sanity 150e
Changes to fallocate have increase the time to allocate
space. The increased run time is making sanity test
150e run long enough to time out. We need to reduce the
amount of space santiy test 150e fallocates from 90%
to 100 MB per OST or 60% of available space if running
in SLOW mode to allow the test to complete.
Test-Parameters: trivial env=ONLY=150e fstype=ldiskfs testlist=sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ic6ab41daf8f451c31d092849ccf6906747baa72d
Reviewed-on: https://review.whamcloud.com/41211
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Oleg Drokin [Mon, 11 Jan 2021 23:57:40 +0000 (18:57 -0500)]
LU-14324 tests: Fix fsx fallocate detection
Avoid kernel playing fake tricks on us by optimizing away small
increases in file size by fallocate, so to ensure the truth make
large fallocate call instead.
Additionally always truncate to zero after all call, as even a
failed call can result in increased file size.
Test-Parameters: trivial testlist=sanity env=ONLY=150
Test-Parameters: testlist=sanityn env=ONLY=16,COUNT=10000
Test-Parameters: testlist=sanity-dom env=ONLY=fsx,sanityn,SANITYN_ONLY=16
Test-Parameters: fstype=zfs testlist=sanity env=ONLY=150
Test-Parameters: fstype=zfs testlist=sanityn env=ONLY=16,COUNT=10000
Test-Parameters: fstype=zfs testlist=sanity-dom env=ONLY=fsx,sanityn,SANITYN_ONLY=16
Change-Id: I8cddb7d0d30bdf89bf13d18ae36a5f950adc392d
Fixes:
03247bda765c ("LU-14286 osd-ldiskfs: fallocate() should zero new blocks")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41202
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Mon, 9 Nov 2020 15:20:10 +0000 (23:20 +0800)]
LU-14128 lov: correctly set OST obj size
When extends a PFL file to a size locating at a boundary of a stripe
in a component, the truncate won't set the size of the OST object
in the prior stripe.
This patch record the prior stripe in
lov_layout_raid0::lo_trunc_stripeno and add the stripe in the
truncate IO and enqueue the lock covering it.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic5d8e3c16f950003736cd6dbd5af404613f818c7
Reviewed-on: https://review.whamcloud.com/40581
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Tue, 5 Jan 2021 22:08:34 +0000 (16:08 -0600)]
LU-14296 lnet: use an unbound cred in kiblnd_resolve_addr()
In kiblnd_resolve_addr() call prepare_kernel_cred(NULL) rather than
prepare_creds() to get a cred with unbound capabilities.
Fixes:
1e4bd16acfa ("LU-14006 o2ib: raise bind cap before resolving address")
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ib2500d82aa4eab5590fa95ae73fd75b2ab542bcd
Reviewed-on: https://review.whamcloud.com/41137
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Wed, 6 Jan 2021 19:48:35 +0000 (13:48 -0600)]
LU-14301 llite: return EOPNOTSUPP if fallocate is not supported
In ll_fallocate() if the server returns the NFSv3 specific error code
ENOTSUPP then replace it with EOPNOTSUPP to avoid confusing
applications.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I85952be27808704112a6902684900cd938d576ac
Reviewed-on: https://review.whamcloud.com/41148
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Tue, 5 Jan 2021 09:23:32 +0000 (10:23 +0100)]
LU-14281 tests: fix sanity-sec test_56 for ZFS
Skip sanity-sec test_56 in case of ZFS backend, as it does not
support FIEMAP.
Also simplify sanity-sec test_56, as all we care about is whether the
"encrypted" flag is set at all.
Fixes:
33322f3a2488 ("LU-14149 llite: fiemap set flags for encrypted files")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=56 clientdistro=el8.3 serverdistro=el8.3 fstype=ldiskfs
Test-Parameters: testlist=sanity-sec env=ONLY=56 clientdistro=el8.3 serverdistro=el8.3 fstype=zfs
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I69125d3e06acb89dbda421e573d18a2c22beac37
Reviewed-on: https://review.whamcloud.com/41130
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Mr NeilBrown [Fri, 16 Oct 2020 04:53:34 +0000 (15:53 +1100)]
LU-13783 o2iblnd: make FMR-pool support optional.
Linux 5.8 removes the FMR-pool API. This patch makes
all use for this API optional, selected only if the
support exists in the kernel.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4c40f3a766f5b46ae4f26d7d3ecf8434a6e5a0cb
Reviewed-on: https://review.whamcloud.com/40287
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Gorenko <sergeygo@nvidia.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Thu, 31 Dec 2020 19:40:34 +0000 (01:10 +0530)]
LU-14286 osd-ldiskfs: fallocate() should zero new blocks
The ldiskfs osd_fallocate() does not correctly zero the
allocated blocks. This is not noticed during autotest and
local developer testing because the underlying storage is
zero-filled due to using a sparse backing file and will
usually read back as zero. Also, the fsx data correctness
test was not properly detecting fallocate() support, and
as a result this behavior was not detected properly.
'fsx' uses test_fallocate() to determine if fallocate()
along with its various FLAGS is enabled and could be
called. It was found that test_fallocate() always returned
false as a bug. Resulting is fallocate being skipped always
for all fsx runs.
This patches fixes test_fallocate() to properly determine
if fallocate calls could be supported or not and return
appropriately and correctly (1 or 0) true/false.
After patch:
~~~~~~~~~~~
$ MOUNT_2="yes" bash ./lustre/tests/llmount.sh
$ lfs setstripe -c -1 /mnt/lustre/f110
$ fsx -c 50 -p 100 -N 2500 -S 0 -l
10485760 /mnt/lustre*/f110
Chance of close/open is 1 in 50
Seed set to 4481
fd 0: /mnt/lustre/f110
fd 1: /mnt/lustre2/f110
fsx: test_fallocate: does not support fallocate mode 0x3, disabling!
fsx: test_fallocate: does not support fallocate mode 0x10, disabling!
skipping zero size read
:
fallocating to largest ever: 0x30ce
:
1900[0]
1609444486.316868 fallocate 0x7f6c thru 0xab3a (0x2bcf bytes)
:
2300[1]
1609444487.241910 fallocate 0x0697 thru 0xed8e (0xe6f8 bytes)
:
All operations completed A-OK!
$
Test-Parameters: trivial testlist=sanity,sanityn,sanity-dom env=COUNT=10000
Fixes:
48457868a02a ("LU-3606 fallocate: Implement fallocate preallocate operation")
Fixes:
853d180121a6 ("LU-3606 fsx: Add fallocate operation to fsx")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Idbb092db6bd09cfda129f6077dcbfae4f4c9d0d0
Reviewed-on: https://review.whamcloud.com/41119
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 6 Jan 2021 10:13:56 +0000 (11:13 +0100)]
LU-14298 tests: fix sanity-sec test_44 for aarch64 client
PAGE_SIZE on aarch64 client is 64KB instead of 4KB, so sizes of IOs
performed in sanity-sec test_44 must be adapted.
Test-Parameters: testgroup=review-ldiskfs-arm env=ONLY=44 clientdistro=el8.3 serverdistro=el8.3
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I3764a75f6df871d71d751764627ed618bedf8e0b
Reviewed-on: https://review.whamcloud.com/41146
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 16 Dec 2020 07:10:56 +0000 (15:10 +0800)]
LU-14187 osd-ldiskfs: fix locking in write commit
Restart transaction in osd layer break rules that
locks are taken afer transaction start.
This patch try to fix by moving transaction restart
to OFD layer.
We will record how many extents we declare with
@oh_declared_ext. And during IO we check if it
run out before we restart whole transaction.
We track an average extent bytes per filesystem and
use it as a hint during declare write commit. This
will avoid us restart transaction ofen if filesystem
is heavily fragmented.
Fixes:
0271b17b ("LU-14134 osd-ldiskfs: reduce credits for new writing")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I289a3a6775befe159b7fef29004eaaaff873e2c6
Reviewed-on: https://review.whamcloud.com/40991
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Serguei Smirnov [Thu, 24 Dec 2020 01:43:21 +0000 (17:43 -0800)]
LU-13641 socklnd: announce deprecation of 'use_tcp_bonding'
Add warning to be printed if 'use_tcp_bonding' option is used
notifying the user that the feature is being deprecated.
It is suggested to use MR configuration with dynamic discovery
instead.
Multi-Rail feature doesn't need to be explicitly enabled.
To use MR instead of tcp bonding, group the interfaces
on the same network using the lnetctl utility:
lnetctl net add --net tcp --if eth2,eth3
or via the modprobe configuration file (/etc/modprobe.d/lnet.conf
or /etc/modprobe.d/lustre.conf):
options lnet networks="tcp(eth2,eth3)"
and make sure dynamic discovery is enabled:
lnetctl set discovery 1
MR will aggregate the throughput of all configured and available
networks/interfaces shared between peer nodes.
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I34288ae1c8a1c4092f88b45a571312691f145218
Reviewed-on: https://review.whamcloud.com/41088
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Tue, 22 Dec 2020 07:08:16 +0000 (10:08 +0300)]
LU-14267 osd: do not update inode each write
there is no need to update in-bh inode each write,
overwrites can skip that. otherwise:
1) transaction can exceed reserved credits as
osd_declare_write() doesn't count inode for overwrites
2) copy is not free and this can be called few times for
each operation: last_rcvd data, changelog.
Fixes:
a23aac2219 ("LU-13416 ldiskfs: don't corrupt data on journal replay")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I47f463d9421cea7fb32e03fa5ba0ab6524511a29
Reviewed-on: https://review.whamcloud.com/41069
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Andreas Dilger [Fri, 18 Dec 2020 22:47:09 +0000 (15:47 -0700)]
LU-14265 utils: allow 'lfs mkdir' without arguments
If no arguments are given to "lfs mkdir <newdir>" assume that
"-i -1 -c 1" were given, to create the specified <newdir> on
an MDT with above-average free space/inodes.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I319f68240832a905a86833a70a4631cde5ce7057
Reviewed-on: https://review.whamcloud.com/41054
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Thu, 17 Dec 2020 18:15:50 +0000 (10:15 -0800)]
LU-14218 kernel: kernel update SLES15 SP1 [4.12.14-197.75.1]
Update SLES15 SP1 kernel to 4.12.14-197.75.1 for Lustre client.
Test-Parameters: trivial \
env=SANITY_EXCEPT="56oc 100 130 136 817" \
clientdistro=sles15sp1 serverdistro=el7.8 \
testlist=sanity
Change-Id: I79d217d3c87da6a108b58d8ce8c02da60d9401a2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41030
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Mon, 21 Dec 2020 02:18:45 +0000 (10:18 +0800)]
LU-12702 quota: wait pending write before acquiring remotely
There is a window that ZFS has updated usage, but
pending_write has not been reduced. This could cause
us to grant more space.
With soft least qunit introduced, after timer reach,
local slave might still have some space to consume,
it is possible that we might exceed this limit.
Test-Parameters: fstype=zfs testlist=sanity-quota,sanity-quota,sanity-quota
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I0e1a94fd0b637e154642de5922a0670780c56ef2
Reviewed-on: https://review.whamcloud.com/41023
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Wed, 28 Oct 2020 13:03:47 +0000 (21:03 +0800)]
LU-14082 ldlm: Use req_mode while lock cleanup
For local lock, the decref cannot count exactly by
granted_mode if the lock has not been granted.
LustreError: (ldlm_lock.c:354:ldlm_lock_destroy_internal())
### lock still has references ns: ??
lock:
ffff88342aa07200/0x9b92ad3407bea22a
lrc: 4/0,1 mode: --/PW res: ?? rrc=?? type: ???
flags: 0x10106400000000 nid: local
remote: 0x5248822d3123ac19 expref: -99
pid: 14515 timeout: 0 lvb_type: 0
LustreError: (ldlm_lock.c:355:ldlm_lock_destroy_internal()) LBUG
Pid: 14562, comm: ll_imp_inval 3.10.0-693.21.1.el7.x86_64 #1 SMP
Call Trace:
[] save_stack_trace_tsk+0x22/0x40
[] libcfs_call_trace+0x8c/0xc0 [libcfs]
[] lbug_with_loc+0x4c/0xa0 [libcfs]
[] ldlm_lock_destroy_internal+0x269/0x2a0 [ptlrpc]
[] ldlm_lock_destroy_nolock+0x2b/0x110 [ptlrpc]
[] ldlm_flock_completion_ast+0x4f5/0x1080 [ptlrpc]
[] cleanup_resource+0x18e/0x370 [ptlrpc]
[] ldlm_resource_clean+0x53/0x60 [ptlrpc]
[] cfs_hash_for_each_relax+0x250/0x450 [libcfs]
[] cfs_hash_for_each_nolock+0x75/0x1c0 [libcfs]
[] ldlm_namespace_cleanup+0x30/0xc0 [ptlrpc]
[] mdc_import_event+0x1b6/0xa20 [mdc]
[] ptlrpc_invalidate_import+0x220/0x8f0 [ptlrpc]
[] ptlrpc_invalidate_import_thread+0x48/0x2b0 [ptlrpc]
[] kthread+0xd1/0xe0
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ic029b3fb82a3dedc507e2bada265322efae5b40c
Reviewed-on: https://review.whamcloud.com/40433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Sat, 31 Oct 2020 05:55:10 +0000 (08:55 +0300)]
LU-14097 tests: conf-sanity/91 should check specific MDT
for exports afer eviction. otherwise 91 fails if few MDTs
share same node as in local setup.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I047473311c01f94e210e8f87aced4cbd7de76b7f
Reviewed-on: https://review.whamcloud.com/40501
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Tue, 11 Sep 2018 17:54:43 +0000 (20:54 +0300)]
LU-11365 tests: test-framework::add() to load ldiskfs
when the tests run from in-tree build, then mkfs.lustre
can not rely on modprobe and ldiskfs needs to be loaded
explicitly. otherwise some tests like conf-sanity/55 fail.
Change-Id: Ic15f0eab8e483d5de9f63606433fbb26f754fd55
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33140
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log