Whamcloud - gitweb
Serguei Smirnov [Mon, 18 Jan 2021 19:20:47 +0000 (11:20 -0800)]
LU-14334 lnet: update changelog
Updated changelog to indicate changes in OFED/MOFED support
as well as new features added in this version of LNet.
Test-Parameters: trivial
Change-Id: I264f6566324da42fd51a8e159d172cbf0ae1a28b
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41263
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Sat, 16 Jan 2021 12:46:13 +0000 (20:46 +0800)]
LU-14283 obdclass: connect vs disconnect race
There might be a possible race if setup (connect)
and cleanup (disconnect) are tangled together(similar
comments in osc_disconnect()):
Thread1: Thread2:
connecting class_cleanup
ptlrpc_connect_interpret
obd->obd_setup = 0
obd_import_event
if (obd->obd_set_up)
osc_init_grant() /*skipped*/
ptlrpc_activate_import..
And If RPC was waked up and send out before
class_disconnect_exports(), It might hit divide zero crash
in osc_announce_cached() because @cl_max_extent_pages is zero.
The problem is we clear @obd_setup too early, It should be cleared
when OBD is really shutdown.
Fixes: 45900a ("LU-4134 obdclass: obd_device improvement")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I898b6f53602c05221a3154a61615a0e270167ac6
Reviewed-on: https://review.whamcloud.com/41256
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Mon, 27 Jul 2020 03:56:22 +0000 (11:56 +0800)]
LU-13602 flr: skip unknown FLR component types
Currently, in lov_init_composite() it will quit with error when
reading an unknown LOV pattern.
Since FLR will be used for upcoming new features, like PCC-RO,
an old client should be able to read the old format of the
component types, ignore and skip the new types of FLR component.
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ica3fe98203d44b52cf25b085c34c83b1a4702464
Reviewed-on: https://review.whamcloud.com/39513
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sat, 9 Jan 2021 07:23:39 +0000 (00:23 -0700)]
LU-12125 llite: send file mode with rename RPC
In preparation for parallel rename operations, send renamed file mode
to the MDS in order to allow the rename locking to be more efficient.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4691f30d151a8ff81e443d24109234341b3ebbe5
Reviewed-on: https://review.whamcloud.com/41184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Wed, 13 Jan 2021 15:48:54 +0000 (18:48 +0300)]
LU-14286 osd-ldiskfs: don't read unwritten blocks
which were allocated using fallocate(2), instead fill
the pages with zeroes.
Add a test to verify that fallocated blocks read as zeroes.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9c0c90f93fd33f26f834144e225b2643cf9fffb7
Reviewed-on: https://review.whamcloud.com/41216
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mikhail Pershin [Tue, 15 Dec 2020 13:56:21 +0000 (16:56 +0300)]
LU-14194 cksum: add lprocfs checksum support in MDC/MDT
Add missed support for checksum parameters in MDC and MDT
Handle T10-PI parameters in MDT similar to OFD, move all
functionality to target code and unify its usage in both
targets
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I7d397067304e028bf597d5c3ab16250731ccba9d
Reviewed-on: https://review.whamcloud.com/40971
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Andreas Dilger [Sun, 13 Dec 2020 11:57:23 +0000 (04:57 -0700)]
LU-13676 tools: find unique stacks from sysrq-t
Enhance the crash-find-unique-traces.awk script to be more useful:
- allow parsing output from sysrq-t
- skip leading fields from input (e.g. timestamps, syslog header)
- sort messages to avoid duplicates
- handle some variance in kernel function names
- print thread names as well as PIDs for all matching threads
- print number of threads matching a particular stack trace
- warn if parts of the stack traces were lost due to buffer overflow
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Icfd70fbf255f413ca1aab0b94c62754e4c00628d
Reviewed-on: https://review.whamcloud.com/40956
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Mon, 11 Jan 2021 03:46:42 +0000 (19:46 -0800)]
LU-14317 ldiskfs: ‘%llu’ mismatch with type ‘long int’ on arm64
This patch fixes build issues like this one on arm64:
lustre-release/ldiskfs/mballoc.c: In function ‘ldiskfs_mb_release’:
lustre-release/ldiskfs/mballoc.c:3180:4: warning:
format ‘%llu’ expects argument of type ‘long long unsigned int’,
but argument 4 has type ‘long int’ [-Wformat=]
"mballoc: (%llu, %llu, %llu) useless c(0,1,2) loops",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Change-Id: I07018302fae8da3b71c0cc649753bdcccdc0d34f
Test-Parameters: trivial
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41191
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Sebastien Buisson [Fri, 18 Dec 2020 14:49:11 +0000 (23:49 +0900)]
LU-14263 gss: unlink revoked key
When a GSS context is destroyed, it is unbound from its key, marking
the key as revoked.
The key also needs to be unlinked from the session keyring. This way,
a subsequent context initialization will manage to create a new valid
key and link it to the keyring.
Similarly, add a new '-r' flag to 'lfs flushctx', in order to reap the
revoked keys from the keyring when flushing the GSS context.
Test-Parameters: trivial
Test-Parameters: clientdistro=el7.9 testgroup=review-dne-ssk
Test-Parameters: clientdistro=el8.3 testgroup=review-dne-ssk
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ida4b4ea53202c1f40ad93816fb4ec96fec2bf8bc
Reviewed-on: https://review.whamcloud.com/41047
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Tue, 12 Jan 2021 17:56:32 +0000 (10:56 -0700)]
LU-14286 tests: reduce fallocate space sanity 150e
Changes to fallocate have increase the time to allocate
space. The increased run time is making sanity test
150e run long enough to time out. We need to reduce the
amount of space santiy test 150e fallocates from 90%
to 100 MB per OST or 60% of available space if running
in SLOW mode to allow the test to complete.
Test-Parameters: trivial env=ONLY=150e fstype=ldiskfs testlist=sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ic6ab41daf8f451c31d092849ccf6906747baa72d
Reviewed-on: https://review.whamcloud.com/41211
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Oleg Drokin [Mon, 11 Jan 2021 23:57:40 +0000 (18:57 -0500)]
LU-14324 tests: Fix fsx fallocate detection
Avoid kernel playing fake tricks on us by optimizing away small
increases in file size by fallocate, so to ensure the truth make
large fallocate call instead.
Additionally always truncate to zero after all call, as even a
failed call can result in increased file size.
Test-Parameters: trivial testlist=sanity env=ONLY=150
Test-Parameters: testlist=sanityn env=ONLY=16,COUNT=10000
Test-Parameters: testlist=sanity-dom env=ONLY=fsx,sanityn,SANITYN_ONLY=16
Test-Parameters: fstype=zfs testlist=sanity env=ONLY=150
Test-Parameters: fstype=zfs testlist=sanityn env=ONLY=16,COUNT=10000
Test-Parameters: fstype=zfs testlist=sanity-dom env=ONLY=fsx,sanityn,SANITYN_ONLY=16
Change-Id: I8cddb7d0d30bdf89bf13d18ae36a5f950adc392d
Fixes:
03247bda765c ("LU-14286 osd-ldiskfs: fallocate() should zero new blocks")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41202
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Mon, 9 Nov 2020 15:20:10 +0000 (23:20 +0800)]
LU-14128 lov: correctly set OST obj size
When extends a PFL file to a size locating at a boundary of a stripe
in a component, the truncate won't set the size of the OST object
in the prior stripe.
This patch record the prior stripe in
lov_layout_raid0::lo_trunc_stripeno and add the stripe in the
truncate IO and enqueue the lock covering it.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic5d8e3c16f950003736cd6dbd5af404613f818c7
Reviewed-on: https://review.whamcloud.com/40581
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Tue, 5 Jan 2021 22:08:34 +0000 (16:08 -0600)]
LU-14296 lnet: use an unbound cred in kiblnd_resolve_addr()
In kiblnd_resolve_addr() call prepare_kernel_cred(NULL) rather than
prepare_creds() to get a cred with unbound capabilities.
Fixes:
1e4bd16acfa ("LU-14006 o2ib: raise bind cap before resolving address")
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ib2500d82aa4eab5590fa95ae73fd75b2ab542bcd
Reviewed-on: https://review.whamcloud.com/41137
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Wed, 6 Jan 2021 19:48:35 +0000 (13:48 -0600)]
LU-14301 llite: return EOPNOTSUPP if fallocate is not supported
In ll_fallocate() if the server returns the NFSv3 specific error code
ENOTSUPP then replace it with EOPNOTSUPP to avoid confusing
applications.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I85952be27808704112a6902684900cd938d576ac
Reviewed-on: https://review.whamcloud.com/41148
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Tue, 5 Jan 2021 09:23:32 +0000 (10:23 +0100)]
LU-14281 tests: fix sanity-sec test_56 for ZFS
Skip sanity-sec test_56 in case of ZFS backend, as it does not
support FIEMAP.
Also simplify sanity-sec test_56, as all we care about is whether the
"encrypted" flag is set at all.
Fixes:
33322f3a2488 ("LU-14149 llite: fiemap set flags for encrypted files")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=56 clientdistro=el8.3 serverdistro=el8.3 fstype=ldiskfs
Test-Parameters: testlist=sanity-sec env=ONLY=56 clientdistro=el8.3 serverdistro=el8.3 fstype=zfs
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I69125d3e06acb89dbda421e573d18a2c22beac37
Reviewed-on: https://review.whamcloud.com/41130
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Mr NeilBrown [Fri, 16 Oct 2020 04:53:34 +0000 (15:53 +1100)]
LU-13783 o2iblnd: make FMR-pool support optional.
Linux 5.8 removes the FMR-pool API. This patch makes
all use for this API optional, selected only if the
support exists in the kernel.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4c40f3a766f5b46ae4f26d7d3ecf8434a6e5a0cb
Reviewed-on: https://review.whamcloud.com/40287
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Gorenko <sergeygo@nvidia.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Thu, 31 Dec 2020 19:40:34 +0000 (01:10 +0530)]
LU-14286 osd-ldiskfs: fallocate() should zero new blocks
The ldiskfs osd_fallocate() does not correctly zero the
allocated blocks. This is not noticed during autotest and
local developer testing because the underlying storage is
zero-filled due to using a sparse backing file and will
usually read back as zero. Also, the fsx data correctness
test was not properly detecting fallocate() support, and
as a result this behavior was not detected properly.
'fsx' uses test_fallocate() to determine if fallocate()
along with its various FLAGS is enabled and could be
called. It was found that test_fallocate() always returned
false as a bug. Resulting is fallocate being skipped always
for all fsx runs.
This patches fixes test_fallocate() to properly determine
if fallocate calls could be supported or not and return
appropriately and correctly (1 or 0) true/false.
After patch:
~~~~~~~~~~~
$ MOUNT_2="yes" bash ./lustre/tests/llmount.sh
$ lfs setstripe -c -1 /mnt/lustre/f110
$ fsx -c 50 -p 100 -N 2500 -S 0 -l
10485760 /mnt/lustre*/f110
Chance of close/open is 1 in 50
Seed set to 4481
fd 0: /mnt/lustre/f110
fd 1: /mnt/lustre2/f110
fsx: test_fallocate: does not support fallocate mode 0x3, disabling!
fsx: test_fallocate: does not support fallocate mode 0x10, disabling!
skipping zero size read
:
fallocating to largest ever: 0x30ce
:
1900[0]
1609444486.316868 fallocate 0x7f6c thru 0xab3a (0x2bcf bytes)
:
2300[1]
1609444487.241910 fallocate 0x0697 thru 0xed8e (0xe6f8 bytes)
:
All operations completed A-OK!
$
Test-Parameters: trivial testlist=sanity,sanityn,sanity-dom env=COUNT=10000
Fixes:
48457868a02a ("LU-3606 fallocate: Implement fallocate preallocate operation")
Fixes:
853d180121a6 ("LU-3606 fsx: Add fallocate operation to fsx")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Idbb092db6bd09cfda129f6077dcbfae4f4c9d0d0
Reviewed-on: https://review.whamcloud.com/41119
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 6 Jan 2021 10:13:56 +0000 (11:13 +0100)]
LU-14298 tests: fix sanity-sec test_44 for aarch64 client
PAGE_SIZE on aarch64 client is 64KB instead of 4KB, so sizes of IOs
performed in sanity-sec test_44 must be adapted.
Test-Parameters: testgroup=review-ldiskfs-arm env=ONLY=44 clientdistro=el8.3 serverdistro=el8.3
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I3764a75f6df871d71d751764627ed618bedf8e0b
Reviewed-on: https://review.whamcloud.com/41146
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 16 Dec 2020 07:10:56 +0000 (15:10 +0800)]
LU-14187 osd-ldiskfs: fix locking in write commit
Restart transaction in osd layer break rules that
locks are taken afer transaction start.
This patch try to fix by moving transaction restart
to OFD layer.
We will record how many extents we declare with
@oh_declared_ext. And during IO we check if it
run out before we restart whole transaction.
We track an average extent bytes per filesystem and
use it as a hint during declare write commit. This
will avoid us restart transaction ofen if filesystem
is heavily fragmented.
Fixes:
0271b17b ("LU-14134 osd-ldiskfs: reduce credits for new writing")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I289a3a6775befe159b7fef29004eaaaff873e2c6
Reviewed-on: https://review.whamcloud.com/40991
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Serguei Smirnov [Thu, 24 Dec 2020 01:43:21 +0000 (17:43 -0800)]
LU-13641 socklnd: announce deprecation of 'use_tcp_bonding'
Add warning to be printed if 'use_tcp_bonding' option is used
notifying the user that the feature is being deprecated.
It is suggested to use MR configuration with dynamic discovery
instead.
Multi-Rail feature doesn't need to be explicitly enabled.
To use MR instead of tcp bonding, group the interfaces
on the same network using the lnetctl utility:
lnetctl net add --net tcp --if eth2,eth3
or via the modprobe configuration file (/etc/modprobe.d/lnet.conf
or /etc/modprobe.d/lustre.conf):
options lnet networks="tcp(eth2,eth3)"
and make sure dynamic discovery is enabled:
lnetctl set discovery 1
MR will aggregate the throughput of all configured and available
networks/interfaces shared between peer nodes.
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I34288ae1c8a1c4092f88b45a571312691f145218
Reviewed-on: https://review.whamcloud.com/41088
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Tue, 22 Dec 2020 07:08:16 +0000 (10:08 +0300)]
LU-14267 osd: do not update inode each write
there is no need to update in-bh inode each write,
overwrites can skip that. otherwise:
1) transaction can exceed reserved credits as
osd_declare_write() doesn't count inode for overwrites
2) copy is not free and this can be called few times for
each operation: last_rcvd data, changelog.
Fixes:
a23aac2219 ("LU-13416 ldiskfs: don't corrupt data on journal replay")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I47f463d9421cea7fb32e03fa5ba0ab6524511a29
Reviewed-on: https://review.whamcloud.com/41069
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Andreas Dilger [Fri, 18 Dec 2020 22:47:09 +0000 (15:47 -0700)]
LU-14265 utils: allow 'lfs mkdir' without arguments
If no arguments are given to "lfs mkdir <newdir>" assume that
"-i -1 -c 1" were given, to create the specified <newdir> on
an MDT with above-average free space/inodes.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I319f68240832a905a86833a70a4631cde5ce7057
Reviewed-on: https://review.whamcloud.com/41054
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Thu, 17 Dec 2020 18:15:50 +0000 (10:15 -0800)]
LU-14218 kernel: kernel update SLES15 SP1 [4.12.14-197.75.1]
Update SLES15 SP1 kernel to 4.12.14-197.75.1 for Lustre client.
Test-Parameters: trivial \
env=SANITY_EXCEPT="56oc 100 130 136 817" \
clientdistro=sles15sp1 serverdistro=el7.8 \
testlist=sanity
Change-Id: I79d217d3c87da6a108b58d8ce8c02da60d9401a2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41030
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Mon, 21 Dec 2020 02:18:45 +0000 (10:18 +0800)]
LU-12702 quota: wait pending write before acquiring remotely
There is a window that ZFS has updated usage, but
pending_write has not been reduced. This could cause
us to grant more space.
With soft least qunit introduced, after timer reach,
local slave might still have some space to consume,
it is possible that we might exceed this limit.
Test-Parameters: fstype=zfs testlist=sanity-quota,sanity-quota,sanity-quota
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I0e1a94fd0b637e154642de5922a0670780c56ef2
Reviewed-on: https://review.whamcloud.com/41023
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Wed, 28 Oct 2020 13:03:47 +0000 (21:03 +0800)]
LU-14082 ldlm: Use req_mode while lock cleanup
For local lock, the decref cannot count exactly by
granted_mode if the lock has not been granted.
LustreError: (ldlm_lock.c:354:ldlm_lock_destroy_internal())
### lock still has references ns: ??
lock:
ffff88342aa07200/0x9b92ad3407bea22a
lrc: 4/0,1 mode: --/PW res: ?? rrc=?? type: ???
flags: 0x10106400000000 nid: local
remote: 0x5248822d3123ac19 expref: -99
pid: 14515 timeout: 0 lvb_type: 0
LustreError: (ldlm_lock.c:355:ldlm_lock_destroy_internal()) LBUG
Pid: 14562, comm: ll_imp_inval 3.10.0-693.21.1.el7.x86_64 #1 SMP
Call Trace:
[] save_stack_trace_tsk+0x22/0x40
[] libcfs_call_trace+0x8c/0xc0 [libcfs]
[] lbug_with_loc+0x4c/0xa0 [libcfs]
[] ldlm_lock_destroy_internal+0x269/0x2a0 [ptlrpc]
[] ldlm_lock_destroy_nolock+0x2b/0x110 [ptlrpc]
[] ldlm_flock_completion_ast+0x4f5/0x1080 [ptlrpc]
[] cleanup_resource+0x18e/0x370 [ptlrpc]
[] ldlm_resource_clean+0x53/0x60 [ptlrpc]
[] cfs_hash_for_each_relax+0x250/0x450 [libcfs]
[] cfs_hash_for_each_nolock+0x75/0x1c0 [libcfs]
[] ldlm_namespace_cleanup+0x30/0xc0 [ptlrpc]
[] mdc_import_event+0x1b6/0xa20 [mdc]
[] ptlrpc_invalidate_import+0x220/0x8f0 [ptlrpc]
[] ptlrpc_invalidate_import_thread+0x48/0x2b0 [ptlrpc]
[] kthread+0xd1/0xe0
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ic029b3fb82a3dedc507e2bada265322efae5b40c
Reviewed-on: https://review.whamcloud.com/40433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Sat, 31 Oct 2020 05:55:10 +0000 (08:55 +0300)]
LU-14097 tests: conf-sanity/91 should check specific MDT
for exports afer eviction. otherwise 91 fails if few MDTs
share same node as in local setup.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I047473311c01f94e210e8f87aced4cbd7de76b7f
Reviewed-on: https://review.whamcloud.com/40501
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Tue, 11 Sep 2018 17:54:43 +0000 (20:54 +0300)]
LU-11365 tests: test-framework::add() to load ldiskfs
when the tests run from in-tree build, then mkfs.lustre
can not rely on modprobe and ldiskfs needs to be loaded
explicitly. otherwise some tests like conf-sanity/55 fail.
Change-Id: Ic15f0eab8e483d5de9f63606433fbb26f754fd55
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33140
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 15 Dec 2020 05:44:59 +0000 (08:44 +0300)]
LU-14214 ofd: fix locking in ofd_object_fallocate()
the rule is that locks are taken after transaction start.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5252be0033611f03b355c1f98e21422cdde55c9e
Reviewed-on: https://review.whamcloud.com/40969
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 14 Dec 2020 23:08:46 +0000 (16:08 -0700)]
LU-8066 osd-ldiskfs: quiet debug mount message
We don't need a message printed to the console for every
mount reporting that tunable parameters were configured.
Test-Parameters: trivial
Fixes:
493cd8088388 ("LU-8066 osd: migrate from proc to sysfs")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I12fb89f8f15a86657fe5c1f46359f184ce3ebbe5
Reviewed-on: https://review.whamcloud.com/40968
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaly Fertman [Mon, 19 Oct 2020 20:46:15 +0000 (23:46 +0300)]
LU-14192 mdt: check for overstriping correctly
check for overstriping in the first component as well
HPE-bug-id: LUS-9430
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Ice125b885681b0cf71dc55c5238c4c4d7d7bb970
Reviewed-on: https://es-gerrit.dev.cray.com/157999
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/40896
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaly Fertman [Thu, 26 Nov 2020 18:37:43 +0000 (21:37 +0300)]
LU-14191 lod: comp stripe count limit check
when creating a PFL file of 2K stripes, they are shrinked to 700+,
because lod_get_stripe_count() checks the space left after taking
given 2K stripes into account, and the total amount of stripes that
fits into EA is 2700+.
when creating a SEL file of 2K stipes, it fails because the extension
component is also taking into account despite the fact it is never
initialised and due to the above logic 2x2k stripes leaves no space
in EA.
HPE-bug-id: LUS-9589
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Ibeb063472d477e393646860010823fed5acb8382
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://review.whamcloud.com/40895
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Etienne AUJAMES [Fri, 27 Nov 2020 20:26:15 +0000 (21:26 +0100)]
LU-14158 mdc: process changelogs_catalog from the oldest rec
The chlg_load use the LLOG_CAT_FIRST to process changelogs. This
values will process record in the catalog always starting with index
0 to the newest record. So when catalog reach the end of indexes and
when records are saved at the begining of catalog, the
llog_cat_process will ignore records at the end.
This patch change the "startcat" value LLOG_CAT_FIRST to 0 to scan
the catalog from the oldest record to the newest.
Fixes:
e2150028 (LU-12553 mdc: polling mode for changelog reader)
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ib00a4373d87fd63bfb5d05731f1fdbd286c0b6f5
Reviewed-on: https://review.whamcloud.com/40786
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 25 Nov 2020 22:50:04 +0000 (15:50 -0700)]
LU-4165 tests: restore skipped sanity-lfsck tests
Many subtests in sanity-lfsck were incorrectly being skipped due
to a cleanup patch that was intended to skip those tests for very
old Lustre versions being run on the MDS, or for ZFS MDTs.
Instead, a large number of tests were *always* being skipped.
Restore those accidentally-skipped tests. Hopefully they still pass.
Clean up code style for the script:
- remove the needless "return" after "skip"
- use mds1_FSTYPE and MDS1_VERSION instead of getting values from MDS
Fixes:
c54b6ca2bdb ("LU-13718 tests: add LU numbers to skipped tests")
Test-Parameters: trivial fstype=ldiskfs testlist=sanity-lfsck
Test-Parameters: fstype=ldiskfs mdscount=2 mdtcount=4 testlist=sanity-lfsck
Test-Parameters: fstype=zfs testlist=sanity-lfsck
Test-Parameters: fstype=zfs mdscount=2 mdtcount=4 testlist=sanity-lfsck
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4e49d79826369a546576672f77f7012f4a3ebbe5
Reviewed-on: https://review.whamcloud.com/40760
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 15 Dec 2020 08:13:20 +0000 (00:13 -0800)]
LU-14075 kernel: kernel update RHEL8.2 [4.18.0-193.28.1.el8_2]
Update RHEL8.2 kernel to 4.18.0-193.28.1.el8_2.
Test-Parameters: trivial \
clientdistro=el8.2 serverdistro=el8.2 \
testlist=sanity
Change-Id: I34e1e51241c3090d1041dedef8379c2e212f58a5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40417
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Mon, 26 Oct 2020 02:37:26 +0000 (10:37 +0800)]
LU-13745 pcc: fall back normal splice read for detached file
For the file that was once cached into PCC but detached from PCC
later, it should fall back into the normal I/O path to direct I/Os
to Lustre OSTs for splice read, not just return zero.
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iff3449069a0a12ac56b58618e474749361c31bb7
Reviewed-on: https://review.whamcloud.com/40396
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Wed, 16 Dec 2020 02:50:24 +0000 (10:50 +0800)]
LU-14205 osd-ldiskfs: fix return of osd_extend_restart_trans()
ldiskfs_journal_ensure_credits() could possibly return 1 if
transaction restarted, Lustre osd expect it return 0 or negative.
Fixes:
cf48a2d23b ("LU-13839 kernel: RHEL 8.3 server support")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I51ff0bd50575902b66529cbd71ff14710c202d03
Reviewed-on: https://review.whamcloud.com/40989
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 14 Dec 2020 21:34:55 +0000 (14:34 -0700)]
LU-11848 misc: update e2fsprogs to 1.45.6.wc3
Update Changelog to reference new e2fsprogs release.
23547cbe57df LU-14126 e2fsck: update mmp block race
0b6718f5e22d LU-11848 filefrag: support PFL and FLR file
502f2e17bd5b LU-13797 e2fsck: quiet 'extent tree could be narrower'
94318cef3508 misc: replace remaining loff_t with ext2_loff_t
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4b3859dfe3950f3b23e47f69ecf8b3ab053ebbe5
Reviewed-on: https://review.whamcloud.com/40967
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 7 Dec 2020 20:08:45 +0000 (13:08 -0700)]
LU-11848 tests: handle 4-digit version numbers properly
Allow version_code() to handle and compare 4-digit version numbers
properly.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8f57537f87bb1c087def277d0d6192def43ebbe5
Reviewed-on: https://review.whamcloud.com/40897
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Wed, 16 Dec 2020 14:51:40 +0000 (09:51 -0500)]
LU-14227 tests: Add missing space in sanity-lnet test 21
otherwise echo"Add peer with nidrange (gni)" is not a valid command
Change-Id: Ibdff3f38338bb880dd4ae23e31ca2517f4626010
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
6a8335be5c2b ("LU-12410 tests: Add gni tests to sanity-lnet")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41003
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Bobi Jam [Thu, 26 Nov 2020 06:13:28 +0000 (14:13 +0800)]
LU-11848 lov: FIEMAP support for PFL and FLR file
* use the high 16 bits of fe_device to record the absolute stripe
number from the beginning we are processing, so that continuous call
can resume from the stripe specified by it.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ie519a9361b2d3ae4977ed2ccf925c92065aa9787
Reviewed-on: https://review.whamcloud.com/40766
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Mr NeilBrown [Thu, 10 Dec 2020 21:41:34 +0000 (08:41 +1100)]
LU-13839 ldiskfs: fix detecting ext4_journal_ensure_credits
As __ext4_journal_ensure_credits() is not exported, the current
detection doesn't work.
So add a local definition so that the build succeeds.
Fixes:
cf48a2d23bca ("LU-13839 kernel: RHEL 8.3 server support")
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5f4b0883e8425d26760ea77e89379103dce30855
Reviewed-on: https://review.whamcloud.com/40939
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Vitaly Fertman [Fri, 4 Dec 2020 19:18:48 +0000 (22:18 +0300)]
LU-14185 osp: alloc fails for OSP obj xattr cache
osp_oac_xattr_find_or_add() may get ENOMEM as it tries to allocate
over len=buf->lb_len=XATTR_SIZE_MAX, thus order:5. Use large alloc
instead.
HPE-bug-id: LUS-9551
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I98c08d16d0343cb60f6efcb63d20739103f2af1c
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/40871
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Shaun Tancheff [Tue, 27 Oct 2020 21:03:13 +0000 (16:03 -0500)]
LU-14080 gnilnd: updates for SUSE 15 SP2
gnilnd updates for SUSE 15 SP2
Use -Werror when checking for -Wno-stringop-truncation to
ensure the compile test is valid
Adjust for kernel dropping time_t, timeval, timespec
Adjust for kernel switching to timer_setup
Adjust for kernel dropping global_page_state(), use
nr_free_pages instead
Cleanup string format for stricter checking
Test-Parameters: trivial
HPE-bug-id: LUS-9453
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I5dcda1497fa5b1f2cf4a215517700f07374fcf7f
Reviewed-on: https://review.whamcloud.com/40426
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 3 Dec 2020 00:48:41 +0000 (11:48 +1100)]
LU-14161 obdclass: fix some problems with obd_nid_hash
There are a few of problems with the handling of the obd_nid_hash
rhl-table.
- obd_export_evict_by_nid() drops out of rcu_readlock() while holding
a reference into the table. This is theoretically unsafe as changed
to the table might cause entries to be missed.
- nid_keycmp() ignores entries with exp_failed set. This is a problem
because rhltable_lookup() only compares the key for the *first*
object in a list of objects with the same key - on a match it
returns the whole list.
So if exp_failed was set on the first in the list, all would become
invisible.
These can be fixed by moving the test on ->exp_failed to after the
call to rhltabke_lookup(), and by using obd_nid_export_for_each() in
obd_export_evict_by_nid(), and repeatedly dealing with the first exp
returned.
Fixes:
580ef453d1d6 ("LU-8130 obd: convert obd_nid_hash to rhashtable")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I65861c179e93c26cf39c5db8a58e3e4c9b962fe3
Reviewed-on: https://review.whamcloud.com/40846
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 17 Jul 2020 10:08:08 +0000 (04:08 -0600)]
LU-9114 ldlm: don't compute sumsq for pool stats
Remove the calculation of sumsq from the LDLM pool stats, since
these stats are almost never used, while conversely the pools
are updated frequently.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I90612b613fb76f4500ca6cd5613f5c121b3ebbe5
Reviewed-on: https://review.whamcloud.com/39435
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 10 Dec 2020 10:15:54 +0000 (04:15 -0600)]
LU-14100 gnilnd: fix kgnilnd_setup_immediate_buffer
struct kvec * removed from kgnilnd_setup_immediate_buffer
Fixes:
c87f41dd20 ("LU-13004 gnilnd: discard struct kvec arg.")
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ie3c511326af6ddb09b6bfa13bb779fc094ba30ec
Reviewed-on: https://review.whamcloud.com/40504
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Sun, 6 Dec 2020 02:03:22 +0000 (21:03 -0500)]
LU-9215 build: Re-add the lustre-devel package
Traditonally on rpm systems a special devel package exist that
contains userland headers and symlinks for the libraries. This
patch creates such a package.
The introduction of this new package changes the dependency
of lustre-test packages since the user land headers are tested
to make sure they compile for user land applications. Update
the dpk package as well.
Lastly we add pkg-config support to Lustre as well. This means
that if do configure --prefix=special_place it will not break
external applications if they use pkg-config.
Change-Id: I3e856d85239d1712e21f7ecc633d9fae569238e6
Test-Parameters: trivial
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/40728
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Fri, 11 Dec 2020 07:17:50 +0000 (15:17 +0800)]
LU-14205 osd-ldiskfs: return correct error after end io
@bi_status could be positive in newer kernel, we need use
blk_status_to_errno() helper to convert error properly.
Otherwise, a positive value could be returned to client
which might trigger unexpected error and make client
fail to resend.
Test-Parameters: clientdistro=el8.3 serverdistro=el8.3
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I2e8f257814eff311b72416788b7f6d9e37bad84a
Reviewed-on: https://review.whamcloud.com/40944
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Sebastien Buisson [Thu, 3 Dec 2020 14:43:29 +0000 (15:43 +0100)]
LU-14149 llite: fiemap set flags for encrypted files
FIEMAP ioctl needs to set
FIEMAP_EXTENT_DATA_ENCRYPTED|FIEMAP_EXTENT_ENCODED flags for all
extents of files encrypted by fscrypt.
Also add sanity-sec test_56 to exercise the flags, with just a
plain file layout for now.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I26f788b5c4b37e96de57334f48847b40886e14c2
Reviewed-on: https://review.whamcloud.com/40852
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
John L. Hammond [Mon, 14 Dec 2020 18:36:32 +0000 (12:36 -0600)]
LU-14208 utils: remove copy_file_range() usage from copytool
Revert the lhsmtool_posix part of "LU-11621 utils: optimize
lhsmtool_posix with copy_file_range()". This is failing sanity-hsm
test_55() on RHEL 8.3.
Test-Parameters: clientdistro=el8.3 serverdistro=el8.3 testlist=sanity-hsm
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I0b410c2bb5f56a32c10f0ec39190735908cbb556
Reviewed-on: https://review.whamcloud.com/40966
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mikhail Pershin [Sun, 1 Dec 2019 06:41:37 +0000 (09:41 +0300)]
LU-10664 dom: non-blocking enqueue for DOM locks
DOM lock enqueue waits for blocking locks on MDT due to
ATOMIC flag, so MDT thread is blocked until lock is granted.
When many clients attempt to write to shared file that may
cause server thread starvation and lock contention. Switch
to non-atomic lock enqueue for DOM locks.
- switch IO lock to non-intent enqueue, so it doesn't consume
server thread for a long time being blocked
- on client take LVB from l_lvb_data updated by completion AST and
update l_ost_lvb used by DoM
- make glimpse performing similarly on MDT and OST, it uses one
format with no intent buffer and return data in LVB buffer
- introduce new connect flag 'dom_lvb' for compatibility reasons
- on server handle glimpse for both old and new clients by filling
either LVB reply buffer or mdt_body buffer
- don't take RPC slot for a DOM enqueue like it is done for EXTENT
locks, update ldlm_cli_enqueue_fini() to accept ldlm_enqueue_info
as parameter
- check that there is no atomic local lock issued with mandatory DOM
bit, trybits should be used
Test-Parameters: mdssizegb=20 testlist=dom-performance
Test-Parameters: testlist=sanity-dom clientversion=2.12
Test-Parameters: testlist=sanity-dom serverversion=2.12
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I06573018126f0663a8f76dce07e126a1879cfba5
Reviewed-on: https://review.whamcloud.com/36903
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Wed, 9 Dec 2020 12:34:05 +0000 (13:34 +0100)]
LU-14199 sec: find policy version in use for sepol
SELinux exports the maximum kernel policy version that can be used.
When building SELinux status checking representation 'sepol', we need
to look for all possible versions of the policy, not only the max one.
Test-Parameters: clientdistro=el8.3 serverdistro=el8.2 testgroup=review-dne-selinux
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iae4b66403ce953e5a7c0df585900713c597ff033
Reviewed-on: https://review.whamcloud.com/40918
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Tue, 8 Dec 2020 14:17:40 +0000 (15:17 +0100)]
LU-14095 gss: use RCU protection for sunrpc cache
Leverage sunrpc cache RCU protection for cache entry removal,
if the kernel supports it.
Test-Parameters: clientdistro=el8.2 serverdistro=el8.2 testgroup=review-dne-ssk
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I50f9dc9e9dc434b502df1e1013215b5506289ff9
Reviewed-on: https://review.whamcloud.com/40914
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Mon, 7 Dec 2020 03:06:08 +0000 (11:06 +0800)]
LU-14186 lov: instantiate components layout for fallocate
fallocate() need send intent lock to MDS to instantiate layout
like PFL.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ie14574ca0981ea00aa1867489d2a40b75eb511cc
Reviewed-on: https://review.whamcloud.com/40885
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Mon, 19 Oct 2020 02:11:36 +0000 (13:11 +1100)]
LU-13783 libcfs: don't lookup symbol when not needed
Linux 5.8 and later do not export kallsyms_lookup_name() so we need to
remove dependence on it.
Looking up radix_tree_node_cachep is not needed in these kernels,
or any kernel were HAVE_XARRAY_SUPPORT is defined.
So avoid the lookup in those cases.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia61fe9a025653985b7dcf67b1234d32667719956
Reviewed-on: https://review.whamcloud.com/40825
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Andreas Dilger [Mon, 30 Nov 2020 20:35:46 +0000 (13:35 -0700)]
LU-13855 tests: remove the need for libiberty
The statx.c test program uses a bunch of GNU-specific xstr*
functions that have very little, if any, benefit to the code.
They allocate copies of static format strings, but those
strings are not modified, and are never freed. Since the
use of xstr* drags in libiberty and associated build issues
on a variety of platforms, instead avoid those functions.
Remove the use of this library completely from the statx.c
test tool, with minimal replacements to keep the build happy.
Remove the corresponding configure checks.
Fixes:
3f7853b31ef6 ("LU-10934 llite: integrate statx() API with Lustre")
Test-Parameters: trivial testlist=sanityn env=ONLY=106 clientdistro=ubuntu2004
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2c11bac8e2a4b4754ee78caabfa9cbdba98bffff
Reviewed-on: https://review.whamcloud.com/40802
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 26 Nov 2020 16:22:49 +0000 (11:22 -0500)]
LU-14151 gss: use skcipher when sync_skcipher not available
Linux 4.2 introduced support for skcipher in commit
7a7ffe65c8c5 crypto: skcipher - Add top-level skcipher interface
Based on the existence of crypto_alloc_skcipher, use the skcipher API
instead of the blkcipher API, in case the sync_skcipher API is not
available.
Test-Parameters: clientdistro=el8.2 env=SHARED_KEY=true,SK_FLAVOR=skpi mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity
Test-Parameters: clientdistro=el7.8 env=SHARED_KEY=true,SK_FLAVOR=skpi mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4e62cd5c8219a215c5528f81d525c054215073e3
Reviewed-on: https://review.whamcloud.com/40768
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Thu, 19 Nov 2020 13:24:18 +0000 (16:24 +0300)]
LU-14143 lov: fix SEEK_HOLE calcs at component end
If data ends exactly at component end then LOV assumed that
is not yet hole in file and the next component will take care.
Meanwhile there can be no next component initialized yet if file
ends exactly at component boundary, so no hole offset is returned
but error
Patch fixes that issue. If component reports hole offset at
component end then it is saved to be used as result when no
other components report valid hole offset.
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I8bffd8b76546b560a3d34034701eea4963f41593
Reviewed-on: https://review.whamcloud.com/40713
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Sebastien Buisson [Fri, 4 Dec 2020 08:22:04 +0000 (09:22 +0100)]
LU-14181 tests: except sanity test_64e 64f with SHARED_KEY
Add sanity test_64e and test_64f to ALWAYS_EXCEPT when
SHARED_KEY is used.
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaa9f5038a59f9ddc50dd9ac81ca81effd8bb9b1b
Reviewed-on: https://review.whamcloud.com/40865
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 13 Nov 2020 10:36:14 +0000 (19:36 +0900)]
LU-14121 nodemap: do not force fsuid/fsgid squashing
In the current implementation, if the real uid is squashed, then the
fsuid is similarly squashed, no matter what is the value of the
effective uid.
This squashing is a little bit too strict, and we should instead trust
mapped fsuid and fsgid values.
Also add euid_access test program and sanity-sec test_55 to verify
the issue is fixed.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iecaecac5054b105cd42206b0a9a3868cde0269b4
Reviewed-on: https://review.whamcloud.com/40645
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 17 Nov 2020 16:13:08 +0000 (17:13 +0100)]
LU-14095 ssk: default rounds of Miller-Rabin for DH_check
OpenSSL 1.1.1c increased the number of rounds used for Miller-Rabin
testing of the prime provided as input parameter to DH_check(). This
makes the check roughly x10 longer, and can lead to request timeouts
when an SSK flavor is being used.
Instead, use a dynamic number of rounds based on the speed of the
check, evaluated when the lsvcgssd daemon starts. If DH_check()
runtime is fine, just use it instead of our own check.
Test-Parameters: clientdistro=el8.2 serverdistro=el8.2 testgroup=review-dne-ssk
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id392cdd76ede196094b146c68d230bc52852aa34
Reviewed-on: https://review.whamcloud.com/40686
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Serguei Smirnov [Tue, 24 Nov 2020 19:59:46 +0000 (14:59 -0500)]
LU-12901 o2iblnd: retry qp creation with reduced queue depth
If negotiated number of frags * queue depth is too large for
successful qp creation, reduce the queue depth in a loop
until qp creation succeeds or the queue depth dips below 2.
Remember the reduced queue depth value to use for later
connections to the same peer.
Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Iaa91510d6f80d813218a06a9bc52f5f9251e8b87
Reviewed-on: https://review.whamcloud.com/40748
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Wed, 9 Dec 2020 20:29:02 +0000 (15:29 -0500)]
New tag 2.13.57
Change-Id: I931c35e61fe13760586d3402afa729bce029384e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Bobi Jam [Fri, 16 Oct 2020 11:44:25 +0000 (19:44 +0800)]
LU-14042 llite: don't check layout info for page discard
The CIT_MISC+ignore_layout is indicating locks/pages manipulation
from the OSC layer, it does not care/access lov layout related info.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ibdf22f0d47712d370385af4c6c984052b91fc7c6
Reviewed-on: https://review.whamcloud.com/40267
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Jian Yu [Mon, 7 Dec 2020 22:31:56 +0000 (14:31 -0800)]
LU-13839 kernel: RHEL 8.3 server support
This patch makes changes to support RHEL 8.3 release with
kernel 4.18.0-240.1.1.el8_3 for Lustre server.
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.3 serverdistro=el8.3 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.3 serverdistro=el8.3 testlist=sanity
Change-Id: I2aa2efaf86b5f244fcb1dd95be1d0bfe04e8d245
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40729
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 2 Dec 2020 00:07:09 +0000 (16:07 -0800)]
LU-14170 build: Update ZFS version to 2.0.0
Major New Features
* Sequential resilver
* Persistent L2ARC
* ZStandard compression
* Redacted zfs send/receive
Notable Changes
* Added fallocate(mode-0/2) compatibility to preallocate space.
* Reorganized the zfs and zpool man pages by splitting out each
subcommand in to its own page.
* Enabled the systemd zfs-mount-generator by default on Linux.
* More relevant and useful ZED syslog entries.
* Provided pam module for automatically loading zfs encryption
keys for home datasets.
* Support for inheriting and setting user properties in channel
programs.
* Improved bootloader support.
* Optionally colorized zpool status output.
Change-Id: Ia00d46a4cb54be94d4fdc0145a2d474ea987cf74
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40822
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Fri, 4 Dec 2020 02:44:25 +0000 (10:44 +0800)]
LU-13826 utils: fix compatibility for LL_IOC_MDC_GETINFO
The landed patch "LU-11367 som: integrate LSOM with lfs find"
uses "LL_IOC_MDC_GETINFO_OLD", so while the IOCTL number/structs
are ABI compatible, it is not API compatible and applications
using for the header including the definition LL_IOC_MDC_GETINFO
is broken.
This patch defines versioned IOCTL number: LL_IOC_MDC_GETINFO_V1,
LL_IOC_MDC_GETINFO_V2. Then we can use the explicitly verioned
constrants everywhere for the in-tree code, and declare
LL_IOC_MDC_GETINFO in a compatible way, but external applications
can select the version that they want explicitly.
And this patch does the same fix for IOC_MDC_GETFILEINFO.
Fixes:
11aa7f8704c4 ("LU-11367 som: integrate LSOM with lfs find")
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I39e0273c4a7331ec782de3e8686b8a756e56beec
Reviewed-on: https://review.whamcloud.com/40858
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andriy Skulysh [Sun, 29 Mar 2020 13:15:29 +0000 (16:15 +0300)]
LU-13994 ldlm: osc_object_ast_clear() is called for mdc object on eviction
Replace osc_object_prune() with cl_object_prune()
PID: 3477 TASK:
ffff9360d82fa0e0 CPU: 0 COMMAND: "ll_imp_inval"
#0 [
ffff9360d5c5b990] machine_kexec at
ffffffff86865704
#1 [
ffff9360d5c5b9f0] __crash_kexec at
ffffffff869209a2
#2 [
ffff9360d5c5bac0] panic at
ffffffff86f7294c
#3 [
ffff9360d5c5bb40] lbug_with_loc at
ffffffffc04b78cb [libcfs]
#4 [
ffff9360d5c5bb60] osc_object_ast_clear at
ffffffffc0956471 [osc]
#5 [
ffff9360d5c5bbc8] ldlm_resource_foreach at
ffffffffc07e2fd6 [ptlrpc]
#6 [
ffff9360d5c5bc08] ldlm_resource_iterate at
ffffffffc07e3266 [ptlrpc]
#7 [
ffff9360d5c5bc38] osc_object_prune at
ffffffffc0956140 [osc]
#8 [
ffff9360d5c5bc58] osc_object_invalidate at
ffffffffc0956e12 [osc]
#9 [
ffff9360d5c5bcd0] osc_ldlm_resource_invalidate at
ffffffffc09477bf [osc]
Change-Id: I83fd02feb3f5559dfe7b93647429f8eb57c7ad0e
HPE-bug-id: LUS-8399
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/40052
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Dominique Martinet [Fri, 31 Aug 2018 09:03:36 +0000 (18:03 +0900)]
LU-10262 mdt: mdt_reint_open: check EEXIST without lock
Many applications blindly open files with O_CREAT, and the mds gets a
write lock to the parent directory for these even if the file already
exists.
Checking for file existence first lets us take a PR lock if file
already existed even if O_CREAT was specified.
This opens up multiple races between the first lookup and the actual
locking, in each of them drop the resources we aquired and retry from
scratch to keep things as far from complicated as possible, with mixed
success.
Update (eaujames):
- rebase the patch
- update tests
Performance tests results:
The array below presents the average "open" syscall latency for 20
files in a single directory accessed by 400 different clients.
_______________________________________________________________
| Test cases | without patch | with patch | %improvement |
|___________________|_______________|____________|______________|
| readonly | 0.960s | 0.973s | -1.40% |
|___________________|_______________|____________|______________|
| readonly cached | 0.372s | 0.372s | +0.01% |
|___________________|_______________|____________|______________|
| O_CREAT+precreate | 1.645s | 0.968s | +41.13% |
|___________________|_______________|____________|______________|
| O_CREAT cached | 0.632s | 0.623s | +1.34% |
|___________________|_______________|____________|______________|
| O_CREAT | 1.261s | 1.093s | +13.32% |
|___________________|_______________|____________|______________|
(for more detail, see the ticket comments section)
This patch optimizes concurent opens with O_CREAT flag when dentry are
not cached by clients.
Change-Id: I247b579d14d20036f89033c99ece457d70ba19e7
Signed-off-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-on: https://review.whamcloud.com/33098
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Thu, 3 Dec 2020 21:07:01 +0000 (05:07 +0800)]
LU-14172 lmv: optimize dir shard revalidate
mdt_is_remote_object() will check whether child is directory shard
if parent and child are on different MDTs, which needs to read LMV
from disk, and hurt striped directory stat performance.
This can be optimized, client can just set CROSS_REF flag to do a
cross reference getattr, which avoids lots of checks.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib2d5a510b27c90a26f979f9cccfd40948e32d91a
Reviewed-on: https://review.whamcloud.com/40863
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Yang Sheng [Fri, 27 Nov 2020 14:57:51 +0000 (22:57 +0800)]
LU-14155 mdc: avoid easize set to 0
The cl_default_mds_easize could be set to 0 in some case. Then
check it before package.
Fixes:
3ec712bd183a ("LU-11868 osd: Set max ea size to XATTR_SIZE_MAX")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Icc045bfba345a61dd7904bfc7772930fd737334e
Reviewed-on: https://review.whamcloud.com/40785
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Wed, 25 Nov 2020 23:20:19 +0000 (16:20 -0700)]
LU-5855 tests: enable skipped sanity-lfsck DNE ZFS tests
LFSCK has been enabled for DNE striped directories on ZFS for
some time, but the skipped tests have never been enabled for
this configuration. Re-enable them for ZFS backends.
Fixes:
26995a3588e7 ("LU-5855 lfsck: misc fixes for zfs-based backend")
Test-Parameters: trivial fstype=zfs testlist=sanity-lfsck
Test-Parameters: fstype=zfs mdscount=2 mdtcount=4 testlist=sanity-lfsck
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I33be173c98faa61311ab2b435555cf49e1e2bacc
Reviewed-on: https://review.whamcloud.com/40761
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Fri, 13 Nov 2020 16:17:37 +0000 (16:17 +0000)]
LU-14073 ptlrpc: remove debugging assert using segment_eq()
In Linux 5.9-rc1 commit
428e2976a5, segment_eq() is removed as it was
only used in uaccess_kernel(). But uaccess_kernel() was introduced in Linux
v4.12-rc1, which does not cover all supported kernel versions.
As segment_eq() is only used in 1 actual debugging call, avoid adding a
new configuration test for uaccess_kernel() and just remove the 2
related LASSERTs.
Test-Parameters: trivial
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: I1595e9b085176d2106a2b958e1fd6f4031cc2f69
Reviewed-on: https://review.whamcloud.com/40797
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Sebastien Piechurski [Fri, 15 May 2020 18:22:44 +0000 (20:22 +0200)]
LU-14159 build: fix gcc8 warnings on kthread_run calls
Fixes warnings from gcc8 about kthread_run calls with no format
parameter.
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Signed-off-by: Sebastien Piechurski <sebastien.piechurski@atos.net>
Change-Id: I30a68a1d0e318c98f2da547ece8018564ced69c0
Reviewed-on: https://review.whamcloud.com/40790
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Fri, 27 Nov 2020 07:11:27 +0000 (10:11 +0300)]
LU-14154 ldiskfs: gcc10 support
gcc10 complains about uninitialized fields:
ldiskfs/extents.c: In function ‘ldiskfs_ext_map_blocks’:
ldiskfs/ldiskfs_extents.h:214:12: error: 'zero_ex2.ee_start_hi' may
be used uninitialized in this function [-Werror=maybe-uninitialized]
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I17ef56352194f6ad76bf6f08dab0be3e28d0323c
Reviewed-on: https://review.whamcloud.com/40776
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
John L. Hammond [Fri, 16 Oct 2020 18:37:36 +0000 (13:37 -0500)]
LU-14043 lfs: use buffered IO in mirror extend
In lfs_mirror_extend() use buffered IO to read from the source
file. Mirror resync will continue to use direct IO on the source.
Call posix_fadvise() at the end of mirroring to discard cached data
from the old and new mirrors.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I7703ae9e30cf30a7e6fbf274114d5986a80d4d3a
Reviewed-on: https://review.whamcloud.com/40275
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 21 Aug 2020 19:27:07 +0000 (14:27 -0500)]
LU-13569 lnet: Add lnet_recovery_limit to lnetctl
Allow setting/reading lnet_recovery_limit via lnetctl.
Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I5aac297bad15e43a52d8b8531da08a1d3f559bea
Reviewed-on: https://review.whamcloud.com/39717
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 21 Aug 2020 18:33:12 +0000 (13:33 -0500)]
LU-13569 lnet: Introduce lnet_recovery_limit parameter
This parameter controls how long LNet will attempt to recover an
unhealthy interface.
Defaults to 0 to indicate indefinite recovery. This maintains the
current behavior.
Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I2f7897d9a293f0979f7402de2b91e160c77790d1
Reviewed-on: https://review.whamcloud.com/39716
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Sun, 28 Jun 2020 08:35:07 +0000 (16:35 +0800)]
LU-13669 llite: try to improve mmap performance
We have observed slow mmap read performances for some
applications. The problem is if access pattern is neither
sequential nor stride, but could be still adjacent in a
small range and then seek a random position.
So the pattern could be something like this:
[1M data] [hole] [0.5M data] [hole] [0.7M data] [1M data]
Every time an application reads mmap data, it may not only
read a single 4KB page, but aslo a cluster of nearby pages in
a range(e.g. 1MB) of the first page after a cache miss.
The readahead engine is modified to track the range size of
a cluster of mmap reads, so that after a seek and/or cache miss,
the range size is used to efficiently prefetch multiple pages
in a single RPC rather than many small RPCs.
Benchmark:
fio --name=randread --directory=/ai400/fio --rw=randread
--ioengine=mmap --bs=128K --numjobs=32 --filesize=200G
--filename=randread --time_based --status-interval=10s
--runtime=30s --allow_file_create=1 --group_reporting
--disable_lat=1 --disable_clat=1 --disable_slat=1
--disk_util=0 --aux-path=/tmp --randrepeat=0
--unique_filename=0 --fallocate=0
| master | patched | speedup |
---------------+-----------+------------+-----------+
page_fault_avg | 512usec | 52usec | 9.75x
page_fault_max | 37698usec| 6543usec| 5.76x
Change-Id: I86436cbd027ec6df5094599e54b4acfd0e018930
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38916
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Mon, 4 May 2020 10:57:33 +0000 (18:57 +0800)]
LU-13368 lnet: discard the callback
Lustre need a completion callback for event that request
has been sent. And then need other callback when reply
arrived. Sometime the request completion callback maybe
lost by some reason even reply has been received.
system will wait forever even timeout. We needn't to wait
request completion in such case. So provide a way to
discard the callback.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: If9cd8420ee76947ee5053180e0f5219f76bb94c2
Reviewed-on: https://review.whamcloud.com/38845
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Fri, 24 Jan 2020 12:12:54 +0000 (15:12 +0300)]
LU-13170 osd: positive lookup shouldn't return an error
because of stale OI, otherwise LFSCK has no chance to
fix direntry. lookup should initiate OI scrubber and
give it a hint about FID->dnode mapping.
This should fix sanity-lfsck/23b failing with ZFS often.
Test-Parameters: fstype=zfs testlist=sanity-lfsck env=ONLY=23b,ONLY_REPEAT=200
Change-Id: Iea826de9995e34939eff2d67a831deb6b192bd4d
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37322
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Wang Shilong [Wed, 6 Nov 2019 13:01:18 +0000 (21:01 +0800)]
LU-9555 quota: df should return projid-specific values
With local ext4 and XFS filesystems, it is possible to use
"df /path/to/directory" (statfs()) to return the current
project quota usage for that directory as "used", and
min(projid quota limit, free space) as "total".
statfs() is a natural interface for users/applications, since
it represents the used/maximum space for that subdirectory.
Otherwise, the user will get EDQUOT back when the project
quota runs out for that directory and applications will not
be able to figure out how much data they could write into
that directory.
Change-Id: I7f357e6e4de6a1465a48c2c08c4aa86c4c848dbb
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36685
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Mikhail Pershin [Wed, 25 Nov 2020 19:45:38 +0000 (22:45 +0300)]
LU-13543 lustre: update *pos in seq_file .next functions
Patch fixes mdt_hsm_actions_debugfs seq operations to work with
new seq file logic in newer kernels
Meanwhile it finishes remaining work originating in LU-13985 which
updates ->next functions to always increment *pos as it is expected
by kernel
Test-Parameters: clientdistro=el8.2 serverdistro=el8.2 testlist=sanity-hsm
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I4f077b633fbf7d420bfda10cce6b0141cef07761
Reviewed-on: https://review.whamcloud.com/40757
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Andreas Dilger [Tue, 24 Nov 2020 07:35:17 +0000 (00:35 -0700)]
LU-13980 osd-ldiskfs: print label instead of device
Print out the volume label (e.g. fsname-OST0000) in error messages
instead of the device name (e.g. dm-2) so that it is easier to
correlate this with other messages in the log.
Fix error strings to be on one line to match code style.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iff31bc962ca32b1f3a13f579700d03835a3ebbe5
Reviewed-on: https://review.whamcloud.com/40738
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 24 Nov 2020 09:34:00 +0000 (12:34 +0300)]
LU-14140 osd: don't panic meeting OI dups
instead dump all info (FID->ino, LMAs) and return an error
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I4c0c7ad12cc808617c748b0a720323a968668fc9
Reviewed-on: https://review.whamcloud.com/40666
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Thu, 12 Nov 2020 09:38:15 +0000 (17:38 +0800)]
LU-14134 osd-ldiskfs: reduce credits for new writing
For new writing, Lustre reserve credits in worst cases,
eg for 1M write, we need reserve more than 500 credits.
This means we need much more journal space than data
blocks for each new writing thus a large journal needed
to reach good performance.
This patch try to improve the case by allowing transaction
to be restarted, only reserve few extent credits during
declare phase. Before every block allocation, one extent
insertion credits must be guaranteed, otherwise transaction
will be restarted.
A simple dd test of following in 10G device shows credits
removed from 260+ to 7 credits:
$ dd if=/dev/zero of=data bs=1M count=1 oflag=direct
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ide541cbd72d8d4aa286eb8b84ca5ce09c3b5f8b6
Reviewed-on: https://review.whamcloud.com/40626
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Wed, 11 Nov 2020 20:02:55 +0000 (04:02 +0800)]
LU-14130 osc: skip 0 row for rpc_stats
Fix the rpc_stats statistic it should not print
0 row as it makes nosense.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Id3534bb43b5453daeed4db078c18912c4cc2f51e
Reviewed-on: https://review.whamcloud.com/40613
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Lai Siyao [Tue, 3 Nov 2020 02:17:36 +0000 (10:17 +0800)]
LU-14105 lfsck: don't LBUG() on disk data
lfsck_namespace_assitant_handler_p1() will LBUG() if file FID is
zero, and file name is not "..", but generally lfsck shouldn't LBUG()
on disk data. The correct way is to treat such file as bad dirent and
remove it, which is done below.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia4092ada5d55583508f276e5c9d59920ec31a33b
Reviewed-on: https://review.whamcloud.com/40524
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Mon, 2 Nov 2020 06:46:42 +0000 (14:46 +0800)]
LU-14043 llite: allow DIO with unaligned IO count
DIO only require user buffer page aligned and
IO offset page aligned, it is ok that io count is
not page aligned, remove this unnecessary limit
so that we could use DIO with file not aligned
with PAGE SIZE.
Change-Id: Iaf1e224b3f180f91d1b54016c6e6b7adaf674bf8
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/40392
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 27 Apr 2020 11:44:53 +0000 (13:44 +0200)]
LU-13474 gss: do not return -ERESTART when gss rpc times out
When a client receives an RPC time out while it tries to negotiate
a GSS context with the servers, it must not return -ERESTART.
Otherwise, the client would keep on sending GSS negotiation requests
to the same peer, instead of letting the HA failover mechanism try
different service nodes.
Test-Parameters: trivial
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ifc9cc02ea3ddefe6c5095cbb464a054c9027c920
Reviewed-on: https://review.whamcloud.com/38310
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andrew Perepechko [Wed, 11 Jul 2018 13:32:33 +0000 (16:32 +0300)]
LU-12353 ldiskfs: speedup quota journalling
ldiskfs_mark_dquot_dirty() need not call
ldiskfs_write_dquot() if the dquot is already
dirty since we are guaranteed that another
thread is already writing the dquot and
will update the buffer with the latest
dquot data.
Change-Id: Ia65c5987bbb0d6d2ac94c5499cb8b51edc3c49c3
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-5342
Reviewed-on: https://review.whamcloud.com/34992
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Mon, 2 Nov 2020 14:27:00 +0000 (23:27 +0900)]
LU-14095 gss: use hlist_unhashed() instead of ->next
In cache_detail list-mutation primitives, verifying the status of an
entry must be done using hlist_unhashed(), in case 'struct cache_head'
has a 'cache_list' field.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1410eca9a647b74127cf40b8f3d6b68d055f773a
Reviewed-on: https://review.whamcloud.com/40514
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Lai Siyao [Sat, 7 Sep 2019 09:38:46 +0000 (17:38 +0800)]
LU-12960 lod: don't set index for 2nd stripe if specific
When MDTs are specific, don't set index for the second stripe
allocation, otherwise it's not created on specific MDT.
Add sanity 31q.
Fixes:
c1d0a355a6 ("LU-12624 lod: alloc dir stripes by QoS")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I415be0fa7c6e702ec72dcaac88bba55290463d44
Reviewed-on: https://review.whamcloud.com/36735
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 11 Sep 2020 19:13:32 +0000 (14:13 -0500)]
LU-13571 lnd: Use NETWORK_TIMEOUT for some conn failures
For -EHOSTUNREACH and -ETIMEDOUT we cannot tell whether the
connnection failure was due to a problem with the remote or local NI,
so we should return the LNET_MSG_STATUS_NETWORK_TIMEOUT to LNet in
these cases.
HPE-bug-id: LUS-9342
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I9036f5698061ce74fe29683b8249b8f9a05f3433
Reviewed-on: https://review.whamcloud.com/39900
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 11 Sep 2020 18:42:42 +0000 (13:42 -0500)]
LU-13571 lnd: Use NETWORK_TIMEOUT for txs on ibp_tx_queue
TXs on the ibp_tx_queue are waiting for a connection to be
established. Failure to establish a connection could be due to a
problem with either the local NI or the remote NI, and o2iblnd cannot
currently distinguish between these failures. As such, it should
return LNET_MSG_STATUS_NETWORK_TIMEOUT to LNet so that LNet will
decrement the health value of both the local NI and the remote NI and
future sends can take these health values into account.
Test-Parameters: trivial
HPE-bug-id: LUS-9342
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Idbbbe95483d25ec48b83e33a00685f72fa5292e6
Reviewed-on: https://review.whamcloud.com/39899
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Thu, 1 Oct 2020 05:51:55 +0000 (01:51 -0400)]
LU-12564 libcfs: Use vfree_atomic instead of vfree
Since vfree is unsafe to use in atomic context, implement our own
libcfs_vfree_atomic heavily based on code from linux 4.10 commit
bf22e37a641327e34681b7b6959d9646e3886770
We can't use the one in the kernel because it's not exported.
unconditionally use it in *_FREE_LARGE() macros since in_atomic()
is not recommended to be used outside of core kernel code.
Not everything is present on 3.10 (rhel7) so we also add
llist primitive and a replacement for raw_cpu_ptr there.
Change-Id: I50892f231e54a284f4d8a14d910ea9ab2fbe6a16
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40136
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: Maloo <maloo@whamcloud.com>
Oleg Drokin [Mon, 12 Oct 2020 20:12:55 +0000 (16:12 -0400)]
LU-14024 ofd: Avoid use after free in ofd_inconsistency_verification_main
The ofd_inconsistency_lock should not be unlocked after we woken up
a different thread that is going to free the structure containing
said lock.
Change-Id: I913e7470664e1128a250597b0a803f791d99099e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40222
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Wang Shilong [Sat, 24 Oct 2020 01:47:23 +0000 (09:47 +0800)]
LU-14072 llite: fix client evicition with DIO
We set lockless in file open if O_DIRECT flag is passed,
however O_DIRECT flag could be cleared by
fcntl(..., F_SETFL, ...).
Finally we comes to a case where buffer IO without lock
held properly, and hit hang:
[<
ffffffffc0d421ed>] osc_extent_wait+0x21d/0x7c0 [osc]
[<
ffffffffc0d44897>] osc_cache_wait_range+0x2e7/0x940 [osc]
[<
ffffffffc0d4585e>] osc_cache_writeback_range+0x96e/0xff0 [osc]
[<
ffffffffc0d31c45>] osc_lock_flush+0x195/0x290 [osc]
[<
ffffffffc0d31d7c>] osc_lock_lockless_cancel+0x3c/0xe0 [osc]
[<
ffffffffc081f488>] cl_lock_cancel+0x78/0x160 [obdclass]
[<
ffffffffc0cd8079>] lov_lock_cancel+0x99/0x190 [lov]
[<
ffffffffc081f488>] cl_lock_cancel+0x78/0x160 [obdclass]
[<
ffffffffc081f9a2>] cl_lock_release+0x52/0x140 [obdclass]
[<
ffffffffc08238a9>] cl_io_unlock+0x139/0x290 [obdclass]
[<
ffffffffc08242e8>] cl_io_loop+0xb8/0x200 [obdclass]
[<
ffffffffc0e1d36b>] ll_file_io_generic+0x91b/0xdf0 [lustre]
[<
ffffffffc0e1dd0c>] ll_file_aio_write+0x29c/0x6e0 [lustre]
[<
ffffffffc0e1e250>] ll_file_write+0x100/0x1c0 [lustre]
[<
ffffffffa984aa90>] vfs_write+0xc0/0x1f0
[<
ffffffffa984b8af>] SyS_write+0x7f/0xf0
[<
ffffffffa9d8eede>] system_call_fastpath+0x25/0x2a
[<
ffffffffffffffff>] 0xffffffffffffffff
Lock cancel time out in the server side and client
eviction happen.
Fix this problem by testing O_DIRECT flag to decide if
we could issue lockless IO.
Fixes:
6bce536725 ("LU-4198 clio: turn on lockless for some kind of IO")
Change-Id: Idbf1c748684a6540aee5f6e35c017929fbcc60b9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/40389
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Fri, 23 Oct 2020 13:43:07 +0000 (16:43 +0300)]
LU-14071 doc: lfs setquota/quota doc for PQ
Add description of key --pool for lfs setquota
and lfs quota.
Change-Id: Ie6bfb3240f7eb8b4239d95aac29989103f009578
Test-Parameters: trivial
HPE-bug-id: LUS-9341
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/40384
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Minh Diep [Wed, 14 Oct 2020 23:57:51 +0000 (16:57 -0700)]
LU-14036 build: fix lbuild for MOFED 5.1
Starting MOFED 5.1, rdma-core is required for libib*mad
Test-Parameters: trivial
Change-Id: Id26f3cdb0552933577e1b27384ac82f9f48e2b3a
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40254
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log