Whamcloud - gitweb
fs/lustre-release.git
3 months agoLU-13746 utils: check argument logname presence in llog_print 63/39263/3
Etienne AUJAMES [Fri, 3 Jul 2020 08:17:29 +0000 (10:17 +0200)]
LU-13746 utils: check argument logname presence in llog_print

Correction of segfault in llog_print when no logname specified.
example: lctl --device MGS llog_print

Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I03a6c08dfc73ff8cb5861d162e21ce5aa581e197
Reviewed-on: https://review.whamcloud.com/39263
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
3 months agoLU-13159 osd: osd-zfs to release index back structures 93/37293/12
Alex Zhuravlev [Mon, 20 Jan 2020 17:12:58 +0000 (20:12 +0300)]
LU-13159 osd: osd-zfs to release index back structures

otherwise those can be leaked in case of failed mount

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I7dc848a8397c11b1f56d36ef0a7155314ca9afc2
Reviewed-on: https://review.whamcloud.com/37293
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
3 months agoLU-14125 obdclass: add grant fields to export procfile 63/40563/4
Olaf Faaland [Thu, 2 Jul 2020 21:25:32 +0000 (14:25 -0700)]
LU-14125 obdclass: add grant fields to export procfile

Add ted_{grant,reserved,dirty} to the export
procfile for OSTs, to allow comparison between the
OST's idea of grants allocated to the client with
the client's idea.

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: Ib34582e2be55fe2007363b52cea4dee211b7f540
Reviewed-on: https://review.whamcloud.com/40563
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14031 tests: check client reconnect 97/40297/7
Alexander Boyko [Mon, 19 Oct 2020 13:07:10 +0000 (09:07 -0400)]
LU-14031 tests: check client reconnect

The patch adds test 147 for recovery-small. It checks how often
client reconnects to a server with obd_timeout=200.

HPE-bug-id: LUS-8520
Test-Parameters: trivial testlist=recovery-small env=ONLY=147
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Id5bd732fe4d949cfa45ea0272f197809cca3290d
Reviewed-on: https://review.whamcloud.com/40297
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13571 lnet: Correct handling of NETWORK_TIMEOUT status 98/39898/14
Chris Horn [Fri, 11 Sep 2020 18:41:39 +0000 (13:41 -0500)]
LU-13571 lnet: Correct handling of NETWORK_TIMEOUT status

The original intent of the LNET_MSG_STATUS_NETWORK_TIMEOUT health
status was to handle cases where the LND was unsure whether the
failure was due to the local or remote NI. In this case, we'll want
to decrement both the local and remote NI health and allow recovery
to ascertain which interface is actually healthy.

Test-Parameters: trivial
HPE-bug-id: LUS-9342
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ib00ac260640100123e4e97e9c566289e92fb0b6e
Reviewed-on: https://review.whamcloud.com/39898
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14027 ldlm: Do not hang if recovery restarted during lock replay 38/40238/4
Oleg Drokin [Wed, 14 Oct 2020 03:55:02 +0000 (23:55 -0400)]
LU-14027 ldlm: Do not hang if recovery restarted during lock replay

LU-13600 introduced lock ratelimiting logic, but it did not take
into account that if there's a disconnection in the REPLAY_LOCKS
phase then yet unsent locks get stuck in the sending queue so
the replay locks thread hangs with imp_replay_inflight elevated
above zero.

The direct consequence from that is recovery state machine never
advances from REPLAY to REPLAY_LOCKS status when imp_replay_inflight
is non zero.

Adjust __ldlm_replay_locks() to check if the import state changed
before attempting to send any more requests.

Add a testcase.

Change-Id: Idbaf5461f33d1884088269d67d01071c7e1bf8a5
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Fixes: 3b613a442b ("LU-13600 ptlrpc: limit rate of lock replays")
Reviewed-on: https://review.whamcloud.com/40238
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
3 months agoLU-13195 obdclass: show FID for corrupted llog 77/38977/2
Alex Zhuravlev [Thu, 18 Jun 2020 07:00:26 +0000 (10:00 +0300)]
LU-13195 obdclass: show FID for corrupted llog

to be able to remove that easily.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iefead860b5e60e74e0eb445f715508a3b01fac87
Reviewed-on: https://review.whamcloud.com/38977
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14053 utils: detect hangup in ofd_access_log_reader 11/40311/7
John L. Hammond [Thu, 24 Sep 2020 21:05:39 +0000 (16:05 -0500)]
LU-14053 utils: detect hangup in ofd_access_log_reader

In ofd_access_log_reader, when the batch file is a pipe or socket,
detect hangups and exit.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: If7509174b992011305d8e4a7aa2766a3a1980831
Reviewed-on: https://review.whamcloud.com/40311
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14052 ofd: support for multiple access readers 06/39906/15
Alex Zhuravlev [Tue, 15 Sep 2020 12:42:17 +0000 (15:42 +0300)]
LU-14052 ofd: support for multiple access readers

ofd_access_log_reader can be passed -I, --mdt-index-filter=INDEX to
print only FIDs located on INDEX.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9a4f09c6b7ca15623d459df17939895301a57a8b
Reviewed-on: https://review.whamcloud.com/39906
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14009 osd: missing ldiskfs_htree_unlock() 37/40137/4
Alex Zhuravlev [Sun, 4 Oct 2020 10:41:26 +0000 (13:41 +0300)]
LU-14009 osd: missing ldiskfs_htree_unlock()

in osd_ldiskfs_it_fill()

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5d2242e0864cbaa72af096b263d8758966a6be22
Reviewed-on: https://review.whamcloud.com/40137
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14027 ldlm: Do not wait for lock replay sending if import dsconnected 72/40272/4
Oleg Drokin [Fri, 16 Oct 2020 14:25:58 +0000 (10:25 -0400)]
LU-14027 ldlm: Do not wait for lock replay sending if import dsconnected

If import disconnected while we were preparing to send some lock replays
the sending RPC would get stuck on the sending list and would keep
the reconnected import state from progressing from REPLAY
to REPLAY_LOCKS state waiting for the queued replay RPCs to finish.

Set them as no_delay to ensure they don't wait.

LU-13600 exacerbated this issue some but it certainly exist
before it as well.

Change-Id: Id276a0be7657d9ad6cf40ad8e7a165d5cd341cb8
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40272
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
3 months agoLU-14061 utils: prefer mounting with specified fstype 74/40474/3
Andreas Dilger [Thu, 29 Oct 2020 20:47:40 +0000 (14:47 -0600)]
LU-14061 utils: prefer mounting with specified fstype

If the server filesystem is mounted with "-t lustre" use that type
for mounting instead of automatically selecting "lustre_tgt" for
server mounts, so that it is appearing in /proc/mounts correctly,
and "mount -t lustre" will list all of these filesystems.

Only if "mount -t lustre_tgt" is used will mount.lustre_tgt be called,
and that will internally use "lustre_tgt" as the filesystem type, and
only fall back to type "lustre" if that does not work.  This will give
userspace plenty of time to transition to using "lustre_tgt" for
server mountpoints.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifd7560c800acdfe14c9564bbf955ecad1224f2e3
Reviewed-on: https://review.whamcloud.com/40474
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-12632 hsm: wait longer in sanity-hsm test_90() 87/40387/6
John L. Hammond [Wed, 28 Oct 2020 15:30:04 +0000 (10:30 -0500)]
LU-12632 hsm: wait longer in sanity-hsm test_90()

Due to the huge (51) number of HSM requests to be completed we must
wait longer for this test to pass when run using an advanced (zfs)
backend filesystem.

Test-Parameters: trivial fstype=zfs env=ONLY=90 testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ia20086bf5c072c5d120eed5b0937d37d7b4342db
Reviewed-on: https://review.whamcloud.com/40387
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14109 doc: lnetctl manpage missing import and export options 23/40523/3
Olaf Faaland [Tue, 3 Nov 2020 01:38:49 +0000 (17:38 -0800)]
LU-14109 doc: lnetctl manpage missing import and export options

Describe missing options in the lnetctl man page:
lnetctl import "--exec"
lnetctl export "--backup"

Test-Parameters: trivial
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: I47e6c5e264dc8d5673f1229291be873996c02f55
Reviewed-on: https://review.whamcloud.com/40523
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14102 tests: add a pause in open-close in sanity/160l 08/40508/3
Alex Zhuravlev [Sun, 1 Nov 2020 11:04:54 +0000 (14:04 +0300)]
LU-14102 tests: add a pause in open-close in sanity/160l

so that close has to update mtime and generate CL_MTIME
record in the changelog.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I118172229c86ed5c201299de7476678689bf4cab
Reviewed-on: https://review.whamcloud.com/40508
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13683 lfs: return -ENOENT when invoked on non-existing file 53/38953/6
Sebastien Piechurski [Tue, 16 Jun 2020 16:14:55 +0000 (18:14 +0200)]
LU-13683 lfs: return -ENOENT when invoked on non-existing file

Since merge of LU-11510, lfs migrate on a non-existing file will give
the following error "lfs migrate: can't create composite layout from
file /some/path/to/file" and will exit with code 0, potentially
leaving a calling script unaware of the error.

This patch fixes it by using errno which is set in the call to
llapi_layout_get_by_path()

Signed-off-by: Sebastien Piechurski <sebastien.piechurski@atos.net>
Change-Id: I910eae78445f6071ff4e741afd43d140f554ab22
Fixes: 8bedfa377fbd ("LU-11510 lfs: migrate a composite layout file correctly")
Reviewed-on: https://review.whamcloud.com/38953
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
3 months agoLU-14115 lod: fix to set inherit flag for stripe directroy 43/40543/3
Wang Shilong [Thu, 5 Nov 2020 02:05:14 +0000 (10:05 +0800)]
LU-14115 lod: fix to set inherit flag for stripe directroy

lod_xattr_set_lmv() is called to set attributes from parent
to all stripes of the directroy, however, flags are ignored
which will cause project inherit bits missed for stripes.

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I10a2de303b4b8430e560752cf0d66466c93616a4
Reviewed-on: https://review.whamcloud.com/40543
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-12956 ldlm: fix hrtimer using 13/40513/3
Alexander Boyko [Mon, 2 Nov 2020 11:02:47 +0000 (06:02 -0500)]
LU-12956 ldlm: fix hrtimer using

A race could happen between hrtimer_start() and
hrtimer_expires_remaning(), cause the second one doesn't hold a lock
on timer->base. And a first one could change it between different CPU.
The following failure happened:
BUG: unable to handle kernel NULL pointer dereference at 000000000028
IP: [<ffffffffc0fc773f>] target_handle_connect+0x12ff/0x2b50 [ptlrpc]
at remaining = hrtimer_expires_remaining(timer), timer->base was NULL

The fix changes hrtimer_expires_remaining() to hrtimer_get_remaining()
which helds a lock and prevents race.

Fixes: 9334f1d51249 ("LU-11771 ldlm: use hrtimer for recovery to fix timeout messages")
HPE-bug-id: LUS-9514
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I2cea1e5e2d523f131f1acb3346cf0324adae624e
Reviewed-on: https://review.whamcloud.com/40513
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14077 kernel: kernel update SLES15 SP1 [4.12.14-197.64.1] 77/40477/3
Jian Yu [Tue, 3 Nov 2020 07:20:01 +0000 (23:20 -0800)]
LU-14077 kernel: kernel update SLES15 SP1 [4.12.14-197.64.1]

Update SLES15 SP1 kernel to 4.12.14-197.64.1 for Lustre client.

Test-Parameters: trivial \
env=SANITY_EXCEPT="56oc 100 130 136 817" \
clientdistro=sles15sp1 serverdistro=el7.8 \
testlist=sanity

Change-Id: If2d8ecd89b307c0e49cf39a7ef33c298a2690f27
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14083 build: Don't overwrite KBUILD_EXTRA_SYMBOLS 35/40435/3
Chris Horn [Thu, 15 Oct 2020 15:40:04 +0000 (10:40 -0500)]
LU-14083 build: Don't overwrite KBUILD_EXTRA_SYMBOLS

The gnilnd requires that KBUILD_EXTRA_SYMBOLS be populated with the
symbols for some of its dependencies. Don't overwrite
KBUILD_EXTRA_SYMBOLS for the kernel tests.

Test-Parameters: trivial
HPE-bug-id: LUS-9448
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ie10d3be0078e163f3e36f1c880d87ebc185c49c7
Reviewed-on: https://review.whamcloud.com/40435
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13988 mdt: ASSERTION(!lustre_handle_is_used(&lh->mlh_reg_lh)) failed 45/40045/3
Andriy Skulysh [Tue, 21 Apr 2020 10:13:15 +0000 (13:13 +0300)]
LU-13988 mdt: ASSERTION(!lustre_handle_is_used(&lh->mlh_reg_lh)) failed

mdt_brw_enqueue() should clear lock handle on error.

HPE-bug-id: LUS-8740
Change-Id: Ic2201189af6bcb2b1a114a599e138f9c22f3d99d
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/40045
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13822 ptlrpc: fixes for RCU-related stalls 14/39514/4
Andrew Perepechko [Mon, 27 Jul 2020 09:31:46 +0000 (12:31 +0300)]
LU-13822 ptlrpc: fixes for RCU-related stalls

ptlrpc_expired_set() may need to process a lot
of requests, so the processing loop needs to
schedule from time to time to avoid RCU-related
stalls.

Change-Id: I14b0aaf14ab127805699729adbb26459a2f4f07c
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
HPE-bug-id: LUS-8939
Reviewed-on: https://review.whamcloud.com/39514
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13709 tests: test lfs mkdir -c without -i 57/39457/5
Olaf Faaland [Thu, 16 Jul 2020 22:50:29 +0000 (15:50 -0700)]
LU-13709 tests: test lfs mkdir -c without -i

Almost every test with lfs mkdir -c in the test suite also
uses option -i, so lfs mkdir -c (same as -i -1, where lustre
chooses the MDTs) is poorly tested.  Add a test for that
case, sanity test_300s.

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: Iede537d52cf445c9c9a6353338670e55a11364da
Reviewed-on: https://review.whamcloud.com/39457
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-11290 ldlm: page discard speedup 27/39327/10
Alexander Zarochentsev [Sun, 31 May 2020 16:29:25 +0000 (19:29 +0300)]
LU-11290 ldlm: page discard speedup

Improving check_and_discard_cb, allowing to cache
negative result of dlm lock lookup and avoid
excessive osc_dlm_lock_at_pgoff() calls.

HPE-bug-id: LUS-6432
Change-Id: I253f944bf430b06d0e1a300d22e5d9b2e97412bf
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/39327
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13839 kernel: new kernel [RHEL 8.3 4.18.0-240.1.1.el8_3] 36/40536/3
Jian Yu [Fri, 6 Nov 2020 09:08:08 +0000 (01:08 -0800)]
LU-13839 kernel: new kernel [RHEL 8.3 4.18.0-240.1.1.el8_3]

This patch makes changes to support new RHEL 8.3 release
for Lustre client.

Test-Parameters: trivial clientdistro=el8.3

Change-Id: I06a46735b42ac258e576b1dd5c0beb17f4fd3e47
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40536
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13344 gss: Update crypto to use sync_skcipher 86/38586/13
Shaun Tancheff [Sun, 24 May 2020 19:29:41 +0000 (14:29 -0500)]
LU-13344 gss: Update crypto to use sync_skcipher

As of linux v4.19-rc2-66-gb350bee5ea0f the change
   crypto: skcipher - Introduce crypto_sync_skcipher

Enabled the deprecation of blkcipher which was dropped
as of linux v5.4-rc1-159-gc65058b7587f
    crypto: skcipher - remove the "blkcipher" algorithm type

Based on the existence of SYNC_SKCIPHER_REQUEST_ON_STACK
use the sync_skcipher API or provide wrappers for the
blkcipher API

Test-Parameters: testlist=sanity,recovery-small,sanity-sec mdscount=2 mdtcount=4 ostcount=8 clientcount=2 env=SHARED_KEY=true,SK_FLAVOR=skn
Test-Parameters: testlist=sanity,recovery-small,sanity-sec mdscount=2 mdtcount=4 ostcount=8 clientcount=2 env=SHARED_KEY=true,SK_FLAVOR=ska
Test-Parameters: testlist=sanity,recovery-small,sanity-sec mdscount=2 mdtcount=4 ostcount=8 clientcount=2 env=SHARED_KEY=true,SK_FLAVOR=ski
Test-Parameters: testlist=sanity,recovery-small,sanity-sec mdscount=2 mdtcount=4 ostcount=8 clientcount=2 env=SHARED_KEY=true,SK_FLAVOR=skpi
HPE-bug-id: LUS-8589
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I7683c20957213fd687ef5cf6dea64c842928db5b
Reviewed-on: https://review.whamcloud.com/38586
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-10059 tests: sanityn 32a error messages 96/40496/2
James Nunez [Fri, 30 Oct 2020 17:44:00 +0000 (11:44 -0600)]
LU-10059 tests: sanityn 32a error messages

There are three checks for file size in sanityn test 32a
that, if they fail, have the same error message
"wrong file size".  Let's add additional information to
the error messages to help distinguish the different errors.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I71aae8de67125d5be93c4fb4728b2c20d26df49c
Reviewed-on: https://review.whamcloud.com/40496
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Vikentsi Lapa <vlapa@whamcloud.com>
3 months agoLU-14093 ptlrpc: redefine uop_params_off to support gcc10 86/40486/2
Alex Zhuravlev [Fri, 30 Oct 2020 07:02:18 +0000 (10:02 +0300)]
LU-14093 ptlrpc: redefine uop_params_off to support gcc10

otherwise gcc10 complains about out-of-boundry access:

llog_swab.c: In function lustre_swab_update_ops:
llog_swab.c:137:46: error: array subscript 65534 is outside the bounds
of an interior zero-length array__u16[0] {aka short unsigned int[]}
[-Werror=zero-length-bounds]
  137 |    __swab16s(&uops->uops_op[i].uop_params_off[j]);

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I27981cbc79991cbd7a79cb90aec97bd1dc8b2f1b
Reviewed-on: https://review.whamcloud.com/40486
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14045 sec: fix O_DIRECT and encrypted files 95/40295/5
Sebastien Buisson [Mon, 19 Oct 2020 08:56:37 +0000 (08:56 +0000)]
LU-14045 sec: fix O_DIRECT and encrypted files

Sometimes, we can end up in a situation where
osc_release_bounce_pages() mistakenly consider pages as fscrypt
bounce pages, and tries to free them.
Fix the way we consider bounce pages by always setting the PageChecked
flag on them.
Also remove sanity test_426 from the ALWAYS_EXCEPT list.

Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=ubuntu2004 fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=ubuntu2004 fstype=zfs mdscount=2 mdtcount=4
Fixes: 728036f256 ("LU-12275 sec: O_DIRECT for encrypted file")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic54ae031c3b0baa17ffed8a6b6b90ff44f87ff58
Reviewed-on: https://review.whamcloud.com/40295
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14039 obdclass: set LA_TYPE when update_log init 64/40264/2
Yang Sheng [Thu, 15 Oct 2020 14:57:46 +0000 (22:57 +0800)]
LU-14039 obdclass: set LA_TYPE when update_log init

Fix for LBUG:
osp_md_create()) ASSERTION( attr->la_valid & LA_TYPE ) failed:
osp_md_create()) LBUG
Pid: 6024, comm: lod0002_rec0001 3.10.0-1062.18.1.el7_lustre
Call Trace:
 libcfs_call_trace+0x8c/0xc0 [libcfs]
 lbug_with_loc+0x4c/0xa0 [libcfs]
 osp_md_create+0x42a/0x470 [osp]
 llog_osd_get_cat_list+0x8d4/0xbd0 [obdclass]
 lod_sub_prep_llog+0xb9/0x783 [lod]
 lod_sub_recovery_thread+0x383/0xcf0 [lod]
 kthread+0xd1/0xe0
 ret_from_fork_nospec_begin+0x7/0x21

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I035f3f2da5dba54b86431cec65c48e9c5010224c
Reviewed-on: https://review.whamcloud.com/40264
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14037 osd: track commit cb in flight 58/40258/5
Alex Zhuravlev [Wed, 14 Oct 2020 12:20:18 +0000 (15:20 +0300)]
LU-14037 osd: track commit cb in flight

and wait for completion in osd-ldiskfs as mntput() is not synchronous
and returns quickly scheduling real umount to another thread while
osd is shutting down. as a result commit callbacks may come when
osd has been released yet.
the patch replaces lu_device_get() and lu_device_put() for transactions
with a dedicated counter which later used to wait for callback completion.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iaa6089b9158bc2a4bea8e33f4cfcce27395689ca
Reviewed-on: https://review.whamcloud.com/40258
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14051 utils: flush alr batch file in thread 10/40310/7
John L. Hammond [Thu, 17 Sep 2020 20:56:55 +0000 (15:56 -0500)]
LU-14051 utils: flush alr batch file in thread

In ofd_access_log_reader, move flushing of the batch file thread to
the sort and print thread.

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Id1e008ede6c05e24ea2e2459520d6585007acc7d
Reviewed-on: https://review.whamcloud.com/40310
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14050 utils: fix fraction output logic 34/39934/6
Alex Zhuravlev [Wed, 16 Sep 2020 19:14:42 +0000 (22:14 +0300)]
LU-14050 utils: fix fraction output logic

so that it doesn't suppress single line reports

Test-Parameters: trivial
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9e374f8e769afcadc5cecbf529fa403deb544544
Reviewed-on: https://review.whamcloud.com/39934
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-12546 mdt: abort recovery between MDTs 27/36027/10
Hongchao Zhang [Sun, 12 Apr 2020 12:40:27 +0000 (20:40 +0800)]
LU-12546 mdt: abort recovery between MDTs

Add an option to abort recovery between MDTs in case there is a
problem during recovery (e.g. MDT is missing or has broken logs),
but don't abort recovery between MDT and clients.

Change-Id: Id88f2b2ebae5cfa722dcac67c087b9b9a448721e
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36027
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
3 months agoLU-13765 osd-ldiskfs: Rename dt_declare_falloc to dt_declare_fallocate 09/40509/2
Arshad Hussain [Sun, 1 Nov 2020 03:33:13 +0000 (09:03 +0530)]
LU-13765 osd-ldiskfs: Rename dt_declare_falloc to dt_declare_fallocate

This patch is the follow up of the patch: 93f700ca24
(LU-13765 osd-ldiskfs: Extend credit correctly for fallocate) and
it makes these changes:

01. Rename dt_declare_falloc() to dt_declare_fallocate()
for better readability.

02. Removes fallocate mode check under osd_fallocate()
as mode check is already done under declare phase.

03. Minor space/tabs changes

Test-Parameters: trivial testlist=sanity ostsizegb=12 env=ONLY="150e"
Test-Parameters: testlist=sanity-quota
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: If911a59a9c944e660e9926f4c436a4aeb2919284
Reviewed-on: https://review.whamcloud.com/40509
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
3 months agoLU-10810 test: test lseek support in tools 02/40502/10
Mikhail Pershin [Sat, 31 Oct 2020 09:03:33 +0000 (12:03 +0300)]
LU-10810 test: test lseek support in tools

Check that SEEK_HOLE/SEEK_DATA are preforming in external tools
as expected.

Need 'cp' version 8.33+ and 'tar' version 1.29+, so check tools
version and measure runtime of sparse file handling if applicable

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I1424bf57c88f69d054c1646be66e10dd7fde8a1a
Reviewed-on: https://review.whamcloud.com/40502
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
3 months agoLU-10810 clio: SEEK_HOLE/SEEK_DATA on client side 08/39708/16
Mikhail Pershin [Fri, 21 Aug 2020 16:22:38 +0000 (19:22 +0300)]
LU-10810 clio: SEEK_HOLE/SEEK_DATA on client side

Patch introduces basic support for lseek SEEK_HOLE/SEEK_DATA
parameters in lustre client.

- introduce new IO type CIT_LSEEK in CLIO stack
- LOV splits request to all stripes involved and merges
  results back.
- OSC sends OST LSEEK RPC asynchronously
- if target doesn't support LSEEK RPC then OSC assumes
  whole related object is data with virtual hole at the end
- lseek restores released files assuming it is done prior
  the file copying.
- tool is added to request needed lseek on file
- basic tests are added in sanity, sanityn and sanity-hsm

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I0728329d4bce71c441de581a439cde1aa873fd46
Reviewed-on: https://review.whamcloud.com/39708
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-10810 ptlrpc: introduce OST_SEEK RPC 07/39707/9
Mikhail Pershin [Mon, 17 Aug 2020 11:06:30 +0000 (14:06 +0300)]
LU-10810 ptlrpc: introduce OST_SEEK RPC

For the purposes of SEEK_HOLE/SEEK_DATA support introduce
new OST_SEEK RPC.

Patch add RPC layout, unified handler and connect flag for
compatibility needs.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I1580902b6b773d9a6d6f9beaa1ee1da60fbc20f8
Reviewed-on: https://review.whamcloud.com/39707
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14116 autoconf: check if DES3 enctype is supported 54/40554/3
Jian Yu [Fri, 6 Nov 2020 06:31:28 +0000 (22:31 -0800)]
LU-14116 autoconf: check if DES3 enctype is supported

krb5 releases 1.18 and later completely remove support for
all DES3 enctypes (des3-cbc-raw, des3-hmac-sha1, des3-cbc-sha1-kd).

This patch adds HAVE_DES3_SUPPORT to check if DES3 enctype
is supported.

Change-Id: Ibb51ec7961e8c775ea92dec6119f4de01e2d9b1d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40554
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
3 months agoLU-13182 llite: Avoid eternel retry loops with MAP_POPULATE 21/40221/6
Oleg Drokin [Mon, 12 Oct 2020 20:10:15 +0000 (16:10 -0400)]
LU-13182 llite: Avoid eternel retry loops with MAP_POPULATE

Kernels 5.4+ have an infinite retry loop from MAP_POPULATE mmap
option. Use the FAULT_FLAG_RETRY_NOWAIT to instruct filemap_fault
to not drop the mmap_sem so if the call fails, we could use
the slow path and break the loop from forming.
(Idea by Neil Brown)

Test-Parameters: testlist=sanity-hsm env=ONLY=1 clientdistro=ubuntu2004
Change-Id: I320ab9ca447282aea15ef2030ef8671c4260d895
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40221
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
3 months agoLU-13651 hsm: call hsm_find_compatible_cb() only for cancel 67/38867/18
Kirill Malkin [Sun, 17 May 2020 03:17:43 +0000 (20:17 -0700)]
LU-13651 hsm: call hsm_find_compatible_cb() only for cancel

The HSM action queue is scanned linearly in hsm_find_compatible_cb()
for existing requests on the same file so that duplicate or
conflicting requests are not added and cancel requests are assigned
the correct cookie, but this can cause a large delay in adding new
requests when the action queue is very large, as access to it is
locked for the duration of the search. Scanning the queue does not
guarantee that duplicate or conflicting requests are not added as
scanning (in hsm_find_compatible_cb()) and adding requests (in
mdt_agent_record_add()) are distinct operations that are not
serialized by a lock and so a race window exists between these two
function calls within which duplicate or conflicting requests can be
added. This is hopefully not a big problem though, as the CDT thread
will not send duplicate archive requests to a copytool serving a
different HSM backend (and it could probably be prevented from sending
duplicate archive requests to a copytool serving the same backend with
a small change in mdt_hsm_is_action_compat()) and duplicate restore
requests are serialized by taking the layout lock on the file before
being added to the action queue, which effectively serializes
them (although this blocks the caller, e.g. lfs, so it might not be
ideal). Since calling hsm_compatible_cb() does not protect completely
against this issue and can cause large delays in adding new requests,
we skip calling it for all requests apart from cancel requests that
don't specify a cookie (which should be all cancel requests in current
code), hopefully safely.

Test-Parameters: testlist=sanity-hsm
Signed-off-by: Kirill Malkin <kirill.malkin@hpe.com>
Signed-off-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Cray-bug-id: LUS-8717
Change-Id: Id82b2a0720e46a9c12c4d9df323ce5a7bd7aff37
Reviewed-on: https://review.whamcloud.com/38867
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathan Rutman <nrutman@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13949 build: add autogen.sh into distribution tarball 25/40425/2
Jian Yu [Tue, 27 Oct 2020 17:33:03 +0000 (10:33 -0700)]
LU-13949 build: add autogen.sh into distribution tarball

This patch adds autogen.sh and config/lustre-version.m4 into
Lustre distribution tarball so that customers can regenerate
aclocal.m4, config.h.in, autoMakefile.in and configure in
their build environments.

Change-Id: Ic6c5430b9a8b504ebc6a7618e141f1ea23b046a2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40425
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
3 months agoLU-14073 build: fix autoconf test for clean_bdev_aliases() 95/40395/3
Mr NeilBrown [Mon, 26 Oct 2020 02:38:21 +0000 (13:38 +1100)]
LU-14073 build: fix autoconf test for clean_bdev_aliases()

From 5.9, buffer_head.h no longer provides a declaration for
'struct block_device' so the code fragment fails because the compiler
doesn't know the size of that structure.

Instead, simple pass NULL rather than the address of a real structure.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I1775572fbd56d22822b6e440fe95bd105042e7b8
Reviewed-on: https://review.whamcloud.com/40395
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
3 months agoLU-14070 tgt: check obd_recovering in tgt_brw_unlock() 82/40382/4
Mikhail Pershin [Fri, 23 Oct 2020 09:30:58 +0000 (12:30 +0300)]
LU-14070 tgt: check obd_recovering in tgt_brw_unlock()

Since tgt_brw_lock() never takes a lock during recovery,
the tgt_brw_unlock() should check that also to prevent
false-positive triggering of assertion.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic6b6f6fa16678622460101d26df14f523e56a47a
Reviewed-on: https://review.whamcloud.com/40382
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14057 ptlrpc: don't log connection 'restored' inappropriately 31/40331/3
Aurelien Degremont [Fri, 16 Oct 2020 13:42:42 +0000 (13:42 +0000)]
LU-14057 ptlrpc: don't log connection 'restored' inappropriately

Reverse imports maintain a target->client connection which
does not support recovery as client don't run a recovery.
At every connection, the reverse import state goes from
NEW to RECOVER to FULL which triggers a `Connection restored`
log message, even if this is the first connection from
this client.

Suppress this log message for reverse import to avoid
this wrong logging.

Test-Parameters: trivial
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: I6f35b8d916a4ae535d55ba39b491f57e1553986c
Reviewed-on: https://review.whamcloud.com/40331
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13728 utils: add missing global parameters 98/40298/2
Cyril Bordage [Mon, 19 Oct 2020 16:10:23 +0000 (18:10 +0200)]
LU-13728 utils: add missing global parameters

lnetctl export shows the complete set of global parameters as
with lnetctl global.

Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Change-Id: I4d864fb4734679106ac6c49ec7f57f5e00ba3434
Reviewed-on: https://review.whamcloud.com/40298
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-10728 utils: fix str length in error string 68/40268/3
Cyril Bordage [Fri, 16 Oct 2020 13:39:06 +0000 (15:39 +0200)]
LU-10728 utils: fix str length in error string

sizeof on pointers was used to get the length of the string. Use
instead string length from function inputs. Also remove useless uses
of snprintf and terminating null bytes.

Test-Parameters: @lnet
Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Change-Id: I7053f39828ababd5782b360ef5c27c607ddb740d
Reviewed-on: https://review.whamcloud.com/40268
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14029 kernel: new kernel [SLES15 SP2 5.3.18-24.24.1] 65/40265/3
Jian Yu [Thu, 15 Oct 2020 19:58:01 +0000 (12:58 -0700)]
LU-14029 kernel: new kernel [SLES15 SP2 5.3.18-24.24.1]

This patch makes changes to support new SLES15 SP2 release
with kernel 5.3.18-24.24.1 for Lustre client.

Test-Parameters: trivial \
env=SANITY_EXCEPT="100 130 136 817" \
clientdistro=sles15sp2 serverdistro=el7.8 \
testlist=sanity

Change-Id: Icf97678ebb0c6495d956f13d57e0cea65a20b108
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40265
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13989 ldlm: BL AST vs failed lock enqueue race 46/40046/3
Andriy Skulysh [Tue, 11 Feb 2020 12:00:18 +0000 (14:00 +0200)]
LU-13989 ldlm: BL AST vs failed lock enqueue race

failed_lock_cleanup() marks the lock with LDLM_FL_LOCAL_ONLY,
so cancel request isn't sent.

Mark failed lock with LDLM_FL_LOCAL_ONLY only
if BL AST wasn't received.
Add server's lock handle to BL AST RPC.
So client will be able to cancel the lock
even if enqueue fails.

Change-Id: I3201bc29abd877cddc334ca27a9d208cb55c5d8f
HPE-bug-id: LUS-8493, LUS-8830
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-on: https://review.whamcloud.com/40046
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13984 ptlrpc: throttle RPC resend if network error 20/40020/9
Aurelien Degremont [Wed, 23 Sep 2020 19:20:08 +0000 (19:20 +0000)]
LU-13984 ptlrpc: throttle RPC resend if network error

When sending a callback AST to a non-responding client, the server
retries endlessly until the client is eventually evicted. When using
ksocklnd, it will retry after each AST timeout, until the socket is
eventually closed, after sock_timeout sec, where the retry will fail
immediately, returning -110, as no socket could be established.

The thread will spin on retrying and failing, until eventual client
eviction. This will cause high thread CPU usage and possible resource
denial.

To workaround that, this patch avoids re-trying callback resend if:
 - the request is flagged with network error and timeout
 - last try was less than 1 sec ago

In worst case, retry will happen after a timeout based on req->rq_deadline.
If there is nothing else to handle, thread will be sleeping during that
time, removing CPU overhead.

Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: Ie5028761c978b26e833fd0a5d30d313addf57984
Reviewed-on: https://review.whamcloud.com/40020
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13961 kernel: kernel update RHEL8.2 [4.18.0-193.19.1.el8_2] 87/39987/4
Jian Yu [Tue, 27 Oct 2020 22:52:32 +0000 (15:52 -0700)]
LU-13961 kernel: kernel update RHEL8.2 [4.18.0-193.19.1.el8_2]

Update RHEL8.2 kernel to 4.18.0-193.19.1.el8_2.

Test-Parameters: trivial \
clientdistro=el8.2 serverdistro=el8.2 \
testlist=sanity

Change-Id: I32d65790adcd5829cdc4447e9b116a83bf1efd63
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39987
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-14049 utils: manage thread resources in alr_batch_print() 09/40309/5
John L. Hammond [Mon, 14 Sep 2020 17:58:31 +0000 (12:58 -0500)]
LU-14049 utils: manage thread resources in alr_batch_print()

In alr_batch_print(), create the sort and print thread with the
detached attribute. Protect against concurrent write the the batch
output file. Ensure that memory is freed in error cases.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ibf1b299bd15f5d189a2302ce476bf2ef986a85b1
Reviewed-on: https://review.whamcloud.com/40309
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
3 months agoLU-13665 tests: skip sanity subtests for new features 90/39890/10
Andreas Dilger [Fri, 11 Sep 2020 23:01:57 +0000 (17:01 -0600)]
LU-13665 tests: skip sanity subtests for new features

Skip sanity.sh test_165 (OAL) and part of test_56oc (btime) during
interop testing for features that were added recently.

Skip test_56oc timestamp parsing test to avoid timezone issues in
test environment.

Fixes: 3f7853b31ef6 ("LU-10934 llite: integrate statx() API with Lustre")
Fixes: 66172e3274ca ("LU-13238 ofd: add OFD access logs")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib09b60dccb563fcedadd1da55eea11ddca6ecde5
Reviewed-on: https://review.whamcloud.com/39890
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14031 ptlrpc: decrease time between reconnection 44/40244/4
Alexander Boyko [Wed, 14 Oct 2020 08:20:58 +0000 (04:20 -0400)]
LU-14031 ptlrpc: decrease time between reconnection

When a connection get a timeout or get an error reply from a sever,
the next attempt happens after PING_INTERVAL. It is equal to
obd_timeout/4. When a first reconnection fails, a second go to
failover pair. And a third connection go to a original server.
Only 3 reconnection before server evicts client base on blocking
ast timeout. Some times a first failed and the last is a bit late,
so client is evicted. It is better to try reconnect with a timeout
equal to a connection request deadline, it would increase a number
of attempts in 5 times for a large obd_timeout. For example,
    obd_timeout=200
     - [ 1597902357, CONNECTING ]
     - [ 1597902357, FULL ]
     - [ 1597902422, DISCONN ]
     - [ 1597902422, CONNECTING ]
     - [ 1597902433, DISCONN ]
     - [ 1597902473, CONNECTING ]
     - [ 1597902473, DISCONN ] <- ENODEV from a failover pair
     - [ 1597902523, CONNECTING ]
     - [ 1597902539, DISCONN ]

The patch adds a logic to wakeup pinger for failed connection request
with ETIMEDOUT or ENODEV. It adds imp_next_ping processing for
ptlrpc_pinger_main() time_to_next_wake calculation, and fixes setting
of imp_next_ping value.

HPE-bug-id: LUS-8520
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ia0891a8ead1922810037f7d71092cd57c061dab9
Reviewed-on: https://review.whamcloud.com/40244
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13645 ldlm: extra checks for DOM locks 78/39878/6
Vitaly Fertman [Wed, 2 Sep 2020 17:14:06 +0000 (20:14 +0300)]
LU-13645 ldlm: extra checks for DOM locks

a couple of checks are added:
- only DOM lock can be a group lock;
- DOM bit must be the only mandatory one, or optional;

Signed-off-by: Vitaly Fertman <c17818@cray.com>
HPE-bug-id: LUS-8987
Change-Id: Iaf7a14a66eb0f125d2f6f7d06f5de0add387e101
Reviewed-on: https://review.whamcloud.com/39878
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13645 ldlm: group locks for DOM IBIT lock 06/39406/7
Vitaly Fertman [Tue, 4 Aug 2020 12:12:04 +0000 (15:12 +0300)]
LU-13645 ldlm: group locks for DOM IBIT lock

Group lock is supposed to be taken on such operations as layout swap
used for e.g. HSM, and is to be taken for DOM locks as well.

HPE-bug-id: LUS-8987
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I97888e1aee853d7fe04548681b2ed6805cb494ae
Reviewed-on: https://review.whamcloud.com/39406
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13666 lod: update .do_index_ops on layout detach 26/39226/2
Lai Siyao [Wed, 1 Jul 2020 14:03:19 +0000 (22:03 +0800)]
LU-13666 lod: update .do_index_ops on layout detach

Directory migration detaches stripes from source, and then attaches
them to target if source is a striped directory. This will convert
source from striped directory to plain directory, it needs update
.do_index_ops from lod_striped_index_ops to lod_index_ops to avoid
trigger assertion in index ops.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia8f66a8a3fd5e96f0dba4d60eb2443107d320418
Reviewed-on: https://review.whamcloud.com/39226
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-11631 mdd: migrate symlink for cross-MDT rename 97/39897/3
Lai Siyao [Wed, 9 Sep 2020 21:50:11 +0000 (05:50 +0800)]
LU-11631 mdd: migrate symlink for cross-MDT rename

If symlink rename is cross MDTs, it's ineffective to turn this
symlink into a remote object, instead migrate it to where target
MDT is. The following changes are made:
* change migration code to allow source and target have different
  name.
* if symlink is renamed to other MDT, and it doesn't have other
  hard link and target doesn't exist, migrate it to target MDT.
* remove mdd_rename_order() which is obsolete.

Add sanity 24G.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib5fafe3122172ac582bbcc907c72a9f391baf0e1
Reviewed-on: https://review.whamcloud.com/39897
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
4 months agoLU-13692 ldlm: Ensure we reprocess the resource on ast error 98/39598/8
Oleg Drokin [Fri, 7 Aug 2020 07:38:51 +0000 (03:38 -0400)]
LU-13692 ldlm: Ensure we reprocess the resource on ast error

When we are trying to grant a lock and met an AST error, rerunning
the policy is pointless since it cannot grant a potentially now eligible
lock and our lock is already in all the queues, just be like all the other
handlers for ERESTART return and run a full resource reprocess instead.

Change-Id: I3edb37bf084b2e26ba03cf2079d3358779c84b6e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
4 months agoLU-14069 ldlm: Fix unbounded OBD_FAIL_LDLM_CANCEL_BL_CB_RACE wait 75/40375/3
Oleg Drokin [Fri, 23 Oct 2020 06:56:04 +0000 (02:56 -0400)]
LU-14069 ldlm: Fix unbounded OBD_FAIL_LDLM_CANCEL_BL_CB_RACE wait

in ldlm_handle_cp_callback the while loop is clearly supposed
to be limited by the "to" value of 1 second, but is not.
Seems to have been broken by all the Solaris porting in HEAD
all the way back in 2008.
Restore the to assignment to make it not hang indefinitely.

Change-Id: I449bfd7f585ab7db475fb3fd4cbbd876126ff789
Fixes: adde80ff ("Land b_head_libcfs onto HEAD")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40375
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
4 months agoLU-1742 o2iblnd: 'Timed out tx' error message 22/3622/4
Brian Behlendorf [Mon, 13 Aug 2012 23:58:20 +0000 (16:58 -0700)]
LU-1742 o2iblnd: 'Timed out tx' error message

Trivial fix to report the total RDMA time outstanding rather
than the number of seconds past the deadline.

Change-Id: I0ef9b7b9b31a4d27adf4f3f33da46c503e5ca49e
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/3622
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14010 build: Ensure dkms installs all Lustre modules 25/40225/2
Amin Tootoonchian [Mon, 12 Oct 2020 21:07:50 +0000 (16:07 -0500)]
LU-14010 build: Ensure dkms installs all Lustre modules

Add --force to dkms install in:
  debian/lustre-client-modules-dkms.postinst

Without it older than available modules are skipped.

Signed-off-by: Amin Tootoonchian <amint@openai.com>
Change-Id: I1d549e7d48d60294810e11ed2588a512f1527eda
Reviewed-on: https://review.whamcloud.com/40225
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13437 llite: pass name in getattr by FID 19/40219/6
Lai Siyao [Mon, 12 Oct 2020 14:22:07 +0000 (22:22 +0800)]
LU-13437 llite: pass name in getattr by FID

Now parent FID is packed in getattr_by_FID request
(see https://review.whamcloud.com/39290), it should also pass in name
from llite, so that lmv can replace fid1 with stripe FID, otherwise
MDS may treat sub files under striped directory as remote object.

Note, the name is not packed in request, because if it's packed, MDS
will getattr by name instead of FID.

Fixes: 5f2c44bf6 ("LU-13437 llite: pack parent FID in getattr")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If8215667bcb10ea3c4c5cd2c9034d81fd1cda3b5
Reviewed-on: https://review.whamcloud.com/40219
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13437 mdc: remote object support getattr from cache 18/40218/4
Lai Siyao [Sat, 10 Oct 2020 14:34:19 +0000 (22:34 +0800)]
LU-13437 mdc: remote object support getattr from cache

For historical reason, IT_GETATTR lock revalidate matches
LOOKUP|UPDATE|PERM lock bits because for MDS < 2.4, permission is
protected by LOOKUP lock, but this will cause remote object not
able to match the cached lock because LOOKUP and UPDATE lock are
fetched separately.

Add sanity 803b, and rename 803 to 803a.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I3ac38fe34472736849307bb7f1eebb5de9343a5c
Reviewed-on: https://review.whamcloud.com/40218
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14016 libcfs: use atomic64_t for libcfs_kmem 68/40168/5
Amir Shehata [Wed, 7 Oct 2020 21:27:14 +0000 (14:27 -0700)]
LU-14016 libcfs: use atomic64_t for libcfs_kmem

libcfs_kmem keeps track of LNet's memory usage. It uses an
int type, so it could wrap around if usage grows beyond 2.14 GB.
Use atomic64_t to avoid this issue.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: If96fb8391c6ffb1924e47cef3dfca02eabc5f912
Reviewed-on: https://review.whamcloud.com/40168
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13498 tests: remove tests from ALWAYS_EXCEPT with SSK 61/40161/8
Sebastien Buisson [Wed, 7 Oct 2020 06:36:49 +0000 (08:36 +0200)]
LU-13498 tests: remove tests from ALWAYS_EXCEPT with SSK

A number of tests had previously been added to ALWAYS_EXCEPT
when SHARED_KEY was in use.
These tests are now passing with SSK, so remove them from the
exception list.

Test-Parameters: trivial
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,replay-dual,replay-single,sanity-hsm,conf-sanity
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,replay-dual,replay-single,sanity-hsm,conf-sanity
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,replay-dual,replay-single,sanity-hsm,conf-sanity
Test-Parameters: env=SHARED_KEY=true mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,replay-dual,replay-single,sanity-hsm,conf-sanity
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: If72b212a23b915afdb723acf7254908e1c043e07
Reviewed-on: https://review.whamcloud.com/40161
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
4 months agoLU-13824 test: test sanity 230q with fewer files on ZFS 28/39528/4
Lai Siyao [Tue, 28 Jul 2020 13:44:09 +0000 (21:44 +0800)]
LU-13824 test: test sanity 230q with fewer files on ZFS

Sanity 230q may timeout on ZFS backend, test with fewer files.

Test-Parameters: trivial fstype=zfs testlist=sanity mdscount=2 mdtcount=4 env=ONLY=230q,ONLY_REPEAT=100
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Iaf9e4e6d68244937819305af72df33e59df19f1f
Reviewed-on: https://review.whamcloud.com/39528
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14031 ptlrpc: remove unused code at pinger 43/40243/3
Alexander Boyko [Wed, 14 Oct 2020 07:45:21 +0000 (03:45 -0400)]
LU-14031 ptlrpc: remove unused code at pinger

The timeout_list was previously used for grant shrinking,
but right now is dead code.

HPE-bug-id: LUS-8520
Fixes: fc915a43786e ("LU-8708 osc: depart grant shrinking from pinger")
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ia7a77b4ac19da768ebe1b0879d7123941f4490b5
Reviewed-on: https://review.whamcloud.com/40243
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13740 tests: improve sanity-sec test_45 46/40146/2
Sebastien Buisson [Tue, 6 Oct 2020 09:43:19 +0000 (11:43 +0200)]
LU-13740 tests: improve sanity-sec test_45

Improve sanity-sec test_45 by referencing the entire mmap-ed region
thanks to multiop.
Also make sure encryption tests are passing on newly supported
Ubuntu 20.04 distro.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=ubuntu2004 fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=ubuntu2004 fstype=zfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=zfs mdscount=2 mdtcount=4
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I983b6bf94d9f51486fd6b688267af46ed4188a98
Reviewed-on: https://review.whamcloud.com/40146
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
4 months agoLU-13765 osd-ldiskfs: Extend credit correctly for fallocate 42/39342/18
Arshad Hussain [Wed, 9 Sep 2020 23:18:13 +0000 (04:48 +0530)]
LU-13765 osd-ldiskfs: Extend credit correctly for fallocate

In OSD layer, before call ->fallocate(), Lustre has already
created journal handle for the fallocate transcation. In
ldiskfs/ext4, for very large range fallocate, the operation
may split into multiple transaction and call journal start/stop
multiple times inside fallocate. However, nested journal will
ignore requested credits, this result in running out of credits
at the end.

As we can not predict the total number of credits needed in
advance especially for large fallocate, thus in this patch, we
move fallocate logic into Lustre OSD, so that it could reserve
credits correctly. It extends credits for the current transaction
when found the left buffer credits is less than needed, and then
restart the transaction.

Testcase sanity/150e and sanity-quota/1h added to verify the
issue.

Test-Parameters: trivial testlist=sanity ostsizegb=12 env=ONLY="150e"
Test-Parameters: testlist=sanity-quota
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ib7565ed2c1ae72eef4832fbcb710e0ee70c53aec
Reviewed-on: https://review.whamcloud.com/39342
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13719 lov: doesn't check lov_refcount 02/39702/2
Hongchao Zhang [Fri, 21 Aug 2020 10:17:12 +0000 (18:17 +0800)]
LU-13719 lov: doesn't check lov_refcount

In lov_cleanup, the check of each OSC is protected by
lov_tgt_getrefs, which will increment the "lov_refcount",
so the "lov_refcount" shouldn't be checked inside because
it is always larger than 0.

Change-Id: I21423d4345190b3e02eb00734c127e35cbc9b1af
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39702
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-12214 build: fix suse require krb5 14/40214/6
Minh Diep [Sun, 11 Oct 2020 22:50:46 +0000 (15:50 -0700)]
LU-12214 build: fix suse require krb5

Test-Parameters: trivial clientdistro=sles15sp1

Change-Id: If5bbe77bda84381b363c733f763cfc81e29aedb7
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40214
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
4 months agoLU-13745 tests: skip sanity test_426 for 4.15+ 66/40366/2
John L. Hammond [Thu, 22 Oct 2020 22:09:03 +0000 (17:09 -0500)]
LU-13745 tests: skip sanity test_426 for 4.15+

Add sanity test_426 to ALWAYS_EXCEPT for newer client kernels because
it is crashing 100% since "LU-13745 test: add splice test for lustre"
was landed.

Test-Parameters: trivial clientdistro=ubuntu1804
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I84a722a27c3e8a572c20b46ca9daaf44e8720b54
Reviewed-on: https://review.whamcloud.com/40366
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
4 months agoLU-14067 test: skip compile tests on aarch64 65/40365/3
John L. Hammond [Thu, 22 Oct 2020 21:57:47 +0000 (16:57 -0500)]
LU-14067 test: skip compile tests on aarch64

aarch64 gcc segfaults trying to compile our headers so skip sanity
400a, 400b and sanity-lnet 300 on aarch64.

Test-Parameters: trivial clientdistro=el8.1 clientarch=aarch64
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I8322107919084c86a0cb6fc15730a49f96c03b22
Reviewed-on: https://review.whamcloud.com/40365
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
4 months agoLU-13636 osd: create agent inode with explicit owner 42/38842/5
Alex Zhuravlev [Fri, 5 Jun 2020 05:16:32 +0000 (08:16 +0300)]
LU-13636 osd: create agent inode with explicit owner

to avoid quota misaccounting.

Test-Parameters: fstype=ldiskfs
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5a02e6e7de71821a10704ac3516ee087998c9c21
Reviewed-on: https://review.whamcloud.com/38842
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13498 gss: update sequence in case of target disconnect 22/40122/5
Sebastien Buisson [Fri, 2 Oct 2020 12:05:55 +0000 (21:05 +0900)]
LU-13498 gss: update sequence in case of target disconnect

Client to OST connections can go idle, leading to target disconnect.
In this event, maintaining correct sequence number ensures that GSS
does not erroneously consider requests as replays.
Sequence is normally updated on export destroy, but this can occur too
late, ie after a new target connect request has been processed. So
explicitly update sec context at disconnect time.

Test-Parameters: env=SHARED_KEY=true,SK_FLAVOR=skn mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Test-Parameters: env=SHARED_KEY=true,SK_FLAVOR=ska mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Test-Parameters: env=SHARED_KEY=true,SK_FLAVOR=ski mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Test-Parameters: env=SHARED_KEY=true,SK_FLAVOR=skpi mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity,recovery-small,sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I65c27e1ab459b2a29670580121ef6e1a00f18918
Reviewed-on: https://review.whamcloud.com/40122
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13745 tests: skip sanity test_426 for 4.18+ 26/40326/3
Andreas Dilger [Wed, 21 Oct 2020 02:42:29 +0000 (20:42 -0600)]
LU-13745 tests: skip sanity test_426 for 4.18+

Add sanity test_426 to ALWAYS_EXCEPT for newer client kernels because
it is crashing 100% since "LU-13745 test: add splice test for lustre"
was landed.

 Unable to handle NULL pointer dereference at address 0000000000000004
 user pgtable: 64k pages, 48-bit VAs, pgdp = 000000009f14b2d0
 Internal error: Oops: 96000005 [#1] SMP
 CPU: 1 PID: 11273 Comm: ptlrpcd_01_01 4.18.0-147.8.1.el8_1.aarch64 #1
 Process ptlrpcd_01_01 (pid: 11273, stack limit = 0x00000000f9135a93)
 Call trace:
  mempool_free+0x24/0xe0
  llcrypt_free_bounce_page.part.1+0x38/0x48 [libcfs]
  llcrypt_free_bounce_page+0x24/0x30 [libcfs]
  brw_interpret+0x124/0x10c8 [osc]
  ptlrpc_check_set+0x688/0x3318 [ptlrpc]
  ptlrpcd_check+0x470/0x820 [ptlrpc]
  ptlrpcd+0x3d4/0x5c8 [ptlrpc]
  kthread+0x130/0x138

Test-Parameters: trivial clientdistro=el8.1 clientarch=aarch64
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8f7b1d5e3ee69a3e0a6dfe3944949741a74cb62a
Reviewed-on: https://review.whamcloud.com/40326
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13699 mdt: Improve message reporting for mdt_identity.c 03/39203/5
Arshad Hussain [Sat, 27 Jun 2020 11:27:06 +0000 (16:57 +0530)]
LU-13699 mdt: Improve message reporting for mdt_identity.c

This patch Improves Error handling and message reporting
for file lustre/mdt/mdt_identity.c

This patch also replaces ERR_PTR(PTR_ERR()) with ERR_CAST()
which is reported by coccinelle

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I59e262e743e7b1926fe75920239d6086c183b30f
Reviewed-on: https://review.whamcloud.com/39203
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
4 months agoLU-12542 osd: consolidate RCU handling 04/40204/2
Andreas Dilger [Thu, 3 Sep 2020 22:37:47 +0000 (16:37 -0600)]
LU-12542 osd: consolidate RCU handling

Consolidate lu_object_header_fini() and kfree_rcu() into a
single lu_object_header_free() function so that callers
which do not need a container for lu_object_header can
avoid duplicating common RCU and OBD_FREE handling code.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0fa68f11b5008ede5a498d38b69ccaeecf3ebbe5
Reviewed-on: https://review.whamcloud.com/40204
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14020 ldiskfs: SUSE 15 SP2 and mainline 5.4.21 and newer 79/40179/2
Shaun Tancheff [Thu, 8 Oct 2020 20:09:51 +0000 (15:09 -0500)]
LU-14020 ldiskfs: SUSE 15 SP2 and mainline 5.4.21 and newer

The updated pdirop for Ubuntu 5.4.0-42 kernel also aligns
with Linux LTS 5.4.21 through to current 5.4.60

In addition this series is also suitable for the current
SUSE 15 SP2 kernel, tested with 5.3.18-24.15.

Test-Parameters: trivial
HPE-bug-id: LUS-9092
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I75ee53c4096b31c3a2d13748401ed1a12306215b
Reviewed-on: https://review.whamcloud.com/40179
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14012 lod: properly initialize lcm in lod_layout_convert() 53/40153/2
John L. Hammond [Tue, 6 Oct 2020 19:14:29 +0000 (14:14 -0500)]
LU-14012 lod: properly initialize lcm in lod_layout_convert()

In lod_layout_convert() zero out lcm and lcme before constructing the
converted layout.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I40f96d51cb63816a9bfc34217f02ff7c450de974
Reviewed-on: https://review.whamcloud.com/40153
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
4 months agoLU-14010 build: Add missing deps for dkms client 38/40138/3
Amin Tootoonchian [Mon, 5 Oct 2020 08:42:54 +0000 (03:42 -0500)]
LU-14010 build: Add missing deps for dkms client

dkms installs also need libyaml-dev and zlib1g to build and
install successfully.

Test-Parameters: trivial
HPE-bug-id: LUS-9369
Signed-off-by: Amin Tootoonchian <amint@openai.com>
Change-Id: Idbd79295590987b7dace07b4da3ee731d0714a8d
Reviewed-on: https://review.whamcloud.com/40138
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-10753 osd-zfs: initialize obj attr correctly 62/40062/4
Lai Siyao [Thu, 24 Sep 2020 16:05:15 +0000 (00:05 +0800)]
LU-10753 osd-zfs: initialize obj attr correctly

mdt_thread_info.mti_attr is used to initialize object attr in create,
currently it's copied to object.oo_attr directly, but some fields
in mti_attr may contain bogus data because it's not cleared in each
use, though la_valid is correctly set, but la_flags is used without
checking la_valid in __mdd_permission_internal().

Another minor fix in osd_create(): set size/nlink to zero since they
are set in valid.

Test-Parameters: testlist=sanity env=ONLY=300,ONLY_REPEAT=100

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I64816b66a0b3c7aa50e62680d5251141697a8e0f
Reviewed-on: https://review.whamcloud.com/40062
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13983 llite: rmdir releases inode on client 11/40011/4
Lai Siyao [Tue, 22 Sep 2020 11:05:51 +0000 (19:05 +0800)]
LU-13983 llite: rmdir releases inode on client

Same as file unlink, rmdir should release inode on client, to achieve
this, ll_rmdir() update inode i_nlink after rmdir, then the last
iput() will release the inode.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I9181151de1830b48986afec30c83120e9f112a85
Reviewed-on: https://review.whamcloud.com/40011
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13975 sec: require enc key in case of O_CREAT only 83/39983/6
Sebastien Buisson [Mon, 21 Sep 2020 12:45:49 +0000 (12:45 +0000)]
LU-13975 sec: require enc key in case of O_CREAT only

In ll_atomic_open(), do not return -ENOKEY when trying to open
either a directory or a file without the encryption key, unless
O_CREAT flag is specified.
Indeed, listing directory content is allowed even without the key.
And in case of regular file, ll_file_open() already checks for the
presence of an encryption key.

Improve sanity-sec test_54 to verify this is working properly.

Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=zfs mdscount=2 mdtcount=4
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I813d4ec938e00c8463b1d3ee9766d180806b40ba
Reviewed-on: https://review.whamcloud.com/39983
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13972 o2iblnd: Don't retry indefinitely 81/39981/6
Amir Shehata [Sat, 19 Sep 2020 08:38:07 +0000 (01:38 -0700)]
LU-13972 o2iblnd: Don't retry indefinitely

If peer is down don't retry indefinitely. Use the retry_count
parameter to restrict the number of retries. After which the
connection fails and error is propagated up.

This prevents long timeouts when mounting a file system with
nodes which might have their NIDs configured in the FS, but the
nodes have been taken offline.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I04faf690ed13357e3ed50c2adaadee265db269c7
Reviewed-on: https://review.whamcloud.com/39981
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13946 build: OpenZFS 2.0 compatibility 22/39822/5
Brian Behlendorf [Thu, 3 Sep 2020 16:23:17 +0000 (09:23 -0700)]
LU-13946 build: OpenZFS 2.0 compatibility

* Update zfs_refcount_add() configure check.  OpenZFS 2.0 has renamed
  the sys/refcount.h header to sys/zfs_refcount.h to avoid a conflict
  with this header on FreeBSD.  Since Lustre never directly includes
  this header by name adjust the configure check to indirectly include
  it through sys/dnode.h.  This was we don't need a separate check to
  determine the expected header name.

* Add db->db_dirty_records check.  The db->db_last_dirty field was
  replaced by a proper db->db_dirty_records list_t.  Detect the code
  change and add a osd_db_dirty_txg() helper function which returns
  the largest dirty txg for the dbuf or zero when clean.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: Ifc9573ec410f50f46f2c601368639453e78b291d
Reviewed-on: https://review.whamcloud.com/39822
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
4 months agoLU-13498 sec: fix credentials with nodemap and SSK 40/40140/2
Sebastien Buisson [Mon, 5 Oct 2020 12:14:09 +0000 (21:14 +0900)]
LU-13498 sec: fix credentials with nodemap and SSK

When SSK is enabled, credentials are evaluated in new_init_ucred().
In case a nodemap entry is defined with squash UID/GID, it must
prevail over normally mapped UID/GID.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1adfd98759e5b98ec78f0477846e1820fed5d8b3
Reviewed-on: https://review.whamcloud.com/40140
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13740 build: announce Ubuntu20 support 88/40088/3
James Simmons [Tue, 29 Sep 2020 14:25:24 +0000 (10:25 -0400)]
LU-13740 build: announce Ubuntu20 support

Now that Ubuntu20 is supported for both servers and clients lets
list it in the Changelog

Test-Parameters: trivial
Change-Id: Id413dc5bf5f8e418f379b8b9c8efc4e2d5521311
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/40088
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Sebastien Buisson <sbuisson@ddn.com>
4 months agoLU-13745 test: add splice test for lustre 95/39695/6
Wang Shilong [Sun, 6 Sep 2020 21:56:56 +0000 (17:56 -0400)]
LU-13745 test: add splice test for lustre

copied from xfstests with adjustment with PAGE SIZE
alignement for DIO, and codes style cleanup.

Test-Parameters: trivial envdefinitions=ONLY=425 testlist=sanity
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I28e83abb4a1181db20a2564a10f40ca208bb2756
Reviewed-on: https://review.whamcloud.com/39695
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
4 months agoLU-13845 utils: Quota id 0xFFFFFFFF is invalid 59/39559/5
Etienne AUJAMES [Fri, 31 Jul 2020 18:29:43 +0000 (20:29 +0200)]
LU-13845 utils: Quota id 0xFFFFFFFF is invalid

"lfs setquota" and "lfs quota" should consider as invalid quota id
value 0xFFFFFFFF (aka. (uid_t)-1)

Fixes: 3d9900e78e ("LU-12549 utils: Check range of quota ID for lfs")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Idbd5970a6f53a544c15bdf22bcf24a7aeba772a8
Reviewed-on: https://review.whamcloud.com/39559
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-12931 libcfs: skip cfs_time_seconds() indirection 45/40145/3
Andreas Dilger [Thu, 3 Sep 2020 22:45:02 +0000 (16:45 -0600)]
LU-12931 libcfs: skip cfs_time_seconds() indirection

Avoid one level of indirection when calling nsecs_to_jiffies64().

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib5ff2aae5352bc86d75a8ae2a2a9f1b406887376
Reviewed-on: https://review.whamcloud.com/40145
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13590 kernel: RHEL 7.9 server support 60/40160/2
Jian Yu [Wed, 7 Oct 2020 01:25:11 +0000 (18:25 -0700)]
LU-13590 kernel: RHEL 7.9 server support

This patch makes changes to support new RHEL 7.9 release
for Lustre server (kernel 3.10.0-1160.2.1.el7).

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I7653091f2bd6a579447edb12045984d2829a8235
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40160
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13590 kernel: new kernel [RHEL 7.9 3.10.0-1160.2.1.el7] 42/40142/3
Jian Yu [Wed, 7 Oct 2020 00:09:32 +0000 (17:09 -0700)]
LU-13590 kernel: new kernel [RHEL 7.9 3.10.0-1160.2.1.el7]

This patch makes changes to support new RHEL 7.9 release
for Lustre client.

Test-Parameters: trivial clientdistro=el7.9

Change-Id: I7a2846de48a6710d6d720d6ccc3176dba4afc6bb
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40142
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
4 months agoLU-13614 ldlm: revert LU-11762 32/39532/7
Vladimir Saveliev [Tue, 28 Jul 2020 22:28:22 +0000 (01:28 +0300)]
LU-13614 ldlm: revert LU-11762

Commit fe5c801657 introduced a problem for recovery.

When recovery timeout reaches hard recovery timeout
target_recovery_overseer() leaves obd_recovery_expired flag set. That
makes check_for_next_transno() to not wait until next replay request
arrives which leads to assertion:
LASSERT(atomic_read(&obd->obd_req_replay_clients) == 0);

Test to illustrace the issue is added.

replay-single.sh:test_59 is added to EXCEPT_ALWAYS list:
  it was broken harmlessly before this patch and this patch made that
  test really fail due to that defect.

Fixes: fe5c80165 ("LU-11762 ldlm: ensure the recovery timer is armed")
HPE-bug-id: LUS-8299
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: Ia694a519b5d73620be3014e92fd671d388550979
Reviewed-on: https://review.whamcloud.com/39532
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-14006 o2ib: raise bind cap before resolving address 27/40127/2
John L. Hammond [Fri, 2 Oct 2020 18:55:01 +0000 (13:55 -0500)]
LU-14006 o2ib: raise bind cap before resolving address

In kiblnd_resolve_addr(), ensure that the current task has
CAP_NET_BIND_SERVICE before calling rdma_resolve_addr() with a
protected source port.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I0552fdd64648ccb8c74667bd93852697f99f0c33
Reviewed-on: https://review.whamcloud.com/40127
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
4 months agoLU-13998 tests: sanityn test_104 incorrect MDT times shown 61/40061/3
Serguei Smirnov [Sat, 26 Sep 2020 17:23:21 +0000 (10:23 -0700)]
LU-13998 tests: sanityn test_104 incorrect MDT times shown

sanityn test_104 prints STAT times instead of MDT times.
Fix it so it prints both for comparison.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I4ae0c1fe704c02c3463043830783cd1d6cd46b98
Reviewed-on: https://review.whamcloud.com/40061
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
4 months agoLU-13992 llite: ASSERTION( last_oap_count > 0 ) failed 50/40050/2
Andriy Skulysh [Tue, 16 Jun 2020 09:49:07 +0000 (12:49 +0300)]
LU-13992 llite: ASSERTION( last_oap_count > 0 ) failed

Punch uses o_blocks to send end of a region. So it
can be mixed with real blocks count on error.

Update blocks count only on success.

Change-Id: I86241c4e5723079b20401805b853d356130f58d9
HPE-bug-id: LUS-7407
Test-Parameters: fstype=zfs
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://review.whamcloud.com/40050
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13986 target: fix possible liveloop in distribute_txn thd 43/40043/3
Mr NeilBrown [Fri, 25 Sep 2020 05:02:42 +0000 (15:02 +1000)]
LU-13986 target: fix possible liveloop in distribute_txn thd

A recent patch to update_trans.c changed how distribute_txn_thread()
waited for more work to do.

It previously had an explicit "wait_event()" which listed all the
conditions to wait for.  It would then recheck each condition and
possibly perform an appropriate action.

It was changed to check each condition only once (per loop).  If the
condition was true, the action would be performed and a flag set.  If
no conditions were true (indicated by flag), it would wait, otherwise
it would loop and recheck all condition.

One of the "if (condition) { do work }" stanzas in the loop tested a
condition that was *not* a condition that should wake up the loop.
"batchid" was not tested at all in the wait_event().  The flag
mentioned above was, however, set when that condition tested true.
This can cause the loop to spin indefinitely.

So remove the "__set_current_state(TASK_RUNNING);" so that the value
of batchid cannot stop the loop from sleeping (calling 'schedule()').

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I124ee3e8250dc63fa927f72dc4d29ed3e7b53005
Reviewed-on: https://review.whamcloud.com/40043
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 months agoLU-13701 utils: fix usage of getopt_long in portals 36/40036/4
Serguei Smirnov [Thu, 24 Sep 2020 23:49:05 +0000 (16:49 -0700)]
LU-13701 utils: fix usage of getopt_long in portals

Due to char being unsigned by default, using this type to store
the int return value of getopt_long causes parsing issues on arm
platform, seen with lctl net_drop_add options. Fix it.

Test-Parameters: trivial testgroup=review-ldiskfs-arm
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ibef6ba347a90d9ed3cd4bd9346ed9f79f0120b87
Reviewed-on: https://review.whamcloud.com/40036
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>