LU-15070 mdt: revoke remote LOOKUP lock for default LMV

When setting default LMV, it will revoke LOOKUP lock, while if dir
is remote dir, its LOOKUP lock is on MDT where its parent is located.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I9f079a0bcff530603725ce72cd89c14935ba913b
Reviewed-on: https://review.whamcloud.com/45236
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15151 tests: use facet check instead of node check

Change wait_update_cond() call to wait_update_facet_cond()
call in test_119.

Fixes: 98f107b53e4d ("LU-9699 osp: don't assert on OSP duplicating")
Test-Parameters: trivial testlist=conf-sanity env=ONLY=119
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-10557
Change-Id: Ia1ff7921026212814ec71e0c3aa60f23fbd7278f
Reviewed-on: https://review.whamcloud.com/45369
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-15160 kernel: kernel update SLES12 SP5 [4.12.14-122.91.2]

Update SLES12 SP5 kernel to 4.12.14-122.91.2 for Lustre client.

Test-Parameters: trivial clientdistro=sles12sp5 \
env=SANITY_EXCEPT="56oc 430c 817" testlist=sanity

Change-Id: Ia6620869fa84d72f8d22c4a8a039600037ddb2d9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/45358
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15149 lnet: Missing newline in lnet_add_route

CWARN string is missing a newline character.

Test-Parameters: trivial
Fixes: 3f2844dc93 ("LU-14945 lnet: don't use hops to determine the route state")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I06370c36e9d88b7e02e000bfb573297ff281aef1
Reviewed-on: https://review.whamcloud.com/45340
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15136 socklnd: default conns_per_peer to 0

Setting conns_per_peer to 0 triggers socklnd to choose the
(heuristically) optimal setting for the interface given its speed.
Make 0 the default for socklnd conns_per_peer.

Test-parameters: trivial testlist=sanity-lnet

Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Fixes: c44afcfb72 ("LU-12815 socklnd: set conns_per_peer based on link speed")
Change-Id: Ie6e76eaee8693472384cce362b394b216142884e
Reviewed-on: https://review.whamcloud.com/45319
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15150 tests: sanity-lnet removes testsuite log on failure

cleanup_testsuite() needs to be more selective when removing files
created by sub-tests.

Test-Parameters: trivial testlist=sanity-lnet
Fixes: aa739144551 ("LU-13569 tests: Check LNet Health recovery logic")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ic17a68ff2aa552594a0f1ea470c39177abe985fc
Reviewed-on: https://review.whamcloud.com/45342
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15131 tests: check parameter correctly

sanity test_254() always skipped due to wrong
parameter path: MDT0 not initialized before get_param.

Fixes: 33aad7829de5 ("LU-10461 tests: call exit in the skip routine")
Test-Parameters: trivial testlist=sanity
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-9969
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: I02e38a27ca4656128e62339d63425f85386fa905
Reviewed-on: https://review.whamcloud.com/45300
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15061 tests: fix sanity-dom exit status

If any of sanity tests fails during "ONLY=sanity sh sanity-dom.sh"
execution -- the next "ONLY=sanityn sh sanity-dom.sh" is always
detected as failed even if all sanityn tests pass. This is
because of ${TMP}/sanity.log exists and is checked for
"FAIL" also, while ${TMP}/sanityn.log is to be proceeded only.

Fixes e2cb43c409b9 ("LU-13773 tests: subscript failure propagation")

Test-Parameters: trivial testlist=sanity-dom
HPE-bug-id: LUS-10480
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Change-Id: I764d1d6df08da13acfecd445e67ced1b455ddce8
Reviewed-on: https://review.whamcloud.com/45299
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15098 tests: sanity-sec 27a exec commands on right node

In nodemap_exercise_fileset called from sanity-sec test 27a,
make sure all commands are executed on first client, as we are
testing properties of nodemaps 'default' and 'c0'.
And make sure 'default' nodemap has admin and trusted properties
set to 1, as we are carrying operations as root.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec clientcount=2 env=ONLY=27a
Fixes: 0daeebcbdc ("LU-14797 nodemap: map project id")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Idd9f391db60475721f3a3856b5e3bee1a18bbbca
Reviewed-on: https://review.whamcloud.com/45293
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15103 tests: clean up busy cifs mount

Busy cifs mount point makes cleanup_cifs fail which
will infact fails lustre unmount as cifs mount point is
lustre mount.

Test-Parameters: trivial
Signed-off-by: gaurav mahajan <gaurav.mahajan@seagate.com>
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-4146
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: I4b7ec7e0a6a706198e328dad337648bf3cb2c3be
Reviewed-on: https://review.whamcloud.com/45238
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15099 kernel: kernel update RHEL7.9 [3.10.0-1160.45.1.el7]

Update RHEL7.9 kernel to 3.10.0-1160.45.1.el7.

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I11c307bfd6a6b353bc7b6fe40bb5d604bc9b3fdc
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/45226
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13717 sec: missing defs in includes for client encryption

Add a few missing definitions in lustre_crypto.h that are required
in case Lustre client encryption is built against the in-kernel
fscrypt library.

Fixes: 028281ae19 ("LU-13717 sec: rework includes for client encryption")
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1965503554dcf660770d201444cfafd54aa84dce
Reviewed-on: https://review.whamcloud.com/45221
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15093 libcfs: Check if param_set_uint_minmax is provided

Linux kernel v5.15 commit 2a14c9ae15a38148484a128b84bff7e9ffd90d68
moved param_set_uint_minmax to common code.

HPE-bug-id: LUS-10469
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ifd1d72ae531f0f6c7cd96cc28fbc07c8a8b70886
Reviewed-on: https://review.whamcloud.com/45214
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14999 mdt: Deadlock on parent during resend

Parent-child lock order gets broken during resend as
there is child lock already but there isn't parent lock
and MDS tries to lock it again.

Don't lookup child on resend, extract fid from the lock instead.

Change-Id: I443648a8162e770c63fd087dd534c27e7c637c40
HPE-bug-id: LUS-9306
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/44885
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14929 gss: detect libkeyutils dependency

When building GSS support, gss_keyring requires libkeyutils.
So make sure this dependency is properly detected at configure time,
and include keyutils.h only when required.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9fa5750f4609250ecdc1c47f68b97bff9be13ace
Reviewed-on: https://review.whamcloud.com/44597
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14124 target: set OBD_MD_FLGRANT in read's reply

If tgt_grant_shrink() decides to not shrink grants - a client is
supposed to restore its cl_grant_avail in osc_update_grant(). In case
of read OBD_MD_FLGRANT is not set on reply's body->oa.o_valid, so
osc_update_grant() misses the cl_grant_avail update. As result server
keeps thinking that client has a lot of grants while a client thinks
that it is missing grants badly. That may lead to performance
degradation.

A test to illustrate the issue is included.

Test-Parameters: trivial testlist=sanity
Change-Id: Ibe7ce0af5701226c8be3ae3f9ad57c354791fa0f
Signed-off-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
HPE-bug-id: LUS-9943
Reviewed-on: https://review.whamcloud.com/43375
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15146 mdt: mdt_lvb2reply crash fix

mdt_lvb2body may crash if res->lr_lvb_data is
not allocated, make it tolerant to not initialized
lvb_data pointer.

HPE-bug-id: LUS-9549
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: Ie31cbba9187f9b04b3d1f8d2abc59e0612a44b41
Reviewed-on: https://review.whamcloud.com/45334
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15047 gss: gss integrity check with multi-rail

With multi-rail, a primary NID is used as node identifier, but LNet
decides which NID is actually used for sending/receiving data, on a
per request basis.
For the integrity check mechanism implemented as part of GSS, the
primary NID must be used in order to compute HMAC with the correct
key, independently of the actual NID for the current request.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I2bf3974d3aa0e8365a9413dca56c69ee3734c12b
Reviewed-on: https://review.whamcloud.com/45277
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14587 ptlrpc: remove LASSERT in nrs_polices proc handler

It's not necessary to LASSERT() in nrs_polices proc handler.
CERROR() and returning error is good enough.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I09f06dc4ab90e49b2df66a9b47a74678c64cdd2f
Reviewed-on: https://review.whamcloud.com/45200
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15089 tests: allow enough time to create tcp connections

Allow enough time to create tcp connections before counting them
when testing socklnd conns_per_peer setting in sanity-lnet test_230

Test-Parameters: trivial testlist=sanity-lnet
Fixes: a5cbe7883db6 ("LU-12815 socklnd: allow dynamic setting of conns_per_peer")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ia3e25de157da03d6129b108c1af9632a8faf8efd
Reviewed-on: https://review.whamcloud.com/45331
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15125 o2iblnd: wrong list used for kib_connd_waits

The ibc_list field of struct kib_conn is used for the kib_connd_waits
list, but kiblnd_connd() uses ibc_sched_list in the
list_first_entry_or_null macro.

Test-Parameters: trivial
Fixes: 34b57a6f8f ("LU-12678 lnet: use list_first_entry() in lnet/klnds subdirectory.")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I0cc8a94636a5129956c53e48ae96b27ece5f0228
Reviewed-on: https://review.whamcloud.com/45316
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15118 ldlm: no free thread to process resend request

MDS grants lock request and sends a reply,
input request queue can be filled immediately
with conflicting lock enqueue request.
So there isn't any free thread to process resend of
first lock enqueue request if the client has failed
to receive the reply.

Process lock enqueue resends with existing lock on MDS
in high priority queue.

Change-Id: If7b94690100b44c774dc14231ed4902f701ed807
HPE-bug-id: LUS-9444
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/45272
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15106 ofd: quiet deprecated param warning

There are a number of obdfilter parameter files that report a
warning even when they are read, which is confusing for users
if there is a tool that is scraping all available parameters:

    # lctl get_param obdfilter.*.*
    ofd: 'obdfilter.*.read_cache_enabled' is deprecated,
         use 'osd-*.read_cache_enabled' instead
    ofd: 'obdfilter.*.readcache_max_filesize' is deprecated,
         use 'osd-*.readcache_max_filesize' instead
    ofd: 'obdfilter.*.sync_on_lock_cancel' is deprecated,
         use 'obdfilter.*.sync_lock_cancel' instead
    ofd: 'obdfilter.*.writethrough_cache_enabled' is deprecated,
         use 'osd-*.writethrough_cache_enabled' instead

It should only print a message if the parameters are actually written.
Also fix the messages to reference the correct parameter names.

Most of these parameter links were added in 2.4 with the addition of
osd-ldiskfs.  However, the deprecation warnings were only added in
2.12.53 and slated for removal in 2.15, but were not backported to
2.12 LTS, and there hasn't been an LTS release since then, so it is
better bump removal so the upcoming 2.15 LTS release includes them.

Fix the test scripts to only use the new parameter names, to avoid
spurious warning messages.  We don't test interop with 2.3 anymore.

Test-Parameters: trivial
Fixes: 7df7347b7b18 ("LU-12967 ofd: restore sync_on_lock_cancel tunable")
Fixes: 493cd8088388 ("LU-8066 osd: migrate from proc to sysfs")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie548e5b6af5463959fb4774e31996097373ebbe5
Reviewed-on: https://review.whamcloud.com/45246
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15104 tests: create client dirs with custom stripe count

Random ha_dir_stripe_count allows clients to create directories
with different stripe counts in one ha.sh test session.
Please use
  DSTRIPECOUNT=3 DSTRIPECOUNTRAND=true
to create the directories with stripe counts 1,2 or 3.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-10435
Reviewed-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Change-Id: I0092958a9bcf1991adc9f45ac1fbed9340a06c57
Reviewed-on: https://review.whamcloud.com/45240
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14195 obdclass: change list_sort to use const pointers

Kernel 5.10.70 commit 4f0f586bf0c898233d8f316f471a21db2abd522d
defines the list_cmp_func_t type and changes the comparison
function types of all list_sort() callers to use const pointers
to avoid type mismatches.

Change-Id: I40d37ec0f0d485d0deebaa9dc3493f2865f76ec9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/45219
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15081 vfs: set_nlink() is not race-safe

set_nlink() is not atomic wrt race with itself and
the following warning may be triggered by VFS:

WARNING: CPU: 5 PID: 195090 at fs/inode.c:241 __destroy_inode+0xdb/0xf0

It does not seem important what exact nlink value is the result
of the race. However, we need to protect the superblock remove
counter.

Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
HPE-bug-id: LUS-9825
Change-Id: I67bc345b9a9e43fb88d919a83246759d11604b03
Reviewed-on: https://review.whamcloud.com/45191
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15045 utils: fix lfs_migrate for files with spaces

Fix the lfs_migrate script to properly quote "$OLDNAME" so that
it works for filenames with spaces and other characters in them.

Test-Parameters: trivial
Fixes: 8bedfa377fbd ("LU-11510 lfs: migrate a composite layout file correctly")
Fixes: 128137adfc53 ("LU-13090 utils: fix lfs_migrate -p for file with pool")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic00f41f3a91ad9dfa491ff57768a3da0c6300c1e
Reviewed-on: https://review.whamcloud.com/45173
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15071 utils: tunefs erease-params for zfs

The patch exclude special zfs params for tunefs erase-params,
skip nvlist modifying. Also fixes test_89 conf-sanity.

tunefs --erase-params produced segmentation fault with old code.

Test-Parameters: trivial fstype=zfs testlist=conf-sanity
HPE-bug-id: LUS-10314
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ic8385a99ca896ce6d855692b3f77e198bf583d94
Reviewed-on: https://review.whamcloud.com/45145
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15067 lod: fix error handling in lod_new_pool

- correct error handling in lod_new_pool - ENOMEM
 from tgt_pool_init may cause incorrect pool_count.
- optimisation in lu_tgt_pool_add. Do not extend
 a pool, if the target is already exists.

HPE-bug-id: LUS-6995
Change-Id: I0ea00d335217f3994334bd02ae36c34653e7da98
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/45137
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15011 lod: count all spilling events

when target pool is used to as the original has no enough space.
lctl lod.*.pool.<poolname>.spill_hit can be used to get the counter.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I6d54a2b910705da182b5f4118e535d196cdab004
Reviewed-on: https://review.whamcloud.com/44947
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13076 dne: dir migrate in QOS mode

Support "lfs migrate -m -1 ..." to migrate directory to MDTs by
space and inode usage, if system is balanced, the target MDT is
chosen in roundrobin mode, otherwise the less full MDTs will be
chosen, and the most full MDT is avoided.

Another minor change: if directory is migrated to specific MDTs,
and the target stripe count is more than 1, its subdirs may not be
migrated to the specified MDT in the command, but migrated to the
MDT where its parent stripe is located (subdir will be striped too),
as can avoid unnecessary remote directories. NB, for command like
"lfs migrate -m 0,1,2 ...", though the subdir may be located on
either MDT0, MDT1 or MDT2, its stripes will be striped over these
three MDTs, but for command like "lfs migrate -m 0 -c 3...", the
subdir may be striped on other MDTs if the subdir is not located on
MDT0.

Add sanity 230u.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I6e9c3d75bfc240b21c65ba27cd5e4bcca7058325
Reviewed-on: https://review.whamcloud.com/44886
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14927 quota: move qsd_transfer to lquota module

With osd-zfs inheriting the tainted state of ZFS we can no longer
directly use kernel internals that exported GPL only. The osd-zfs
modules quota code uses some of these internal functions. The
osd-zfs qsd_transfer() function is generic enough that we can
move it to the lquota modules.

Change-Id: I735db0266306477cd5558968f1f3ed1f6e1b32da
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/44735
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14938 tests: fail_abort() in t-f to take care of MDTs

fail_abort() in test-framework ensures that the clients
are back after evictions. the same should be done for
MDTs as otherwise any subsequent test may fail due to
another MDT observing eviction and interrupting current
request with -EIO.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0a00ece52d28c6d28eef029a4f87a348efaa041c
Reviewed-on: https://review.whamcloud.com/44671
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14937 build: re-use config cache in 'make rpms/debs'

Idea is to pass along the value of the -C or --cache-file options from
the initial ./configure to the one launched as part of the rpm or deb
build.
But all the environment variables related cache info must be removed
otherwise the config cache file cannot be reused.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iab4ae2815961ba10132d9cb44f82ca58d313e908
Reviewed-on: https://review.whamcloud.com/44659
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14909 test: mkdir_on_mdt0 to mkdir on MDT0

Many sub tests in recovery-small.sh and replay-single.sh need to mkdir
on MDT0, use mkdir_on_mdt0() to create such directories.

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=recovery-small.sh,replay-single.sh
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id4a44da062350ea284f51c8c821302aebbfe9dee
Reviewed-on: https://review.whamcloud.com/44544
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14831 osd-ldiskfs: uninited osd_inode_id is used

In osd_fid_lookup, the "osd_inode_id" could be used uninitializedly
if the FID doesn't exist in OI, which cause some faked FID/inode
pair to be inserted into OI file and the OI scrub thread could be
triggered repeatedly.

Change-Id: I9100dece9e94d3e590f17fb4498601876aa1edaa
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/44349
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14174 lfs: llapi_mirror_find() signed return

Check return codes from llapi_mirror_find() as signed value
to don't count errors as valid mirror ids

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I9754eaa063c9a2d07d8b815a86e7597922201f9c
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41757
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-14409 ldiskfs: Add support for SUSE 5.3.18-24.46.1

Linux-commit: f902b216501094495ff75834035656e8119c537f
ext4: fix bogus warning in ext4_update_dx_flag()

The update breaks the ldiskfs pdirop patch which disables
ext4_update_dx_flag.

SUSE 5.3.18-24.46 can directly use the 5.4.0-66-ubuntu20.series

Test-Parameters: trivial
HPE-bug-id: LUS-9684, LUS-9758
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I9271ee80c05715d7dcec78535cfde1e384ba40e9
Reviewed-on: https://review.whamcloud.com/41473
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12391 tests: mdsrate tests improvements

Patch improves mdsrate tests to work with
striped directories which are created if
MDSRATE_ENABLE_DNE=true.

mdsrate.c is fixed to not fail if --mdtcount
option is used and ndirs=1. Without this fix
  mdsrate --mdcount
fails as:
cannot create stripe dir: File exists
when the ranks do system(mkdir_cmd) on the
directory which was already created by the
first executed rank.

Patch sets mdt.*.enable_remote_dir_gid to "-1"
to allow mpiuser to create remote directories.

Patch makes the mdsrate based tests a bit more
verbose: mdsrate create/mknod is called with
--debug option if VERBOSE set to "true".

Test-Parameters: trivial testlist=performance-sanity
Fixes: f31c60c97328 ("LU-1187 tests: Add mntfmt/mntcount/mdtcount to mdsrate")
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-7262
Reviewed-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ieb32ad7dfad838fc9124740236889a5fe47cb901
Reviewed-on: https://review.whamcloud.com/35069
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11407 obdclass: add start time to stats files

When the stats files are initialized or reset, store the current
timestamp with the stats.  That allows computing average IO and
RPC rates over the accumulated stats lifetime, in addition to the
normal incremental operation rates found by comparing successive
values read from the stats file with the read interval.

Any stats that currently print the "snapshot_time:" header will
now also print "start_time:" and "elapsed_time:" fields as well.
Consolodate this printing into a helper function instead of
duplicating very similar code in many different functions.  Output
can't be exactly the same for all callers, because these fields are
embedded into different types of output files, but it is very close.

Change struct rename_stats and brw_stats to use a common name prefix.

Change the obd_job_stats timestamps to ktime_t so that we can use the
common helper function for printing the header.  It is easier to store
ojs_cleanup_interval internally as 1/2 of the maximum stats age, since
since division is more easily done when the value is initially set as
seconds compared to when it is ktime_t.  This may also be a tiny bit
more efficient since we don't do a divide/shift on each access.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iacefa17def455ef53a28fd14b6d4c670463ebbe5
Reviewed-on: https://review.whamcloud.com/33201
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15115 ptlrpc: recalc timer on EINPROGRESS reply

ptlrpcd doesn't recalculate wait queue timer after
getting -EINPROGRESS reply. It may delay request resend
till its timing out.

HPE-bug-id: LUS-10366
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: Idc76c688a0f7ff8e110446fd1fe13dd83f636f3b
Reviewed-on: https://review.whamcloud.com/45266
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2084 lnet: don't retry allocating router buffers

Don't loop indefinitely trying to allocate router buffer pools if
the number of requested buffers is too large for the system.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic0f2ccf0f7b38dfa254e46e268b27092342efdb5
Reviewed-on: https://review.whamcloud.com/45174
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14976 ptlrpc: align function names with param names

Change the internal function names for the ptlrpc proc tunables
to match the parameter names exposed to userspace.  Otherwise it
is needlessly complex to find the function that implements the
"nrs_policies" parameter, since the parameter use itself is wrapped
in a macro that generates the proc handling structure.

Clean up code style in related functions.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9079766cccade963f1510cfcce228da9be3ebbe5
Reviewed-on: https://review.whamcloud.com/44817
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15076 socklnd: lock ksnc_tx_queue list processing

A GFP occurred in the ksocknal_find_timed_out_conn() while processing
ksnc_tx_queue list.

Add locking to this list.

Change-Id: I1f76683e5798c5015f11e3fa285db9613b1af906
Signed-off-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
HPE-bug-id: LUS-10248
Fixes: 25c1cb2c4d ("LU-9120 lnet: handle socklnd tx failure")
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-on: https://review.whamcloud.com/45179
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13997 tests: fix sanity test_418 lock cancellation

Use "do_nodes" directly to cancel DLM locks, rather than
"do_rpc_nodes", since that is very heavy to use in a loop
(each call takes 3s, but the loop delay is only 0.2s).

Due to DoM reserving grant space for the DoM files, the "avail"
space shown by "df" may be smaller in the aggregate returned by
the MDT compared to the individual values from "lfs df".

Skip this part of the check until MDC grant cancel is fixed.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I83d989688ce671f0ff9c62ebdf3144746a3ebbe5
Reviewed-on: https://review.whamcloud.com/45231
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14448 lod: verify LOV before set/inherit

DoM layout can only be set as entry in composite layout, and its
stripe count should always be 0.

Verify LOV before setting and inheriting.

Add sanity 270i.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I016d1a202960bfebc72dd808de5f80e09051a01e
Reviewed-on: https://review.whamcloud.com/45039
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13195 osp: osp_send_update_req() should check generation

and don't send requests depending on just failed one

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I27a2b21130e33287168204ad829c0a53002b517e
Reviewed-on: https://review.whamcloud.com/45042
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12807 tests: fix intermittent runtests failure

Occasional runtests failures are seen in full testing on ldiskfs.
Increase the llog space limit to 72KB from 50KB due to seeing a
regular failures in the 52/64KB range.

Test-Parameters: trivial testlist=runtests
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6e272fe9fec21a650110a42efe31a1dc48e35854
Reviewed-on: https://review.whamcloud.com/44614
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Vikentsi Lapa <vlapa@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15074 build: Use strlcpy if strscpy is not available

Linux commit v4.2-rc1-2-g30035e45753b
    string: provide strscpy()

    The strscpy() API is intended to be used instead of strlcpy(),
    and instead of most uses of strncpy().

Unfortunatley strscpy is not always available.

Test for strscpy and fallback to strlcpy when strscpy is
not available.

Test-Parameters: trivial
Fixes: b77a6d86936 ("LU-14665 lnet: simplify lnet_ni_add_interface")
HPE-bug-id: LUS-9546
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I43038e4a6260dafb57195ec3417ce009f5a3fad4
Reviewed-on: https://review.whamcloud.com/45175
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15064 tests: sanity-sec test_58 must read its own dir

sanity-sec test_58 should restrict file listing to its own
directory, and not try to list content of the entire Lustre tree.
This is useless for the purpose of the test, and exposes to unclean
remnants from previous tests.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=58
Fixes: 1faf54e8bf ("LU-14989 sec: access to enc file's xattrs")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic9ab0860da0ab86355a207ad9d50feb3975adf68
Reviewed-on: https://review.whamcloud.com/45150
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15060 tests: sanity-flr test_208[a,b] fix

sanity-flr  test_208a  and test_208b failed as:
  test_208a returned 2
  test_208b returned 2
on Lustre setup where osts are located not on one host
because of stack_trap "do_nodes $osts $LCTL set_param $old"
returns 2. Let's use save_lustre_params() instead of trying to
set not-existing parameters.

Fixes: 8507472dd37e ("LU-14996 lov: prefer mirrors on non-rotational OSTs")
Test-Parameters: trivial testlist=sanity-flr env=ONLY=208
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Change-Id: I19cedc0a9745d0d112ac05fe3a800347ab4c40d3
Reviewed-on: https://review.whamcloud.com/45130
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15011 tests: ost-pools to remove big files

otherwise those files can affect subtest 29

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia07c10882aba97758a3d11965693134eb2238e9a
Reviewed-on: https://review.whamcloud.com/45120
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-15050 tests: do not ignore SANITY_ONLY and SANITYN_ONLY

sanity 150[b,bb,c,d,f,g] and sanityn 107 tests are to be included
into SANITY_ONLY and SANITYN_ONLY lists only if these lists not
set by user.

Fixes: 3c75d2522786 ("LU-10664 dom: non-blocking enqueue for DOM locks")
Fixes: 163870abfb7c ("LU-14382 mdt: implement fallocate in MDC/MDT")
Test-Parameters: trivial testlist=sanity-dom env=SANITY_ONLY=1,SANITYN_ONLY=1
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Change-Id: I686731df3dfbfe1b7d4ae2e8621d1b0c10c48a22
Reviewed-on: https://review.whamcloud.com/45107
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>

LU-15039 lnet: Fix reference leak in lnet_parse

We need to drop the reference taken by lnet_nid2peerni_locked() if we
determine that we need to drop the message because of asymmetric
route.

Test-Parameters: trivial
HPE-bug-id: LUS-9186
Fixes: 955080c3ae ("LU-13779 lnet: Correct asymmetric route detection")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I799c9522b1ce5f4caffc5848a829995e5b5484e7
Reviewed-on: https://review.whamcloud.com/45067
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15010 mdc: add support for grant shrink

just re-use existing mechanism used in OSC

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I4cdca057d35eaff6493d047127f1fe5eee9e9620
Reviewed-on: https://review.whamcloud.com/44956
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14659 test: improve generate_uneven_mdts() in sanity.sh

Improve generate_uneven_mdts() in several places:
1. set qos maxage to 1, so the result is up to date, and avoid filling
   up MDT.
2. fill MDT with files of size 64K other than 1M, so MDT imbalance is
   quicker to achieve.
3. when checking minimum imbalance after test, lookup max value from
   the result, other than by index stored before directory creation,
   because the result is dynamic if several MDTs have almost the same
   free space and inodes.

Test-Parameter: trivial mdscount=2 mdtcount=4 testlist=sanity
Fixes: 233344d451e ("LU-13417 test: generate uneven MDTs early for sanity 413")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I2807101ff632404e25fdb640840d83d1991c88d9
Reviewed-on: https://review.whamcloud.com/44649
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-14905 lfsck: linkEA overflow handling fix

An absent link in EA is not an issue and not to be fixed if EA is
overflowed. lfsck should not report it is an issue if there is no
space for this link, and should not report it is fixed whereas it
is not (linkea_add_buf() returns 0 if so without having a new entry
added into EA and lfsck_namespace_assistant_handler_p1() later
reports it is repaired).

HPE-bug-id: LUS-8810
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Change-Id: Iba1549045c8c3889adf55c99cdd88756e5643073
Reviewed-on: https://es-gerrit.dev.cray.com/158706
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Tested-by: Jenkins Build User <nssreleng@cray.com>
Reviewed-on: https://review.whamcloud.com/44469
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14866 lod: remove duplicate OST_TGT

Remove duplicate OST_TGT from lod_ost_alloc_qos.

Change-Id: I4fbe2daa057f23a60e31e59d7c0db592945a5363
Fixes: 2112ccb3c4 ("LU-13073 osp: don't block waiting for new objects")
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/44351
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14989 sec: keep encryption context in xattr cache

When an inode is being cleared, its xattr cache must be completely
wiped. But in case of lock cancel, we want to keep the encryption
context, as further processing might need to check it.

Fixes: 1faf54e8bf ("LU-14989 sec: access to enc file's xattrs")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8a2f4497129353a7fbf86cdaaa13fae6e0988790
Reviewed-on: https://review.whamcloud.com/45148
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13941 osp: Silently lower requested create_count to maximum

When setting create_count it should silently accept a larger value
and truncate it to the current maximum.

This would avoid issues if that limit is changed in the future.

HPE-bug-id: LUS-5960
Test-Parameters: trivial testlist=parallel-scale,sanity
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4727ba6fca747e1ae9850188ef63c7abb89904be
Reviewed-on: https://review.whamcloud.com/39967
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14797 nodemap: map project id

Add calls to nodemap_map_id() in order to map project IDs from
client ID to server ID and conversely.
Also extend nodemap_can_setquota() to allow setquota on project
only if ID is not squashed or deny_unknown is not set.
Update sanity-sec test_27a to exercise the feature.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id66458550d312404b1993ead8940c3d12eaadccd
Reviewed-on: https://review.whamcloud.com/44119
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15052 lnet: include linux/ethtool.h

Kernel 5.11.0-36 removes including linux/ethtool.h from
linux/netdevice.h, which caused the following build error:

dereferencing pointer to incomplete type 'const struct ethtool_ops'

This patch fixes the above issue by adding the include into
the file that uses the structure.

Test-Parameters: trivial

Change-Id: Ifc25de5acaebf2b5fd5bb6f1c303366ab9ea6ef6
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/45109
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 obdclass: Remove lprocfs_obd_short_io_bytes_* declarations

The functions themselves were long renamed

Change-Id: Ic97d83a56d065ff1dadfc9a01d878e246e06a847
Test-Parameters: trivial
Fixes: 32fb31f3bf ("LU-8066 osc: move suitable values from procfs to sysfs")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/45096
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>

LU-15040 mdc: update max_easize on reconnect

If MDS was restarted to enable ea_inode, clients should get new
max_easize value. However, cl_max_mds_easize is not updated. This may
cause lfs getstripe to fail if file has huge stripe number
(2000 for example):

*** Error in `lfs': free(): invalid pointer: 0x0000000000de09d0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81299)[0x7f0623c03299]
/lib64/libc.so.6(closedir+0xd)[0x7f0623c42ddd]
/lib/liblustreapi.so.1(+0xa557)[0x7f06248b5557]
/lib/liblustreapi.so.1(+0xad74)[0x7f06248b5d74]
lfs[0x4105b3]
/lib/liblustreapi.so.1(Parser_execarg+0x51)[0x7f06248c88e1]
lfs[0x40448e]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f0623ba4555]
lfs[0x4044fc]

HPE-bug-id: LUS-9478
Change-Id: If155a63e2f07536c6500b37b5e6191cb8b0d0607
Reviewed-on: https://es-gerrit.dev.cray.com/158100
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Nikitas Angelinas <nangelinas@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/45073
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12268 osd: BUG_ON for IAM corruption

The patch adds strict checks of buffer head overflow
for IAM dx blocks.

HPE-bug-id: LUS-10178
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I1608f6cbf00b5120fbc36d0c65fcfe37c43e375f
Reviewed-on: https://review.whamcloud.com/45072
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15038 mgc: release cl_mgc_mutex on error

If local_oid_storage_init() returns an error, the cl_mgc_mutex()
should be released.

Fixes: 3e38436dc09 ("LU-2059 llog: MGC to use OSD API for backup logs")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I921dde4e9202733874d8e7f980e95af23739a655
Reviewed-on: https://review.whamcloud.com/45063
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15011 tests: pool spill test modifications

Make the following modifications to the ost-pools
test suite:
test 29 - change check for 'when striping is specified
          explicitly' file from 'file-2' to 'file-3'

test 30 - Add bad parameter check for setting the threshold
          below zero

test 31 - 'do_nodes $mdts $LCTL get_param lod.*.pool.*'
          doesn’t print anything. Change to
          'do_nodes $mdts $LCTL get_param lod.*.pool.*.spill*'

Fixes: 0a998f4723 (“LU-14825 lod: pool spilling”)
Test-Parameters: trivial testlist=ost-pools
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Icbdc3d42b7f7609bc57cc37830975d831125d659
Reviewed-on: https://review.whamcloud.com/45056
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-15028 tests: improve ha.sh to be more verbose

Patch adds some informing messages to make the failure
reason detection simpler.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-10286
Reviewed-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Change-Id: I3bef165f497d745c3e8ee3c8a91532096100bb99
Reviewed-on: https://review.whamcloud.com/45025
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15026 zfs: Fix ZFS(2.0.0-1) build error on CentOS (3.10)

ZFS: (2.0.0-1)
Lustre: 608cce73d51 LU-15007 tests: quota enable cmd fix
CentOS: 3.10.0-1160.15.2.el7.x86_64

This patch fixes two build failures seens as below for
the above configuration

First
~~~~~
In file included from:
/root/zfs/zfs_git_lustre_build/zfs/include/sys/spa.h:39:0,
from libmount_utils_zfs.c:32:
/root/zfs/<path>/.../sys/zfs_context.h:110:27:
fatal error: sys/byteorder.h: No such file or directory
#include <sys/byteorder.h>

Second
~~~~~~
gcc -rdynamic -shared -export-dynamic -pthread \
-L/root/zfs/zfs_git_lustre_build/zfs/lib/libzfs/.libs/
-L/root/zfs/zfs_git_lustre_build/zfs/lib/libnvpair/.libs
-o mount_osd_zfs.so \
`ar -t libmount_utils_zfs.a` \
-ldl -lzfs -lnvpair -lzpool
/usr/bin/ld: cannot find -lzpool
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
collect2: error: ld returned 1 exit status

Test-Parameters: trivial fstype=zfs
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I5be8cc846da1ca213f1bd3c29b6b55acc22928f2
Reviewed-on: https://review.whamcloud.com/45016
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13575 lnet: Ensure round robin selection of peer NIs

Use the peer net sequence number to set the peer NI sequence number to
ensure round robin selection of peer NIs on each peer net.

HPE-bug-id: LUS-10349
Test-Parameters: trivial
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I1fa14ad675ead4ae2c5b1d4edad250caa4498df2
Reviewed-on: https://review.whamcloud.com/45004
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13575 lnet: Ensure round robin selection of local NIs

Use the net sequence number to set the NI sequence number to ensure
round robin selection of NIs on each net.

Test-Parameters: trivial
HPE-bug-id: LUS-10349
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I6ce0b088fcad6312186e6fbad4ab14283aee55eb
Reviewed-on: https://review.whamcloud.com/45003
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15005 build: Ubuntu dkms packages missing dependancies

It was noted that on astandard ubuntu install the dkms package
would fail to install as libnl-genl-3-dev was missing. As a
test we tried installing the dkms package on an upstream
cloud image for 18.04 and 20.04. We noted that a number of
packages were needed before dkms would install cleanly.

Test-Parameters: trivial testgroup=review-ldiskfs-ubuntu

Signed-off-by: James Beal <jb23@sanger.ac.uk>
Change-Id: I53a2f143dd2154a2d0b598db8c60fd8ff1421860
Reviewed-on: https://review.whamcloud.com/44999
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14474 llog: don't destroy next llog

do not destroy empty llog if it's referenced as
the next one in a catalog.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I78bfeb90435aaee2b8536b647aa3acec56642ea0
Reviewed-on: https://review.whamcloud.com/44998
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15014 obdclass: lu_ref_add() called in atomic context

For the native Linux client testing I turn on lu_ref
debugging. When turned on the following errors occur:

[ 2885.946815] Call Trace:
[ 2885.951240]  dump_stack+0x68/0x9b
[ 2885.956523]  ___might_sleep+0x205/0x260
[ 2885.962245]  lu_ref_add+0x25/0x40 [obdclass]
[ 2885.968442]  vvp_pgcache_current+0x101/0x1a0 [lustre]
[ 2885.975370]  seq_read+0x1ab/0x3c0

and

[ 7042.102529]  dump_stack+0x68/0x9b
[ 7042.107328]  ___might_sleep+0x205/0x260
[ 7042.112647]  lu_ref_add+0x25/0x40 [obdclass]
[ 7042.118385]  mdc_lock_upcall+0x154/0x4d0 [mdc]
[ 7042.124275]  mdc_enqueue_send+0x508/0x580 [mdc]
[ 7042.130225]  ? mdc_lock_lvb_update+0x280/0x280 [mdc]

This is easily fixed with introducing a lu_object_ref_add_atomic()
function.

Test-Parameters: trivial
Change-Id: Ife7d255079a836570661f669c1e9c7c0ce6de4aa
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/44969
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15013 osc: use original cli for osc_lru_reclaim for debug msg

Before the list cleanup introduced in osc_lru_reclaim() the
variable cli was both passed in and used to scan the
cl_client_cache. After the scan was done then we use cli in
a debug message. It appears to be the original intent was to
use the original cli passed in for the debug message, not the
last scanned item. After the list cleanup patch landed now
cli can be NULL which can crash the node. The fix is to use
a separate struct client_obd variable for the scan and use
the original cli passed in for the debug message.

Test-Parameters: trivial
Fixes: 8c166f6bf42c ("LU-6142 lustre: use list_first_entry() in lustre subdirectory.")
Change-Id: I5f0f1b986744fdd30af7f7856c1278b41447a373
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/44966
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14927 scrub: create shared scrub_needs_check() function.

The current functions osd_consistency_check() in both ldiskfs and
zfs use ktime_* functions which are exported for pure GPL modules.
This is not the case for ZFS. We can refactor the code to create
a new common function scrub_needs_check() that can be used along
side osd_consistency_check(). Fix a few cases where the error
code is not checked for ZFS.

Change-Id: I0cc6cd84a35ecc10b511096f4e749a2961da3bbf
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/44689
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12262 llite: harden ll_sbi ll_flags

For most file systems mount flags are straight forward but this
is not the case for Lustre. We have to consider if the server
backend supports a mount option. Additionally its possible to
disable or enable a feature using sysfs during run time. Some
features can't be managed with a mount option but still can
be managed with sysfs or based on what is enabled on the server
node. All these states are reported together in the debugfs
file sbi_flags. The mount specific options are reported in
the super block show_option ll_show_option().

With all this complexity it is easy for it to get out of sync
and report incorrect things. We consolidate this handling by
moving to using match_table_t that is used by various Linux
file system to parse options. LL_SBI_FLAGS is replaced by our
match_table_t, ll_sbi_flags_name, that can be used for mount
options as well as reporting the sbi_flags in debugfs. We take
advantage of the fact that mount option parse will stop at the
first NULL in ll_sbi_flags_name and after that NULL list the
other features flags that are managed with other methods besides
mount options.

The next change is the move of ll_flags to a bitmap which gives
us two advantages. The first is that we can support more than
32 flags in the future. Second is no need to use bit shifting
math since we can use th enum LL_SBI_* values directly with
clear_bit() / set_bit() / test_bit(). Allow these changes should
miminize future problems with keeping all these states in sync.

Change-Id: Ia4f08cdde54c0fd11440dcf6b60b5fcb8bfb4d63
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/44541
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14895 client: allow case-insensitive checksum types

The current t10ip4K and t10crc4K checksum types use an upper-case 'K'
in the name, unlike the other checksum types which are all lower-case.
This is distinction is difficult to see in some fonts, and can cause
usage errors.  Accept upper-case variants of the checksum type names.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I97673ffa98cf8e5fc601ac7df5aaafb24b3ebbe5
Reviewed-on: https://review.whamcloud.com/44530
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14911 osp: release thandle if it was created

osp_statfs_update() could leak thandle if transaction couldn't
start for a reason.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I541a5e4a7860008eb179d905ac57997b737f178c
Reviewed-on: https://review.whamcloud.com/44504
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14801: utils: improve performance of 'lfs find -perm'

The current implementation of the "-perm" predicate queries the
stat information on the OST while it is not necessary. This patch
fixes that issue by moving the check to the correct location in
`cb_find_init`

On a simple setup with 1 MDT and 2 OSTs we observed for around 8000
files:
- 2.3s without the patch
- 1.9s with the patch

Test-Parameters: trivial
Change-Id: I30c0e89d136556058eadf6bede062577c6d36eaf
Signed-off-by: Courrier Guillaume <guillaume.courrier@cea.fr>
Reviewed-on: https://review.whamcloud.com/44118
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Rick Mohr <mohrrf@ornl.gov>
Reviewed-by: Anjus George <georgea@ornl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-14797 sec: add projid to nodemap

Add the ability to create id maps of a new type, projid. This also
requires adding a new value to map_mode, projid_only. Finally, a new
property named squash_projid is used to map all project ID to a
default one.
Update lctl man pages to mention these additions.
Update sanity-sec test_12 and test_15 to exercise projid mapping and
squash_projid property.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I63eba8b0d33feaa7ece8c1788cb587fcb330357a
Reviewed-on: https://review.whamcloud.com/44108
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14739 quota: fix quota with root squash enabled

This patch tries to fix several problems:

1. OSD will ignore quota if IO comes from client
cache or root, however since following change:

LU-12687 osc: consume grants for direct I/O

DIO now consumes grant too, following check for
sync IO is wrong now:

(lnb[i].lnb_flags & (OBD_BRW_FROM_GRANT | OBD_BRW_SYNC))
        == OBD_BRW_FROM_GRANT)

This wass originally added to support 1.8 client, it is
going to be 2.15 now, so let's remove this broken check.

2. Server side will clear OBD_BRW_NOQUOTA if root squash
is enabled, this will revert fixes from:

"LU-13228 clio: mmap write when overquota"

We need to separate @ci_noquota and @oi_cap_sys_resource cases,
introduce a new flag OBD_BRW_SYS_RESOURCE, and extend test_75
to cover this case.

3. LU-14739 missed case that DoM quota should be considered
as well.

4. If EDQUOT is returned for root, we check the new root squash
flag OBD_FL_ROOT_SQUASH from server side. If this flag is not set,
we bypass quota for root, otherwise all root writes become sync
writes.

5. Fix a leftover problem with LU-9671 for DOM

Fixes: a4fbe7341baf12 ("LU-14739 quota: nodemap squashed root cannot bypass quota")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I3fd23da7d56acb5b485540333208e5d5b0b48023
Reviewed-on: https://review.whamcloud.com/44347
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14475 log: Rewrite some log messages

Some log messages are too short to be meaningful. Rewrite them.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I9ae7d7da23c7e227c4e2b84010fb0c2a06b8cc87
Reviewed-on: https://review.whamcloud.com/41892
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>

LU-12362 ptlrpc: use wait_woken() in ptlrpcd()

Using wait_event() to wait for ptlrpcd_check() to succeed is
problematic.  ptlrpcd_check() is complex and can wait for other
events.  This nested waiting can behave differently to expectation and
generates a warning

   do not call blocking ops when !TASK_RUNNING

This happens because the task state is set to TASK_IDLE before
ptlrpcd_check() is calls.

A better approach (introduce for precisely this use-case) is to use
wait_woken() and woken_wake_function().

When a wake_up is requested on the waitq, woken_wake_function() sets a
flag to record the wakeup.  wait_woken() will wait until this flag is
set.  This way, the task state doesn't need to be set until after
ptlrpcd_check() has completed.

wait_woken() was introduced in Linux 3.19, so libcfs is enhance to
provide the functionality on older kernels.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Iaddf56e2e76c204435bbef3c857e54ce0a6772bc
Reviewed-on: https://review.whamcloud.com/45069
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 lod: return pools_hash_params to being static.

A recent patch changes pools_hash_params in lod_pool.c to no longer
be 'static'.  This is not ideal.

rhashtable interfaces are mostly 'static inlines' which contain a lot
of code which is mostly optimised away providing that the 'params'
structure is const and locally visible.  When these interfaces are
called with a params structure in another file, the code produces is
quite inefficient and wasteful.

It is generally cleaner to provide accessor functions which can be
exported to other compilation units.  It is even beneficial to do that
within the one file.

This patch introduces
   lod_pool_exists()
and
   lod_pool_find()

The first is 'extern' and thus 'pools_hash_params' can not be static.
The second is used in several places in lod_pool.c, improving code
quality and maintainability.

Fixes: 0a998f4723f5 ("LU-14825 lod: pool spilling")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ieafe2f23fe5cc71d9bdce73cbe7360f5cb540edf
Reviewed-on: https://review.whamcloud.com/45070
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14734 ldiskfs: improve message for large_dir

Make it more clear that the large_dir feature has already been
enabled, rather than making the admin think that they need to
enable the feature themselves.

Test-Parameters: trivial
Fixes: f5967b06aac5 ("LU-14734 osd-ldiskfs: enable large_dir automatically")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ica59d3370148ed277d3541c05be065c4638daf8d
Reviewed-on: https://review.whamcloud.com/45046
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12567 ptlrpc: handle reply and resend reorder

ptlrpc can't detect a bulk transfer timeout
if rpc and bulk are reordered on router.
We should fail a bulk for situations where bulk is not
completed (after bulk timeout LNET_EVENT_UNLINK is set).

HPE-bug-id: LUS-7445, LUS-7569
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: Iaf099d31f8fbc68c3edbfcff77ae424862e0adc1
Reviewed-on: https://review.whamcloud.com/35571
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14711 osc: Do not attempt sending empty pages

Do not crash if trying to send a lock-prolonging emtpy read
to an old server, if the server does not support short reads.
Otherwise the client crashes when access the NULL page.

Test-Parameters: trivial
Fixes: 564070343ac4 ("LU-14711 osc: Notify server if cache discard takes a long time")
Change-Id: Icae7bf3ef16c45d33894b3c5fbac15b1a98c39d9
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/44654
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>

New tag 2.14.55

Change-Id: I4dcb816ff9e6b27ebc6c870fce187f8122ba2bde
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-14629 sec: do not block rename of topmost encrypted dir

We intentionally forbid file and directory rename from encrypted to
unencrypted directory. But we must not block rename of the topmost
encrypted directory.

Fixes: 1158386ac9 ("LU-14629 sec: forbid file rename from enc to unencrypted dir")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I480a24b2b0327e1d9104f216da54720e4f351636
Reviewed-on: https://review.whamcloud.com/45054
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14989 sec: access to enc file's xattrs

Encryption context is stored in 'security.c' xattr. This is put in the
xattr cache via ll_xattr_cache_insert() to avoid sending a getxattr
request to the server. But this operation declares the xattr cache for
the inode as 'valid', with two consequences. It prevents any further
filling with other xattrs, and trying to read an xattr value will
directly return -ENODATA, without any attempt to fetch the xattr from
the server.
This is solved by adding a new ll_file_flags 'LLIF_XATTR_CACHE_FILLED'
that tells if the xattr cache for the inode has been filled. This bit
is set only by ll_xattr_cache_refill(), and 'valid' now just means the
xattr cache for the inode has been initialized.

Fixes: 40d91eafe2 ("LU-12275 sec: atomicity of encryption context getting/setting")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6c2b6870df29f26f048dedeb7212d1c801ca69e1
Reviewed-on: https://review.whamcloud.com/44855
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10848 test: wait to process inodes in phase2

In test_8 of sanity-lfsck, the "LF_INCONSISTENT" flag was set
when processing the inodes with corrupted LinkEA in phase2,
LFSCK could have no chance to process it yet because of the delay
OBD_FAIL_LFSCK_DELAY3.

Test-Parameters: trivial testlist=sanity-lfsck,sanity-lfsck,sanity-lfsck,sanity-lfsck,sanity-lfsck,sanity-lfsck,sanity-lfsck,sanity-lfsck,sanity-lfsck
Change-Id: Id414728c998d527fbc27f877c6d31dcedcc12457
Signed-off-by: Hongchao Zhagn <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/44658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14777 lod: fix E2BIG on create

A fix solves 2 cases that caused create to fail
with -E2BIG.
1. Stripe count number should be calculated depending
on LOV_PATTERN_OVERSTRIPING flag.
2. In a case of failover lod_comp_entry_stripe_count
may return 0 if all OST targets have been disconnected.
Return EAGAIN in such case to calculate this later,
when at least one OST would be connected.

HPE-bug-id: LUS-9485
Fixes: aa72de32 ("LU-11691 lov: Limit layout size to max ea size")
Change-Id: I26cad4903d5dd6197fe1384013fbba8b2c76487c
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/44043
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14543 target: prevent overflowing of tgd->tgd_tot_granted

If tgd->tgd_tot_granted < ted->ted_grant then there should not be:
   tgd->tgd_tot_granted -= ted->ted_grant;
which breaks tgd->tgd_tot_granted.
In case of obvious ted->ted_grant damage, recalculate
tgd->tgd_tot_granted using list of exports.

The same change is made for tgd->tgd_tot_dirty.

This patch also adds sanity check for exp->exp_target_data.ted_grant
increase in tgt_grant_alloc() to catch grant counting corruption as
soon as it happened. By default, the detected corruption is
CERROR()-ed, if needed that can be switched to LBUG() using lctl
set_param *.*.lbug_on_grant_miscount.
test-framework.sh:init_param_vars() enables LBUG().

Fixes: af2d3ac30e ("LU-11939 tgt: Do not assert during grant cleanup")
Change-Id: I36ba7496f7b72b4881e98c06ec254a8eefd4c13f
Signed-off-by: Vladimir Saveliev <vlaidimir.saveliev@hpe.com>
Cray-bug-id: LUS-9875
Reviewed-on: https://review.whamcloud.com/42129
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15027 sec: initialize ll_inode_info for fake inode

When creating an encrypted symlink, we need to make use of a fake
inode in order to be able to encrypt the target name before sending
the create request to the MDS.
This fake inode needs minimal initialization, but it is at least
required to properly initialize the ll_inode_info associated with this
fake inode.

Fixes: e735298935 ("LU-13717 sec: filename encryption - symlink support")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I20c30d873f9bffdbdc8b5f272cb8b80e5be7fbfb
Reviewed-on: https://review.whamcloud.com/45023
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15007 tests: quota enable cmd fix

the tunable should be osd-*.*.quota_slave.enabled
not osd-*.*.quota_slave.enable.

Fixes: b9c359a70d ("LU-7004 tests: move from lctl conf_param to lctl set_param -P")

Test-Parameters: trivial testlist=ost-pools,sanity-quota
HPE-bug-id: LUS-10413
Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: I321436520ab48eff4bcc93611a0ada68fa33205e
Reviewed-on: https://review.whamcloud.com/44919
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15008 kernel: kernel update RHEL8.4 [4.18.0-305.19.1.el8_4]

Update RHEL8.4 kernel to 4.18.0-305.19.1.el8_4.

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.4 serverdistro=el8.4 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.4 serverdistro=el8.4 testlist=sanity

Change-Id: Icedc6cf2a5678cfbce76c47507137c0ea41d0b06
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/44950
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14033 ldiskfs: Fix mballoc prefetch patch

ext4-mballoc-prefetch patch was inadvertently broken during patch
rebasing.

In ext4_read_block_bitmap():
    ext4_read_block_bitmap_nowait() should not ignore locked

Test-Parameters: trivial
HPE-bug-id: LUS-9805
Fixes: fc87b01f96e8 ("LU-12477 ldiskfs: remove obsolete ext4 patches")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I6ebe9dfe48f48706da3623e3d32d33dddf35b395
Reviewed-on: https://review.whamcloud.com/44894
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-14996 lov: prefer mirrors on non-rotational OSTs

consider non-rotational OSTs as preferred unless explicit prefer
flag is set on a mirror.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I787bcba0b5e45842c9d4762c7f97a8f44a4fc9cb
Reviewed-on: https://review.whamcloud.com/44883
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14994 kernel: kernel update RHEL7.9 [3.10.0-1160.42.2.el7]

Update RHEL7.9 kernel to 3.10.0-1160.42.2.el7.

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I377ea5d1e28c50b1087dfca7cb32f44afb9bf5f5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/44875
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>