LU-12845 utils: fix typos in 'lctl pcc' help

The help message for 'lctl pcc ...' has typos that
should be corrected.

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I880a126d38a7cf9b2fc65d1a05a5d4eb554be1e5
Reviewed-on: https://review.whamcloud.com/36416
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10496 tests: enable 39k for DoM

Test was disabled temporary, enable it back as all needed
fixes are landed.

This reverts commit 68dd8a8acff9ad2295a1fcba318fc8ed5f140026.

Test-Parameters: trivial testlist=sanity-dom,sanity-dom,sanity-dom
Test-Parameters: fstype=zfs testlist=sanity-dom,sanity-dom,sanity-dom
Test-Parameters: fstype=zfs testlist=sanity-dom,sanity-dom,sanity-dom
Test-Parameters: fstype=zfs testlist=sanity-dom,sanity-dom,sanity-dom
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ib230d2849450bb4642ae8a286a84e501e0dafde1
Reviewed-on: https://review.whamcloud.com/36352
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12760 tests: stack_trap defaults to sigspec=EXIT

If the "sigspec" argument is not specified for stack_trap(), default
to "EXIT" as the signal, since this is what we use for all callers
of stack_trap() today anyway.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2c8d986cdf8743e1d956cd7941a47bd4cd772592
Reviewed-on: https://review.whamcloud.com/36186
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>

LU-12712 lod: fix warning message for non-SEL file

The warning message printed when the LCME_FL_EXTENSION flag is set on
a non-SEL file was incorrectly checking the component magic (usually
LOV_MAGIC_V3) instead of the file magic (should be LOV_MAGIC_SEL).
Don't redefine "magic" for each component, since this is only used in
one other place to mean the component magic, and use the actual file
or component magic in the few places where this is checked.

Fix the warning message to be rate-limited on the console by using
CWARN(...) instead of CDEBUG(D_WARNING, ...), though it is somewhat
questionable whether this should be a console message at all as there
is nothing that the administrator can do about this problem and it
doesn't appear to have any side-effects.  Also print the component
ID/index and restructure the message text so that it is more clear
about what the actual problem is.

Fix a few minor style issues in nearby code.

Test-Parameters: trivial testlist=sanity-pfl
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I81c3f9914512b1959b8483bb2b988ea4597cab07
Reviewed-on: https://review.whamcloud.com/36351
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12800 mdd: do not cache atributes on mdd_parent_fid

mdd_is_parent() brings link xattrs of not locked objects into osp
cache.

Invalidate osp cache built during mdd_is_subdir().

Change-Id: Id9e34af3ff4712af9d4f3ae984e8082448e5fd3f
Cray-bug-id: LUS-7634
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-on: https://review.whamcloud.com/36287
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12769 recovery: use monotonic timer

instead of real one. also use absolute values for timer.

One of the reasons for the move from jiffies based timer
to a hrtimer timer was to avoid the issue of time drift.
It was discovered due to test failures with recovery on
VMs that the high resolution wall clock can drift as well.
Moving to the monotonic clock for the hrtimer avoids this
drift completely and it is safe to use since the recovery
timestamp is not shared between nodes.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I8b75121934c229dec8df7be0a4e69c1cda940d3f
Reviewed-on: https://review.whamcloud.com/36274
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12691 ldlm: obd_max_recoverable_clients is not atomic

Originally obd_max_recoverable_clients is not increased
at the same moment. But because of LU-3540,
it will be increased by multiple processes.

The type of obd_max_recoverable_clients should be
atomic_t and be handled by atomic operations.

Signed-off-by: Tatsushi Takamura <takamr.tatsushi@jp.fujitsu.com>
Change-Id: I9a67bbbfacab2e05858243f649e3a4e0d4b5d7f7
Reviewed-on: https://review.whamcloud.com/35914
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12824 o2ib: Reintroduce kiblnd_dev_search

If we add an interface to multiple nets then we need to re-use the
struct ib_dev object for each of the nets.

Cray-bug-id: LUS-7935
Fixes: 75ab841 ("LU-11893 lnet: consoldate secondary IP address handling")
Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I1790e24458f47d632fd137b78de076d408fe5260
Reviewed-on: https://review.whamcloud.com/36326
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12824 o2ib: Record rc in debug log on startup failure

Since kiblnd_startup() return -ENETDOWN on failure, let's record the
rc value for the failure case in the debug log.

Cray-bug-id: LUS-7935
Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ied934642bc567b8d3f51293d7dd095d47ff134df
Reviewed-on: https://review.whamcloud.com/36325
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12802 tests: speedup cleanup of racer

After racer test survives for a given time, it starts to cleanup.
And the parent racer.sh script waits the child racer/racer.sh
to exit. However sometimes, somehow, this stucks for a long time.
Sending a signal to remaining dd(or other) processes will wake up
the wait in parent racer.sh script immediately.

Lustre-change: https://review.whamcloud.com/35101

Test-Parameter: trivial testlist=racer
DDN-Bug-ID: DDN-256
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I2ff2784b76faa0532c39af29b1586a48f2b90a21
Reviewed-on: https://review.whamcloud.com/36289
Reviewed-by: Shilong Wang <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12777 test: fix to pass facet to facet_fstype

Function facet_fstype() expect mgs1 mds1 etc as its
argument, and we used it wrong to pass $mds1 which will
cause following error.

line 1192: lustre-ost1/ost1_FSTYPE: bad substitution

And we fail to detect this is ZFS based OSD, and pool
reimporting will be missed thus failed to mount.

Test-Parameters: trivial clientdistro=el8 testlist=conf-sanity \
                 fstype=zfs envdefinitions=ONLY=103
Test-Parameters: trivial clientdistro=el8 testlist=conf-sanity \
                 fstype=ldiskfs envdefinitions=ONLY=103
Change-Id: Id8fd5b9f17e666614e83e5c1a2399fde8b91b023
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36298
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12795 tests: Prefer FAILF in mpi tests

Prefer FAILF() to the sprintf()/FAIL() pattern and remove
the errmsg temporary buffer.

Cleanup use of assignment in conditional statements.

Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Icb04b188209ad3717260aa1238de1d9788d76f79
Reviewed-on: https://review.whamcloud.com/36276
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12790 obdclass: print jobid error message properly

Modify unlikely() condition to print error message properly when
(rc == -EOVERFLOW).

Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I19bfb353c71b55a0dfb6eec78c1af915494acd71
Reviewed-on: https://review.whamcloud.com/36272
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12795 tests: Introduce FAILF and start using it

Unify the sprintf(msg, ...); FAIL(msg); pattern into a single
FAILF() macro and avoid the needless use of a temporary buffer.

Remove the duplicate lp_utils.c and lp_utils.h files.

Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Ia0ed463e305ec671909a049d847eb8518291f72f
Reviewed-on: https://review.whamcloud.com/36255
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12229 tests: fix "bad substitution" error

In newer bash version, the special characters is invalid in the
usage of indirect variable expansion {!word}. For example,
# a=lustre,pool
# echo ${!a}
-bash: lustre,pool: bad substitution
To avoid "bad sustitution" error, pool_new command is used in
test_1j and test_1k directly.

Test-Parameters:trivial clientdistro=el8 testlist=ost-pools
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ifce4616cd7f314416fe5fa09f8fba846ae45bcef
Reviewed-on: https://review.whamcloud.com/36243
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12764 lnet: eliminate uninitialized warning

lustre-release/lnet/lnet/router.c: In function ‘lnet_del_route’:
include/linux/compiler.h:177:26: error: ‘lp’ may be used uninitialized
in this function [-Werror=maybe-uninitialized]
  case 8: *(__u64 *)res = *(volatile __u64 *)p; break;  \
                          ^
/home/wangsl/lustre-release/lnet/lnet/router.c:754:20: note: ‘lp’ was declared here
  struct lnet_peer *lp;
                    ^
/home/wangsl/lustre-release/lnet/lnet/router.c: At top level:
cc1: error: unrecognized command line option ‘-Wno-stringop-overflow’ [-Werror]
cc1: error: unrecognized command line option ‘-Wno-stringop-truncation’ [-Werror]
cc1: error: unrecognized command line option ‘-Wno-format-truncation’ [-Werror]
cc1: all warnings being treated as errors

codes logic gurantee @lpi and @lpni are inited at the same time,
but let's init @lpi to make gcc happy.

Change-Id: I1ccd88ca5061b5f29a530bf2b755585c92612a69
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36189
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11467 utils: add lfs mirror delete command

Add "lfs mirror delete" as an alias for "lfs mirror split -d", to
balance "lfs mirror create" and simplify the interface for users.

Add lfs-mirror-create.1 man page, and convert some tests in
sanity-flr.sh to use the new interface.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I4399878dc2fd435c517a2ff529b91480583ebbe5
Reviewed-on: https://review.whamcloud.com/36185
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12704 tests: component end must be multiple of stripesize

In racer file_create.sh, it was generating a random multiple of 64KB
in [64K..1024K] for the stripe_size, but using a fixed 1MB component
end.  However, the component end must be a multiple of stripe_size,
or an error will be returned by "lfs setstripe":

  Invalid layout: component end must be aligned by the stripe size

Generate the component_end argument as a random multiple [1..8] of
the stripe size for PFL, DoM, and FLR files.

Fixes: 0b66b11523cb ("LU-3285 tests: add dom into racer test suite")
Test-Parameters: trivial testlist=racer
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib79131fc71d0eaa4a398cd8c8adf6e53473ebbe5
Reviewed-on: https://review.whamcloud.com/36174
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12724 kernel: kernel update RHEL7.7 [3.10.0-1062.1.1.el7]

Update RHEL7.7 kernel to 3.10.0-1062.1.1.el7.

Test-Parameters: trivial clientdistro=el7.7 serverdistro=el7.7

Change-Id: Iad40fb93b8a15d875b72749a05666a23e4755fcc
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36074
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11607 tests: replace lustre_version/fstype in posix/perf

The routine get_lustre_env() is available to all Lustre test
suites and sets environment variables for the Lustre
version and file system type for servers.

In posix, performance-sanity, and parallel-scale, replace
calls to lustre_version_code() and facet_fstype() for all
server types with definitions from get_lustre_env().

While doing this, replace "$SINGLEMDS" with "$MDS1_VERSION"
or "$mds1_FSTYPE" in lustre_version_code() and facet_fstype().

Clean up around any modifications by converting spaces to
tabs.

Test-Parameters: trivial testlist=posix,performance-sanity,parallel-scale
Test-Parameters: fstype=zfs testlist=posix,performance-sanity,parallel-scale
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Id7d3c2a50c7b880c1147e9b6c721fddff07861fa
Reviewed-on: https://review.whamcloud.com/35934
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>

LU-4315 doc: add separate lctl-list_param man page

Split the lctl list_param man page from the main lctl.8 man page.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib3f125e953427ec6ace1709588f5d40b1e3ebbe5
Reviewed-on: https://review.whamcloud.com/35649
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12400 zfs: zfs mainline 0.8+ with mainline (5.2) kernel

Compile tests need to resolve using Module.symvers from the zfs
kmod kernel version.

dsl_pool_config_enter signature changed

Use spa_get_hostid() if it is available (zfs 0.7.0 and later).
For zfs 0.6.x use /proc/sys/kernel/spl/hostid with a fallback
to reading the spl module parameters and manually decoding
the hostid file.

list_move_tail() in libzfs conflicts with normal kernel/lustre
list API. Move it out of the way before libcfs/util/list.h is
included.

Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Ia16e226239d33555ba7d906b39e37e20f012a02c
Reviewed-on: https://review.whamcloud.com/35518
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4322 tests: re-enable 101a in dne config

We should re-enable 101a in the dne config, and also make
it more strict on discards.  This test should normally
result in 0 discards, because every page brought in by
readahead is used.

It is possible for randomness with the reads to lead to a
few discards, but no more than that.

Test-Parameters: trivial testlist=sanity,sanity,sanity,sanity
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I53c6263c33c27d36b746c8fc56c8deebb4b713c7
Reviewed-on: https://review.whamcloud.com/35027
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 obd_type: discard obd_types linked list.

As all obd_types are kobjects in the lustre_kset kset,
they are linked together in that kset and don't
need any extra linkage.

There are non-obd_type objects in lustre_kset, added by
class_setup_tunables().  These have a different ->ktype, so we are
careful to only return objects with the correct ->ktype.

As kset_find_obj() returns a counted reference, we need
to put that reference when done.

On the server side it is possible to have an obd_type partially
initialized by one subsystem and latter fully initialized by
another subsystem. We use typ_sym_filter to notify us if the
obd_type is only partially setup. If it only paritially setup
then we let the original subsystem that created the obd_type
to clean it up. If the obd_type was latter completely setup
then we let the latter subsystem do the cleanup for us.

Linux-commit: 881bc9b58ef5e8c9be297b121187ea6c26572cf1

Change-Id: I4316644f7fb12e358b799af64deb57836e796066
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/34718
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12127 lfsck: skip orphan processing for namespace LFSCK

LFSCK can reconnect a recently-deleted orphan object back
into the normal namespace when it shouldn't. This can cause
access to the deleted data (potential security risk), and
sometimes cause an assertion if orphan is later deleted.

The commit 077570483e75e0610fd45149b926097547c434b8 skips
the orphan object during LFSCK scan. But what needs to be
skipped is only the namespace LFSCK logic for the orphan
object, but the layout LFSCK processing is still necessary
for the orphan object to verify the consistency between the
orphan MDT-object and related OST-object(s). This patch
just does that.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I9b2809b95efa4b3c3e3b2c7d0a501624ed3ebbe5
Reviewed-on: https://review.whamcloud.com/34174
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10496 ofd: serialize fmd check and set

Serialize FMD check and set in OFD to prevent update
with old data.
Update also sanity tests 39j,k to cancel LRU locks on
both mdc and osc namespaces because for DOM files both
MDT and OST stripes are being used

Test-Parameters: testlist=sanity-dom
Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com>
Change-Id: I02e28e1e3e8e533d9c7450d798fceb9261b27ea0
Reviewed-on: https://review.whamcloud.com/36228
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for munlink.c

This patch fixes issues reported by checkpatch
for file lustre/tests/munlink.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I1722de906c529045fc9a2c8edc877d695840677d
Reviewed-on: https://review.whamcloud.com/35912
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for openfile.c

This patch fixes issues reported by checkpatch
for file lustre/tests/openfile.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ied280e919397d27b087ec9d5ac3c7e55d44ec0b2
Reviewed-on: https://review.whamcloud.com/35911
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for opendevunlink.c

This patch fixes issues reported by checkpatch
for file lustre/tests/opendevunlink.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I2f488b77610429e7c17ac3975a66aab271b27409
Reviewed-on: https://review.whamcloud.com/35910
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for opendirunlink.c

This patch fixes issues reported by checkpatch
for file lustre/tests/opendirunlink.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I245b37c93e53313c96a5f69d11d41bc93b3844e2
Reviewed-on: https://review.whamcloud.com/35909
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for utime.c

This patch fixes issues reported by checkpatch
for file lustre/tests/utime.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Iae92c34b2f6d68204bb3f9846227eb507c04e27d
Reviewed-on: https://review.whamcloud.com/35908
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for openfilleddirunlink.c

This patch fixes issues reported by checkpatch
for file lustre/tests/openfilleddirunlink.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ic6771392bf8fdb477ce7d5632c09780667a8abe0
Reviewed-on: https://review.whamcloud.com/35907
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for writemany.c

This patch fixes issues reported by checkpatch
for file lustre/tests/writemany.c

This patch also removes locally defined difftime
macro with C library defined difftime function

Test-Parameters: trivial testlist=recovery-small
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I10458cd1467639ba613b303f2fbe8cd24efc9942
Reviewed-on: https://review.whamcloud.com/35906
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for multifstat.c

This patch fixes issues reported by checkpatch
for file lustre/tests/multifstat.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ief81a0fb80c5863ee92b5e6e602924e8ebdd0031
Reviewed-on: https://review.whamcloud.com/35905
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for mkdirmany.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mkdirmany.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I79a86ddf661c3149b1de67a29b329809dd87fbec
Reviewed-on: https://review.whamcloud.com/35903
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for mmap_sanity.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mmap_sanity.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I37a842ef9a193b554c26deb916b9fe9187b79042
Reviewed-on: https://review.whamcloud.com/35902
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for ll_sparseness_write.c

This patch fixes issues reported by checkpatch
for file lustre/tests/ll_sparseness_write.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Icf1b9488f75791cd859e975c21341e67b160992c
Reviewed-on: https://review.whamcloud.com/35821
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for ll_sparseness_verify.c

This patch fixes issues reported by checkpatch
for file lustre/tests/ll_sparseness_verify.c

Change-Id: I7049be84c43169a2b21d0d2cff980f48f6fd27d0
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/35820
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for ll_dirstripe_verify.c

This patch fixes issues reported by checkpatch
for file lustre/tests/ll_dirstripe_verify.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I552d27b9517ef36b6ad66db6222bf55239c16c04
Reviewed-on: https://review.whamcloud.com/35819
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Remove file iam_ut.c

This patch removes file lustre/tests/iam_ut.c
This file currently is not used at all by any
tests or binary.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ia27a94a4cc7ed1a5ff788b07ce1f2487f9427c83
Reviewed-on: https://review.whamcloud.com/35818
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Remove file flock.c

This patch removes file lustre/tests/flock.c
This file currently is not used at all by any
tests or binary.

For flock tests validation file
lustre/tests/flocks_test.c is used instead.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ie9b55c7a93c77234ef21a5bc7827fbaf9e9b20cd
Reviewed-on: https://review.whamcloud.com/35817
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 tests: Fix style issues for directio.c

This patch fixes issues reported by checkpatch
for file lustre/tests/directio.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ic1b39dd84aaef75c4bd94c357b6f917b735175cb
Reviewed-on: https://review.whamcloud.com/35816
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for chownmany.c

This patch fixes issues reported by checkpatch
for file lustre/tests/chownmany.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I57a89193c683c12881215394fa417ab0aefa004f
Reviewed-on: https://review.whamcloud.com/35815
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for checkstat.c

This patch fixes issues reported by checkpatch
for file lustre/tests/checkstat.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I2ed9c9b4325f9285643a21114dbaf2e1e48c1757
Reviewed-on: https://review.whamcloud.com/35814
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for mcreate.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mcreate.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I51651c52e81ca64302a02cedcc97faa11fff656e
Reviewed-on: https://review.whamcloud.com/35813
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12410 tests: Add auster option to skip setup

Add an option to auster to skip the initial setup of Lustre.

Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ie3de93c8a4d3f812aaf1f032e39c351827c6eaef
Reviewed-on: https://review.whamcloud.com/35389
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12824 o2ib: Fix whitespace in kiblnd_startup

Convert whitespace to tabs where appropriate in kiblnd_startup()

Cray-bug-id: LUS-7935
Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I11aaaa8e47d754b219fb773d74e37190476e4eeb
Reviewed-on: https://review.whamcloud.com/36324
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11670 tests: do not fail the first half in sanityn test 103

There are two halves in sanityn test 103. The first half is to
reproduce the problem of incorrect size when using lockahead
and the second half is to verify that the fix works. Sometimes,
the problem cannot be reproduced in the first half test, so we
should not fail the whole test.

Test-Parameters: trivial fstype=zfs \
mdscount=2 mdtcount=4 testlist=sanityn,sanityn,sanityn

Test-Parameters: trivial fstype=ldiskfs \
mdscount=2 mdtcount=4 testlist=sanityn,sanityn,sanityn

Change-Id: Ib6c82bfe512ac072104abfcb406e2ef1bd6a6a02
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36303
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>

LU-11967 mdt: reint layout_change in standard way

Layout_change is a reint operation, and it should be handled the
same as other reint operations, so that resent and replay can
work correctly.

Also replace the lock passed in ldlm_handle_enqueue0() with the
lock taken in mdt_layout_change(). This avoids taking lock again
in ldlm_handle_enqueue0(), and also makes replay eaiser. Note,
before replacing, the mode is downgraded from EX to CR, because
client only needs this mode, as can avoid unnecessary lock cancel
later.

Add missing resent reconstructor for REINT_RESYNC.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I328044dacbf18d03232c9bbb51271f6202e9b939
Reviewed-on: https://review.whamcloud.com/35465
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11739 lod: subdir under ROOT should honor default layout

Though sub directories under ROOT don't inherit ROOT default
layout, they should hornor ROOT default layout in creation.

Add sub test in sanity.sh 65n to verify this.

Fixes: 0a988cae95 ("LU-11739 lod: don't inherit default
                    layout from root directory")

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I1edf91da7944a8871652df7bca2104d00f66163a
Reviewed-on: https://review.whamcloud.com/35204
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12825 build: change lbuild to support MOFED 4.7

* Remove 'alternate' name in MOFED tar
* use MLNX_LIBS to download rpms

Test-Parameters: trivial

Change-Id: Ia5a4f51455be836a7df4fa6b3e9eccc17cffef2c
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36333
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11444 ptlrpc: resend may corrupt the data

Late resend if arrives much later than another modification RPC
which has been already handled on this slot, may be still applied
and therefore overrides the last one

Send RPCs from client in increasing order for each tag
and check it on server to check late resend.

A slot can be reused by a client after kill while
the server continue to rely on it.

Add flag for such obsolete requests, here we trust the
client and perform xid check for all in progress requests.

Cray-bug-id: LUS-6272, LUS-7277, LUS-7339
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Change-Id: I97806577cf979f49a75379ffc55947cc3dcd02b1
Reviewed-on: https://review.whamcloud.com/35114
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9920 vvp: dirty pages with pagevec

When doing i/o from multiple writers to a single file, the
per-file page cache lock (the mapping lock) becomes a
bottleneck.

Most current uses are single page at a time.  This converts
one prominent use, marking page as dirty, to use a pagevec.

When many threads are writing to one file, this improves
write performance by around 25%.

This requires implementing our own version of the
set_page_dirty-->__set_page_dirty_nobuffers functions.

This was modeled on upstream tip of tree:
v5.2-rc4-224-ge01e060fe0 (7/13/2019)

The relevant code is unchanged since Linux 4.17, and has
changed only minimally since before Linux 2.6.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ifff9cd01f8b4e960bb4ebea560b9a9a01376698d
Reviewed-on: https://review.whamcloud.com/28711
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 target: remove lwi arg from target_bulk_io

The callers of target_bulk_io() pass in an lwi pointer but never put
any information into it or take any information out of it.  Also
target_bulk_io() always re-initializes the struct before using it, so
it doesn't communicate info from one call to the next.

All that this achieves it to make stack usage slightly less
in the few cases where the lwi pointer is tti_wait_info in
struct tgt_thread_info.  That is not worth it, and a future
patch will remove the use of the struct completely.

So make lwi local to target_bulk_io, and remove it from
all callers.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: Ib6039006d0168393abf3995877acde2d7c796b1f
Reviewed-on: https://review.whamcloud.com/35969
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 lustre: add wait_event macros suitable for upstream

This patch adds three sorts of wait_event macros.

1/ wait_event_idle_* which are available upstream, but not
   in older kernels.
   if TASK_NOLOAD is not available, we use TASK_UNINTERRUPTIBLE,
   and block all interrupts.

   We cannot use ___wait_cond_timeout() as it changed signature
   in 3.13. so we define our own ___wait_cond_timeout1().

2/ wait_event_idle_exclusive_lifo() and
   wait_event_idle_exclusive_lifo_timeout()
    which might be accepted upstream if we can make a strong case

   prepare_to_wait_event() doesn't support this directly, but
   as it won't relink a wait_entry that is already linked, it
   is sufficient to link to the head of the queue before calling
   prepare_to_wait_event().

3/ l_wait_event_abortable
   l_wait_event_abortable_timeout
   l_wait_event_abortable_exclusive
    which are unlikely to be accepted upstream, but match the general
    approach of upstream wait_event macros, and are useful
    to lustre.
    Possibly some or all of these should become wait_event_killable_*
    LUSTRE_FATAL_SIGS is moved over to linux-wait.h.

___wait_event() and related macros are copied from upstream linux,
and modified slightly to work across all supported kernels.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: I2d260fc159dbe5b1a3cc7a26e4aeedf30150d85a
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/35962
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12681 osc: wrong cache of LVB attrs, part2

It may happen that osc oinfo lvb cache has size < kms.

It occurs if a reply re-ordering happens and an older size is applied
to oinfo unconditionally.

Another possibility is RA, when osc_match_base() attaches the dlm lock
to osc object but does not cache the lvb. The next layout change will
overwrites the lock lvb by the oinfo cache (previous LUS-7731 fix),
presumably smaller values. Therefore, the next lock re-use may run
into a problem with partial page write which thinks the preliminary
read is not needed.

Do not let the cached oinfo lvb size to become less than kms.
Also, cache the lock's lvb in the oinfo on osc_match_base().

Signed-off-by: Vitaly Fertman <c17818@cray.com>
Cray-bug-id: LUS-7731
Change-Id: I50136f57491364146ce7b6a81b814e474e3edb86
Reviewed-on: https://review.whamcloud.com/36200
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12681 osc: wrong cache of LVB attrs

osc object keeps the cache of LVB, obtained on lock enqueue, in
lov_oinfo. This cache gets all the modifications happenning on
the client, whereas the original LVB in locks does not get them.
At the same time, this cache is lost on object destroy, which
may appear on layout change in particular.

ldlm locks are left in LRU and could be matched on next operations.
First enqueue does not match a lock in LRU due to @kms_ignore in
enqueue_base, however if the lock will be obtained on a small offset
with some locks existent in LRU on larger offsets, the obtained size
will be cut by the policy region when set to KMS.

2nd enqueue can already match and add stale data to oinfo. Thus the
OSC cache is left with a small KMS. However the logic of preparing
a partial page code checks the KMS to decide if to read a page and
as it is small,the page is not read and therefore the non-read part
of the page is zeroed.

The object destroy detaches dlm locks from osc object, offload the
current osc oinfo cache to all the locks, so that it could be
reconstructed for the next osc oinfo. Introduce per-lock flag to
control the cached attribute status and drop re-enqueue after osc
object replacement.

This patch also fixes the handling of KMS_IGNORE added in LU-11964. It
is used only for skip the self lock in a search there is no other logic
for it and it is not needed for DOM locks at all - all the relevant
semantics is supposed to be accomplished by cbpending flag.

Signed-off-by: Vitaly Fertman <c17818@cray.com>
Cray-bug-id: LUS-7731
Change-Id: Iba45bb3e5ee181c82c2f22deb299228b1519cddb
Reviewed-on: https://review.whamcloud.com/36199
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12495 obdclass: qos penalties miscalculated

In lqos_calc_penalties(), the penalty_per_obj is miscalculated.

Also improve sanity test_413b: take both blocks and inodes into
account to make the test more robost.

Fixes: d3090bb ("LU-11213 lod: share object alloc QoS code with LMV")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ie965fc3bfa3e303c27f93a6e1a428cc4a90f8548
Reviewed-on: https://review.whamcloud.com/36269
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12758 quota: clear default flag for new ID

When setting the quota limits as 0 by "lfs setquota", the default
flag won't be cleared if the lquota_entry is just created for some
quota ID at the first time because the quota limits are the same.

Change-Id: I7f44ce0cb13783ca5bede2f55cd0707f1ccbc8ca
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36236
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shilong Wang <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11743 utils: allow lctl pool_list on separate MGS

Change lctl pool_list command to parse the configuration log directly
when run on a standalone MGS node.  This also allows the pool commands
to be run when only the MGS is started.

Also, those test scripts from the patch of LU-9899 to mount a client
on the standalone MGS to allow OST pools to work properly are cleared.

Change-Id: Ic25931d49c2cf747da2a3f2ac3c25a21f6878991
Test-Parameters: standalonemgs=true testlist=ost-pools.sh,sanity.sh,conf-sanity.sh
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-on: https://review.whamcloud.com/35895
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12755 ldiskfs: fix project quota unpon unpatched kernel

The value of MAXQUOTAS is the number of quota types supported
by kernel. With project quotas patch applied, MAXQUOTAS is
equal to EXT4_MAXQUOTAS. However, on an unpatched kernel,
project quota type is not supported and MAXQUOTAS is one less
than EXT4_MAXQUOTAS.

In ldiskfs, we need to make sure that the loop in
ext4_quota_off_umount() is limiting the EXT4_MAXQUOTAS loop
to the kernel MAXQUOTAS value. Otherwise, it is trying to
dereference sb_dqopt(sb)->files[2] which is not an inode at all,
and cause the kernel stick on a spinlock in ext4_quota_off()
as follows during unmount:

Call Trace:
[<ffffffffb9d733c5>] queued_spin_lock_slowpath+0xb/0xf
[<ffffffffb9d81b30>] _raw_spin_lock+0x20/0x30
[<ffffffffb9865e2e>] igrab+0x1e/0x60
[<ffffffffc08a8c4b>] ldiskfs_quota_off+0x3b/0x130 [ldiskfs]
[<ffffffffc08abcdd>] ldiskfs_put_super+0x4d/0x400 [ldiskfs]
[<ffffffffb984b13d>] generic_shutdown_super+0x6d/0x100
[<ffffffffb984b5b7>] kill_block_super+0x27/0x70
[<ffffffffb984b91e>] deactivate_locked_super+0x4e/0x70
[<ffffffffb984c0a6>] deactivate_super+0x46/0x60
[<ffffffffb986abff>] cleanup_mnt+0x3f/0x80
[<ffffffffb986ac92>] __cleanup_mnt+0x12/0x20
[<ffffffffb96c1c0b>] task_work_run+0xbb/0xe0
[<ffffffffb962cc65>] do_notify_resume+0xa5/0xc0
[<ffffffffb9d8d23b>] int_signal+0x12/0x17

Test-Parameters: clientdistro=el7.7 serverdistro=el7.7

Change-Id: I18a4d97656e2f8478754943424c0fac927f843ca
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36203
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12789 o2ib: fix configure checks

Fix configure checks for modern kernels / MOFED 4.7
1) sg_dma_address() and sg_dma_len() always have only one argument.
2) Make configure checks executed in proper enviroment

Change-Id: I9910de888371776758376743ab4418778e1d85e4
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/36245
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12763 lnet: Use alternate ping processing for non-mr peers

Router peers without multi-rail capabilities (i.e. older Lustre
versions) or router peers that have discovery disabled need to use
the alternate ping processing introduced by LU-12422. Otherwise,
these peers go through the normal discovery processing, but their
remote network interfaces are never added to the peer object. This
causes routes through these peers to be considered down when
avoid_asym_router_failure is enabled.

Cray-bug-id: LUS-7866
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ib567b66c871abdad9b39b4f29b38eca424d4cd8d
Reviewed-on: https://review.whamcloud.com/36182
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12739 lnet: Don't queue msg when discovery has completed

In lnet_initiate_peer_discovery(), it is possible for the peer object
to change after the call to lnet_discover_peer_locked(), and it is
also possible for the peer to complete discovery between the first
call to lnet_peer_is_uptodate() and our placing the lnet_msg onto
the peer's lp_dc_pendq. After the call to lnet_discover_peer_locked()
check whether the, potentially new, peer object is up to date while
holding the lp_lock. If the peer is up to date, then we needn't
queue the message. Otherwise, we continue to hold the lock to place
the message on the peer's lp_dc_pendq.

Cray-bug-id: LUS-7596
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ib3da7447588479bb35afcc3fe176b9120d915a89
Reviewed-on: https://review.whamcloud.com/36139
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12705 utils: cleanup unnecessary typecasting

There're a bunch of variables typeecasted in utils/lfs.c where
they are not needed, so cleanup them here.

Change-Id: I6c944f18137fd1ff1162d9b6567c9328dfa185eb
Test-Parameters: trivial
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/36224
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11426 llog: changelog records reordering

Changelog records can get reordered because of a race
window between cr_index generation and llog file
space allocation. This can lead to llog records
loss.

llog_write() holds loghandle->lgh_lock semaphore,
so it seems an appropriate place to generate a
new changelog index.

Change-Id: I034d1a696bde1d0f780e494ab65073e4018ceec9
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Cray-bug-id: LUS-7691
Reviewed-on: https://review.whamcloud.com/36187
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9859 libcfs: move misc-device registration closer to related code.

The ioctl handler for the misc device is in  lnet/libcfs/module.c
but is it registered in lnet/libcfs/linux/linux-module.c.

Keeping related code together make maintenance easier, so move the
code.

Linux-commit: b4ded66db93bbe1f5323ad38ce51bb1be114934f

Test-Parameters: trivial

Change-Id: Ia2b3590a769214fe964dab7a63fd5edcfd6c5042
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/36118
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12690 llite: error handling of ll_och_fill()

The return error of ll_och_fill() should be handled.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I4e750001cb124104836fa24e39ec8ae203b51a83
Reviewed-on: https://review.whamcloud.com/35913
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11380 llapi: separate FID man pages

Add separate man pages for the lfs commands and llapi functions.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I05abfaf888a5474d62feebab4e8db543ba3ebbe5
Reviewed-on: https://review.whamcloud.com/35673
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11933 mdt: clear sp_cr_flags in migrate unpack

mdt_thread_info.mti_spec is not cleared after operation handling, so
mdt_migrate_unpack() should clear it in case the old values are used.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib3d5d39a4a072621c8da8b6ef7869cb4d8178aac
Reviewed-on: https://review.whamcloud.com/36154
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for multiop.c

This patch fixes issues reported by checkpatch
for file lustre/tests/multiop.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I08c038f034e26eafacc99a3f3d7966704a79709c
Reviewed-on: https://review.whamcloud.com/35904
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 ptlrpc: Fix style issues for llog_net.c

This patch fixes issues reported by checkpatch
for file lustre/ptlrpc/llog_net.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ieee60565e5332a1546999e6c8edaffc833074271
Reviewed-on: https://review.whamcloud.com/35823
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12570 mdt: request env for DT threads

as part of lock enqueue MDT thread can call ldlm_reclaim_full() to
cancel old unused LDLM locks and that scans all presented namespace
including OFD-originated (with extent locks). thus MDT ends with
calls into OFD code which needs own env marked with LCT_DT_THREAD.

Test-Parameters: testlist=sanity,sanity,sanity,sanity envdefinitions=ONLY="134a",SHARED_KEY=true
Test-Parameters: testlist=sanity,sanity,sanity,sanity envdefinitions=ONLY="134a",SHARED_KEY=true
Test-Parameters: testlist=sanity,sanity,sanity,sanity envdefinitions=ONLY="134a",SHARED_KEY=true
Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com>
Change-Id: I231b88159978bc3ce7a3fa0f27e57eb32137c343
Reviewed-on: https://review.whamcloud.com/36179
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12103 ldiskfs: don't search large block range if disk full

Block allocator tries to find:
1) group with the same range as required
2) group with the same average range as required
3) group with required amount of space
4) any group

For quite full disk step 1 is failed with higth
probability, but takes a lot of time.

Skip 1st step if disk space < 25%
Skip 2d step if disk space < 15%
Skip 3d step if disk space < 5%
Also check if group has any free space on step 4.

This three thresholds can be adjusted through added interface.

Variables added which counts unsuccessfull group processing loops.
This can show allocator effectiveness in different circumstances.

This statistics output through mb_alloc file. This file is
useful to track allocator activity.

Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: I18c7147e32951c49e12a2444803aa2995bb4ae2d
Cray-bug-id: LUS-6746
Reviewed-on: https://review.whamcloud.com/35180
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12776 doc: update Changelog to reflect primary kernels

This patch updates the lustre/Changelog file to distinguish
between primary kernels that are built and tested during the
current release cycle from other kernels that are also known
to have built at some point.

Test-Parameters: trivial
Signed-off-by: Joseph Gmitter <jgmitter@whamcloud.com>
Change-Id: I7b6b5ce2dc813da91e2d9cd7a499168443e41f59
Reviewed-on: https://review.whamcloud.com/36214
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12612 osd: add lnb size down to osd

so that each OSD can check for lnb array overflow.
the patch isn't final - there will be proper
implementation in osd-zfs and a new test.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I43683c84e48006b4075f9a8b3e87cdfeae28c02b
Reviewed-on: https://review.whamcloud.com/35801
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11367 som: integrate LSOM with lfs find

The patch integrates LSOM functionality with lfs find so that it
is possible to use LSOM functionality directly on the client. The
MDS fills in the mbo_size and mbo_blocks fields from the LSOM
xattr, if the actual size/blocks are not available, and then set
new OBD_MD_FLLSIZE and OBD_MD_FLLBLOCKS flags in the reply so that
the client knows these fields are valid.

The lfs find command adds "-l|--lazy" option to allow the use of
LSOM data from the MDS.

Add a new version of ioctl(LL_IOC_MDC_GETINFO) call that also returns
valid flags from the MDS RPC to userspace in struct lov_user_mds_data
so that it is possible to determine whether the size and blocks are
returned by the call.  The old LL_IOC_MDC_GETINFO ioctl number is
renamed to LL_IOC_MDC_GETINFO_OLD and is binary compatible, but
newly-compiled applications will use the new struct lov_user_mds_data.

New llapi interfaces llapi_get_lum_file(), llapi_get_lum_dir(),
llapi_get_lum_file_fd(), llapi_get_lum_dir_fd() are added to fetch
valid stat() attributes and LOV info to the user.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I21dfae7c2633dead5d83b438ec340fea4d3ebbe5
Reviewed-on: https://review.whamcloud.com/35167
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>

LU-12621 o2iblnd: cache max_qp_wr

When creating the device the maximum number of work requests per qp
which can be allocated is already known. Cache that internally,
and when creating the qp make sure the qp's max_send_wr does not
exceed that max. If it does then cap max_send_wr to max_qp_wr.
Recalculate the connection's queue depth based on the max_qp_wr.

Test-Parameter: nettype=o2ib
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I6d9a642d03633264f5f14445a051dd14515709c1
Reviewed-on: https://review.whamcloud.com/36073
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12751 tests: add missing error()

nothing else I can say

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I040771e57ec6f6c6bfbde5a21358c6747f4f20dc
Reviewed-on: https://review.whamcloud.com/36159
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>

LU-12753 tests: wait for mds2 recovery in sanity 278

At the end of sanity.sh test_278() the mds2 facet is remounted.
However, test_300a() which runs immediately afterward expects
all MDTs to be available, and occasionally fails creating a
fully-striped directory with:

    stripe_count is 1, expect 2

because the mds2 facet has not finished mounting at mkdir time.

Add a wait at the end of test_278() to ensure that mds2 has finished
mounting so that it doesn't affect any later tests.

Test-Parameters: trivial envdefinitions=ONLY=270-300 mdscount=1 mdtcount=2
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9421101be6394288ccd4ce5777d118523a3ebbe5
Reviewed-on: https://review.whamcloud.com/36167
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12734 misc: add bash completion for lctl set/get_param

Add some start of bash completion for lctl, mainly set_param and
get_param, and modify build system to install it.

Test-Parameters: trivial
Signed-off-by: Dominique Martinet <dominique.martinet@cea.fr>
Change-Id: I16d2698e782702375c7fa3edf3bfde2e3b197297
Reviewed-on: https://review.whamcloud.com/36105
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12707 obdecho: avoid panic with partially object init

in some cases (like ENOMEM) init function can't called, so
any init related code should placed in the object delete handler,
not in the object free.

Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: I1fca56423de9a045aac2c495fbc45069c3bbc97c
Reviewed-on: https://review.whamcloud.com/35950
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12569 o2iblnd: Make credits hiw connection aware

The IBLND_CREDITS_HIGHWATER mark check currently looks only
at the global peer credits tunable, ignoring the connection
specific queue depth when determining the threshold at
which to send a NOOP message to return credits.

This is incorrect because while connection queue depth
defaults to the same as peer credits, it can be less than
that global value for specific connections.

So we must check for this case when setting the threshold.

Test-Parameter: nettype=o2ib
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie028ae11cdbd0f75a38b265b7ab5830f92f08d90
Reviewed-on: https://review.whamcloud.com/35578
Reviewed-by: Chris Horn <hornc@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12514 utils: add "lustre_tgt" filesystem type

Add a "lustre_tgt" filesystem type separate from the "lustre"
filesystem type for forward compatibility to allow the client and
server mount code to be disentangled.  This does not actually make
any changes to the mount code itself, but allows testing interop
and fallback after an upgrade if the new filesystem type is used.

Remove the check for the "lustre_lite" filesystem type from
llapi_is_lustre_mnttype() since that was unused since Lustre 1.4,
and replace it with a check for filesystem type "lustre_tgt".

Test-Parameters: trivial testlist=conf-sanity
Change-Id: If635834dfff502e6ecf8a85a2665de09efd52372
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36165
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11956 mdd: do not reset original lu_buf.lb_len

In mdd_iterate_xattrs(), we are resetting the xbuf.lb_len
to a smaller value returned by linkea_overflow_shrink().

If that's the last xattr we gonna process, we could deduct
less than originally allocated size from obd_memory stats,
failing the memleak check later.

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I6175a91c61ceb0e37ab889d0cfd904f4993ab5cc
Reviewed-on: https://review.whamcloud.com/35333
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12745 build: Account for optional SPL for ZFS 0.8+

With ZFS 0.8.0 and later, SPL is not longer present.
Some zfs packages provide vestigial spl package contents, but zfs-dkms
does not.  This makes testing SPL directories optional depending on
version of ZFS, this also accounts for the new location of the spl
include directory under the zfs include directory.

Test-Parameters: trivial
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I8afcff079f25543a3c86df0c404146a859b226aa
Reviewed-on: https://review.whamcloud.com/36161
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11485 lod: disallow setting the last non-stale mirror as stale

"lfs setstripe" allows setting stale flag on the last
non-stale mirror of a file, which makes the file have
no valid component to read and return IO error.

This patch fixes the above issue by disallowing that.
It also disallows "lfs mirror split" to destroy the
last non-stale mirror of a file.

Change-Id: I6934cfe0190cd1ea83de1cf28ddf840b9f96193a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36141
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 lustre: introduce CONFIG_LUSTRE_FS_POSIX_ACL

Lustre (or any file system) should not conditionally
compile code based on CONFIG_FS_POSIX_ACL.  This config
option enables library support.
A file system can define it's own config option,
which then selects CONFIG_FS_POSIX_ACL (if needed).  It should
act on its own option, not the library one.

This patch makes that change.

While lustre is out-of-tree, it cannot select CONFIG_FS_POSIX_ACL
and must work with the configruation of the kernel it is being built
against. So the new CONFIG_LUSTRE_FS_POSIX_ACL can only be selected
if the based kernel has CONFIG_FS_POSIX_ACL selected. With that
restriction this change becomes little more than an indirection.
However it means that the out-of-tree code can be closer to the
(eventual) in-tree code.

Test-Parameters: trivial

Signed-off-by: NeilBrown <neilb@suse.com>
Change-Id: I3ba283f295d1c7217e7ff9917573be0fe92bb2ca
Reviewed-on: https://review.whamcloud.com/36085
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12385 lnet: update opa defaults

Testing reveals no significant performance improvements
when using peer_credits > 32. Adjusted the default
peer_credits, peer_credits_hiw and concurrent_sends
to take that into account.

This has the advantage of avoiding an issue observed
on multiple opa sites where the qp can not be created because
of large initial queue_depth. The queue depth is then
reduced gradually until the qp creation succeeds.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I6036ec1da7063e30b567446e5db89040f21bc701
Reviewed-on: https://review.whamcloud.com/36072
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11542 import: Fix missing spin_unlock()

A recent patch moved the spin_unlock() down into
each branch of an 'if', but missed the final 'else'.
Add the spin_unlock in the else.

Fixes: 29904135df67 ("LU-11542 import: fix race between imp_state & imp_invalid")
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: I6ee399050aad0fe9df9c0e3ddf8ec0be8eae1641
Reviewed-on: https://review.whamcloud.com/35999
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 ptlrpc: fix indents in ptlrpc_recover_import()

Next patch will make code changes in ptlrpc_recover_import(),
so fix up indenting to use TABs first.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: I1d839e3087a5d12473b0428509455dbbb4b5d9e8
Reviewed-on: https://review.whamcloud.com/35976
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 lustre: use TABs for indents in a few places.

Each of the functions changed here will have code changes
in the next patch, so fix up all the indentation first.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: Ib10e999a8c58eb96d3312878be91b465da3a2df8
Reviewed-on: https://review.whamcloud.com/35970
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 fid: style cleanups in seq_client_alloc_meta()

Prior to code changes in seq_client_alloc_meta, make indents
consistently tabs, and change "env != NULL" to "env"

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: If72d737f843a5f6e56ef82aabb7a779b358f4382
Reviewed-on: https://review.whamcloud.com/35963
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9341 lod: Add special O_APPEND striping

Files opened with O_APPEND are almost always log files,
which generally stay small and do not benefit from being
striped widely.  Additionally, PFL files accessed with
O_APPEND are fully instantiated, meaning that because the
files usually stay small, these objects usually wasted.

This patch adds special striping for files created with
O_APPEND.  This is controlled on the MDS by two new proc
variables:
mdd_append_stripe_count
mdd_append_pool

If the stripe count is set to 0 and the pool is not set,
this functionality is disabled and files created with
O_APPEND will be striped like any other file.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I433d1b8c80488a851b8eb26c78cf5519a6cd75bf
Reviewed-on: https://review.whamcloud.com/35617
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11213 lod: share object alloc QoS code with LMV

Move object alloc QoS code to obdclass, so that LMV and LOD
can share the same code.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I451a43fa9a254ec709b2acd43538fdcba0be4a88
Reviewed-on: https://review.whamcloud.com/35219
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11213 lmv: use lu_tgt_descs to manage tgts

Like LOD, use lu_tgt_descs to manage tgts, so that they can
share tgt management code.

TODO: use the same tgt management code for LOV/LFSCK.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I843e85e0f8b05e21056567cebfea0a8fff1d4ef8
Reviewed-on: https://review.whamcloud.com/35218
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11670 osc: glimpse - search for active lock

When there are lock-ahead write locks on a file, the server
sends one glimpse AST RPC to each client having such (it
may have many) locks. This callback is sent to the lock
having the highest offset.

Client's glimpse callback goes up to the clio layers and
gets the global (not lock-specific) view of size.  The clio
layers are connected to the extent lock through the
l_ast_data (which points to the OSC object).

Speculative locks (AGL, lockahead) do not have l_ast_data
initialised until an IO happens under the lock. Thus, some
speculative locks may not have l_ast_data initialized.

It is possible for the client to do a write using one lock
(changing file size), but for the glimpse AST to be sent to
another lock without l_ast_data initialized.  Currently, a
lock with no l_ast_data set returns ELDLM_NO_LOCK_DATA to
the server.  In this case, this means we do not return the
updated size.

The solution is to search the granted lock tree for any lock
with initialized l_ast_data (it points to the OSC object
which is the same for all the extent locks) and to reach the
clio layers for the size through this lock instead.

cray-bug-id: LUS-6747
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6c60f4133154a3d6652315f155af24bbc5752dd2
Reviewed-on: https://review.whamcloud.com/33660
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12634 lnet: for_ifa removed. Use in_dev_for_each_ifa_rtnl

Linux 5.3 removed for_ifa and replaced it with an _rntl and _rcu
versions for use with their respective locking primitives.

kernel-commit: ef11db3310e272d3d8dbe8739e0770820dd20e52

Test-Parameters: trivial
Cray-bug-id: LUS-7689
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Iea07222b9abb3f9c219d28fe2c660d9eaf21af80
Reviewed-on: https://review.whamcloud.com/35744
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12559 ptlrpc: Hold imp lock for idle reconnect

Idle reconnect sets import state to IMP_NEW, then releases
the import lock before calling ptlrpc_connect_import.  This
creates a gap where an import in IMP_NEW state is exposed,
which can cause new requests to fail with EIO.

Hold the lock across the call so as not to expose imports
in this state.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I9f8509d11c4d5a8917a313349534d98b964cd588
Reviewed-on: https://review.whamcloud.com/35530
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12269 kernel: RHEL 8.0 server support

This patch makes changes to support RHEL 8.0 release
for Lustre server.

The ldiskfs patch series was added by commit
5f7bd9a806d5ba5b0e70ae1b299a9b9a87fcc0a6.

Test-Parameters: trivial \
envdefinitions=SANITY_EXCEPT="421a 817" \
clientdistro=el8 serverdistro=el8 \
testlist=sanity

Change-Id: I6380c6cb6280e49bd1b193a639f1c5cd6f4e7ef1
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35665
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>