LU-10754 tests: Clear mdc locks before tests

On ZFS testing, a sync stemming from a lock cancellation
from a previous test sometimes causes us to run longer than
the sleep times allowed for forked processes to be ready.

So, cancel the MDC lru locks first.  This will only incur a
sync if there is data to sync, but will wait for one if
necessary.

Test-Parameters: testlist=sanityn,sanityn,sanityn,sanityn

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I865de238aadd6da719066e6f22e2a36d1d3f368e
Reviewed-on: https://review.whamcloud.com/34848
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12242 kernel: kernel update RHEL7.6 [3.10.0-957.12.1.el7]

Update RHEL7.6 kernel to 3.10.0-957.12.1.el7.

Test-Parameters: clientdistro=el7.6 serverdistro=el7.6

Change-Id: I71d3bc18dbc16ed1ad7a3083dc19f52b56f60e40
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34784
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12248 lov: fix ost objects calculation in lod_statfs

Wen OSTs report fewer free objects than MDTs, the statfs
objects results are presented with the numbers reported
by OSTs. Fix the calculation of OST objetcs to make it
work with statfs aggregation via the MDT.

Make the lfs code consistent with ll_statfs_internal()
and lod_statfs().

Fixes: a829595add ("LU-11721 lod: limit statfs ffree if less ...")

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I838a1527ed6411a412b63e2855ca7247755a3bcf
Reviewed-on: https://review.whamcloud.com/34777
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>

LU-12225 obdclass: fix race access vs removal of jobid_hash

We added @pidmap into hash and reference count will be 1.
However, another thread might reclaim this newely added
@pidmap from hash list, we try to access this @pidmap
will become a user-after-free operation.

Fix this problem by init reference count as 1 before
adding hash list, which gurantee memory could be not
freed during our access.

Check other places where memory reclaim used did similar
idea like this.

Change-Id: Idd5f429b97e064e29b6883243f8a012c2b4b4ae7
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34763
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 lnet: getname dropping addrlen argument

Since kernel 4.17 ->getname() does not take int *addrlen
argument anymore, instead it's returning the length to
the caller.

Linux-commit: 9b2c45d479d0fb8647c9e83359df69162b5fbe5f

Test-Parameters:trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I4ad5de4a22f3fb23c07a356650ea7925acf07eed
Reviewed-on: https://review.whamcloud.com/34672
Tested-by: Jenkins
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12139 kernel: kernel update [SLES12 SP3 4.4.176-94.88]

Update SLES12 SP3 kernel to 4.4.176-94.88.

Test-Parameters: trivial clientdistro=sles12sp3 serverdistro=sles12sp3

Change-Id: Iecf77e056fc571eb5118ac8c96d440e5f3ceebc0
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34670
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11157 obd: round values to nearest MiB for *_mb syfs files

Several sysfs files report their settings with the functions
lprocfs_[seq]_read_frac_helper() which has the intent of showing
fractional values i.e 1.5 MiB. This approach has caused problems
with shells which don't handle fractional representation and the
values reported don't faithfully represent the original value the
configurator passed into the sysfs file. To resolve this lets
instead always round up the value the configurator passed into
the sysfs file to the nearest MiB value. This way it is always
guaranteed the values reported are always exactly some MiB value.

Change-Id: Ia2e8cf8421784853aa33d4bb87c54aee00953835
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34317
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2233 tests: improve tests sanityn/40-47

sanity/40-46 usually take 800-900s which is almost a half
of the whole sanityn pass. 99.(9)% of time the tests just
wait to ensure specific order the operations execute in.

the patch changes cfs_fail_timeout_set() so that it can
interrupt waiting if fail_loc is set to 0 - polling with
1/10s frequency is used.

the tests itself are modified to reset fail_loc. to be
able to do so both operations (referenced as OP1 and OP2
in the tests) are run in background. once started and then
ensured with pdo_sched() helper that MDS threads got to the
blocking points, we can interrupt OP1 and do usual checks.

ONLY=40-47 sh sanityn.sh take: 1017s before and 78s after.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie01bd6a077333f6f57e533a73f38588a073a2381
Reviewed-on: https://review.whamcloud.com/4392
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>

LU-12276 lnet: check const parameters for ib_post_send and ib_post_recv

In MOFED 4.6, the second and third parameters for ib_post_send() and
ib_post_recv() are declared with 'const'. This patch adds the check in
configure file to resolve build failure.

Change-Id: If7193a6a4fcb7b238f5d4ee64e878a5816433e7b
Test-Parameters: trivial
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34837
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12266 mdd: fix up non-dir creation in SGID dirs

sgid directories have special semantics, making newly created files in
the directory belong to the group of the directory, and newly created
subdirectories will also become sgid. This is historically used for
group-shared directories.

But group directories writable by non-group members should not imply
that such non-group members can magically join the group, so make sure
to clear the sgid bit on non-directories for non-members (but remember
that sgid without group execute means "mandatory locking", just to
confuse things even more).

Adapt fix from inode_init_owner() to use in mdd_create_sanity_check().

Linux-commit: 0fa3ecd87848c9c93c2c828ef4c3a8ca36ce46c7

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iae253c5cc7865fc81574760ce0ed4d93698b7314
Reviewed-on: https://review.whamcloud.com/34809
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12227 scripts: check for mounted ZFS devices too

lustre init script skips several checks if the device type is ZFS. If
some ZFS devices are already mounted, the script will return a
non-zero exit code.

The label and mount point check is valid for ZFS devices, so let's do
it and avoid this error case. With this patch, when starting ZFS
devices the script will only start the not already started ones and if
it succeeds, return 0.

Test-Parameters: trivial
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Change-Id: I152ca4d62d444193cc66896173873587f0761493
Reviewed-on: https://review.whamcloud.com/34766
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10602 utils: fix file heat support

Change the LL_IOC_HEAT_SET ioctl number assignment to reduce the
number of different values used, since we are running out.  Use
a __u64 as the IOC struct argument instead of a "long" since that
is what is actually passed, and it avoids being CPU-dependent.

Move the LU_HEAT_FLAG_* values into an enum to avoid a generic
"flags" argument in the code.  This makes it clear what is passed.

Clean up code style for lfs_heat_get() and lfs_heat_set().

Fixes: ae723cf8161f ("LU-10602 llite: add file heat support")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If06212d2d62d085a2104cf54ae9a10e512eb2efd
Reviewed-on: https://review.whamcloud.com/34757
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 obd: embed typ_kobj in obd_type

As there is a 1-1 mapping between obd_types and their ->typ_kobj, it
is simple and more normal to embed the kobj in the obd_type, rather
than allocate it separately.

This requires calling "kobject_init_and_add()" earlier, so we
open-code relevant part of class_setup_tunables() in
class_register_type(). Now class_setup_tunables() is needed only
for server side code.

With typ_kobj embedded in obd_type we change class_setup_tunables()
to return an obd_type object instead of a kobject. This way we
can use kobject_put() to cleanup the obd_type created with
class_setup_tunables(). The reason for class_setup_tunables() is
for the creation of a lightweight obd_type which is never added
to the typ_chain list to avoid potential duplicates which can
happen on single node setups with lod / lov and osp /osc.

Change-Id: Iac160e6817a7c520e4462a3fc133ddfee6a7ccdc
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34612
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11690 lod: fix LBUG with wide striping

When striping extremely widely (~1600+ stripes), we reach
more than half of the theoretical limit of layout size,
and LBUG.

It is also possible to trigger this assert with
multi-component PFL files, where all the components are
below the stripe count limit, but together they exceed it.

PFL makes asserting based on LOV_MAX_STRIPE_COUNT
unworkable, so just remove the assert.  Further work is
planned to match up maximum allowed layout size with
the real maximum EA size.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id0240785792e7d4084ea6e53b44469a40e59043d
Reviewed-on: https://review.whamcloud.com/33708
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 ptlrpc: Fix style issues for service.c

This patch fixes issues reported by checkpatch
for file lustre/ptlrpc/service.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ibaffcdfaeac48176ba05b5e4f4471f9db96d9cbe
Reviewed-on: https://review.whamcloud.com/34605
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 ptlrpc: Fix style issues for sec_null.c

This patch fixes issues reported by checkpatch
for file lustre/ptlrpc/sec_null.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I67631d35ae4461ca92516975ab71f69d01378e19
Reviewed-on: https://review.whamcloud.com/34549
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6142 ldlm: Fix style issues for interval_tree.c

This patch fixes issues reported by checkpatch
for file lustre/ldlm/interval_tree.c

Test-Parameters: trivial
Change-Id: Ida99aa8f7a5928e87611c73aa7b5d0dc4a5246e9
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/34498
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11803 obd: replace class_uuid with linux kernel version.

We can replace the lustre custom class_uuid_t with the linux
kernels uuid handling.

Change-Id: I9a59b0b6027ccb95994a87f3a5dcdf80a8a56480
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33916
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11376 lmv: new foreign LMV format

This patch introduces a new striping/LMV format in order to
allow to specify an arbitrary external reference for a dir
in Lustre namespace.
The new LMV format is made of {newmagic, length, type, flags,
string[length]} to be as flexible as possible.
Foreign dir can be created by using the ioctl(LL_IOC_LMV_SETDIRSTRIPE)
operation and it can only be and remain an empty dir until removed.
A new API method llapi_dir_create_foreign() has been introduced
and "lfs {get,set}dirstripe" and "lfs find" modified to understand
new format.
The idea behind this is to provide Lustre namespace support and
striping prefetch/caching under lock protection, for user/external
usage.

This patch is the LMV/dirs complement of LOV/files previous change
(Change-Id: I5d9c0642fe8e7009c30918bfa946cac7c00c9af8) and has
been rebased on top of the latter along with some with obvious
mutualizations and simplifications.

Code has been added for lfsck to handle foreign dirs, and
a new sub-test has been added in sanity-lfsck in order to verify
if does not break foreign dir and that reverse is also true.

Also fixes a bug causing SEGVs during
"lfs find [--mdt-count=[+,-]<count>, --mdt-hash=<hashtype>]" when
handling a file (ie, "DIR *dir" is NULLL) in cb_find_init().

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I3721b8f14578bf926a92da76375dae92dc8d764d
Reviewed-on: https://review.whamcloud.com/34087
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11376 lov: new foreign LOV format

This patch introduces a new layout/LOV format in order to
allow to specify an arbitrary external reference for a file
in Lustre namespace.
The new LOV format is made of {newmagic, length, type, flags,
string[length]} to be as flexible as possible.
Foreign file can be created by using the open(O_LOV_DELAY_CREATE) +
ioctl(LL_IOC_LOV_SETSTRIPE) operations and it can only be and remain
an empty file until removed.
A new API method llapi_file_create_foreign() has been introduced
and "lfs [[get,set]stripe,find" modified to understand new layout.
The idea behind this is to provide Lustre namespace support and
layout prefetch/caching under layout protection, for user/external
usage.

Code has been added for lfsck to handle foreign files, and
a new sub-test has been added in sanity-lfsck in order to verify
if does not break foreign file and that reverse is also true.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I5d9c0642fe8e7009c30918bfa946cac7c00c9af8
Reviewed-on: https://review.whamcloud.com/33755
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>

LU-11403 tests: Fix $tfile usage

We cannot just use raw $tfile - we must use something under
$DIR.  This is resulting in failures because $tfile exists.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iea6356cabb1623606bf926ce80c55a3210c0b535
Reviewed-on: https://review.whamcloud.com/34698
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11233 utils: fix double-free of params fields

Call find_param_fini() on error so that the params are not leaked
during initialization if there is an intermediate error.

Zero out the parameters as they are freed, so if find_param_fini()
is called multiple times (as it is in some error paths) it does
not corrupt the heap by double freeing pointers.  This can be hit
by calling "lfs getstripe -m" on multiple pathnames, some of which
do not exist.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie0d7e9ee134deb0633af2f8052b8a458333ebbe5
Reviewed-on: https://review.whamcloud.com/34711
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6951 tests: sanity test_27m failure

sanity 27m fails with "OST0 was full but new created file
still use it" if the test runs with more than 1 client.
The issue can be easily reproduced with qos_threshold_rr=100.

The reason is grants. Every client initially gets 2 Mb grant.
When dd from the first client receives ENOSPC, it does not mean
the OST is filled up, since the client is not allowed to use
other clients' grants. When creating a new file, the MDS still
sees free space on OST0 equal to the amount of unused grants
and allocates new objects on OST0.

This situation does not seem to reflect any defect in Lustre.
Rather, the original author's intent seems to be that
the test should always run with a single client. So, this patch
simply disables the test if the test is running with more than
one client.

Change-Id: I47cd1a6806e8fa5203aeb5bcf57a6b31b424f24d
Seagate-bug-id: MRP-1690
Signed-off-by: Alexander Boyko <c17825@cray.com>
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/23506
Tested-by: Jenkins
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11090 quota: Oops in qsd_config

It's quota config vs umount race
Remove qsd from the list of fsinfo before
freeing per-quota type data.

Change-Id: Ib7c3a94b3222ffd229da1a384113b3befc19665b
Cray-bug-id: LUS-5896
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-on: https://review.whamcloud.com/32715
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11251 mdt: ASSERTION (req_transno < next_transno) failed

An update request is checked for duplicates by xid in
is_req_replayed_by_update(). However xid is unique per
client only. It may happen that there are 2 requests
with the same xid from different clients.

Perform lookup by transno, it is unique per MDT.

Change-Id: If00b69f01451c659292c004aa296a6ea36680d3c
Cray-bug-id: LUS-6015
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://review.whamcloud.com/33001
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9010 ptlrpc: Change static defines to use macro for gss_krb5_mech.c

This patch replaces spinlock which are defined statically
in file lustre/ptlrpc/gss/gss_krb5_mech.c with kernel provided macro.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I5da319ce013c29043fc4bde4a4946cfbdf6c2491
Reviewed-on: https://review.whamcloud.com/33936
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12043 llite, readahead: fix to call ll_ras_enter() properly

ll_ras_enter() is expected to be called per syscall.
However, with fast read enabled, it will be no longer true that
We will call vvp_io_read_start() for every syscall.

To fix this problem, we should move this to file read handler.

Change-Id: I8d70714b2e8bc04b7c4ab996d189f10f37488d97
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34755
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12159 utils: improve lfs getname functionality

Add "-n" and "-i" options to lfs getname to allow printing only
the fsname or instance ID of the filesystem(s).

Split out the documentation to a separate lfs-getname.1 man page.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie132513325b6630fc5103a89b469271ba7392cb2
Reviewed-on: https://review.whamcloud.com/34595
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9846 obd: Add overstriping CONNECT flag

This patch reserves the OBD_CONNECT flag for overstriping,
and also does some cleanup of OBD_CONNECT flags, putting
them in the correct order and adding some missing ones in
proc and the wire{test,check} checks.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I5d7c8f30d16cc2541d3202582fe55177022ccede
Reviewed-on: https://review.whamcloud.com/34743
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>

LU-12218 ptlrpc: Bulk assertion fails on -ENOMEM

Recalculate rq_mbits on ENOMEM resend if OBD_CONNECT_BULK_MBITS
isn't used.

Change-Id: I3bd5f7536372558a264bf5fe3247b8b1946f84fd
Cray-bug-id: LUS-7159
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/34753
Tested-by: Jenkins
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11233 build: support for gcc8

this patch covers kernel portion of Lustre

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I3fac8b89eef2291b5cb91ea05ee0b6ff32d11741
Reviewed-on: https://review.whamcloud.com/34660
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-12131 tests: only create lgssc.conf file if necessary

lgssc.conf file is now packaged by Lustre, and installed under
/etc/request-key.d/.
So, unless run from build tree, init_gss() must not create its own
anymore. So adjust corresponding commands in init_gss() and
cleanup_sk().

Fixes: e299df1e9eea ("LU-7854 gss: install lgssc.conf under /etc/request-key.d")
Whamcloud-bug-id: ATM-1283
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9cc76fddb8a622d7c40d6348913df42ae063254a
Reviewed-on: https://review.whamcloud.com/34520
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12263 build: push depreciation of LMV_HASH_FLAG_DEAD to 2.12.55

This is to quickly restore buildability for now.
The actual proper removal TBD.

Change-Id: Ib28d658d614716307c984dcf77b2451138bc0e1b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34793

New tag 2.12.53

Change-Id: I913a2b175ba5ea02c8489a9baa64e2932a8bdbe8
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-12261 tests: Race between exec and truncate

Execing '$tdir/sleep' with & doesn't guarantee the file is
actually open before returning, so it is sometimes losing
the race with truncate, resulting in errors like this:
/usr/lib64/lustre/tests/sanity.sh: line 4172:
/mnt/lustre/d43b.sanity/sleep: Text file busy

Where $tdir/sleep gets ETXTBSY, instead of truncate as
expected.

A 1 second delay should be enough to guarantee exec wins
the race vs truncate.

Test-Parameters: trivial
Test-Parameters: testgroup=review-ldiskfs-arm
Test-Parameters: testgroup=review-ldiskfs
Test-Parameters: testgroup=review-ldiskfs-arm

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ice6f4567805e64c3be755b6c684b6a086a348dd8
Reviewed-on: https://review.whamcloud.com/34791
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

Revert "LU-11367 som: integrate LSOM with lfs find"

This is causing LU-12253

This reverts commit 5b6569affc9a0e33fa5d7d2061834397da13e0cb.

Change-Id: I5a70d4cec5bb81f8067e847cec99c77bc8f94093
Reviewed-on: https://review.whamcloud.com/34780
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>

LU-12138 kernel: kernel update SLES12 SP4 [4.12.14-95.13.1]

Update SLES12 SP4 kernel to 4.12.14-95.13.1 for Lustre client.

Test-Parameters: trivial clientdistro=sles12sp4 \
envdefinitions=LNET_SELFTEST_EXCEPT=smoke,SANITY_EXCEPT=103a

Change-Id: I0831e611caa1ad51775e5f73d7989212f1347c2e
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34619
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11367 som: integrate LSOM with lfs find

The patch integrates LSOM functionality with lfs find so that it
is possible to use LSOM functionality directly on the client. The
MDS fills in the mbo_size and mbo_blocks fields from the LSOM
xattr, if the actual size/blocks are not available, and then set
new OBD_MD_FLLSIZE and OBD_MD_FLLBLOCKS flags in the reply so that
the client knows these fields are valid.

The lfs find command adds "--lazy" option to allow the use of LSOM
data from the MDS.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I21dfae7c2633dead5d83b438ec340fea4d56c52b
Reviewed-on: https://review.whamcloud.com/33545
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12147 utils: statone doesn't place \0

as strncpy() is not supposed to do, the caller has to
take care of that.

Change-Id: I858a7f0eb6c7cdcb70e8a8e445c96f1187c73c2f
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34572
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8434 tests: add script language option to auster

auster is a scipt that kicks off the Lustre test suites.
auster assumes that all scirpts are written in bash and
runs all script using bash. We may want to run other scripts
to test Lustre and we need to allow the user to choose what
scripting language to use to kick off their scripts.

Test-Parameters: trivial

Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ifbd3707171de57912306cf051a98922249c4b2a9
Reviewed-on: https://review.whamcloud.com/34737
Tested-by: Jenkins
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12208 tests: fix interop with old clients

We are unable to run the tests on the setup with mixed
clients versions installed because the clients older than
2.8.54 do not have set_hostid() added by LU-7134.

The hostid is to be set on all sever nodes including the
facets falover nodes. This is not done currently, and tests
fail on the setup like the following: ost1_HOST=host1
ost1failover_HOST=host2.

Patch fixes formatall() to call set_hostid() on server nodes
only, including the failover nodes.

Test-Parameters: trivial
Fixes: 7b654fc673 ("LU-7134 utils: Ensure hostid set for ZFS during mkfs")
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-7242
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I56f36c4b4e6cdadcb9940abfc9d2da8183b52509
Reviewed-on: https://review.whamcloud.com/34725
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12203 tests: skip sanity-lfsck 36[b,c] for FILESET

When FILESET is set the tests accessing .lustre directly
are to be skipped.

Test-Parameters: trivial envdefinitions=FILESET=/subdir envdefinitions=ONLY="36" testlist=sanity-lfsck ostcount=4
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-7175
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I7aa1ca40efdeebfb3ee52edc8d8758bb6ce73ad4
Reviewed-on: https://review.whamcloud.com/34722
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12195 tests: use sleep instead of wrapped multiop

in sanity/43* and sanity/14* tests as multiop is not a binary,
but libtool-wrapped script. the tests fail when started from a
build tree.

Change-Id: Iaec3433f03aab23583052373e5f0252d9eac7f04
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34721
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12027 utils: fix "lfs find -amctime" comparison

For matches that are "equal" to the specified time, they must match
within the smallest unit specified.  For example, "-mtime 24h" would
match anything within 1h of 24h ago, similar to how "-size 100M" will
match anything within 1MB of 100MB.

Test-Parameters: trivial fstype=zfs
Fixes: 355c8a529a3a ("LU-12027 utils: add units to 'lfs find -amctime'")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib1eb4e626b712bb75f13b075849f959f203ebbe5
Reviewed-on: https://review.whamcloud.com/34658
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 obd: update udev event handling

Add a timestamp that users have requested so it can be recorded
then a sysfs lustre file changed. Second the PARAM field only
was created with the kobject source and parent name but the
sysfs file could be deeper in the lustre sysfs tree. Add handling
for deeper sysfs tree paths.

Change-Id: I46d4861addd16a6d5b6c28ad8bc2ce446e554d62
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34624
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12172 kernel: kernel update RHEL7.6 [3.10.0-957.10.1.el7]

Update RHEL7.6 kernel to 3.10.0-957.10.1.el7.

Test-Parameters: clientdistro=el7.6 serverdistro=el7.6

Change-Id: I5471cf0f88201fdeac960eeaf6579bc454591d6b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34618
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12169 llite: fill copied dentry name's ending char properly

Dentry name expect an extra '\0'. and dentry_len won't calcualte
extra '\0' for it, but we should allocate memory and fill it
when copying dentry name by ourselves.

Otherwise, lu_name_is_valid_2() will try to access @name[len]
and check whether it is '\0'. this is invalid memory access.
We will possibly hit a crash if the first access that bit is '\0'.
and the bit overwritten by someone else, and finally we failed
sanity check in mdc_name_pack().

LustreError: 157839:0:(mdc_lib.c:137:mdc_pack_name()) LBUG

Fixes: f575b65("LU-12020 llite: make sure name pack atomic")
Change-Id: I533e19a0e6efb0fca5a46bcdbdb0006d1b1bedab
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34611
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-930 doc: improve mkfs.lustre --fsname description

Specify what valid values for the --fsname option are and add some
additional description for this option and --index.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I256c97aaa51cec18cef44b3f34466c4fdb3ebbe5
Reviewed-on: https://review.whamcloud.com/34433
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12055 lod: ignore root layout if parent's one includes pool

When parent directory provides striping information which includes a
pool, don't inherit the stripe index from the root directory striping.
Otherwise, pool config and root layout mismatch leads to create
failure.

Test illustrating the issue is added.

Fix for sanity.sh:test_255c: the test assumes that test dir has
stripe count set to 1. Specify it explicitly to avoid using stripe
count of root directory which might be not 1.

Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Cray-bug-id: LUS-7093
Change-Id: Ib0f61edf4973bcb260dad617c27d1c1729223fa0
Reviewed-on: https://review.whamcloud.com/34386
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-11925 hsm: attributes aren't updated after RESTORE

MDS returns file size to a client with UPDATE lock
while file is RELEASED. It isn't cancelled after RESTORE
and the client has old file size after appending data.

Flush update lock after RESTORE completed.

Change-Id: Ib956dbd075691ce5fac1ce552df9519f9fa768e4
Cray-bug-id: LUS-6945
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-on: https://review.whamcloud.com/34180
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12043 llite: switch to use ll_fsname directly

There are many places which try to access filesystem
fsname, instead of parsing it everytime, just store
it into @sbi, we could use @ll_fsname directly whenever we need.

Change-Id: I25b8505b9aac5d4f6e97fe3691321be2b800f35d
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34602
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11756 o2iblnd: kib_conn leak

A new tx can be queued while kiblnd_finalise_conn()
aborts txs. Thus a reference from new tx will
prevent connection from moving into kib_connd_zombies.

Insert new tx after IBLND_CONN_DISCONNECTED into
ibc_zombie_txs list and abort it during
kiblnd_destroy_conn().

Change-Id: Ib92d8d02e6e3f66f7140041a330fc00b7ad44ae3
Cray-bug-id: LUS-6412
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-on: https://review.whamcloud.com/33828
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11673 tests: replace obsolete '-o' to '||'

Since use of -o and -a are marked as obsolete in shell
test ([), we need to switch from using [ expr1 –o expr2 ]
to [ expr1] || [ expr2 ].

Make this change for sanity tests.

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Id87580d0280a716a6939a1203ae5b370e762d6ec
Reviewed-on: https://review.whamcloud.com/33670
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11355 lustre: enable fstrim on lustre device

pass the FITRIM ioctl through the OST/MDT
mountpoint to the underlying filesystem, which
allows us to run fstrim on server mount point directly.

Change-Id: Ia6f9b43e48245ee7907a47f05c3924b3640bc734
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33131
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11796 lov: Remove unnecessary assert

This is asserting on network data from the server, and
additionally, the LU-9846 (overstriping) work shows this
condition is not a problem if it does somehow occur.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7b53eb63914f6e9d31a0747a40d09df9ffedaa91
Reviewed-on: https://review.whamcloud.com/33882
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>

LU-11691 lov: Limit layout size to max ea size

The layout code does not currently prevent the creation of
layouts which (once instantiated) will exceed the maximum
xattr size.

This patch modifies the code which calculates the maximum
allowed stripe count for a component to also evaluate the
full size of new layouts and report a count of zero if the
new layout is too large.  The server will then return
 -E2BIG to the client asking for such a layout.

Unfortunately, it's not practical to test this without
overstriping.  LU-9846 adds tests for this.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I2ead7702780b2600cf09485e06393ee9bcfb4a1e
Reviewed-on: https://review.whamcloud.com/34171
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11868 osd: Set max ea size to XATTR_SIZE_MAX

Lustre currently limits EA size to either ~1 MiB (ldiskfs)
or 32K (ZFS).  VFS has its own limit, XATTR_SIZE_MAX,
which we must respect to interoperate correctly with
userspace tools like tar, getattr, and the getxattr()
syscall.

Set this as the new max EA size for both ldiskfs and ZFS.

(The current 32K on ZFS is too small for
LOV_MAX_STRIPE_COUNT [2000] files, so needs to be raised
regardless.)

In order to use this correctly, we have to use the real ea
size on the client.  The previous code for maximum ea size
on the client (KEY_MAX_EASIZE, llite.max_easize) used a
calculated value based on number of targets.

With one exception, the mdc code already uses the default
ea size rather than the max.  Default ea size adjusts
automatically to the largest size sent by the server.

The exception is the open code, which uses the max so it
never has to resend a layout request.  This patch changes
it to use default, which means that the first time a very
widely striped file is opened, the open will be resent.

Add limit checks on client & server so the xattr size limit
is honored.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4da62691f30fa276d20959810116cf558cccc515
Reviewed-on: https://review.whamcloud.com/34058
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10777 dom: disable read-on-open with resend

The read-on-open can fill more data on reply buffer than
client allocated, this causes buffer re-allocation followed
by resend. Meanwhile FIO read test shows that such resends
perform worse than separate READ RPC. For example:
FIO 8k read is ~50% better without buffer re-allocation
with resend. Considering that there is parameter on MDC
'mdc_dom_min_repsize' to control read-on-open inline buffer
size, there is no sense to keep 'reallocation+resend'
option on MDT. Patch removes it.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I7eb9d64f5551789e93b1f7676f61c0e7a5149f76
Reviewed-on: https://review.whamcloud.com/34700
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11678 quota: make overquota flag for old req

For the old request with over quota flag, the over quota flag
should still be marked at OSC, because the old request could be
processed afther the new request at OST, then it won't break the
quota enforement at OST.

Change-Id: Ic34c438fe3f018c3b596b26ad6dc945547c8fada
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34645
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shilong Wang <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12168 utils: obdfilter fix for SHORT msgs

Sometimes obdfilter-survey shows SHORT instead of min,max.
This could happen when two signals for a parent process comes
during a verbose time. The counters are updated and start_time
is dropped. By default timeperiod is 1 second.

ost  1 sz 16777216K rsz 2048K obj    4 thr    8
write 3662.99 [4286.00,4528.95] rewrite 3873.87 [4746.85, 4857.48]
read 8088.39      SHORT

The patch fixes this issue and drops counters and time when
statistics are printed or all threads are started.

Obdfilter-survey can print SHORT after patch when subtest time
is too small 1-2 seconds. The detail log shows this case as

Total: total 8192 threads 4 sec 1.692006 4841.590396/second

Test-Parameters: trivial
Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-7110
Change-Id: I9b1521c23e9360216a279ab5c28c39bcaca9974b
Reviewed-on: https://review.whamcloud.com/34610
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12152 lnet: Cleanup lnet_get_rtr_pool_cfg

The cfs_percpt_for_each loop contains an off-by-one error that causes
memory corruption. In addition, the way these loops are nested results
in unnecessary iterations. We only need to iterate through the cpts
until we match the cpt number passed as an argument. At that point we
want to copy the router buffer pools for that cpt.

Cray-bug-id: LUS-7240
Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I8c0dc7bab7ca42dbce04a9e6efa4343da4139239
Reviewed-on: https://review.whamcloud.com/34591
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11845 osd-zfs: Support encrypted ZFS datasets

Call zfs::dmu_objset_own and zfs::dmu_objset_disown with
decrypt=B_TRUE

This is called the same way as in zfs modules.

Fixes: 0fedb017c1 ("LU-9890 osd-zfs: dmu_objset_own/disown changes")
Test-Parameters: envdefinitions=ZFS_MKFS_OPTS="encryption=on -o keylocation=file:///etc/adjtime -o keyformat=passphrase" testlist=sanity fstype=zfs
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I1d9bc1a579ac26706a9f6cc5a0d52649ce005228
Reviewed-on: https://review.whamcloud.com/33999
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>

LU-12040 mdc: reset lmm->lmm_stripe_offset in mdc_save_lovea

In order to prepare for replay lmm->lmm_stripe_offset (which contains
layout generation) has to be set to -1 (LOV_OFFSET_DEFAULT) in order
to not confuse lod_verify_v1v3

Fixes: f90abfdc96 ("LU-169 lov: add generation number to LOV EA")
Fixes: 89693927f0 ("LU-8998 lod: accomodate to composite layout")
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Cray-bug-id: LUS-7008
Change-Id: I911d3c659b6c11cc8847f0517062dd8e4df89dff
Reviewed-on: https://review.whamcloud.com/34371
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12178 osd: do not rebalance quota under memory pressure

this will happen eventually.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ibe4ef9e45deed5ea19169f3affed322351785357
Reviewed-on: https://review.whamcloud.com/34741
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11986 lnet: properly cleanup lnet debugfs files

The function lnet_router_debugfs_remove() is suppose to cleanup
the lnet specific debugfs files but that is not happening at all.
Change lnet_remove_debugfs() from doing the final debugfs lnet
and libcfs cleanup to doing specific debugfs file removal. We
can make libcfs module unloading to directly finish the entire
libcfs and debugfs tree removal instead. With this change we can
make lnet_router_debugfs_fini() call lnet_remove_debugfs().

Change-Id: I9e314e7efde806073b621166ff2e1b344e550875
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34669
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12037 mdt: add option for cross-MDT rename

Add option mdt.mdt_remote_rename, if it's not set (it's set by
default), do cross-MDT rename as cp, this is used for debug or
user want to move inode in rename.

Add sanity test_24z.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia0d122f1716f17078b375f770a193347a6e50708
Reviewed-on: https://review.whamcloud.com/34410
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11931 lnd: bring back concurrent_sends

Revert "LU-10291 lnd: remove concurrent_sends tunable"

This reverts commit 8d35d6c9bd85ed3a282aa124b672e50c02322a7d.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Icb63d7383f0d2a3cab82c1565f66670dca1f698d
Reviewed-on: https://review.whamcloud.com/34396
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11986 lnet: Avoid lnet debugfs read/write if ctl_table does not exist

Running command "lctl get param -n stats" after lnet
is taken down leads to kernel panic because it
tries to read from the file which doesnt exist
anymore.

In lnet_debugfs_read() and lnet_debugfs_write(),
check if struct ctl_table is valid before trying
to read/write to it.

Change-Id: I2450d2f89c2e8a7db793680a4df581282ee46a16
Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34622
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11779 tests: add version check for sanity-hsm

sanity-hsm test 255 was added to Lustre tag 2.12.0.
sanity-hsm test 260c was modified with Lustre tag 2.12.0.
Thus, we need to check that the server is 2.12.0
or later before running these tests.

Fixes: e7d5c1681c07 (LU-11653 hsm: copytool registration wakes the coordinator)
Fixes: b84bc6d895a0 (LU-11572 tests: make sanity-hsm test_260c reliable)
Test-Parameters: trivial serverjob=lustre-b2_10 serverbuildno=168 testlist=sanity-hsm
Test-Parameters: testlist=sanity-hsm
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I1a5369ec864432a241a875c3430baa5a064b0dfe
Reviewed-on: https://review.whamcloud.com/34589
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-947 ptlrpc: allow stopping threads above threads_max

If a service "threads_max" parameter is set below the number of
running threads, stop each highest-numbered running thread until
the running thread count is below threads_max.  Stopping nly the
last thread ensures the thread t_id numbers are always contiguous
rather than having gaps.  If the threads are started again they
will again be assigned contiguous t_id values.

Each thread is stopped only after it has finished processing an
incoming request, so running threads may not immediately stop
when the tunable is changed.

Also fix function declarations in file to match proper coding style.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I106f841e62c26b488ae837564c858a44263ebbe5
Reviewed-on: https://review.whamcloud.com/34400
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-930 doc: man page for l_getsepol

Man page for l_getsepol.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I338492cebca9a088657ff8bd5122274e7e49a5c7
Reviewed-on: https://review.whamcloud.com/34184
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>

LU-930 doc: man page for lctl nodemap_set_sepol

Man page for lctl nodemap_set_sepol.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9e27aaa7d5653fcd6225a424bdbb920471b01555
Reviewed-on: https://review.whamcloud.com/34084
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11213 uapi: reserve connect flag for plain layout

Reserve OBD_CONNECT2_PLAIN_LAYOUT flag, so that client supporting
plain layout won't enable plain layout if MDT doesn't support,
and in contrary, MDT supporting plain layout won't send such layout
to client that doesn't support.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia629e17d83b5b48c94518de428e5abd79e5a37f0
Reviewed-on: https://review.whamcloud.com/34656
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10092 pcc: Reserve a new connection flag for PCC

Reserve OBD_CONNECT2_PCC connection flag that will be set
(in ocd_connect_flags2) if a Lustre server or a client supports
Persistent Client Cache (PCC).

Test-Parameters: trivial
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ibe20c668a649be69475dc326ce56dc8708772d32
Reviewed-on: https://review.whamcloud.com/34356
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>

LU-12021 lsom: Add an OBD_CONNECT2_LSOM connect flag

Add an OBD_CONNECT2_LSOM connect flag so that clients do not send
MDS_ATTR_LSIZE and MDS_ATTR_LBLOCKS flags to the old servers that
do not support them.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I266c74e56c2cb1462e204d6fd4f1399f10621416
Reviewed-on: https://review.whamcloud.com/34343
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>

LU-12175 tests: Partial revert of LU-11636

Since landing:
LU-11636/https://review.whamcloud.com/33611/
07b271c0972757772a129e9a6370dbb163f16a06

we have seen high failure rates in several tests:
sanity 208 (LU-12175)
sanity 133g (LU-12171)
recovery-small 107 (LU-12176)
recovery-small 134 (LU-11560)

Testing with a full revert of LU-11636 showed that removing
it stops those failure.  Our best guess is that the
randomization of stripe count and mdt index is the cause,
so this patch reverts just those changes.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id6fa3079e29178555827f4e1f39d51cf8d62cf31
Reviewed-on: https://review.whamcloud.com/34705
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11394 utils: Adjust HOSTID constant

Use constant defined in spl / post-0.8.0 libspl, for HOSTID file.
Also allows get_system_hostid() to be pulled in from ZFS 0.8.x

Test-Parameters: trivial
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: Iba70d4f3b7f237260bdc964b28b601deeee81208
Reviewed-on: https://review.whamcloud.com/34278
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12166 test: fix broken detection on ZFS

We intent to run the command on mds, otherwise
project quota will never be tested.

Test-Parameters:trivial fstype=zfs
Fixes: a046e87 ("LU-7991 quota: project quota against ZFS backend")
Change-Id: I8650a0e1065f0bb465da01556472d3d23b22a530
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34609
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12160 osd-ldiskfs: use-after-free in osd_object_delete()

store a local copy of projid to avoid use-after-free.

Fixes: 39f63cf54c62 ("LU-4017 quota: add setting/getting project id function")

Change-Id: I60e19de3485cae3df1cc2e8aae6eeed4b5de3a11
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34596
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-930 doc: improve formatting of lfs.1 synopsis

Add proper command formatting for the lfs.1 SYNOPSIS section.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic70d6ccc7127510fd2df17cf6d70b0af8e3ebbe5
Reviewed-on: https://review.whamcloud.com/34594
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12151 osd-ldiskfs: pass owner down rather than transfer it

Currently, for object creation, initially uid/gid set as 0,
and then osd_quota_trasfer() is called to correct space accounting
for non-root users, function call is like:

|->osd_create
  |->osd_create_type_f
     |->osd_mkreg
        |->ldiskfs_create_inode
           |->ext4_new_inode() ->owner as NULL, create 0 as uid/gid
      |->osd_attr_init
         |->osd_quota_transfer  ->which will change uid/gid again for above.

This is inefficient since osd_quota_transfer() is a more
heavy operations, we could just pass downer owner(uid,gid),
project quota will inherit from its' parents automatically
when creating inode.

Some distros ext4 still did not support passing @owner down,
that is (rhel6,sles11) we just added extra @owner arg in
ldiskfs_create_inode() to make build system happy, and we
could add similar support to older kernel if that is really needed.

Command:
 $ salloc -N 32 --ntasks-per-node=24 mpirun -np 768 mdtest -n 2000
-F -u -d <mnt>

Without Patch:
Users Speed
root 175741.938 ops/sec
non-root  108631.673 ops/sec

Patched:
Users Speed
root 184775.286 ops/sec
non-root  185218.466 ops/sec

Patch improved ~80% for non-root users and we reached
same speed for both root and non-root users.

Change-Id: I57b0d2a6913268448c0ed90cfe76bd9f051b0b40
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34581
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12141 build: correct the required kernel version for lustre kmod

Use %kversion rather than %kver when creating preamble for lustre kmods
in lustre spec, to avoid *Requires kernel version* mismatch.

Test-Parameters: trivial

Change-Id: I9929471abd48b214510bcb499e25793ad120e6d1
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/34559
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-930 doc: man pages for lctl pool_new, pool_add

Add man pages for lctl pool_new and lctl pool_add.

More pages are needed for other commands, pool_remove,
pool_destroy, and pool_list.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie7cbb80d28610b9f74fe8f58c74c37a72e3ebbe5
Reviewed-on: https://review.whamcloud.com/34524
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12027 utils: add units to "lfs find -amctime"

The normal find command can only specify time arguments in terms
of whole days. The MacOS find(1) man page reports the use of unit
suffixes to give better control over the time range, such as "1d4h".

Possible time units are as follows:

    s       second
    m       minute (60 seconds)
    h       hour (60 minutes)
    d       day (24 hours)
    w       week (7 days)
    y       year (365 days)

There is no "month" specifier here or in find(1), since the length
of a month is not fixed, and it would conflict with the 'm' minutes
unit.  Units may be combined in one argument, e.g. "-atime -1h30m".

For matches that are "equal" to the specified time, they must match
within the smallest unit specified.  For example, "-mtime 24h" would
match anything within 1h of 24h ago, similar to "-size 100M" will
match anything within 1MB of 100MB.

Test-Parameters: trivial fstype=zfs
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib1eb4e626b712bb75f13b075849f959f003ebbe5
Reviewed-on: https://review.whamcloud.com/34367
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11403 llite: ll_fault fixes

Various error conditions in the fault path can cause us to
not return a page in vm_fault.  Check if it's present
before accessing it.

Additionally, it's not valid to return VM_FAULT_NOPAGE for
page faults.  The correct return when accessing a page that
does not exist is VM_FAULT_SIGBUS.  Correcting this avoids
looping infinitely in the testcase.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I53fc16d91462ac5d4555855dfa067d7fd6716c90
Reviewed-on: https://review.whamcloud.com/34247
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-930 doc: add audit_mode in lctl-nodemap-modify man page

audit_mode is a nodemap property added by patch under LU-9727.
It can be set via 'lctl nodemap_modify' command, so add reference
in corresponding man page.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic600756ada257e3e2cfe92a1c30e9b7342f2e4d1
Reviewed-on: https://review.whamcloud.com/34088
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11394 build: Remove SPL requirements

Because ZFS and SPL are version locked, and ZFS has explicit
requirements for SPL, remove lustre's SPL requirements.

lbuild: Make building spl optional when version is changed.

Test-Parameters: trivial
Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs ostcount=2
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: Iedba8d4047ba1fa852a2f99db2cd1b6caff33326
Reviewed-on: https://review.whamcloud.com/33743
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10717 tests: tests should not start mgs

The conf-sanity prolog do reformat_and_config which leaves
mgs service started, if it is not combined.
So, in general, test should not start mgs service, if it don't
stop mgs. And test should start mgs after reformat.

The client mount requires start of all MDTs, because of
MDT0000-osp-MDT000X synchronization.

Test-Parameters: trivial testlist=conf-sanity
Test-Parameters: mdscount=2 mdtcount=4 testlist=conf-sanity
Test-Parameters: standalonemgs=true testlist=conf-sanity
Test-Parameters: standalonemgs=true mdscount=2 mdtcount=4 testlist=conf-sanity
Cray-bug-id: LUS-2524
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I226eab5683afc36efe908b200f46b710f6235374
Reviewed-on: https://review.whamcloud.com/33589
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11601 ptlrpc: IR doesn't reconnect after EAGAIN

There is a chance that client is connecting to OST
before recovery start when OST is not configured.
In such case OST returns EAGAIN(target->obd_no_conn == 1).
There is no problem when pinger_recov is enabled
because ptlrpc_pinger_main will reconnect later.
But it doesn't reconnect when pinger_recov is 0.

Move setting imp_connect_error to ptlrpc_connect_interpret.
It is needed to store there only connection errors.

Cray-bug-id: LUS-2034
Change-Id: I35ad57e43825162f4056ad346e22a8dddea0e191
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://es-gerrit.dev.cray.com/153542
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-on: https://review.whamcloud.com/33557
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12165 quota: fix to use correct fsname array size

Max fsname is allowed to be LUSTRE_MAXFSNAME, plus '\0',
we expected arrary size should be LUSTRE_MAXFSNAME + 1.

Otherwise, we will hit following crash easily.

[864870.292204] [<ffffffff9230e84e>] dump_stack+0x19/0x1b
[864870.293186] [<ffffffff92308b50>] panic+0xe8/0x21f
[864870.294104] [<ffffffffc0f3f805>] ? qsd_enabled_seq_write+0x205/0x210 [lquota]
[864870.295418] [<ffffffff91c91b8b>] __stack_chk_fail+0x1b/0x20
[864870.296437] [<ffffffffc0f3f805>] qsd_enabled_seq_write+0x205/0x210 [lquota]
[864870.297760] [<ffffffff91e1e418>] ? __sb_start_write+0x58/0x110
[864870.298894] [<ffffffff91e91050>] proc_reg_write+0x40/0x80
[864870.299883] [<ffffffff91e1b490>] vfs_write+0xc0/0x1f0
[864870.300765] [<ffffffff91e1c2bf>] SyS_write+0x7f/0xf0
[864870.301711] [<ffffffff92320795>] system_call_fastpath+0x1c/0x21

Test-parameters: trivial
Change-Id: I33dd331a83ddac0e0c36a82480e7e90ad0ed2c2a
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34608
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>

LU-6142 ldlm: Fix style issues for ptlrpcd.c

This patch fixes issues reported by checkpatch
for file lustre/ptlrpc/ptlrpcd.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Id8186c98b272e0863fc48d63abbe33e1d703c408
Reviewed-on: https://review.whamcloud.com/34604
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 ptlrpc: Fix style issues for sec_bulk.c

This patch fixes issues reported by checkpatch
for file lustre/ptlrpc/sec_bulk.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I90ee1627561098f391776cda1958c9fd73067c51
Reviewed-on: https://review.whamcloud.com/34548
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 ldlm: Fix style issues for ldlm_request.c

This patch fixes issues reported by checkpatch
for file lustre/ldlm/ldlm_request.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ifd675677bae2279cc4a541d81a14d8ffbed64268
Reviewed-on: https://review.whamcloud.com/34547
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 ldlm: Fix style issues for ldlm_lockd.c

This patch fixes issues reported by checkpatch
for file lustre/ldlm/ldlm_lockd.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ife51cc65388966da8b25bb2e40ed30c3144c2e8e
Reviewed-on: https://review.whamcloud.com/34544
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

Revert "LU-11771 ldlm: use hrtimer for recovery to fix timeout messages"

This reverts commit 1ba794f6ec9e7ce7ad65fd74f170089fffc31d91.

We've seen several new or an increase in test failures for:
LU-12175 for sanity test 208 failures
LU-11560 for recovery-small test 134 failures
LU-12176 is for recovery-small test 107 failures
LU-12177 is for sanity test 160a failures

We are trying to pinpoint what patch landed that is causing these failures.

Test-Parameters: trivial testgroup=review-dne-part-1
Test-Parameters: testgroup=review-dne-zfs-part-1
Test-Parameters: testgroup=review-dne-part-1
Test-Parameters: testgroup=review-dne-zfs-part-1
Test-Parameters: testgroup=review-dne-part-1
Test-Parameters: testgroup=review-dne-zfs-part-1
Test-Parameters: testgroup=review-dne-part-1
Test-Parameters: testgroup=review-dne-zfs-part-1

Change-Id: I2ed33fb14726e29cae8745d671d4e25e276a7a66
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34629
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8955 tests: exercise SELinux policy info

Add new tests 21a and 21b to sanity-selinux.sh. Goal is to test
that SELinux policy info is properly sent by the client, and
checked by the server, in the following cases:
- connection
- create
- open
- unlink
- rename
- getxattr
- setxattr
- setattr
- getattr
- symlink
- hardlink

Test-Parameters: trivial testlist=sanity-selinux clientselinux
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ibd9c5added027e12d8126641c56f21fdbc791941
Reviewed-on: https://review.whamcloud.com/33699
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12133 tests: sanityn test_35 syntax error

Patch fixes test_35() trivial syntax error.

Test-Parameters: trivial envdefinitions=ONLY=35 testlist=sanityn
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-5882
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: Id81b9f071920a2111314c869fe2700e6ddf5981a
Reviewed-on: https://review.whamcloud.com/34542
Tested-by: Jenkins
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11585 quota: no IS_ERR() check in qsd_lqe_read

qsd_lqe_read() should check lqe_locate() return value with
IS_ERR() instead of != NULL.

Cray-bug-id: LUS-6636
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Change-Id: I930a16a789ece6ca52ca82ce69626d6678472c9a
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/34522
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12072 lov: remove KEY_CACHE_SET to simplify the code

We must invoke obd_set_info_async with KEY_CACHE_SET after
obd_connect for OSC device. In fact, It can be combined
in obd_connect to simplify the code.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I8cf235658a3e4af1685a7454ebfe887e5a28eccc
Reviewed-on: https://review.whamcloud.com/34419
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9625 utils: remove old lfs "cp" and "ls" sub-commands

Remove the obsolete "lfs cp" and "lfs ls" sub-commands for handling
"remote" users in a different namespace.  They have been non-working
since commit v2_8_54_0-73-g9d06de3 and were never in use before that.

Instead we have nodemap to handle UID/GID mapping.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8db90d388f8fa621d61fc65ab677b1589b3ebbe5
Reviewed-on: https://review.whamcloud.com/34240
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Nikitas Angelinas <nangelinas@cray.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>