LU-11749 tests: sanity-sec 23b exec commands on right node

In sanity-sec test 23b, make sure commands are executed on
client 1, then client 2.
Otherwise, ACL mapping cannot be correctly demonstrated.

Test-Parameters: trivial clientcount=2 envdefinitions=ONLY=23b testlist=sanity-sec,sanity-sec,sanity-sec,sanity-sec,sanity-sec,sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iffa3946be149313af696907fa6c83a6ea58cb3f6
Reviewed-on: https://review.whamcloud.com/33846
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11586 tests: test_10() fix for FILESET set

Patch fixes sanity-pfl test_10() to set root layout
properly: subdirectory could be mounted on $MOUNT if FILESET
is set, test now mounts root on $MOUNT2 for this case and
sets the root layout via $MOUNT2.

Test-Parameters: trivial testlist=sanity-pfl envdefinitions=ONLY=10
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: 6557
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: If94474d92686a37518c1dd51793e742967fdfbc1
Reviewed-on: https://review.whamcloud.com/33514
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10496 tests: disable 39k for DoM for a while

as too many runs fail due to missing FMD.

Test-Parameters: trivial

Change-Id: Ie391917fd7e251283289a4d6930d26c977810a47
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34115
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11720 spec: srpm should be free of kernel requiements

This moves the fix for LU-9731 into spec file and out of lbuild.
This lets "make rpms" benefit from the fix.
This also prevents the srpm from being incorrectly locked to the
kernel present when lbuild was used to create it (via
kmp-lustre.preamble).

Test-Parameters: trivial
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I15f61c0e37182c0efbea3566d43b1e89f180d3e5
Reviewed-on: https://review.whamcloud.com/33771
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11915 tests: stop running conf-sanity test 110

conf-sanity test 115 started hanging about the time
test 110 and 111 were added to conf-sanity. Stop running
conf-sanity test 110 by adding it to the ALWAYS_EXCEPT
list.

Test-Parameters: trivial
Test-Parameters: fstype=ldiskfs mdscount=2 mdtcount=4 testlist=conf-sanity
Test-Parameters: fstype=ldiskfs mdscount=2 mdtcount=4 testlist=conf-sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I5b36df4f0e852ba4f00945a0718bb2f6f89cc925
Reviewed-on: https://review.whamcloud.com/34166
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>

New tag 2.12.51

Change-Id: Icfc5df666e4f1d3bd384d6519069bdbeb0b07e7c

LU-11721 tests: wait for statfs to update on DNE

Wait for the statfs to update properly when there are multiple
MDTs so that the test doesn't gratuitously fail.

Fixes: 757403191c3 ("LU-11721 utils: print used inodes ratio ...")
Test-Parameters: trivial testlist=sanity mdscount=2 mdtcount=4 ostcount=7
Test-Parameters: testlist=sanity fstype=zfs mdscount=2 mdtcount=4
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia75f7bd4d3027c91f10ce990730b2bd7123ebbe5
Reviewed-on: https://review.whamcloud.com/34164
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-11605 osp: max_create_count and create_count changes

Setting max_create_count to 0 causes setting create_count
to 0. Set create_count to OST_MIN_PRECREATE when setting
back max_create_count.
Without the patch create_count remains equal to 0 despite
on changing max_create_count to something != 0.
This causes create to stuck in osp_precreate_reserve
because osp_precreate_send doesn't send new request to OST.
To understand the number of objects to precreate(grow) it
uses opd_pre_create_count that is equal to 0.

Cray-bug-id: LUS-6435
Change-Id: I940c48f91e9c7d49b766bd85ea271ce229424c7f
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://es-gerrit.dev.cray.com/153888
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://review.whamcloud.com/33559
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10059 tests: sanityn 32a restore parameters

Test sanityn test 32a does not restore parameters on error,
making other tests fail because lockless truncate is still
enabled.

Add cleanup.

Also fix use of stack_trap with restore_params in two other
tests to clean up temporary file.

Test-Parameters: trivial testlist=sanityn

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I315eb5c53cc1722833b1159c5ad908c3b7bf5550
Reviewed-on: https://review.whamcloud.com/34070
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-11868 mdc: Improve xattr buffer allocations

Many of the xattr related buffers in the mdc/mdt code are
allocated at max_easize, but they are used for normal POSIX
xattrs (primarily ACLs) and so they are guaranteed not to
exceed XATTR_SIZE_MAX.

HSM xattrs should also be less than XATTR_SIZE_MAX.

Reduce allocations to MIN(XATTR_SIZE_MAX, max_easize).

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I228486ebd79b35358baeff490d6181cefa5a4586
Reviewed-on: https://review.whamcloud.com/34059
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

LU-11871 doc: man page for lctl nodemap_set_fileset

Man page for lctl nodemap_set_fileset.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Icce0b1558621bca84b14f76037a5000002855881
Reviewed-on: https://review.whamcloud.com/34057
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-11161 tests: start running sanity 160g again

sanity test 160g was failing when run in a DNE configuration
and we stopped running this test meaning added to the
ALWAYS_EXCEPT list. The problem is that the test did not
write enough files to exceed changelog idle index threshold
for deregistering users.

Start running sanity test 160g with DNE testing again.

Test-Parameters: trivial
Test-Parameters: ostfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs clientcount=2 mdscount=2 mdtcount=4 osscount=1 ostcount=8 testlist=sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I286ef8eb7c4638ff8f357db54c4926d5a2f20ac4
Reviewed-on: https://review.whamcloud.com/33994
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11658 tests: fix typo in previous patch

Fixes: 42e83c44eb ("LU-11658 lov: cl_cache could miss initialize")

Test-Parameters: trivial testlist=conf-sanity envdefinitions=ONLY=50h

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I9981b60293f95bbc96ca838dc157f98685a17338
Reviewed-on: https://review.whamcloud.com/33983
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11838 o2iblnd: get IP address more directly.

Use dev_get_by_name() and for_primary_ifa() to
get IP address for a named device.  This is more
direct.

Linux-commit: 10e138e41a4343fd1a88e4543990205d134e562a
Linux-commit: 9eb957b98aa6322abde33240bf50dd483c5d1190

Change-Id: Ic4562c3948934bacb8613e9f6f57f609ecc04de7
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/33970
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11739 lod: don't inherit default layout from root directory

There is no need to inherit the default directory layout from
the root directory when subdirectories are created therein.
This consumes xattr space on the subdirectories, and makes it
more complex to change the filesystem default layout in the future.

This patch fixes the above issue in lod_ah_init() to check if
the parent directory is the root directory and not copy
the default layout xattr to the new subdirectory.

Change-Id: Ie0d286785bdbcd73e2ae60b429e66d5d54b44eef
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33956
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-930 doc: move server man pages to server package

Don't install the ll_decode_filter_fid.8 and llverdev.8 man pages
with the client build.  These utilities are only on the server.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I713107f6e8f09866650245d5835f41db6abcab07
Reviewed-on: https://review.whamcloud.com/33952
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 ldlm: match store macro prototype

Use size_t in the LDLM_POOL_SYSFS_WRITER_STORE() function declaration
to match the struct lustre_attr (*store) function prototype.

Test-Paramters: trivial
Fixes: 1196bd199e7 ("LU-8066 ldlm: move /proc/fs/lustre/ldlm to /sys")

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I41012a66507a048c63bfe6b6e732cf858ba5a5b1
Reviewed-on: https://review.whamcloud.com/33920
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11828 clio: fix incorrect invariant in cl_io_iter_fini()

It was discovered during PFL testing that if LINVRNT() is enabled
that cl_io_iter_fini() will crash with the following backtrace:

    kernel: LustreError: 16009:0:(cl_io.c:439:cl_io_iter_fini())
            ASSERTION( io->ci_state == CIS_UNLOCKED ) failed
    kernel: cl_io_iter_fini+0x10c/0x110 [obdclass]
    kernel: cl_io_loop+0x46/0x220 [obdclass]
    kernel: cl_setattr_ost+0x1ed/0x2a0 [lustre]
    kernel: ll_setattr_raw+0x7b0/0x9a0 [lustre]
    kernel: notify_change+0x1dc/0x430
    kernel: do_truncate+0x72/0xc0
    kernel: do_sys_ftruncate+0xf5/0x160

This is due to the incorrect assumption that the ci_state will
always be CIS_UNLOCKED, but by looking at the behavior of
cl_io_loop() it can be seen that is not the case with PFL.
We do want to make sure the IO state is not in the middle of
some other action (up to CIS_IT_STARTED or CIS_IO_FINISHED or
later) when cl_io_iter_fini() is called.

Change-Id: I41c4d3ae5618d0e138f850e63edfd2dc6d4df19a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33915
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11770 osd-ldiskfs: clean up inline macro usage

Remove inline macro usage as much as possible in the osd-ldiskfs
code, per Linux and Lustre code style.  Balance start/end braces
for functions to keep syntax highlighting working properly.

Change the autoconf macros to be checking for the new structure
fields instead of the old fields, and add compat macros for older
kernels so that they can later be removed without code churn.

Remove direct inode->truncate() usage and autoconf check, since
ldiskfs_truncate() has been exported since at least RHEL6.3.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifc3187275c1841918228fac2368b18a9fa89a47d
Reviewed-on: https://review.whamcloud.com/33841
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11566 utils: fix lctl llog_print for large configs

If "lctl llog_print" is called for a large configuration, it will
overflow the 8KB buffer limit for OBD ioctl commands.  The kernel
snprintf calls try to overflow the supplied buffer.  Avoid that.
If the configuration is large, fetch the configuration records in
chunks and print them incrementally.

Add --start and --end options to llog_print and deprecate the use of
positional parameters, since positional parameters are increasingly
complex to parse as options are added, and are harder to use.

The callback for the configuration records will allow "lctl pool_*"
commands to be processed directly on the MGS.

Move existing llog_print test_60aa, test_60ab to conf-sanity as
test_123aa and test_123ab (rename set_param -F test_123 to test_123F).
Add new test_123ac and test_123ad for the new llog_print --start and
--end param, and update test_123aa to test old positional parameters.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib7d2ae893033bd4594646c980b7d0ddbd2b3a089
Reviewed-on: https://review.whamcloud.com/33815
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11665 tests: check number of pages correctly

On ARM or other 64KB PAGE_SIZE systems, check the read cache size
against the actual PAGE_SIZE instead of just checking the number
of pages being read.

Test-Parameters: trivial clientarch=aarch64 testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8fb90325a18b343c5f5af01df603a25fe33ebbe5
Reviewed-on: https://review.whamcloud.com/33772
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11721 utils: print used inodes ratio when using "lfs df -i"

"lfs df -i" prints the used blocks percentage, instead of the used
inodes percentage. Fix this by allowing obd_statfs_ratio() to
distinguish when "-i" is used.

Round up the ratio returned from obd_statfs_ratio() in a ceiling
manner, to match the output of df(1). Add a sanity test to check
that the outputs from df(1) and lfs df match.

Signed-off-by: Nikitas Angelinas <nangelinas@cray.com>
Cray-bug-id: LUS-6748
Test-Parameters: trivial
Change-Id: I0b31ecb7371875c93bc07dda1f1c89e04d5b4576
Reviewed-on: https://review.whamcloud.com/33758
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>

LU-8955 sec: create new function sptlrpc_get_sepol()

Create new function sptlrpc_get_sepol() in ptlrpc/sec.c to compute
SELinux policy info, by calling new userland command l_getsepol.

The SELinux policy info syntax is the following:
<mode>:<name>:<version>:<hash>
where:
- <mode> is a digit telling if SELinux is in Permissive mode (0)
  or Enforcing mode (1)
- <name> is the name of the SELinux policy
- <version> is the version of the SELinux policy
- <hash> is the computed hash of the binary representation of the
  policy, as exported in /etc/selinux/<name>/policy/policy.<version>

Userland command l_getsepol can be called on the command line by a
security administrator to get SELinux status information to store into
'sepol' field of nodemap.

SELinux status information is reported by Lustre client only if
new 'send_sepol' ptlrpc kernel module's parameter is not zero, and
SELinux is enabled on the client.
'send_sepol' accepts various values:
- 0: do not send SELinux policy info;
- -1: send SELinux policy info for every request;
- N > 0: only send SELinux policy info every N seconds. Use max value
  2^31-1 (signed int on 32 bits) to make sure SELinux policy info is
  only checked at mount time.
Independently from 'send_sepol' value, SELinux policy info has an
associated mtime. l_getsepol checks mtime and recalculates whole
SELinux policy info (including SHA) only if mtime changed.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I7a4b955f36c5e0f4eca1561beae2b40fec88d570
Reviewed-on: https://review.whamcloud.com/24421
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8955 nodemap: add SELinux policy info to nodemap

Give the ability to set SELinux policy information on a nodemap,
in a new nodemap field named 'sepol'.
When set, a client pertaining to this nodemap will be allowed to
connect only if the SELinux policy information it sends matches
the one stored in the nodemap.

Expected 'sepol' string format is:
<1-digit>:<policy name>:<policy version>:<policy hash>
1-digit is 0 for SELinux Permissive mode, 1 for Enforcing mode.

SELinux policy info of nodemap is stored permanently by using
'lctl set_param -P' commands. It means MDS and OSS will be able to
retrieve SELinux policy info of nodemap after a restart (when they
read params llog).
MGS will not see SElinux policy info after restart, but this does
not prevent the feature from working.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia1d178dd34f05ede020b490b1797a71dbae15d7b
Reviewed-on: https://review.whamcloud.com/24420
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1365 tests: LARGEDIR and 3-level htree for local ldiskfs

Add conf-sanity.sh test for the LARGEDIR feature with Lustre + ldiskfs
to exceed both 2-level htree (with 1024-byte blocksize to reduce the
number of entries per htree level) and 2GB directory size (marked SLOW
since this takes a while to run). This is done using a smaller number
of hard-linked inodes (nlink_max = 65000), to avoid the overhead of
accessing and caching a large number of different inodes.

Running multiple threads on a single directory is serialized by the
VFS directory mutex, so the tests are run in parallel on all available
clients and mountpoints to speed up the test as much as possible.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: If52ef346a39a1c3236871edbb0327bf44fe1a392
Test-Parameters: trivial testlist=conf-sanity envdefinitions=SLOW=yes,FLAKEY=false
Reviewed-on: https://review.whamcloud.com/22009
Tested-by: Jenkins
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-1365 utils: allow set block size for ldiskfs backend

Add “-b” option to mkfs.lustre that allows to set backend block size.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: I83fc76f64ce2a0b4bf500841b695d64d3dea78de
Reviewed-on: https://review.whamcloud.com/33757
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

LU-11790 ldiskfs: add terminating u32 when expanding inodes

In ext4_expand_extra_isize_ea(), we calculate the total size of the
xattr header, plus the xattr entries so we know how much of the
beginning part of the xattrs to move when expanding the inode extra
size.  We need to include the terminating u32 at the end of the xattr
entries, or else if there is uninitialized, non-zero bytes after the
xattr entries and before the xattr values, the list of xattr entries
won't be properly terminated.

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I247b935b3cf315481dc4658133a7eee02b6350e9
Reviewed-on: https://review.whamcloud.com/33893
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>

LU-11889 tests: disable sanity/156 for ZFS OST

disable sanity/156 with ZFS OST as osd-zfs doesn't
maintain cache hit/miss stats yet.

Test-Parameters: trivial

Change-Id: I34eac3dcc49962ead0c0d6b17d8d605036872f4f
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34114
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11607 tests: create routine to get Lustre env

The Lustre tests in the test suites make repeated calls
to a small number functions that relate to the Lustre
environment. Some examples are the version of the Lustre
server or client and the file system type of the server.

Collect these calls into a routine called get_lustre_env()
in the test-framework.sh library and replace calls in
sanity with the global variables.

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I01dd00dd50cca39c964c5fd8abc3f51ab3c8e6b8
Reviewed-on: https://review.whamcloud.com/33938
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9859 libcfs: add watchdog for ptlrpc service threads.

If a ptlrpc thread takes longer than expected, it
can be used to get a stack trace to see where the
problem is.

This patch moves this functionality out of libcfs
by using a delayed work struct to report a back
trace of the thread if it ever fires.

Change-Id: I8f54df430322f0751b179f67284018e576defc08
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33018
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8939 nodemap: remove deprecated lproc files

Before the MGS was able to transfer the nodemap configuration to the
MDSes and OSSes, people had to use lproc interfaces in order to load
the configuration. This patch removes those deprecated interfaces.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I8c8f38755cf2870fbabca9e96d7d51a9ac290193
Reviewed-on: https://review.whamcloud.com/24352
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7239 mdd: make mdd_attr_set() synchronous less often

In permission_needs_sync(), test mode changes for S_ISUID or S_ISVTX
rather than simply checking whether the new mode in the incoming
request has those bits set.  Similarly check for uid and gid changes.

Prior to these changes, copying a directory tree into lustre, whose
directories' modes have S_ISUID | S_ISGID | S_ISVTX set, is
unnecessarily slow because chmod() system calls are synchronous
whether they change the actual permissions or not.

Change-Id: I41d3297a007eb4b796790b3b3de0ad355d17cb58
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/16699
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7159 tests: fix 224c fault injection

Sanity 224c test always passes because no fault injection happened.

This Patch adds additional step in fault injection and fix this test.

Test-Parameters: clientarch=aarch64 testlist=node-provisioning-1,lustre-initialization-1,lnet-selftest,sanity

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@seagate.com>
Xyratex-bug-id: MRP-2472
Change-Id: I5f637e98b46ddda6acbafb956f9b85fccb7cae15
Reviewed-on: https://review.whamcloud.com/16426
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>

LU-11852 osc: skip cache entry wakeup without grants

No need to wakeup osc_entry_cache waiters with EDQUOT or insufficent
grants, cause they will sleep again.
osc_enter_cache could skip queue of waiters if client has enough
grants. The fix adds list_empty checking and improves FIFO.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6119
Change-Id: I0a2415a19b2c8ecd7df3411d88cab9fee2470a10
Reviewed-on: https://review.whamcloud.com/34013
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11845 zfs: put configure checks in version order

Put the ZFS feature checks in release version order, so that it is
easier to track when they apply and when they can be removed in
the future.

Make the configure checks use decrypt=B_TRUE just to illustrate
more correct usage.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I22053638d72b41b51b6f56dea5668e78535cab07
Reviewed-on: https://review.whamcloud.com/34000
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11716 tests: fix scrub tests

scrub_prep set MDT failloc to 0x198. It prevents the inserting
of osd_obj to OI, and brokes the logic of osd_fid_lookup. The
creating and searching quota slv file could happen during this
failloc. osd_fid_lookup() returned osd object with oo_inode equal
to zero and a bit later, osd_index_try got BUG cause it tried to
access to &LDISKFS_I(inode)->i_es_lock.

The patch adds OST object creation for quota synchronization before
setting failloc.

Test-Parameters: trivial testlist=sanity-scrub
Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6051
Change-Id: I8a37c476c2c95b235b0f0ecc7ec1ab0066ae3883
Reviewed-on: https://review.whamcloud.com/33750
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11625 ofd: handle upgraded filter_fid properly

Since there have been several iterations of struct filter_fid stored
on disk, the current code wasn't checking for all of the possible
cases when trying to decide what action to take when accessing and
upgrading the xattr for new capabilities.

Properly check for the various different struct filter_fid sizes and
handle them appropriately.  Add a more verbose description of the
various cases so that this is more clear to others in the future.

Add decoding of filter_fid fields added for FLR in 2.11.

We should already be testing for upgrading the filter_fid xattr
from different OST versions in conf-sanity test_32d.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifef2292296236cb06ff7e8cd50caff4b133ebbe5
Reviewed-on: https://review.whamcloud.com/33627
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11555 utils: ZFS check multihost enabled in read_ldd()

For ZFS check that multihost is enabled if failover host is defined.
Print a warning if it's not.

Test-Parameters: trivial mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: Iddb5871afc6fb6808a25921c8d3e8516d675f15c
Reviewed-on: https://review.whamcloud.com/33491
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10953 tests: disable sanity/300k on small setups

sanity/300k simulates creation of 512-stripes directory and
fails on small setup causing all subsequent tests to fail as
well.

Change-Id: I5e1d8d25ea364c0f49d9e114cc447cc40f951e2f
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/32148
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 lnet: change lnet_ipaddr_enumerate() to use for_each_netdev()

for_each_netdev() is a more direct interface than
lnet_ipif_enumerate(), so use it instead.  Also get
address and 'up' status directly from the device.

This means we need to possible re-allocate the storage
space if there are lots of IP addresses.

However there is no need to resize the allocation down if we
over-allocated.  This is only used once, and is freed soon
after it is allocated, so that is a false optimization.

Linux-commit: 0400cf406c32ac3968241cd528747d922b6c55c3

Change-Id: I1c1e7722c7b2b267dcb8134ae295a54f976d96ad
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/33969
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 kernel: harden current_time autoconf test

In newer kernels CURRENT_TIME was replaced by current_time(). The
return value of current_time() was struct timespec but to support
time after 2038 the return value was changed to struct timespec64.
This change broke the autoconf test. The solution is to use one
of the struct iattr field in the autoconf test since it hides
the return value type.

Test-Parameters: trivial

Change-Id: I95abd2cd2b777f99cbf6ab78370ee2171e5fca67
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33963
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-9325 msg: replace obsolete simple_strtoul use

The kernel function simple_strtoul() has been superseded by the
kstrto* family of functions. Replace all uses of simple_strto*()
with the equivalent kstrto*() function in the msg subsystem.

Change-Id: I5ca5df29b8b2f34107ee40628ab3b77d57547b99
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33959
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10447 tests: replace $SET/$GETSTRIPE in replay tests

$SETSTRIPE and $GETSTRIPE were needed when we used the
standalone 'lstripe' utility. 'lstripe' hasn't been used
for years and we need to clean up all remnants of it.

Replace all instances of $SETSTRIPE with '$LFS setstripe'
and $GETSTRIPE with '$LFS getstripe' in the replay-dual,
replay-ost-single, and replay-single test suites.

Test-Parameters: trivial testlist=replay-dual,replay-ost-single,replay-single
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I3902142061ddb3c8860b6983d5ec55383aeb25a9
Reviewed-on: https://review.whamcloud.com/33918
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10966 utils: Fix `lfs check` documentation and arguments

Several bugs exist for 'lfs check'. Add checking for MGS servers.
Add 'lfs check mdts' which is inline with the other options but
still keep mds for backwards compatibility. Do the same with the
'all' option which is equivalent to older 'servers' that is also
kept for backwards compatibility. Update man pages to reflect
these changes

Change-Id: Ifffec8fd1498616036122f9af311cd6366d693bc
Signed-off-by: George Melikov <mail@gmelikov.ru>
Reviewed-on: https://review.whamcloud.com/33775
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11418 mdd: delete name if orphan doesn't exist

mdd_orphan_destroy() should delete name if orphan object doesn't
exist, otherwise the orphan clean thread will try to destroy this
orphan in dead loop.

add sanity test_811.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id22b2fab0ac87dfb81ca9f01d8ed0338f1b12120
Reviewed-on: https://review.whamcloud.com/33661
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11402 tests: add version check sanity-quota 60

sanity-quota test 60 was added to Lustre tag 2.11.53. Thus,
we need to check that the server is 2.11.53 or later before
running test 60.

Test-Parameters: trivial serverjob=lustre-b2_10 serverbuildno=152 testlist=sanity-quota
Test-Parameters: testlist=sanity-quota
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I5db738f0776b5615165977df1708f197f215b994
Reviewed-on: https://review.whamcloud.com/33418
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11843 tests: customise sanity-quota

Patch adds the possibility to not turn project quotas OFF
at the end of sanity-quota if it is ON initially.
ENABLE_PROJECT_QUOTA=false is ignored in this case.

Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6162
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Test-Parameters: trivial testlist=sanity-quota
Change-Id: Ic7d58f4a077acdbeec426613777e3c7c0778f0a0
Reviewed-on: https://review.whamcloud.com/33997
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 lustre: discard LTIME_S macro

Rather than using a macro, just access the required
field directly.

Linux-commit: 5b7cc4e4ce3cc15f67462ae75c55eecc7edc3a40

Change-Id: I325cac7458265d1cf6ad9f195a513f2612865906
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/33984
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-11834 llite: fix temporary instance buffer size

The formatting of the cfg_instance variable was changed in LU-11809
to always use a fixed "%016llu" format, but the temporary buffer
allocations for the instance string were not changed to match the
printed value.

This results in string truncation in some situations.  Change the temp
buffer size to always have 16 bytes for the instance instead of using
sizeof(cfg_instance).

Fixes: cd294a1255 ("LU-11809 llite: don't use %p for cfg_instance")

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5eca99afa2787cc57e739489b252b12af68cab07
Reviewed-on: https://review.whamcloud.com/33951
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11825 clio: Remove pio feature

The pio feature was an interesting concept, but never
achieved real use in production, primarily because it was
actually slower than normal i/o for some common workloads,
so it could not be on by default.

It has had a trivial deadlock in it (with truncate) since
release, and this has never been reported by a customer.

The attemp to rewrite readahead to use ptasks was never
fully successful, and is being superseded by a different
implementation.

Given all this, there is no reason to keep the extra
complexity in the already complicated clio code.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ie558db946af94189e35e985e8894706c14ed5f39
Reviewed-on: https://review.whamcloud.com/33912
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11810 misc: allow Fixes: tag in commit signoff block

Allow the "Fixes:" tag in the signoff block of the commit message.
The Fixes: tag should contain a valid git commit hash, and may
optionally be followed by a description of the original patch.
For example:

Fixes: 5760b34c48b ("LU-1145 test: add Test-Parameters tag")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I49f712dd8da173510e5941b66eb050e53a1cab07
Reviewed-on: https://review.whamcloud.com/33888
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11683 build: Update ZFS to 0.7.12

0.7.12

ZFS
 * Add BuildRequires gcc, make, elfutils-libelf-devel #8095
 * Fix flake8 "invalid escape sequence 'x'" warning #8056
 * ZTS: Update O_TMPFILE support check #7528
 * Allow use of pool GUID as root pool #8052
 * Fix arc_release() refcount #8000
 * Prefix all refcount functions with zfs_ #7963
 * Linux 4.19-rc3+ compat: Remove refcount_t compat #7885
 * Zpool iostat: remove latency/queue scaling #7945
 * Fix statfs(2) for 32-bit user space #7122
 * Skip import activity test in more zdb code paths #7797
 * Reduce taskq and context-switch cost of zio pipe #7736
 * Fix race in dnode_check_slots_free() #7147

SPL
 * Add BuildRequires gcc, make, elfutils-libelf-devel #8095
 * Linux 4.20 compat: current_kernel_time() #8074
 * Backport vnode.h changes from zfs:93ce2b4c
 * Define timestruc_t for Lustre compatibility #8014

0.7.11

ZFS
 * tx_waited -> tx_dirty_delayed in trace_dmu.h #7096
 * Revert "zpool reopen should detect expanded devices"

SPL
 * Linux 4.18 compat: Kconfig #7890)

0.7.10

ZFS
 * Correctly handle errors from kern_path #7764
 * Fix build with CONFIG_GCC_PLUGIN_RANDSTRUCT {{f4ee628}}
 * Fix 'zfs recv' of non large_dnode send streams #7617
 * Fix object reclaim when using large dnodes #7097
 * Fix problems receiving reallocated dnodes #6821
 * Fedora 28: Fix misc bounds check compiler warnings #7826
 * Fix libaio-devel requirement for Debian-based distributions #7829
 * Add libaio-devel BuildRequires #7821
 * Add missing zfs-dracut RPM dependencies #7729
 * Don't modify argv] in user tools #7760
 * Fix arcstat.py handling of unsupported options #7799
 * Allow inherited properties in zfs_check_settable() #7755
 * Fix zfs incremental send remove '-o' properties #7478
 * OpenZFS 8906 - uts: illumos rootfs should support salted cksum
 * Fix zpl_mount() deadlock #7598
 * Fix kernel unaligned access on sparc64 #7642
 * Default ashift for Amazon EC2 NVMe devices #7676
 * Linux 4.14 compat: blk_queue_stackable() #7645
 * Add pool state /proc entry, "SUSPENDED" pools #7331
 * zpool reopen should detect expanded devices #7546
 * Support Debian DKMS builds #7358
 * module param callbacks check for initialized spa #7496
 * Trim new line from zfs_vdev_scheduler #3356
 * Fix ENOSPC in "Handle zap_add() failures in ..." #7401
 * Fix divide-by-zero in mmp_delay_update() #7391
 * OpenZFS 8997 - ztest assertion failure in zil_lwb_write_issue #7084
 * Linux compat 4.18: check_disk_size_change() #7611
 * Linux 4.18 compat: inode timespec -> timespec64 #7643
 * zv_suspend_lock in zvol_open()/zvol_release() #6342

SPL
 * Add rwsem_tryupgrade for 4.9.20-rt16 kernel (SPL) #7589
 * Add pool state /proc entry, "SUSPENDED" pools (SPL) #7331
 * Linux 4.18 compat: inode timespec -> timespec64 #7643

Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I7b108d659d9d6cd6df3d4c55d8b8380f68ccaa6e
Reviewed-on: https://review.whamcloud.com/33687
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11661 test: improve sanityn test_47g

'stat' may be run before 'mkdir', to avoid this, sync data before
test and wait longer time after 'mkdir' in background.

Test-Parameters: trivial testlist=sanityn,sanityn,sanityn,sanityn mdtfilesystemtype=zfs mdscount=2 mdtcount=4 envdefinitions=ONLY=47g
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I314bc9d36629a5185efc5ef8281a03337ea77776
Reviewed-on: https://review.whamcloud.com/33647
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11579 llite: remove cl_file_inode_init() LASSERT

If there is some corruption or other reason that the file layout
cannot be used, the first call to cl_file_inode_init() will fail.
If it is called a second time on the same file then it will hit
an LASSERT() since I_NEW is no longer set on the inode.

It would be good to handle the error in lov_init_raid0() better,
but we still want to avoid this LASSERT() if there is an error.

Convert the LASSERT() in cl_file_inode_init() into a CERROR() and
error return.  This is being triggered due to corruption on the
server, but that shouldn't cause the client to assert.

    lov_dump_lmm_common() oid 0xdf4e:311367, magic 0x0bd10bd0
    lov_dump_lmm_common() stripe_size 1048576, stripe_count 4
    lov_dump_lmm_objects() stripe 0 idx 10 subobj 0x0:151194471
    lov_dump_lmm_objects() stripe 1 idx 12 subobj 0x0:152477530
    lov_dump_lmm_objects() stripe 2 idx 25 subobj 0x0:151589797
    lov_dump_lmm_objects() stripe 3 idx 2 subobj 0x0:150332564
    lov_init_raid0() fsname-clilov: OST0019 is not initialized
    cl_file_inode_init() Failure to initialize cl object
        [0x20004c047:0xdf4e:0x0]: -5

    cl_file_inode_init() ASSERTION(inode->i_state & (1 << 3) ) failed
    cl_file_inode_init() LBUG
    Pid: 37233, comm: ll_sa_4709 3.10.0-862.14.4.el7.x86_64 #1 SMP
    Call Trace:
    libcfs_call_trace+0x8c/0xc0 [libcfs]
    lbug_with_loc+0x4c/0xa0 [libcfs]
    cl_file_inode_init+0x2ac/0x300 [lustre]
    ll_update_inode+0x315/0x600 [lustre]
    ll_iget+0x163/0x350 [lustre]
    ll_prep_inode+0x232/0xc80 [lustre]
    sa_handle_callback+0x3a4/0xf70 [lustre]
    ll_statahead_thread+0x40e/0x2080 [lustre]

Instead, return an IO error instead of killing the client.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If8a6eb24df09e7e158b61f02e2517132893ebbe5
Reviewed-on: https://review.whamcloud.com/33505
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11496 tests: start_client_loads () fix

recovery-*-scale tests are not functional with LOADS=IOR
because of MPIRUN and MPIRUN_OPTIONS are not set.
Patch fixes this defect.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6220
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I9b6d3810978260c0cec4a5482f093f788651aac0
Reviewed-on: https://review.whamcloud.com/33339
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10447 tests: deprecate $SETSTRIPE/$GETSTRIPE in sanity

$SETSTRIPE and $GETSTRIPE were needed when we used the
standalone 'lstripe' utility. 'lstripe' hasn't been used
for years and we need to clean up all remnants of it.

Replace all instances of $SETSTRIPE with '$LFS setstripe'
and $GETSTRIPE with '$LFS getstripe' in the sanity and
sanityn test suites.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I492d261c7568634c8f0dd0d5a100ef5bb8433ad8
Reviewed-on: https://review.whamcloud.com/32809
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8955 cfg: reserve flags for SELinux status checking

Reserve LCFG_NODEMAP_SET_SEPOL config flag that will be used to
define sepol parameter on nodemap entries.
Reserve OBD_CONNECT2_SELINUX_POLICY connection flag that will be set
(in ocd_connect_flags2) if a client supports sending the SELinux
policy status info.
Add checks for all lcfg_command_type constants, along with lustre_cfg
and cfg_record_type.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I009eb8f1597b51acce7dc21498a43096e41cd034
Reviewed-on: https://review.whamcloud.com/33797
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>

LU-10937 sptlrpc: split sptlrpc_process_config()

Make sptlrpc_process_config() more than a single line wapper
exporting function. Instead migrate the lcfg parsing out of
__sptlrpc_process_config() so that we can use this function
for both LCFG_PARAM and LCFG_SET_PARAM handling.

The first field parsed from struct lustre_cfg *lcfg is the
target. This can be "_mgs", file system name, or an obd target
i.e fsname-MDT0000. We can move to extracting the file system
name out of the target string using server_name2fsname().

Test-Parameters: testlist=sanity-sec envdefinitions=SHARED_KEY=true

Change-Id: If94ee8164ab2b49fe10d098b32faea9e331f04ac
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33760
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11647 ptlrpc: always unregister bulk

In ptlrpc_check_set, the bulk should be unregistered before
ptl_send_rpc in any case.

Change-Id: Icf963002f934b43ccbb9d6ef02ba7f9d11f297f8
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/22378
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7558 ptlrpc: connect vs import invalidate race

Connect can't be sent while import invalidate is
in progress, thus it leaves the import in not
initialized state.

Don't allow reconnect in evicted state.

Change-Id: I79a1a1eb05fede30e100ba09b6f3f98636a46213
Cray-bug-id: LUS-6322
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33718
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7631 tests: add debug info to conf-sanity 82a

In the routine check_stripe_count, the different error
messages need to be modified so when an error occurs,
a user can tell what error was hit. Also, print precreated
object information at the beginning of the test and on
error.

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=conf-sanity envdefinitions=ONLY=82a
Test-Parameters: mdscount=1 mdtcount=1 testlist=conf-sanity envdefinitions=ONLY=82a
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ifc75d52d38d9cb401118ef7baa4014bddf6298f2
Reviewed-on: https://review.whamcloud.com/33689
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11723 kernel: kernel update RHEL7.6 [3.10.0-957.1.3.el7]

Update RHEL7.6 kernel to 3.10.0-957.1.3.el7.

Test-Parameters: clientdistro=el7.6 ossdistro=el7.6 mdsdistro=el7.6

Change-Id: I9e2679104b226d2ab334b7f870ccdd495b2345f9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33763
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9010 tests: Change static defines to use macro for kinode.c

This patch replaces completion which is defined statically in
file lustre/tests/kernel/kinode.c with kernel provided macro.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I7b41d3f0a67f471fe7f5dfa840521dc690a163d2
Reviewed-on: https://review.whamcloud.com/33935
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9010 ptlrpc: Change static defines to use macro for gss_svc_upcall.c

This patch replaces spinlock which are defined statically
in file lustre/ptlrpc/gss/gss_svc_upcall.c with kernel
provided macro.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I9978fcf50763df1b908c41377eef89ce284cf835
Reviewed-on: https://review.whamcloud.com/33934
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10866 pacemaker: Support additional mount options

Add support for additional mount options for pacemaker resources.
Logging is also added for the output of the mount and umount commands.

Signed-off-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Change-Id: Ideca8e37b12015b9f455634c85295137e00b97fd
Reviewed-on: https://review.whamcloud.com/31836
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Joe Grund <jgrund@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11737 lfsck: do not ignore dryrun

lfsck_layout_recreate_lovea() shouldn't ignore dryrun.

Change-Id: Ia8bafc13f148b03573dee5db26b6aff9386b5b5f
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33826
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>

LU-10815 tests: all_mdts_nodes() all_osts_nodes() defect

When mds_HOST and ost_HOST are not set (mds1_HOST,
ost1_HOST, ost2_HOST, etc are used), all_mdts_nodes() and
all_osts_nodes() have client in the list set due to cfg/local.sh:
  mds_HOST=${mds_HOST:-$(hostname)}
  ost_HOST=${ost_HOST:-$(hostname)}
With this change these functions include mds_HOST and ost_HOST
into the list only if mdsN_HOST and ostN_HOST are not set.

Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-5784
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I1ccfc94a186db98fbf87698cb8fc7d08ee008fdc
Reviewed-on: https://review.whamcloud.com/33342
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11755 ldlm: remove LASSERT() in ns_is_{client,server}()

(ns->ns_client == LDLM_NAMESPACE_CLIENT ||
ns->ns_client == LDLM_NAMESPACE_SERVER) implies that
(!(ns->ns_client & ~(LDLM_NAMESPACE_CLIENT |LDLM_NAMESPACE_SERVER)))
is also true, so the latter LASSERT() can be removed from
ns_is_{client,server}().

Signed-off-by: Nikitas Angelinas <nangelinas@cray.com>
Cray-bug-id: LUS-6762
Change-Id: Ib165d97ca397d5109c046415d2ec513357b07cbd
Reviewed-on: https://review.whamcloud.com/33822
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11820 tests: skip conf-sanity test_106 for SLOW=no

This test currently takes 40 minutes to run, so it should be
skipped for SLOW=no test runs.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I120747e21d046bd28667e2ff26013ae51f9cab07
Reviewed-on: https://review.whamcloud.com/33902
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11606 utils: Call llapi_error() for errors in liblustreapi files

Change calls to remaining printf() and fprintf() to use
llapi_printf() and llapi_error() so that it is possible
to override these calls when liblustreapi is linked into
some other application.

Change-Id: I6531d29b05b34766a68dabf17299cd00babe2517
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33877
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-8066 lnet: properly isolate kernel_param_ops

Don't reference kernel_param_ops if not available.  There is
already a HAVE_KERNEL_PARAM_OPS configure check for this, but
it is just misplaced.

Fixes: 7092309f32516cbfb95a964c87b8030129edeb27

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I56f86257c6bc8a9c53c7901bc2765e10587cab07
Reviewed-on: https://review.whamcloud.com/33874
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11771 osd: avoid use of HZ in brw_stats

Don't use HZ when computing the brw_stats IO time.  This brw_stats
field was formerly in jiffies, but since the conversion to ktime_t
(now always in msec) the output is incorrect when HZ != 1000.

Fixes: 4c2f028a95063aa30006a071a039e8a3eefe52f2

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia01705b94ddedec2e375621c6e01a957f53ebbe5
Reviewed-on: https://review.whamcloud.com/33857
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11757 lod: use calculated stripe count

lod_prep_md_striped_create() tries to allocat big
chunk of memory because
lum->lum_stripe_count == -1 and is converted to __u32.

ldo_dir_stripe_count was calculated already in lod_ah_init()

Change-Id: Id99d9e024638dfb1b34262840d2e543c808a9cdc
Cray-bug-id: LUS-6694
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-on: https://review.whamcloud.com/33829
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11736 utils: don't set max_sectors_kb on MDT/MGT

The max_sectors_kb tunable should not be applied to MDT and MGT
devices. This tuning is needed for efficiency of large IOs for
spinning disks, but is not needed for SSDs or regular IO. It can
cause problems with DM Multipath configurations for minimal
benefits, so should be limited to OST devices.

This only applies to ldiskfs backend filesystems, no such tuning
is currently done for any ZFS devices.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I496603da13aae042f63cc37c0dea221a393ebbe5
Reviewed-on: https://review.whamcloud.com/33796
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 tests: Fix replay-single test_85b

test_85b of replay single sets a default striping on $DIR
and does not remove it.  This makes it impossible to
correctly test self-extending layouts, so fix this first.

cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I0057c8403e3dae2437cf0c8810af8086e2971c35
Reviewed-on: https://review.whamcloud.com/33777
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 sysfs: make ping sysfs file read and writable

Starting with 4.15 kernels any sysfs read only is limited to
root access only. To retain the ability for non root users
to detect if a remote server is alive using the 'ping' sysfs
file we need to change it to writable. Retain the read ability
so older tools will work.

Change-Id: I6560c119328d723a20a2b32e1fa8c68dce5d407a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33776
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-11719 ldlm: Adjust search_* functions

The search_itree and search_queue functions should both
return either a pointer to a found lock or NULL.

Currently, search_itree just returns the contents of
data->lmd_lock, whether or not a lock was found.

search_queue will do the same under certain cirumstances.

Zero lmd_lock in both search_* functions, and also stop
searching in search_itree once a lock is found.

cray-bug-id: LUS-6783
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ie231166756e60c228370f8f1a019ccfe14dfda6a
Reviewed-on: https://review.whamcloud.com/33754
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11717 tests: exit_status () defect

exist_status() should search a word FAIL, otherwise
all the tests skipped due to reason which contains
FAILOVER_MODE have status = 1 and exited with the error.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6711
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I5a69e0a4aed6cc8d75d97fe83c6f304b666701ea
Reviewed-on: https://review.whamcloud.com/33751
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11415 socklnd: improve scheduling algorithm

Modified the scheduling algorithm to use all scheduler threads
available. Previously a connection is assigned a single thread
and can only use that one. With this patch any scheduler thread
available on the assigned CPT can pick up and work on requests
queued on the connection.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I7e8e5220e1914155fd793994485e3a443fe1ba44
Reviewed-on: https://review.whamcloud.com/33740
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11700 osp: GPF in osp_reserved_mb_low_seq_write()

opd_pre pointer may be uninitialized on statrup

Change-Id: If6ff4289545503e07719c7e9a282be1bd393465b
Cray-bug-id: LUS-6118
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33717
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11699 lfsck: Umount while running LFSCK

lod_device_free() ASSERTION( atomic_read(&lu->ld_ref) == 0)failed

Wake up master thread after releasing all refernces.

Change-Id: I2933f7c58c12f4e3cd2510b01babbf02814c8768
Cray-bug-id: LUS-6510
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33715
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11696 utils: "lfs getsom" returns "24" to userspace

The "lfs getsom" command always returns "24" to userspace because
"rc = 24" (sizeof(struct lustre_som_attrs)) after fetching the
xattr from the kernel.
In this patch, rc is set to 0 if the lfs_getsom()->lgetxattr()
call returns a positive value.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ie3151f67b5ce2b5b2bc35a4b6528ba9a20a5db9f
Reviewed-on: https://review.whamcloud.com/33714
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11689 lfs: make sure project proceed all dirs

Leftover fix since LU-10986 lfs: make lfs project tolerant errors
We should proceed other dirs if we hit errors, otherwise,
some dirtree like following will fail if aaaa not exists.

testdir/
├── subdir
│   └── 1
├── bbbb -> aaaa
└── cccc

Also remove extra error output since we have output failing
messages inside every action function.

Test-Parameters: trivial testlist=sanity-quota,sanity-quota
Change-Id: I0062dbc3f4d1925c9e9e1a509ee35ac569bd9b74
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33707
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11669 tests: add project in yml_test_group()

This patch fixes yml_test_group() in yaml.sh to
add test project name, which is required in
results.yml for Maloo to parse.

Test-Parameters: trivial

Change-Id: I0ae563d855dc2d28eaea85e86b1cb23d2428988b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33658
Tested-by: Jenkins
Reviewed-by: Leonel Ochoa <lochoa@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11658 lov: cl_cache could miss initialize

The cl_cache may be missed initialize when we mount
a client with deactivate osc and then active it.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I92cd44375d70624fb55ef7a0218e7178211a8687
Reviewed-on: https://review.whamcloud.com/33650
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11620 lfsck: change llsd_rb_lock to rwsemaphore

llsd_rb_lock is taken in ->init, and released in ->fini, but during
this period it may getxattr which can sleep. Change it to rwsemaphore.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Idc68eb886e60dc45ccfb7ac9bf5bf06db42d690d
Reviewed-on: https://review.whamcloud.com/33603
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11579 lov: quiet lov_dump_lmm_ console messages

Limit messages in lov_dump_lmm_objects() and lov_dump_lmm_common()
printing to the console repeatedly when D_ERROR is used.  Change
CDEBUG() to CDEBUG_LIMIT() so that rate-limiting is applied.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I32fd70cdf6422222ab0a8f599aa60bc2f6da229e
Reviewed-on: https://review.whamcloud.com/33513
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11575 build: install systemd stuff only for debian with systemd enabled

Add a precheck for systemd, to avoid to try to package systemd
stuff intolustre-client/server-utils deb when building on debian
series without systemd supported.

Test-Parameters: clientdistro=ubuntu1604 trivial

Change-Id: If58b64acc035e621594ab420a8b900b18a34a211
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/33492
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11504 tests: trigger lfsck after/during failover/failback

Patch adds the possibility to trigger lfsck
during and/or after failover/failback tests on
specified node with spectified device if set and
with lfsck custom parameters if required.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6123,LUS-6591
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Change-Id: I4bdafdb5e6840f13e23a41abcda55de78db9d8b4
Reviewed-on: https://review.whamcloud.com/33351
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11503 tests: fix "division by 0" defect

if stop file is created before ha_repeat_mpi_load() or
ha_repeat_nonmpi_load() are started nr_loops is equal to 0
and test fails with
   (1476438773 - start_time) / nr_loops: division by 0
Patch fixes this case.

Patch correctly processes ha_dump_logs()->ha_on() rc
to avoid the test failure when nodes are not accessible for
logs dumping.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-2205
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I97957875685492ddec66daa7c353ff6ef930933c
Reviewed-on: https://review.whamcloud.com/33350
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11398 ptlrpc: clean up rq_interpret_reply callbacks

Clean up the function prototypes of several rq_interpret_reply
callback functions to match the function pointer type instead
of using typecasting to avoid the risk of bad function pointers.

Clean up related code to match code style.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2df73304a8884c1cc838b9c9344d3fd9443ebbe5
Reviewed-on: https://review.whamcloud.com/33203
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 mdd: migrate from proc to sysfs

Move the ofd module from using proc for most single value files
to sysfs. The more complex proc entries are moved to debugfs.

Change-Id: I01eebf1c58f1a13c2f5e8c599a1363c80468b0bd
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33632
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11078 tests: customise run_ior()

Sometimes it is required to run ior with parameters
missing in run_ior() cmd, custom mpi threads per
cient and custom striping.

Now these parameters can be specified by:
  ior_custom_params
  mpi_ior_custom_threads
  ior_stripe_params

Test-Parameters:trivial testlist=parallel-scale
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I785d7aece107b2b53d0c422fbaec5c469b5bbc97
Reviewed-on: https://review.whamcloud.com/32688
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10862 ldlm: remove trace from ldlm_pool_{count,skrink}()

The trace in ldlm_pool_{count,shrink}() is too noisy given its
information value so remove it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iefed8000368c66b678aeae203e3afed06faa75c4
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/31820
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-8066 obdclass: report all obd states for OBD_IOC_GETDEVICE

The wrong state '--' which is reported when the obd device is
inactive. Reporting the "IN" state cover all the information that
is provided by 'devices' debugfs file. Now all the information
from 'devices' can be collected from the lustre sysfs tree.

Change-Id: I2479b15578babeb6f45da8129ffc0de99044730a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33774
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10171 headers: define pct(a,b) once

pct is defined 6 times in different places.  Define it in one.
Also change it to a static inline to do a better job of
enforcing types.

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: If61132a1096c351a9bcb7debb868351206267535
Reviewed-on: https://review.whamcloud.com/29852
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-6142 llite: Fix style issues for symlink.c

This patch fixes issues reported by checkpatch
for file lustre/llite/symlink.c

Change-Id: I053c5fe8ca1354e49fc8f1c4d54ba620253e6888
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33811
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for lcommon_misc.c

This patch fixes issues reported by checkpatch
for file lustre/llite/lcommon_misc.c

Change-Id: I06410af949183bc054444b8189fc39aecc48a4ce
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33810
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for llite_nfs.c

This patch fixes issues reported by checkpatch
for file lustre/llite/llite_nfs.c

Change-Id: I6457af5be663985f2e978115d1a1986a4ee15b08
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33809
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for glimpse.c

This patch fixes issues reported by checkpatch for file
lustre/llite/glimpse.c

Change-Id: Idf5b811fd5def8976d97e928b451d0b0956ea9af
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33808
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>