LU-8955 nodemap: add SELinux policy info to nodemap

Give the ability to set SELinux policy information on a nodemap,
in a new nodemap field named 'sepol'.
When set, a client pertaining to this nodemap will be allowed to
connect only if the SELinux policy information it sends matches
the one stored in the nodemap.

Expected 'sepol' string format is:
<1-digit>:<policy name>:<policy version>:<policy hash>
1-digit is 0 for SELinux Permissive mode, 1 for Enforcing mode.

SELinux policy info of nodemap is stored permanently by using
'lctl set_param -P' commands. It means MDS and OSS will be able to
retrieve SELinux policy info of nodemap after a restart (when they
read params llog).
MGS will not see SElinux policy info after restart, but this does
not prevent the feature from working.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia1d178dd34f05ede020b490b1797a71dbae15d7b
Reviewed-on: https://review.whamcloud.com/24420
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1365 tests: LARGEDIR and 3-level htree for local ldiskfs

Add conf-sanity.sh test for the LARGEDIR feature with Lustre + ldiskfs
to exceed both 2-level htree (with 1024-byte blocksize to reduce the
number of entries per htree level) and 2GB directory size (marked SLOW
since this takes a while to run). This is done using a smaller number
of hard-linked inodes (nlink_max = 65000), to avoid the overhead of
accessing and caching a large number of different inodes.

Running multiple threads on a single directory is serialized by the
VFS directory mutex, so the tests are run in parallel on all available
clients and mountpoints to speed up the test as much as possible.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: If52ef346a39a1c3236871edbb0327bf44fe1a392
Test-Parameters: trivial testlist=conf-sanity envdefinitions=SLOW=yes,FLAKEY=false
Reviewed-on: https://review.whamcloud.com/22009
Tested-by: Jenkins
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-1365 utils: allow set block size for ldiskfs backend

Add “-b” option to mkfs.lustre that allows to set backend block size.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: I83fc76f64ce2a0b4bf500841b695d64d3dea78de
Reviewed-on: https://review.whamcloud.com/33757
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

LU-11790 ldiskfs: add terminating u32 when expanding inodes

In ext4_expand_extra_isize_ea(), we calculate the total size of the
xattr header, plus the xattr entries so we know how much of the
beginning part of the xattrs to move when expanding the inode extra
size.  We need to include the terminating u32 at the end of the xattr
entries, or else if there is uninitialized, non-zero bytes after the
xattr entries and before the xattr values, the list of xattr entries
won't be properly terminated.

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I247b935b3cf315481dc4658133a7eee02b6350e9
Reviewed-on: https://review.whamcloud.com/33893
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>

LU-11889 tests: disable sanity/156 for ZFS OST

disable sanity/156 with ZFS OST as osd-zfs doesn't
maintain cache hit/miss stats yet.

Test-Parameters: trivial

Change-Id: I34eac3dcc49962ead0c0d6b17d8d605036872f4f
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34114
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11607 tests: create routine to get Lustre env

The Lustre tests in the test suites make repeated calls
to a small number functions that relate to the Lustre
environment. Some examples are the version of the Lustre
server or client and the file system type of the server.

Collect these calls into a routine called get_lustre_env()
in the test-framework.sh library and replace calls in
sanity with the global variables.

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I01dd00dd50cca39c964c5fd8abc3f51ab3c8e6b8
Reviewed-on: https://review.whamcloud.com/33938
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9859 libcfs: add watchdog for ptlrpc service threads.

If a ptlrpc thread takes longer than expected, it
can be used to get a stack trace to see where the
problem is.

This patch moves this functionality out of libcfs
by using a delayed work struct to report a back
trace of the thread if it ever fires.

Change-Id: I8f54df430322f0751b179f67284018e576defc08
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33018
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8939 nodemap: remove deprecated lproc files

Before the MGS was able to transfer the nodemap configuration to the
MDSes and OSSes, people had to use lproc interfaces in order to load
the configuration. This patch removes those deprecated interfaces.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I8c8f38755cf2870fbabca9e96d7d51a9ac290193
Reviewed-on: https://review.whamcloud.com/24352
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7239 mdd: make mdd_attr_set() synchronous less often

In permission_needs_sync(), test mode changes for S_ISUID or S_ISVTX
rather than simply checking whether the new mode in the incoming
request has those bits set.  Similarly check for uid and gid changes.

Prior to these changes, copying a directory tree into lustre, whose
directories' modes have S_ISUID | S_ISGID | S_ISVTX set, is
unnecessarily slow because chmod() system calls are synchronous
whether they change the actual permissions or not.

Change-Id: I41d3297a007eb4b796790b3b3de0ad355d17cb58
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/16699
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7159 tests: fix 224c fault injection

Sanity 224c test always passes because no fault injection happened.

This Patch adds additional step in fault injection and fix this test.

Test-Parameters: clientarch=aarch64 testlist=node-provisioning-1,lustre-initialization-1,lnet-selftest,sanity

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@seagate.com>
Xyratex-bug-id: MRP-2472
Change-Id: I5f637e98b46ddda6acbafb956f9b85fccb7cae15
Reviewed-on: https://review.whamcloud.com/16426
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>

LU-11852 osc: skip cache entry wakeup without grants

No need to wakeup osc_entry_cache waiters with EDQUOT or insufficent
grants, cause they will sleep again.
osc_enter_cache could skip queue of waiters if client has enough
grants. The fix adds list_empty checking and improves FIFO.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6119
Change-Id: I0a2415a19b2c8ecd7df3411d88cab9fee2470a10
Reviewed-on: https://review.whamcloud.com/34013
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11845 zfs: put configure checks in version order

Put the ZFS feature checks in release version order, so that it is
easier to track when they apply and when they can be removed in
the future.

Make the configure checks use decrypt=B_TRUE just to illustrate
more correct usage.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I22053638d72b41b51b6f56dea5668e78535cab07
Reviewed-on: https://review.whamcloud.com/34000
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11716 tests: fix scrub tests

scrub_prep set MDT failloc to 0x198. It prevents the inserting
of osd_obj to OI, and brokes the logic of osd_fid_lookup. The
creating and searching quota slv file could happen during this
failloc. osd_fid_lookup() returned osd object with oo_inode equal
to zero and a bit later, osd_index_try got BUG cause it tried to
access to &LDISKFS_I(inode)->i_es_lock.

The patch adds OST object creation for quota synchronization before
setting failloc.

Test-Parameters: trivial testlist=sanity-scrub
Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6051
Change-Id: I8a37c476c2c95b235b0f0ecc7ec1ab0066ae3883
Reviewed-on: https://review.whamcloud.com/33750
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11625 ofd: handle upgraded filter_fid properly

Since there have been several iterations of struct filter_fid stored
on disk, the current code wasn't checking for all of the possible
cases when trying to decide what action to take when accessing and
upgrading the xattr for new capabilities.

Properly check for the various different struct filter_fid sizes and
handle them appropriately.  Add a more verbose description of the
various cases so that this is more clear to others in the future.

Add decoding of filter_fid fields added for FLR in 2.11.

We should already be testing for upgrading the filter_fid xattr
from different OST versions in conf-sanity test_32d.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifef2292296236cb06ff7e8cd50caff4b133ebbe5
Reviewed-on: https://review.whamcloud.com/33627
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11555 utils: ZFS check multihost enabled in read_ldd()

For ZFS check that multihost is enabled if failover host is defined.
Print a warning if it's not.

Test-Parameters: trivial mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: Iddb5871afc6fb6808a25921c8d3e8516d675f15c
Reviewed-on: https://review.whamcloud.com/33491
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10953 tests: disable sanity/300k on small setups

sanity/300k simulates creation of 512-stripes directory and
fails on small setup causing all subsequent tests to fail as
well.

Change-Id: I5e1d8d25ea364c0f49d9e114cc447cc40f951e2f
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/32148
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 lnet: change lnet_ipaddr_enumerate() to use for_each_netdev()

for_each_netdev() is a more direct interface than
lnet_ipif_enumerate(), so use it instead.  Also get
address and 'up' status directly from the device.

This means we need to possible re-allocate the storage
space if there are lots of IP addresses.

However there is no need to resize the allocation down if we
over-allocated.  This is only used once, and is freed soon
after it is allocated, so that is a false optimization.

Linux-commit: 0400cf406c32ac3968241cd528747d922b6c55c3

Change-Id: I1c1e7722c7b2b267dcb8134ae295a54f976d96ad
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/33969
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 kernel: harden current_time autoconf test

In newer kernels CURRENT_TIME was replaced by current_time(). The
return value of current_time() was struct timespec but to support
time after 2038 the return value was changed to struct timespec64.
This change broke the autoconf test. The solution is to use one
of the struct iattr field in the autoconf test since it hides
the return value type.

Test-Parameters: trivial

Change-Id: I95abd2cd2b777f99cbf6ab78370ee2171e5fca67
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33963
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-9325 msg: replace obsolete simple_strtoul use

The kernel function simple_strtoul() has been superseded by the
kstrto* family of functions. Replace all uses of simple_strto*()
with the equivalent kstrto*() function in the msg subsystem.

Change-Id: I5ca5df29b8b2f34107ee40628ab3b77d57547b99
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33959
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10447 tests: replace $SET/$GETSTRIPE in replay tests

$SETSTRIPE and $GETSTRIPE were needed when we used the
standalone 'lstripe' utility. 'lstripe' hasn't been used
for years and we need to clean up all remnants of it.

Replace all instances of $SETSTRIPE with '$LFS setstripe'
and $GETSTRIPE with '$LFS getstripe' in the replay-dual,
replay-ost-single, and replay-single test suites.

Test-Parameters: trivial testlist=replay-dual,replay-ost-single,replay-single
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I3902142061ddb3c8860b6983d5ec55383aeb25a9
Reviewed-on: https://review.whamcloud.com/33918
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10966 utils: Fix `lfs check` documentation and arguments

Several bugs exist for 'lfs check'. Add checking for MGS servers.
Add 'lfs check mdts' which is inline with the other options but
still keep mds for backwards compatibility. Do the same with the
'all' option which is equivalent to older 'servers' that is also
kept for backwards compatibility. Update man pages to reflect
these changes

Change-Id: Ifffec8fd1498616036122f9af311cd6366d693bc
Signed-off-by: George Melikov <mail@gmelikov.ru>
Reviewed-on: https://review.whamcloud.com/33775
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11418 mdd: delete name if orphan doesn't exist

mdd_orphan_destroy() should delete name if orphan object doesn't
exist, otherwise the orphan clean thread will try to destroy this
orphan in dead loop.

add sanity test_811.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id22b2fab0ac87dfb81ca9f01d8ed0338f1b12120
Reviewed-on: https://review.whamcloud.com/33661
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11402 tests: add version check sanity-quota 60

sanity-quota test 60 was added to Lustre tag 2.11.53. Thus,
we need to check that the server is 2.11.53 or later before
running test 60.

Test-Parameters: trivial serverjob=lustre-b2_10 serverbuildno=152 testlist=sanity-quota
Test-Parameters: testlist=sanity-quota
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I5db738f0776b5615165977df1708f197f215b994
Reviewed-on: https://review.whamcloud.com/33418
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11843 tests: customise sanity-quota

Patch adds the possibility to not turn project quotas OFF
at the end of sanity-quota if it is ON initially.
ENABLE_PROJECT_QUOTA=false is ignored in this case.

Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6162
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Test-Parameters: trivial testlist=sanity-quota
Change-Id: Ic7d58f4a077acdbeec426613777e3c7c0778f0a0
Reviewed-on: https://review.whamcloud.com/33997
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11838 lustre: discard LTIME_S macro

Rather than using a macro, just access the required
field directly.

Linux-commit: 5b7cc4e4ce3cc15f67462ae75c55eecc7edc3a40

Change-Id: I325cac7458265d1cf6ad9f195a513f2612865906
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/33984
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-11834 llite: fix temporary instance buffer size

The formatting of the cfg_instance variable was changed in LU-11809
to always use a fixed "%016llu" format, but the temporary buffer
allocations for the instance string were not changed to match the
printed value.

This results in string truncation in some situations.  Change the temp
buffer size to always have 16 bytes for the instance instead of using
sizeof(cfg_instance).

Fixes: cd294a1255 ("LU-11809 llite: don't use %p for cfg_instance")

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5eca99afa2787cc57e739489b252b12af68cab07
Reviewed-on: https://review.whamcloud.com/33951
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11825 clio: Remove pio feature

The pio feature was an interesting concept, but never
achieved real use in production, primarily because it was
actually slower than normal i/o for some common workloads,
so it could not be on by default.

It has had a trivial deadlock in it (with truncate) since
release, and this has never been reported by a customer.

The attemp to rewrite readahead to use ptasks was never
fully successful, and is being superseded by a different
implementation.

Given all this, there is no reason to keep the extra
complexity in the already complicated clio code.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ie558db946af94189e35e985e8894706c14ed5f39
Reviewed-on: https://review.whamcloud.com/33912
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11810 misc: allow Fixes: tag in commit signoff block

Allow the "Fixes:" tag in the signoff block of the commit message.
The Fixes: tag should contain a valid git commit hash, and may
optionally be followed by a description of the original patch.
For example:

Fixes: 5760b34c48b ("LU-1145 test: add Test-Parameters tag")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I49f712dd8da173510e5941b66eb050e53a1cab07
Reviewed-on: https://review.whamcloud.com/33888
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11683 build: Update ZFS to 0.7.12

0.7.12

ZFS
 * Add BuildRequires gcc, make, elfutils-libelf-devel #8095
 * Fix flake8 "invalid escape sequence 'x'" warning #8056
 * ZTS: Update O_TMPFILE support check #7528
 * Allow use of pool GUID as root pool #8052
 * Fix arc_release() refcount #8000
 * Prefix all refcount functions with zfs_ #7963
 * Linux 4.19-rc3+ compat: Remove refcount_t compat #7885
 * Zpool iostat: remove latency/queue scaling #7945
 * Fix statfs(2) for 32-bit user space #7122
 * Skip import activity test in more zdb code paths #7797
 * Reduce taskq and context-switch cost of zio pipe #7736
 * Fix race in dnode_check_slots_free() #7147

SPL
 * Add BuildRequires gcc, make, elfutils-libelf-devel #8095
 * Linux 4.20 compat: current_kernel_time() #8074
 * Backport vnode.h changes from zfs:93ce2b4c
 * Define timestruc_t for Lustre compatibility #8014

0.7.11

ZFS
 * tx_waited -> tx_dirty_delayed in trace_dmu.h #7096
 * Revert "zpool reopen should detect expanded devices"

SPL
 * Linux 4.18 compat: Kconfig #7890)

0.7.10

ZFS
 * Correctly handle errors from kern_path #7764
 * Fix build with CONFIG_GCC_PLUGIN_RANDSTRUCT {{f4ee628}}
 * Fix 'zfs recv' of non large_dnode send streams #7617
 * Fix object reclaim when using large dnodes #7097
 * Fix problems receiving reallocated dnodes #6821
 * Fedora 28: Fix misc bounds check compiler warnings #7826
 * Fix libaio-devel requirement for Debian-based distributions #7829
 * Add libaio-devel BuildRequires #7821
 * Add missing zfs-dracut RPM dependencies #7729
 * Don't modify argv] in user tools #7760
 * Fix arcstat.py handling of unsupported options #7799
 * Allow inherited properties in zfs_check_settable() #7755
 * Fix zfs incremental send remove '-o' properties #7478
 * OpenZFS 8906 - uts: illumos rootfs should support salted cksum
 * Fix zpl_mount() deadlock #7598
 * Fix kernel unaligned access on sparc64 #7642
 * Default ashift for Amazon EC2 NVMe devices #7676
 * Linux 4.14 compat: blk_queue_stackable() #7645
 * Add pool state /proc entry, "SUSPENDED" pools #7331
 * zpool reopen should detect expanded devices #7546
 * Support Debian DKMS builds #7358
 * module param callbacks check for initialized spa #7496
 * Trim new line from zfs_vdev_scheduler #3356
 * Fix ENOSPC in "Handle zap_add() failures in ..." #7401
 * Fix divide-by-zero in mmp_delay_update() #7391
 * OpenZFS 8997 - ztest assertion failure in zil_lwb_write_issue #7084
 * Linux compat 4.18: check_disk_size_change() #7611
 * Linux 4.18 compat: inode timespec -> timespec64 #7643
 * zv_suspend_lock in zvol_open()/zvol_release() #6342

SPL
 * Add rwsem_tryupgrade for 4.9.20-rt16 kernel (SPL) #7589
 * Add pool state /proc entry, "SUSPENDED" pools (SPL) #7331
 * Linux 4.18 compat: inode timespec -> timespec64 #7643

Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I7b108d659d9d6cd6df3d4c55d8b8380f68ccaa6e
Reviewed-on: https://review.whamcloud.com/33687
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11661 test: improve sanityn test_47g

'stat' may be run before 'mkdir', to avoid this, sync data before
test and wait longer time after 'mkdir' in background.

Test-Parameters: trivial testlist=sanityn,sanityn,sanityn,sanityn mdtfilesystemtype=zfs mdscount=2 mdtcount=4 envdefinitions=ONLY=47g
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I314bc9d36629a5185efc5ef8281a03337ea77776
Reviewed-on: https://review.whamcloud.com/33647
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11579 llite: remove cl_file_inode_init() LASSERT

If there is some corruption or other reason that the file layout
cannot be used, the first call to cl_file_inode_init() will fail.
If it is called a second time on the same file then it will hit
an LASSERT() since I_NEW is no longer set on the inode.

It would be good to handle the error in lov_init_raid0() better,
but we still want to avoid this LASSERT() if there is an error.

Convert the LASSERT() in cl_file_inode_init() into a CERROR() and
error return.  This is being triggered due to corruption on the
server, but that shouldn't cause the client to assert.

    lov_dump_lmm_common() oid 0xdf4e:311367, magic 0x0bd10bd0
    lov_dump_lmm_common() stripe_size 1048576, stripe_count 4
    lov_dump_lmm_objects() stripe 0 idx 10 subobj 0x0:151194471
    lov_dump_lmm_objects() stripe 1 idx 12 subobj 0x0:152477530
    lov_dump_lmm_objects() stripe 2 idx 25 subobj 0x0:151589797
    lov_dump_lmm_objects() stripe 3 idx 2 subobj 0x0:150332564
    lov_init_raid0() fsname-clilov: OST0019 is not initialized
    cl_file_inode_init() Failure to initialize cl object
        [0x20004c047:0xdf4e:0x0]: -5

    cl_file_inode_init() ASSERTION(inode->i_state & (1 << 3) ) failed
    cl_file_inode_init() LBUG
    Pid: 37233, comm: ll_sa_4709 3.10.0-862.14.4.el7.x86_64 #1 SMP
    Call Trace:
    libcfs_call_trace+0x8c/0xc0 [libcfs]
    lbug_with_loc+0x4c/0xa0 [libcfs]
    cl_file_inode_init+0x2ac/0x300 [lustre]
    ll_update_inode+0x315/0x600 [lustre]
    ll_iget+0x163/0x350 [lustre]
    ll_prep_inode+0x232/0xc80 [lustre]
    sa_handle_callback+0x3a4/0xf70 [lustre]
    ll_statahead_thread+0x40e/0x2080 [lustre]

Instead, return an IO error instead of killing the client.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If8a6eb24df09e7e158b61f02e2517132893ebbe5
Reviewed-on: https://review.whamcloud.com/33505
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11496 tests: start_client_loads () fix

recovery-*-scale tests are not functional with LOADS=IOR
because of MPIRUN and MPIRUN_OPTIONS are not set.
Patch fixes this defect.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6220
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I9b6d3810978260c0cec4a5482f093f788651aac0
Reviewed-on: https://review.whamcloud.com/33339
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10447 tests: deprecate $SETSTRIPE/$GETSTRIPE in sanity

$SETSTRIPE and $GETSTRIPE were needed when we used the
standalone 'lstripe' utility. 'lstripe' hasn't been used
for years and we need to clean up all remnants of it.

Replace all instances of $SETSTRIPE with '$LFS setstripe'
and $GETSTRIPE with '$LFS getstripe' in the sanity and
sanityn test suites.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I492d261c7568634c8f0dd0d5a100ef5bb8433ad8
Reviewed-on: https://review.whamcloud.com/32809
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8955 cfg: reserve flags for SELinux status checking

Reserve LCFG_NODEMAP_SET_SEPOL config flag that will be used to
define sepol parameter on nodemap entries.
Reserve OBD_CONNECT2_SELINUX_POLICY connection flag that will be set
(in ocd_connect_flags2) if a client supports sending the SELinux
policy status info.
Add checks for all lcfg_command_type constants, along with lustre_cfg
and cfg_record_type.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I009eb8f1597b51acce7dc21498a43096e41cd034
Reviewed-on: https://review.whamcloud.com/33797
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>

LU-10937 sptlrpc: split sptlrpc_process_config()

Make sptlrpc_process_config() more than a single line wapper
exporting function. Instead migrate the lcfg parsing out of
__sptlrpc_process_config() so that we can use this function
for both LCFG_PARAM and LCFG_SET_PARAM handling.

The first field parsed from struct lustre_cfg *lcfg is the
target. This can be "_mgs", file system name, or an obd target
i.e fsname-MDT0000. We can move to extracting the file system
name out of the target string using server_name2fsname().

Test-Parameters: testlist=sanity-sec envdefinitions=SHARED_KEY=true

Change-Id: If94ee8164ab2b49fe10d098b32faea9e331f04ac
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33760
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11647 ptlrpc: always unregister bulk

In ptlrpc_check_set, the bulk should be unregistered before
ptl_send_rpc in any case.

Change-Id: Icf963002f934b43ccbb9d6ef02ba7f9d11f297f8
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/22378
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7558 ptlrpc: connect vs import invalidate race

Connect can't be sent while import invalidate is
in progress, thus it leaves the import in not
initialized state.

Don't allow reconnect in evicted state.

Change-Id: I79a1a1eb05fede30e100ba09b6f3f98636a46213
Cray-bug-id: LUS-6322
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33718
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-7631 tests: add debug info to conf-sanity 82a

In the routine check_stripe_count, the different error
messages need to be modified so when an error occurs,
a user can tell what error was hit. Also, print precreated
object information at the beginning of the test and on
error.

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=conf-sanity envdefinitions=ONLY=82a
Test-Parameters: mdscount=1 mdtcount=1 testlist=conf-sanity envdefinitions=ONLY=82a
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ifc75d52d38d9cb401118ef7baa4014bddf6298f2
Reviewed-on: https://review.whamcloud.com/33689
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11723 kernel: kernel update RHEL7.6 [3.10.0-957.1.3.el7]

Update RHEL7.6 kernel to 3.10.0-957.1.3.el7.

Test-Parameters: clientdistro=el7.6 ossdistro=el7.6 mdsdistro=el7.6

Change-Id: I9e2679104b226d2ab334b7f870ccdd495b2345f9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33763
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9010 tests: Change static defines to use macro for kinode.c

This patch replaces completion which is defined statically in
file lustre/tests/kernel/kinode.c with kernel provided macro.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I7b41d3f0a67f471fe7f5dfa840521dc690a163d2
Reviewed-on: https://review.whamcloud.com/33935
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9010 ptlrpc: Change static defines to use macro for gss_svc_upcall.c

This patch replaces spinlock which are defined statically
in file lustre/ptlrpc/gss/gss_svc_upcall.c with kernel
provided macro.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I9978fcf50763df1b908c41377eef89ce284cf835
Reviewed-on: https://review.whamcloud.com/33934
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10866 pacemaker: Support additional mount options

Add support for additional mount options for pacemaker resources.
Logging is also added for the output of the mount and umount commands.

Signed-off-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Change-Id: Ideca8e37b12015b9f455634c85295137e00b97fd
Reviewed-on: https://review.whamcloud.com/31836
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Joe Grund <jgrund@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11737 lfsck: do not ignore dryrun

lfsck_layout_recreate_lovea() shouldn't ignore dryrun.

Change-Id: Ia8bafc13f148b03573dee5db26b6aff9386b5b5f
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33826
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>

LU-10815 tests: all_mdts_nodes() all_osts_nodes() defect

When mds_HOST and ost_HOST are not set (mds1_HOST,
ost1_HOST, ost2_HOST, etc are used), all_mdts_nodes() and
all_osts_nodes() have client in the list set due to cfg/local.sh:
  mds_HOST=${mds_HOST:-$(hostname)}
  ost_HOST=${ost_HOST:-$(hostname)}
With this change these functions include mds_HOST and ost_HOST
into the list only if mdsN_HOST and ostN_HOST are not set.

Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-5784
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I1ccfc94a186db98fbf87698cb8fc7d08ee008fdc
Reviewed-on: https://review.whamcloud.com/33342
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11755 ldlm: remove LASSERT() in ns_is_{client,server}()

(ns->ns_client == LDLM_NAMESPACE_CLIENT ||
ns->ns_client == LDLM_NAMESPACE_SERVER) implies that
(!(ns->ns_client & ~(LDLM_NAMESPACE_CLIENT |LDLM_NAMESPACE_SERVER)))
is also true, so the latter LASSERT() can be removed from
ns_is_{client,server}().

Signed-off-by: Nikitas Angelinas <nangelinas@cray.com>
Cray-bug-id: LUS-6762
Change-Id: Ib165d97ca397d5109c046415d2ec513357b07cbd
Reviewed-on: https://review.whamcloud.com/33822
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11820 tests: skip conf-sanity test_106 for SLOW=no

This test currently takes 40 minutes to run, so it should be
skipped for SLOW=no test runs.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I120747e21d046bd28667e2ff26013ae51f9cab07
Reviewed-on: https://review.whamcloud.com/33902
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11606 utils: Call llapi_error() for errors in liblustreapi files

Change calls to remaining printf() and fprintf() to use
llapi_printf() and llapi_error() so that it is possible
to override these calls when liblustreapi is linked into
some other application.

Change-Id: I6531d29b05b34766a68dabf17299cd00babe2517
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33877
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-8066 lnet: properly isolate kernel_param_ops

Don't reference kernel_param_ops if not available.  There is
already a HAVE_KERNEL_PARAM_OPS configure check for this, but
it is just misplaced.

Fixes: 7092309f32516cbfb95a964c87b8030129edeb27

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I56f86257c6bc8a9c53c7901bc2765e10587cab07
Reviewed-on: https://review.whamcloud.com/33874
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11771 osd: avoid use of HZ in brw_stats

Don't use HZ when computing the brw_stats IO time.  This brw_stats
field was formerly in jiffies, but since the conversion to ktime_t
(now always in msec) the output is incorrect when HZ != 1000.

Fixes: 4c2f028a95063aa30006a071a039e8a3eefe52f2

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia01705b94ddedec2e375621c6e01a957f53ebbe5
Reviewed-on: https://review.whamcloud.com/33857
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11757 lod: use calculated stripe count

lod_prep_md_striped_create() tries to allocat big
chunk of memory because
lum->lum_stripe_count == -1 and is converted to __u32.

ldo_dir_stripe_count was calculated already in lod_ah_init()

Change-Id: Id99d9e024638dfb1b34262840d2e543c808a9cdc
Cray-bug-id: LUS-6694
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-on: https://review.whamcloud.com/33829
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11736 utils: don't set max_sectors_kb on MDT/MGT

The max_sectors_kb tunable should not be applied to MDT and MGT
devices. This tuning is needed for efficiency of large IOs for
spinning disks, but is not needed for SSDs or regular IO. It can
cause problems with DM Multipath configurations for minimal
benefits, so should be limited to OST devices.

This only applies to ldiskfs backend filesystems, no such tuning
is currently done for any ZFS devices.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I496603da13aae042f63cc37c0dea221a393ebbe5
Reviewed-on: https://review.whamcloud.com/33796
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 tests: Fix replay-single test_85b

test_85b of replay single sets a default striping on $DIR
and does not remove it.  This makes it impossible to
correctly test self-extending layouts, so fix this first.

cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I0057c8403e3dae2437cf0c8810af8086e2971c35
Reviewed-on: https://review.whamcloud.com/33777
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 sysfs: make ping sysfs file read and writable

Starting with 4.15 kernels any sysfs read only is limited to
root access only. To retain the ability for non root users
to detect if a remote server is alive using the 'ping' sysfs
file we need to change it to writable. Retain the read ability
so older tools will work.

Change-Id: I6560c119328d723a20a2b32e1fa8c68dce5d407a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33776
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-11719 ldlm: Adjust search_* functions

The search_itree and search_queue functions should both
return either a pointer to a found lock or NULL.

Currently, search_itree just returns the contents of
data->lmd_lock, whether or not a lock was found.

search_queue will do the same under certain cirumstances.

Zero lmd_lock in both search_* functions, and also stop
searching in search_itree once a lock is found.

cray-bug-id: LUS-6783
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ie231166756e60c228370f8f1a019ccfe14dfda6a
Reviewed-on: https://review.whamcloud.com/33754
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11717 tests: exit_status () defect

exist_status() should search a word FAIL, otherwise
all the tests skipped due to reason which contains
FAILOVER_MODE have status = 1 and exited with the error.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6711
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I5a69e0a4aed6cc8d75d97fe83c6f304b666701ea
Reviewed-on: https://review.whamcloud.com/33751
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11415 socklnd: improve scheduling algorithm

Modified the scheduling algorithm to use all scheduler threads
available. Previously a connection is assigned a single thread
and can only use that one. With this patch any scheduler thread
available on the assigned CPT can pick up and work on requests
queued on the connection.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I7e8e5220e1914155fd793994485e3a443fe1ba44
Reviewed-on: https://review.whamcloud.com/33740
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11700 osp: GPF in osp_reserved_mb_low_seq_write()

opd_pre pointer may be uninitialized on statrup

Change-Id: If6ff4289545503e07719c7e9a282be1bd393465b
Cray-bug-id: LUS-6118
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33717
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11699 lfsck: Umount while running LFSCK

lod_device_free() ASSERTION( atomic_read(&lu->ld_ref) == 0)failed

Wake up master thread after releasing all refernces.

Change-Id: I2933f7c58c12f4e3cd2510b01babbf02814c8768
Cray-bug-id: LUS-6510
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33715
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11696 utils: "lfs getsom" returns "24" to userspace

The "lfs getsom" command always returns "24" to userspace because
"rc = 24" (sizeof(struct lustre_som_attrs)) after fetching the
xattr from the kernel.
In this patch, rc is set to 0 if the lfs_getsom()->lgetxattr()
call returns a positive value.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ie3151f67b5ce2b5b2bc35a4b6528ba9a20a5db9f
Reviewed-on: https://review.whamcloud.com/33714
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11689 lfs: make sure project proceed all dirs

Leftover fix since LU-10986 lfs: make lfs project tolerant errors
We should proceed other dirs if we hit errors, otherwise,
some dirtree like following will fail if aaaa not exists.

testdir/
├── subdir
│   └── 1
├── bbbb -> aaaa
└── cccc

Also remove extra error output since we have output failing
messages inside every action function.

Test-Parameters: trivial testlist=sanity-quota,sanity-quota
Change-Id: I0062dbc3f4d1925c9e9e1a509ee35ac569bd9b74
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33707
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11669 tests: add project in yml_test_group()

This patch fixes yml_test_group() in yaml.sh to
add test project name, which is required in
results.yml for Maloo to parse.

Test-Parameters: trivial

Change-Id: I0ae563d855dc2d28eaea85e86b1cb23d2428988b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33658
Tested-by: Jenkins
Reviewed-by: Leonel Ochoa <lochoa@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11658 lov: cl_cache could miss initialize

The cl_cache may be missed initialize when we mount
a client with deactivate osc and then active it.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I92cd44375d70624fb55ef7a0218e7178211a8687
Reviewed-on: https://review.whamcloud.com/33650
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11620 lfsck: change llsd_rb_lock to rwsemaphore

llsd_rb_lock is taken in ->init, and released in ->fini, but during
this period it may getxattr which can sleep. Change it to rwsemaphore.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Idc68eb886e60dc45ccfb7ac9bf5bf06db42d690d
Reviewed-on: https://review.whamcloud.com/33603
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11579 lov: quiet lov_dump_lmm_ console messages

Limit messages in lov_dump_lmm_objects() and lov_dump_lmm_common()
printing to the console repeatedly when D_ERROR is used.  Change
CDEBUG() to CDEBUG_LIMIT() so that rate-limiting is applied.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I32fd70cdf6422222ab0a8f599aa60bc2f6da229e
Reviewed-on: https://review.whamcloud.com/33513
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11575 build: install systemd stuff only for debian with systemd enabled

Add a precheck for systemd, to avoid to try to package systemd
stuff intolustre-client/server-utils deb when building on debian
series without systemd supported.

Test-Parameters: clientdistro=ubuntu1604 trivial

Change-Id: If58b64acc035e621594ab420a8b900b18a34a211
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/33492
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11504 tests: trigger lfsck after/during failover/failback

Patch adds the possibility to trigger lfsck
during and/or after failover/failback tests on
specified node with spectified device if set and
with lfsck custom parameters if required.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6123,LUS-6591
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Change-Id: I4bdafdb5e6840f13e23a41abcda55de78db9d8b4
Reviewed-on: https://review.whamcloud.com/33351
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11503 tests: fix "division by 0" defect

if stop file is created before ha_repeat_mpi_load() or
ha_repeat_nonmpi_load() are started nr_loops is equal to 0
and test fails with
   (1476438773 - start_time) / nr_loops: division by 0
Patch fixes this case.

Patch correctly processes ha_dump_logs()->ha_on() rc
to avoid the test failure when nodes are not accessible for
logs dumping.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-2205
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I97957875685492ddec66daa7c353ff6ef930933c
Reviewed-on: https://review.whamcloud.com/33350
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11398 ptlrpc: clean up rq_interpret_reply callbacks

Clean up the function prototypes of several rq_interpret_reply
callback functions to match the function pointer type instead
of using typecasting to avoid the risk of bad function pointers.

Clean up related code to match code style.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2df73304a8884c1cc838b9c9344d3fd9443ebbe5
Reviewed-on: https://review.whamcloud.com/33203
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 mdd: migrate from proc to sysfs

Move the ofd module from using proc for most single value files
to sysfs. The more complex proc entries are moved to debugfs.

Change-Id: I01eebf1c58f1a13c2f5e8c599a1363c80468b0bd
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33632
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11078 tests: customise run_ior()

Sometimes it is required to run ior with parameters
missing in run_ior() cmd, custom mpi threads per
cient and custom striping.

Now these parameters can be specified by:
  ior_custom_params
  mpi_ior_custom_threads
  ior_stripe_params

Test-Parameters:trivial testlist=parallel-scale
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I785d7aece107b2b53d0c422fbaec5c469b5bbc97
Reviewed-on: https://review.whamcloud.com/32688
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10862 ldlm: remove trace from ldlm_pool_{count,skrink}()

The trace in ldlm_pool_{count,shrink}() is too noisy given its
information value so remove it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iefed8000368c66b678aeae203e3afed06faa75c4
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/31820
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-8066 obdclass: report all obd states for OBD_IOC_GETDEVICE

The wrong state '--' which is reported when the obd device is
inactive. Reporting the "IN" state cover all the information that
is provided by 'devices' debugfs file. Now all the information
from 'devices' can be collected from the lustre sysfs tree.

Change-Id: I2479b15578babeb6f45da8129ffc0de99044730a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33774
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10171 headers: define pct(a,b) once

pct is defined 6 times in different places.  Define it in one.
Also change it to a static inline to do a better job of
enforcing types.

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: If61132a1096c351a9bcb7debb868351206267535
Reviewed-on: https://review.whamcloud.com/29852
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-6142 llite: Fix style issues for symlink.c

This patch fixes issues reported by checkpatch
for file lustre/llite/symlink.c

Change-Id: I053c5fe8ca1354e49fc8f1c4d54ba620253e6888
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33811
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for lcommon_misc.c

This patch fixes issues reported by checkpatch
for file lustre/llite/lcommon_misc.c

Change-Id: I06410af949183bc054444b8189fc39aecc48a4ce
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33810
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for llite_nfs.c

This patch fixes issues reported by checkpatch
for file lustre/llite/llite_nfs.c

Change-Id: I6457af5be663985f2e978115d1a1986a4ee15b08
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33809
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for glimpse.c

This patch fixes issues reported by checkpatch for file
lustre/llite/glimpse.c

Change-Id: Idf5b811fd5def8976d97e928b451d0b0956ea9af
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33808
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-11712 osd-ldiskfs: Wrap blk integrity with config check

Build is currently broken for kernels without
CONFIG_BLK_DEV_INTEGRITY. Build failure introduced by LU-11096
commit c8505c2e70d03ba20edf9fcbf431888e87a21147
https://review.whamcloud.com/#/c/32725/
Use of blk integrity should be wrapped in the config check for
HAVE_BLK_INTEGRITY_ENABLED

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Iac9e4a2572024c026132c87c11042cf353b14d48
Reviewed-on: https://review.whamcloud.com/33745
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 mgs: Fix style issues for mgs_fs.c

This patch fixes issues reported by checkpatch
for file lustre/mgs/mgs_fs.c

Change-Id: I2865df1a6663e288935f65353a408e3676eaa015
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33712
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 lod: Fix style issues for lod_dev.c

This patch fixes issues reported by checkpatch for file
lustre/lod/lod_dev.c

Change-Id: I72eaa79a12769e61889e567e5f28fdf3e8045c94
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33594
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 lmv: Fix style issues for lmv_fld.c

This patch fixes issues reported by checkpatch for file
lustre/lmv/lmv_fld.c

Change-Id: I98a5bdca61eff990370a7b2518b54e7f4137d120
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33566
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11537 osp: avoid nested transaction

Don't create and start new transaction in
osp_precreate_reserve (osp_sync_force)
because it has been already started in mdd_create.
New transaction rewrites oti_declare_ops_cred
resulting in assert in osd_trans_exec_op:

osd_trans_exec_op()) lustre-MDT0000: opcode 3: credits = 0, rollback = 4
osd_trans_exec_op()) ASSERTION( !ldiskfs_track_declares_assert ) failed:
...
 #2 [ffff88008983f600] panic at ffffffff816a863f
 #3 [ffff88008983f680] lbug_with_loc at ffffffffc0513854 [libcfs]
 #4 [ffff88008983f6a0] osd_create at ffffffffc0dfac32 [osd_ldiskfs]
 #5 [ffff88008983f718] lod_sub_create at ffffffffc101b585 [lod]
 #6 [ffff88008983f7c0] lod_create at ffffffffc100d6e9 [lod]
 #7 [ffff88008983f800] mdd_create_object_internal at ffffffffc0ed8888 [mdd]
 #8 [ffff88008983f838] mdd_create_object at ffffffffc0ec3e05 [mdd]
 #9 [ffff88008983f8b0] mdd_create at ffffffffc0ecc673 [mdd]

Change-Id: Ic2c4589a9a1f640c7a0aa989fc62d81ca08f917f
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Cray-bug-id: LUS-6098
Reviewed-on: https://es-gerrit.dev.cray.com/153461
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/33391
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11296 osc: speed up page cache cleanup during blocking ASTs

While we are cleaning a write lock, we don't need to check if
page cache pages under this lock are covered by another lock.

If a client needs to give up its lock, cleaning gigabytes of
page cache can take quite a long time.

Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-6352
Change-Id: I576130216ed4de4e352ea697bddb5ff83046443a
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/33090
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10883 llog: fix backup file name comparison

Last 3 bytes of key are wrongly compared with ".bak" extention.
4 bytes need to be compared.

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Change-Id: I90c1af2e41569780bf81df3975603acdc828edd6
Test-Parameters: testlist=conf-sanity
Reviewed-on: https://review.whamcloud.com/31886
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>

LU-11453 misc: add compat for INIT_LIST_HEAD_RCU

Add a compat version of INIT_LIST_HEAD_RCU() if unavailable.

Fixes: 68bc3984975bb72f730d8a8ab7aa2d836e50abe5

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9c1296206e002b895bb6bdf55ddd0d8ec70cab07
Reviewed-on: https://review.whamcloud.com/33875
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11750 krb5: krb5int_derive_key has 'hash' extra parameter

From Kerberos 5 release 1.15, and introduction of support of
aes-sha2, krb5int_derive_key() groks an additional 'hash' parameter.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I7c6ea5ac2d6844371b254b7361d28c462afe5afa
Reviewed-on: https://review.whamcloud.com/33817
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-7004 mgs: remove using obdname2fsname() from mgs layer.

Further testing shows that with the complex of obd device names
as well as wild card handling makes covering every case in
obdname2fsname() very difficult. Instead of mining the file
system name out of obd device name take advantage of the fact
the MGS device will know all the file system names and test if
the file system name is present in the lctl set_param -P
cmd.

Fix up the PERM_CMD test in the test suite which was wrong.
Make config-sanity test 29 wait until the tunables are set
after mounting the client.

Test-Parameters: envdefinitions="PERM_CMD=$LCTL set_param -P"

Change-Id: Ie3f5c8e1e1ff5ec309a31ba598f7cdf2f19c791f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32784
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10239 osc: limit chunk number of write submit

Don't queue too many pages in an extent for a write RPC, we need
to take care of the chunk limit in write submit as well (refers to
LU-8135 for more details).

Change-Id: Idaca6f365a6f8bb467fea43c811f775b37f49818
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/30627
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10384 mgs: replace_nids large string and failover support

Replace_nids uses nids list as new UUID. UUID string
length is limited by 38 symbols. So new nids list need
to be less then 38 symbols.

With this patch first nid in list string representation
is used for UUID as this done for failover nids.

Replace nids finds records for given device and regenerates
lines that contain old nids. add_uuid and add_conn lines for
failover used to be deleted during replace_nids which breaks
failover configuration.

This patch adds failover support to replace_nids command.
For example:

lctl replace_nids lustre-MDT0000 nid1,nid2:nid3,nid4:nid5,nid6

nid3,nid4 - nids from first failover node
nid5,nid6 - nids from second failover node

Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Cray-bug-id: MRP-4505
Change-Id: I4e9a35e8fa8781909ecbaa74785700f4ca04cf92
Reviewed-on: https://review.whamcloud.com/30624
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New tag 2.12.50

Start of 2.13 development effort.

Change-Id: Id378f5ad264aca50d31d6fdf0e2364904b76a4e2
Signed-off-by: Oleg Drokin <green@whamcloud.com>

New release 2.12.0

Change-Id: Icc5da4f5d1d032982a144ee5d13f214b04389d76
Signed-off-by: Oleg Drokin <green@whamcloud.com>

New release candidate 2.12.0-RC4

Change-Id: I5aa92f8a4232f5293e3253223d9447d23b3b0337
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11809 llite: don't use %p to generate cfg_instance

In kernel 4.15 and later, using "%p" in a string format to
print a kernel pointer will result in the pointer being
hashed with a random value, and the high bytes will be masked
on 64-bit CPUs to prevent leaking kernel address-space info
to userspace to defeat ASLR.  In early boot, the "%p" pointer
may resolve to "        (ptrval)", if there is not enough
entropy in the system to generate a random hash value.

The superblock pointer is used on the client to uniquely
identify all of the OBD devices connected to it, and to
find the configuration llog that was used to mount the
filesystem, so that it can also be used at unmount time.
The sb pointer is also used in the OBD device names, and
the "        (ptrval)" expansion breaks /sys filenames,
and also breaks the uniqueness of the config instance.

On the server, there is also a pointer value used for the
FLDB SEQ servers of the OSTs.

For the short term, bypass the "%p" hashing, so that mount
continues to work properly, and this can be resolved in a
later patch to change ll_get_cfg_instance() to provide a
unique value that is not directly a kernel pointer.

In llapi_getname() don't depend on the cfg_instance being
exactly 16 characters long, if this changes in the future.

Test-Parameters: clientdistro=ubuntu1804 testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I166de0248af8fe57535628a64bb770a4e03ebbe5
Reviewed-on: https://review.whamcloud.com/33900
Tested-by: Jenkins
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11783 build: remove lustre_user.h deprecation warning

The "lustre/lustre_user.h" header has been in use for many years.
The patch https://review.whamcloud.com/25246 "LU-6401 uapi: migrate
remaining uapi headers to uapi directory" moved the header to
"linux/lustre/lustre_user.h" and left a stub "lustre/lustre_user.h"
behind that generates a compiler warning that this header is
deprecated.

However, no window was given between the introduction of the new
header and the deprecation of the old header, which makes it harder
for applications to smoothly transition to the new header location.
Also, installing Lustre headers into the "linux/" directory before
Lustre is actually part of the kernel may potentially cause problems.

Disable the deprecation warning in the old header for several
releases, until the new header location has been available for a good
time and it is safe for applications to assume that it is available.

Test-Parameters: trivial
Fixes: 6712478e79588e73e28c7ccac3afc7ac2368a4f3
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If5a62587e2d3627178a0f7a09c3a4c10801cab07
Reviewed-on: https://review.whamcloud.com/33872
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New release candidate 2.12.0-RC3

Change-Id: I841a274fd9ab3b35c8c3a5b93e33469a9e230af5
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11753 obdclass: lu_dirent record length missing '0'

In lu_dirent packing, a '0' is appended after name, but it's not
counted in size calcuation, which may cause crash.

Add sanity test_230l.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Iab4947dea8e26ea798d5f64e218268200a5fabe8
Reviewed-on: https://review.whamcloud.com/33865
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11753 obdclass: index_page support variable length rec

mdd_dir_is_empty() may readdir from other MDT if directory
is striped or remote, in this case, it will issue OBD_IDX_READ
RPC to fetch dir page, and on remote MDT dt_index_page_build()
is called to build page, but this function doesn't support
variable length record, so it may miscalculate offset in
reading, which may cause crash.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia25a0aca52fb1323ea64a7ff72bf6022754af32c
Reviewed-on: https://review.whamcloud.com/33837
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11729 tests: skip sanity test 810 for ARM

Skip sanity.sh test_810 for ARM clients as it always failing.

Test-Parameters: trivial clientarch=aarch64 testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I84117aebb277d4ddcb7787b715587e330f3ebbe5
Reviewed-on: https://review.whamcloud.com/33864
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11770 osd-ldiskfs: preserve bio_integrity API

Preserve the existing kernel API for bio_integrity when the T10-PI
patches are applied, so that any other code that may be using this
interface do not break.

In particular, keep the EXPORT_SYMBOL(bio_integrity_alloc) and
EXPORT_SYMBOL(bio_integrity_prep) in place to avoid module breakage.

In struct bio_integrity_payload put the *bip_generate_fn and
*bip_verify_fn pointers after *bip_vec, since bip_vec is the last
field directly accessed by callers.

In struct blk_integrity_exchg the bi_idx field only needs to be an
unsigned short since the bio->bi_idx and bio->bi_vcnt values used
with it are also unsigned short.  This saves 8 bytes of padding in
the struct and puts the added bi_bio field at the end to preserve
the structure field alignment for external callers.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9e29723d5d581a65b1c2ca2611d012c05b953514
Reviewed-on: https://review.whamcloud.com/33840
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11753 utils: print out DNE2 directory hash flags

There may be flags stored in the lmv_hash_type field, such as
"LMV_HASH_FLAG_MIGRATION" that is set while the directory is
being migrated.  Print out the flag from "lfs getdirstripe".

This is still missing support for "lfs find" to find directories
that have incomplete migration.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib6b362f9eb993b5fa0562b3a51b54eaee1ccab07
Reviewed-on: https://review.whamcloud.com/33843
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>