LU-12400 llite: Use the new vm_fault_t type

Linux 4.17 created the new vm_fault_t type

Linux-commit: 1c8f422059ae5da07db7406ab916203f9417e396

Linux 5.1 changed the vm_fault_t type to bitwise unsigned int
which changes the interfaces registered to struct vm_operations_struct

Linux-commit: 3d3539018d2cbd12e5af4a132636ee7fd8d43ef0

Prefer to match the upstream API and fallback to 'int'
where vm_fault_t is not available.

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I7122fb0d4af3ee9a19c1a5d0b77c4f13f6850181
Reviewed-on: https://review.whamcloud.com/35500
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>

LU-12400 lnet: Infiniband sg_dma changes for linux 5.1

IB/core: Remove ib_sg_dma_address() and ib_sg_dma_len()
Linux-commit: a163afc88556e099271a7b423295bc5176fcecce

This simplification can be applied to mainline 3.15 and later
however the test should remain for 3rd party ib driver support

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I4824b3b737388a3fc0aec43b2d8e5d10f871ccdd
Reviewed-on: https://review.whamcloud.com/35497
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12536 llite: release active extent on sync write commit

Processes can wait forever in osc_extent_wait() for the extent state
to change because the extent write is not started before the wait
begins. A 4.7 kernel change to generic_write_sync() modified it to
check IOCB_DSYNC instead of O_SYNC. Thus an active extent is not
released (written) in osc_io_commit_async() in the synchronous case.

Linux commit: dde0c2e79848298cc25621ad080d47f94dbd7cce

Cray-bug-id: LUS-7435
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I03fd3f17c0c21bcb176730a0edf449d2e5e0c108
Reviewed-on: https://review.whamcloud.com/35472
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12089 scripts: Fix style issues for system-profile.sh

This patch fixes issues reported by checkpatch for file
lustre/scripts/system-profile.sh

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I83ba9d78046077d8b0f20c6877415355a105e70a
Reviewed-on: https://review.whamcloud.com/35432
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-12089 scripts: Fix style issues for statechange-lustre.sh

This patch fixes issues reported by checkpatch
for file lustre/scripts/statechange-lustre.sh

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I202200f16681ea56916beb4b56f7406fccdf39d9
Reviewed-on: https://review.whamcloud.com/35429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-12431 clio: remove default csi_end_io handler

Since the csi_end_io is empty remove it.
Provide an initialize that supports a custom end_io handler.

Cray-bug-id: LUS-7330
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If62c2536772136bfafc6e99ad1dbd9b466735201
Reviewed-on: https://review.whamcloud.com/35400
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12478 build: rhel8 missing module packaing tools

On RHEL8 kmodtool and kernel_module_package_buildreqs
are not installed with kernel-devel

This helps to bootstrap the developer into a working configuration.

Test-Parameters: trivial
Cray-bug-id: LUS-7385
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I30cf5c11174b2aa94c663c72e34f8f88c5c90ed8
Reviewed-on: https://review.whamcloud.com/35356
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12479 utils: cleanup gcc8 string warnings

Cleanup some trivial buffer overflows

Test-Parameters: trivial
Cray-bug-id: LUS-6962
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I17dd1d042c4050e351aca856931ce1107fd5b08f
Reviewed-on: https://review.whamcloud.com/35354
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>

LU-11022 lfs: remove mirror by pool name

lfs mirror split --pool <poolname> <file>

Change-Id: I012e68729b94657236ba3fc530fc7b7485529ed2
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35329
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9859 libcfs: initialize bit_wait_table

With older platforms wait_event_var() is missing so we needed to
provide are own version. This included creating a local
bit_wait_table like the kernel has. That table was never properly
initialized so it can cause failures under the right conditions.

Test-Parameters: trivial

Fixes: 372ef85512 ("LU-11089 obd: use wait_event_var() in lu_context_key_degister()")
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I310d37da7c1b54166224b367446cc905c02ab8bc
Reviewed-on: https://review.whamcloud.com/35567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9835 utils: Properly maintain remaining buffer length in jt_opt_threads

When we write to a buffer, we do need to reduce the size of it,
not just move the starting pointer.

Test-Parameters: trivial
Change-Id: I688e31d5434fb3040985b65a092d56ca30b69941
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/28363
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12540 tests: squash multiply -O -E options

mkfs.lustre and e2fsprogs don't support a multiply
-O -E options. after checking a code, i think easy
way is handle it situation in the test-framework.

Fixes: 591a9b4cebc5 ("LU-9846 lod: Add overstriping support")
Test-Parameter: trivial

Cray-bug-id: LUS-7479
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: I54cab1e02093136a3550a76d1262aeb281118bae
Reviewed-on: https://review.whamcloud.com/35482
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10100 llite: swab LOV EA user data

Many sub-tests failed with "Invalid argument" failures
on PPC client because of the endianness issue.

This patch fixes the issue by adding a common function
lustre_swab_lov_user_md() to swab the LOV EA user data.

Test-Parameters: clientarch=ppc64 \
envdefinitions=ONLY=27 testlist=sanity

Change-Id: I46bab0788300cd79c4e66e1a4990c3e1f7192391
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35291
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12561 build: Remove old lbuild entries

Remove lbuild entries for 2.6 based kernels (and earlier).

Test-Parameters: trivial

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4bb3dcf9e9b4402d7f4e86bbd54f7d9d62efdc48
Reviewed-on: https://review.whamcloud.com/35553
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 ldiskfs: Remove unused 2.6.x patches

Lustre no longer supports 2.6.x kernels, so remove the
ldiskfs patches which are no longer used after removing
RHEL6 and non-3.x SLES11 support.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Icc31c9704669367ece16ec0c09746611cad32ee8
Reviewed-on: https://review.whamcloud.com/35550
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 ldiskfs: Remove 2.6 kernel series

Lustre no longer supports 2.6 kernels, so remove the
ldiskfs patch series for these kernels.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I2c7941582ee90416149cecd47fd2914bc5265a7f
Reviewed-on: https://review.whamcloud.com/35549
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: remove txt directory

The kernel_patches/txt directory was last updated in 2007,
and isn't used for anything today.  Let's remove it.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iad69e7dd92791156752573f8bf2f3c619cdcb637
Reviewed-on: https://review.whamcloud.com/35548
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove unused patches

With 2.6 kernel support removed, these patches are no
longer used.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If374a060f4a4522a10111923c92fa7ccaf791df3
Reviewed-on: https://review.whamcloud.com/35547
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove 2.6 based SLES11 support

2.6 based SLES11 (SLES11SP{0,1,2}) is no longer supported
(and does not build) so remove the series & target for it.

3.x based SLES11 (SP{3,4}) is still supported, and this
should not affect those.

Test-Parameters: trivial

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie73c39d24bad26e696a66e907c161348ab4afad5
Reviewed-on: https://review.whamcloud.com/35546
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12561 kernel: Remove RHEL6 series and targets

Remove the RHEL6 series and target files.

Also remove the RHEL5 (wow!) targets, and the outdated
Fedora (fc) targets.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I00fc47cac656bc3b6f220f3994f0a25ed73879f9
Reviewed-on: https://review.whamcloud.com/35545
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove RHEL6 kernel configs

First in a series of patches to remove RHEL6 support, this
removes the kernel configs.

This should be a build only change, so trivial testing
should be OK.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0b68754f8921b82e8d0c5eaa321d58187da49a70
Reviewed-on: https://review.whamcloud.com/35544
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove RHEL6 from which_patch

RHEL6 is not supported any more, and no longer builds, so
remove it from which_patch.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id71e41ea76e96542cfa090e9a72bca024353601f
Reviewed-on: https://review.whamcloud.com/35543
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12296 llite: improve ll_dom_lock_cancel

ll_dom_lock_cancel() should zero kms attribute similar to
mdc_ldlm_blocking_ast0().

In order to avoid code duplication between mdc_ldlm_blocking_ast0()
and ll_dom_lock_cancel() - add cl_object_operations method to be able
to reach mdc's blocking ast from llite level.

Test illustrating the issue is added.

Cray-bug-id: LUS-7118
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I2b100ead6d420dbf561bc61be973d64dad317214
Reviewed-on: https://review.whamcloud.com/34858
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12516 mdd: support for volatile creation in .lustre

this is useful to enable striping manipulation by FIDs.

Change-Id: I4d5b1b13acdfef21ac46bf3557e9ab6d5ccc796b
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35258
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12484 tests: correct typo in test-framework::is_project_quota_supported()

do_facet was misspelled as do_fact on the zfs side.

Change-Id: Idc112802a6817e5128799cb6059040c6b3021791
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Fixes: f172b116885 (LU-10092 llite: Add persistent cache on client)
Reviewed-on: https://review.whamcloud.com/35362
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>

LU-12489 tests: Recategorize some "Slow" tests as normal.

replay-single 44b is really fast now
replay-ost-single 8[ab] both take under a minute in my testing
ost-pools 23b only takes a minute now and test 18 is still
          under 10 minutes to bother excluding it.

Change-Id: I57a35c39cadd1728c38d332d83cbe50da6d6a8fe
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35367
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-4423 obdclass: use list_sort() to sort a list.

Rather than a bespoke bubble-sort, use list_sort() to
sort this linked list.

As this would become a 1-line function that is only called once,
call list_sort() directly from the one call site.

Linux-commit: e714d3559e964d1547d20b54ad5fd6bbb3401f56

Signed-off-by: NeilBrown <neilb@suse.com>
Change-Id: Ied197a4fdba43d793c5ebbb9afc837a986609469
Reviewed-on: https://review.whamcloud.com/35512
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: MS_* flags and SB_* flags split

In kernel 4.20 the MS_* flags should only be used for mount
time flags and SB_* flags for checking super_block.s_flags
The MS_* flags have moved to a uapi header

Linux-commit: e262e32d6bde0f77fb0c95d977482fc872c51996

Test-Parameters: trivial
Change-Id: Ifd64efb16c7795377ece066d01ae04dc004a13ac
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/35019
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-12510 osd: osd-zfs to release zrlock quickly

otherwise few threads trying to access same dnode can get stuck.
this patch is a quick workaround for the issue, it's supposed
to be replaced with a better patch using regular DMU API.

Change-Id: I24d9ed7f8e68080c6a46409476a80799dbb45230
Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35524
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9019 libcfs: avoid using HZ and msecs_to_jiffies()

HZ is a constant selected with the configuration of the kernel
and msecs_to_jiffies() is a inline function in jiffies.h. Because
we are out of tree that means prebuilt lustre packages could be
installed on a node with different values which impact the
behavior of the file system. This was addressed earlier but
regression have crept back in. Fix up all those instances by
replacing msec_to_jiffies() and HZ with cfs_time_seconds() which
translates seconds to jiffies with nsec_to_jiffies(). Add to
spelling.txt to avoid new regressions.

Change-Id: Ie13efe83774db498bb5475ed47a057bbd42d47bf
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35520
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12462 osc: Do not assert for first extent

In the discard case, the OSC fsync/writeback code asserts
that each OSC extent is fully covered by the fsync request.

This is not valid for the DOM case, because OSC extent
alignment requirements can create OSC extents which start
before the OST region of the layout (ie, they cross in to
the DOM region).  This is OK because the layout prevents
them from ever being used for i/o, but this same behavior
means that the OSC fsync start/end is aligned with the
layout, and so does not necessarily cover that first
extent.

The simplest solution is just to not assert on the first
extent.  (There is no way at the OSC layer to recognize the
DOM case.)

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If66f8d81fb9dd4546a5647a10f6ca551e2cf98e3
Reviewed-on: https://review.whamcloud.com/35525
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12326 tests: recognize Bochs as VM

Bochs is reported by qemu-system-x86_64 and few tests (e.g. 399a)
depends on that.

Change-Id: I3c6cfca1c0cb811425a09b3958cd0626891e73da
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34942
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12545 llite: cleanup stats of LPROC_LL_*

Some LPROC_LL_ stats are not used for a long time. This patch
removes them. LPROC_LL_STAFS is changed to LPROC_LL_STATFS in
this patch too.

Change-Id: I64b033e186733147e2a2984d8afec97240d03573
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/35514
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4423 ptlrpc: make ptlrpc_last_xid an atomic64_t

This variable is treated like an atomic64_t,
so change it's type and simplify the code.

Change-Id: I4b219342222fd784ac1d7dc17660feb816bacd57
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35510
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-4423 ptlrpc: Fix using smp_processor_id() in preemptible context

This warning show up with kernels that enable preemptible
[ 1877.516799] BUG: using smp_processor_id() in preemptible [00000000] code: mount.lustre/14077

Change it to disable preemption around smp_processor_id().

Linux-commit: c369772e78a7383ba4e68673128fe2d6ef2863ee

Fixes: ef94e4d1bb ("LU-8710 ptlrpc: use current CPU instead of hardcoded 0")

Change-Id: If66107f0843c5d0c4bcf874e64a20251fdf9704e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35495
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12542 ldlm: simplify lock_mode_to_index()

This function has the same effect as ilog2(), so just use ilog2
directly.

Change-Id: If90207c328b549e85cb6d38a6604dfb8c7b6c8a0
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35486
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11672 ldlm: awalys cancel aged locks regardless enabling or disabling lru resize

Currently cancelling aged locks is handled by of ldlm_pool_recalc routine,
and it only works when lru resize is enabled, means if we disabled lru
resize, old aged locks are still cached even though they reach the
ns_max_age.

But theoretically, even lru resize disabled, lru_max_age should behave
same as enabling lru resize. At the end, lru_size is like hard limit of
number of locks, but ns_max_age/lru_max_age is a elimination mechanism,
regardless enabling or disabling lru resize meaning once it gets
lru_max_age, locks need to be cancelled.

So fix it here with changing the lru flags when invoking ldlm_cancel_lru
to do the real cancel work, if lru resize is enabled, set flag to
LDLM_LRU_FLAG_LRUR, otherwise LDLM_LRU_FLAG_AGED.

Change-Id: Ic2df2550af87fd7209fdb31ca3730683d727a74d
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/35467
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12137 osd-ldiskfs: use ERR_CAST in osd_oi_index_open()

In osd_oi_index_open() when dentry is invalid it is void casting
the invalid dentry being returned. This is what ERR_CAST was
invented for so use it.

Test-Parameters: trivial

Change-Id: I71016ff9c9dfc4408db8ab14576ba87dd6dc352d
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35455
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12137 osd-ldiskfs: migrate ll_lookup_one_len to osd_compat.c

The function ll_lookup_one_len() in lvfs.h is only used for the
osd-ldiskfs layer so relocate it to osd_compat.c and rename it
to osd_lookup_one_len_unlocked(). We use unlocked in the name
since this signifies it must always be called with the inode lock
not taken since it will take the inode lock itself.

Test-Parameters: trivial

Change-Id: I507100a676f98448f4cacc94e902294903a67efb
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35454
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12431 clio: issue wake with waitqueue lock held

Remove the barrier and rely on the wait queue lock for wake
synchronization.

Leave csi_end_io empty but available for future customization

Inspired by c3973b4aca6df794c492f6856ffbf02f2f8a9592

Cray-bug-id: LUS-7330
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Idc632140256cccfa6046a52cbd1c6432955e2b11
Reviewed-on: https://review.whamcloud.com/35381
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL cleanup

some cleanups
- dt_statfs with an extra paremeter to be dt_statfs_info;
- lod_statfs_and_check does not need an extra parameter and
to be static again;
- move asserts to a better place;
- test component-add with wrong paremeters;
- print out the layout sanity errors wherever needed;
- make an array of layout_sanity errors;
- an HSM sanity test is added;

and one defect:
- the last component cannot be 0-lenght;

Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: If832579ce27cb6ab87d36a594c04363deaea8711
Reviewed-on: https://review.whamcloud.com/35414
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 utils: setstripe component-add support for SEL

the math of the SEL component sizes calculation does not work for the
component end when some components already exist.

Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I793ea7e5a5ac8f4639f694f83af7d413bd2b982c
Reviewed-on: https://review.whamcloud.com/35314
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4423 lustre: convert rsi_sem to a spinlock.

This lock is never held over code that sleeps, and is
only ever held for short periods of time.
So a simple spinlock is best.

Change-Id: I3280f52bf64ae2b896bd67436d8d8a42cab38ac2
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35279
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12400 libcfs: save_stack_trace_tsk if ARCH_STACKWALK

Along with CONFIG_ARCH_STACKWALK save_stack_trace_tsk is not
directly available. Try using symbol_get() to acquire it.

Linux-commit: 214d8ca6ee854f696f75e75511fe66b409e656db

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I923b718eadc6c58fa2676a6d2fbd48523c615f62
Reviewed-on: https://review.whamcloud.com/35239
Reviewed-by: Chris Horn <hornc@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12400 lnet: SO_SNDTIMEO, SO_RCVTIMEO removed

Y2038 64-bit time removed socket options that specify time.
The previous interface is available under _OLD and versions
expecting timespec64 are _NEW.

Linux-commit: 7f1bc6e95d7840d4305595b3e4025cddda88cee5

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I73ad3aa61afdfeba6160d95470b22cea03ed17f9
Reviewed-on: https://review.whamcloud.com/35237
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12427 lnet: warn if discovery is off

Output a warning if discovery is off and admin is
either trying to add a route or enable routing

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Iacd7762c5d19c6e0c45ff6a58693a05761f1336f
Reviewed-on: https://review.whamcloud.com/35200
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12331 llite: create obd_device with usercopy whitelist

Since kernel 4.16 hardened usercopy has been added,
whitelist the struct obd_device to silence the warning.

 Bad or missing usercopy whitelist? Kernel memory exposure attempt
 detected from SLUB object 'll_obd_dev_cache' (offset 1256, size 40)!
 WARNING: CPU: 1 PID: 17534 at mm/usercopy.c:83 usercopy_warn+0x7d/0xa0
 Call Trace:
   __check_object_size+0xfa/0x181
   lmv_iocontrol+0x1146/0x1880 [lmv]
   ll_obd_statfs+0x356/0x860 [lustre]
   ll_dir_ioctl+0x1e37/0x6760 [lustre]
   do_vfs_ioctl+0xa4/0x630

Linux-commit: 8eb8284b412906181357c2b0110d879d5af95e52

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ie863e8a5e2cebd3fd716e7ccc4e0491f83f6fabc
Reviewed-on: https://review.whamcloud.com/34946
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12501 utils: fix 'lfs df' printing loop

If the OS_STATE_NONROT flag is set for a device, the showdf() state
printing loop will spin endlessly because this bit is not printed,
so it is never cleared from the loop's state mask.

Declaring the obd_statfs_state_names[] array indexed by OS_STATE_*
flags also is problematic because the array will double in size as
new binary flags are added (already OS_STATE_NONROT results in an
array size of 0x200 = 512 entries).  Instead, declare a struct that
is indexed linearly and stores the OS_STATE_* flag in a field,
along with the name and whether the flag indicates a problem state.

The flag printing loop can iterate over the array of flags instead
of the os_state bits, which clarifies the for-loop iteration and is
equally efficient.

This also allows printing informational flags with "lfs df -v" so
that OS_STATE_NONROT and similar flags can be visible to users.

Fixes: 68635c3d9b3 ("LU-11963 osd: Add nonrotational flag to statfs")
Change-Id: Ib62e949ca56d691c4699d5f2d9439c42643ebbe5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35456
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 obdecho: Fix style issues for echo_client.c

This patch fixes issues reported by checkpatch
for file lustre/obdecho/echo_client.c

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ia6c218e3ccf35e9ea91b323eac09fb21284bb1aa
Reviewed-on: https://review.whamcloud.com/34489
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 utils: Fix style issues for libiam.c

This patch fixes issues reported by checkpatch for
file lustre/utils/libiam.c

Change-Id: I441edce554e84cdab9eb874666b86fa929ef9f67
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/34438
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6504 socklnd: fix indentation issue highlighted by smatch

lnet/klnds/socklnd/socklnd.c:1459 ksocknal_close_conn_locked() warn: inconsistent indenting

Change-Id: I66139687e82ea526ad9df1a973dc4a408b490ff4
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/14599
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10756 ptlrpc: change IMPORT_SET_* macros into real functions

Make the IMPORT_SET_STATE_NOLOCK and IMPORT_SET_STATE macros into
normal functions. Since import_set_state_nolock() is basically a
wrapper around __import_set_state() we can merge both functions.

Change-Id: Idaa6aeb81ff2282e2f83d758a267129e686bd794
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35463
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12477 lustre: use delete_from_page_cache() for directory pages.

lustre sometimes uses the internal function truncate_complete_page()
to remove a page of a directory.
Much of what this function does, does not apply to directory pages
as there is no invalidatepage function, and at these times, the
page is not dirty.
The only useful part of the function is delete_from_page_cache(),
so just call that directly.

Linux-commit: d17fa2f3a0b9b40be48e0c3cc88eb3b3cea1b701

Change-Id: I54795e71e107c50f662bd2015c6f621bfe436e0a
Acked-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35462
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12477 llite: use generic_error_remove_page()

lustre's internal ll_invalidate_page() is behaviourally identical to
generic_error_remove_page().
In the case of lustre it isn't a memory hardware error that requires
the page being invalidated, it is the loss of a lock, which will likely
result in the data changing on the server.
In either case, we don't want the page to be accessed any more, so the
same removal is appropriate.

Linux-commit: d5419b40599b4d6e030695dad30f15347679be66

Change-Id: I92686b5332eec02580563c1bee779688e8e591a3
Acked-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35461
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12137 osd-ldiskfs: migrate osd_ios_lookup_one_len() to osd_compat.c

The function osd_ios_lookup_one_len() was created for the LFSCK code
to look for a dentry by name and if the inode of that dentry was
NULL treat it as an -ENOENT so LFSCK would repair the file. This
function will be used for more the scrub infrastructure in future
patches so move it to osd_compat.c.

Test-Parameters: trivial

Change-Id: Ic34c1110f8ced7a4a2f7c0fa3b8a9403be9940ca
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35453
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12524 libcfs: Reduce memory frag due to HA debug msg

The dynamic allocation and freeing of Lustre trace pages has been
shown to cause memory fragmentation that sometimes prevents
applications from getting the contiguous memory they need to run. In
one such occurrence over 99% of the messages were the matched open
trace messages issued by mdc_close():

DEBUG_REQ(D_HA, mod->mod_open_req, "matched open; tag %d", tag);

D_HA is included in the default set of debug flags. This has proven
to be quite useful in debugging connection issues particularly at
mount time. So removing all HA message from the default tracing is
not a good option.

However, the matched open debug message has not proven itself to be
as generally useful. So moving the message under a different debug
flag, one that must be explicitly enabled, reduces the amount of
default tracing and thereby helps reduce fragmentation without
causing much loss of functionality. Using D_RPCTRACE to match the
corresponding open debug message in mdc_set_open_replay_data.

Test-Parameters: trivial
Cray-bug-id: LUS-7560
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: Iee267fba517c20b82dccf8d2ac10f8e7f15354f8
Reviewed-on: https://review.whamcloud.com/35449
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12523 ptlrpc: Add jobid to rpctrace debug messages

This mod adds the jobid string found in the ptlrpc_body of an rpc
to the output of rpctrace messages. If jobids are not in use the
string will be empty. If jobids are in use, the string can be
useful in analyzing Lustre activity.

Test-Parameters: trivial
Cray-bug-id: LUS-7557
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: Ib7ec75e28581f3ac420314812e2521fa49f021dd
Reviewed-on: https://review.whamcloud.com/35445
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12513 utils: add handling YAML_NO_TOKEN

YAML_NO_TOKEN is something that is not a key value pair, either
an entry or exit into a sub-structure. The parser equates it to
an error. This flaw makes a YAML format that is valid be treated
as invalid. Add handling YAML_NO_TOKEN as a valid setting. This
flaw was discovered in the patch for the LU-6081 work.

Test-Parameters: trivial

Change-Id: Ibe0c0a2bea22b26a0dd2d900b7a4a5957b96e3da
Signed-off-by: Ben Evans <bevans@cray.com>
Reviewed-on: https://review.whamcloud.com/35428
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12513 utils: change cy_valueint to 64 bits

While testing the lustre YAML implementation with netlink I found
that it didn't support 64 bit integer values. Change cy_valueint
to a long long.

Test-Parameter: trivial

Change-Id: I1746a4907a83e7c5733a0681a66e7f4a54a4c392
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35423
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>

LU-12482 tests: use MDS1_VERSION in sanity-hsm

MDS_VERSION_CODE is long deprecated and undefined.

Change-Id: I007cea54cb4ddbc5705656a84d12448dd84aa0a1
Test-Parameters: trivial
Test-Parameters: testlist=sanity-hsm,sanity-hsm
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35353
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12043 llite: make sure readahead cover current read

When doing readahead, @ria_end_min is used to indicate
how far we are expected to read to cover current
read.

update @ria_end_min unconditionally with IO end.
also @ria_end_min is closed interval which should be
calculated as start + count - 1;

Change-Id: If7f8da44da31623a73b363d5a18c1ec8b54da745
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35215
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12401 gss: fix checksum for Kerberos and SSK

When computing checksum for Kerberos, krb5 wire token header is
appended to the plain text. Make sure the actual header is appended
in gss_digest_hash().
For interop with older clients, introduce new server side tunable
'sptlrpc.gss.krb5_allow_old_client_csum'. When not set, servers refuse
Kerberos connection from older clients.

In gss_crypt_generic(), protect against an undefined behavior by
switching from memcpy to memmove.

When computing checksum for SSK, make sure the actual token is used
to store the checksum.

Fixes: a21c13d4df ("LU-8602 gss: Properly port gss to newer crypto api.")
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small,sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true clientbuildno=6308 clientjob=lustre-reviews-patchless testlist=sanity,recovery-small,sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0233ada481f132af112bf88c065f5421902c942e
Reviewed-on: https://review.whamcloud.com/35099
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12131 tests: fix test_802a for GSS

test_802a should not overwrite already existing client mount options
when trying to mount client as read-only.

Test-Parameters: trivial
Test-Parameters: envdefinitions=ONLY=802a testlist=sanity
Test-Parameters: envdefinitions=SHARED_KEY=true,ONLY=802a testlist=sanity
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8189c245870fb0caf48006db11621f0af48e1878
Reviewed-on: https://review.whamcloud.com/35335
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12131 tests: properly handle GSS in server failover

In case of server failover, a number of aspects must be handled when
GSS based features (SSK or Kerberos) are activated:
- lsvcgssd daemon must be restarted;
- targets must be mounted with proper skpath option;
- permissions on keys must be adjusted.
When service is initially started, all that is managed in setupall().
fail() and facet_failover() have to be improved to take GSS aspects
into account.

Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small,sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8db686f406629c7eec655496cf83c0539c1bfb33
Reviewed-on: https://review.whamcloud.com/35041
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for cascading_rw.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mpi/cascading_rw.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I033db1a7ee23042cc3ec0dedced48830e00d1230
Reviewed-on: https://review.whamcloud.com/35433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11761 fld: let's caller to retry FLD_QUERY

In fld_client_rpc(), if the FLD_QUERY request between MDTs fails
with -EWOUDBLOCK because the connection is lost, return -EAGAIN
to notify the caller to retry.

It also reverts the patch https://review.whamcloud.com/12586/, which
was landed on b2_6_90_0-5-g6db07f0 to avoid returning -EAGAIN from
lod_object_init() to confuse lu_object_find_at() (thinks the object
was dying when it encounters -EAGAIN). In current Lustre version,
lu_object_find_at() just returned found object and let's caller to
check whether it's dying.

Fixes: 6db07f095fba ("LU-5871 lod: Do not return EAGAIN in lod_object_init")
Change-Id: Ie83ebfdae2bd50c96a59a065f7f3c3dcfad04e42
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34962
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12538 lod: Add missed qos_rr_init

The new lmv space hash code uses the lu_qos_rr struct, but
forgot to init it fully.  Specifically, the spin lock isn't
inited, causing failures.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id410a8dc61980b880eab7e151b85c417a8439fd5
Reviewed-on: https://review.whamcloud.com/35490
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

New tag 2.12.56

Change-Id: Iad3fd72a2720f5dba2b7dae667b088eb73199d6a
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-7912 osc: Remove stale comment in osc_page_transfer_add

Ever since LU-3321 the fileds were not shared,
but then ops_inflight was completely deleted too.

Test-Parameters: trivial
Change-Id: I7b235f10ddb26a7ddbd4de7e502d33ee81a4f2e3
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/19115
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12410 tests: ignore return status of removal of 99-lustre-test.rules

we don't care if we cannot remove it on th server side.

Change-Id: Id6833505c1e7cd39df9845a16b01c31c9d65e794
Test-Parameters: trivial
Test-Parameters: testlist=sanity-dlc
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35398
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>

LU-9862 lov: Correct bounds checking

While Dan Carpenter ran his smatch tool against the lustre code
base he encountered the following static checker warning:

lustre/lov/lov_ea.c:207 lsm_unpackmd_common()
warn: signed overflow undefined. 'min_stripe_maxbytes * stripe_count < min_stripe_maxbytes'

The current code doesn't properly handle the potential overflow
with the min_stripe_maxbytes * stripe_count. This fixes the
overflow detection for maxbytes in lsme_unpack().

Change-Id: I34646df3d59cadcb42a4defb58e16cb840acc99
Fixes: 3ddcf5b4a138 ("LU-7890 lov: Ensure correct operation for large object sizes")
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/28484
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12495 test: shorten qos_maxage to update statfs

sanity test_413b() should shorten lmv->desc.qos_maxage to update
cached statfs in time.

Test-Parameter: trivial envdefinitions=ONLY=413b
Test-Parameter: testlist=sanity,sanity,sanity,sanity,sanity,sanity

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I58672590669be5eaa5c0d679c51cb6cd533bc0d7
Reviewed-on: https://review.whamcloud.com/35395
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>

LU-12514 obdclass: Drop FS_HAS_FIEMAP compat macro

FS_HAS_FIEMAP was some sort of old RHEL5 construct that's not
really important anymore

Linux-commit: 5c8eae72ff46f0e70d03ae2e86e631d7a1ca4fe6

Change-Id: Ia9941fa32eeb6114f9404014b78c29465d524d07
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/35424
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>

LU-12368 obdclass: don't send multiple statfs RPCs

If multiple threads are racing to send a non-cached OST_STATFS or
MDS_STATFS RPC, this can cause a significant RPC storm for systems
with many-core clients and many OSTs due to amplification of the
requests, and the fact that STATFS RPCs are sent asynchronously.
Some logs have shown few 96-core clients have 20k+ OST_STATFS RPCs
in flight concurrently, which can overload the network if many OSTs
are on the same OSS nodes (osc.*.max_rpcs_in_flight is per OST).

This was not previously a significant issue when core counts were
smaller on the clients, or with fewer OSTs per OSS.

If a thread can't use the cached statfs values, limit statfs to one
thread at a time, since the thread(s) would be blocked waiting for
the RPC replies anyway, which can't finish faster if many are sent.

Also add a llite.*.statfs_max_age parameter that can be tuned on
to control the maximum age (in seconds) of the statfs cache.  This
can avoid overhead for workloads that are statfs heavy, given that
the filesystem is _probably_ not running out of space this second,
and even so "statfs" does not guarantee space in parallel workloads.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I95690e37aecbac08ac5768a5e5c6c70ca258a832
Reviewed-on: https://review.whamcloud.com/35380
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12481 osd-ldiskfs: allow full 64KB xattr size

When the 'ea_inode' feature is enabled, allow the full 64KB xattr
size, since the xattr data is stored directly in the ea_inode data
blocks, while the ext4_xattr_entry and ext4_xattr_hdr structures are
stored separately in the parent inode or external xattr block.

This avoids errors on the client trying to set a full-sized inode:

    setfattr: /mnt/lustre/f61.conf-sanity: Argument list too long

Fixes: 3ec712bd183a ("LU-11868 osd: Set max ea size to XATTR_SIZE_MAX")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1320c32af98ab0feeeb147d8dbbc66ec7d1b8e1f
Reviewed-on: https://review.whamcloud.com/35352
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9859 libcfs: remove prng

The cfs prng is no longer used, so discard it.

Linux-commit: 508d5e0f4d45a815a0759c6aea69fef62359cf74

Test-Parameters: trivial

Change-Id: If780690dba196c8bc5935be223a952442f6a33ae
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/35351
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12473 llapi: fix pool_list by path

lfs/lctl pool_list <fs_path> would print the FS path as pool prefix.
print fsname properly instead.

Fixes: 8813fdf2a4f2 ("LU-5030 util: migrate liblustreapi to use cfs_get_paths()")
Change-Id: I016b794fabd3d161d4651b41989637aebdf31f36
Signed-off-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-on: https://review.whamcloud.com/35320
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4423 ptlrpc: remove inline on non-inlined functions.

These three functions are never inlined.  The only time they
are used, their address is taken, and this forces them to
be compiled as stand-alone functions.  So having the "inline"
declaration is misleading.

Move the functions to the place where their address is used, and
remove the 'inline' tag.

Test-Parameters: trivial

Change-Id: I0824d362f05e7397dd828f06464ad5aa156673d4
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35296
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12400 osd-ldiskfs: get rid of legacy 'get_ds()' function

get rid of legacy 'get_ds()' function

Every in-kernel use of this function defined it to KERNEL_DS
(either as an actual define, or as an inline function).

Linux-commit: 736706bee3298208343a76096370e4f6a5c55915

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I3aabe74802f1a953b140728f22c83125dae270c3
Reviewed-on: https://review.whamcloud.com/35238
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12423 lnet: honor discovery setting

If discovery is off do not push out any updates. This could be
triggered in case of a gateway's interface changing.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ie421318ae85b895327ec170ffb436c9b679f6866
Reviewed-on: https://review.whamcloud.com/35192
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: interoperability support

Add a new SEL magic for storing SEL components on disk.
It is never gets out of LOD, converted on read/write to COMP_V1.
A the result, old MDS is not able to open SEL files.
At the same time old clients are able to work with existing files
seamlessly. Old clients still lacks lustre utils support, thus not
possible to create new SEL files, etc.

Cray-bug-id: LUS-2528
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Ib3f0b1402cd920e56beaad78a74da485bd7ad342
Reviewed-on: https://review.whamcloud.com/35144
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: totalram_pages changed to atomic_long_t

Kernel 5.0 changed totalram_pages to atomic_long_t
Provide an abstracted accessor now that totalram_pages
is now a function

Linux-commit: ca79b0c211af63fa3276f0e3fd7dd9ada2439839

Test-Parameters: trivial
Change-Id: I558e42074004e2ee5f79deea0d363e5bea332729
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/35025
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 utils: SEL: lfs find & getstripe support

The support includes:
- add --extension-size option to lfs find & getstripe along
  with +/- functionality;
- do not take the extension components into account for
  lfs find --stripe-size and --stripe-count;
- add appropriate tests;

Cray-bug-id: LUS-2528
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Ic3ad4c713e8c676998cf7d02b524ba266c992924
Reviewed-on: https://review.whamcloud.com/34909
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 obd_type: use typ_kobj.name as typ_name

As the kobject has a name (after kobject_add has been called),
we don't need to also store it in typ_name.
So use typ_kobj.name instead of typ_name.

This requires changing some "char *" to "const char *" as
typ.kobj.name is const.

Change-Id: Iaf0ef192e91ba1b4bd1c1b124dc1068de632d341
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/34717
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>

LU-12101 socklnd: fix infinite loop in ksocknal_push()

If the list_for_each_entry() loop in ksocknal_push()
ever finds a match, then it will increment 'i', and the outer
loop will continue.

Once peer_off becomes larger than the number of matches
in a given chain, 'peer_ni' will be an invalid pointer, and
ksocknal_push_peer() will probably crash when called on it.

To abort the outer loop properly, we need to test if
"i <= peer_off", which indicates that all patching peers
have been found.

This bug can easily be reproduced by running
  lctl --net tcp push

Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: I9468214c7e1a0154213586cac0deb61afaa1d53d
Reviewed-on: https://review.whamcloud.com/34499
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: Repeated components

This changes behavior when there is no next component to
spill over to.  Currently, in that case, we just extend
the current component regardless of available space.

Now, if there is no following component, we try repeating
the current component, creating a new component using the
current one as a striping template.  We try assigning
striping for this component.  If there is sufficient free
space on the OSTs chosen for this component, it is
instantiated and i/o continues there.

If there is not sufficient space on the OSTs chosen for the
new component, we remove it & extend the current component.

This is a behavioral improvement, with no implications for
layout sanity checking.

Cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: If9f364b4105a4bb892dfe673c724e04781c46336
Reviewed-on: https://review.whamcloud.com/33786
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: Add FLR support

Add FLR support for self-extending layouts.

The basic model is that when a layout intent would modify
an FLR replica, we first run the extent_update code to
perform any layout extent changes for self extending
layouts.

This treats the FLR operations (stale, resync, etc)
similarly to i/o, creating initialized layout where those
operations need it.  This makes the interaction between SEL
and FLR fairly simple.

Add FLR tests for self-extending layouts

Cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ia23df8e226955f64e9b19df993b66d2d4f820f33
Reviewed-on: https://review.whamcloud.com/33785
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: Layout sanity checking

Add layout sanity checking for self-extending layouts.

This requires a more complex method checking layouts,
checking the entire layout rather than just individual
components against their immediate neighbors.  This is
implemented with a layout sanity callback which walks the
layout.

Incorporate mirror sanity checks from lfs.c.

Cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I960a4ce96ace54f7fe4305b9197e27c540f81211
Reviewed-on: https://review.whamcloud.com/33784
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: Implement basic spillover space

This is a barebones implementation of spillover space.
This allows the creation of extendable layout components,
which are normal layout components followed by "extension
components".  These extension components are never
initialized, instead, when i/o reaches them, the server
checks if there is sufficient space on the preceding normal
layout component, and if so, it modifies the extent of the
component to give space to the preceding component.

If there is not sufficient space on those OSTs, the special
extension space component can be removed, and the next
component of the layout is moved down to meet the existing
component.  This allows i/o to "spill over" to this new
layout component, which is expected to be on different
OSTs.

For multi-tiered systems, this makes it possible to avoid
the situation where an inner tier is low on space, but a
an outer tier has plenty, and PFL files cannot use the
space in the outer tier because the inner is full.

This patch requires the next patch in the series for FLR
support, but does not depend on the other subsequent
patches in this series.

Cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I8f6c6df8ee155033d5278535dc456e604552e409
Reviewed-on: https://review.whamcloud.com/33783
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: Add flag & setstripe support

The self-extending layouts feature adds a new layout flag
and also uses the stripe size field differently.

This patch implements this basic functionality, to be used
in subsequent patches.

Cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I4392b70266cbab5bc8fa42afc3c360b954d5918a
Reviewed-on: https://review.whamcloud.com/33782
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 lod: SEL: split lod_del_layout

SEL deletes layout components as part of other operations,
rather than only as a separate delete operation.

So we split lod_layout_del in to a function that prepares
the layout and one that writes it out.  The prep_layout
function will be used in later patches.

Cray-bug-id:  LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I5d7270db9a8d9bc94f4571906ed9e2d4a17a151b
Reviewed-on: https://review.whamcloud.com/33780
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 tests: New test-framework functionality

The self-extending layout tests will make heavy use of
setting OST low & high watermarks to simulate low/out of
space conditions.  To this end, add improved ways of
working with these to the test framework and use them in
sanity 253.

Add a component-count helper in sanity-pfl.

Fix pool_add_targets so it can add only 1 target.

Also move one helper from sanity to test-framework so it
can be used from sanity-pfl.

Cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I4e75c7db07b201ff2c410734d5daa991e74bd5c1
Reviewed-on: https://review.whamcloud.com/33778
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11762 ldlm: don't exceed hard timeout

For recovery lustre has both a soft timeout, obd_recovery_timeout
and a hard timeout, obd_recovery_time_hard. When the recovery
timer is adjust with the function extend_recovery_timer() you
can control if it takes in consideration what is left of the
timer. The current code is not very clear on its intent so this
patch attempts to make the code understandable. No function
change should happen with this patch.

Change-Id: I5701a6cd813ad64b6b4422863767af135eb8e94b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34408
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12491 obdclass: add comment for rcu handling in lu_env_remove

During the review it was pointed out why the RCU lock was dropped
in lu_env_remove() but the code itself doesn't explain why. Add
a comment giving the details why RCU locking is not needed.

Test-parameters: trivial

Change-Id: I4fd761d2e1b4adad8e970904d56cdcd057dfe7d5
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35447
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11518 osc: cancel osc_lock list traversal once found the lock is being used

Currently, in osc_ldlm_weigh_ast, it walks osc_lock list (oo_ol_list)
to check whether target dlm is being used, normally, if found, it needs
to skip the rest ones and cancel the traversal, but it doesn't, let's
fix it here.

Change-Id: I2e64d2938cdacb6c5baca73647d74c9fb8f54f8c
Fixes: 3f3a24dc5d7d ("LU-3259 clio: cl_lock simplification")
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/35396
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11213 uapi: change "space" hash type to hash flag

Change LMV_HASH_TYPE_SPACE to LMV_HASH_FLAG_SPACE to make it flexible
in directory layout inheritance in the future. But it's still exposed
to user as hash type "space" in "lfs setdirstripe" command to make
it easy to understand.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ifa302204ed62dff8cc9d12fdc1f9ea86f8491d40
Reviewed-on: https://review.whamcloud.com/35318
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11264 llapi: clean up llapi_search_tgt() code

Clean up llapi_search_tgt() and helper functions llapi_search_ost()
and llapi_search_mdt() to set errno on return.

Add man pages for all of these functions.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ieb2e93208fbc1b1492f632d8ce1383ca9fdec5f2
Reviewed-on: https://review.whamcloud.com/35092
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11285 mdt: improve IBITS lock definitions

Move MDS_INODELOCK_* flags into a named enum, and add the definitions
for the newer flags into wirecheck/wiretest to ensure consistency.

Rename MDS_INODELOCK_MAXSHIFT to MDS_INODELOCK_NUMBITS to hold current
number of lockbits, rather than one less than the number of lockbits,
since the only two places that use it expect it to be one larger than
it is.  Fix uses of MDS_INODELOCK_NUMBITS to be number of locks.  This
does not change the value of MDS_INODELOCK_FULL, which is used in the
protocol to exchange supported lock bits between client and server.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0c2985bcc602b7182d5db2cf8d590923be2cab07
Reviewed-on: https://review.whamcloud.com/35045
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12491 obdclass: use RCU to release lu_env_item

as rhashtable_lookup_fast() is lockless and can
find just released objects.

Fixes: aa82cc8361 ("obdclass: put all service's env on the list")
Change-Id: I6ed8ccc5bb5b192eed90b55103d11b822ec90692
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35038
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11893 lnet: consoldate secondary IP address handling

The last piece of code with broken secondary IP address
support is lnet_parse_ip2nets(). We could fix it like
o2iblnd or socklnd was done but since the LND drivers
resolved those issues instead we can move the handling
out of the LND drivers into one place in the LNet core.
To do this we introduce struct lnet_inetdev which is
a collection of data that the current LNet layer requires.
The new function lnet_inet_enumerate() is used to collect
this information.

Change-Id: I0c532caa3cf6b2178eb1ab65e55e5883d408a185
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34993
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>