git://git.whamcloud.com - fs/lustre-release.git/log

LU-13416 ldiskfs: don't corrupt data on journal replay

Journalled write want a special attention on blocks release,
revoke records must added to avoid replace new write blocks
with stale data. Mark inode as "journal write" to generate
valid revoke records. Large EA inode updates affected
with this bug also.

large ea fix is

Linux-commit: ddfa17e4adc4bd19c32216aaa6250dc38b0579df
Author: Tahsin Erdogan <tahsin@google.com>
Date: Wed Jun 21 21:36:51 2017 -0400
ext4: call journal revoke when freeing ea_inode blocks

Lustre-change: https://review.whamcloud.com/38281
Lustre-commit: a23aac2219047cb04ed1fa555f31fa39e5c499dc

Change-Id: I605128c4ba70331a48715dc95546430909efb893
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38705
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13589 utils: fix lfs setstripe unit parsing

The "size_units" variable was not being reset while parsing different
"lfs setstripe" arguments, so e.g. "lfs setstripe -E 1M -S 65536 ..."
ended up using the 'M' unit for the stripe size, which resulted in a
stripe_size of 65536MiB = 64GiB, which resulted in an error.

This only appeared with PFL or other layout patterns which had more
than one unit being parsed, and was already fixed in master via SEL.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib3f9be86f5104aaea4f77d87853255a518cbc3a0
Reviewed-on: https://review.whamcloud.com/38680
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13535 lfsck: fix possible PFL layout corruption

While checking lmm_oi in composite layout the pointer to 'lmm'
is re-assigned to component entry but the same pointer is used
for LOV EA buffer to update EA. Therefore if lmm_oi was fixed in
some component then just current entry is saved as new layout.

Lustre-change: https://review.whamcloud.com/38584
Lustre-commit: be009cb4a73b3bef7302083bec7d1d6289d515b7

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ifbd984a71b383ab4ca35ad59ed9cd8cf57b6d7cc
Reviewed-on: https://review.whamcloud.com/38585
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>

LU-13111 kernel: new kernel [SLES12 SP5 4.12.14-122.20.1]

This patch makes changes to support new SLES12 SP5 release
for Lustre client.

Test-Parameters: trivial clientdistro=sles12sp5 \
env=SANITY_EXCEPT="56oc 817" testlist=sanity

Change-Id: Ia4b856b03801e02da9a2e584efeb8759b4dd30c3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38640
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13556 kernel: kernel update RHEL7.8 [3.10.0-1127.8.2.el7]

Update RHEL7.8 kernel to 3.10.0-1127.8.2.el7.

Test-Parameters: trivial clientdistro=el7.8 serverdistro=el7.8

Change-Id: If7ac6f4b5f1fe32a15c63f51589a2e320001b4a5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38635
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13488 kernel: new kernel [RHEL 8.2 4.18.0-193.1.2.el8]

This patch makes changes to support new RHEL 8.2 release
for Lustre client.

Test-Parameters: trivial clientdistro=el8.2 \
env=SANITY_EXCEPT="130" testlist=sanity

Change-Id: Icb1db3afd2e94423a45354acfdd559f8f1e294cb
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38461
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12904 o2ib: ib_destroy_cq() returns void

Kernel destroy CQ flows can't fail and the returned value of
ib_destroy_cq() is not interested in those flows.

kernel-commit: 890ac8d97e6722a9e4a66a0bd836d1b028d075fe

This patch is back-ported from the following one:
Lustre-commit: 7d2ea1e5bbd80f23e6935174c969b34b58048443
Lustre-change: https://review.whamcloud.com/36578

Test-Parameters: trivial
Cray-bug-id: LUS-8042
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I873bf76a33bd80d5e6de4d1b16a79ff5ea930f3a
Reviewed-on: https://review.whamcloud.com/38489
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12634 lnet: for_ifa removed. Use in_dev_for_each_ifa_rtnl

Linux 5.3 removed for_ifa and replaced it with an _rntl and _rcu
versions for use with their respective locking primitives.

kernel-commit: ef11db3310e272d3d8dbe8739e0770820dd20e52

This patch is back-ported from the following one:
Lustre-commit: 6e0d0146276353559c821916e193c90d167b14e0
Lustre-change: https://review.whamcloud.com/35744

Test-Parameters: trivial
Cray-bug-id: LUS-7689
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Iea07222b9abb3f9c219d28fe2c660d9eaf21af80
Reviewed-on: https://review.whamcloud.com/38487
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12400 ptlrpc: Sun RPC changes for RCU locking

In kernel 4.20 SUNRPC cache_detail->hash_lock changed to spinlock_t

Now that the reader functions are all RCU protected, use a regular
spinlock rather than a reader/writer lock.

Linux-commit: 1863d77f15da0addcd293a1719fa5d3ef8cde3ca

This patch is back-ported from the following one:
Lustre-commit: 77d53777e32c80047cb75293d5f9a4c0d23bbea8
Lustre-change: https://review.whamcloud.com/35499

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If0df38337d5a2bb0ac4b8cb645dbe89f65e0f352
Reviewed-on: https://review.whamcloud.com/38483
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: include file linux/selinux.h removed

In kernel 5.1 linux/selinux.h was removed with
SELinux: Remove unused selinux_is_enabled

Linux-commit: 3d252529480c68bfd6a6774652df7c8968b28e41

This patch is back-ported from the following one:
Lustre-commit: 39e5bfa73414d18738001761b42ea0e3264c2983
Lustre-change: https://review.whamcloud.com/35035

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If963e6b22b7b07899de5b970f934bb157c5f7cec
Reviewed-on: https://review.whamcloud.com/38480
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: Lustre specific iov_for_each broken (removed)

Kernel 4.20 introduced iov_iter_type and broke iov_for_each

As iov_for_each is only used once so drop the macro entirely.
When iov_iter_type is available ignore invalid iter types.

Linux-commit: 8a363970d1dc38c4ec4ad575c862f776f468d057

Kernel 3.15 added type to iov_iter. Use the type to provide
a sensible replacement for iov_iter_type when it is available.

Linux-commit: 71d8e532b1549a478e6a6a8a44f309d050294d00

This patch is back-ported from the following one:
Lustre-commit: d93aa0171a25f8ffca51bed35a2d477a45fda0f3
Lustre-change: https://review.whamcloud.com/35024

Cray-bug-id: LUS-6962
Change-Id: I97cdce1c85803ac2d4436d4eedf67a545ea2cdb8
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/38479
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12400 llite: Use the new vm_fault_t type

Linux 4.17 created the new vm_fault_t type

Linux-commit: 1c8f422059ae5da07db7406ab916203f9417e396

Linux 5.1 changed the vm_fault_t type to bitwise unsigned int
which changes the interfaces registered to struct vm_operations_struct

Linux-commit: 3d3539018d2cbd12e5af4a132636ee7fd8d43ef0

Prefer to match the upstream API and fallback to 'int'
where vm_fault_t is not available.

This patch is back-ported from the following one:
Lustre-commit: f2b224a48cb00f885b9df2cc56e349dae5f27f9e
Lustre-change: https://review.whamcloud.com/35500

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I7122fb0d4af3ee9a19c1a5d0b77c4f13f6850181
Reviewed-on: https://review.whamcloud.com/38478
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13168 tests: verify truncated xattr is handled

Verify that a truncated trusted.lov xattr is handled properly,
for both plain and PFL layouts.

Add a test case that verifies this is fixed for both layout types.

Lustre-change: https://review.whamcloud.com/38434
Lustre-commit: cb74546354201434a6fd3d53acd1a0808fbfcb1c

Fixes: f2d06d3c76 ("LU-12911 llite: Don't access lov_md fields before size check")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I11d420c7fdc2362f64689a545b95c76e893ebbe5
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38604
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13131 osc: Ensure immediate departure of sync write pages

Except for the case of direct-io and server-lock, we are
hold potentially multiple locks that are next to impossible
to find and cross reference.
So instead just send it all right away - should only
be a factor in rare cases of out of quota or close to out
of space.

Lustre-change: https://review.whamcloud.com/38453
Lustre-commit: 13b7cf4fabdd55977b68eb856bfdc82f0a349e73

Change-Id: I961cd9ba7f3266d22dfc5eff758c2f4ebbe148a4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13131 osc: Do not wait for grants for too long

obd_timeout is way too long considering we are holding a lock
that might be contended. If OST is slow to respond, we might
get evicted, so limit us to a half of the shortest possible
max wait a server might have before switching to synchronous IO.

Lustre-change: https://review.whamcloud.com/38283
Lustre-commit: 1eee11c75ca13745d083410e1ced3a1a8b088ee9

Change-Id: I36653194c1b8b95ba3cc2ed9240df7b0888cf7ed
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38672
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-9679 osc: convert cl_cache_waiters to a wait_queue.

cli->cl_cache_waiters is a list of tasks that need
to be woken when grant-space becomes available.  This
means it is acting much like a wait queue.
So let's change it to really be a wait queue.

The current implementation adds new waiters to the end of the list,
and calls osc_enter_cache_try() on each in order.  We can provide the
same behaviour by using an exclusive wait, and having each waiter wake
the next task when it succeeds.

If a waiter notices that success has become impossible, it wakes all
other waiters.

If a waiter times out, it doesn't wake other - just leaves them to
time out themselves.

Note that the old code handled -EINTR from the wait function.  That is
not a possible return value when wait_event_idle* is used, so that
case is discarded.

As we need wait_event_idle_exclusive_timeout_cmd(), we should fix the
bug in that macro - the "might_sleep()" is wrong, as a spinlock might
be held at that point.

Linux-Commit: 31f45f56ecdf ("lustre: osc_cache: convert
cl_cache_waiters to a wait_queue.")

Lustre-change: https://review.whamcloud.com/37605
Lustre-commit: b2ede01d1ed77ddc512c013220f6ea8b509e9541

Change-Id: Ib7622ea2daea8f6e59bef95d3b6c5a80d209b81e
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/38575
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13131 osc: Always send all HP RPCs requests

that are resulting from lock cancel activity.

Lustre-change: https://review.whamcloud.com/38057
Lustre-commit: 0f1743916be6605fcd8f57993d6ce7d8d06ce12c

Change-Id: I4007167f0f39b0699e977c14bd160f475d8288ad
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37468
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9859 libcfs: initialize bit_wait_table

With older platforms wait_event_var() is missing so we needed to
provide are own version. This included creating a local
bit_wait_table like the kernel has. That table was never properly
initialized so it can cause failures under the right conditions.

Test-Parameters: trivial

Lustre-commit: https://review.whamcloud.com/35567
Lustre-change: 916643c4b8d136c9c96afa5cd21d1eb6030d148c

Fixes: 372ef85512 ("LU-11089 obd: use wait_event_var() in lu_context_key_degister()")
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I310d37da7c1b54166224b367446cc905c02ab8bc
Reviewed-on: https://review.whamcloud.com/38574
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11089 obd: use wait_event_var() in lu_context_key_degister()

lu_context_key_degister() has an open coded loop which calls
schedule() without setting a new task state.  This is generally
a bad idea - it could easily just spin.

Instead, use wait_event_var() to wait for ->lct_used to be zero,
and arrange to get a wakeup when that happens.
Previously ->lct_used would only fall down to 1.  Now we decrement
it an extra time so that wake_up, which only happens when the
count reaches zero, will only happen when lu_context_key_degister()
is actually waiting for it.

Note that this patch removes key_fini() from protection by
lu_keys_guard.  key_fini() calls are not always protected
by a lock, and there seems to be no need here.  Nothing else
can be acting on the given key in that context at this point,
so no race is possible.

Linux-commit: ef84c07364211bb4e398a9de45d1c13a32059cee
Lustre-commit: 372ef85512dd2a722415fba9a3df66f81029508b
Lustre-change: https://review.whamcloud.com/33667

Change-Id: I9514bd21916f75fce00e393612967fb197e3a1c4
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/38573
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12911 llite: Don't access lov_md fields before size check

When 'struct lov_user_md' is passed in via setxattr, it comes with
a size. If thatt size is too small, some function that check exactly
what version is present might access beyond the end of allocation
memory, which can have undesirable effects, such as triggering
a KASAN warning (and possibly worse).

So check that the size is sane before looking inside the structure
at all.

Lustre-change: https://review.whamcloud.com/36589
Lustre-commit f2d06d3c76a1d69447e7bd6fd29d8165be558d73

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib3f071a3ff77a039fdfa38c903d87999108b3322
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/38433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13102 llog: fix processing of a wrapped catalog

The logic for rereading a llog buffer had an exception
for a full catalog, when lgh_last_idx = llh_cat_idx and a first
processing index is a llh_cat_idx+1. This check is based on
a value lh_last_idx, which stays unchanged between buffer read.
But llh_cat_idx could go forward, and this lead to a wrong check
where reread doesn't happen. As a result Lustre got ENOENT for
a record and stoped osp processing.

llog_cat_set_first_idx())
catlog [0x6:0x1:0x0] first idx 34730, last_idx 34731
osp_sync_process_queues()) 1 changes, 0 in progress, 0 in flight
llog_process_thread())
stop processing plain 0x76941:1:0 index 64767 count 1
llog_process_thread())
index: 34731, lh_last_idx: 34730 synced_idx: 34730 lgh_last_idx: 34731
llog_cat_process_common()) processing log [0x2780f:0x1:0x0]:0
at index 34731 of catalog [0x6:0x1:0x0]
llog_cat_id2handle()) snx11281-OST0001-osc-MDT0001:
error opening log id [0x2780f:0x1:0x0]:0:rc = -2

The patch fixes logic and also adds/changes debugging for
llog and osp.

Lustre-change: https://review.whamcloud.com/37102
Lustre-commit: a4f049b96562fd502b1948fb082767351e040a1c

Cray-bug-id: LUS-8193
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I9463223a1ea904b96643b19e1580f5894142c12b
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38541
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13274 uapi: fix build on older kernels

The recent changes to lustre_user.h broke building on older kernels,
because it resulted in the user tools including both <linux/quota.h>
and <sys/quota.h>, which results in conflicts for the declaration of
the quotactl() function.

Also, restore the compat declaration of __ALIGN_KERNEL(), though it
only in lustre_user.h, since it is included in other headers anyway.

Test-Parameters: trivial
Fixes: 0417dce9fc75 ("LU-13274 uapi: make lustre UAPI headers C99 compliant")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9b31cabaf2732eb5872e88686e6af7d12e7d3564
Reviewed-on: https://review.whamcloud.com/38520
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12030 tests: Properly detect debug kernel use on rhel7.6

kmalloc-128 slab seems to be gone, so let's use dma-kmalloc-128
instead.

Lustre-commit: 673e8b0b3c404b14a02e7e874c3aed991bc52eb3
Lustre-change: https://review.whamcloud.com/34342

Test-Parameters: trivial testlist=conf-sanity env=ONLY=63
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: Ice7f350ba2bc6cc733c0a98b0037e6f0980216c9
Reviewed-on: https://review.whamcloud.com/38572
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12846 mdd: return error while delete failed

Since we use a global buffer, So avoid to replace the
index name while iterate the orphan directory. Also
return error code in mdd_orphan_destroy while dt_delete
failed. Else will cause a dead loop.

Fixes: e1ace3751f ("LU-8514 mdd: transaction failure should be checked")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I6fc3e992333ffa61900074309223555264cfe66b
Reviewed-on: https://review.whamcloud.com/37570
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>

LU-11986 libcfs: add compat for d_hash_and_lookup()

Add compatibility for older kernels that do not export the
d_hash_and_lookup() function, even though it exists forever.

Fixes: 9d42660e173e ("LU-11986 lnet: properly cleanup lnet debugfs files")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I10d6547f6b17665880cacfbf87d4dbfd386b92ff
Reviewed-on: https://review.whamcloud.com/38529
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12236 gss: remove unused code in gss_svc_upcall.c

Delete rsc_flush() related functions which are never
used.

This patch is back-ported from the following one:
Lustre-commit: 25b0bf5a23032394055b7b94b3169f5cf4068570
Lustre-change: https://review.whamcloud.com/34794

Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small,sanity-sec
Change-Id: Iedd6339b5fafdea81147c83e5f0499fa3ad60251
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-on: https://review.whamcloud.com/38476
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13228 clio: mmap write when overquota

Flagging client by overquota flag should not
cause mmap write access to sigbus the app.

Lustre-change: https://review.whamcloud.com/37495
Lustre-commit: b651089f859e8269af7272b91f5e60aa25f24226

Cray-bug-id: LUS-8221
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Change-Id: I29d5901fa5078b5cfca40391a02531cf27efce93
Reviewed-on: https://review.whamcloud.com/38292
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12402 lnet: handle recursion in resend

When we're resending a message we have to decommit it first. This
could potentially result in another message being picked up from the
queue and sent, which could fail immediately and be finalized, causing
recursion. This problem was observed when a router was being shutdown.

This patch uses the same mechanism used in lnet_finalize() to limit
recursion. If a thread is already finalizing a message and it gets
into path where it starts finalizing a second, then that message
is queued and handled later.

Lustre-change: https://review.whamcloud.com/35431
Lustre-commit: ad9243693c9a5a5b2c34165ad853ddf5ceec4617

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Iace64c7ddb1f56a0a63b030df6a5ab103ae6c645
Reviewed-on: https://review.whamcloud.com/38367
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12436 lov: return error if cl_env_get fails

When cl_env_get() fails with an error return the error.

Lustre-change: https://review.whamcloud.com/35229
Lustre-commit: a7997c836bbfe2a0674007f1c23b9593e596e0ba

Test-Parameters: trivial
Cray-bug-id: LUS-7310
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Ia065aeb142a772f4d620b84111af423e27c06b90
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38360
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13369 kernel: kernel update RHEL7.7 [3.10.0-1062.18.1.el7]

Update RHEL7.7 kernel to 3.10.0-1062.18.1.el7.

Test-Parameters: trivial clientdistro=el7.7 serverdistro=el7.7

Change-Id: Ifc00fca35a0ad28ba8326e56e693ea39360a8114
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38241
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13347 kernel: RHEL 7.8 server support

This patch makes changes to support new RHEL 7.8 release
for Lustre server (kernel 3.10.0-1127.el7).

Test-Parameters: trivial clientdistro=el7.8 serverdistro=el7.8

Change-Id: I4817fd2f9512111aa7d26109454104945fd2778f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38144
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12816 ptlrpc: ptlrpc_register_bulk LBUG on ENOMEM

Another path through ptl_send_rpc() can cause the assert reported
in LU-10643. The assertion in ptlrpc_register_bulk() on
!desc->bd_registered fails when an rpc is resent and the first
send attempt failed to successfully attach the reply buffer. The
bulk error cleanup in ptl_send_rpc() does not reset the
bd_registered flag.

Lustre-change: https://review.whamcloud.com/36309
Lustre-commit: e6225c07ce4c0037a127a41b2bc539364dfd1f4d

Cray-bug-id: LUS-7946
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I474211f196ea9bd83a036747e25c91c37c85ffbb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38266
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13063 tests: stop running sanity test 411

sanity test 411 hits a kernel bug for RHEL 8.1. Since this
is an issue with the kernel and not Lustre, let's stop
running this test until the kernel is patched. Thus, we
need to add sanity test 411 to the ALWAYS_EXCEPT list.

Also change the ALWAYS_EXCEPT condition for test smoke for
lnet-selftest to be based on kernel version and not
architecture, so that the custom test for this patch can
pass.

This patch is back-ported from the following one:
Lustre-commit: 34e4c37474b3d9328aac1dd9228a018ac7d4f47e
Lustre-change: https://review.whamcloud.com/37270

Test-Parameters: trivial clientdistro=el8.1
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I60174dcd4776b53ac5b44be6c208d40e1f022445
Reviewed-on: https://review.whamcloud.com/38260
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12661 tests: skip sanity 817 if kernel version >= 4.14

sanity test_817 is in the ALWAYS_EXCEPT list for aarch64,
however it's failing because the test was done on kernel-alt
which is 4.14.x, it's not related with the architecture.

On new kernels nfsd is not releasing the file after write,
it will fail with ETXTBSY regardless of whether the nfs export
is backed by a lustre mount or not.

Skip the test on new kernels for now.

This patch is back-ported from the following one:
Lustre-commit: 4fed33473ca2964ff19f61fdb8501b2210f923de
Lustre-change: https://review.whamcloud.com/36712

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ie18ceb961eee2313fca7d60a35159a7496075029
Reviewed-on: https://review.whamcloud.com/38259
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12852 pfl: restrict the stripe count correctly

In function lod_get_stripe_count(), when restricting the stripe
count to the maximum xattr size, the xattr overhead should be
taken into count correctly.

Lustre-change: https://review.whamcloud.com/36947
Lustre-commit: 6dc37759cfb22727ac5d776c38b72e8638563fd8

Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ief548e47ce4d375f2e189860ccfe05d0f3c7e890
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37512
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13224 utils: expose llapi_param* functions

Added an interface to find files in sysfs,procfs,etc.,
and read them into a file or buffer.
The interface is meant to return the same results as
the "lctl list_param" and "lctl get_param" commands,
but doesn't require starting an lctl process.

The interface is being used by lctl for parts of
its "list_param" and "get_param" commands.

The output of lctl get_param is altered slightly.
Previously, for some multi-line files lctl would
print the param name and first line of the file on
the same line such as:

param=line 1
line 2
line 3

Now multi-line files consistently put a newline
after the param name and before the content such as:

param=
line 1
line 2
line 3

Added debug output to failing test sanity 300d.

Lustre-commit: 9b44cf70a95b2ee97a17697dc97fbe462ad1f5b9
Lustre-change: https://review.whamcloud.com/37545

Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Change-Id: I2726b27b0042d58c97284f8348970deb6efc43d1
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38239
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13324 llite: page delete race

Page could be raced by truncate during its deletion by
another thread, check vmpage mapping before accessing it.

Lustre-change: https://review.whamcloud.com/37793
Lustre-commit: 8ccaed513006f0582578ff42c9917b5aa76f2f4b

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I3172f5ffd3928926b16ab6fd7362b05da0c7cfd5
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38111
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12904 gss: struct cache_detail readers changed to writers

Linux 5.3 changed struct cache_detail readers to writers
SUNRPC: Track writers of the 'channel' file to improve ...

kernel-commit: 64a38e840ce5940253208eaba40265c73decc4ee

This patch is back-ported from the following one:
Lustre-commit: 81fb8bc7d214394bbc504379a5a84441e871b60a
Lustre-change: https://review.whamcloud.com/36580

Test-Parameters: trivial
Cray-bug-id: LUS-8042
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I7750303937cd6fc560e458efa79f25e521fefec7
Reviewed-on: https://review.whamcloud.com/38193
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13355 crypto: crypto engine wrappers in libcfs

libcfs has wrappers in order to be able to use crypto engines as
standard crypto modules. But libcfs should not be considered as the
owner of these implementations.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1e22907c10837042606095ed6089c3c675fffe27
Reviewed-on: https://review.whamcloud.com/38205
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13347 kernel: new kernel [RHEL 7.8 3.10.0-1127.el7]

This patch makes changes to support new RHEL 7.8 release
for Lustre client.

Test-Parameters: trivial clientdistro=el7.8

Change-Id: I82f89495d5ab1d46a539a016a899307d7c8f37b7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38143
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13101 llite: eviction during ll_open_cleanup()

On error ll_open_cleanup() is called while
intent lock remains pinned. So eviction can
happen while close request waits for a mod rpc slot.

Release intent lock before ll_open_cleanup()

Lustre-change: https://review.whamcloud.com/37096
Lustre-commit: 6d5d7c6bdb4f19f9db485a774d9259d452cf220e

Change-Id: Ia422351f3f54fc652078f742f2ead0bf278c9d17
Cray-bug-id: LUS-8055
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-on: https://review.whamcloud.com/37096
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38147
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>

LU-13136 dom: check read-on-open buffer presents in reply

The ll_dom_finish_open() uses req_capsule_has_field() wronly,
it check only format but not buffer presence in reply, that
causes unneeded console errors about missing buffer later in
req_capsule_server_get()

Patch replaces that with req_capsule_field_present() to check
if server pack that field in reply or not and properly skip
responses from an old server.

Lustre-change: https://review.whamcloud.com/37249
Lustre-commit: 58bea527100b50abf3df2dbab0ed6d6b42b69d86

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ia6114879c90e3e6b8c5020c4912e988cad90df30
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38188
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13131 osc: Make sure we don't accidentally deprioritize hp requests

Looks like in some cases HP requests could migrade off HP list back
onto urgent list

Lustre-change: https://review.whamcloud.com/37967
Lustre-commit: f1c55f97d2d4b21fe87031b2bddfb9813f7df488

Change-Id: I96bf0a3a005b166f34dba215463c0806dfe2526a
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37918
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>

LU-12773 tests: sanity test_805 Use do_facet

do_node cannot really work with $SINGLEMDS, that's the
facet name.

This fixes error message below (and a following syntax error):
mds1: ssh: Could not resolve hostname mds1: Name or service not known

Lustre-change: https://review.whamcloud.com/36204
Lustre-commit: fa8b581aea27acc00f6ac48e76ec261dddf8631a

Fixes: 106abc184d8b ("LU-8856 osd: mark specific transactions netfree")
Test-Parameters: trivial fstype=zfs
Change-Id: I0d842dbccbfd934c524ae01cca7399dd52158064
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37901
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13377 llite: fix dead loop for short write

|->vvp_io_write_start()
|->__generic_file_write_iter()
    |->iov_iter_advance() if write succeed()
  |->vvp_io_write_commit()
     |->update ci_nob

The problem is we will move forward iov iter inside
__generic_file_write_iter(), but @ci_nob will be updated
after vvp_io_write_commit().

If out of quota or some other problems happen, this could
cause a mismatch with @ci_nob and @vui_iter.

And @vui_iter->count will be reset using @ci_nob in
iov_iter_reexpand(), this will make @vui_iter->count
more than what it really left, and we could dead loop
in vvp_mmap_locks() if IO need be retried or restarted:

vvp_io_write_lock+0x45/0x80 [lustre]
cl_io_lock+0x5f/0x3d0 [obdclass]
cl_io_loop+0x92/0x190 [obdclass]
ll_file_io_generic+0x7b3/0xc90 [lustre]
ll_file_aio_write+0x12d/0x1f0 [lustre]
ll_file_write+0xce/0x1e0 [lustre]
vfs_write+0xc0/0x1f0
SyS_write+0x7f/0xf0
system_call_fastpath+0x22/0x27

Lustre-change: https://review.whamcloud.com/38018
Lustre-commit: 13dfe0df4956afb50b323a11615b0b34ed014e53

Change-Id: I5fb4c18cf02fb17bf50122b63decacef678caa01
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-on: https://review.whamcloud.com/38163
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12988 ldiskfs: skip non-loaded groups at cr=0/1

cr=0 is supposed to be an optimization to save CPU cycles,
but if buddy data (in memory) is not initialized then all
this makes no sense as we have to do sync IO taking a lot
of cycles. also, at cr=0 mballoc doesn't store any avaibale
chunk. cr=1 also skips groups using heruistic based on avg.
fragment size.
it's more useful to skip such groups and switch to cr=2 where
groups will be scanned for available chunks.

using sparse image and dm-slow virtual device of 120TB was
simulated. then the image was formatted as OST and filled
using debugfs to mark ~85% of available space as busy.
mount as OST w/o the patch couldn't complete in half an hour
(according to vmstat it would take ~10-11 hours). with the
patch applied mount took ~20 seconds.

Lustre-change: https://review.whamcloud.com/36891
Lustre-commit: 6a7a700a1490dfde6b60c2fb36df92a052059866

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I88c8c1b01b386af0fa438bfeb97acb6110bd00ec
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13157 mdd: migrate readlink from wrong place

In osd_read(), if symlink name buf length is smaller than i_size,
return -EOVERFLOW, and compare inline data length with i_size
instead of buflen.

Updated sanity.sh 230b.

Lustre-change: https://review.whamcloud.com/37285
Lustre-commit: a3b30423c6067b3e8644ecfb3269f8837af7e4cd

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia13b1f1079efc4ebd22ec400f1a909ff7ec2095d
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38112
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12345 ldiskfs: optimize nodelalloc mode

We found performance regression when using bigalloc with "nodelalloc"
(1MB cluster size):

1. mke2fs -C 1048576 -O ^has_journal,bigalloc /dev/sda
2. mount -o nodelalloc /dev/sda /test/
3. time dd if=/dev/zero of=/test/io bs=1048576 count=1024

The "dd" will cost about 2 seconds to finish, but if we mke2fs without
"bigalloc", "dd" will only cost less than 1 second.

The reason is: when using ext4 with "nodelalloc", it will call
ext4_find_delalloc_cluster() nearly everytime it call
ext4_ext_map_blocks(), and ext4_find_delalloc_range() will also scan
all pages in cluster because no buffer is "delayed".  A cluster has
256 pages (1MB cluster), so it will scan 256 * 256k pags when creating
a 1G file. That severely hurts the performance.

Therefore, we return immediately from ext4_find_delalloc_range() in
nodelalloc mode, since by definition there can't be any delalloc
pages.

The same optimization also added for ldiskfs_find_delayed_extent()
function that improve performance dromaticaly.

Here is results of testing on two node system.
Without the patch:
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 56.30 0.06 0.00 43.63

Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await r_await w_await svctm %util
sds 0.00 0.00 0.00 1174.00 0.00 4.59
8.00 0.84 0.71 0.00 0.71 0.01 1.20

With patch:
08/29/2018 01:13:22 AM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
0.00    0.00    4.13   30.37    0.00   65.50

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s      wMB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm %util
sds               0.00     0.00    0.00 54117.82     0.00     211.43
8.00   152.59    2.82    0.00    2.82   0.02 99.01

Lustre-change: https://review.whamcloud.com/34982
Lustre-commit: af48ae8bff289b2bc083a888efeafa3c48df91e2

Cray-bug-id: LUS-5835
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: Ie33410d4481778ee4f76a054ab8cfc11cc19a0ed
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37538
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11114 llite: Update mdc and lite stats on open|creat

Increment "create" counter in mdc/<instance>/md_stats, and
"mknod" counter in llite/<instance>stats when an open with
the CREAT flag results in a newly created file.

The mknod counter is chosen for consistency with
patch http://review.whamcloud.com/20246
"LU-8150 mdt: Track open+create as mknod"
but the mdc counter set does not include mknod.

Lustre-change: https://review.whamcloud.com/36948
Lustre-commit: 4b8518ee4fa542f45fcdaeaec580d858dfcaf137

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: If082b911e415c0bc46248728e47ce0f37b9efa83
Reviewed-on: https://review.whamcloud.com/38158
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>

LU-12580 lov: fix out of bound usercopy

When handling ioctl LL_IOC_LOV_GETSTRIPE, the user
could pass a limited buffer which is bigger than
lov_comp_md_size(), it will crash the client because
we are doing the usercopy with the user provided buffer
size.

Make sure the copy works, also for the PFL file,
we should only copy the chosen component.

Lustre-change: https://review.whamcloud.com/37469
Lustre-commit: 2f1beb33144523467b596f4b6fab882b0a839187

Change-Id: I92bcf6d7b7f7a4387a9936a0b58332e50a88e542
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38051
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12198 libcfs: always copy ioctl header back to user

lnetctl_get_peer_list fills back the required size in header if the
given buffer was too small. Userspace needs the info back to grow
the buffer and try again.

Note we only replace err on failure if err was previously not set

Lustre-change: https://review.whamcloud.com/37559
Lustre-commit: 9e02ef474f8caa833d6a1b5e0068d5323a57e8c4

Fixes: fba98579efc4 ("LU-6202 libcfs: replace libcfs_register_ioctl with a blocking notifier_chain")
Signed-off-by: Dominique Martinet <dominique.martinet@cea.fr>
Change-Id: I2b6e319aceeb00d488572053d27023891afe1928
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37720
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13294 libcfs: incorrect rotor behaviour

Signed int cpt rotor is set to -1 on initialization.
cfs_cpt_spread_node() improperly handles this value
via "if (!rotor--)" check. The condition is never true
with negative rotor values, so for_each_node_mask()
only exits with node = MAX_NUMNODES.

kmalloc_node() attempts to determine the zonelist based
on the passed node id and maps MAX_NUMNODES to some
random pointer. Crash.

BUG: unable to handle kernel paging request at 0000000100002007
IP: [<ffffffff847c0da7>] __alloc_pages_nodemask+0x97/0x420

Lustre-change: https://review.whamcloud.com/37709
Lustre-commit: f8aa86dd1622804d81020a7dbb1116f276b340f3

Change-Id: I4df74e394bdfc2a918d66aa12e6852ff0f6738ab
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-8492
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38049
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11986 libcfs: provide QSTR_INIT compat macro

Provide a compat macro for QSTR_INIT() for older kernels.

Fixes: 9d42660e173e ("LU-11986 lnet: properly cleanup lnet debugfs files")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ice19a4dad8456551ba398034a8d3942068006512
Reviewed-on: https://review.whamcloud.com/38090
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13148 tests: clean up sanity 56ra add debugging

Consolidate duplicate code from sanity.sh test_56ra() into a
helper function to make it easier to see what is being run.

Print out the before and after values for each test.

Skip test_56ra for versions older than 2.12.4, since it was
backported in commit v2_12_3-24-gd55982d842.

Test-Parameters: trivial testlist=sanity env=ONLY=56ra,ONLY_REPEAT=50
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia28c1b556f53ea88643805cbf4ada725a53ebbe5
Reviewed-on: https://review.whamcloud.com/38021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12580 lov: fix typo in lov_comp_md_size

If the component of a PFL file is not initialized,
we should use 0 as the stripe size when calculating
the LOVEA size.

Lustre-change: https://review.whamcloud.com/37493
Lustre-commit: d41716533682ed88b8a77654f9b5b050ef5c672c

Change-Id: I4ff5f4a78bc1d432cc1ac6fa3733461bd6b762e6
Fixes: 62f64a1077 ("LU-9489 lod: keep minimum LOVEA size")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38050
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13260 lov: fix size check when stripe is zero

Set correct max size while stripe is zero.

Lustre-change: https://review.whamcloud.com/37623
Lustre-commit: fe57ce6adf1a00e14269b230d07a4548a58d77c3

Fixes: f3f6515562 (LU-8998 lov: add composite layout unpacking)
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I9b76283fcc65f58e3be6adf49f035236687ac85c
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38048
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11758 osp: remove assertion from statfs

Sequence can't be changed or overflowed
in case of IDIF. Thus don't tigger kernel
panic for below case:
last_created [0x100000001:0x15:0x0], next_fid [0x100000000:0xfffffff6:0x0]
The same assertion that excepts IDIFs exists
in osp_fid_diff.
Also the patch is adding several optimizations
in osp_precreate_send.

Lustre-commit: bcfd0e040d1536410ba6c301f64d4f8ea6a8797a
Lustre-change: https://review.whamcloud.com/33832

Change-Id: I3966dfc621999d065c9b485d387938085fccb140
Cray-bug-id: LUS-2386
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-on: https://review.whamcloud.com/37993
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12299 libcfs: fix panic for too large cpu partions

If cpu partions larger than online cpus, following calcuation
will be 0:

num = num_online_cpus() / ncpt;

And it will trigger following panic in cfs_cpt_choose_ncpus()

LASSERT(number > 0);

We actually did not support this, instead of panic
it, return failure is better.

Also fix a invalid pointer access if we failed to init @cfs_cpt_table,
as it will be converted to ERR_PTR() if error happen.

Lustre-change: https://review.whamcloud.com/34864
Lustre-commit: 77771ff24c03a59fc96a7f41199a6b73530a418a

Change-Id: I49daadd8f0c7d22aa78d08248d8c085781740768
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37332
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13160 tests: fix sanity-hsm monitor setup

On RHEL8, even we are using pdsh -R ssh,
the ssh still waits for the remote cat process
to finish.
Use the subshell to avoid the time out.

Lustre-change: https://review.whamcloud.com/37595
Lustre-commit: 6724d8ca58e9b8474a180b013a4723cbdd8900d9

Change-Id: Id5b8d492b5ce9a235da73448ade475ade145bbed
Test-Parameters: trivial clientdistro=el8.1 testlist=sanity-hsm
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37773
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12775 test: reorder 'tar' command options

'tar' in RHEL8 is stricter in command option order.

Test-Parameters: trivial \
envdefinitions=ONLY="32c" \
clientdistro=el8.1 serverdistro=el8.1 \
mdscount=2 mdtcount=4 testlist=conf-sanity

Lustre-change: https://review.whamcloud.com/36907
Lustre-commit: f3e101a36310c0c2b9d516c09ec0166eb24524d2

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I814203808efae4a746166abd3ba08f2bc5fce8f7
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13274 uapi: make lustre UAPI headers C99 compliant

Attempting to compile strict C99 user land applications or
libraries with the Lustre UAPI headers will fail. These same
errors can be seen by enabling CONFIG_UAPI_HEADER_TEST as well.
Update the Lustre UAPI headers to be compilable with -std=c99.
Enhance our current test covering UAPI header handling.

For OpenSFS branch we can't include <linux/stat.h> since we support
kernels before struct statx existed and they will collide with the
special definitions in lustre_user.h.

Lustre-change: https://review.whamcloud.com/37678
Lustre-commit: 7a7309fa849577ddd5a4f6bb5bfb69e84a7fec89

Signed-off-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Change-Id: Ifb0da33180dc3c7e116d6bf2b7f603ad0528277a
Reviewed-on: https://review.whamcloud.com/37973
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11510 lfs: migrate a composite layout file correctly

The patch fixes the following issues:
- in function migrate_open_files(), "layout" pointer should be used
  instead of "param" pointer to tell whether a comp file should be
  created or not, because "param" pointer is always not null and
  the composite layout file will never be created;
- make --copy and --yaml options work correctly in lfs_migrate tool;
- when a composite layout file is migrated, "--copy" option will be
  added to preserve its layout in both lfs_migrate and "lfs migrate",
  and in such situation, pool name will be saved as well;
- when a file is restriped with -R option by lfs_migrate, the file
  will be set with its parent's stripe by default, by adding
  "--copy $parent_dir" option;
- do some code cleanup in lfs_migrate and sanity.sh test_56wb/c

sanity.sh test_56xd/xe are added to verify this patch.

Lustre-change: https://review.whamcloud.com/36082
Lustre-commit: 8bedfa377fbd0c9f1b6ea2c40d36fdcaa52137df

Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I85779c69e74444eb869f28add4363ad3a6835b97
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37814
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13090 utils: fix lfs_migrate -p for file with pool

If "lfs_migrate -p <pool>" is run to migrate a file with an existing
pool, the given pool is overridden by the existing pool from the file
during migration. Fix this to use the OST pool requested by the user.

Don't print a warning about deprecated -n option if --dry-run is used.

If a pool is specified, use it with "lfs df" to find OST free space.

Change temp filename to work better with new DNE "crush" hash.

Don't return an error if falling back to rsync and no links are found.

Add test for "lfs_migrate -p" and update man page and usage to match.
Clean up debug-level helpers in test-framework.sh.

Lustre-change: https://review.whamcloud.com/37067
Lustre-commit: 128137adfc539dd2dd92040c14a63ff27f969820

Test-Parameters: trivial testlist=ost-pools
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ief69a620fc969aeff24ec0633a3314c3b83ebbe5
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37813
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12157 utils: fix lfs_migrate output and testing

Don't pass the "-v" option through to "lfs migrate", as this causes
the filename to be printed twice when run with the "-v" option.

Don't use "echo -e" to process escape characters in filenames unless
this is needed, but add it where it is needed.  Don't add ANSI escape
characters to the output.  The output previously looked like:

     /mnt/testfs/l0: stripe count=1,size=1048576,pool=/mnt/testfs/l0
     done migrate
     nr[K/mnt/testfs/l1: /mnt/testfs/l1: already migrated ...
     nr[K/mnt/testfs/l2: /mnt/testfs/l1: already migrated ...
     nr[K/mnt/testfs/l3: /mnt/testfs/l1: already migrated ...

Print out the "pool=" and "mirror_count=" parameters only if needed.

Fix "-A" option to round up the number of stripes when the migrated.
Skip sanity test_56xc 1GB test if there is not enough space on OSTs.

Fixes: 60c5bc2502 ("LU-8235 scripts: pass unrecognized options to lfs migrate")
Fixes: 80a2ff7137 ("LU-6051 utils: allow lfs_migrate to handle hard links")
Fixes: 99d7a8ed43 ("LU-8207 scripts: add auto-stripe option to lfs_migrate")

Lustre-change: https://review.whamcloud.com/34592
Lustre-commit: 9b8e8e0e54e5055c02469cb16d186c94fa2040e0

Test-Parameters: trivial fstype=zfs testlist=sanity envdefinitions=ONLY=56
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I059e7daeb2fa82e7607fd9d862797433053ebbe5
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37812
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13169 tests: add ONLY_REPEAT parameter to repeat subtests

Add the ONLY_REPEAT environment variable, to allow tests specified
by ONLY to be run multiple times, to ensure that the test is passing
consistently (or fixing an intermittent bug).  This is faster than
restarting the test session multiple times for only a few subtests.

Have the iteration around the subshell started for run_one() so that
any registered stack_trap EXIT calls are triggered between iterations,
the fail_loc is reset, grant/health/error checks are done, and so on.

Remove $tdir and $tfile files after each iteration to avoid failures
with the subsequent subtest runs.  For tests that do not follow the
standard naming convention for test directories and files, they need
to be updated to use $tdir and $tfile, which is good in any case.

YAML output splits each iteration into a separate subtest for Maloo.
The output from run_one() is appended to a single output file for all
iterations so all output is captured instead of just the last one.

The iterations will continue until $ONLY_REPEAT loops pass, or until
the subtest hits an error.  Trying to continue for all iterations in
the face of errors would likely end up with all of later iterations
failing also due to leftover state from the previous failure, and the
goal is for the subtests to pass consistently.  If we are trying to
determine rates of intermittent failures, this can be computed using
1/num_passes about the same as num_failures/ONLY_REPEAT iterations.

Rename variables in subtests to avoid clash with testnum, testname,
and TESTNAME, and use them consistently in functions and subtests.

Lustre-change: https://review.whamcloud.com/37321
Lustre-commit: e16e3d46ee8c44e691c5cd3d25161f2f297fa0fd

Test-Parameters: testlist=sanity envdefinitions=ONLY=27l,ONLY_REPEAT=100
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Change-Id: I5449590dc3e25c113b059974fb7b96c892434380
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Charlie Olmstead <charlie@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37586
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13296 obd: make statfs cache working again

Once statfs raced on mutex, lets read a cached data instead
of trash.

Lustre-change: https://review.whamcloud.com/37753
Lustre-commit: 7281635521a823548d497bce2f19acfa3318dfe9

Test-Parameters: testlist=sanity envdefinitions=ONLY=423,ONLY_REPEAT=500
Fixes: 1c41a6ac390b ("LU-12368 obdclass: don't send multiple statfs RPCs")
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: I268782875c30c078f239c194f69cdf7506d66169
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37819
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13261 mdt: PFL layout changed while accessing

The PFL layout EA could be enlarged when the corresponding layout
of some IO range is started to be written, which can cause other
thread to get incorrect layout size at "mdt_intent_layout" and cause
"mdt_lvbo_fill" to fail checking the real layout size.

In Lustre, "ldlm_handle_enqueue0" has processed the error "-ERANGE"
and it will retry after expanding the layout buffer size, then it
only needs to decrease the debug level of the log in "mdt_lvbo_fill"

Lustre-change: https://review.whamcloud.com/37684
Lustre-commit: 35d01a0fc7b2933d589f5a6bc4878382cbc15b52

Change-Id: Iad722d1dac187f57ae77606a4d4587525412cd68
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37821
Tested-by: Maloo <maloo@whamcloud.com>

LU-11891 utils: getstripe use --mdt-index consistently

LU-10856 fixed most usages of "warning: '-M' deprecated,
use '--mdt-index' or '-m' instead" but missed a few in
cases in sanity test_271d, test_271e, and test_271f.
Fix those tests to use "--mdt-index".

Also, lfs has a few places were the usage of "--mdt-index"
and "--mdt" is inconsistent. Fix those options to be used
consistently across all commands.

Lustre-change: https://review.whamcloud.com/34116
Lustre-commit: 88d8f0f86bd4994e07aa12bd00cbc7ad3192205c

Fixes: 6c617a3d56 ("LU-10856 tests: remove deprecated lfs ...")

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I013b2198f3a39533da9a0067a0bf5846604b3052
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37728
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11810 misc: allow Fixes: tag in commit signoff block

Allow the "Fixes:" tag in the signoff block of the commit message.
The Fixes: tag should contain a valid git commit hash, and be
followed by a description of the original patch.

Lustre-change: https://review.whamcloud.com/33888
Lustre-commit: 4d6bff6be51ff6e336f73b2822ac5100254b9431

Fixes: 5760b34c48b4 ("LU-1145 test: add Test-Parameters tag")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I49f712dd8da173510e5941b66eb050e53a1cab07
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-on: https://review.whamcloud.com/37724
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13071 lnet: reduce log severity for health events

No need to print an error when the health of an interface is
reduced. Changed it to debug level.

Lustre-change: https://review.whamcloud.com/37002
Lustre-commit: 5567aaba1086217e021938e5f9543c640b2d007c

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ia60ade12efab732ea4b0388a3803976bf65938ab
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37718
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-11269 ptlrpc: do not expose transient IDLE state

to avoid cases when anyone sending an RPC observes the connection
in this state while it's going to reconnect right away.

Lustre-change: https://review.whamcloud.com/37523
Lustre-commit: ea8d2ecc783fbaff12c581935ac426b9b8567031

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9ca89051c4176fe321262f8b2f52969c382e401e
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37649
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13110 kernel: kernel update SLES12 SP4 [4.12.14-95.45.1]

Update SLES12 SP4 kernel to 4.12.14-95.45.1 for Lustre client.

Test-Parameters: trivial clientdistro=sles12sp4 \
envdefinitions=LNET_SELFTEST_EXCEPT=smoke,SANITY_EXCEPT="103a 817"

Change-Id: I1f7024465b4b6334488b7314f1073fafa10331d6
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37124
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13191 osp: handle -EROFS in osp_sync_interpret()

Upon OST disk failure, osp_sync_interpret() may get -EROFS,
which is a valid errno.

Lustre-change: https://review.whamcloud.com/37404
Lustre-commit: 868089cd309506719b814afecebf825effc6c93f

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I5c3cff3019aa47c6d5803f0f0b373bc704f18118
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37516
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-10198 llog: keep llog handle alive until last reference

Llog handle keeps related dt_object pinned until llog_close()
call, meanwhile llog handle can still have other users which
took llog handle via llog_cat_id2handle()

Patch changes llog_handle_put() to call lop_close() upon last
reference drop. So llog_osd_close() will put dt_object only
when llog_handle has no more references.
The llog_handle_get() checks and reports if llog_handle has
zero reference.
Also patch modifies checks for destroyed llogs, llog handle
has new lgh_destroyed flag which is set when llog is destroyed,
llog_osd_exist() checks dt_object_exist() and lgh_destroyed
flag, so destroyed llogs are considered as non-existent too.
Previously it uses lu_object_is_dying() check which is not
reliable because means only that object is not to be kept in
cache.

Lustre-change: https://review.whamcloud.com/37367
Lustre-commit: d6bd5e9cc49b3bb9901ada503107e8b0eca44f7e

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: If7df41646c243c0d40b20a30a33e86c688d24508
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37514
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12651 osc: always call update_next_shrink

Call update_next_shrink in case of clients not
supporting grant shrinking or clients with grant
shrinking explicitely disabled. Otherwise
osc_grant_work_handler() schedules itself immediately
after its completion causing excessive CPU consumption.

Fixes: 3e070e30a98d ("LU-8708 osc: enable/disable OSC grant shrink")

Lustre-change: https://review.whamcloud.com/37429
Lustre-commit: 117f587bc3e60f4dd1c939f8488e43cb752c12ca

Cray-bug-id: LUS-8460
Change-Id: I507b3d10dd5374772456853098bc26053cbd140d
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37572
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13163 mdc: new kernel function xa_is_value()

xa_is_value() is added in kernel 4.19-rc6 to replace
radix_tree_entry_exceptional().

This patch is back-ported from the following one:
Lustre-commit: a0a3a29deb82656f9639f46847deac2689973893
Lustre-change: https://review.whamcloud.com/37399

Test-Parameters: trivial clientdistro=el8.1 \
envdefinitions=ONLY=65i mdscount=2 mdtcount=4 \
testlist=sanity,sanity,sanity,sanity,sanity

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If89aa19c37af8a67debe782d1c77f4ef4dc6f923
Reviewed-on: https://review.whamcloud.com/37481
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13142 lod: cleanup layout checking

Cleanup layout checking in lod layer and lfs command-line utility,
for DoM components.

Lustre-change: https://review.whamcloud.com/37267
Lustre-commit: 4ce8a29d8bfc5b77893b642cdf2c33ceed960866

Reported-by: Clement Barthelemy <clement.barthelemy@nextino.eu>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ib8b184a31d26442ed10241dc12a0452e5243d0e8
Reviewed-on: https://review.whamcloud.com/37410
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13152 llapi: llapi_layout_get_by_xattr groks DoM

llapi_layout_get_by_xattr() function must be updated to handle
lov component with LOV_PATTERN_MDT pattern.

Lustre-change: https://review.whamcloud.com/37269
Lustre-commit: 3e23353201a753104d1fcdab28353646e40644dc

Signed-off-by: Clement Barthelemy <clement.barthelemy@nextino.eu>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6553e66cd4f3b5acc65790da94555350c98fe179
Reviewed-on: https://review.whamcloud.com/37409
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12474 tests: Do not run check_progs_installed for racer

it's run from within racer so racer is already there for sure

Lustre-change: https://review.whamcloud.com/35327
Lustre-commit: e7b7433571a748cdc651c5f50f01ff5ee0656c28

Change-Id: Ifd78cd051842c9663130b650c6e35d60332250e7
Test-Parameters: testlist=racer
Test-Parameters: trivial
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37521
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-13117 libcfs: fix to match right key in cfs_get_environ()

It does the memcmp() to match the environment variable
with the desired key, then accounts for the "=" when
calculating length. But it fails to check that the next
character is actually an equals sign. In the case of
any key which is also the prefix to some other variable

Also add debug information for debugging similar issue
in the future.

Lustre-change: https://review.whamcloud.com/37156
Lustre-commit: 31170f9ceca91684ea66e0b16757881563a8cf26

Test-Parameters: trivial
Change-Id: Ia2b4ccd1f10c89059cecc224d4e2ba8d1d75b825
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37396
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11607 tests: replace version/fstype calls in conf-sanity

The routine get_lustre_env() is available to all Lustre test
suites and sets an environment variable for the file system
type for MDS1 and OST1 and sets a variable for the Lustre
version of servers.

In conf-sanity, replace the calls to facet_fstype() and
lustre_version_code() for all server types defined in
get_lustre_env(). While doing this, replace SINGLEMDS with
mds1 in these calls.

Clean up around any modifications with
- converting spaces to tabs
- removing calls to return after skip() or skip_env()

Lustre-change: https://review.whamcloud.com/35721
Lustre-commit: b4c955fe72d8598c4eaf98b809ae42be94f8c40b

Test-Parameters: trivial testlist=conf-sanity
Test-Parameters: fstype=zfs testlist=conf-sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I17707883d46aa66c32e1229107646bc7a9df5e4e
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36942
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13154 test: skip sanity-quota 66 if MDS version < 2.12.4

Since LU-12826 landed after this version, add version check to
make interop test pass.

Lustre-change: https://review.whamcloud.com/37276
Lustre-commit: f99fa029fd904ac13f33ab82de37fd07a69aea84

Test-Parameters: trivial envdefinitions=ONLY=66 testlist=sanity-quota
Change-Id: I829f424b9bb103e18c06de6f797827f82e1874d1
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37350
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

New release 2.12.4

Change-Id: I5c3d94ca134daae770414b220966fe3e92e5a10d
Signed-off-by: Oleg Drokin <green@whamcloud.com>

New tag 2.12.4-RC2

Change-Id: I637563481546e29b6fd648e2269287cdd2577910
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-13194 tests: check server version sanityn 104

Check the server version before running sanityn test 104.
If the server version is less than 2.12.4, skip the test.

Lustre-change: https://review.whamcloud.com/37461/
Lustre-commit: a8b9a123fea3762b999e80f56fdbbdf2ea10e280

Fixes: d2f7cb7934a0 ("LU-12026 mdt: MDS stores atime|mtime|ctime")

Test-Parameters: trivial serverversion=2.11.0 serverdistro=el7 envdefinitions=ONLY=104 testlist=sanityn
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I625fb0163c078dc95ed670d169dc5744bc16d4e8
Reviewed-on: https://review.whamcloud.com/37420
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13145 lnet: use conservative health timeouts

Use more conservative lnet_transaction_timeout and lnet_retry_count
values by default. Currently with timeout=10 and retry=3 there is
only a 3s window for the RPC to be sent before it is timed out.
This has caused fault injection rather than fault tolerance.
Increase the default timeout to 50s with retry=2, which is hopefully
long enough to cover virtually all uses, but still allows LNet Health
to be enabled by default and resend before Lustre times out itself.

Fixes: 8632e94aeb7e ("LU-11816 lnet: setup health timeout defaults")
Lustre-change: https://review.whamcloud.com/37430
Lustre-commit: 361e9eaef13c0f472ad45388d3e147dabc32b737

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6bfc4d61cebab38c1554e1b42834b1f38fc34ba8
Reviewed-on: https://review.whamcloud.com/37390
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12593 osd: up i_append_sem during errors

There is a potential leak of i_append_sem during errors for
buffer head read and ldiskfs_joural_get_write_access() at
osd_ldiskfs_write_record().
The patch adds up(i_append_sem) for errors paths.

Lustre-change: https://review.whamcloud.com/37406/
Lustre-commit: 7599dd3d20d6bb4ee89634c5a76730481ca62470

Fixes: f832a7dc33c6 ("LU-12593 osd: zeroing a freshly allocated block buffer")
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I245d0c45af03519c66b75731e5d57f42de41fe95
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37445
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13099 lmv: disable statahead for remote objects

Statahead for remote objects is supposed to be disabled by
LU-11681 lmv: disable remote file statahead.

However due to typo it is not and statahead for remote objects is
accompanied by warnings like:
ll_set_inode()) Can not initialize inode .. without object type..
ll_prep_inode()) new_inode -fatal: rc -12

Fix the typo.

Test to illustrate the issue is added.

Lustre-change: https://review.whamcloud.com/37089
Lustre-commit: 68330379b01cb6bf9b24235a80a4666d24c0e343

Fixes: 02b5a407081c ("LU-11681 lmv: disable remote file statahead")
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Cray-bug-id: LUS-8262
Change-Id: Id9cc7f30ba75918658bf8eb1c8f3249993da6699
Reviewed-on: https://review.whamcloud.com/37370
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13121 llite: fix deadlock in ll_update_lsm_md()

Deadlock may happen in in following senario: a lookup process called
ll_update_lsm_md(), it found lli->lli_lsm_md is NULL, then
down_write(&lli->lli_lsm_sem). but another lookup process initialized
lli->lli_lsm_md after this check and before write lock, so the first
lookup process called up_read(&lli->lli_lsm_sem) and return, so the
write lock is never released, which cause subsequent lookups deadlock.

Rearrange the code to simplify the locking:
1. take read lock.
2. if lsm was initialized and unchanged, release read lock and return.
3. otherwise release read lock and take write lock.
4. free current lsm and initialize with new lsm.
5. release write lock.
6. initialize stripes with read lock.

Lustre-change: https://review.whamcloud.com/37182
Lustre-commit: 3746550282c865deebb07bfd92bcb4d1dabdc675

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ifcc25a957983512db6f29105b5ca5b6ec914cb4b
Reviewed-on: https://review.whamcloud.com/37325
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New RC 2.12.4-RC1

Change-Id: Ia0ed234bd5b7ffb74f1c1ec73190a34504f05496
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11385 odbclass: Handle gracefully if nsproxy is NULL

Gracefully handle the case if current->nsproxy is NULL:
check for the condition and return an error, avoiding attempts
to dereference the pointer.

Lustre-change: https://review.whamcloud.com/36802
Lustre-commit: 15278c6d32a5a9a7a2b8ac9e08c8702383e0c2ff

Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ia102d2bacdb0e54b0339985396447e6d25465c56
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37314
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12637 kernel: new kernel [RHEL 8.1 4.18.0-147.3.1.el8_1]

This patch makes changes to support new RHEL 8.1 release
for Lustre client.

Test-Parameters: trivial clientdistro=el8.1 \
envdefinitions=SANITY_EXCEPT="411 817" \
testlist=sanity

Lustre-change: https://review.whamcloud.com/36946
Lustre-commit: 97e93c8f267a7d9fb9ee6d96b040236172a7f247

Change-Id: Ifcc0a15c3ad9afa99b670641f91b23c1a5c0668e
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37186
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11385 lnet: check if current->nsproxy is NULL before using

A crash is seen at few sites in the function
rdma_create_id(current->nsproxy->net_ns, cb, dev, ps, qpt).
The issue is identified with the first param in this
function - current->nsproxy->net_ns. There is a
possibility that this value is NULL and resulting in
"kernel NULL pointer dereference" crash.

Handle the case of NULL value gracefully by adding
a check and using init_net if current or
current->nsproxy is NULL.

Lustre-change: https://review.whamcloud.com/34577
Lustre-commit: ef1783e282f6eba9d69b0957f1b5fed00be0cbd6

Change-Id: I06349e081f2c4ba0480b3924fc304f94ca765891
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37313
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12853 ptlrpc: zero session enviroment

handle_recovery_req() set le_ses for request processing,
and doesn't zero it after. This leads to accessing freed memory
at keys_fill() later.

The patch also adds a cleanup for xxx_env_info, makes them equal
and combines to a single function.

Lustre-change: https://review.whamcloud.com/36443
Lustre-commit: 2a620f07e23b3b044f429f049bcc5ffa96f6d844

Cray-bug-id: LUS-7676
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: Ifad95c1177258b6f71effe5fa815f68c8426c516
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37305
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13098 ptlrpc: supress connection restored message

if that happens on idling connection.

Lustre-change: https://review.whamcloud.com/37086
Lustre-commit: 7aa58847b94d0ebb2796774a2de2183ba7f8cc4b

Fixes: 5a6ceb664f07 ("LU-7236 ptlrpc: idle connections can disconnect")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I506665d427f3e77477f53e2d3059bcb1daaf0318
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37315
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12799 ptlrpc: return proper error code

from ptlrpc_disconnect_prep_req() using ERR_PTR()
as the callers expect.

Lustre-change: https://review.whamcloud.com/36282
Lustre-commit: 9e2620d75cce1e1b4855704ddd9a994ce8e8d650

Fixes: 5a6ceb664f07 ("LU-7236 ptlrpc: idle connections can disconnect")
Change-Id: I5493194a1f18f3d0b559921b7859bf835585ba58
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/37164
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13092 lbuild: include lbuild-{fc,rhel,sles} to SIGNATURE

We should include these files to calculate SIGNATURE, for example
bump kernel extra tags could happen there.

Lustre-change: https://review.whamcloud.com/37076
Lustre-commit: b39e1e6e3e4ea396ad842ec3695f45cfd5dfb79e

Test-Parameters: trivial
Change-Id: I2c62ad765d3c6a1b9e99affe3be95a404d6140c5
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37312
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12988 osd: do not use preallocation during mount

as cold mballoc cache can cause very lengthy search.

Lustre-commit: ae21fce625ec6cd134fa4764683f00bc692132cb
Lustre-change: https://review.whamcloud.com/36704

Change-Id: I821b023d392336f0085a96e821dc22e92dbf23b7
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/37155
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13087 target: init lcd last transno from reply data

Init lcd_last_transno value from reply data to keep it
valid so tgt_release_reply_data() will keep a slot with
the highest transno and on-disk data is not lost.

Lustre-change: https://review.whamcloud.com/37060
Lustre-commit: 52c1cbaa7db7505642b64b2d85448d506a444661

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id31b3b250616fb6afd3d145c31b12af30ac86be8
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37187
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>