LU-12202 misc: remove obsolete contrib files

nn-final-symbol-list.txt: originally used for Solaris port
mptlinux.spec.patch: for ancient SCSI drivers
01-dont-include-openib-initscript-rhel5.ed: old RHEL5 patch
rdac_spec: old driver file

I only wish there was a "really_trivial" keyword...

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If66db22caf604736949e18289af74cc4f0fa6613
Reviewed-on: https://review.whamcloud.com/34719
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11681 lfsck: misc fixes in inserting shard

* To insert a shard, we need to convert parent to a plain
directory by removing its LMV, and set back after insertion.
* If parent lost its LMV, after inserting shard, also copy
shard LMV to parent.
* Currenly remove striped directory LMV will remote its shard
LMV too, but if directory LMV is corrupt, removing will fail.
As a work around, don't remove shard LMV when removing striped
directory LMV, normally when we do this, we will remove shards.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ic0f6c2eadb607e82b281ffaf9eaa75284ab90a16
Reviewed-on: https://review.whamcloud.com/33928
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11681 lfsck: misc fixes for dangling entry repair

* lock child in dangling entry repair.
* if child is directory, create it as plain directory, because
  it may have shards already, which will be inserted later, besides,
  it may be remote, and creating striped directory remotely is not
  supported.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ide0d90f99e6d24f4a9507d98bbc7ad7f876a907d
Reviewed-on: https://review.whamcloud.com/33927
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11681 lfsck: read LMV from bottom object

LFSCK should read directory LMV from bottom object because it
doesn't want to read shard FIDs, and the allocated buffer size
can only contain struct lmv_mds_md_v1.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I2160b029ea4d19c5c056a0d06c340194a4811b00
Reviewed-on: https://review.whamcloud.com/33926
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11542 import: fix race between imp_state & imp_invalid

We set import to LUSTRE_IMP_DISCON and then deactive when
it is unreplayable. Someone may set this import up between
those two operations. So we will get a invalid import with
FULL state.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Ib4cec0bcaf6f4b221ba260edb94749a4e523f5e6
Reviewed-on: https://review.whamcloud.com/33395
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10756 osp: properly order sysfs registration

When generating udev events for import states I found that the
order of setting up the sysfs tree for osp was important. Setup
the osp obd device sysfs tree before lwp sysfs device. Always
call client_obd_cleanup() first on shutdown.

Change-Id: I29257f3e91f10f8266509b535e36cc8b62ce2362
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35464
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12627 ofd: reset fti_attr in ofd_lvbo_update()

This patch try to fix following panic:

(ofd_internal.h:440:tsi2ofd_info()) ASSERTION( info->fti_attr.la_valid == 0 ) failed:
(ofd_internal.h:440:tsi2ofd_info()) LBUG
[ 5321.108598] Call Trace:
[ 5321.109347]  [<ffffffffc06fc8bc>] libcfs_call_trace+0x8c/0xc0 [libcfs]
[ 5321.111342]  [<ffffffffc06fc96c>] lbug_with_loc+0x4c/0xa0 [libcfs]
[ 5321.113026]  [<ffffffffc147631a>] ofd_preprw+0xcfa/0x1160 [ofd]
[ 5321.114643]  [<ffffffffc0bb934c>] tgt_brw_write+0xc7c/0x1cf0 [ptlrpc]
[ 5321.116373]  [<ffffffffc0bbc50a>] tgt_request_handle+0x91a/0x15c0 [ptlrpc]
[ 5321.118230]  [<ffffffffc0b61636>] ptlrpc_server_handle_request+0x256/0xb00 [ptlrpc]
[ 5321.120318]  [<ffffffffc0b6516c>] ptlrpc_main+0xbac/0x1560 [ptlrpc]
[ 5321.122001]  [<ffffffff84cc1c31>] kthread+0xd1/0xe0
[ 5321.123023]  [<ffffffff85374c37>] ret_from_fork_nospec_end+0x0/0x39
[ 5321.124066]  [<ffffffffffffffff>] 0xffffffffffffffff

If this is server lock, tgt_brw_lock() will finally call
ofd_lvbo_update() upon lock canceling which will use @fti_attr
and pollute value:

|->ptlrpc_main
 |->lu_context_enter(le_ctx)
  |->tgt_brw_write
   |->tgt_brw_lock
    |->tgt_extent_lock
     |->ldlm_cli_enqueue_local
      |->ldlm_lock_enqueue
       |->ldlm_run_ast_work
        |->ptlrpc_check_set
          |->ldlm_cb_interpret
           |->ldlm_handle_ast_error
            |->ofd_lvbo_update
             |->ofd_attr_get polluted @info->fti_attr

  |->tgt_brw_write
   |->ofd_preprw
    |->tsi2ofd_info
      |->ASSERTION(info->fti_attr.la_valid == 0)

 |->lu_context_exit(le_ctx)--->memset @fti_attr

To fix this problem, reset fti_attr->la_valid before
ofd_lvbo_update() return just like what offd_lvbo_init() did.

Change-Id: Ib6b448dd21603cfe0305d8425862a96ef3f7fee8
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35685
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12566 mdc: hold lock while walking changelog dev list

In mdc_changelog_cdev_finish() we need chlg_registered_dev_lock
while walking and changing entries on the chlog_registered_devs
and ced_obds lists in chlg_registered_dev_find_by_obd().

Move the calling of chlg_registered_dev_find_by_obd() under the
mutex, and add assertions to the places where the lists are walked
and changed that the mutex is held.

Fixes: 1d40214d96dd ("LU-7659 mdc: expose changelog through char devices")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib62fdff87cde6a4bcfb9bea24a2ea72a933ebbe5
Reviewed-on: https://review.whamcloud.com/35668
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Fix style issues for parallel_grouplock.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mpi/parallel_grouplock.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I746b77db6fe27ee7b9306014f57887ee7504baf3
Reviewed-on: https://review.whamcloud.com/35436
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 tests: Fix style issues for createmany-mpi.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mpi/createmany-mpi.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I365f533bbd68a256be971486cd1232bddca4342f
Reviewed-on: https://review.whamcloud.com/35435
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 tests: Fix style issues for mdsrate.c

This patch fixes issues reported by checkpatch
for file lustre/tests/mpi/mdsrate.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ibfbadca6af5c915193363a772bc8f6c9a7107ffd
Reviewed-on: https://review.whamcloud.com/35434
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 util: Fix style issues for parser.c

This patch fixes issues reported by checkpatch
for file libcfs/libcfs/util/parser.c

Change-Id: I6e5f4d40634e4b3a86640fb2793a70cfd646c725
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/35384
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 util: Fix style issues for l_ioctl.c

This patch fixes issues reported by checkpatch
for file libcfs/libcfs/util/l_ioctl.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I75a8dad667e62bf128af79d64f0ce001ffc9f12c
Reviewed-on: https://review.whamcloud.com/35382
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>

LU-11743 utils: allow lctl pool commands on separate MGS

The current lctl code checks for the presence of configured pools on
the client and MDS via /proc or /sys files.  However, the MGS does
not parse the client/MDS configuration logs, so it does not create
the various files for the pools, which causes the pool commands to
fail verification.

Change lctl pool_new, pool_add, pool_remove and pool_destroy commands
to parse the configuration log directly when run on a standalone MGS
node.  This also allows the pool commands to be run when only the MGS
is started.

Test-Parameters: standalonemgs=true testlist=ost-pools.sh
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ib6fdb367c919f7b726fbf551dcfa6015593ebbe5
Reviewed-on: https://review.whamcloud.com/34110
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12604 mdt: check field size of sec context name

In request received from client, check that claimed size of
RMF_FILE_SECCTX_NAME field is consistent with expected content,
which is supposed to be an extended attribute name.

Test-Parameters: clientselinux testlist=sanity,recovery-small,sanity-selinux envdefinitions=SANITY_EXCEPT="271f"
Reported-by: Alibaba Cloud <yunye.ry@alibaba-inc.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ice96f0e03f790b334fcdf64ae4becef2e39738f4
Reviewed-on: https://review.whamcloud.com/35655
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12600 tgt: shortio size should be unsigned

The short_io_size value is accepting unsigned values from
req_capsule_get_size, and so needs to be unsigned as well.

If it's not, it's possible for the short_io_size memcopy to
act on an incorrect value and cause memory corruption.

Reported-by: Alibaba Cloud <yunye.ry@alibaba-inc.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I043e314cd43a7b40519f951a605fa5a36ff91dcf
Reviewed-on: https://review.whamcloud.com/35653
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12586 lov: Correct write_intent end for trunc

When instantiating a layout, the server interprets the
write intent from the client as the range [start, end), not
including the last byte.

This is correct for writes because the last byte given for
a write is actually 'endpos', the resulting file pointer
position, and so is not included.

However, truncate is specifiying a size, not an endpos, so
truncate is [start, size].  To make this work with the
[start, end) processing for write_intents, we have to add
1 to the size when sending a write intent.

Without this, a truncate operation to the first byte of a
new layout component fails silently because the component
is not instantiated.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id2b07abe73455bf1f0ed841ad08c5f381a871315
Reviewed-on: https://review.whamcloud.com/35607
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12568 lnet: Defer rspt cleanup when MD queued for unlink

When an MD is queued for unlink its lnet_libhandle is invalidated so
that future lookups of the MD fail. As a result, the monitor thread
cannot detach the response tracker from such an MD, and instead must
wait for the remaining operations on the MD to complete before it can
safely free the response tracker and remove it from the list. Freeing
the memory while there are pending operations on the MD can result
in a use after free situation when the final operation on the MD
completes and we attempt to remove the response tracker from the MD
via the lnet_msg_detach_md()->lnet_detach_rsp_tracker() call chain.

Here we introduce zombie lists for such response trackers. This will
allow us to also handle the case where there are response trackers
on the monitor queue during LNet shutdown. In this instance the
zombie response trackers will be freed when either all the operations
on the MD have completed (this free'ing is performed by
lnet_detach_rsp_tracker()) or after the LND Nets have shutdown since
we are ensured there will not be any more operations on the
associated MDs (this free'ing is performed by
lnet_clean_zombie_rstqs()).

Three other small changes are included in this patch:
 - When deleting the response tracker from the monitor's list we
   should use list_del() rather than list_del_init() since we'll
   be freeing the response tracker after removing it from the list.
 - Perform a single ktime_get() call for each local queue.
 - Move the check of whether the local queue is empty outside of
   the net lock.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I2a62ceb5b259a094204a1500527443e942483386
Reviewed-on: https://review.whamcloud.com/35576
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11617 mdc: fix possible deadlock in chlg_open()

Lockdep reports a possible deadlock between chlg_open() and
mdc_changelog_cdev_init()

mdc_changelog_cdev_init() takes chlg_registered_dev_lock and then
calls misc_register() which takes misc_mtx.
chlg_open() is called while misc_mtx is held, and tries to take
chlg_registered_dev_lock.
If these two functions race, a deadlock can occur as each thread will
hold one of the locks while trying to take the other.

chlg_open() does not need to take a lock.  It only uses the
lock to stablize a list while looking for the matching
chlg_registered_dev, and this can be found directly by examining
file->private_data.

So remove chlg_obd_get(), and use file->private_data to find the
obd_device.
Also ensure the device is fully initialized before calling
misc_register().  This means setting up some list linkage before the
call, and tearing it down if there is an error.

Change-Id: Ibe099f23a08fa3ef846d4ef8fcb85659f8f99245
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33572
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12523 ptlrpc: Don't get jobid in body_v2

Some Lustre messages are still sent with ptlrpc_body_v2,
which does not have space for the jobid.

This results in errors like this when getting the jobid
from these messages, which we do now that the jobid is in
all RPC debug:
LustreError: 6817:0:(pack_generic.c:425:lustre_msg_buf_v2()) msg
000000005c83b7a2 buffer[0] size 152 too small (required 184, opc=-1)

While we should stop sending ptlrpc_body_v2 messages, we
we still have to support these messages from older servers.
So put a check in lustre_msg_get_jobid so it won't try to
get the jobid if it's the old, smaller RPC body.

Fixes: 9ae40e4c5ecb ("LU-12523 ptlrpc: Add jobid to rpctrace debug messages")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I8af3c7cd921c7448ed12fdd30d1e48b2ccc89fdc
Reviewed-on: https://review.whamcloud.com/35584
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ann Koehler <amk@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11023 quota: remove quota pool ID

Remove pool ID support in quotas as it is
totally unused. It is also needed to start
implementation of quota pools that may
include 2 type of slaves - OST's or MDT's.
At the first stage quota pools will be the
mirror of LOD pools. Further when/if MDT pools
will appear, these could be aslo used as quota
pools.
The main part of quota pool's is not
changed because it will be used for new quota
pools. Remove qmt's pool hash as pool's look up
mainly will be performed by OST/MDT index and
all code connected with quota pool ID.

Cray-bug-id: LUS-5801
Change-Id: Icb72b69e1d194f23a58fcfbd2655969ad73be46b
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://review.whamcloud.com/34389
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12595 lnet: Return EHOSTUNREACH for unreachable gateway

Commit 43b35351e9ca258773e89c2d68047e939fb822fb contains a flaw in
that it shouldn't be a fatal error to encounter an unreachable
gateway when parsing routes. Parsing should continue in case there
are any valid, reachable routes that are being added. Returning EINAL
here will cause a failure to load the LNet module. lnet_parse_route()
explicitly allows for lnet_add_route() to return EHOSTUNREACH for
just this purpose.

Test-parameters: trivial
Fixes: 43b35351e9 ("LU-12411 lnet: Do not allow gateways on remote nets")
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ia0f28779a3505eff02dafdc23a6e01c1d0cbc84b
Reviewed-on: https://review.whamcloud.com/35630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12558 kernel: kernel update [SLES12 SP3 4.4.180-94.100]

Update SLES12 SP3 kernel to 4.4.180-94.100.

Test-Parameters: trivial clientdistro=sles12sp3 serverdistro=sles12sp3

Change-Id: Ie0b0f742ce98f67d74ae0583010a177ab60febd5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35589
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9897 build: use pkgconf for detecting libblkid

Currently we have extra complexity to determine where packages are
installed. This approach is also fragile if packages are not
installed in the standard location. This can be handled in a much
cleaner way with pkgconfig. Use pkgconf to discover libblkid.

Test-Parameters: trivial

Change-Id: I5241c2c38b1114d8f061ee5f2a4cb0c677ce2c71
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35531
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4423 ptlrpc: simplify locking in ptlrpc_add_rqs_to_pool()

This spinlock is only needed in this function to
protect the list_add_tail(), so make that more
obvious.

This change means that prp_rq_size is set outside of the spinlock, but
as it is only ever read outside of the spinlock, that cannot matter.

Change-Id: Ie986bf3737f0da157d991d8880148512bc4dd777
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35511
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12539 build: pass --with-o2ib when building deb packages

When building deb packages (make debs), '--with-o2ib' option is
not passed to ./configure called by package mechanism.
So Lustre deb packages are possibly built against wrong OFED headers.

Test-Parameters: trivial
Test-Parameters: trivial clientdistro=ubuntu1804
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9cd1db54e77b97f46c0e0bdfe35084f1a268b70b
Reviewed-on: https://review.whamcloud.com/35481
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12441 lnet: Detach rspt when md_threshold is infinite

MDs for pings use the infinite threshold on MD operations.
As such they aren't normally unlinkable as determined by
lnet_md_unlinkable(). We can cover this case by checking whether the
refcount is zero and threshold is LNET_MD_THRESH_INF.

Cray-bug-id: LUS-7366
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I4b9499bc6327523f1e911ebc4514002511e42236
Reviewed-on: https://review.whamcloud.com/35452
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12509 ofd: ofd_precreate_objects lockdep warning

ofd_precreate_objects()->ofd_write_lock() is used to
get the write locks, later dt_write_lock() is called
to do the same on the last_id object.

This triggers a false positive with lockdep enabled,
use a different sublass on the last_id to silence
the warning.

Consolidate lockdep subclasses into a single place,
dt_object_role.

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I283782352bef8422e6c327ce5cfac306da289e37
Reviewed-on: https://review.whamcloud.com/35420
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12492 lnet: Deprecate live and dead router check params

Rather than delete these params let's deprecate them for one release
and print a warning to console if the user is setting them.

Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I5796996dd00b06a59a872e7e8453ab64afd65dd0
Reviewed-on: https://review.whamcloud.com/35387
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12470 tests: clear MDT-MDT locks for pdo tests

It is not sufficient to clear client locks to avoid
spillover from one tests to the next in the pdo tests, we
must also clear MDT-MDT locks or we can end up waiting for
one of those.

Test-Parameters: trivial testlist=sanityn
Test-Parameters: fstype=zfs testlist=sanityn
Test-Parameters: mdscount=2 mdtcount=4 testlist=sanityn
Test-Parameters: mdscount=2 mdtcount=4 fstype=zfs testlist=sanityn
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I8b6a1a6e9a1268a5d87bcb216f54736118ae7ba0
Reviewed-on: https://review.whamcloud.com/35321
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: include file linux/selinux.h removed

In kernel 5.1 linux/selinux.h was removed with
SELinux: Remove unused selinux_is_enabled

Linux-commit: 3d252529480c68bfd6a6774652df7c8968b28e41

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If963e6b22b7b07899de5b970f934bb157c5f7cec
Reviewed-on: https://review.whamcloud.com/35035
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: Lustre specific iov_for_each broken (removed)

Kernel 4.20 introduced iov_iter_type and broke iov_for_each

As iov_for_each is only used once so drop the macro entirely.
When iov_iter_type is available ignore invalid iter types.

Linux-commit: 8a363970d1dc38c4ec4ad575c862f776f468d057

Kernel 3.15 added type to iov_iter. Use the type to provide
a sensible replacement for iov_iter_type when it is available.

Linux-commit: 71d8e532b1549a478e6a6a8a44f309d050294d00

Cray-bug-id: LUS-6962
Change-Id: I97cdce1c85803ac2d4436d4eedf67a545ea2cdb8
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/35024
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12157 utils: fix lfs_migrate output and testing

Don't pass the "-v" option through to "lfs migrate", as this causes
the filename to be printed twice when run with the "-v" option.

Don't use "echo -e" to process escape characters in filenames unless
this is needed, but add it where it is needed.  Don't add ANSI escape
characters to the output.  The output previously looked like:

     /mnt/testfs/l0: stripe count=1,size=1048576,pool=/mnt/testfs/l0
     done migrate
     nr[K/mnt/testfs/l1: /mnt/testfs/l1: already migrated ...
     nr[K/mnt/testfs/l2: /mnt/testfs/l1: already migrated ...
     nr[K/mnt/testfs/l3: /mnt/testfs/l1: already migrated ...

Print out the "pool=" and "mirror_count=" parameters only if needed.

Fix "-A" option to round up the number of stripes when the migrated.
Skip sanity test_56xc 1GB test if there is not enough space on OSTs.

Fixes: 60c5bc2502 ("LU-8235 scripts: pass unrecognized options to lfs migrate")
Fixes: 80a2ff7137 ("LU-6051 utils: allow lfs_migrate to handle hard links")
Fixes: 99d7a8ed43 ("LU-8207 scripts: add auto-stripe option to lfs_migrate")
Test-Parameters: trivial fstype=zfs testlist=sanity envdefinitions=ONLY=56
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I059e7daeb2fa82e7607fd9d862797433053ebbe5
Reviewed-on: https://review.whamcloud.com/34592
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11462 osd-zfs: skip sync if object is not dirty

there is no need to flush whole filesystem if the object is not
dirty or being synced.

in SLOW=yes FSTYPE=zfs sh sanity-benchmark.sh ~24% sync reqs
can be skipped.

Change-Id: I13b15ed646a4603858002e8be25ef5f5f8cd2b9b
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33276
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11456 mdt: mdt_object_sync() can skip sync

if object's version is found committed. OST implements this logic yet.

Change-Id: I17a05c0202bba3f97f80b165d6a3c9c2aff01996
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33266
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12587 llite: don't check vmpage refcount in ll_releasepage()

We could not use vmpage refcount to check whether page could be
released because it break invalidate_complete_page2():

See comments:
/*
 * This is like invalidate_complete_page(), except it ignores the page's
 * refcount.  We do this because invalidate_inode_pages2() needs stronger
 * invalidation guarantees, and cannot afford to leave pages behind because
 * shrink_page_list() has a temp ref on them, or because they're transiently
 * sitting in the lru_cache_add() pagevecs.
 */

So checking refcount > 3 might be wrong here, one common
case is page might be transiently in lru_cache_add().

Since we have checked whether vmpage is used by cl_page later in the
function, and vmpage will be locked before called, it should be safe
to remove vmpage refcount check.

One of problem currently is following DIO will mostly
fall back to Buffer IO:

 $ dd if=/dev/zero of=data bs=1M count=1
 $ dd if=/dev/zero of=data bs=1M count=1 oflag=direct conv=notrunc

Which is because DIO will firstly try to writeback and invalidate
clean page which fail because vmpage refcount could be 4 here.

Function calls come from linux-3.10.0-957.1.3.el7.x86_64:

|->generic_file_direct_write()
  |->filemap_write_and_wait_range()
    |->invalidate_inode_pages2_range()
          |->invalidate_complete_page2() If a page can not be invalidated,
 return 0 to fall back to buffered write.
                |->try_to_release_page()
  |->ll_releasepage()
return 0 becuase of vmpage count is 4 > 3
   |->generic_file_buffered_write

Change-Id: I14ed0877a789d68dee2de80574688a0e699556a3
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9341 utils: fix lfs find for composite files

Running "lfs getstripe -c" on a composite file returns the stripe
count of the last initialized component, but "lfs find -c N" does
not find this file because it was adding the total stripe_count
of all components.  "lfs find" should also check the stripe_count
of the last initialized component, as described in the man page.
Also use the last component stripe_size instead of any component.

Add a test case for the correct usage.

Fixes: 5a76aee24476 ("LU-8998 lfs: user space tools for PFL")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1f0097aa002b29febcbf183cab02519b202540e5
Reviewed-on: https://review.whamcloud.com/35611
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12537 lnet: Sync the start of discovery and monitor threads

The discovery thread starts up before the monitor thread so it may
issue PUTs or GETs before the monitor thread has a chance to
initialize its data structures (namely the_lnet.ln_mt_rstq). This can
result in an OOPs when we attempt to attach response trackers to MDs.

Introduce a completion to synchronize the startup of these threads.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I5d7356269090d8cbd1eab59fa29bee7ef211832f
Reviewed-on: https://review.whamcloud.com/35478
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11264 llapi: clean up llapi_file_create_foreign()

Clean up the error handling in llapi_file_create_foreign() to always
set errno, and avoid printing a duplicate error message when run on
a non-Lustre filesystem.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I610748f5d07566fb81c2f0b6f59507e97e03158f
Reviewed-on: https://review.whamcloud.com/35093
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11221 osd: allow concurrent bulks from pagecache

drop page lock earlier, once IO is complete so that page can be
read by few clients simultanously.

Change-Id: Iee28f578e937744f07f7c5be7eae99e59e625e6e
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33521
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12553  mdc: polling mode for changelog reader

this allows the user (like lsom_sync and similar) to follow
the changelog and don't rescan getting duplicates.

Change-Id: I78dc163838c1b88f9447a4731ad4bfe00fec7eff
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35262
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12405 lnet: Oracle OFED extensions default to on

Oracle UEK kernel ships with OFED extensions on by default.
Enable building if Oracle OFED extensions are not suppressed.

Introduce and use kib_fmr_pool_map wrapper for ib_fmr_pool_map_phys

Test-Parameters: trivial
Cray-bug-id: LUS-7440
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I2b15293454ce2e4f55ca0a47117c8072326e2d3a
Reviewed-on: https://review.whamcloud.com/35292
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 ldiskfs: Remove old map blocks support

Oldest supported kernel has ext4_map_blocks and ext_pblock
Remove them instead of fixing them for newer (4.14+) kernels

Test-Parameters: trivial
Cray-bug-id: LUS-6962
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I415dcfa4b0a5bde5f7240b802ea81199715bdb88
Reviewed-on: https://review.whamcloud.com/35496
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12560 tests: Use full path for test-groups

Acceptance-small doesn't use the full path when accessing
the test-groups directory, so it will only run when
executed from the directory containing test-groups.

Correct it to use the full path, so it can be run from any
directory.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I788b2993e8b9a5e83e18940aae503fb21ade9cdc
Reviewed-on: https://review.whamcloud.com/35541
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-12472 tests: update sanity-krb5.sh

With current Lustre version, some behaviors have changed.

When a user tries to access Lustre without valid Kerberos credentials,
access is denied, and the key generated in the user's session keyring
is marked as revoked. This key must be unlinked so that further
access with valid Kerberos credentials regenerates a new, valid key in
the user's session keyring.

When Kerberos credentials presented by the user cannot be validated
on server side because the lsvcgssd daemon is down, client's request
immediately fails instead of being blocked until lsvcgssd daemon
is up and runnning again.

*gssd daemon is automatically started in the background, no need for a
'&' on the command line.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1b929ac593fd427c199cffa02d364b40426d32bf
Reviewed-on: https://review.whamcloud.com/35317
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4315 doc: split lctl get_param and set_param man pages

Split the "lctl get_param" and "lctl set_param" commands from
"lctl.8" into their own man pages, and improve the description
and examples for each.

Still need to split "lctl list_param" and "lctl conf_param" in
a separate patch.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia03fe707af9bbcaad7a3c4da903d8dfbd23ebbe5
Reviewed-on: https://review.whamcloud.com/35242
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 mdt: migrate procfs files to sysfs

Move the simple one item MDT proc files to sysfs. For all the
proc files for the HSM directory move them to both sysfs and the
more complex ones to debugfs since HSM is managed by admins.

Change-Id: I4603d04853e82c29059dfc043b3ddc75757ffe27
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34779
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 test: llapi_layout_test enhancements

This patch improves the interactive behavior of
llapi_layout_test, by adding the ability to specify a
run list, rather than just a skip list.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I2ec9de94c228f309c157a3d4e75615ab8581330a
Reviewed-on: https://review.whamcloud.com/33779
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 tests: Fix style issues for write_disjoint.c

This patch fixes issues reported by checkpatch for file
lustre/tests/mpi/write_disjoint.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: If9c581925a10106592fc62e5915f029cc712ca0b
Reviewed-on: https://review.whamcloud.com/35494
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 tests: Fix style issues for write_append_truncate.c

This patch fixes issues reported by checkpatch for file
lustre/tests/mpi/write_append_truncate.c

Change-Id: Iab8f0ab0e6c501f31aca8f2a58eab8fbea0c2685
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/35493
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 tests: Fix style issues for lp_utils.c

This patch fixes issues reported by checkpatch for file
lustre/tests/mpi/lp_utils.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I6595161378936c4cb5f6229ca1f37617d0dc657c
Reviewed-on: https://review.whamcloud.com/35492
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-4398 llite: do not cache write open lock for exec file

This is to avoid the problem that the MDT needs an extra lock
revocation to make the file be able to execute.

Signed-off-by: Jinshan Xiong <jinshan.xiong@uber.com>
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Change-Id: Ibb42a9a8cb56a9bf48a6e972b72d3d71ed7fbaf5
Reviewed-on: https://review.whamcloud.com/32265
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6202 utils: remove obsolete l_ioctl2() wrapper

The l_ioctl2() wrapper was only used for liblustre configuration,
so is obsolete since Lustre 2.3 or earlier.

Fixup whitespace in nearby code.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3fd345bb36f51938ef297811ecee9f43483ebbe5
Reviewed-on: https://review.whamcloud.com/33693
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12440 lnet: Misleading error from lnet_is_health_check

In the case of sending to 0@lo we never set msg_txpeer nor
msg_rxpeer. This results in failing this lnet_is_health_check
condition and a misleading error message. The condition is only an
error the msg status is non-zero.

An additional case where we can have msg_rx_committed, but not
msg_rxpeer is for optimized GETs. In this case we allocate a reply
message but do not set msg_rxpeer.  We cannot perform further health
checking on this message, but it is not an error condition.

Test-parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I56b1f2aa98945939e4b7ae8266de66cf27f35869
Reviewed-on: https://review.whamcloud.com/35235
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12439 lnet: Convert noisy timeout error to cdebug

This error message in lnet_finalize_expired_responses is very noisy
when nodes go down or are rebooted, and it does not provide much value
to system administrators. Convert it to a CDEBUG instead

Test-parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Iccf43943de18f147aa804049b7804ea415cc99bb
Reviewed-on: https://review.whamcloud.com/35233
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12575 build: add ibutils2 for MOFED build

MOFED 4.6 include ibutils2 instead of ibutils
Remove ofed rhel5 patch which we don't need

Change-Id: I46c51eb8a194ea86bd8c951944e5c1427d0f37d0
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35590
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12090 utils: lfs rmfid

a new RPC_REINT_RMFID has been introduced by the patch.
it's supposed to be used with corresponding llapi_rmfid()
to unlink a batch of MDS files by their FIDs. the caller
has to have permission to modify parent dir(s) and the objects
themselves.

Change-Id: I50421d85babc74d448842acea489321a5d40052d
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34449
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 obd: harden debugfs handling

While the seq_file private data shouldn't disappear from under
us just in case always test if the private field is set. If
not return -ENODEV for debugfs read and write operations.

Change-Id: I28614994e8ec29f360ab534534a5869b0bf60d4b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35575
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12477 kernel: remove < 2.6.39 kernel support

We no longer support kernels 2.6.38 or older so we can remove all
the various autoconf test and simplify the lustre code. Only
leave LC_EXPORT_TRUNCATE_COMPLETE_PAGE since 4.X kernel test are
linked to this test.

Change-Id: I0532507dd3310b7ef4f4e4201dfa8b47c2991163
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35551
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4423 ptlrpc: use list_move where appropriate.

There are several places in lustre where "list_del" (or occasionally
"list_del_init") is followed by "list_add" or "list_add_tail" which
moves the object to a different list.
These can be combined into "list_move" or "list_move_tail".

Change-Id: I9ca0df11cd655b9a63a4889dd4388b3b8c8bc992
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35507
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12400 llite: Use the new vm_fault_t type

Linux 4.17 created the new vm_fault_t type

Linux-commit: 1c8f422059ae5da07db7406ab916203f9417e396

Linux 5.1 changed the vm_fault_t type to bitwise unsigned int
which changes the interfaces registered to struct vm_operations_struct

Linux-commit: 3d3539018d2cbd12e5af4a132636ee7fd8d43ef0

Prefer to match the upstream API and fallback to 'int'
where vm_fault_t is not available.

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I7122fb0d4af3ee9a19c1a5d0b77c4f13f6850181
Reviewed-on: https://review.whamcloud.com/35500
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>

LU-12400 lnet: Infiniband sg_dma changes for linux 5.1

IB/core: Remove ib_sg_dma_address() and ib_sg_dma_len()
Linux-commit: a163afc88556e099271a7b423295bc5176fcecce

This simplification can be applied to mainline 3.15 and later
however the test should remain for 3rd party ib driver support

Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I4824b3b737388a3fc0aec43b2d8e5d10f871ccdd
Reviewed-on: https://review.whamcloud.com/35497
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12536 llite: release active extent on sync write commit

Processes can wait forever in osc_extent_wait() for the extent state
to change because the extent write is not started before the wait
begins. A 4.7 kernel change to generic_write_sync() modified it to
check IOCB_DSYNC instead of O_SYNC. Thus an active extent is not
released (written) in osc_io_commit_async() in the synchronous case.

Linux commit: dde0c2e79848298cc25621ad080d47f94dbd7cce

Cray-bug-id: LUS-7435
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I03fd3f17c0c21bcb176730a0edf449d2e5e0c108
Reviewed-on: https://review.whamcloud.com/35472
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12089 scripts: Fix style issues for system-profile.sh

This patch fixes issues reported by checkpatch for file
lustre/scripts/system-profile.sh

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I83ba9d78046077d8b0f20c6877415355a105e70a
Reviewed-on: https://review.whamcloud.com/35432
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-12089 scripts: Fix style issues for statechange-lustre.sh

This patch fixes issues reported by checkpatch
for file lustre/scripts/statechange-lustre.sh

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I202200f16681ea56916beb4b56f7406fccdf39d9
Reviewed-on: https://review.whamcloud.com/35429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-12431 clio: remove default csi_end_io handler

Since the csi_end_io is empty remove it.
Provide an initialize that supports a custom end_io handler.

Cray-bug-id: LUS-7330
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If62c2536772136bfafc6e99ad1dbd9b466735201
Reviewed-on: https://review.whamcloud.com/35400
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12478 build: rhel8 missing module packaing tools

On RHEL8 kmodtool and kernel_module_package_buildreqs
are not installed with kernel-devel

This helps to bootstrap the developer into a working configuration.

Test-Parameters: trivial
Cray-bug-id: LUS-7385
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I30cf5c11174b2aa94c663c72e34f8f88c5c90ed8
Reviewed-on: https://review.whamcloud.com/35356
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12479 utils: cleanup gcc8 string warnings

Cleanup some trivial buffer overflows

Test-Parameters: trivial
Cray-bug-id: LUS-6962
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I17dd1d042c4050e351aca856931ce1107fd5b08f
Reviewed-on: https://review.whamcloud.com/35354
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>

LU-11022 lfs: remove mirror by pool name

lfs mirror split --pool <poolname> <file>

Change-Id: I012e68729b94657236ba3fc530fc7b7485529ed2
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35329
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9859 libcfs: initialize bit_wait_table

With older platforms wait_event_var() is missing so we needed to
provide are own version. This included creating a local
bit_wait_table like the kernel has. That table was never properly
initialized so it can cause failures under the right conditions.

Test-Parameters: trivial

Fixes: 372ef85512 ("LU-11089 obd: use wait_event_var() in lu_context_key_degister()")
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I310d37da7c1b54166224b367446cc905c02ab8bc
Reviewed-on: https://review.whamcloud.com/35567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9835 utils: Properly maintain remaining buffer length in jt_opt_threads

When we write to a buffer, we do need to reduce the size of it,
not just move the starting pointer.

Test-Parameters: trivial
Change-Id: I688e31d5434fb3040985b65a092d56ca30b69941
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/28363
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12540 tests: squash multiply -O -E options

mkfs.lustre and e2fsprogs don't support a multiply
-O -E options. after checking a code, i think easy
way is handle it situation in the test-framework.

Fixes: 591a9b4cebc5 ("LU-9846 lod: Add overstriping support")
Test-Parameter: trivial

Cray-bug-id: LUS-7479
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: I54cab1e02093136a3550a76d1262aeb281118bae
Reviewed-on: https://review.whamcloud.com/35482
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10100 llite: swab LOV EA user data

Many sub-tests failed with "Invalid argument" failures
on PPC client because of the endianness issue.

This patch fixes the issue by adding a common function
lustre_swab_lov_user_md() to swab the LOV EA user data.

Test-Parameters: clientarch=ppc64 \
envdefinitions=ONLY=27 testlist=sanity

Change-Id: I46bab0788300cd79c4e66e1a4990c3e1f7192391
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35291
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12561 build: Remove old lbuild entries

Remove lbuild entries for 2.6 based kernels (and earlier).

Test-Parameters: trivial

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4bb3dcf9e9b4402d7f4e86bbd54f7d9d62efdc48
Reviewed-on: https://review.whamcloud.com/35553
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 ldiskfs: Remove unused 2.6.x patches

Lustre no longer supports 2.6.x kernels, so remove the
ldiskfs patches which are no longer used after removing
RHEL6 and non-3.x SLES11 support.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Icc31c9704669367ece16ec0c09746611cad32ee8
Reviewed-on: https://review.whamcloud.com/35550
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 ldiskfs: Remove 2.6 kernel series

Lustre no longer supports 2.6 kernels, so remove the
ldiskfs patch series for these kernels.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I2c7941582ee90416149cecd47fd2914bc5265a7f
Reviewed-on: https://review.whamcloud.com/35549
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: remove txt directory

The kernel_patches/txt directory was last updated in 2007,
and isn't used for anything today.  Let's remove it.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iad69e7dd92791156752573f8bf2f3c619cdcb637
Reviewed-on: https://review.whamcloud.com/35548
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove unused patches

With 2.6 kernel support removed, these patches are no
longer used.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If374a060f4a4522a10111923c92fa7ccaf791df3
Reviewed-on: https://review.whamcloud.com/35547
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove 2.6 based SLES11 support

2.6 based SLES11 (SLES11SP{0,1,2}) is no longer supported
(and does not build) so remove the series & target for it.

3.x based SLES11 (SP{3,4}) is still supported, and this
should not affect those.

Test-Parameters: trivial

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie73c39d24bad26e696a66e907c161348ab4afad5
Reviewed-on: https://review.whamcloud.com/35546
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12561 kernel: Remove RHEL6 series and targets

Remove the RHEL6 series and target files.

Also remove the RHEL5 (wow!) targets, and the outdated
Fedora (fc) targets.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I00fc47cac656bc3b6f220f3994f0a25ed73879f9
Reviewed-on: https://review.whamcloud.com/35545
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove RHEL6 kernel configs

First in a series of patches to remove RHEL6 support, this
removes the kernel configs.

This should be a build only change, so trivial testing
should be OK.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0b68754f8921b82e8d0c5eaa321d58187da49a70
Reviewed-on: https://review.whamcloud.com/35544
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12561 kernel: Remove RHEL6 from which_patch

RHEL6 is not supported any more, and no longer builds, so
remove it from which_patch.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id71e41ea76e96542cfa090e9a72bca024353601f
Reviewed-on: https://review.whamcloud.com/35543
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12296 llite: improve ll_dom_lock_cancel

ll_dom_lock_cancel() should zero kms attribute similar to
mdc_ldlm_blocking_ast0().

In order to avoid code duplication between mdc_ldlm_blocking_ast0()
and ll_dom_lock_cancel() - add cl_object_operations method to be able
to reach mdc's blocking ast from llite level.

Test illustrating the issue is added.

Cray-bug-id: LUS-7118
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I2b100ead6d420dbf561bc61be973d64dad317214
Reviewed-on: https://review.whamcloud.com/34858
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12516 mdd: support for volatile creation in .lustre

this is useful to enable striping manipulation by FIDs.

Change-Id: I4d5b1b13acdfef21ac46bf3557e9ab6d5ccc796b
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35258
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12484 tests: correct typo in test-framework::is_project_quota_supported()

do_facet was misspelled as do_fact on the zfs side.

Change-Id: Idc112802a6817e5128799cb6059040c6b3021791
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Fixes: f172b116885 (LU-10092 llite: Add persistent cache on client)
Reviewed-on: https://review.whamcloud.com/35362
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>

LU-12489 tests: Recategorize some "Slow" tests as normal.

replay-single 44b is really fast now
replay-ost-single 8[ab] both take under a minute in my testing
ost-pools 23b only takes a minute now and test 18 is still
          under 10 minutes to bother excluding it.

Change-Id: I57a35c39cadd1728c38d332d83cbe50da6d6a8fe
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35367
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-4423 obdclass: use list_sort() to sort a list.

Rather than a bespoke bubble-sort, use list_sort() to
sort this linked list.

As this would become a 1-line function that is only called once,
call list_sort() directly from the one call site.

Linux-commit: e714d3559e964d1547d20b54ad5fd6bbb3401f56

Signed-off-by: NeilBrown <neilb@suse.com>
Change-Id: Ied197a4fdba43d793c5ebbb9afc837a986609469
Reviewed-on: https://review.whamcloud.com/35512
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12355 llite: MS_* flags and SB_* flags split

In kernel 4.20 the MS_* flags should only be used for mount
time flags and SB_* flags for checking super_block.s_flags
The MS_* flags have moved to a uapi header

Linux-commit: e262e32d6bde0f77fb0c95d977482fc872c51996

Test-Parameters: trivial
Change-Id: Ifd64efb16c7795377ece066d01ae04dc004a13ac
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/35019
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-12510 osd: osd-zfs to release zrlock quickly

otherwise few threads trying to access same dnode can get stuck.
this patch is a quick workaround for the issue, it's supposed
to be replaced with a better patch using regular DMU API.

Change-Id: I24d9ed7f8e68080c6a46409476a80799dbb45230
Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35524
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9019 libcfs: avoid using HZ and msecs_to_jiffies()

HZ is a constant selected with the configuration of the kernel
and msecs_to_jiffies() is a inline function in jiffies.h. Because
we are out of tree that means prebuilt lustre packages could be
installed on a node with different values which impact the
behavior of the file system. This was addressed earlier but
regression have crept back in. Fix up all those instances by
replacing msec_to_jiffies() and HZ with cfs_time_seconds() which
translates seconds to jiffies with nsec_to_jiffies(). Add to
spelling.txt to avoid new regressions.

Change-Id: Ie13efe83774db498bb5475ed47a057bbd42d47bf
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35520
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12462 osc: Do not assert for first extent

In the discard case, the OSC fsync/writeback code asserts
that each OSC extent is fully covered by the fsync request.

This is not valid for the DOM case, because OSC extent
alignment requirements can create OSC extents which start
before the OST region of the layout (ie, they cross in to
the DOM region).  This is OK because the layout prevents
them from ever being used for i/o, but this same behavior
means that the OSC fsync start/end is aligned with the
layout, and so does not necessarily cover that first
extent.

The simplest solution is just to not assert on the first
extent.  (There is no way at the OSC layer to recognize the
DOM case.)

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If66f8d81fb9dd4546a5647a10f6ca551e2cf98e3
Reviewed-on: https://review.whamcloud.com/35525
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12326 tests: recognize Bochs as VM

Bochs is reported by qemu-system-x86_64 and few tests (e.g. 399a)
depends on that.

Change-Id: I3c6cfca1c0cb811425a09b3958cd0626891e73da
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34942
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12545 llite: cleanup stats of LPROC_LL_*

Some LPROC_LL_ stats are not used for a long time. This patch
removes them. LPROC_LL_STAFS is changed to LPROC_LL_STATFS in
this patch too.

Change-Id: I64b033e186733147e2a2984d8afec97240d03573
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/35514
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4423 ptlrpc: make ptlrpc_last_xid an atomic64_t

This variable is treated like an atomic64_t,
so change it's type and simplify the code.

Change-Id: I4b219342222fd784ac1d7dc17660feb816bacd57
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35510
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-4423 ptlrpc: Fix using smp_processor_id() in preemptible context

This warning show up with kernels that enable preemptible
[ 1877.516799] BUG: using smp_processor_id() in preemptible [00000000] code: mount.lustre/14077

Change it to disable preemption around smp_processor_id().

Linux-commit: c369772e78a7383ba4e68673128fe2d6ef2863ee

Fixes: ef94e4d1bb ("LU-8710 ptlrpc: use current CPU instead of hardcoded 0")

Change-Id: If66107f0843c5d0c4bcf874e64a20251fdf9704e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35495
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12542 ldlm: simplify lock_mode_to_index()

This function has the same effect as ilog2(), so just use ilog2
directly.

Change-Id: If90207c328b549e85cb6d38a6604dfb8c7b6c8a0
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/35486
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11672 ldlm: awalys cancel aged locks regardless enabling or disabling lru resize

Currently cancelling aged locks is handled by of ldlm_pool_recalc routine,
and it only works when lru resize is enabled, means if we disabled lru
resize, old aged locks are still cached even though they reach the
ns_max_age.

But theoretically, even lru resize disabled, lru_max_age should behave
same as enabling lru resize. At the end, lru_size is like hard limit of
number of locks, but ns_max_age/lru_max_age is a elimination mechanism,
regardless enabling or disabling lru resize meaning once it gets
lru_max_age, locks need to be cancelled.

So fix it here with changing the lru flags when invoking ldlm_cancel_lru
to do the real cancel work, if lru resize is enabled, set flag to
LDLM_LRU_FLAG_LRUR, otherwise LDLM_LRU_FLAG_AGED.

Change-Id: Ic2df2550af87fd7209fdb31ca3730683d727a74d
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/35467
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12137 osd-ldiskfs: use ERR_CAST in osd_oi_index_open()

In osd_oi_index_open() when dentry is invalid it is void casting
the invalid dentry being returned. This is what ERR_CAST was
invented for so use it.

Test-Parameters: trivial

Change-Id: I71016ff9c9dfc4408db8ab14576ba87dd6dc352d
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35455
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12137 osd-ldiskfs: migrate ll_lookup_one_len to osd_compat.c

The function ll_lookup_one_len() in lvfs.h is only used for the
osd-ldiskfs layer so relocate it to osd_compat.c and rename it
to osd_lookup_one_len_unlocked(). We use unlocked in the name
since this signifies it must always be called with the inode lock
not taken since it will take the inode lock itself.

Test-Parameters: trivial

Change-Id: I507100a676f98448f4cacc94e902294903a67efb
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/35454
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12431 clio: issue wake with waitqueue lock held

Remove the barrier and rely on the wait queue lock for wake
synchronization.

Leave csi_end_io empty but available for future customization

Inspired by c3973b4aca6df794c492f6856ffbf02f2f8a9592

Cray-bug-id: LUS-7330
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Idc632140256cccfa6046a52cbd1c6432955e2b11
Reviewed-on: https://review.whamcloud.com/35381
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>