LU-13485 build: Enable 2 stage configure tests

This idea was implemented by OpenZFS a while ago. This
is heavily inspired by the OpenZFS work.

Here we enable splitting tests compile tests into two
distinct parts that share an internal unique name.

The source half can then be built in parallel and
the results can be determined based on the build
artifacts.

Tests which depend on order of execution and/or the
result of a previous test are not well suited for
being converted. However the majority of lustre
compile tests can be run in parallel.

Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If01ccdfdf4810ecc2d616da3fa6b7ca786fe760f
Reviewed-on: https://review.whamcloud.com/38347
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13472 lnet: set route aliveness properly

In the case when the discover is toggled from on to off, the route
aliveness might become stale due to not updating the route->lr_alive
variable correctly. It will get updated once the gateway is pinged.
However, there is a period of max alive_router_check_interval where
the route can be down.

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ic1754d6e7ddc9398efc7a64f823a70e5546e9ca6
Reviewed-on: https://review.whamcloud.com/38323
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13477 lnet: Force full discovery cycle

There are scenarios where there could be a discrepancy between
cached peer information and reality. In these cases what could
end-up happening is incomplete interface information might be
cached because one side determined that the peer didn't require
a PUSH. This will lead to undesired MR behavior, where not all
the interfaces are used for a period of time.

Therefore, it is safer to always force a full discovery cycle:
GET/PUSH to ensure both sides are up-to-date.

In the NMR case, when discovery is turned off, make sure to flag
discovery as complete to avoid stalling the state machine.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ie49ad11e8ff874206baa268a4ef2d58ebb536ed5
Reviewed-on: https://review.whamcloud.com/38322
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13478 lnet: handle discovery off properly

Peers need to only be updated when discovery is toggled from
on to off. This way the peers don't attempt to send to a
non-primary NID of the node. However, when discovery is
toggled from off to on, the peer will attempt rediscovery
and the peer information will eventually consolidate.

In order to properly delete the peer only when it makes sense
we have to differentiate between the case when we get the
initial message and when we get a push for an already discovered
peer. We only want to delete our local representation if the peer
is one we have already had in our records.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Id6a7353276fec82fddf90e0fa9d85d165b459c8d
Reviewed-on: https://review.whamcloud.com/38321
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13464 target: abort recovery if timer fail

During target recovery, the recovery timer should be kept to be
armed to ensure the recovery doesn't take too long time, there
should be some problem if the deadline of the recovery timer is
passed and the recovery is not completed yet, the recovery should
be aborted in this case.

Change-Id: Id44f2a2d1a3183ad8dd13f4d34392713c55a2cb3
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38277
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13258 obdclass: bind zombie export cleanup workqueue

Lustre uses a workqueue to clear out stale exports. Bind this
workqueue to the cores used by Lustre defined by the CPT setup.

Move the code handling workqueue binding to libcfs so it can be
used by everyone.

Rename CONFIG_LUSTRE_PINGER to CONFIG_LUSTRE_FS_PINGER to match
linux client.

Change-Id: Ifa109f6a93e6ec6bbdef5e91fe8ca1cde0eaea3e
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/38212
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13412 llite: fix read if readahead window smaller than rpc size

Readahead always try to align readahead with RPC size, but this
could introduce a problem if readahead window is smaller than RPC size.

With current codes, it will fallback a lot of 4k read because
RPC aligned window start plus window pages will be behind of
current read. Fix this to align with readahead window rather
than RPC size in this case.

Change-Id: I0cd33ac7f92a75f38c926db33630f3036bbfd6c7
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38132
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 lnet: always pass struct lnet_md by reference.

Both LNetMDAttach and LNetMDBind expected a struct lnet_md to be
passed by value.  This requires copying the data structure onto the
stack, which is a waste of stack space and brings no value.

So change them to expect a reference, and declare it 'const' to be
sure it doesn't get changed.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I343797d1e70cc85fde92d544e56536e982e02973
Reviewed-on: https://review.whamcloud.com/37853
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 gnilnd: remove support for GNILND_BUF_VIRT_*

GNILND_BUF_VIRT_UNMAPPED and GNILND_BUF_VIRT_MAPPED are
not longer set, so remove them and any code that only
runs when they are set.
    gnd_map_nvirt  gnd_map_virtnob
can go too.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: If394bc2cf64f903ed4cdb1e1e80a2a017accd562
Reviewed-on: https://review.whamcloud.com/37847
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 gnilnd: discard struct kvec arg.

The 'struct kvec *' are to kgnilnd_setup_rdma_buffer()
and kgnilnd_setup_immediate_buffer() is now always
NULL.  So we can remove the arg and code that handles
non-NULL values.
This means that kgnilnd_setup_virt_buffer() can
disappear completely.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib38494693fba521a6e3dc4e6dc0cbb33dea1595b
Reviewed-on: https://review.whamcloud.com/37846
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13262 ldlm: no current source if lu_ref_del not in same tsk

Running with USE_LU_REF ("configure --enable-lu_ref") configured
triggers a LBUG (because "ref->lf_failed > 0" condition false)
due to to using "current" as the lu_ref source, but in some cases
lu_ref_del() occurs within a different task context.
To avoid this, lu_ref source is changed to ldlm_lock address by
this patch.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ia35e31c1a722c03f97672025e2abff40486b3f76
Reviewed-on: https://review.whamcloud.com/37624
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>

LU-10934 llite: integrate statx() API with Lustre

System call statx() interface can specify a bitmask to fetch
specific attributes from a file (e.g. st_uid, st_gid, st_mode, and
st_btime = file creation time), rather than fetching all of the
normal stat() attributes (such as st_size and st_blocks). It also
has a AT_STATX_DONT_SYNC mode which allows the kernel to return
cached attributes without flushing all of the client data and
fetching an accurate result from the server.
The conditions for adding statx() API for Lustre are mature:
1. statx() is added to Linux 4.11+;
2. glibc supports statx() (glibc 2.28+ -> RHEL 8, Ubuntun 18.10+)
3. The support for stat(1) and ls(1) to use statx(3) to fetch
   only the required attributes has landed to the upstream GNU
   coreutils package.

This patch integrates statx() API with Lustre so that we can take
advantage of the efficiencies available:
- Only fetch MDS attributes if STATX_SIZE, STATX_BLOCKS and
  STATX_MTIME are not requested, and avoid OSS glimpse RPCs
  completely;
- Hook this into statahead to avoid async glimpse locks (AGL) if
  OST information not needed;
- Enhance the MDS RPC interface to return the file creation time
  stored in both ldiskfs and ZFS already, and enable STATX_BTIME;
- Better support with AT_STATX_DONT_SYNC mode. Return the "lazy"
  attributes or cached attributes (even stale) on a client if
  available without any RPCs to servers (MDS and OSS).
- statx (lustre/test/statx): port coreutils ls/stat by using
  statx(3) system call if OS supported it.
- Test scripts. Using statx() to verify btime attribute and the
  advantage described above.

Test-Parameters: clientdistro=el8
Test-Parameters: clientdistro=ubuntu1804
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I8432c9029bad9dea3e1ebc13a0d6978131d9b929
Reviewed-on: https://review.whamcloud.com/36674
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13528 llite: prevent MAX_DIO_SIZE 32-bit truncation

On 4kB PAGE_SIZE systems, kmalloc can allocate up to 4MB, which makes
MAX_DIO_SIZE up to 682MB. This number can fit into 32 bits.
But on 64kB PAGE_SIZE systems, kmalloc can allocate up to 512MB, which
then makes MAX_DIO_SIZE up to 1365GB. This needs 64 bits to fit.
Make sure that for every platform MAX_DIO_SIZE is not abusively
truncated, by casting it to size_t.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9d2c8c4a1ccf0abf0b7647e569b8454365369e8a
Reviewed-on: https://review.whamcloud.com/38526
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 utils: Fix style issues for lustre_cfg.c

This patch fixes issues reported by checkpatch
for file lustre/utils/lustre_cfg.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I4617daabd111309cac11c975df6ee0a897379115
Reviewed-on: https://review.whamcloud.com/38643
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 utils: Fix style issues for mkfs_lustre.c

This patch fixes issues reported by checkpatch
for file lustre/utils/mkfs_lustre.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ibf5833907b38dc67f84e88e7d2c188c6eb51773e
Reviewed-on: https://review.whamcloud.com/38556
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-9679 lnet: tidy lnet_discover and fix mem accounting bug.

A recent patch introduce a memory accounting bug because "n_ids"
can change between the ALLOC call and the FREE call.

With this patch we fix that by ensuring n_ids doesn't change - the
current change is not needed.
Also:
 - discard 'max_intf' var.  It is always exactly lnet_interfaces_max,
   so just use that directly.
 - only copy back the number of interfaces found
 - report the number of interfaces actually copied.
 - Move the copy_to_user until after all locks and references are
   dropped so there is no need to re-take any locks.

Test-Parameters: trivial
Fixes: b1f6f3becedc ("LU-9679 libcfs: Add CFS_ALLOC_PTR_ARRAY and free")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib807ca4ce5235b28e7ae11d90e1942aff4454cfc
Reviewed-on: https://review.whamcloud.com/38644
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 llite: use %pd to report dentry names.

Since Linux 3.12, it has been possible use the "%pd" format specifier
to print a dentry name, so use that instead of "%.*s" and having
to pass both the length and the name.

Redhat's 3.10 kernels also have this functionality backported.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I70303a2c375d54825e89a0ba6703914528c54c05
Reviewed-on: https://review.whamcloud.com/38442
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 ptlrpc: change LONG_UNLINK to PTLRPC_REQ_LONG_UNLINK

The name "LONG_UNLINK" is vague and generic.  Change it to
  PTLRPC_REQ_LONG_UNLINK
to make it clear it is about requests taking a long time,
and of interest to PTLRPC.

Test-Parameters: trivial
Suggested-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I990657da3ec780c982a7ae31c21e4c8c9064be17
Reviewed-on: https://review.whamcloud.com/38405
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-6142 lustre: convert some container_of0 to container_of

Each of these calls to container_of0() can be determined from local
context to be passed a valid pointer, so it is best to use
container_of() directly to make this clear.
Either:
 - the returned pointer is dereferenced with out be tests, or
 - the passed-in pointer is dereferened before the call, or
 - the passed-in pointer cannot be NULL, such as when
   it is a '.next' of a list_head or returned by lu_obecjt_next()

So convert all of these to container_of()

... except one which *should* be container_of(), but cannot
be as it won't compile cleanly on older kernels.  Change
that one to container_of_safe() with a big comment.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idcd954f89ed366882563810ce042a5ddaba5a1e5
Reviewed-on: https://review.whamcloud.com/38383
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 osd-ldiskfs: convert container_of0() to container_of()

Every use of container_of0() in osd-ldiskfs can safely use
container_of() instead.  Doing so makes the intent of the code
clearer.

In most cases, the pointer returned is later dereferenced without any
subsequent checks.  In a few cases (e.g.  osd_obj()), the pointer
passed in is dereferenced before the container_of() call.  These
patterns assure us that the pointer in valid (not NULL or an ERR_PTR),
so container_of() is the correct interface to use.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I11cb51cd2dc459a3ab5c420ef7bf3324a28eeffc
Reviewed-on: https://review.whamcloud.com/38378
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 lustre: use BIT() macro where appropriate

When accessing a bit in a bitmap/mask/flags-word it can be more
readable to use BIT(num) rather than "1 << num".

This patch makes that change to various places in lustre

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9f10e7e11e0f310c46c9b216f138799c62308279
Reviewed-on: https://review.whamcloud.com/38377
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 lustre: use BIT() macro where appropriate in include

When accessing a bit in a bitmap/mask/flags-word it can be more
readable to use BIT(num) rather than "1 << num".

This patch makes that change to various places in lustre/include.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6a242fa843ff26320b299b8a532b6bb4388f0233
Reviewed-on: https://review.whamcloud.com/38376
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9679 ptlrpc: use OBD_ALLOC_PTR_ARRAY() and FREE

Use:
  OBD_ALLOC_PTR_ARRAY
  OBD_FREE_PTR_ARRAY

for allocating and freeing arrays in ptlrpc.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic0323ce4d50d5ced3ac6e529f7477cc4779478e6
Reviewed-on: https://review.whamcloud.com/38253
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-9679 mdt: use OBD_ALLOC_PTR_ARRAY() and FREE

Use:
  OBD_ALLOC_PTR_ARRAY
  OBD_FREE_PTR_ARRAY

for allocating and freeing arrays in mdt.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5e69f03370fc42fdfe770a4c6fc72a994a6749f6
Reviewed-on: https://review.whamcloud.com/38251
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13535 lfsck: fix possible PFL layout corruption

While checking lmm_oi in composite layout the pointer to 'lmm'
is re-assigned to component entry but the same pointer is used
for LOV EA buffer to update EA. Therefore if lmm_oi was fixed in
some component then just current entry is saved as new layout.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ifbd984a71b383ab4ca35ad59ed9cd8cf57b6d7cc
Reviewed-on: https://review.whamcloud.com/38584
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>

LU-10401 tests: add -F so list_param prints entry type

In sanity.sh test_133g add "-F" so that the list_param
command prints '=' for entries that can be written.
Otherwise, the grep will not find any entries to modify
and the test does not really work. Also change 113f to
use lctl rather than hardcode proc and sysfs paths.

Test-Parameters: trivial testlist=sanity
Fixes: 83b6c6608e94 ("LU-11644 ptlrpc: show target name in req_history")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I3c94bb13f3345161580ef1b7944945b871ea2c60
Reviewed-on: https://review.whamcloud.com/38567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>

LU-13187 tests: improve sanity test_129 checking

When running sanity test_129 repeatedly, it may fail to print the
warning/error messages to the console due to throttling.  Fix the
test to only warn if the messages were not printed, so long as the
size limit is correct.

Always reset max_dir_limit on exit, and delete the test files.

Fix test_129 to use mcreate instead of multiop to be faster, and
fix mcreate to return the actual error code returned by the syscall.

Fix the console error messages to be complete strings, rather than
being dynamically generated, so that they can be more easily found.
To compensate, don't have separate messages for with/without FID.

Test-Parameters: trivial testlist=sanity env=ONLY=129,ONLY_REPEAT=100
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I06593441151fa508ef2db7a3e68be34cc0ce7057
Reviewed-on: https://review.whamcloud.com/38550
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13539 all: Cleanup LASSERTF uses missing newlines

LASSERTF() usage that does not include a terminating newline
present an unclear syslog entry from admins.

This adds the terminating newline for a few cases.

Test-Parameters: trivial
HPE-bug-id: LUS-8853
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I3f088601c2ad6d59e2868e86bfbd5315e36ece98
Reviewed-on: https://review.whamcloud.com/38540
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-9859 libcfs: move tcd locking across to tracefile.c

No need to have this in linux-tracefile.c

Test-Parameters: trivial
Change-Id: I3fdc70ad5f32ea7ff78c778565f01eaaa78f1e94
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/38601
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13549 build: fix zfs/spl config checks

zfs/spl config checks should only proceed to variable substitution in
case system supports it. Otherwise, it would end up adding 'Not found'
in list of include dirs/libs.

Test-Parameters: trivial testgroup=review-zfs
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia76a67e42cc6a2f116013c142b6c0b2143838548
Reviewed-on: https://review.whamcloud.com/38592
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13481 dne: improve temp file name check

Previously if all but two characters in file name suffix are digit,
it's not treated as temp file, as is too strict if suffix length is
short, e.g. 6. Change it to allow one character, and this non-digit
character should not be the starting character.

Test-Parameters: trivial testlist=sanity env=ONLY=33h,ONLY_REPEAT=500
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ie36e6e15c1e593f47f4d3fab7f8c567d1d587f28
Reviewed-on: https://review.whamcloud.com/38539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>

LU-9859 libcfs: discard cfs_block_sigsinv()

cfs_block_sigsinv() and cfs_restore_sigs() are simple
wrappers which save a couple of line of code and
hurt readability for people not familiar with them.
They aren't used often enough to be worthwhile,
so discard them and open-code the functionality.

The sigorsets() call isn't needed as or-ing with current->blocked is
exactly what sigprocmask(SIG_BLOCK) does.

Linux-commit: 6afe572bc76688cd840032254217a4877b66e916

Change-Id: Ia9189e0885dffb098df7abef09db42ecb49196cd
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/38530
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 lnet: Correct signature in gnilnd.h

kgnilnd_recv was updated correctly but the function prototype
was not included.

Test-Parameters: trivial
Fixes: e35b7751f49 ("LU-13004 lnet: remove the 'struct kvec' arg from lnd_send")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I29db966ce54801391aa3c5c6426f03a327fea4b1
Reviewed-on: https://review.whamcloud.com/38491
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13493 llite: check if page truncated in ll_write_begin()

See following function flows:

CPU0 CPU1
|->grab_cache_page_nowait
  |->find_get_page
    |->__find_get_page (page unlocked)
|->truncate page
   |->trylock_page —->page might has been truncated after this.

So it is possible that page might has been truncated after
grab_cache_page_nowait() return even page lock is held.

We need check wheather vmpage->mapping change in ll_write_begin()
otherwise, we will have truncated page with NULL mapping, which
will trigger assertions in vvp_set_pagevec_dirty().

This patch also fix assertion string doesn't end in newline.

Change-Id: I46e14f560378a39d8ae1353d60cc49c0f0b241c0
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38425
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13471 lnet: use the same src nid for discovery

When discovering a remote peer (not on the same network) a GET is
sent to the peer to retrieve the peer's interfaces.  This is followed
by a PUSH, if discovery is on, to push the node's interfaces However,
if both node and peer have multiple interfaces it is likely that the
GET and the PUSH will originate on different interfaces. When the
peer receives the PUSH it will not be able to connect the two NIDs
and will not be able to consolidate the node's NIDs.  This issue is
specific for remote peers because at the time the push handler is
invoked the remote lpni has not been created yet. lnet_parse()
creates the lpni of the gateway.

Similar to the strategy already in place of using the same source NID
for all the messages of an RPC, discovery should use the same source
NID for both the GET and PUSH.

This patch stores the source NID interfaces the GET was sent on and
uses it for the PUSH.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I5a13ab7799b2ddc47714202bcbed786b0d3940b7
Reviewed-on: https://review.whamcloud.com/38320
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13369 kernel: kernel update RHEL7.7 [3.10.0-1062.18.1.el7]

Update RHEL7.7 kernel to 3.10.0-1062.18.1.el7.

Test-Parameters: trivial clientdistro=el7.7 serverdistro=el7.7

Change-Id: Ifc00fca35a0ad28ba8326e56e693ea39360a8114
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38240
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13427 lmv: do not print MDTs that are inactive

Have lmv return -EAGAIN instead of -ENODATA for unconfigured MDTs.
That avoids "lfs df -v" from printing a long list of invalid MDTs
when trying to get the target state for non-rotational devices.

Add test for "lfs df -v" printing nonrotational state, as well as
limiting the reported OST and MDT to configured devices.

Fixes: 4c76eb64a9ff ("LU-8920 utils: don't print deactivated OSTs in 'lfs df')
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia3dcf85f4cf52caa39848f0f6ed8d47d42ce7057
Reviewed-on: https://review.whamcloud.com/38165
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10973 lnet: initialize buffers properly

Initialize the error message buffer properly to avoid
garbage characters printed out.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I9c0d08ea2baf0cda990e3abb7e7de497a5226bfe
Reviewed-on: https://review.whamcloud.com/38083
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13366 utils: SEL yaml and copy file support

create a SEL file by a yaml template or by copying the template
from another file (setstripe --yaml and setstripe --copy).

Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: I001226a9dcfa5706295cef1c13a2e7aeb5c498cc
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://es-gerrit.dev.cray.com/156286
HPE-bug-id: LUS-8039
Reviewed-on: https://review.whamcloud.com/37960
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13366 doc: add SEL options to util's man pages

a couple of documents missed the SEL info:
- lfs man page
- lfs setstripe help message
- lfs find help message

Test-Parameters: trivial
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Ifd4e55877309a29cafdd05d007ace6660d0f775b
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://es-gerrit.dev.cray.com/156672
HPE-bug-id: LUS-8149
Reviewed-on: https://review.whamcloud.com/37962
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 socklnd: discard tx_iov.

tx_iov always points to tx_hdr, so we can discard tx_iov, and just use
&tx_hdr.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9738b6f953184fdea859da7a9c4187227fa61143
Reviewed-on: https://review.whamcloud.com/37851
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 lnet: simplify ksock_tx.

The tx_frags union in 'struct ksock_tx' is largely unnecessary.  The
payload is always lnet_kiov_t, the only kvec is a header.  So replace
the union with just those two fields.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I2c4bd5397ddf412b11e9f741406de617c534c736
Reviewed-on: https://review.whamcloud.com/37850
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 lnet: remove lnet_copy_flat2iov and ..iov2flat

This functions are no longer used, so remove them.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I748643c38430ddb21f35493f0ef704528167716f
Reviewed-on: https://review.whamcloud.com/37849
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13004 lnet: remove lnet_extract_iov()

The only place this is called, the src kvec is
NULL with length 0, so it returns 0.
So remove it.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I72fd39d0b4330fd4f811f2ee1476a653407855b4
Reviewed-on: https://review.whamcloud.com/37848
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6006 tests: add sleep 1 after command in background

otherwise subsequent command may race with the one in
background and fail:
 mkdir a & touch a/b

Test-Parameters: trivial env=ONLY="22-23",ONLY_REPEAT=50 testlist=replay-dual
Test-Parameters: env=ONLY="22-23",ONLY_REPEAT=50 fstype=zfs testlist=replay-dual
Test-Parameters: env=ONLY="22-23",ONLY_REPEAT=50 mdscount=2 mdtcount=4 testlist=replay-dual
Test-Parameters: env=ONLY="22-23",ONLY_REPEAT=50 fstype=zfs mdscount=2 mdtcount=4 testlist=replay-dual

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0574d315e596cd899f7c4ea20c70b4c3da99b9b4
Reviewed-on: https://review.whamcloud.com/37343
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13522 mdt: count mdt_mds_mds_conns upon reconnect

mdt.mdt_mds_mds_conns is inter-MDT connection count, it should be
increased upon reconnect (MDT stop and start).

Update sanityn 33c.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ic0dabee10431f39665cb2bc4fa8a014fc78fbd60
Reviewed-on: https://review.whamcloud.com/38512
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13065 tests: clean up runtests

The runtests test suite needs to be cleaned up with the
following:

runtests used a verbose flag that initialized a ‘V’
variable, but setting the verbose option was removed.
The flag is still in the script.  The ‘V’ flag should be
replaced by the global VERBOSE flag set in
test-frameworks.sh.

Insert tabs at the beginning of each line of test 1

Remove ‘\’ after ||

The error() function does not need a second argument and may
be confusing when reading the error message.  Remove the
second argument from the calls to error()

Replace exit() calls with calls to error() where appropriate

Use lower case for all letters of local variables and
declare them 'local'

Test-Parameters: trivial testlist=runtests
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I4e9f1eb467b004e5e2384dba4b7f047762304e8b
Reviewed-on: https://review.whamcloud.com/37016
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12100 tests: Use least qunit to set limit

Use least qunit to set lower limit for inodes in sanity-quota/2
This ensures that the limit is set at or above the minimum size.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-quota
Test-Parameters: testlist=sanity-quota fstype=zfs
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I80e2c3cb66870d11f74f34c435e266a46630479b
Reviewed-on: https://review.whamcloud.com/36797
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12854 nodemap: allow boolean value for audit_mode

Allow "true/false" or "on/off" for audit_mode property
on nodemap entries.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ibc6123dc64a88a80f474908d82e00269daacac69
Reviewed-on: https://review.whamcloud.com/36450
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>

LU-6142 libcfs: fix tab and alignment issues for libcfs_string.c

Resolve non-code changes for checkpatch errors.

Test-Parameters: trivial
Change-Id: Ie2848a83fdf58853ec32c4b188bcae1bcf4afa89
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/38623
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 utils: Fix style issues for obd.c

This patch fixes issues reported by checkpatch
for file lustre/utils/obd.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ic350a87f27fe13909d18f735d5044cb61e1654ea
Reviewed-on: https://review.whamcloud.com/38558
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13131 osc: Ensure immediate departure of sync write pages

Except for the case of direct-io and server-lock, we are
hold potentially multiple locks that are next to impossible
to find and cross reference.
So instead just send it all right away - should only
be a factor in rare cases of out of quota or close to out
of space.

Change-Id: I961cd9ba7f3266d22dfc5eff758c2f4ebbe148a4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38453
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-6142 utils: Fix style issues for lustre_rsync.c

This patch fixes issues reported by checkpatch
for file lustre/utils/lustre_rsync.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I4f4b3bd2455f2012561eb7567ba508e47c06749e
Reviewed-on: https://review.whamcloud.com/38441
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 kernel: use kmem_cache_zalloc as appropriate.

Rather than passing __GFP_ZERO to kmem_cache_alloc(), or calling
memset(0) after the allocation, use kmem_cache_zalloc().

Also update spelling.txt to encourage use of kmem_cache_zalloc().

kmem_cache_zalloc() has been part of Linux since 2.6.17.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic4ccbe0e223121e54699f7667e35db14d0f0da70
Reviewed-on: https://review.whamcloud.com/38439
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12222 ptlrpc: Check if NID is local, not just lolnd NID

There's a couple places where we check whether a NID is the lolnd NID
but we really want to know whether the NID is local. Use
LNetIsPeerLocal() to accomplish this.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ia17b9b4b54fd1063c42a6f8bdd0e593be1086683
Reviewed-on: https://review.whamcloud.com/38388
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 utils: Fix style issues for portals.c

This patch fixes issues reported by checkpatch
for file lustre/utils/portals.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I97eb576fe12a75e26d36dcf2228d5f161712e3e5
Reviewed-on: https://review.whamcloud.com/38362
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13131 osc: Do not wait for grants for too long

obd_timeout is way too long considering we are holding a lock
that might be contended. If OST is slow to respond, we might
get evicted, so limit us to a half of the shortest possible
max wait a server might have before switching to synchronous IO.

Change-Id: I36653194c1b8b95ba3cc2ed9240df7b0888cf7ed
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38283
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-9679 target: use OBD_ALLOC_PTR_ARRAY() and FREE

Use:
  OBD_ALLOC_PTR_ARRAY
  OBD_FREE_PTR_ARRAY

for allocating and freeing arrays in target.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I844a12e810c8a0bd6fb13ca047000de7f265988d
Reviewed-on: https://review.whamcloud.com/38255
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-9679 lodlov: use OBD_ALLOC_PTR_ARRAY() and others

Use:
  OBD_ALLOC_PTR_ARRAY
  OBD_FREE_PTR_ARRAY
  OBD_ALLOC_PTR_ARRAY_LARGE
  OBD_FREE_PTR_ARRAY_LARGE

for allocating and freeing arrays in lod and lov.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia782fc903761e21be982e4553ec40035c57f73f3
Reviewed-on: https://review.whamcloud.com/38254
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>

LU-13357 lod: implement striped directory .dio_lookup

Add function lod_striped_lookup() for
lod_striped_index_ops.dio_lookup to allow name lookup under striped
directory.

Currently this is used by subdir mount, which needs to lookup FID
of the subdir on server side.

Function lfsck_namespace_repair_dirent() should call dt_lookup() with
bottom object, because child may be shard.

Add sanity 247f.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Iba844d1a34a318bcbd42b00186ed6fa9d165effc
Reviewed-on: https://review.whamcloud.com/37903
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12003 osd: take reference to object in osd_trunc_lock()

normally the references to objects are held until a transaction
is over, but in few cases reference is released before. and then
such an object can be release, so OSD should have own reference
to prevent early release.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I81647fdec8d42f123e990553edb5e371636f45c0
Reviewed-on: https://review.whamcloud.com/37170
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11025 dne: support directory restripe

This patch adds directory restripe support:
* 'lfs setdirstripe -m -1 -c <stripe_count>' on an existed directory
  will change this directory layout, if 'stripe_count' is larger than
  current count, new stripes are allocated after current stripes,
  otherwise merge stripes of this directory, NB, if stripe count is
  unchanged, but hash type changed, it's treated as merging, but
  rehashing actually.
* mdt_restripe() ia added to restripe directory.
* mdd_dir_declare_layout_split() is added to split directory, which
  handles both plain and striped directory split.
* lod_dir_declare_layout_split() will handle the internal of directory
  split.
* directory merge is simple compared to split, which just records
  target stripe count in LMV, and update it.

NB. this patch only restripe directory, but doesn't add the code to
migrate sub files, which will be implemented in the following patch.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I526f7423b909eb83cf8723e65981d713b3e42499
Reviewed-on: https://review.whamcloud.com/36898
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11025 osd: osd_attr_get() returns dirent count

For osd-ldiskfs, to get dirent count it needs to iterate directory
entries and sum it up, while for osd-zfs, zap_count() can get it
from ZAP directly.

Add a new field 'la_dirent_count' in struct lu_attr, and set it
to directory entry count in osd_attr_get() if object is directory, and
this value will be cached in osd_object for osd-ldiskfs, if directory
is newly created, it will be set to '0', and later index_insert/delete
will update its value, so the following osd_attr_get() can use the
cached value.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If4225aecdba1c428d64d97c35b6c982c4932a265
Reviewed-on: https://review.whamcloud.com/38097
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11025 obdclass: add lu_device_operations::ldo_fid_alloc()

Add an interface ldo_fid_alloc in lu_device_operations, which is to
allocate a FID by parent object and sub file name, this will be
used to migrate sub files for directory restripe.

The existing osd_fid_alloc() and osp_fid_alloc() will switch to this
interface from obd_ops::o_fid_alloc().

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ide480fe492fd2d4d5de675bbc61aee7e2a9e3ce3
Reviewed-on: https://review.whamcloud.com/37282
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12931 ldlm: use proper units for timeouts

Use timeout_t for ns_ctime_age_limit since this is a relative time
and not an absolute time.

Use ktime_t for ns_dirty_age_limit internally, even though the user
interface is in seconds, since this is frequenty used together with
other ktime_t values in the kernel.

Fixes: e920be681451 ("LU-9019 ldlm: migrate the rest of the code to 64 bit time")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Idb5ece834e56a95cd781cb871b1b7c20bf3ebbe5
Reviewed-on: https://review.whamcloud.com/38365
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13411 llog: allow delete of zero size llog

1) all plain logs belonging to catalog should have flag
LLOG_F_ZAP_WHEN_EMPTY base on llog_cat_new_log(). When
llog_cat_process_common processing plain log with zero file size,
this flag is not set during llog_cat_id2handle LLOG_EMPTY, so these
plain llogs are not canceled/destroyed. They appeared during cross
MDT updates. Fix adds flag LLOG_F_ZAP_WHEN_EMPTY for any plain llog
at catalog.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-8634
Change-Id: Ieebee67bf9e7bebb9ecc51b858a9976a00583c7b
Reviewed-on: https://review.whamcloud.com/38131
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13416 ldiskfs: don't corrupt data on journal replay

Journalled write want a special attention on blocks release, revoke records must
added to avoid replace a new write blocks with stale data. Mark inode as
“journal write” generate a right revoke records. Large EA inode updates affected
with this bug also.

large ea fix is

commit ddfa17e4adc4bd19c32216aaa6250dc38b0579df
Author: Tahsin Erdogan <tahsin@google.com>
Date:   Wed Jun 21 21:36:51 2017 -0400
    ext4: call journal revoke when freeing ea_inode blocks

Change-Id: I605128c4ba70331a48715dc95546430909efb893
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/38281
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11643 tests: revert new images and tests for upgrade patch

Revert "LU-11643 tests: add new images and tests for upgrade tests".
This patch seem to cause conf-sanity test 32a to hang.
Let's revert the patch until this issue is understood.

This reverts commit 6b979daaffc36aeef145316b41d0e2fe8abcf20f.

Test-Parameters: trivial testlist=conf-sanity env=ONLY=32
Change-Id: Ic4a354420797a5968234925960278eaab86d22ca
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38619
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13541 llite: fix possible divide zero in ll_use_fast_io()

ll_use_fast_io() is used to check wheather we could use fast IO.
Since it is called in fast path, we don't hold ras_lock to protect
access, there might have the race @ras_stride_bytes is reset after
stride_io_mode() check.

Fixes: 9e4c5bdaaec5 ("LU-12644 llite: try fast io for stride io correctly")
Change-Id: If57ad074ecfa6560cc527f9f52e7adc2b0a456fd
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38545
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13246 osd: unlock os_lock if it was locked

do not use the global state (which can change concurrently) for that.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Idbdc3639bca50006dac00112205e1fee9c9a0e30
Reviewed-on: https://review.whamcloud.com/37547
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12662 tests: Add new pjdfstest into tests

Create a new POSIX test suite based on pjdfstest.

Test-Parameters: trivial
Test-Parameters: fstype=ldiskfs testlist=pjdfstest
Test-Parameters: fstype=zfs testlist=pjdfstest
Change-Id: Iec37e2248ce5ccf89319aaffb3ead9b407ad1931
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35841
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-11643 tests: add new images and tests for upgrade tests

Add new images for conf-sanity.sh 32

disk2_10-ldiskfs.tar.bz2
disk2_12-ldiskfs.tar.bz2

Test-Parameters: trivial
Test-Parameters: fstype=ldiskfs envdefinitions=ONLY="32f 32g" testlist=conf-sanity

Change-Id: I235c1d255db34fe8e6900e88d80265a3cd3eff18
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35049
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-3606 fallocate: Implement fallocate preallocate operation

This patch adds fallocate(2) preallocate operation support
for Lustre. fallocate(2) method of the inode_operations or
file_operations is implemented and transported to the OSTs
to interface with the underlying OSD's fallocate(2) code.

In a saperate patch, a new RPC, OST_FALLOCATE has been
added and reserved for space preallocation. The fallocate
functionality (prealloc) in CLIO has been multiplexed
with CIT_SETATTR. (https://review.whamcloud.com/37277)

Lustre fsx(File system exerciser) is updated in a saperate
patch to handle fallocate calls. (https://review.whamcloud.com/37277)

Only fallocate preallocate operation is supported by this
patch for now. Other operations like, FALLOC_FL_PUNCH
(deallocate), FALLOC_FL_ZERO_RANGE, FALLOC_FL_COLLAPSE_RANGE
and FALLOC_FL_INSPECT_RANGE is not supported by this patch
and will be addressed by a separate patch.

ZFS operation is not supported by this patch. ZFS fallocate(2)
will be addressed by patch (https://review.whamcloud.com/36506/)

New test case under sanity is added to verify fallocate
call.

Test-Parameters: fstype=ldiskfs testlist=sanity,sanityn,sanity-dom
Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Abrarahmed Momin <abrar.momin@gmail.com>
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I03f27d356616fbf3a3ab8e6309af26c00434d81b
Reviewed-on: https://review.whamcloud.com/9275
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13432 tests: test-framework to preserver MODOPTS_LIBCFS

do not reset MODOPTS_LIBCFS at module loading so that
if some one passes extra option to a test that's preserved
over a whole run rather than reset that at the first module
reload.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I235d09dc7ed4084b95e7f85528b952b75b16744d
Reviewed-on: https://review.whamcloud.com/38173
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13174 tests: sanity-quota to convert minutes into seconds

before arithmetic opertions

Change-Id: Iaefcd01dfe2066256010c2d195414a689c9f7573
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37327
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13486 llite: restore ll_dcompare()

Revert "LU-9868 llite: Get rid of ll_dcompare" as it is causing
failures in conf-sanity test_32* due to errors with sub-mounted
ext2 filesystem dentries being invalidated.

This reverts commit 787231f53ab63c72634250f8fe9d27bc66cc4e46.

Change-Id: I5cff42df30cf91523c1dbb8ee5b8eefa1ba3a5ad
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38498
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12477 ldiskfs: finish removing HAVE_CLEAR_RDONLY_ON_PUT

A recent patch removed almost all occurances of
HAVE_CLEAR_RDONLY_ON_PUT.  There is still one in
an ldiskfs patch, and it is the only reason for that
patch.

So remove the patch.

Test-Parameters: trivial
Fixes: 20553c985a51 ("LU-12477 kernel: remove dev_read_only handling")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5faf01aecd665b49bd3a74f9d968994c8db53c1c
Reviewed-on: https://review.whamcloud.com/38438
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13433 doc: fix lfs mirror "delete" and "split" doc

Add --mirror-id and --component-id in "lfs mirror delete" usage
Add examples with --comp-id in lfs mirror delete and split manpages
Add --comp-id in lfs mirror delete and split manpages

Test-Parameters: trivial
Signed-off-by: Kévin Baillergeau <kevin.baillergeau.ocre@cea.fr>
Change-Id: Icdc010eec9141b2582f9d43418523643b11018e5
Reviewed-on: https://review.whamcloud.com/38394
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13461 lnet: restrict gateway selection

This patch fixes a conflict between
LU-13026 lnet: MR selection of gateway ni
and
LU-12919 lnet: Fix source specified route selection

LU-12919 patch relied on lnet_find_best_lpni_on_net() to restrict
lpni selection on a specific network. However, it is necessary
to allow lpni selection on any network if the src net is not
specified. LU-13026 removed the restriction imposed by
lnet_find_best_lpni_on_net(), which broke the restriction
on the source net selection in some configuration setups.

This patch renames lnet_find_best_lpni_on_net() to
lnet_find_best_lpni(). If passed a peer_net it will find the
best lpni on the peer_net or return NULL if none is available.
If passed 'any' net, then it'll find the best reachable lpni
on any available net.

Fixes: e5614b45793 ("LU-13026 lnet: MR selection of gateway ni")
Fixes: f0aa632d425 ("LU-12919 lnet: Fix source specified route selection")
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ia2a655a6efcbdd1ac067bd622b030bb677b813cd
Reviewed-on: https://review.whamcloud.com/38298
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11310 ldiskfs: Repair support for SUSE 15 GA and SP1

At about the same time that support for SLE15 was added, some of the
patches mentioned in the series were removed by a separate patch.

This patch restores correct compiltation for ldiskfs with SUSE 4.12
kernels.

Specifically
     SLE15-LTSS release 4.12.14-150.47
     and 4.12.14-150.22
and
     SLES15-SP1 release 4.12.14-197.37
     and 4.12.14-197.7
which are the current releases as of 2020-apr-21 that  are supported.

Test-Parameters: trivial
Fixes: fc87b01f96e8 ("LU-12477 ldiskfs: remove obsolete ext4 patches")
Fixes: 862e9bf632dc ("LU-11310 ldiskfs: Support for SUSE 15 GA and SP1")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I76077feec52097345647048cefd2bb375c72e35e
Reviewed-on: https://review.whamcloud.com/38256
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13455 ptlrpc: connect to MDT stucks

mdc_ioc_hsm_ct_register() is executed on
IMP_EVENT_ACTIVE event.
If HSM_CT_REGISTER request expires import becomes
disconnected but reconnect can't start because
previous connect waits for completion of
HSM_CT_REGISTER request.

Mark HSM_CT_REGISTER request as no_resend.

Change-Id: Ia131f30453dc49cc800b56f256f5fb4bbb4147d6
Cray-bug-id: LUS-8409
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Ben Evans <jevans@cray.com>
Reviewed-on: https://review.whamcloud.com/38243
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9859 libcfs: replace memory_presure functions by standard interfaces

Use memalloc_noreclaim_save() and memalloc_noreclaim_restore(),
and for testing, just directly test the flag in current->flags

Linux-commit: fa399093620b867afe7686be47f197c8f17908c5

Change-Id: Id8f68164a7532ee4a816c71aff922f484c04877d
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/38211
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13426 mdd: correctly swap OIs during migration

While swapping layouts mdd_swap_layouts() swaps also OI(FID)
and uses lmm->lmm_oi directly in one place which is wrong if
corresponding layout is composite. Layout OI should be taken
by using mdd_get_lmm_oi() which handles all layout types
correctly.

Patch fixes that by using mdd_get_lmm_oi() instead of direct
access to lmm_oi field.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I6a7dc9ece3b272f75a5f206b32937cf9f674ead9
Reviewed-on: https://review.whamcloud.com/38172
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>

LU-13415 dom: use DoM optimization for DOM+OST files

Most DoM optimizations are used while DoM file has only
DoM stripe, assuming that with initialized OST stripes
most IO activity are shifted to OSTs. That is not quite
true because OST stripes can be initialized by any append
to the DoM file, so it still uses only DoM stripe but has
OST objects allocated.

Patch keeps DoM optimizations even if OST stripes are
initialized with few extra checks:
- for read-on-open it doesn't read file tail if whole DoM
  stripe is filled
- for IO lock-on-open feature it uses 'trylock' mode when
  taking DOM IO lock for files having OST stripes

Test-Parameters: mdssizegb=20 testlist=dom-performance
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I268bbbc947ca3e5d86339ce1a27d4641ca405a80
Reviewed-on: https://review.whamcloud.com/38152
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13344 lnet: stop using struct timeval

The struct timeval is not 2038 safe so the Linux kernel is moving
away from its use. The use of rpe_stamp hasn't been used since
Lustre 2.2 so remove the userland use of this field. This frees
use to change rpe_stamp to an equivalent struct timespec64 for
future use. Greatly simplify lnet_sock_[read|write] by using
jiffies values of sk_sndtimeo, sk_rcvtimeo cached in struct sock.

Change-Id: Ib58193756ec4a526e55bc810c05abd3920b2b269
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/38105
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13279 tests: sanityn.sh:test_20 takes DoM into account

sanityn.sh:test_20() should also cancel MDC locks as layout of files
may include DoM component.

Test-Parameters: testlist=sanityn
Cray-bug-id: LUS-8348
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I6157038235590d3c5b06d910c3b3d2600968a812
Reviewed-on: https://review.whamcloud.com/37662
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12974 lfsck: dryrun mode shouldn't repair for layout

"lfsck_start -t layout --dryrun" shouldn't repair but report
inconsistencies.

Update sanity-lfsck.sh 18b.

Test-parameters: trivial testlist=sanity-lfsck envdefinitions=ONLY=18
DDN-Bug-ID: DDN-892

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ie1f4d0d1726e8f3f39f5a8b7be9cc149df1c1e44
Reviewed-on: https://review.whamcloud.com/36767
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>

LU-11971 utils: Send file creation time to clients

Both ext4 and ZFS allow storing the file creation time in the
on-disk inode, and the new statx() API allows returning it to
userspace, but as yet we do not have any mechanism to send it
from the servers to the client.

Add fields into struct mdt_body and an OBD_MD_FLBTIME flag to
allow it to be requested and returned directly from MDTs. It
does not need to get it from the OSTs or in the LVB since those
objects are precreated and their creation time (birth time) is
not accurate.

This patch also adds support to lfs find to query on this field.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ib122b336695cde511fa6d8fb9eca95a4da701430
Reviewed-on: https://review.whamcloud.com/36507
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12214 build: fix build without lustre_utils

fix build without_lustre_utils enabled.
library and tools man pages separated to avoid package
build breakage.

Cray-bug-id: LUS-6036
Test-parameters: trivial
Change-Id: I0dbf821cff85b336d26555b6f2a8b659d9203de9
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/36429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-12652 ldiskfs: disable credits check in 4.18

4.18 has a credits check in xattr_set path with different math
resulting in false ENOSPC and subsequent failure to update LMA.
as a result - many tests fail on 4.18, especially with DNE.

Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com>
Change-Id: I82b19c0c24988aed66d74cf2323e93acffb14f31
Reviewed-on: https://review.whamcloud.com/35750
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13502 lnet: Add MD options for response tracking

These MD options will allow response tracking on a particular MD to
override the behavior specified by the lnet_response_tracking
parameter.

Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I2dee4e863138c1fe2ab1a1243c4c9e75e876f58f
Reviewed-on: https://review.whamcloud.com/38450
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13499 obd: fix printing of client connection UUID

The client connection UUID sent to the servers (ASCII format) was
being truncated to only 16 bytes in size, like '595f3c6a-20ae-4'
instead of a full UUID like '18ae0f9a-4b09-4599-8ced-0f2126eab425'.

This was caused by using UUID_SIZE to limit the size of the "%pU"
string printed to avoid overflowing the target buffer, but in fact
UUID_SIZE is the size of the binary uuid_t (16 bytes) instead of
the size of struct obd_uuid (40 bytes) where the ASCII version of
the UUID is stored.

Fix this to use sizeof(target) rather than an external constant,
which is exactly why sizeof(target) should always be used.  The
usage in osd_scrub.c is not actually broken, but it is still
better to use sizeof(target) to avoid future inconsistencies.

Fixes: 604c266a175b7 ("LU-11803 obd: replace class_uuid with linux kernel version")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I05325646cd632a09997d6632a483909629ce7057
Reviewed-on: https://review.whamcloud.com/38443
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13168 tests: verify truncated xattr is handled

Verify that a truncated trusted.lov xattr is handled properly,
for both plain and PFL layouts.

Add a test case that verifies this is fixed for both layout types.

Fixes: f2d06d3c76 ("LU-12911 llite: Don't access lov_md fields before size check")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I11d420c7fdc2362f64689a545b95c76e893ebbe5
Reviewed-on: https://review.whamcloud.com/38434
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13490 lustre: fix to make jobstats work for async ra

We use workqueue to triger async readahead, this will make
the problem that read jobstats are not accounted by real
IO, we could fix this by overwritting jobid which inited in
vvp_io_init().

Change-Id: I20b00b17f7a90cf94f1e5be54f47aee8ea2e5440
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38426
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 misc: declare static chars as const where possible.

When a char * is statically initialized to a literal string, it is
best if the char is declared 'const' to ensure that literal string can
never be changed, and to allow it to be stored in read-only memory.

Unfortunately this is not possible for strings used as module
parameters.  Similarly osd_0copy_tag in osd-zfs cannot be made const,
so that is change to a simple array (no pointer).

But in many other places it is possible, and it requires all ultimate
users of the string to declare that they are working with const chars.

This patch changes a collection of static char*s to const, and in many
cases arrays of char * to "const char * const", and then propagates
the 'const' annotion as needed.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I3185a6cd0d1b8a7dbd83698f456f65f47b4f2954
Reviewed-on: https://review.whamcloud.com/38421
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10467 llite: clean up pcc_layout_wait()

pcc_layout_wait() contains a pointless while loop (and an unnecessary
blank line) and is a trivial function that is only called once.

So move the code into __pcc_layout_invalidate(), remove the while
loop, and generally tidy up.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4a7c721a562682906dec4996bfbdad03e26a01dd
Reviewed-on: https://review.whamcloud.com/38403
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 osd-zfs: convert container_of0() to container_of()

Every use of container_of0() in osd-zfs can safely use container_of()
instead.  Doing so makes the intent of the code clearer.

In most cases, the pointer returned is later dereferenced without any
subsequent checks.  In a few cases (e.g.  in osd_internal.h), the
pointer passed in is dereferenced before the container_of() call.
These patterns assure us that the pointer in valid (not NULL or an
ERR_PTR), so container_of() is the correct interface to use.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I3f6d35c8b5beee09561e78c5eb20bb89d322590b
Reviewed-on: https://review.whamcloud.com/38379
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 lnet: use BIT() macro where appropriate

When accessing a bit in a bitmap/mask/flags-word it can be more
readable to use BIT(num) rather than "1 << num".

This patch makes that change to various places in lnet.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9a9234221f550c29f7799b0654c89ea9224c6d92
Reviewed-on: https://review.whamcloud.com/38375
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 libcfs: use BIT() macro where appropriate

When accessing a bit in a bitmap/mask/flags-word it can be more
readable to use BIT(num) rather than "1 << num".

This patch makes that change to various places in libcfs.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5e945358d2385599f8fe6f1ddff310efe17b589f
Reviewed-on: https://review.whamcloud.com/38374
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13054 ldiskfs: split htree_lock as separate patch

The htree_lock part is identical in the different
distro version of pdirop patch. So move it out as
separate patch to reduce maintenance effort.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I423cc957de37ccdb097c9893f69481ce947ac78c
Reviewed-on: https://review.whamcloud.com/38372
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>