Whamcloud - gitweb
fs/lustre-release.git
21 months agoLU-11429 mdt: rename mdt_remote_permission 47/33247/2
Andreas Dilger [Thu, 27 Sep 2018 11:41:01 +0000 (13:41 +0200)]
LU-11429 mdt: rename mdt_remote_permission

Rename mdt_remote_permission() to mdt_remote_dir_permission() and
mdt_remote_permission_check() to mdt_remote_dir_permission_check()
to match the "mdt_remote_dir" and "mdt_remote_dir_gid" proc tunable
names.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3336260b3fdc0a1ab3b12a7e2c4722c7a63ebbe5
Reviewed-on: https://review.whamcloud.com/33247
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11417 llapi: add llapi_layout_get_by_xattr(3) API 30/33230/2
Andreas Dilger [Tue, 25 Sep 2018 10:17:46 +0000 (12:17 +0200)]
LU-11417 llapi: add llapi_layout_get_by_xattr(3) API

Add new llapi_layout_get_by_xattr(3) interface to be able to extract
a layout structure from a LOV EA xattr.  This can be useful when the
xattr is retrieved from some external source (e.g. tarball, HSM, or
tools that directly access the underlying ldiskfs or ZFS filesystem).

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9b405bf6b3119e44097f36d49ac5859ff93ebbe5
Reviewed-on: https://review.whamcloud.com/33230
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11282 tests: log skip message for sanityn test_19 14/33214/2
Andreas Dilger [Fri, 21 Sep 2018 21:20:17 +0000 (15:20 -0600)]
LU-11282 tests: log skip message for sanityn test_19

Log the skip message for sanityn test_19() rather than just echoing it
and returning success.  The current behaviour makes it difficult to
see if this test is being run or not.

Test-Parameters: trivial testlist=sanityn
Test-Parameters: testlist=sanityn mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I05e2a9ef7c9fc5fb1be6533eab8bcb885f54ca6c
Reviewed-on: https://review.whamcloud.com/33214
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11010 tests: remove calls to return after skip() 33/32733/7
James Nunez [Tue, 26 Jun 2018 22:45:04 +0000 (16:45 -0600)]
LU-11010 tests: remove calls to return after skip()

The skip() routine now contains a call to exit. All calls
to skip() and skip_env() should be reviewed and calls to
return that follow skip() should be removed.

This is the fourth patch in a series that removes calls
to return after skip() in the Lustre test suites.

Calls to return after skip() are removed for:
mmp.sh
ost-pools.sh
parallel-scale-cifs.sh
parallel-scale-nfs.sh
posix.sh

Test-Parameters: trivial testlist=mmp
Test-Parameters: ostcount=1 osscount=1 testlist=ost-pools
Test-Parameters: mdtfilesystemtype=zfs testlist=posix,mmp
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ib6e4c81069b142652a0c50c339683dca21f03199
Reviewed-on: https://review.whamcloud.com/32733
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11392 tests: check race for llog_process_thread 92/33192/3
Alexander Boyko [Tue, 18 Sep 2018 12:58:41 +0000 (08:58 -0400)]
LU-11392 tests: check race for llog_process_thread

The patch adds 10h test at llog_test which runs at 60a sanity.
It reproduces a race between llog_process_thread and llog_add.
The llog should be wrapped so it has old data on disk and zero at
bitmap.
1. llog_process_thread reads part of llog at buffer.
1. process a last record, checks the next record fields
2. llog_add adds a record and marks new record at bitmap
1. check bitmap flag and process the old record from buffer

Test-Parameters: testlist=sanity envdefinitions=ONLY=60a
Cray-bug-id: LUS-6287
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: Ic89c81dd918d856f441df4d3257377e09b91a8cc
Reviewed-on: https://review.whamcloud.com/33192
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11193 llog: Do not write to read-only devices 57/33157/4
Nathaniel Clark [Thu, 13 Sep 2018 17:41:09 +0000 (13:41 -0400)]
LU-11193 llog: Do not write to read-only devices

Check if device is read-only before trying to start a transaction
on a devices.  Lustre snapshots are generally read-only.

When underlying device is read-only we check in the following places:
* llog_destroy, llog_write - return EROFS
* llog_open_create - on the create sized we return EROFS
* llog_cancel_rec - we return 0 which means success where 1 means
"success & log destroyed"

Test-Parameters: trivial testlist=sanity-lsnapshot mdtcount=4 mdscount=2 ostfilesystemtype=zfs mdtfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: Ia0083c57ceb589698b1422fec57e75aa6e68948a
Reviewed-on: https://review.whamcloud.com/33157
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11440 doc: recommend e2fsprogs 1.44.3.wc1 70/33370/2
Li Dongyang [Mon, 15 Oct 2018 00:07:03 +0000 (11:07 +1100)]
LU-11440 doc: recommend e2fsprogs 1.44.3.wc1

Update the recommended e2fsprogs version to 1.44.3.wc1

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I164f7c12ed718f1939d6fc392eb7bb7f9286f053
Reviewed-on: https://review.whamcloud.com/33370
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
21 months agoLU-11524 tests: fix sanity-sec test_31 for all situations 80/33380/5
Sebastien Buisson [Tue, 16 Oct 2018 12:52:46 +0000 (21:52 +0900)]
LU-11524 tests: fix sanity-sec test_31 for all situations

In case setupall() is called with server_only, this info must be
passed to init_param_vars(), and init_param_vars() must return
immediately. Otherwise, it will try to do client-specific tunings
(including quota settings) whereas no clients are mounted.

Modify cleanup_31() in sanity-sec so that client-specific
tunings are done via setupall().
Also make client umount more robust in test_31.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I2db17c139768d0842ff65ac8313a8e7d1484c4ef
Reviewed-on: https://review.whamcloud.com/33380
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
21 months agoLU-11329 utils: create tests maintainers list 60/33360/2
James Nunez [Fri, 12 Oct 2018 20:21:43 +0000 (14:21 -0600)]
LU-11329 utils: create tests maintainers list

Add the subsystem lustre/tests to the existing Lustre
subsystems in the Maintainers List and add myslef as a
maintainer.

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ic78dbd151d9cdf73fe32299da048bb103ed7592f
Reviewed-on: https://review.whamcloud.com/33360
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
21 months agoLU-11276 ldlm: don't apply ELC to converting and DOM locks 25/33125/5
Mikhail Pershin [Fri, 7 Sep 2018 10:23:48 +0000 (18:23 +0800)]
LU-11276 ldlm: don't apply ELC to converting and DOM locks

Prevent ELC for locks being converted and for locks
having DOM bit set to avoid data flush without need.

Test-Parameters: testlist=racer,racer,racer
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id1429412f2ccd77f037bef2a851d22874a44dce6
Reviewed-on: https://review.whamcloud.com/33125
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11347 osd: do not use pagecache for I/O 75/32875/13
Alex Zhuravlev [Wed, 25 Jul 2018 10:24:27 +0000 (14:24 +0400)]
LU-11347 osd: do not use pagecache for I/O

for testing purposes cache is constantly disabled.

 - with non-rotational storage
 - when both read and write caches are disable
 - sanityn/16c to run fsx with cache disable

Change-Id: If6ea9186485cd0aceb0372b68f4860de3a4fb124
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32875
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11199 mdt: Attempt lookup lock on open 29/32929/5
Patrick Farrell [Tue, 28 Aug 2018 14:28:29 +0000 (09:28 -0500)]
LU-11199 mdt: Attempt lookup lock on open

Commit 4f50273a (LU-10269 ldlm: fix the issues introduced
by try bits) changed the locking behavior on open to not
attempt to grant the LOOKUP lock bit.  This causes a
performance regression in open(), which is up to 75% in
some benchmarks, such as mdsrate (from lustre/tests/mpi):

First create the files:
mpirun -n 4 /usr/lib64/lustre/tests/mdsrate /
-d /mnt/lustre/mdsrate --create --nfile=30000

Then drop caches:
echo 3 > /proc/sys/vm/drop_caches

Then run the open benchmark:
mpirun -n 4 /usr/lib64/lustre/tests/mdsrate /
-d /mnt/lustre/mdsrate --open --iters 8000 --nfile=30000
[More details in LU-1199]

This patch reverts that specific part of 4f50273a, which
restores open() performance to prior levels.

It may not be 100% correct to ask for open only when
also asking for layout, but this is the earlier behavior.

There is a further patch in flight to optimize this
code:
https://review.whamcloud.com/32156

And further changes are left for that patch.

Cray-bug-id: LUS-6358

Change-Id: Iceca88807e99955f28eba6bbcb3585964f7df2f4
Signed-off-by: Patrick Farrell <paf@cray.com>
Reviewed-on: https://review.whamcloud.com/32929
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11473 doc: add lfs-getsom man page 90/33290/2
James Nunez [Thu, 4 Oct 2018 21:32:42 +0000 (15:32 -0600)]
LU-11473 doc: add lfs-getsom man page

The Lazy Size on MDT feature added a flag to 'lfs' to allow
users to get the LSOM data. Add the lfs "getsom" flag to the
lfs man page.

Also, make a minor correction to the llsom_sync man page that
corrects the order of the --device flag for changelog_register
and changelog_deregister.

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I9a5acab6de3b7f32eb246c94e9975e30f63f10e1
Reviewed-on: https://review.whamcloud.com/33290
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-1095 misc: quiet console messages at startup 81/33281/2
Andreas Dilger [Wed, 3 Oct 2018 23:16:23 +0000 (17:16 -0600)]
LU-1095 misc: quiet console messages at startup

Some modules print less-than-useful messages on every load.
Turn these into internal debug messages to reduce noise.

The message in gss_init_svc_upcall() should also be quieted,
but it exposes that this function is waiting 1.5s on each module
load for lsvcgssd to start.  This should be fixed separately.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib51ce0e9a88a94d8d2d5eb0906abef0f544cab07
Reviewed-on: https://review.whamcloud.com/33281
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11454 ptlrpc: Make CPU binding switchable 62/33262/6
Patrick Farrell [Thu, 4 Oct 2018 12:10:11 +0000 (07:10 -0500)]
LU-11454 ptlrpc: Make CPU binding switchable

LU-6325 added CPT binding to the ptlrpc worker threads on
the servers.  This is often desirable, especially where
NUMA latencies are high, but it is not always beneficial.

If NUMA latencies are low, there is little benefit, and
sometimes it can be quite costly:

In particular, if NID-CPT hashing with routers leads to an
unbalanced workload by CPT, it is easy to end up in a
situation where the CPUs in one CPT are maxed out but
others are idle.

To this end, we add module parameters to allow disabling
the strict binding behavior, allowing threads to use all
CPUs.

This is complicated a bit because we still want separate
service partitions - The existing "no affinity" behavior
places all service threads in a single service partition,
which gives only one queue for service wakeups.

So we separate binding behavior from CPT association,
allowing us to keep multiple service partitions where
desired.

Module parameters are added to ldlm, mdt, and ost, of the
form "servicename_cpu_bind", such as "mds_rdpg_cpu_bind".

Setting them to "0" will disable the strict CPU binding
behavior for the threads in that service.

Parameters were not added for certain minor services which
do not have any CPT affinity/binding behavior today.  (This
appears to be because they are not expected to be
performance sensitive.)

cray-bug-id: LUS-6518
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I1f6f9bb7a11da3a3eec7fc14c41d09ed27700f46
Reviewed-on: https://review.whamcloud.com/33262
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11448 kernel: kernel update RHEL7.5 [3.10.0-862.14.4.el7] 54/33254/5
Jian Yu [Sat, 6 Oct 2018 06:35:45 +0000 (23:35 -0700)]
LU-11448 kernel: kernel update RHEL7.5 [3.10.0-862.14.4.el7]

Update RHEL7.5 kernel to 3.10.0-862.14.4.el7.

Change-Id: I4901102347a14d23645547efc84857868acec0f7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33254
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11369 hsm: allow non-owner writers to set HSM state dirty 58/33158/2
John L. Hammond [Thu, 13 Sep 2018 17:52:45 +0000 (12:52 -0500)]
LU-11369 hsm: allow non-owner writers to set HSM state dirty

In mdt_add_dirty_flag(), bump up the capability so that
mdt_hsm_attr_set() will succeed even if the writer (or truncater) is
not the owner of the file.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ibd7e9e039c3a984642b4a01c63cd11d2029e93f1
Reviewed-on: https://review.whamcloud.com/33158
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-8066 llite: make llite/lov and lmv symlinks 16/32516/4
James Simmons [Mon, 8 Oct 2018 15:17:55 +0000 (11:17 -0400)]
LU-8066 llite: make llite/lov and lmv symlinks

old proc code had /proc/sys/fs/lustre/llite/.../lov and lmv
dirs that contained name of the dir in lustre/lov and lustre/lmv
to better be able to find correct obd device there, but
I imagine a better solution would be to just create a symlink with
the same name. The name is then pointless and the target dir would
have uuid file just as if it was the old-style dir.

This is a modified version of

Linux-commit : d8ede3f1d5d94618442a61067c6b98a2afbb0962

Change-Id: I90bc1b75e07f0aaa4c3119671f6d097b0e7353b3
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32516
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-8324 hsm: prioritize one RESTORE once in a while 23/31723/14
Quentin Bouget [Fri, 4 May 2018 12:53:12 +0000 (14:53 +0200)]
LU-8324 hsm: prioritize one RESTORE once in a while

Currently, HSM requests are run in the order they are submitted to
the coordinator. This has the undesirable side effect that the more
interactive requests (RESTORE / REMOVE) can take quite a while to be
run when a huge batch of ARCHIVE requests are already queued.

This patch is not a clean fix to LU-8324, it is merely an attempt at
making things bearable while a proper solution is being developped.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I8697ffae19b28f31901d9e61cce55b40f848fb51
Reviewed-on: https://review.whamcloud.com/31723
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-8950 tests: do not use make_custom_file_for_progress() 94/24394/17
Quentin Bouget [Wed, 20 Dec 2017 15:12:36 +0000 (15:12 +0000)]
LU-8950 tests: do not use make_custom_file_for_progress()

Do not use make_custom_file_for_progress() in sanity-hsm in tests:
12c, 26, 27b, 28, 31b, 31c, 33, 34, 35, 36, 54, 55, 56, 60, 62, 71,
104, 200, 202, 221, 223b, 225, 251, 252, 407

Before this patch make_custom_file_for_progress() was used to create
big files (5MB to 40 MB). There were several use cases for that:
 - in combination with the --bandwidth option of lhsmtool_posix, it
   allows synchronizing HSM operations and other things;
 - to log enough "progress events", HSM operations need to last long
   enough (the --bandwidth option comes in handy too).

It needed to be removed because:
 - archiving and restoring big files at a 1MB/s rate takes too much
   time (admittedly, that was the point);
 - there are other way to have HSM operations occur concurently. For
   example: suspending copytools to delay request processing.
 - make_custom_file_for_progress() does not correctly reflect what
   the function does and should be used for. Wherever big file are
   still needed, the patch uses create_file() instead.

Removing the need to archive or restore big files at a limited rate
represents quite a speed-up on sanity-hsm.

Test-Parameters: trivial testlist=sanity-hsm,sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: Ib9eb7599f9e16d8790630c69c9d1c7be3df416a1
Reviewed-on: https://review.whamcloud.com/24394
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jean-Baptiste Riaux <riaux.jb@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-1757 osc: clarify short_io_bytes is maximum value 73/33173/3
Andreas Dilger [Sat, 15 Sep 2018 18:39:07 +0000 (12:39 -0600)]
LU-1757 osc: clarify short_io_bytes is maximum value

Clarify in the code that the "osc.*.short_io_bytes" parameter is
the maximum IO size to pack into request/reply not the minimum.

Allow short_io to be disabled completely if it is set to zero.

It would be nice to also change the name of the /proc functions
in a similar manner but that also changes the /proc tunable name
(via LUSTRE_RW_ATTR() macro) and has compatibility implications
for sites that may have changed this value.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3a91420d7e0405445558ba9203233d7f977cab07
Reviewed-on: https://review.whamcloud.com/33173
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11057 tests: exercise 'network' client mount option 89/33189/10
Sebastien Buisson [Tue, 18 Sep 2018 09:59:32 +0000 (18:59 +0900)]
LU-11057 tests: exercise 'network' client mount option

Test 'network' client mount option that enables to limit connections
from the client to be on the specified network NID.

This option is currently not compatible with Dynamic Peer Discovery.
test_31 will have to be updated once Dynamic Peer Discovery is
modified so that it does not conflict with 'network' client mount
option.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I13f016fad52a7b73524f652b369c6434985fc0c6
Reviewed-on: https://review.whamcloud.com/33189
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11196 tests: clean up after conf-sanity test_101 68/33268/4
Andreas Dilger [Tue, 2 Oct 2018 20:27:09 +0000 (14:27 -0600)]
LU-11196 tests: clean up after conf-sanity test_101

conf-sanity test_101() creates up to 50000 files in the top-level
test directory, which can sometimes cause the later test_103() setup
to fail, because "rm -rf" fails with "Argument list too long" when
trying to clean up the test directory.

Create the test_101 files in a subdirectory for cleanliness, and
remove them at the end of the test so that we don't run out of space.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1167339d03340e1cc545d2855c4b32eef18cab07
Reviewed-on: https://review.whamcloud.com/33268
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
21 months agoLU-11511 fix build breakage with LMV_HASH_FLAG_DEAD undeclared 56/33356/2
Oleg Drokin [Fri, 12 Oct 2018 03:56:52 +0000 (23:56 -0400)]
LU-11511 fix build breakage with LMV_HASH_FLAG_DEAD undeclared

It was being deprecated with 2.11.56, but actually 2.8 was waaay too
recent to kill it like that, let's postpone this until 2.12.53 for now
and definitely let's fix it by then.

Change-Id: Ibfc2cd858803e7802536041775fb09ae6960f211
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33356
Tested-by: Jenkins
22 months agoNew tag 2.11.56 2.11.56 v2_11_56 v2_11_56_0
Oleg Drokin [Thu, 11 Oct 2018 18:42:45 +0000 (14:42 -0400)]
New tag 2.11.56

Change-Id: I9e75c41c31acdaddffef64eee83d68523ebbd3d2
Signed-off-by: Oleg Drokin <green@whamcloud.com>
22 months agoRevert "LU-11058 test: reenable sanity 77k test" 54/33354/2
Oleg Drokin [Thu, 11 Oct 2018 18:41:54 +0000 (18:41 +0000)]
Revert "LU-11058 test: reenable sanity 77k test"

I guess we spoke too soon and there's now again a bunch of
failures tracked under LU-11058 for this test.

This reverts commit fb1875c314cf3650aa7ba6b2cdb72ef18e812d5f.

Change-Id: I1d17f4b7457f8a057279e58114f93a4affbf115c
Reviewed-on: https://review.whamcloud.com/33354
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11422 lnet: Fix selftest backward compatibility post health 42/33242/8
Sonia Sharma [Sun, 23 Sep 2018 16:15:31 +0000 (12:15 -0400)]
LU-11422 lnet: Fix selftest backward compatibility post health

Post LNet health feature landing, lnet-selftest loses
backward compatibility. This patch fixes that by
adding a new structure lnet_counters_common similar
to lnet_counters(pre-Health version). Now,
lnet_counters_common is the struct that selftest depends on.

Also, adds a struct lnet_counters_health specifically
for health stats.

Change-Id: Ia066ff7b3522738080d192e80c14cf5ac55a33e2
Test-Parameters: trivial testlist=lnet-selftest clientjob=lustre-b2_10 clientbuildno=136
Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33242
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Jenkins
Tested-by: Andreas Dilger <adilger@whamcloud.com>
22 months agoLU-11388 test: disable replay-single test_131b 79/33279/2
Mikhail Pershin [Wed, 3 Oct 2018 18:58:37 +0000 (21:58 +0300)]
LU-11388 test: disable replay-single test_131b

Add test to ALWAYS_EXCEPT list for now due to LU-11388
problem. Change also 'bs' and 'count' in test to have
only one RPC being sent.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I76fb56896d53674c7eb9456a8bcbb1b2a23629c7
Reviewed-on: https://review.whamcloud.com/33279
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11375 mdc: use old statfs format 62/33162/7
Alex Zhuravlev [Thu, 13 Sep 2018 18:07:08 +0000 (21:07 +0300)]
LU-11375 mdc: use old statfs format

when the client talks to old server with no support
for aggregated statfs

Test-Parameters: clientjob=lustre-b2_10 clientbuildno=136 testgroup=review-ldiskfs

Change-Id: I447b312d46db56da152f62835b3f98401f997cf0
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33162
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: James Simmons <uja.ornl@yahoo.com>
22 months agoLU-6142 osd-ldiskfs: Fix style issues for osd_iam_lvar.c 50/32850/3
Arshad Hussain [Sat, 21 Jul 2018 23:41:59 +0000 (05:11 +0530)]
LU-6142 osd-ldiskfs: Fix style issues for osd_iam_lvar.c

This patch fixes issues reported by checkpatch
for file lustre/osd-ldiskfs/osd_iam_lvar.c

Change-Id: I11d4836e869314accc1123f2167db556f82e823f
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/32850
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-7004 tests: move from lctl conf_param to lctl set_param -P 87/30087/24
James Simmons [Thu, 20 Sep 2018 18:32:11 +0000 (14:32 -0400)]
LU-7004 tests: move from lctl conf_param to lctl set_param -P

To ensure that the work to make lctl set_param -P functional
we need to provide a way to test either lctl conf_param or
lctl set_param -P in the test suite.

Change-Id: I2f58a9b5262eed03ddc3a2ff15ac96b752d97735
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30087
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-6202 libcfs: replace libcfs_register_ioctl with a blocking notifier_chain 84/33284/4
NeilBrown [Fri, 5 Oct 2018 13:02:41 +0000 (09:02 -0400)]
LU-6202 libcfs: replace libcfs_register_ioctl with a blocking notifier_chain

libcfs allows other modules to register handlers for ioctls.
The implementation it uses for this is nearly identical to a
blocking notifier chain, so change to use that.

The biggest difference is that the return value from notifier has a
defined format, where libcfs_register_ioctl uses -EINVAL to mean
"continue".  This requires a little bit of conversion.

Linux-commit: 912a846027c203370f4837a2b2db7ba2ebe1b47c

Change-Id: I84657fc1befd004eb09de37b5549af8bff034738
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33284
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11441 obdclass: make mod rpc slot wait queue FIFO 82/33282/3
Vladimir Saveliev [Thu, 4 Oct 2018 11:50:56 +0000 (14:50 +0300)]
LU-11441 obdclass: make mod rpc slot wait queue FIFO

Relatively big load may cause a process to spin for a long time
without successful grabbing a free mod rpc slot. It has been observed
a process spinning more than 100 seconds when there were 72 mdtest-s
and 8 IOR-s.

Make mod rpc slot wait queue to run FIFO so that waiting thread got
free mod rpc slot in order they entered the queue.

Cray-bug-id: LUS-6380
Change-Id: I4ef2cf529cf1bb4c8500d7a2b181a05ca080223e
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-on: https://review.whamcloud.com/33282
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-930 doc: update data_version command in lfs man page 78/33278/2
Nikitas Angelinas [Wed, 3 Oct 2018 18:14:14 +0000 (11:14 -0700)]
LU-930 doc: update data_version command in lfs man page

Update the lfs man page to document the [rw] options of the
data_version command, and list some of the command use cases.

Signed-off-by: Nikitas Angelinas <nangelinas@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Cray-bug-id: LUS-5890
Test-Parameters: trivial
Change-Id: I2522de9f9a987f5c085d370f6dafc0df4080f5fa
Reviewed-on: https://review.whamcloud.com/33278
Tested-by: Jenkins
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11451 utils: fix to dump usage messages to stdout 56/33256/2
Wang Shilong [Sun, 30 Sep 2018 05:48:26 +0000 (13:48 +0800)]
LU-11451 utils: fix to dump usage messages to stdout

-h option output should be expected to stdout, rather
than stderr.

Test-parameters: trivial
Change-Id: I6209327a926c4482eb8b9174bb44aa2c5ba76c2d
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33256
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
22 months agoLU-11370 lnet: cleanup ni_cptlist 55/33155/2
Sonia Sharma [Fri, 31 Aug 2018 08:14:50 +0000 (04:14 -0400)]
LU-11370 lnet: cleanup ni_cptlist

With the changes added for the multi-rail feature,
ni_cptlist related code was removed from
lnet_startup_lndni() and other places. It lurked
behind at few places, cleaning up from there with
this patch

Change-Id: Ied557802df4feb869cea57cb7f44724a550b1ec1
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33155
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11364 osd: remove unused 'ignore quota' parameters 38/33138/3
John L. Hammond [Tue, 11 Sep 2018 16:17:47 +0000 (11:17 -0500)]
LU-11364 osd: remove unused 'ignore quota' parameters

Remove the unused 'ignore quota' parameters from dt_write(),
dt_insert(), and their implementations.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib6e61c5dff8c2b1027ed3a7e0808fe208fd17d84
Reviewed-on: https://review.whamcloud.com/33138
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11058 test: reenable sanity 77k test 99/33099/4
James Simmons [Thu, 4 Oct 2018 18:07:57 +0000 (14:07 -0400)]
LU-11058 test: reenable sanity 77k test

Reenable sanity 77k test because test_77k was fixed via patch
https://review.whamcloud.com/33190 commit bda43cbe36

Test-Parameters:trivial testlist=sanity envdefinitions=ONLY=77k

Change-Id: I01b8c2f2b81bcb4a0de71fa73e71fa32e7f9dc61
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33099
Tested-by: Jenkins
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10595 mgc: don't proccess cld during stopping 90/33190/2
Alexander Boyko [Tue, 18 Sep 2018 12:21:52 +0000 (08:21 -0400)]
LU-10595 mgc: don't proccess cld during stopping

The patch fixes the log processing during stopping. It was general
protection fault at mgc_process_cfg_log() at lsi access. Lsi pointer
was wrong 38323172756f6663, and all cld->cld_cfg.cfg_sb had invalid
data.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6199
Change-Id: Ie4f35ba5be707ab8d274cc26fd2c230ffcb0a16e
Reviewed-on: https://review.whamcloud.com/33190
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: move remaining single item proc files to sysfs 15/32515/4
James Simmons [Wed, 23 May 2018 15:22:11 +0000 (11:22 -0400)]
LU-8066 llite: move remaining single item proc files to sysfs

Remove the remaining single item proc files for llite that are
not present in the linux client to sysfs.

Change-Id: Ic7c8d61a0798e8f2385d878a398f29e8b231c6fe
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32515
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
22 months agoLU-11287 ldlm: update l_blocking_lock under lock 24/33124/6
Mikhail Pershin [Fri, 7 Sep 2018 10:34:46 +0000 (18:34 +0800)]
LU-11287 ldlm: update l_blocking_lock under lock

Update l_blocking_lock under with locking to prevent race
between lock_handle_convert0() and ldlm_work_bl_ast() code.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I881a1daf6f3b09677abcd6a85f6891d409926cc8
Reviewed-on: https://review.whamcloud.com/33124
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-6142 obdclass: Fix style issues for obd_sysfs.c 83/32983/2
Arshad Hussain [Sun, 12 Aug 2018 05:44:27 +0000 (11:14 +0530)]
LU-6142 obdclass: Fix style issues for obd_sysfs.c

This patch fixes issues reported by checkpatch
for file lustre/obdclass//obd_sysfs.c

Change-Id: I77fd2279a2f53ee18557eda3f92bca8286db3ce7
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/32983
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
22 months agoLU-8066 llite: change how "dump_page_cache" walks a hash table 11/33011/3
NeilBrown [Mon, 17 Sep 2018 17:44:56 +0000 (13:44 -0400)]
LU-8066 llite: change how "dump_page_cache" walks a hash table

The "dump_page_cache" seq_file currently tries to encode
a location in the hash table into a 64bit file index so that
the seq_file can seek to any location.

This is not necessary with the current implementation of seq_file.
seq_file performs any seeks needed itself by rewinding and calling
->next and ->show until the required index is reached.

The required behaviour of ->next is that it always returns the next
object after the last one returned by either ->start or ->next.
It can ignore the ppos, but should increment it.

The required behaviour of ->start is one of:
1/ if *ppos is 0, then return the first object
2/ if *ppos is the same value that was passed to the most recent call
   to either ->start or ->next, then return the same object again
3/ if *ppos is anything else, return the next object after the most
   recently returned one.

To implement this we store a vvp_pgcache_id (index into hash table)
in the seq_private data structure, and also store 'prev_pos' as the
last value passed to either ->start or ->next.

We remove all converstion of an id to a pos, and any limits on the
size of the vpi_depth.

vvp_pgcache_obj_get() is changed to ignore dying objects so that
vvp_pgcache_obj only returns NULL when it reaches the end of a hash
chain, and so vpi_bucket needs to be incremented.

A reference to the current ->clob pointer is now kept as long as we
are iterating over the pages in a given object, so we don't have to
try to find it again (and possibly fail) for each page.

And the ->start and ->next functions are changed as described above.

Test-Parameters: trivial envdefinitions="ONLY=63b" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=118" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=20" testlist=sanityn

Change-Id: I2c6d2be47f809cb4a944aa3de6f561ce4f08bbf4
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33011
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11455 build: Remove conflicting lustre and Lustre 65/33265/2
Nathaniel Clark [Tue, 2 Oct 2018 12:08:56 +0000 (08:08 -0400)]
LU-11455 build: Remove conflicting lustre and Lustre

Some systems (Mac) are still case insensivite under some conditions,
so having lustre and Lustre in the same directory causes problems.
Seperate all the resource agents into their own directory.

Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I6df370744ffecec9b9e52ffa9f131f02f351f6c5
Reviewed-on: https://review.whamcloud.com/33265
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
22 months agoLU-11412 kernel: kernel update [SLES12 SP3 4.4.155-94.50] 36/33236/3
Jian Yu [Thu, 27 Sep 2018 17:07:21 +0000 (10:07 -0700)]
LU-11412 kernel: kernel update [SLES12 SP3 4.4.155-94.50]

Update SLES12 SP3 kernel to 4.4.155-94.50.

Test-Parameters: envdefinitions=CONF_SANITY_EXCEPT=103 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
clientdistro=sles12sp3 ossdistro=sles12sp3 mdsdistro=sles12sp3 \
testlist=sanity,conf-sanity,sanity-sec

Change-Id: I47b72420664fe614a2da7c864c401a9729d96c55
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33236
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11368 build: link the shared libraries properly 46/33146/2
Li Dongyang [Wed, 12 Sep 2018 12:13:02 +0000 (22:13 +1000)]
LU-11368 build: link the shared libraries properly

Make sure the -l options come after the objects.
Otherwise on distros(e.g. ubuntu18) use --as-needed
by default, we won't actully link against those libs.

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: If828fc409a54e48e0818f8a11087c266d46652c1
Reviewed-on: https://review.whamcloud.com/33146
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-9966 tests: sanity-411 check LBUG direct 93/32293/6
Yang Sheng [Fri, 4 May 2018 10:20:49 +0000 (18:20 +0800)]
LU-9966 tests: sanity-411 check LBUG direct

The sanity test_411 intends to catch LBUG while IO
under memory pressure. It assumes that 'dd' would
fail because memory alloction failure. But it is
not ture since RHEL7 has some problem in memory
accounting. So we can ignore the 'dd' status.

Test-Parameters: trivial

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ie9af0fe68c85a9e2af4da6b49d0a697d00f3761b
Reviewed-on: https://review.whamcloud.com/32293
Tested-by: Jenkins
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
22 months agoLU-10772 utils: incorrect check in build_layout_from_yaml_node 37/31537/3
Sonia Sharma [Tue, 6 Mar 2018 08:39:27 +0000 (00:39 -0800)]
LU-10772 utils: incorrect check in build_layout_from_yaml_node

In lustre/utils/lfs.c, build_layout_from_yaml_node(),
check and fail for NULL pointer 'string' before
dereferencing it.

Issue found in Static analysis.

Change-Id: I272432d3997f3bfae520ff3aa48d175309ccd386
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/31537
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11424 lnet: copy the correct amount of cpts to lnet_cpts 29/33229/2
James Simmons [Tue, 25 Sep 2018 03:28:57 +0000 (23:28 -0400)]
LU-11424 lnet: copy the correct amount of cpts to lnet_cpts

The incorrect size was used in the memory copy of the requested
cpts to net->lnet_cpts. This lead to the following in testing
RIP: 0010:lnet_match2mt.isra.8+0x2b/0x40 [lnet]

lnet_mt_of_attach+0x72/0x1b0 [lnet]
LNetMEAttach+0x60/0x1f0 [lnet]
ptl_send_rpc+0x26f/0xbb0 [ptlrpc]
libcfs_debug_msg+0x57/0x80 [libcfs]
ptlrpc_send_new_req+0x4c9/0x860 [ptlrpc]
ptlrpc_check_set.part.21+0x855/0x18b0 [ptlrpc]
? try_to_del_timer_sync+0x4d/0x80
? del_timer_sync+0x35/0x40
ptlrpcd_check+0x3ae/0x3f0 [ptlrpc]
ptlrpcd+0x2be/0x320 [ptlrpc]
? wait_woken+0x80/0x80

Changing the size from ncpts to ncpts * sizeof(*net->net_cpts)

Change-Id: I10832430e53ccc5b40ebce3ddfd2cf9ea330b0df
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33229
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11187 ldiskfs: don't mark mmp buffer head dirty 38/33038/10
Li Dongyang [Tue, 21 Aug 2018 00:10:45 +0000 (10:10 +1000)]
LU-11187 ldiskfs: don't mark mmp buffer head dirty

Marking mmp bh dirty before writing it will make writeback
pick up mmp block later and submit a write, we don't want the
duplicate write as kmmpd thread should have full control of
reading and writing the mmp block.
Another reason is we will also have random I/O error on
the writeback request when blk integrity is enabled, because
kmmpd could modify the content of the mmp block(e.g. setting
new seq and time) while the mmp block is under I/O requested
by writeback.

Linux-commit: fe18d649891d813964d3aaeebad873f281627fbc

Test-Parameters: testgroup=review-ldiskfs testlist=mmp
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I5aa9fd384a4ea25ee52f1198528fae4ecc9c28c7
Reviewed-on: https://review.whamcloud.com/33038
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
22 months agoLU-11198 utils: propagate errors for read_param 25/32925/9
Wang Shilong [Fri, 28 Sep 2018 17:31:29 +0000 (13:31 -0400)]
LU-11198 utils: propagate errors for read_param

Steps to reproduce:

 $ lctl get_param -n ost.OSS.ost_io.nrs_tbf_rule
 error: get_param: reading 'ost.OSS.ost_io.nrs_tbf_rule': No such device
 $ echo $?
 0

read_param() might return errors, fix to return errors to
caller.

Change-Id: Id165424342be3109669f173b9b72c16aa8c83425
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/32925
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11414 ptlrpc: don't change buffer when signature is ready 23/33223/2
Mikhail Pershin [Sun, 23 Sep 2018 20:07:37 +0000 (23:07 +0300)]
LU-11414 ptlrpc: don't change buffer when signature is ready

The lm_repsize is part of buffer being used in signature calculation
and must not be changed after calculation is done.

Patch reverts related changes from commit 13372d6c and moves related
lm_repsize update into MDC where DOM read-on-open buffer is prepared

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic277ca26ad8fb8e23ee7e4b6e81d84a1d51b4733
Reviewed-on: https://review.whamcloud.com/33223
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11420 mdt: export mdt_dom_read_open_fops 24/33224/3
Alex Zhuravlev [Mon, 24 Sep 2018 06:55:20 +0000 (09:55 +0300)]
LU-11420 mdt: export mdt_dom_read_open_fops

Read-on-open MDT lproc fops were removed by mistake in
commit 169738e3, restore lost changes and fix compilation
failures with gcc8

Change-Id: I89de27b76e061b67d175d026a7e563d48df78603
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33224
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11436 build: always dist debian stuff 46/33246/3
Gu Zheng [Thu, 27 Sep 2018 08:59:33 +0000 (16:59 +0800)]
LU-11436 build: always dist debian stuff

Always try to dist debian stuff, so that even on non-dpkg-using
build targets (RHEL series), the tar dist will contain all debian
stuff, and can be delivered to users who want to build lustre on
debian series servers.

Test-Parameters: trivial
Change-Id: I8493f5a5e65078c0160c5493efee6906c87db6f1
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/33246
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10906 llite: create checksums to replace checksum_pages 22/33222/3
James Simmons [Sat, 22 Sep 2018 21:13:50 +0000 (17:13 -0400)]
LU-10906 llite: create checksums to replace checksum_pages

Create llite.*.checksums, which matches llite.*.checksum_pages in
functionality. Now the llite layer have something that matches
osc.*.checksums. In time we can retire checksum_pages and change
it to its original purpose of enabling per-page checksums (which
was not implemented in the CLIO development).

Test-Parameters: trivial

Change-Id: I0e381e3626e835a448a6285cb81d27d67af93a54
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33222
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11400 flr: add 'nosync' flag for FLR mirrors 05/33205/6
Bobi Jam [Thu, 20 Sep 2018 07:50:45 +0000 (15:50 +0800)]
LU-11400 flr: add 'nosync' flag for FLR mirrors

This patch allows 'nosync' flag to be set for FLR mirror components,
which makes lfs mirror resync skip on mirrors with this flag unless
mirror resync explicitly requested those mirrors to be resync.

This flag can be cleared by set '^nosync' on any component of the
mirror.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I209ad91fd9a96d82f8c83a3eb047f665ef4ccf86
Reviewed-on: https://review.whamcloud.com/33205
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11397 idl: remove obsolete RPC flags 02/33202/2
Andreas Dilger [Wed, 19 Sep 2018 06:01:43 +0000 (02:01 -0400)]
LU-11397 idl: remove obsolete RPC flags

Remove RPC flags that are no longer in use:
- OBD_MD_FLQOS has never been used in master branch
- OBD_MD_FLEPOCH unused since v2_7_50_0-38-gd5d5b349f2
- OBD_MD_REINT unused since before 1.6
- OBD_MD_FLMDSCAPA unused since v2_7_55_0-15-g353ef58b1d
- OBD_MD_FLOSSCAPA unused since v2_7_55_0-15-g353ef58b1d

Rename OBD_MD_FLGENER to OBD_MD_FLPARENT to more accurately describe
that this flag is only used to mark the parent FID in the OST obdo.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7ef8ee5a403a7f4d0184bb279198d93e1198e9de
Reviewed-on: https://review.whamcloud.com/33202
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11380 llite: zero lum for stripeless files 72/33172/3
John L. Hammond [Fri, 14 Sep 2018 20:49:37 +0000 (15:49 -0500)]
LU-11380 llite: zero lum for stripeless files

In the IOC_MDC_GETFILEINFO/LL_IOC_MDC_GETINFO case of ll_dir_ioctl(),
if the file has no striping then zero out the lum buffer so that
userspace won't be confused by garbage.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I4574b8bfdee122ec1ac571ae3714026c6fa0a7f0
Reviewed-on: https://review.whamcloud.com/33172
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
22 months agoLU-11381 tests: sanity-flr 201 skip information 71/33171/3
James Nunez [Fri, 14 Sep 2018 20:29:47 +0000 (14:29 -0600)]
LU-11381 tests: sanity-flr 201 skip information

sanity-flr test 201 was added to the ALWAYS_EXCEPT list
with no information about why it is being skipped. Add
the JIRA ticket number that documents why test 201
is not being run.

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I33cb0a51908bdba6a3f3faa35b1e337b4ff41764
Reviewed-on: https://review.whamcloud.com/33171
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
22 months agoLU-10899 osc: move obdo_cache to OSC code 41/33141/4
Andreas Dilger [Mon, 10 Sep 2018 23:22:46 +0000 (17:22 -0600)]
LU-10899 osc: move obdo_cache to OSC code

The obdo_cache slab is only used by the OSC code today, so it does
not need to be allocated in obdclass on servers.  Move it to only
be allocated when the OSC module is loaded.

Rename obdo_cachep to osc_obdo_kmem to match other slab caches
created by the OSC.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6f70adb1f5d0e76c72d797a65b413df1533ebbe5
Reviewed-on: https://review.whamcloud.com/33141
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11360 osd-ldiskfs: hold s_umount before calling sync_fs 35/33135/3
Li Dongyang [Tue, 11 Sep 2018 04:42:26 +0000 (14:42 +1000)]
LU-11360 osd-ldiskfs: hold s_umount before calling sync_fs

ldiskfs_sync_fs() calls dquot_writeback_dquots(), which expects
we are holding s_umount since kernel 4.10
Linux-commit: 9d1ccbe70e0b14545caad12dc73adb3605447df0

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Icda84eebe0d6261e9e374fb2a005bb66e586bba8
Reviewed-on: https://review.whamcloud.com/33135
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11111 lfsck: print correct parent FID 35/32935/2
Andreas Dilger [Fri, 3 Aug 2018 18:23:32 +0000 (12:23 -0600)]
LU-11111 lfsck: print correct parent FID

The LFSCK debug message for repairing a striped directory incorrectly
prints the child FID twice, rather than printing the parent FID:

  MDT0002-osd: namespace repair shard 0 [0x68002f9d1:0xa47c:0x0] ...
           ... of the striped directory [0x68002f9d1:0xa47c:0x0]

Fix this to print the parent FID.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I94b530ca6eaa07ee93bb8154bba8d356ce2f2896
Reviewed-on: https://review.whamcloud.com/32935
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11164 ldlm: pass env to lvbo methods 32/32832/17
Alex Zhuravlev [Thu, 19 Jul 2018 12:02:43 +0000 (16:02 +0400)]
LU-11164 ldlm: pass env to lvbo methods

to save on env allocation.

Benchmarks made by Shuichi Ihara demonstrated 13% improvement
for small I/Os: 564k vs 639k IOPS. the details can be found
LU-11164.

Change-Id: I797e3d7e19ef408993004a2b872842d655240525
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32832
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10030 clio: don't use spare bits in iattr.ia_valid 25/32825/15
James Simmons [Fri, 21 Sep 2018 02:18:09 +0000 (22:18 -0400)]
LU-10030 clio: don't use spare bits in iattr.ia_valid

Using spare bits in iattr.ia_valid is not safe as the
bits might get used for some other purpose without
us noticing.

Lustre currently used 4 spare bit, but they are all
defined in different places so this isn't immediately
obvious.

This patch brings all those bit definitions together
and defined a new op_data field (op_xvalid) to hold
the extra validity bits.

It also replaces sa_valid in struct cl_setattr_io
with sa_avalid and sa_xvalid.  Changing the name is
helpful as sa_valid already has another use within
lustre.

Modified version of:
Linux-commit: 599b9fb8d424c7907325037b5a862221b1232526

Change-Id: I475a16d557143c0741472e5873c83d71968d7cb8
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32825
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: move /proc/fs/lustre/llite/xattr_cache to sysfs 07/32507/4
James Simmons [Sat, 22 Sep 2018 20:27:04 +0000 (16:27 -0400)]
LU-8066 llite: move /proc/fs/lustre/llite/xattr_cache to sysfs

Move xattr_cache file from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 070c29ca79ef5a3ba753053928371b257064b15f

Change-Id: I0fa7715cd2ae9c90af1540e406c1645a8eee3b2e
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32507
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: move /proc/fs/lustre/llite/*_easize to sysfs 06/32506/3
James Simmons [Fri, 21 Sep 2018 21:42:36 +0000 (17:42 -0400)]
LU-8066 llite: move /proc/fs/lustre/llite/*_easize to sysfs

Move max_easize and default_easize files from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: f5501d042e5dd2b9932f1cad8ad3a71ec4777926
Linux-commit: 60b65afb700626f0fe43589138c8ca9ac460d634

Change-Id: Ie445d976ba6e21a224adc1894c108a7d30ed3a8b
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32506
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: move /proc/fs/lustre/llite/lazystatfs to sysfs 05/32505/3
James Simmons [Fri, 21 Sep 2018 15:02:18 +0000 (11:02 -0400)]
LU-8066 llite: move /proc/fs/lustre/llite/lazystatfs to sysfs

Move lazystatfs file from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: c7fe64bdd0623f5871800ff42842741175f19562

due to the large amount of changes to the OpenSFS/Intel branch.

Change-Id: I279f3361ae7b779bbd186504f5c4072b265330f9
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32505
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8174 misc: name open file handles as such 53/26953/17
Andreas Dilger [Fri, 8 May 2015 23:26:42 +0000 (17:26 -0600)]
LU-8174 misc: name open file handles as such

In a number of places in the code, rename variables from "*_handle"
or "*_fh" to "*_open_handle" so that it is clear this is referencing
an open file handle rather than something else (e.g. a lock handle).

Also rename the confusingly-named mti_close_handle to mti_open_handle,
since this is referencing an open file handle, even if it is used at
close time to close the file.

   mfd_handle2mfd() -> mfd_open_handle2mfd()
   mdt_file_data.mfd_handle -> mfd_open_handle
   mdt_file_data.mfd_old_handle -> mfd_open_handle_old
   mdt_thread_info.mti_close_handle -> mti_open_handle
   mdt_body.mbo_handle -> mbo_open_handle
   mdt_io_epoch.mio_handle -> mio_open_handle
   md_op_data.op_handle -> op_open_handle
   mdt_rec_create.cr_old_handle -> cr_open_handle_old
   mdt_reint_record.rr_handle -> rr_open_handle
   obd_client_handle.och_fh -> och_open_handle

Change the resync code path to use a "lease_handle" to avoid confusion
with an open handle:

   mdt_rec_resync.rs_handle -> rs_lease_handle
   use md_op_data.op_lease_handle
   add mdt_reint_record.rr_lease_handle

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I6e2abdaee1c32d502ea4f2f4b339a6bb21528f69
Reviewed-on: https://review.whamcloud.com/26953
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-6081 user: adding llapi_create_volatile_param() 42/13242/30
frank zago [Tue, 27 Feb 2018 16:12:16 +0000 (11:12 -0500)]
LU-6081 user: adding llapi_create_volatile_param()

This new function is similar to the existing
llapi_create_volatile_idx() function, but takes 2 additional
parameters: a stripe information, and a file mode.

llapi_create_volatile_idx() is changed to use
llapi_create_volatile_param().

Include a new man page for llapi_create_volatile_param(),
llapi_create_volatile_idx() and llapi_create_volatile_param().

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ia50750d68b8cfa335b53b114a20fccaed49c75cd
Reviewed-on: https://review.whamcloud.com/13242
Tested-by: Jenkins
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10806 target: skip discard for a missing obt_lut 40/33240/2
Alexander Boyko [Wed, 26 Sep 2018 12:45:20 +0000 (08:45 -0400)]
LU-10806 target: skip discard for a missing obt_lut

If error happens at tgt_init, obt_lut is zeroed. The error handle
path destroys export and calls tgt_grant_discard which access to
non allocated memory.

The patch skips release grant for zero obt_lut.

BUG: unable to handle kernel NULL pointer dereference at
0000000000000378
IP: [<ffffffff816b6b5c>] _raw_spin_lock+0xc/0x30
[<ffffffffc0ff7c15>] ? tgt_grant_discard+0x35/0x190 [ptlrpc]
[<ffffffffc0fcbe2e>] ? tgt_client_free+0x17e/0x3b0 [ptlrpc]
[<ffffffffc1414097>] mdt_destroy_export+0x87/0x200 [mdt]

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6260
Change-Id: Iccd80cb753122727156430e2593340c32841451c
Reviewed-on: https://review.whamcloud.com/33240
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8616 utils: document specific MDT index selection 16/33216/3
Andreas Dilger [Sat, 22 Sep 2018 04:08:39 +0000 (22:08 -0600)]
LU-8616 utils: document specific MDT index selection

Document the "lfs setdirstripe -i" ability to specify explicit MDT
index values for striped direcotories.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I23b0b5211677bc935b37ebbbfa0a2999c03ebbe5
Reviewed-on: https://review.whamcloud.com/33216
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11392 llog: fix race llog_process_thread vs llog_add 93/33193/3
Alexander Boyko [Tue, 18 Sep 2018 13:13:36 +0000 (09:13 -0400)]
LU-11392 llog: fix race llog_process_thread vs llog_add

The llog should be wrapped so it has old data on disk and zero at
bitmap.
  1. llog_process_thread reads part of llog at buffer.
  1. process a last record, checks the next record fields
  2. llog_add adds a record and marks new record at bitmap
  1. check bitmap flag and process the old record from buffer

After LU-7001 small window for the race still exists. It locates
between partial_chunk check and ext2_test_bit.
The patch fixes this race.

Cray-bug-id: LUS-6287
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I45a742af0c987aa159a22a3255a4fb9d430b6a84
Reviewed-on: https://review.whamcloud.com/33193
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10177 lfs: support DOM-to-OST migration 79/32979/8
Mikhail Pershin [Fri, 10 Aug 2018 16:20:52 +0000 (19:20 +0300)]
LU-10177 lfs: support DOM-to-OST migration

Migraton of DoM file to an OST file is possible with
'lfs migrate' command in general but would cause space loss
on MDS, because DOM local inode blocks are not deleted.
Patch doesn the following:
- adds truncate for local MDS inode after migration.
- check and prevent migration from non-DOM to DOM file,
  that will be done later with mirroring.
- prohibit 'lfs swap_layouts' command if either file has
  DOM component, 'lfs migrate' should be used instead

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I1ce7b09eefab5e13f81af30c007475c927bd2874
Reviewed-on: https://review.whamcloud.com/32979
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11128 ptlrpc: new request vs disconnect race 80/32980/35
Alex Zhuravlev [Fri, 10 Aug 2018 17:03:11 +0000 (21:03 +0400)]
LU-11128 ptlrpc: new request vs disconnect race

new request can race with disconnect-by-idle process.
disconnect code detect this state and initiate a new connection.

Change-Id: I6acc913c371f2ae63f27151edce457c2a13118c5
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32980
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11373 libcfs: fix wrong check in libcfs_debug_vmsg2() 54/33154/5
Wang Shilong [Tue, 18 Sep 2018 04:45:49 +0000 (12:45 +0800)]
LU-11373 libcfs: fix wrong check in libcfs_debug_vmsg2()

Logic here is we skip output if time is before @cdls_next
reach and increase @cdls_count. however we did it in the
opposite way:

1)libcfs_debug_vmsg2() is called for a long time, that means
current check succeed, we skip print messages and return, we
will skip all messages later too..

2)libcfs_debug_vmsg2() is called frequently, current check
fail every time, message will be bumped out always. the
worst case is we never skip any messages.

Also fix test case to cover this later which is from Andreas:

The test_60a() llog test is being run on the MGS, while the
check in test_60b() to confirm that CDEBUG_LIMIT() works properly
is being run on the client.  There has been a breakage in
CDEBUG_LIMIT() that this test failed to catch, now we need to
track it down.

Change test_60b to dump the dmesg logs on the MGS.

Fixes: fdb5d3d50
Test-parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Idb2ecfe5ddadc72d8931e6623b171b9c773ebbe5
Reviewed-on: https://review.whamcloud.com/33154
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
22 months agoLU-11071 build: create lustre resource package for Ubuntu 52/33152/4
James Simmons [Thu, 13 Sep 2018 22:55:07 +0000 (18:55 -0400)]
LU-11071 build: create lustre resource package for Ubuntu

The new Ubuntu server support is missing on last package that RHEL
currently supports. Enable Debian/Ubuntu packaging for the resource
agent package.

Test-Parameters: trivial

Change-Id: I872ae6b1b02782155db14c5cc971607ca3e93d56
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33152
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11268 mdc: use smaller buffers for getxattr 24/33024/4
John L. Hammond [Tue, 18 Sep 2018 16:45:16 +0000 (12:45 -0400)]
LU-11268 mdc: use smaller buffers for getxattr

In mdc_intent_getxattr_pack() only the xattr values buffer should be
upsized for older MDTs. This refines the interop fix from change
https://review.whamcloud.com/31990 (LU-10912 mdc: use large xattr
buffers for old servers).

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: If7c6fe7be96a6a227c746f90757b7aae1874b14e
Reviewed-on: https://review.whamcloud.com/33024
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: remove redundant lookup in dump_pgcache 10/33010/5
NeilBrown [Sun, 16 Sep 2018 21:28:41 +0000 (17:28 -0400)]
LU-8066 llite: remove redundant lookup in dump_pgcache

Both the 'next' and the 'show' functions for the dump_page_cache
seqfile perform a lookup based on the current file index.  This is
needless duplication.

The reason appears to be that the state that needs to be communicated
from "next" to "show" is two pointers, but seq_file only provides for
a single pointer to be returned from next and passed to show.

So make use of the new 'seq_private' structure to store the extra
pointer.
So when 'next' (or 'start') find something, it returns the page and
stores the clob in the private area.
'show' accepts the page as an argument, and finds the clob where it
was stored.

Linux-commit: 85b6542bbade42935386661cb45d90c4e13766b7

Test-Parameters: trivial envdefinitions="ONLY=63b" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=118" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=20" testlist=sanityn

Change-Id: I6a45430f7ba3007dcd70bdf09c4c7ed5a213533d
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33010
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11237 build: delete obsolete slapd-lustre.conf 89/32989/2
Andreas Dilger [Mon, 13 Aug 2018 17:25:58 +0000 (11:25 -0600)]
LU-11237 build: delete obsolete slapd-lustre.conf

Delete the obsolete slapd-lustre.conf file, as it was only used
for LDAP configuration of clients and servers back before
"zero-conf" (MGS+llog) configuration was available.  Also remove
the use of openldap in Makefile.am.

This has been obsolete since Lustre 1.4 or so.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If99ca0a62a1753ead1b29d217bc2437bd29e0850
Reviewed-on: https://review.whamcloud.com/32989
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11169 obdclass: fix old return code usage 06/32906/8
Alexey Lyashkov [Sat, 15 Sep 2018 07:33:51 +0000 (10:33 +0300)]
LU-11169 obdclass: fix old return code usage

Commit 49d8a7ccd73 from LU-9855 introduce a fail in the
obd_commitrw function which parent error code
replaced with result of exp_check_ops function.

Cray-bugid: LUS-6195

Test-Parameters: trivial
Change-Id: If8643ba39232f94ba9c523fdb9f10c1411a1838a
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/32906
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11020 osd: use right sync 66/32866/6
Alexey Lyashkov [Wed, 1 Aug 2018 15:52:28 +0000 (18:52 +0300)]
LU-11020 osd: use right sync

use a sync_fs instead of journal flush only.
ext4 commit with similar change

commit 06a407f13daf9e48f0ef7189c7e54082b53940c7
Author: Dmitry Monakhov <dmonakhov@openvz.org>
Date:   Wed Jun 12 22:25:07 2013 -0400

ext4: fix data integrity for ext4_sync_fs

Inode's data or non journaled quota may be written w/o jounral so we
 _must_ send a barrier at the end of ext4_sync_fs. But it can be
skipped if journal commit will do it for us.

Cray-bug-id: LUS-5993
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: Ie242b48d0b8ad61cf4d38ac0fb173176193716d9
Reviewed-on: https://review.whamcloud.com/32866
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 ofd: migrate from proc to sysfs 59/33059/4
James Simmons [Sat, 8 Sep 2018 17:11:44 +0000 (13:11 -0400)]
LU-8066 ofd: migrate from proc to sysfs

Move the ofd module from using proc for most single value files
to sysfs. Leave the more complex proc entries for now. Migrate
the sysfs and most of the proc handling to lproc_ofd.c.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ief21636a0dce3fd4946f9a6971a8b09c2812200e
Reviewed-on: https://review.whamcloud.com/33059
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-6864 mdc: move RPC semaphore code to lustre/osp 12/32412/3
Andreas Dilger [Tue, 15 May 2018 21:42:27 +0000 (15:42 -0600)]
LU-6864 mdc: move RPC semaphore code to lustre/osp

The "MDC RPC semaphore" is no longer used by MDC code since patch
http://review.whamcloud.com/14374 "LU-5319 mdc: manage number of
modify RPCs in flight" landed.  It is only still used by the OSP
currently.  While there are plans to remove this from the OSP as
well, it makes sense to move all of this code from MDC to OSP so
that it will also be cleaned up when that functionality lands.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1c4c9a45f308a29600631ac7fed40e592b3ebbe5
Reviewed-on: https://review.whamcloud.com/32412
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8130 ptlrpc: convert conn_hash to rhashtable 36/32036/13
NeilBrown [Tue, 28 Aug 2018 21:05:42 +0000 (17:05 -0400)]
LU-8130 ptlrpc: convert conn_hash to rhashtable

Linux has a resizeable hashtable implementation in lib,
so we should use that instead of having one in libcfs.

This patch converts the ptlrpc conn_hash to use rhashtable.
In the process we gain lockless lookup.

As connections are never deleted until the hash table is destroyed,
there is no need to count the reference in the hash table.  There
is also no need to enable automatic_shrinking.

Linux-commit: ac2370ac2bc5215daf78546cd8d925510065bb7f

Change-Id: I576daf314c3ac31a58df02d731292e1e8bb408c6
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32036
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-4684 lmv: support accessing migrating directory 04/31504/31
Lai Siyao [Thu, 1 Mar 2018 02:34:19 +0000 (10:34 +0800)]
LU-4684 lmv: support accessing migrating directory

Migrating directory contains stripes of both old and new layout, and
its sub files may be located on either one. To avoid race between
access and new creations, there are 4 rules to access migrating
directory:
1. always create new file under new layout.
2. any operation that tries to create new file under old layout will
   be rejected, e.g., 'mv a <migrating_dir>/b', if b exists and is
   under old layout, this rename should fail with -EBUSY.
3. operations that access file by name should try old layout first,
   if file doesn't exist, then it will retry new layout, such
   operations include: lookup, getattr_name, unlink, open-by-name,
   link, rename.
4. according to rule 1, open(O_CREAT | O_EXCL) and create() will
   create new file under new layout, but they should check existing
   file in one transaction, however this can't be done for old
   layout, so check existing file under old layout on client side,
   then issue the open/create request to new layout.

Disable sanity 230d for ZFS backend because it will trigger lots of
sync, which may cause system hung.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Icd587eb17f4c6dc2234dcc3c7ba40759bc3e8d31
Reviewed-on: https://review.whamcloud.com/31504
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-5026 obdclass: Add lbug_on_eviction option 57/10257/13
Ryan Haasken [Wed, 7 May 2014 21:33:24 +0000 (16:33 -0500)]
LU-5026 obdclass: Add lbug_on_eviction option

Add an lbug_on_eviction sysfs interface.  When it is set to a non-zero
value on a client, it will cause the client to LBUG whenever it is
evicted by the server. Note, an MDS is a client to OSTs, and every
server is a client of MGS. Thus, it is probably desireable to leave
this set to zero on servers.

Change-Id: I2f694e4bc1b1ec9daf52f8a6ac46ad87f40f810f
Cray-bug-id: LUS-2591
Signed-off-by: Ryan Haasken <haasken@cray.com>
Signed-off-by: Chris Horn <hornc@cray.com>
Reviewed-on: https://review.whamcloud.com/10257
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11329 utils: add get_maintainer.pl script 04/33104/3
Andreas Dilger [Mon, 3 Sep 2018 14:30:32 +0000 (22:30 +0800)]
LU-11329 utils: add get_maintainer.pl script

The get_maintainer.pl script can be given a patch file or list of
pathnames and will look at the MAINTAINERS file for different code
subsections and the developers that are maintaining those sections.
If no maintainer is found, it will also check the git commit history
and "git blame" to see which developers were active on the specified
file(s) and use them.

This patch adds a copy of the get_maintainers.pl and MAINTAINERS files
to the Lustre tree to make it easier to find reviewers for the code.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5f2e717227a7c86c924e7edb889e087c7c3ebbe5
Reviewed-on: https://review.whamcloud.com/33104
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11128 ptlrpc: add debugging for idle connections 68/33168/3
Andreas Dilger [Fri, 14 Sep 2018 03:44:23 +0000 (21:44 -0600)]
LU-11128 ptlrpc: add debugging for idle connections

Add a "debug" parameter for the idle client disconnection so that
it can log disconnect/reconnect events to the console.

Print the idle time in the "import" file.

Enable the connection debugging for all test runs.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I47a4f73542fb4864e4f35275ecfe8907493ebbe5
Reviewed-on: https://review.whamcloud.com/33168
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11301 target: add lock in sub_trans_stop_cb() 69/33169/2
Lai Siyao [Mon, 27 Aug 2018 20:55:56 +0000 (04:55 +0800)]
LU-11301 target: add lock in sub_trans_stop_cb()

sub_trans st_committed and st_stopped flags take different bit of
the same memory address, so both of them should be set with lock,
but sub_trans_stop_cb() doesn't, which may cause it overriden.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ic3d7d7281b3cf9bd20702be944e14f35200318f1
Reviewed-on: https://review.whamcloud.com/33169
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11372 o2iblnd: add clarifying comment to ko2iblnd.conf 63/33163/2
Sonia Sharma [Fri, 31 Aug 2018 14:24:42 +0000 (10:24 -0400)]
LU-11372 o2iblnd: add clarifying comment to ko2iblnd.conf

The settings in ko2iblnd.conf are recommended only for
OPA but if other IB cards are present along with
OPA, then these settings are applied across all the
configured IB interfaces.

Change-Id: I6daf1f59a100db4e9c2cd12c255ad33fda274093
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33163
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11071 ldiskfs: fix ldiskfs patch for ubuntu18 44/33144/2
Li Dongyang [Wed, 12 Sep 2018 05:50:58 +0000 (15:50 +1000)]
LU-11071 ldiskfs: fix ldiskfs patch for ubuntu18

the original ext4-corrupted-inode-block-bitmaps-handling-patches.patch
missed the changes to ext4_validate_inode_bitmap, add it back

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I42ff97be5c5680be04d69e0e818b622df6bd400d
Reviewed-on: https://review.whamcloud.com/33144
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10899 obdclass: remove unused ll_import_cachep 19/33119/2
Andreas Dilger [Thu, 6 Sep 2018 10:24:30 +0000 (18:24 +0800)]
LU-10899 obdclass: remove unused ll_import_cachep

The ll_import_cache is not used anywhere, and can be removed.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3f43c81ffbf1385117f9ba1f8f9881388a3ebbe5
Reviewed-on: https://review.whamcloud.com/33119
Tested-by: Jenkins
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11303 out: clean up osp_update_rpc_pack() macro 07/33107/3
Andreas Dilger [Tue, 4 Sep 2018 09:50:48 +0000 (17:50 +0800)]
LU-11303 out: clean up osp_update_rpc_pack() macro

The osp_update_rpc_pack() macro was using both an operation "name"
argument as well as an "opcode" argument, but totally ignoring the
"opcode" argument.  This allowed a caller in osp_attr_set() to pass
a non-existent opcode name to the function that was ignored.

The function also concatenated the passed name string to generate
a function name to be called (which supplied its own opcode value).
This name concatenation obscures the code functionality, since the
called functions are not actually referenced directly in the code.

Instead, pass the function name as the argument, and drop the opcode
completely.  This at least makes it possible to see where the function
is being called.  Change the macro to be upper-case, so that it is
more clear that it is a macro rather than a real function.  It would
be better to replace the macro with an actual function, but this is
not very practical due to the use of variable-length argument lists.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7bf5c1d0f7a9abab1d8de8046cc408961f3ebbe5
Reviewed-on: https://review.whamcloud.com/33107
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: use more private data in dump_pgcache 09/33009/14
NeilBrown [Tue, 11 Sep 2018 13:41:52 +0000 (09:41 -0400)]
LU-8066 llite: use more private data in dump_pgcache

The dump_page_cache debugfs file allocates and frees an 'env' in each
call to vvp_pgcache_start,next,show.  This is likely to be fast, but
does introduce the need to check for errors.

It is reasonable to allocate a single 'env' when the file is opened,
and use that throughout.

So create 'seq_private' structure which stores the sbi, env, and
refcheck, and attach this to the seqfile.

Then use it throughout instead of allocating 'env' repeatedly.

Linux-commit: fc5f499af16c247c5bff68d2625a6d1ca48b3332

Test-Parameters: trivial envdefinitions="ONLY=63b" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=118" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=20" testlist=sanityn

Change-Id: Id7e7f5ec1729d1e5af041adcb247ca11299ed734
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33009
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-11133 dkms: Add missing ldiskfs module in dkms config 00/32800/4
Sebastien Piechurski [Mon, 20 Aug 2018 09:06:07 +0000 (11:06 +0200)]
LU-11133 dkms: Add missing ldiskfs module in dkms config

The dkms.conf file generated by the script dkms.mkconf does not
include the ldiskfs kernel module. This results in not being able to
use ldiskfs when using the lustre-ldiskfs-dkms or lustre-all-dkms
packages.
This patch adds the module in the configuration for both of these
packages.

Test-Parameters: trivial
Signed-off-by: Sebastien Piechurski <sebastien.piechurski@atos.net>
Change-Id: Icc98fba208c78c3061fe591e3315af1eb9199f4f
Reviewed-on: https://review.whamcloud.com/32800
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: move /proc/fs/lustre/llite/statahead_{max, agl} to sysfs 04/32504/3
James Simmons [Fri, 10 Aug 2018 01:49:28 +0000 (21:49 -0400)]
LU-8066 llite: move /proc/fs/lustre/llite/statahead_{max, agl} to sysfs

Move statahead_max and statahead_agl files from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 4081a3201ade8b3be715ebb0b7879a2bfe5fcc98

due to the large amount of changes to the OpenSFS/Intel branch.

Change-Id: Ic16d48d0cf1baabb462952de9739544923eba8cd
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32504
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-8066 llite: move /proc/fs/lustre/llite/stats_track* to 03/32503/7
James Simmons [Mon, 10 Sep 2018 17:02:17 +0000 (13:02 -0400)]
LU-8066 llite: move /proc/fs/lustre/llite/stats_track* to
 sysfs

Move stats_track_pid, stats_track_ppid and stats_track_gid files
from /proc/fs/lustre/llite/* to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 62cf4b1315e86a1f373f77acf32a23d76428ba27

due to the large amount of changes to the OpenSFS/Intel branch.

Test-Parameters: trivial

Change-Id: I29dc2dd269f66e40a7d016ba366353ae2ca7fb51
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32503
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-10124 lnet: Correctly add peer MR value while importing 38/31138/8
Sonia Sharma [Thu, 1 Feb 2018 23:40:03 +0000 (15:40 -0800)]
LU-10124 lnet: Correctly add peer MR value while importing

while adding peer using lnetctl import, the MR value of the
peer is not correctly imported.

Checks for MR value other than True/False in
handle_yaml_config_peer() -
1. No value provided - Use default as True
2. Value other than True/False - Error out

Change-Id: I02a21e35086f1c6f29081b464dd1a63aba692cbc
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/31138
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-4684 migrate: shrink dir layout after migration 26/31626/22
Lai Siyao [Thu, 8 Mar 2018 03:24:31 +0000 (11:24 +0800)]
LU-4684 migrate: shrink dir layout after migration

Use setxattr(XATTR_NAME_LMV) to shrink dir layout after migration,
this may change dir fid because it may shrink dir to 1-stripe, if so
replace this dir with its stripe.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I4fd3e83e188ac0c6c4845183c17701276fae94a8
Reviewed-on: https://review.whamcloud.com/31626
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
22 months agoLU-4684 migrate: migrate striped directory 27/31427/26
Lai Siyao [Mon, 22 Jan 2018 18:34:54 +0000 (02:34 +0800)]
LU-4684 migrate: migrate striped directory

Migrate striped directory in below steps:
1. create target object if needed: if source is directory, a
   target object is always created, otherwise if source is
   already located on the target MDT, or source still has
   link on source MDT, then skip creating.
a) if source is directory, detach source stripes and
   attach them to target.
b) migrate source xattrs to target.
c) if source is regular file, update PFID to target
   fid.
d) update fid to target for all links of source
2. update namespace
a) migrate dirent from source parent to target parent.
b) update linkea parent fid to target parent.
c) destroy source object.

This implementation improves following fields:
1. all involved objects are locked to avoid race.
2. directory migration doesn't migrate its dir entries, instead
   it's done in each sub file migration, this avoids timeout in
   migrating dir entries for large directory, and also avoids
   touching dir entries without lock.
3. file/dir is migrated in one transaction, so migrate recovery
   is the same as others.
4. migrating directory can be accessed (modifiable) like normal
   directory.
5. if migration of sub files under a directory fails, user can
   redo migrate to finish migration of this directory.

Disable migrate in racer.sh.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I7906e50a0bf55375eafdf2cf5500979dd2d94d48
Reviewed-on: https://review.whamcloud.com/31427
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>