Whamcloud - gitweb
fs/lustre-release.git
3 years agoRevert "LU-9019 osc: migrate to time64_t" 71/30571/2
Oleg Drokin [Mon, 18 Dec 2017 15:40:38 +0000 (15:40 +0000)]
Revert "LU-9019 osc: migrate to time64_t"

This causes frequent failures tracked in LU-10403

This reverts commit 4f2a5d5887492da9abe320074511811415e0a06c.

Change-Id: I7d255fcee654508b6df233728624a39917853b98
Reviewed-on: https://review.whamcloud.com/30571
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10232 lov: call cl_object_attr_get under cl_attr lock 52/30052/2
Mikhail Pershin [Sun, 12 Nov 2017 21:20:26 +0000 (00:20 +0300)]
LU-10232 lov: call cl_object_attr_get under cl_attr lock

cl_object_attr_get() must be called under cl_object_attr_lock
get. There is place in lov_getstripe where it is called
without that lock.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ia0a2322ba4ff0ff4affb081375cb108fbf2988c4
Reviewed-on: https://review.whamcloud.com/30052
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10005 osp: cache non-exist EA 78/29078/9
Di Wang [Fri, 27 Oct 2017 15:08:11 +0000 (23:08 +0800)]
LU-10005 osp: cache non-exist EA

OSP should also cache non-exist EA, otherwise
it will keep sending RPC to try to get remote
non-exist EA.

So if default stripe EA does not exist on root
MDT, file creation on non-root MDT will always
try to send extra OSP RPC to get the EA.

This patch also fixes a LFSCK bug that may cause
the LFSCK repaired items to be counted repeately.

Test-Parameters: mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs mdscount=2 mdtcount=4 testlist=sanity-lfsck,sanity-lfsck,sanity-lfsck
Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: Ie9c8cb5fd54e404f1af97de47e809f6f96de8d86
Reviewed-on: https://review.whamcloud.com/29078
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
3 years agoLU-10331 out: use OBD_ALLOC_LARGE() for update buffers 55/30455/2
John L. Hammond [Fri, 8 Dec 2017 19:10:21 +0000 (13:10 -0600)]
LU-10331 out: use OBD_ALLOC_LARGE() for update buffers

In out_handle() the update buffers may be up to 100KB so use
OBD_ALLOC_LARGE() to avoid high order page allocation errors.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie9970eb219ff0e5357132e0858cb16db8f703209
Reviewed-on: https://review.whamcloud.com/30455
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
3 years agoLU-10038 test: ignore readdir races in sanity 133g 51/30451/3
John L. Hammond [Fri, 8 Dec 2017 16:19:48 +0000 (10:19 -0600)]
LU-10038 test: ignore readdir races in sanity 133g

In sanity test_133g() writing to some files (for example
mdt/*/exports/clear) may cause cause the proc tree to change while
find is running. Avoid errors in these cases by adding the
-ignore_readdir_race option to the find invocations.

Test-Parameters: trivial testlist=sanity

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3ce61ee0f4e1041041d6872a0fe03488a9df363c
Reviewed-on: https://review.whamcloud.com/30451
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
3 years agoLU-10346 utils: use the correct lib path in mount_utils.c 31/30431/3
Quentin Bouget [Thu, 7 Dec 2017 15:48:28 +0000 (15:48 +0000)]
LU-10346 utils: use the correct lib path in mount_utils.c

The backfs_load_module() function of mount_utils.c used an outdated
to link itself against the osd library. This patch fixes it.

Test-Parameters: trivial
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I90a523b45b5772c05f7520e810eec7eac14b75be
Reviewed-on: https://review.whamcloud.com/30431
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10338 test: handle empty CLIENTS in mountcli() 13/30413/2
John L. Hammond [Wed, 6 Dec 2017 21:27:22 +0000 (15:27 -0600)]
LU-10338 test: handle empty CLIENTS in mountcli()

In mountcli() use if [ -n "$CLIENTS" ]; ... rather than
[ -n "$CLIENTS" ] && ... to avoid immediate exit on error when CLIENTS is
the empty string.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I440c3eefb4bee134caee1db5ff2ecdbdf9c1ee5c
Reviewed-on: https://review.whamcloud.com/30413
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10330 tests: fix version check in sanity.sh test_101g 89/30389/2
Emoly Liu [Wed, 6 Dec 2017 07:46:11 +0000 (15:46 +0800)]
LU-10330 tests: fix version check in sanity.sh test_101g

The 16MB RPC patch is available in 2.7.x, so one more version
check needs to be added to 2.7.17 or later to avoid interop
failure.

Test-Parameters: trivial

Change-Id: I6eeefbf74017aaeeb8998accd6da04d8de75606c
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/30389
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9535 tests: check server version in sanityn 77j 85/30385/3
James Nunez [Tue, 5 Dec 2017 23:31:21 +0000 (16:31 -0700)]
LU-9535 tests: check server version in sanityn 77j

The QoS feature in the TBF policy was added to Lustre with
tag 2.9.53 and a patch with commit hash d2c403363f6. Thus,
sanityn test 77j should only be run for servers with version
2.9.53 and later.

Test-Parameters: trivial mdsjob=lustre-b2_9 ossjob=lustre-b2_9 serverbuildno=2 testlist=sanityn
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ia2a04b5ae688809d4a84a2dc8459598f02932119
Reviewed-on: https://review.whamcloud.com/30385
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10267 llog: fix EOF handling in llog_client_next_block() 13/30313/2
John L. Hammond [Wed, 29 Nov 2017 14:45:47 +0000 (08:45 -0600)]
LU-10267 llog: fix EOF handling in llog_client_next_block()

In llog_client_next_block() update *cur_idx and *cur_offset in the
special case that the handler has returned -EIO after reaching the end
of the log without finding the desired record. This fixes client side
EOF detection in llog_process_thread().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If30ebec065ddd38fb7b06c3e16f96bb3cd76fa1b
Reviewed-on: https://review.whamcloud.com/30313
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10248 mdd: set PFID for swap and merge layout 92/30292/5
Jinshan Xiong [Thu, 16 Nov 2017 22:05:06 +0000 (22:05 +0000)]
LU-10248 mdd: set PFID for swap and merge layout

It should update PFID for the OST objects if the layout has been
moved to another file. So far only swap and merge layout would
have this requirement.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ic2fd7d7a88cd0ef4501d2f02baf58dd177e07973
Reviewed-on: https://review.whamcloud.com/30292
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8999 test: check chown result 43/30243/4
Hongchao Zhang [Fri, 17 Nov 2017 15:05:17 +0000 (23:05 +0800)]
LU-8999 test: check chown result

For test_38 in sanity-quota, the chown operation should be
checked whether it succeeds or not. the patch also collects
the output of quota users if the test failed.

Change-Id: I6c04f9519f3f097af7126064380faf1bdc4fff6a
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/30243
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10257 clio: remove unused cl_lock layers 92/30192/2
John L. Hammond [Fri, 17 Nov 2017 21:20:02 +0000 (15:20 -0600)]
LU-10257 clio: remove unused cl_lock layers

Remove the struct vvp_lock and struct lovsub_lock, omitting these
layers from cl_lock. Adjust cl_lock_enqueue() to allow for empty locks
(corresponding to unstriped files).

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib1478d0cfb9604540ffc38eb9b01da4f4f96212a
Reviewed-on: https://review.whamcloud.com/30192
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9474 tests: rewrite copytool_setup to use stack_trap 97/30097/7
Quentin Bouget [Tue, 14 Nov 2017 14:20:27 +0000 (14:20 +0000)]
LU-9474 tests: rewrite copytool_setup to use stack_trap

Replace copytool_setup with its rewrite (copytool setup) as much
as possible. There are some interaction between copytool_setup()
and other test functions (mainly import_file() and its derivatives).
Later patch(es) will handle those interactions and completely
remove copytool_setup() from sanity-hsm.

Among the benefits of using the new copytool() function:
 - it has a nice command line interface
 - it uses stack_trap() to register clean up actions
 - it makes it easier to run sanity-hsm with other copytools

Test-Parameters: trivial clientcount=3 mdscount=2 testlist=sanity-hsm,sanity-hsm
Change-Id: I81635da39f19d001e0f5c6348b233b45c4298fd0
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-on: https://review.whamcloud.com/30097
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10227 ptlrpc: simplify struct ptlrpc_request_set 60/30060/3
John L. Hammond [Fri, 10 Nov 2017 20:03:30 +0000 (14:03 -0600)]
LU-10227 ptlrpc: simplify struct ptlrpc_request_set

Remove obd_statfs_rqset(), replacing its only use with
obd_statfs(). Collapse lov_statfs_async() and lov_statfs() into a
single function, removing the need for lov_statfs_interpret().

Remove the then unused set_wakeup_ptr, set_cblist, set_interpret, and
set_arg members of struct ptlrpc_request_set. Remove struct
ptlrpc_set_cbdata and ptlrpc_set_add_cb(). On x86_64 this reduces the
size of struct ptlrpc_request_set from 152 bytes to 112.

Add sanity test_118n() to ensure that OST_STATFS requests are still
async.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iec8aa378157367f03de96f82d67158b281ec374c
Reviewed-on: https://review.whamcloud.com/30060
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9774 mdt: mdt_dump_lmm(), don't panic 88/29188/8
Alex Zhuravlev [Sat, 25 Nov 2017 16:26:32 +0000 (11:26 -0500)]
LU-9774 mdt: mdt_dump_lmm(), don't panic

skip compond layouts as mdt_dump_lmm() doesn't understand that yet.

Change-Id: I48d447f81d58466f473de871bc59d011d8b6f7ba
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/29188
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-10190 osd-zfs: create agent object for remote object 55/28855/25
Fan Yong [Tue, 5 Dec 2017 14:28:10 +0000 (22:28 +0800)]
LU-10190 osd-zfs: create agent object for remote object

In DNE environment, the object and its name entry may reside
on different MDTs. Under such case, we will create an agent
object on the MDT where the name entry resides. The agent
object is empty to indicates that the real object for this
name entry resides on another MDT. If without agent object,
related name entry will be skipped when perform MDT side file
level backup/restore via ZPL by userspace tool, such as 'tar'.

Test-Parameters: envdefinitions=ONLY=803 testlist=sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4
Test-Parameters: envdefinitions=ONLY=34 testlist=sanity-lfsck mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I7fb8c54ca774f3877bcd326bc10d8e99083ac90a
Reviewed-on: https://review.whamcloud.com/28855
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10189 osd: handle PFID EA in LMA properly 96/29696/13
Fan Yong [Wed, 29 Nov 2017 04:25:51 +0000 (12:25 +0800)]
LU-10189 osd: handle PFID EA in LMA properly

Originally, the issue was caused by old ldiskfs OST device
with 256-bytes sized inode. Because the inode inline space
was very limited, we have to store the PFID EA inside LMA
EA for stripe and PFL component information.

When we restore the OST from such old OST via server side
file level backup, then such composite LMA will be on the
new OST even if the new OST inode has enough inline space
to hold separated PFID EA.

In futher, if we migrate the old OST from ldiskfs to ZFS,
then such composite LMA will also be on the ZFS based OST
although the PFID EA can be stroed independently on ZFS.

So the OSD logic, in spite of ldiskfs or ZFS, needs to
understand the composite LMA EA, and handle it properly.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2b66787e725e13da7984f1bc2df45760dfbe4c4d
Reviewed-on: https://review.whamcloud.com/29696
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10187 osd-zfs: repair FID-in-dirent for ZFS backend 09/28609/23
Fan Yong [Tue, 5 Dec 2017 02:12:40 +0000 (10:12 +0800)]
LU-10187 osd-zfs: repair FID-in-dirent for ZFS backend

If the ZFS backend is restored from MDT file-level backup, then
the FID-in-dirent will be lost. The patch can regenerate/repair
the lost/corrupted FID-in-dirent during namespace LFSCK.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I747cdb75adc93971a7450f9e6b5e99598c79f656
Reviewed-on: https://review.whamcloud.com/28609
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10186 osd-zfs: move LAST_ID OI mapping out of oi.xx 79/28679/14
Fan Yong [Wed, 29 Nov 2017 04:24:47 +0000 (12:24 +0800)]
LU-10186 osd-zfs: move LAST_ID OI mapping out of oi.xx

Originally, the OI mapping for LAST_ID was put under the
oi.xx file, it may cause some issues:

1) If we remove the oi.xx file, then before the OI scrub
   scanned the whole device, we can not know the last ID
   for related sequence. That will fail the system start
   as to the OI scrub will be stopped by force.

2) It is incompatible with ldiskfs backend, that will cause
   trouble after migrating from ldiskfs-based device.

This patch moves the LAST_ID OI mapping from oi.xx file to
/O/<seq>/LAST_ID as ldiskfs backend does.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ibfef1d9ef5be72f6e04ff79f92302483f7fe8a15
Reviewed-on: https://review.whamcloud.com/28679
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
3 years agoLU-10188 osd-zfs: handle non 4K aligned block size 41/29241/17
Fan Yong [Wed, 6 Dec 2017 13:45:57 +0000 (21:45 +0800)]
LU-10188 osd-zfs: handle non 4K aligned block size

After restored from server side file level backup, the files
created via ZPL may use non 4K-bytes aligned block size. When
the device is mounted as Lustre, the zfs-osd will adjust the
client visible OST-objects' block size as at least 4K-bytes
aligned. For the objects that cannot be reset the block size,
the OSD logic needs to handle the non aligned case properly.
Otherwise, Lustre I/O handler may cause osd-zfs or ZFS panic
when osd_bufs_get_read().

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic07aeec3fc774508cedd6d24cca54b76171d143b
Reviewed-on: https://review.whamcloud.com/29241
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8602 libcfs: call proper crypto algo when keys are passed in 99/25199/26
James Simmons [Thu, 30 Nov 2017 16:06:42 +0000 (11:06 -0500)]
LU-8602 libcfs: call proper crypto algo when keys are passed in

In most cases keys are not passed to cfs_crypto_hash_alloc()
but if they are then crypto_ahash_setkey() will fail. Keys
are only handled by the hmac version of the algorithm requested.
If a key is passed into cfs_crypto_hash_alloc() then we should
request the hmac version of the algorithm when calling
crypto_alloc_ahash().

Change-Id: I080d89bc864b236524ef11f50df41b750ecab9fe
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25199
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5020 llite: don't reconnect MGC if connecting 29/10229/16
Hongchao Zhang [Fri, 13 Oct 2017 22:57:28 +0000 (06:57 +0800)]
LU-5020 llite: don't reconnect MGC if connecting

1. In mgc_set_info_async(KEY=KEY_INIT_RECOV_BACKUP "init_recov_bk"),
   the MGC import should be reconnected only if its state is
   LUSTRE_IMP_DISCON

2. in mgc_target_register, if the target will regenerate the config,
   we should use some longer delay limit to wait the MGC to connect
   to MGS for the target (server) will fail to exit if the request
   expired due to delay limit.

3. In case of parallel mount, the async cleanup of OSS will affects
   the following mount for the OSS can't be setup again, then there
   should be some barrier to sync with the OSS cleanup.

Change-Id: I805b84cf12100ec2cc68f95bb65a9c396e0fbc1b
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/10229
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-618 llite: IO accounting of page read 36/1636/11
Hongchao Zhang [Thu, 30 Nov 2017 10:10:52 +0000 (18:10 +0800)]
LU-618 llite: IO accounting of page read

When CONFIG_TASK_IO_ACCOUNTING is used with Lustre, writes are
accounted for but not read.

The accounting is normally done in the kernel for page writeback
and readahead functionlity, Therefore, as Lustre implements its
own readahead, it must also maintain its own accounting on read
(but not for write)

Change-Id: I19f330be65324a8da002f9d61cb9262345ecb012
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/1636
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-10268 lfsck: postpone lfsck start until initialized 59/30259/5 59/30259/6
Fan Yong [Thu, 7 Dec 2017 07:29:35 +0000 (15:29 +0800)]
LU-10268 lfsck: postpone lfsck start until initialized

Sometimes, the LFSCK start request may comes (from remote server)
before local target initialized. If we start the LFSCK right away
on current server, the LFSCK engine may access NULL pointer, such
as lookup FID with NULL 'ss_server_fld'.

To avoid such trouble, start LFSCK logic will return -EINPROGRESS
to the request sponsor. It is the sponsor duty to retry the start
request some time later.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If7bc44e025b5f3c4f977b3a35e3784ada548a2df
Reviewed-on: https://review.whamcloud.com/30259
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10269 ldlm: fix the issues introduced by try bits 46/30246/11
Jinshan Xiong [Mon, 13 Nov 2017 05:36:33 +0000 (05:36 +0000)]
LU-10269 ldlm: fix the issues introduced by try bits

Try to grant optinal try_bits if lock is not blocked
and to be granted immediately otherwise trybits are cleared,
therefore there are no locks with trybits in both granted and
waiting queue.

Also mdt_object_lock_try() is changed to don't return negative for
try lock only case.

Test-Parameters: mdscount=1 mdtcount=1 testlist=racer,racer,racer
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: If3f5734355511ad1af71d5996026d04c60f3e8c1
Reviewed-on: https://review.whamcloud.com/30246
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9019 osc: migrate to time64_t 63/30063/3
James Simmons [Tue, 21 Nov 2017 03:39:40 +0000 (22:39 -0500)]
LU-9019 osc: migrate to time64_t

Change od_contention_time from int to time64_t to make it clear
this field is in units of seconds. With this change move the other
*_contention_time fields from jiffies to time64_t to avoid the
overhead of switching between the two different time formats.
Change ops_submit_time also to time64_t since using jiffies
doesn't gain us anything and having it in time64_t makes it clear
this is code related to time.

Change-Id: I97d3685ac61781e6b1dc8634bc105bb0ffe76aa0
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30063
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8626 hsm: count the number of started requests of each type 77/28677/9
Quentin Bouget [Thu, 24 Aug 2017 11:16:04 +0000 (13:16 +0200)]
LU-8626 hsm: count the number of started requests of each type

Add counters in the coordinator to keep track of how many started hsm
requests of each type (ARCHIVE, RESTORE, REMOVE) the coordinator is
managing. There is no counter for CANCEL requests as the coordinator
does not keep track of them like it does for the other requests.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I0c1e9881f2f6a4f005dfc3545b1b51714eb91b7b
Reviewed-on: https://review.whamcloud.com/28677
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5216 hsm: cancel hsm actions running on CT when killed 38/24238/28
vinayakswami hariharmath [Mon, 20 Mar 2017 11:35:16 +0000 (17:05 +0530)]
LU-5216 hsm: cancel hsm actions running on CT when killed

This patch handles:

1. Unexpected client (data mover node) eviction could
cause on going hsm requests to be stuck in "STARTED"
state as the copy tool running on the data mover node
is not available anymore and requests could not be
finished. This patch unregisters the copy tool and
cancels all the requests on the copytool's agent.

2. The way to stop a copytool is to KILL it. In this
case also, as explained above, on going hsm operations
get stuck in "STARTED" state and if another copytool
tries to do any hsm activity on that file, it will not be
processed as the hsm status of the file is still STARTED.

In such cases i.e
if the copytool's agent is killed then we are just going
to add the hsm cancel request to agent record
and mark request state as ARS_CANCELED. mdt_cordinator()
thread running in the back ground looks into the request state
and marks it as CANCELED. This allows hsm status to be
maintained at proper state and allows any further
operation to proceed as expected.

Adding test_62 to sanity-hsm.sh to verify the fix

Test-Parameters: trivial testlist=sanity-hsm
Seagate-bug-id: MRP-2464
Signed-off-by: vinayakswami hariharmath <vinayakswami.hariharmath@seagate.com>
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Change-Id: Iceb1a3450bcbcec287fe11c6a9fce45fc6097e3c
Reviewed-on: https://review.whamcloud.com/24238
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-6349 idl: remove obsolete RPC MSG flags 31/17831/8
Andreas Dilger [Wed, 6 Jan 2016 00:15:30 +0000 (17:15 -0700)]
LU-6349 idl: remove obsolete RPC MSG flags

Add comments for MSG_* flags, and delete obsolete flags
MSG_LAST_REPLAY, MSG_AT_SUPPORT, MSG_DELAY_REPLAY, MSG_VERSION_REPLAY.

Remove now-unused imp_no_lock_replay field.

Add comments for MSG_CONNECT_* flags, and delete obsolete flag
MSG_CONNECT_ASYNC.

Delete them from wirecheck files.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I4f6ba93151246d173d796a6625a683cf65f767f7
Reviewed-on: https://review.whamcloud.com/17831
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-6349 idl: clean up and document ptlrpc structures 30/17830/10
Andreas Dilger [Wed, 4 Mar 2015 08:58:47 +0000 (01:58 -0700)]
LU-6349 idl: clean up and document ptlrpc structures

Miscellaneous cleanups for wire-related structures found during the
Lustre protocol documentation project.

Remove obsolete and unused since 2.3 OBD_QC_CALLBACK RPC.

Move definition of constants used by lustre_msg_v2 and ptlrpc_body
fields to immediately before their respective structures.

Add comments for struct members for lustre_msg_v2, ptlrpc_body_v3,
ldlm_request, and obd_statfs.

Rename RQF_MDS_INTENT_CLOSE to RQF_MDS_CLOSE_INTENT to make it more
easily found with RQF_MDS_CLOSE.  Rename mdt_intent_close_client()
and mdc_intent_close_pack() to mdt_close_intent_client() and
mdc_close_intent_pack() to match.

Rename RQF_LDLM_GL_DESC_CALLBACK to RQF_LDLM_GL_CALLBACK_DESC to make
it match RQF_LDLM_FL_CALLBACK and RQF_LDLM_BL_CALLBACK.

Remove unused MSG_OP_FLAG_MASK, MSG_OP_FLAG_SHIFT, MSG_GEN_FLAG_MASK.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I21be2ec0aacef9965ff9c835174b5b017b3ebbe5
Reviewed-on: https://review.whamcloud.com/17830
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10134 lfsck: not add requests if engine out of work 65/30165/5
Fan Yong [Wed, 22 Nov 2017 02:38:43 +0000 (10:38 +0800)]
LU-10134 lfsck: not add requests if engine out of work

There is race condition between LFSCK assistant engine and
LFSCK request generators: before the LFSCK assistant engine
exit, it will mark itself as 'stopping', then cleanup the
in-queue requests, and then mark itself as 'stopped'. It is
expected that the 'stopping' status will prevent generators
adding more LFSCK requests. But current implementation only
checks 'stopped' or not. So if the LFSCK engine thread exit
before the whole system scanned that may because of some
failures or on demand, more LFSCK requests may be added in
the queue after the cleanup.

The patch fixes the wrong logic by checking 'running' or not,
and stop adding more LFSCK requests if not 'running'.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic2b5ca3f5e80b5be5a5c60aa24f0b54682b717d9
Reviewed-on: https://review.whamcloud.com/30165
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9983 osp: align the OSP request size by 4k 70/29270/4
Di Wang [Tue, 26 Sep 2017 00:25:07 +0000 (20:25 -0400)]
LU-9983 osp: align the OSP request size by 4k

In osp_object_update_request_create() round up the size of the object
update request to a multiple of the PAGE_SIZE to avoid issues with IB
HW that cannot handle gaps in memory regions.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: Id5c16b031dcb16d764c4e4f325f51b9ecf454533
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: https://review.whamcloud.com/29270
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7585 osd-ldiskfs: auto scrub control 10/29710/10
Fan Yong [Wed, 29 Nov 2017 02:53:36 +0000 (10:53 +0800)]
LU-7585 osd-ldiskfs: auto scrub control

Originally, there was a lproc interface in OSD, named
"auto_scrub", used for the control of auto detecting
inconsistent OI mapping and trigger OI scrub. But such
switch is too simily, either 'on' or 'off'. It is not
convenient for the real system control. For example:
If we just finished one cycle OI scrub, should we auto
detect OI inconsistency in the subsequent lookup()?
If yes, it will cause some unnecessary overhead. But
if no, then as long as there was once OI scrub, we will
have no more chance to auto detect corrupted OI mapping.

To resolve such trouble, this patch enhances the lproc
interface "auto_scrub" to allow the system administrator
to specify how long (in seconds) the system can be trusted
after the lastest OI scrub. During such trusted interval,
we will not auto detect inconsistent OI mapping. the
default value is one month (60 * 60 * 24 * 30).

The patch also replaces cfs_time_current_sec() with
ktime_get_real_seconds() for some cleanup work.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iae2c7dd1da92c27d40357c62cd94e886228c86f7
Reviewed-on: https://review.whamcloud.com/29710
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7585 scrub: general framework for OI scrub 07/28607/18
Fan Yong [Wed, 29 Nov 2017 02:52:30 +0000 (10:52 +0800)]
LU-7585 scrub: general framework for OI scrub

Reconstruct OI scrub code to make some data structure and
functions can be shared between osd-ldiskfs and osd-ZFS.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4c4e3fcbd11078d10592136e9e55de66516a2b16
Reviewed-on: https://review.whamcloud.com/28607
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
3 years agoLU-10129 lnd: set device capabilities 09/30309/4
Amir Shehata [Tue, 28 Nov 2017 00:13:51 +0000 (16:13 -0800)]
LU-10129 lnd: set device capabilities

MLX-4, MLX-5 and OPA support different capabilities. Query the
device and cache the capabilities of the device for future use.

MLX5 can support fast registration and gaps
MLX4 and OPA only support FMR

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I70d468f8af52d263139e7b51341bf4b5150b89c1
Reviewed-on: https://review.whamcloud.com/30309
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7340 mdd: changelogs garbage collection 03/27103/26
Bruno Faccini [Fri, 12 May 2017 21:34:49 +0000 (23:34 +0200)]
LU-7340 mdd: changelogs garbage collection

When changelogs are almost full (few number of
free entries in catalog), try to recover some space
by unregistering users that are idle since too long,
based on new Changelog User record field (in fact,
using previous cur_padding unused field) to keep
track of last user's changelog cancel request time,
or based on gap between user index and current
ChangeLog record, for older registered user.
sanity/test_160[f,g] have been added to verify feature.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I2100b101772e6d027675e5efa5606d4be24342a0
Reviewed-on: https://review.whamcloud.com/27103
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-10344 test: create files on local node in sanity-hsm 33/30433/3
John L. Hammond [Thu, 7 Dec 2017 16:11:26 +0000 (10:11 -0600)]
LU-10344 test: create files on local node in sanity-hsm

In sanity-hsm, rewrite create_file() to call dd ... directly rather
than do_facet $SINGLEAGT dd ... Rewrite create_archive_file() to call
do_facet $SINGLEAGT ... directly instead of calling
create_file(). Move the Lustre parent directory creation from
create_archive_file() to import_file().

Test-Parameters: trivial testlist=sanity-hsm

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I923475c32fa69b0ac59b95b90b42c147cf361274
Reviewed-on: https://review.whamcloud.com/30433
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
3 years agoNew tag 2.10.56 2.10.56 v2_10_56 v2_10_56_0
Oleg Drokin [Tue, 12 Dec 2017 17:30:53 +0000 (12:30 -0500)]
New tag 2.10.56

Change-Id: I128e691f84fed31ea3b9332be6e254252c6de174
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10296 obdclass: no errors on missing o_cleanup 14/30314/2
Alex Zhuravlev [Wed, 29 Nov 2017 16:02:46 +0000 (19:02 +0300)]
LU-10296 obdclass: no errors on missing o_cleanup

it's legal to have o_cleanup undefined.

Change-Id: Ifd80dbd36fabcbfbd9a1609746d79a37e5a01023
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/30314
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-10297 lod: prepare inuse array always 34/30334/2
Bobi Jam [Fri, 1 Dec 2017 01:56:47 +0000 (09:56 +0800)]
LU-10297 lod: prepare inuse array always

The OST inuse array should be prepared for composite files and for
plain files as well, since creating OST objects in both cases needs
to use it.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ibb37b680c68f8883650cdee6bebebc1c4d844623
Reviewed-on: https://review.whamcloud.com/30334
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-930 man: better describe conf_parm -d in lctl.8 41/30241/2
Andreas Dilger [Thu, 23 Nov 2017 19:11:36 +0000 (12:11 -0700)]
LU-930 man: better describe conf_parm -d in lctl.8

Add a bit more description to "lctl conf_param -d".

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ib7a71122ea3e77e910d817a0ecdc4e6d1ddfd2eb
Reviewed-on: https://review.whamcloud.com/30241
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-4761 tests: clean up spurious messages and files 12/29012/10
Andreas Dilger [Wed, 7 Jun 2017 23:54:56 +0000 (17:54 -0600)]
LU-4761 tests: clean up spurious messages and files

Reduce spurious messages that appear in the test and console logs
that are not useful for debugging, and just clutter things up.

Clean up a few files left over from tests.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ie17888a8f9c5864530dcb455a0cd4e51f83ebbe5
Reviewed-on: https://review.whamcloud.com/29012
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-4134 obdclass: fix double free in failure path 67/29967/8
Yang Sheng [Mon, 27 Nov 2017 21:14:49 +0000 (16:14 -0500)]
LU-4134 obdclass: fix double free in failure path

We should just decref for obd if register is failed.
Since class_export_put will invoke class_free_dev,
then we may face a double free.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ia8d1e487c69c4de1c7c247158cc8615aa6b6093a
Reviewed-on: https://review.whamcloud.com/29967
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9727 lustre: add uid/gid to Changelogs entries 14/28114/18
Sebastien Buisson [Wed, 19 Jul 2017 14:44:27 +0000 (23:44 +0900)]
LU-9727 lustre: add uid/gid to Changelogs entries

Add a new changelog extension named changelog_ext_uidgid to hold
uid/gid information.
uid/gid info is added to every Changelog entry type except MARK, in
the form 'u=<uid>:<gid>':
5 01CREAT 15:44:32.385864793 2017.07.18 0x0 t=[0x200000402:0x3:0x0]
ef=0x1 u=500:500 p=[0x200000402:0x2:0x0] file1

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ie09e4cd146dea75985de00b1da58f75c2a5928f2
Reviewed-on: https://review.whamcloud.com/28114
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
3 years agoLU-10224 obd: free obd_svc_stats when all users are gone 49/30249/4
Bruno Faccini [Tue, 28 Nov 2017 15:15:16 +0000 (10:15 -0500)]
LU-10224 obd: free obd_svc_stats when all users are gone

During object device shutdown obd_svc_stats must only be freed
after all access methods from user-land are no longer possible
to prevent any race and further crash.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Iea4f5b0486779c1721c90f32538af1a723f76a79
Reviewed-on: https://review.whamcloud.com/30249
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5761 tests: re-add test_89 to replay-single ALWAYS_EXCEPT 52/30252/3
Andreas Dilger [Mon, 27 Nov 2017 00:53:49 +0000 (17:53 -0700)]
LU-5761 tests: re-add test_89 to replay-single ALWAYS_EXCEPT

Re-add replay-single test_89 to the ALWAYS_EXCEPT list, as it
is still failing intermittently.

Test-Parameters: trivial testlist=replay-single
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia98e28a2c4b863fd48cf0cd5d5ca9fe9cbafe9f3
Reviewed-on: https://review.whamcloud.com/30252
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoMerge "LU-9771 flr: Merge branch 'flr'"
Intel HPDD Gerrit Code Review [Fri, 1 Dec 2017 05:17:10 +0000 (05:17 +0000)]
Merge "LU-9771 flr: Merge branch 'flr'"

3 years agoLU-9897 utils: make liblnetconfig a hard requirment 04/30204/3
James Simmons [Wed, 22 Nov 2017 02:43:14 +0000 (21:43 -0500)]
LU-9897 utils: make liblnetconfig a hard requirment

With the recent changes to LNet lnetctl is now really a hard
requirment. This patch makes lustre require libyaml libraries
in order to build. In turn liblnetconfig and lnetctl are now
always built.

Test-Parameters: trivial

Change-Id: I26ff9397f3d5ba11da5ab4e76658ffd8c27ed035
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30204
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10256 hsm: enable setting archive_id in hsm_set 50/30150/5
Thomas Stibor [Fri, 17 Nov 2017 14:31:39 +0000 (15:31 +0100)]
LU-10256 hsm: enable setting archive_id in hsm_set

Setting HSM flags in lfs_hsm_change_flags(...,LFS_HSM_SET)
does not allow to specify the archive_id, that is,
in llapi_hsm_state_set(path, mask, 0, 0 /* archive_id */)
archive_id = 0 is always set, which means no identifier change.
For having full flexibility (e.g. for debugging), introduce
the additional option --archive-id in hsm_set. If the option
is not provided, the default behavior (archive_id = 0) is used
and no archive identifier change is done. In addition a
test case is provided to check the functionality and shell
function get_hsm_archive_id() is modified in favor of more
robust grep for contents after pattern approach.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: I2145a18ecf32479527bb045140e5e881e58dd115
Reviewed-on: https://review.whamcloud.com/30150
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8497 osp: handle remote -ENOMEM from osp_send_update_req() 83/30083/2
John L. Hammond [Tue, 14 Nov 2017 18:23:28 +0000 (12:23 -0600)]
LU-8497 osp: handle remote -ENOMEM from osp_send_update_req()

In osp_send_update_req() detect an unsent request by checking
rq_queued_time == 0 rather than rq_set == NULL, which is always true
after returning from ptlrpc_queue_wait().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ief959b71600157a9c3521775cc06994326e50c51
Reviewed-on: https://review.whamcloud.com/30083
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8402 llite: simplify ll_inode_revalidate() 36/30036/2
John L. Hammond [Fri, 10 Nov 2017 16:08:44 +0000 (10:08 -0600)]
LU-8402 llite: simplify ll_inode_revalidate()

ll_inode_revalidate() is only called by ll_getattr() so move the
contents of ll_inode_revalidate() to ll_getattr() and rename
__ll_inode_revalidate() to ll_inode_revalidate().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ieeba1b74551b7a531e1db47ccb0638c72c90adb6
Reviewed-on: https://review.whamcloud.com/30036
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8402 llite: assume OBD_CONNECT_ATTRFID 10/30010/3
John L. Hammond [Wed, 8 Nov 2017 16:12:02 +0000 (10:12 -0600)]
LU-8402 llite: assume OBD_CONNECT_ATTRFID

OBD_CONNECT_ATTRFID has been supported by MDSs since before 1.6 so add
it to the client side required flags and remove some code to handle
servers that do not support it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I841ae21811c0903c10a84e659d2d1f8255c8109f
Reviewed-on: https://review.whamcloud.com/30010
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
3 years agoLU-8402 ldlm: assume OBD_CONNECT_IBITS 09/30009/3
John L. Hammond [Wed, 8 Nov 2017 15:44:51 +0000 (09:44 -0600)]
LU-8402 ldlm: assume OBD_CONNECT_IBITS

Clients and MDSs have supported and required OBD_CONNECT_IBITS since
before 1.6 so remove obsolete code to handle clients that do not
support this flag.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I24a623b063c518755b81d02fc48fa0f76aacd318
Reviewed-on: https://review.whamcloud.com/30009
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10165 llite: disable statahead if starting statahead fail 17/29817/4
Lai Siyao [Fri, 27 Oct 2017 03:48:30 +0000 (11:48 +0800)]
LU-10165 llite: disable statahead if starting statahead fail

Once starting statahead thread fails, it should disable statahead.
Current code only does this when "sai != NULL", instead it should
check whether current process is opening this dir, so for cases
like current file is not the first dirent, or sai allocation fail,
it won't retry statahead.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Iaedddd3659cdffeab51800f45b02f0b39c4a1ec1
Reviewed-on: https://review.whamcloud.com/29817
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5170 lfs: Standardize error messages in macros 50/28250/2
Steve Guminski [Wed, 12 Jul 2017 14:46:33 +0000 (10:46 -0400)]
LU-5170 lfs: Standardize error messages in macros

Error messages in ARG2INT() and ARG2ULL() are updated to a standard
format.  Messages are prefixed with the name of the utility and the
command that caused the error.  User-provided values are delimited
with single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I31a0a30ac15681826e7e25b8a44d56174fb23e08
Reviewed-on: https://review.whamcloud.com/28250
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-9727 lustre: Add an additional set of 64 changelog flags. 45/28045/18
Matthew Sanderson [Fri, 14 Jul 2017 08:14:23 +0000 (18:14 +1000)]
LU-9727 lustre: Add an additional set of 64 changelog flags.

This adds a new changelog extension containing 64 additional flag
bits, to be used for future changelog extensions.
The presence of the extension is signalled using the last remaining
unused changelog flag bit.
The new extension is present in all changelog records by default, but
will be removed from records read by legacy changelog consumers.

Signed-off-by: Matthew Sanderson <matthew.sanderson@anu.edu.au>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iad05880e82400d7e84d927e84e44d1454e240a80
Reviewed-on: https://review.whamcloud.com/28045
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10271 hsm: don't release with wrong size 40/30240/2
Alexander Boyko [Thu, 23 Nov 2017 09:51:02 +0000 (04:51 -0500)]
LU-10271 hsm: don't release with wrong size

The result of ll_merge_attr operation was ignored during
ll_hsm_release. For this case the released file could have
the wrong size on MDS, like
File: ‘released.file’
Size: 0  Blocks: 0  IO Block: 4194304 regular empty file

Patch adds test_253 sanity-hsm, to check hsm release
operation when cl_object_attr_get failed. It produces
the wrong size-on-mds for a released file.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I94df1889265e5b6d03b16745de93e52af95d5b7c
Reviewed-on: https://review.whamcloud.com/30240
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-10176 tests: limit DoM tests to new servers only 85/30085/2
Mikhal Pershin [Tue, 14 Nov 2017 19:40:21 +0000 (22:40 +0300)]
LU-10176 tests: limit DoM tests to new servers only

Run DoM tests only if server has support for DoM by checking
its version is 2.10.55 at least.

Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I79f26cd3c80fbfc4bf48ddc3fcc762248d491034
Reviewed-on: https://review.whamcloud.com/30085
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
3 years agoLU-6142 uapi: Get rid of lustre_fid typedef 49/29849/8
James Simmons [Mon, 20 Nov 2017 21:16:52 +0000 (16:16 -0500)]
LU-6142 uapi: Get rid of lustre_fid typedef

Replace it with struct lu_fid. Update the userland code and man
pages to reflect this change.

Linux-commit: d8f6bc9a53f97d1ea4b2b955672904338643308b

Test-Parameters: trivial

Change-Id: I0b7e0770dd9da9bdac55c02c2ec98aea7cea7100
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/29849
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-10220 mdd: fix buf alloc in mdd_changelog_data_store_by_fid 14/30014/5
Sebastien Buisson [Thu, 9 Nov 2017 15:48:08 +0000 (00:48 +0900)]
LU-10220 mdd: fix buf alloc in mdd_changelog_data_store_by_fid

Fix allocation of mti_big_buf by call to lu_buf_check_and_alloc()
in mdd_changelog_data_store_by_fid().
reclen must take the header size of struct llog_changelog_rec into
account.

Maybe no memory corruptions were seen before because the buffer size
allocated in a previous call to mdd_declare_changelog_store() was
covering the need. But audit will add more information in changelog
records, provoking memory corruptions without this fix.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id0a06c412b54c0ae12c15d53f3e166e3e5d9ed68
Reviewed-on: https://review.whamcloud.com/30014
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-9887 tests: adjust lfsck speek test error range 47/30247/2
Fan Yong [Fri, 24 Nov 2017 14:12:38 +0000 (22:12 +0800)]
LU-9887 tests: adjust lfsck speek test error range

To make sanity-lfsck test_9{a,b} to be more robust for kinds
of environment.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ie0740f2b205f319cbefe54c86734b889e6d03cc6
Reviewed-on: https://review.whamcloud.com/30247
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
4 years agoLU-10255 tests: Fix wrong comparison operator in sanity 129 49/30149/3
Nikolay Borisov [Fri, 17 Nov 2017 14:23:04 +0000 (16:23 +0200)]
LU-10255 tests: Fix wrong comparison operator in sanity 129

Currently sanity 129 uses numerica (-eq) operator to check whether
the has_warning variable is true or false depending on whether
expected warnings have been detected. This is wrong since bash
doesn't really have bolean types and true/false are not evaluated to
1/0. This results in the following bash error when the test is run:

    ./sanity.sh: line 9244: [: true: integer expression expected

Fix it by using the correct string operator '=' to check for
correct value.

Test-Paramters: clientdistro=sles12sp2 testlist=sanity
Test-Paramters: clientdistro=sles12sp3 testlist=sanity

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: I071dcb3001ea9b2e5806d346dcd48bfcaf3297e9
Reviewed-on: https://review.whamcloud.com/30149
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10229 test: Add sanity-sec test_27 to ALWAYS_EXCEPT 35/30135/3
Jim Casper [Thu, 16 Nov 2017 18:48:51 +0000 (13:48 -0500)]
LU-10229 test: Add sanity-sec test_27 to ALWAYS_EXCEPT

Received request to ALWAYS_EXCEPT test_27 since it has been
failing regularly on master.  This will keep it from affecting
other patches.

Test-Parameters: trivial testlist=sanity-sec

Change-Id: I630b5cd1f978025e605ecad38cd5e276afdab694
Signed-off-by: Jim Casper <jamesx.casper@intel.com>
Reviewed-on: https://review.whamcloud.com/30135
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
4 years agoLU-10241 kernel: kernel update RHEL6.9 [2.6.32-696.16.1.el6] 30/30130/2
Bob Glossman [Tue, 14 Nov 2017 20:44:15 +0000 (12:44 -0800)]
LU-10241 kernel: kernel update RHEL6.9 [2.6.32-696.16.1.el6]

Update RHEL6.9 kernel to 2.6.32-696.16.1.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
  ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib2215b56e647fb8e19bc8b7e344a94564c7a8c7f
Reviewed-on: https://review.whamcloud.com/30130
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-9135 osp: do not fail if last_rcvd update failed 29/30129/2
Alex Zhuravlev [Thu, 16 Nov 2017 12:43:32 +0000 (15:43 +0300)]
LU-9135 osp: do not fail if last_rcvd update failed

just ignore an error in this case and hope the next time
OST will be able to serve well and OSP cancel the record.

Change-Id: Ib1bbe77132e81322f353909620b6a413981ae342
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/30129
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10038 test: improve error reporting in sanity test_133g() 05/30105/2
John L. Hammond [Wed, 15 Nov 2017 16:05:57 +0000 (10:05 -0600)]
LU-10038 test: improve error reporting in sanity test_133g()

In sanity test_133g() remove the output redirection to /dev/null so
that errors from do_facet or find go to the test logs. Suppress error
messages in badarea_io.c.

Test-Parameters: trivial envdefinitions=ONLY=133 testlist=sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia51ccdacb995ff51a6467cbdd280c8965cfc5f47
Reviewed-on: https://review.whamcloud.com/30105
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10217 test: Make sanity 248 work with newer dds 05/30005/5
Nikolay Borisov [Thu, 9 Nov 2017 10:39:19 +0000 (12:39 +0200)]
LU-10217 test: Make sanity 248 work with newer dds

This patch fixes 2 problems when working with value from dd.

Firstly, extraction of the readings from dd rely on them always
being in field number 6. However, that's not true for newer dd
versions where the output has changed and the time reading is now in
field number 10. So to make the code more generic and work with both
old and new outputs switch to using an egrep regular expression to
extract the correct field.

Secondly, dd can output its time in scientific notation, with
decimal and floating point portion being separated by a comma rather
than a dot:
   4096 bytes (4,1 kB, 4,0 KiB) copied, 9,1229e-05 s, 44,9 MB/s

Unforuntately this is not understood by bc so it needs to be
processed by substituting the comma for a dot and also expanding
the scientific notation to multiplication of 10 to the correct power.

Test-Parameters: trivial
Test-Parameters: clientdistro=sles12sp2 testlist=sanity
Test-Parameters: clientdistro=sles12sp3 testlist=sanity

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: I83ec659874ffe702c68f94f828827ff93325db72
Reviewed-on: https://review.whamcloud.com/30005
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10204 tests: Fix line continuation 81/29981/4
Nikolay Borisov [Tue, 7 Nov 2017 16:44:11 +0000 (18:44 +0200)]
LU-10204 tests: Fix line continuation

There is a simple  missing in one of the error paths in
ladvise_willread_performance which results in an unknown command
error. Fix it by adding the missing backslash.

Test-Parameters: trivial
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: Ifff854c6cf5c1add71fd1571cb4de01d56204fde
Reviewed-on: https://review.whamcloud.com/29981
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
4 years agoLU-8950 tests: standardize file creation in sanity-hsm 04/24704/8
Quentin Bouget [Thu, 8 Dec 2016 15:30:44 +0000 (16:30 +0100)]
LU-8950 tests: standardize file creation in sanity-hsm

sanity-hsm exposed several functions to create files:
 - make_small()
 - make_small_sync()
 - make_archive()

This patch renames all these functions (respectively):
 - create_small_file()
 - create_small_sync_file()
 - create_archive()

And adds a new one: create_empty_file() which creates an empty file

The actual file_creation is now abstracted in the create_file()
function used by all the aforementionned functions.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: Ib4267827eb2e31ed396da1d711419f45f9e0ef16
Reviewed-on: https://review.whamcloud.com/24704
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-3285 mdt: remove assertion on mdt_pack_size2body() 58/30058/3
Mikhal Pershin [Mon, 13 Nov 2017 14:10:52 +0000 (17:10 +0300)]
LU-3285 mdt: remove assertion on mdt_pack_size2body()

The assertion on result of mdt_pack_size2body() is not
always correct, e.g. in case of resend.

Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: Iff2c9723959db591fdd29074b0186d8ddf0752e9
Reviewed-on: https://review.whamcloud.com/30058
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-3285 mdt: get lu_object_attr for existent object 57/30057/4
Mikhal Pershin [Mon, 13 Nov 2017 10:18:36 +0000 (13:18 +0300)]
LU-3285 mdt: get lu_object_attr for existent object

In mdt_punch_hdl() lu_object_attr() may be taken for
non-existent object with LBUG() as result.
Take lu_object_attr() after mdt_object_exists()check.

Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: If1181caa94b28cf1f251f57fa6f7102dc40c329d
Reviewed-on: https://review.whamcloud.com/30057
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-3285 tests: disable kernel flusher in discard tests 16/30016/3
Mikhail Pershin [Thu, 9 Nov 2017 17:35:43 +0000 (20:35 +0300)]
LU-3285 tests: disable kernel flusher in discard tests

Test 101 checks that client discards data instead of flush,
but it is not reliable because write RPC may happens before
unlink command is executed due to kernel flusher.
Set /proc/sys/vm/dirty_writeback_centisecs to 0 for the test
time and return it back to the default value later.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I2c4da039a456519e25c6d9ffcc015c5baa479740
Reviewed-on: https://review.whamcloud.com/30016
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-9872 tests: modify check space requirements for NFS test 86/29786/14
James Nunez [Mon, 20 Nov 2017 04:22:38 +0000 (21:22 -0700)]
LU-9872 tests: modify check space requirements for NFS test

In parallel-scale-nfs testing, the Lustre clients NFS mounts
the Lustre file system. In some cases, the file system looks
like it has enough space (du) to run NFS testing, but one or
more OSTs is full or is almost full. Since the default striping
for Lustre is a single stripe, some NFS tests will fail because
the file it is writing gets written to that almost full OST.

Increase the default stripe to all OSTs for all
parallel-scale-nfs tests.

Test-Parameters: trivial testlist=parallel-scale,parallel-scale-nfsv3,parallel-scale-nfsv4
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Iff187dc6c158d9063cd36ced0801b88cdc8814ae
Reviewed-on: https://review.whamcloud.com/29786
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Casper <jamesx.casper@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-9771 flr: Merge branch 'flr' 50/30250/2
Jinshan Xiong [Tue, 28 Nov 2017 16:56:24 +0000 (16:56 +0000)]
LU-9771 flr: Merge branch 'flr'

Merge remote-tracking branch 'origin/flr'.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Idee9297fbcab2bea3bd5987c94e4b4e79c49b3b6

4 years agoLU-9771 flr: mirror resync multiple files 42/30242/4
Bobi Jam [Fri, 24 Nov 2017 04:41:59 +0000 (12:41 +0800)]
LU-9771 flr: mirror resync multiple files

mirror resync multiple files in one command line.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I4e301070f678fbad2dd8663d52ded7294ad5dad5
Reviewed-on: https://review.whamcloud.com/30242
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
4 years agoLU-9771 tests: make racer work for flr 74/30074/9
Jinshan Xiong [Fri, 17 Nov 2017 07:11:10 +0000 (23:11 -0800)]
LU-9771 tests: make racer work for flr

Creating flr file in racer tests.

Test-parameters: trivial testlist=racer,racer,racer mdscount=1 mdtcount=1

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I56cef133263a05050ebdb0e3b16ccd0c737fe27f
Reviewed-on: https://review.whamcloud.com/30074
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: support extending a plain file to mirrored file 65/29665/14
Jinshan Xiong [Tue, 14 Nov 2017 00:12:25 +0000 (00:12 +0000)]
LU-9771 flr: support extending a plain file to mirrored file

Support extending a plain file to mirrored file by
'lfs mirror extend -N <a_plain_file>', so that <a_plain_file>
becomes a mirrored file, and it contins 2 mirrors each of them
has its old layout parameters covering [0, EOF).

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I415cd65effa72a4bde70b02063d8b635866947a8
Reviewed-on: https://review.whamcloud.com/29665
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: to not support dom+flr for phase 1 09/29609/17
Jinshan Xiong [Fri, 17 Nov 2017 07:06:24 +0000 (23:06 -0800)]
LU-9771 flr: to not support dom+flr for phase 1

To exclude the support of dom+flr.

Test-Parameters: testlist=sanity-flr

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I3b2ff9bf5dd77f2703d8292226b164a9789134b1
Reviewed-on: https://review.whamcloud.com/29609
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: make sure init size work 81/29581/17
Jinshan Xiong [Sat, 14 Oct 2017 19:59:06 +0000 (12:59 -0700)]
LU-9771 flr: make sure init size work

Lustre allows to create a file with empty layout, and truncate it
up with non-zero file size. The size should be applied later after
a layout is attached.

Test-Parameters: testlist=sanity-flr

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie645f8172e98426a531eb2d9ffdcbec8f73f3731
Reviewed-on: https://review.whamcloud.com/29581
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: inherit mirror options from directory 76/29576/16
Jinshan Xiong [Sat, 14 Oct 2017 07:55:23 +0000 (00:55 -0700)]
LU-9771 flr: inherit mirror options from directory

This patch checks and verifies directory inheritance from parent
works both for regular files and directories.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ief58805ca736acb9a53fd9ad8fb2bb460d46b748
Reviewed-on: https://review.whamcloud.com/29576
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: lfs mirror resync command 43/29243/24
Bobi Jam [Sat, 14 Oct 2017 07:49:27 +0000 (00:49 -0700)]
LU-9771 flr: lfs mirror resync command

This patch adds "lfs mirror resync" command to resynchronize a
mirrored file.

Usage:

lfs mirror resync [--only <mirror_id[,...]>] <mirrored file>

Options:

--only <mirror_id[,...]>
Indicates which mirror(s) specified by mirror_id(s) needs to
be resynchronized. The mirror_id is the numerical unique
identifier for a mirror. Multiple mirror_ids are separated
by comma.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic2aa474d1023a4364aa9d90c59120b1d5c89a269
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: https://review.whamcloud.com/29243
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
4 years agoLU-9771 flr: FLR+PFL test cases 01/29101/31
Jian Yu [Sat, 14 Oct 2017 07:42:39 +0000 (00:42 -0700)]
LU-9771 flr: FLR+PFL test cases

* add some PFLR test cases.
* improve some lfs getstripe output.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Id27ad50456f1c51287df61fc93a2b18a3c6a6361
Reviewed-on: https://review.whamcloud.com/29101
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: lfs mirror create and extend commands 00/29100/28
Jian Yu [Fri, 15 Sep 2017 21:43:10 +0000 (21:43 +0000)]
LU-9771 flr: lfs mirror create and extend commands

This patch adds "lfs mirror create" and "lfs mirror extend"
commands to create or extend a mirrored file.

Usage:

lfs mirror create <--mirror-count|-N[mirror_count]>
                  [setstripe options|--parent]
                  ... <filename|directory>

lfs mirror extend [--no-verify]
                  <--mirror-count|-N[mirror_count]>
                  [setstripe options|--parent|-f <victim_file>]
                  ... <filename>
Options:

--mirror-count|-N[mirror_count]
Number of mirrors to be created with the upcoming setstripe
layout options. The option can be repeated multiple times to
separate mirrors that have different layouts. The mirror_count
argument is optional and defaults to 1 if it's not specified;
if specified, it must follow the option without a space.

setstripe options
The layout of one mirror. The options are the same as those for
lfs setstripe command. If setstripe options are not specified,
then the stripe options inherited from the previous component
will be used.

--parent
This option indicates that the default stripe options inherited
from parent directory will be used.

-f <victim_file>
The layout of victim_file will be split and used as a mirror
added to the mirrored file.

--no-verify
This option indicates not to verify the mirror(s) from victim
file(s) in case the victim file(s) contains the same data as
the original mirrored file.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8fb4a367bd169073f194ddee5c26d65b557e8201
Reviewed-on: https://review.whamcloud.com/29100
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: instantiate component for truncate 99/29099/21
Bobi Jam [Mon, 4 Sep 2017 09:28:58 +0000 (17:28 +0800)]
LU-9771 flr: instantiate component for truncate

layout intent for truncate IO needs stale components for non
primary mirror overlapping [trunc_size, eof), while needs to
instantiate components overlapping [0, trunc_size).

Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ibf31e3e1fd68d1a643f2ed3ac5b9e3c778563895
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: https://review.whamcloud.com/29099
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 lod: support setting mirror layout from CLI 98/29098/20
Jinshan Xiong [Fri, 15 Sep 2017 21:29:58 +0000 (21:29 +0000)]
LU-9771 lod: support setting mirror layout from CLI

Parse and support mirror layout from CLI.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I51d24aecb63af04cb3c1dd1bcfbf5728030b233b
Reviewed-on: https://review.whamcloud.com/29098
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: randomize mirror pick 97/29097/20
Jinshan Xiong [Sat, 26 Aug 2017 06:53:58 +0000 (23:53 -0700)]
LU-9771 flr: randomize mirror pick

Pick mirror randomly for read on client side and write on the MDT.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8dff91e6b7354a9841a2f1595df611717fe51afb
Reviewed-on: https://review.whamcloud.com/29097
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 osp: no delay commit callback for layout version 00/30000/10
Jinshan Xiong [Wed, 8 Nov 2017 23:30:44 +0000 (23:30 +0000)]
LU-9771 osp: no delay commit callback for layout version

OSP batches commit callback by one second interval. However, this
mechanism would cause problem to FLR because the clients has to
wait up to one second before they are able to write OST objects.

This patch registers immediate commit callback if the SETATTR is
for layout version. It also cleans up the code a little bit.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I5fa0fea9a8c8b23379118609bee9a36f412a191d
Reviewed-on: https://review.whamcloud.com/30000
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: resync support and test tool 96/29096/21
Jinshan Xiong [Fri, 15 Sep 2017 21:22:41 +0000 (21:22 +0000)]
LU-9771 flr: resync support and test tool

A tool to resync mirrored file after writing.
It extends the Lustre lease API to support taking file lease and then
sending the MDS_REINT_RESYNC RPC to the MDT so that it can increase
the file's layout version; then the client will start copying
the contents from valid mirror to stale mirrors. At the end of
resync, the copying client will release the lease and revalidate
stale mirrors.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I59f84cd501f6945225c97bdb99a142ae7efbf0fb
Reviewed-on: https://review.whamcloud.com/29096
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-9771 flr: mirror read and write 95/29095/17
Jinshan Xiong [Fri, 15 Sep 2017 21:13:10 +0000 (21:13 +0000)]
LU-9771 flr: mirror read and write

Support to perform I/O to designated mirror.

Three APIs have been exported for mirror IO:
1. llapi_mirror_read();
2. llapi_mirror_copy();
3. llapi_mirror_{set,clear}().

Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iae1d1edea8e72b423f6f46821594a7d7791ff69b
Reviewed-on: https://review.whamcloud.com/29095
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-10226 tests: properly check for fileset clear and error 18/30218/2
Sebastien Buisson [Fri, 10 Nov 2017 16:16:47 +0000 (17:16 +0100)]
LU-10226 tests: properly check for fileset clear and error

Make sure fileset info on nodemap is cleared everywhere before
continuing test.
Also, properly exit on this error.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1d117ffef6c4a1002d3eba47a301e64830be4b4f
Reviewed-on: https://review.whamcloud.com/30037
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit eae0f97355cf9a20066815b004716e4f148cec3d)
Reviewed-on: https://review.whamcloud.com/30218
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
4 years agoLU-10038 test: improve error reporting in sanity test_133g() 19/30219/2
John L. Hammond [Wed, 15 Nov 2017 16:05:57 +0000 (10:05 -0600)]
LU-10038 test: improve error reporting in sanity test_133g()

In sanity test_133g() remove the output redirection to /dev/null so
that errors from do_facet or find go to the test logs. Suppress error
messages in badarea_io.c.

Test-Parameters: trivial envdefinitions=ONLY=133 testlist=sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia51ccdacb995ff51a6467cbdd280c8965cfc5f47
Reviewed-on: https://review.whamcloud.com/30219
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
4 years agoLU-10234 tests: use wait correctly in sanity-hsm test_3[4-6] 02/30102/4
Quentin Bouget [Wed, 15 Nov 2017 15:41:53 +0000 (15:41 +0000)]
LU-10234 tests: use wait correctly in sanity-hsm test_3[4-6]

Test 34, 35 and 36 did not properly check the return code of wait
and potentially missed regressions.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I21e8b16b55b946a8e5837549c3a3698379132e48
Reviewed-on: https://review.whamcloud.com/30102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10218 mdc: propagate changelog errors to readers 40/30040/2
John L. Hammond [Fri, 10 Nov 2017 17:00:58 +0000 (11:00 -0600)]
LU-10218 mdc: propagate changelog errors to readers

Store errors encountered by the changelog llog reader thread in the
crs_err field of struct changelog_reader_state so that they can be
propageted to changelog readers. In chlg_read() improve the error and
EOF reporting. Return -ERESTARTSYS when the blocked reader is
interrupted. Replace uses of l_wait_event() with
wait_event_interruptible().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I279ec8ca11bfeb67f844acf12e3f7cba8f8aa1e6
Reviewed-on: https://review.whamcloud.com/30040
Tested-by: Jenkins
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10226 tests: properly check for fileset clear and error 37/30037/3
Sebastien Buisson [Fri, 10 Nov 2017 16:16:47 +0000 (17:16 +0100)]
LU-10226 tests: properly check for fileset clear and error

Make sure fileset info on nodemap is cleared everywhere before
continuing test.
Also, properly exit on this error.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1d117ffef6c4a1002d3eba47a301e64830be4b4f
Reviewed-on: https://review.whamcloud.com/30037
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10182 utils: change MDT name too long error 84/29884/5
Ben Evans [Wed, 1 Nov 2017 17:02:27 +0000 (13:02 -0400)]
LU-10182 utils: change MDT name too long error

Change the error message to something more generic.
fsname-MDT00 failed with "Name too long", which was wrong

Test-parameters: trivial

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: If7c7de080c75dfdc853ff6d06e98a8ed074279cb
Reviewed-on: https://review.whamcloud.com/29884
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10168 kernel: kernel update [SLES12 SP2 4.4.90-92.45] 39/29839/5
Bob Glossman [Fri, 27 Oct 2017 16:37:41 +0000 (09:37 -0700)]
LU-10168 kernel: kernel update [SLES12 SP2 4.4.90-92.45]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If24eddeeadc825b1b3073439304c0958c0ae1b74
Reviewed-on: https://review.whamcloud.com/29839
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10166 mdc: invalid free in changelog reader 18/29818/9
Henri Doreau [Fri, 27 Oct 2017 07:31:58 +0000 (09:31 +0200)]
LU-10166 mdc: invalid free in changelog reader

Use kthread_stop() to instruct the producer thread to exit
when the device is closed, and only then: release the CRS
structure.

The previous implementation left small time windows open,
during which the producer threads could free a structure
before the consumer thread was completely done with it.

Change-Id: Id2038aa9b7fcfd2c3347f628e749f9d2c265ac6e
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-on: https://review.whamcloud.com/29818
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-10161 lbuild: fix the problem of determine default_iface 76/29776/4
Li Xi [Wed, 25 Oct 2017 17:53:37 +0000 (13:53 -0400)]
LU-10161 lbuild: fix the problem of determine default_iface

The script of determining the default interface is broken.
Remove complexity and replace with hostname

Test-Parameters: trivial
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I0d662e9e24bed17078e31b6b5ca305cff2c0548e
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29776
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
4 years agoLU-10136 test: Lockahead test fixes 84/29684/5
Patrick Farrell [Mon, 13 Nov 2017 16:02:21 +0000 (11:02 -0500)]
LU-10136 test: Lockahead test fixes

Two fixes:
1. Add version check for interop testing
2. Reduce count of locks requested in test15.  At 5000
locks, we would sometimes get some locks reclaimed during
the test due to memory pressure.  Reduce count to 500.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I693f8543b3579819aa64101ef30453627637ead6
Reviewed-on: https://review.whamcloud.com/29684
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>