Whamcloud - gitweb
Andreas Dilger [Wed, 21 Oct 2020 02:42:29 +0000 (20:42 -0600)]
LU-13745 tests: skip sanity test_426 for 4.18+
Add sanity test_426 to ALWAYS_EXCEPT for newer client kernels because
it is crashing 100% since "LU-13745 test: add splice test for lustre"
was landed.
Unable to handle NULL pointer dereference at address
0000000000000004
user pgtable: 64k pages, 48-bit VAs, pgdp =
000000009f14b2d0
Internal error: Oops:
96000005 [#1] SMP
CPU: 1 PID: 11273 Comm: ptlrpcd_01_01 4.18.0-147.8.1.el8_1.aarch64 #1
Process ptlrpcd_01_01 (pid: 11273, stack limit = 0x00000000f9135a93)
Call trace:
mempool_free+0x24/0xe0
llcrypt_free_bounce_page.part.1+0x38/0x48 [libcfs]
llcrypt_free_bounce_page+0x24/0x30 [libcfs]
brw_interpret+0x124/0x10c8 [osc]
ptlrpc_check_set+0x688/0x3318 [ptlrpc]
ptlrpcd_check+0x470/0x820 [ptlrpc]
ptlrpcd+0x3d4/0x5c8 [ptlrpc]
kthread+0x130/0x138
Test-Parameters: trivial clientdistro=el8.1 clientarch=aarch64
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8f7b1d5e3ee69a3e0a6dfe3944949741a74cb62a
Reviewed-on: https://review.whamcloud.com/40326
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Sat, 27 Jun 2020 11:27:06 +0000 (16:57 +0530)]
LU-13699 mdt: Improve message reporting for mdt_identity.c
This patch Improves Error handling and message reporting
for file lustre/mdt/mdt_identity.c
This patch also replaces ERR_PTR(PTR_ERR()) with ERR_CAST()
which is reported by coccinelle
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I59e262e743e7b1926fe75920239d6086c183b30f
Reviewed-on: https://review.whamcloud.com/39203
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Thu, 3 Sep 2020 22:37:47 +0000 (16:37 -0600)]
LU-12542 osd: consolidate RCU handling
Consolidate lu_object_header_fini() and kfree_rcu() into a
single lu_object_header_free() function so that callers
which do not need a container for lu_object_header can
avoid duplicating common RCU and OBD_FREE handling code.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0fa68f11b5008ede5a498d38b69ccaeecf3ebbe5
Reviewed-on: https://review.whamcloud.com/40204
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 8 Oct 2020 20:09:51 +0000 (15:09 -0500)]
LU-14020 ldiskfs: SUSE 15 SP2 and mainline 5.4.21 and newer
The updated pdirop for Ubuntu 5.4.0-42 kernel also aligns
with Linux LTS 5.4.21 through to current 5.4.60
In addition this series is also suitable for the current
SUSE 15 SP2 kernel, tested with 5.3.18-24.15.
Test-Parameters: trivial
HPE-bug-id: LUS-9092
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I75ee53c4096b31c3a2d13748401ed1a12306215b
Reviewed-on: https://review.whamcloud.com/40179
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Tue, 6 Oct 2020 19:14:29 +0000 (14:14 -0500)]
LU-14012 lod: properly initialize lcm in lod_layout_convert()
In lod_layout_convert() zero out lcm and lcme before constructing the
converted layout.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I40f96d51cb63816a9bfc34217f02ff7c450de974
Reviewed-on: https://review.whamcloud.com/40153
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Amin Tootoonchian [Mon, 5 Oct 2020 08:42:54 +0000 (03:42 -0500)]
LU-14010 build: Add missing deps for dkms client
dkms installs also need libyaml-dev and zlib1g to build and
install successfully.
Test-Parameters: trivial
HPE-bug-id: LUS-9369
Signed-off-by: Amin Tootoonchian <amint@openai.com>
Change-Id: Idbd79295590987b7dace07b4da3ee731d0714a8d
Reviewed-on: https://review.whamcloud.com/40138
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Thu, 24 Sep 2020 16:05:15 +0000 (00:05 +0800)]
LU-10753 osd-zfs: initialize obj attr correctly
mdt_thread_info.mti_attr is used to initialize object attr in create,
currently it's copied to object.oo_attr directly, but some fields
in mti_attr may contain bogus data because it's not cleared in each
use, though la_valid is correctly set, but la_flags is used without
checking la_valid in __mdd_permission_internal().
Another minor fix in osd_create(): set size/nlink to zero since they
are set in valid.
Test-Parameters: testlist=sanity env=ONLY=300,ONLY_REPEAT=100
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I64816b66a0b3c7aa50e62680d5251141697a8e0f
Reviewed-on: https://review.whamcloud.com/40062
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Tue, 22 Sep 2020 11:05:51 +0000 (19:05 +0800)]
LU-13983 llite: rmdir releases inode on client
Same as file unlink, rmdir should release inode on client, to achieve
this, ll_rmdir() update inode i_nlink after rmdir, then the last
iput() will release the inode.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I9181151de1830b48986afec30c83120e9f112a85
Reviewed-on: https://review.whamcloud.com/40011
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 21 Sep 2020 12:45:49 +0000 (12:45 +0000)]
LU-13975 sec: require enc key in case of O_CREAT only
In ll_atomic_open(), do not return -ENOKEY when trying to open
either a directory or a file without the encryption key, unless
O_CREAT flag is specified.
Indeed, listing directory content is allowed even without the key.
And in case of regular file, ll_file_open() already checks for the
presence of an encryption key.
Improve sanity-sec test_54 to verify this is working properly.
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=zfs mdscount=2 mdtcount=4
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I813d4ec938e00c8463b1d3ee9766d180806b40ba
Reviewed-on: https://review.whamcloud.com/39983
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Sat, 19 Sep 2020 08:38:07 +0000 (01:38 -0700)]
LU-13972 o2iblnd: Don't retry indefinitely
If peer is down don't retry indefinitely. Use the retry_count
parameter to restrict the number of retries. After which the
connection fails and error is propagated up.
This prevents long timeouts when mounting a file system with
nodes which might have their NIDs configured in the FS, but the
nodes have been taken offline.
Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I04faf690ed13357e3ed50c2adaadee265db269c7
Reviewed-on: https://review.whamcloud.com/39981
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Brian Behlendorf [Thu, 3 Sep 2020 16:23:17 +0000 (09:23 -0700)]
LU-13946 build: OpenZFS 2.0 compatibility
* Update zfs_refcount_add() configure check. OpenZFS 2.0 has renamed
the sys/refcount.h header to sys/zfs_refcount.h to avoid a conflict
with this header on FreeBSD. Since Lustre never directly includes
this header by name adjust the configure check to indirectly include
it through sys/dnode.h. This was we don't need a separate check to
determine the expected header name.
* Add db->db_dirty_records check. The db->db_last_dirty field was
replaced by a proper db->db_dirty_records list_t. Detect the code
change and add a osd_db_dirty_txg() helper function which returns
the largest dirty txg for the dbuf or zero when clean.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: Ifc9573ec410f50f46f2c601368639453e78b291d
Reviewed-on: https://review.whamcloud.com/39822
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Sebastien Buisson [Mon, 5 Oct 2020 12:14:09 +0000 (21:14 +0900)]
LU-13498 sec: fix credentials with nodemap and SSK
When SSK is enabled, credentials are evaluated in new_init_ucred().
In case a nodemap entry is defined with squash UID/GID, it must
prevail over normally mapped UID/GID.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1adfd98759e5b98ec78f0477846e1820fed5d8b3
Reviewed-on: https://review.whamcloud.com/40140
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Tue, 29 Sep 2020 14:25:24 +0000 (10:25 -0400)]
LU-13740 build: announce Ubuntu20 support
Now that Ubuntu20 is supported for both servers and clients lets
list it in the Changelog
Test-Parameters: trivial
Change-Id: Id413dc5bf5f8e418f379b8b9c8efc4e2d5521311
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/40088
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Sebastien Buisson <sbuisson@ddn.com>
Wang Shilong [Sun, 6 Sep 2020 21:56:56 +0000 (17:56 -0400)]
LU-13745 test: add splice test for lustre
copied from xfstests with adjustment with PAGE SIZE
alignement for DIO, and codes style cleanup.
Test-Parameters: trivial envdefinitions=ONLY=425 testlist=sanity
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I28e83abb4a1181db20a2564a10f40ca208bb2756
Reviewed-on: https://review.whamcloud.com/39695
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Etienne AUJAMES [Fri, 31 Jul 2020 18:29:43 +0000 (20:29 +0200)]
LU-13845 utils: Quota id 0xFFFFFFFF is invalid
"lfs setquota" and "lfs quota" should consider as invalid quota id
value 0xFFFFFFFF (aka. (uid_t)-1)
Fixes:
3d9900e78e ("LU-12549 utils: Check range of quota ID for lfs")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Idbd5970a6f53a544c15bdf22bcf24a7aeba772a8
Reviewed-on: https://review.whamcloud.com/39559
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 3 Sep 2020 22:45:02 +0000 (16:45 -0600)]
LU-12931 libcfs: skip cfs_time_seconds() indirection
Avoid one level of indirection when calling nsecs_to_jiffies64().
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib5ff2aae5352bc86d75a8ae2a2a9f1b406887376
Reviewed-on: https://review.whamcloud.com/40145
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 7 Oct 2020 01:25:11 +0000 (18:25 -0700)]
LU-13590 kernel: RHEL 7.9 server support
This patch makes changes to support new RHEL 7.9 release
for Lustre server (kernel 3.10.0-1160.2.1.el7).
Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9
Change-Id: I7653091f2bd6a579447edb12045984d2829a8235
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40160
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 7 Oct 2020 00:09:32 +0000 (17:09 -0700)]
LU-13590 kernel: new kernel [RHEL 7.9 3.10.0-1160.2.1.el7]
This patch makes changes to support new RHEL 7.9 release
for Lustre client.
Test-Parameters: trivial clientdistro=el7.9
Change-Id: I7a2846de48a6710d6d720d6ccc3176dba4afc6bb
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40142
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Vladimir Saveliev [Tue, 28 Jul 2020 22:28:22 +0000 (01:28 +0300)]
LU-13614 ldlm: revert LU-11762
Commit
fe5c801657 introduced a problem for recovery.
When recovery timeout reaches hard recovery timeout
target_recovery_overseer() leaves obd_recovery_expired flag set. That
makes check_for_next_transno() to not wait until next replay request
arrives which leads to assertion:
LASSERT(atomic_read(&obd->obd_req_replay_clients) == 0);
Test to illustrace the issue is added.
replay-single.sh:test_59 is added to EXCEPT_ALWAYS list:
it was broken harmlessly before this patch and this patch made that
test really fail due to that defect.
Fixes:
fe5c80165 ("LU-11762 ldlm: ensure the recovery timer is armed")
HPE-bug-id: LUS-8299
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: Ia694a519b5d73620be3014e92fd671d388550979
Reviewed-on: https://review.whamcloud.com/39532
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Fri, 2 Oct 2020 18:55:01 +0000 (13:55 -0500)]
LU-14006 o2ib: raise bind cap before resolving address
In kiblnd_resolve_addr(), ensure that the current task has
CAP_NET_BIND_SERVICE before calling rdma_resolve_addr() with a
protected source port.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I0552fdd64648ccb8c74667bd93852697f99f0c33
Reviewed-on: https://review.whamcloud.com/40127
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Sat, 26 Sep 2020 17:23:21 +0000 (10:23 -0700)]
LU-13998 tests: sanityn test_104 incorrect MDT times shown
sanityn test_104 prints STAT times instead of MDT times.
Fix it so it prints both for comparison.
Test-Parameters: trivial testlist=sanityn
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I4ae0c1fe704c02c3463043830783cd1d6cd46b98
Reviewed-on: https://review.whamcloud.com/40061
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Andriy Skulysh [Tue, 16 Jun 2020 09:49:07 +0000 (12:49 +0300)]
LU-13992 llite: ASSERTION( last_oap_count > 0 ) failed
Punch uses o_blocks to send end of a region. So it
can be mixed with real blocks count on error.
Update blocks count only on success.
Change-Id: I86241c4e5723079b20401805b853d356130f58d9
HPE-bug-id: LUS-7407
Test-Parameters: fstype=zfs
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://review.whamcloud.com/40050
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Fri, 25 Sep 2020 05:02:42 +0000 (15:02 +1000)]
LU-13986 target: fix possible liveloop in distribute_txn thd
A recent patch to update_trans.c changed how distribute_txn_thread()
waited for more work to do.
It previously had an explicit "wait_event()" which listed all the
conditions to wait for. It would then recheck each condition and
possibly perform an appropriate action.
It was changed to check each condition only once (per loop). If the
condition was true, the action would be performed and a flag set. If
no conditions were true (indicated by flag), it would wait, otherwise
it would loop and recheck all condition.
One of the "if (condition) { do work }" stanzas in the loop tested a
condition that was *not* a condition that should wake up the loop.
"batchid" was not tested at all in the wait_event(). The flag
mentioned above was, however, set when that condition tested true.
This can cause the loop to spin indefinitely.
So remove the "__set_current_state(TASK_RUNNING);" so that the value
of batchid cannot stop the loop from sleeping (calling 'schedule()').
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I124ee3e8250dc63fa927f72dc4d29ed3e7b53005
Reviewed-on: https://review.whamcloud.com/40043
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Serguei Smirnov [Thu, 24 Sep 2020 23:49:05 +0000 (16:49 -0700)]
LU-13701 utils: fix usage of getopt_long in portals
Due to char being unsigned by default, using this type to store
the int return value of getopt_long causes parsing issues on arm
platform, seen with lctl net_drop_add options. Fix it.
Test-Parameters: trivial testgroup=review-ldiskfs-arm
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ibef6ba347a90d9ed3cd4bd9346ed9f79f0120b87
Reviewed-on: https://review.whamcloud.com/40036
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 24 Sep 2020 23:46:24 +0000 (09:46 +1000)]
LU-13985 lustre: seq_file .next functions must update *pos
A seq_file .next function must update *pos on EOF to a value which
will cause a subsequent ->start to also return EOF.
If it doesn't the last record of the file can be returned
twice to a 'read()'. Also the seq_file code will generate
a warning.
This patch fixes various ->next functions to always update
*pos.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia0c432cd50550ecde6b308cbc554b316fa03adae
Reviewed-on: https://review.whamcloud.com/40035
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Sep 2020 05:23:46 +0000 (15:23 +1000)]
LU-13982 tests: fix infinite loop in sanity test_184c
If the dd in test_184c fails to create the file, for example due to
ENOSPC, the subsequent "while" loops indefinitely.
So add a loop-count to ensure it stops eventually.
Also change the test from "-f" to "-s" so we wait for the file to be
non-empty.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5051adf101f08856b97fa994f687b976fda84df4
Reviewed-on: https://review.whamcloud.com/40007
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Mr NeilBrown [Wed, 23 Sep 2020 05:11:53 +0000 (15:11 +1000)]
LU-13981 tests: use $TRUNCATE consistently and correctly.
Two changes:
1/ tests should should always use $TRUNCATE rather than just
"truncate" to ensure the tests/truncate programs is used
rather than anything in /bin /usr/bin. This might not
be needed in practice, but it is good to be consistent.
2/ The arguments provided should match what tests/truncate
expects - "filename size". Using "-s size filename"
as expected by /usr/bin/truncate is confusing and
error prone.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic0adc0148ea0ff625695dbfc4cf339d0eca70241
Reviewed-on: https://review.whamcloud.com/40005
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Tue, 15 Sep 2020 11:34:29 +0000 (14:34 +0300)]
LU-13810 tests: increase limit for 1g
With wide striping file has objects at all
OSTs. As ZFS acquires several Kb for each
inode, each OST reserves minimum qunit 1M
even without write. At clean system with
8 OSTs, it acquires 8MB after file creation.
Increase pool limit from 10M to 20M to
make sanity-quota_1g pass at ZFS with 8 OSTs.
HPE-bug-id: LUS-9349
Test-Parameters: env=ONLY=1g testlist=sanity-quota
Change-Id: I1286e57aadebdd665c51d965220961b1f758c6f5
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/39917
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Elena Gryaznova [Fri, 4 Sep 2020 11:47:04 +0000 (14:47 +0300)]
LU-13948 tests: load modules after reboot_node
LOAD_MODULES_REMOTE should be taken into account
for FAILURE_MODE=HARD.
Test-parameters: envdefinitions="LOAD_MODULES_REMOTE=true"
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-9283
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I85f0a2812ac3be4ac9645d3b165b7371504969f0
Reviewed-on: https://review.whamcloud.com/39829
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Etienne AUJAMES [Fri, 12 Jun 2020 16:06:43 +0000 (18:06 +0200)]
LU-13056 utils: Default MGS device llog_print/llog_catlist
Add the default device "MGS" for lctl llog_catlist, llog_print,
llog_check, llog_remove, llog_cancel and llog_info.
Example:
The two lines below are equivalent.
$ lctl --device MGS llog_catlist
$ lctl llog_catlist
Tests 123xx of conf_sanity have been modified to mix between the old
and new syntax.
lctl-llog_* man page have been modified to match the new syntax.
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: If988dfc5d8171b2ca47bb68dd06f9d2953548cb2
Reviewed-on: https://review.whamcloud.com/38940
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Hongchao Zhang [Mon, 18 Nov 2019 02:56:12 +0000 (21:56 -0500)]
LU-12232 test: call dt_sync after dd
In test_6 of replay-ost-single, calling wait_mds_ost_sync
after "dd" to sync the write transactions.
Test-Parameters: trivial alwaysuploadlogs \
mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=replay-ost-single,replay-ost-single,replay-ost-single
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I26f6b1fe903c8b7b3f4e5fb698baffc4c3b85130
Reviewed-on: https://review.whamcloud.com/36772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Mon, 28 Sep 2020 06:55:14 +0000 (14:55 +0800)]
LU-8465 tests: enable pfsck if possible
To test pfsck widely, try enable pfsck by default
for Lustre tests.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I271422cf9d1f9cd0cc25c228c1f2df003e4f73f9
Reviewed-on: https://review.whamcloud.com/40070
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Thu, 13 Aug 2020 05:26:54 +0000 (15:26 +1000)]
LU-13017 tests: disable statahead_agl for sanity test_56ra
The sanity test_56ra can fail because statahead_agl can cause extra
glimpse request.
If a stat() systemcall is made after an AGL glimpse request is sent,
but before the reply has been received, the code handling the stat
cannot see that glimpse request and so will send another. This
elevates the number of requests counted.
There is a parameter (statahead_agl) which make it easy to disable the
AGL, but it isn't implemented properly. Specifically, inodes can
still be added to the sai_agls list when agl is disabled. They will
never be removed, which causes an assertion to fail.
To clean this up, remove the sai_agl_valid flag, and use a test on
sai_task being non-NULL instead. Also check agl_should_run() while
locked against ->sai_task changing, and before adding anything
to lli_agl_list.
We don't need the 'added' variable. It is perfectly OK to wake_up the
sai_agl_task *before* adding to the list as long is that is all done
under the lock. The task will wait for the lock before checking the
list, so it won't see it being empty.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I12c4e447104a86b3f48eaf57b6cf7ce4b41cc6de
Reviewed-on: https://review.whamcloud.com/39667
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Thu, 24 Sep 2020 12:57:47 +0000 (15:57 +0300)]
LU-13920 hsm: process hsm_actions only after mdd setup
There is no guarantee that MDD setup is finished
at the moment when coordinator is started by
config params processing. If MDD setup is not finished,
hsm actions llog is not inited(mdd_hsm_actions_llog_init).
Hence hsm_pending_restore will not be called, i.e.
RESTORE requests will be sent to agents without taken
layout locks. I believe it may cause different problems.
I faced at least a kernel panic when llog includes
RESTORE request in ARS_WAITING that hasn't bee sent to
agent before failover. And the 2nd one RESTORE request
to the same fid was resent after recovery. Finally
agent handles to RESTORE to the same FID in parallel
that resulted in a panic with following bt:
BUG: unable to handle kernel NULL pointer dereference at
0000000000000040
IP: [<
ffffffffc0b03bec>] thandle_get_sub_by_dt+0x14c/0x420 [ptlrpc]
...
[<
ffffffffc1202732>] lod_sub_get_thandle+0x2f2/0x400 [lod]
[<
ffffffffc1205021>] lod_sub_declare_xattr_set+0x61/0x300 [lod]
[<
ffffffffc11db0d5>] lod_obj_stripe_replace_parent_fid_cb+0x245/0x450 [lod]
[<
ffffffffc11eae0e>] lod_obj_for_each_stripe+0x11e/0x2d0 [lod]
[<
ffffffffc11ebfe2>] lod_replace_parent_fid+0x2a2/0x390 [lod]
[<
ffffffffc11dae90>] ? lod_attr_get+0x110/0x110 [lod]
[<
ffffffffc11f8faf>] lod_declare_xattr_set+0x24f/0xf70 [lod]
[<
ffffffffc077b251>] ? lprocfs_counter_sub+0xc1/0x130 [obdclass]
[<
ffffffffc1091ff4>] mdo_declare_xattr_set+0x74/0x2b0 [mdd]
[<
ffffffffc077b129>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[<
ffffffffc0fa2f7b>] ? osd_trans_create+0xbb/0x620 [osd_ldiskfs]
[<
ffffffffc1094903>] mdd_declare_xattr_set+0x33/0x70 [mdd]
[<
ffffffffc1094b4e>] mdd_object_pfid_replace+0x7e/0x1e0 [mdd]
[<
ffffffffc109c2c6>] mdd_swap_layouts+0xa76/0x1dc0 [mdd]
[<
ffffffffc10a5e1a>] ? mdd_trans_stop+0x3a/0x174 [mdd]
[<
ffffffffc114b489>] hsm_cdt_request_completed.isra.14+0xc89/0xf50 [mdt]
[<
ffffffffc077b129>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[<
ffffffffc114d844>] mdt_hsm_update_request_state+0x544/0x7b0 [mdt]
[<
ffffffffc0a82277>] ? lustre_msg_buf+0x17/0x60 [ptlrpc]
[<
ffffffffc10fec92>] ? ucred_set_audit_enabled.isra.15+0x22/0x60 [mdt]
[<
ffffffffc112b98f>] mdt_hsm_progress+0x1ef/0x3f0 [mdt]
[<
ffffffffc0aefcfa>] tgt_request_handle+0x96a/0x1640 [ptlrpc]
[<
ffffffffc06cca9e>] ? libcfs_nid2str_r+0xfe/0x130 [lnet]
[<
ffffffffc0a91466>] ptlrpc_server_handle_request+0x256/0xb10 [ptlrpc]
[<
ffffffffc0a95fbc>] ptlrpc_main+0xb3c/0x14d0 [ptlrpc]
[<
ffffffffc0a95480>] ? ptlrpc_register_service+0xf90/0xf90 [ptlrpc]
[<
ffffffffa04c1c31>] kthread+0xd1/0xe0
[<
ffffffffa04c1b60>] ? insert_kthread_work+0x40/0x40
[<
ffffffffa0b76c37>] ret_from_fork_nospec_begin+0x21/0x21
[<
ffffffffa04c1b60>] ? insert_kthread_work+0x40/0x40
Code: 74 29 4c 3b a0 50 ff ff ff 75 e4 4d 85 ed 74 1b bf 01 00 00 00 e8 c5 b8 ff ff 85 c0 0f 85 98 00 00 00 49 8b 45 00 e9 04 ff ff ff <49> 8b 44 24 40 48 8b 40 08 48 85 c0 0f 84 b3 02 00 00 4c 89 e6
RIP [<
ffffffffc0b03bec>] thandle_get_sub_by_dt+0x14c/0x420 [ptlrpc]
Note, I faced this panic while testing https://review.whamcloud.com/#/c/38867/,
however I believe the same issue may exist even without 38867.
Patch makes mdt_hsm_cdt_start to wait until MDT initialization becomes
finished. Without this fix you should see below error in dmesg
each time after MDS restart if HSM is enbaled.
mdt_hsm_cdt_start()) lustre-MDT0000: cannot take the layout locks needed for registered restore: -2
Test-Parameters: testlist=sanity-hsm
Change-Id: I4c4edaa72a562576ea71d89a4b60618d099ec4f5
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/40028
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Nathan Rutman <nrutman@gmail.com>
Reviewed-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Mon, 10 Aug 2020 22:45:44 +0000 (08:45 +1000)]
LU-13006 jobid: enhance tests to check per-session jobids.
Lustre allows per-session jobids by writing to "jobid_this_session".
The upstream-linux client does *not* support jobids using the process
environment.
Update the sanity tests to recognize this.
1/ Allow setting jobid_var=USER to fail - if it fails, don't
test use of envionment variables
2/ Check if "jobid_this_session tests - if it does, check
that it is used for generating jobids.
3/ Some tests use jobid_var to test general assignment
to config values. Change those to use jobid_name if
it is available.
Also if jobid_var is set to 'session' and jobid_name contains %j -
use jobid_name as is done when jobid_var is a variable name.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia7d01b8ffb9c6c910d2ce8f0615c802485604bf9
Reviewed-on: https://review.whamcloud.com/39656
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Sep 2020 01:16:06 +0000 (11:16 +1000)]
LU-13977 tests: fix float comparison in sanity test_255a
[ $a -gt $b ]
compares integers.
in ladvise_willread_performance(), average_ladvise and lowest_speedup
are calculated to 2 decimals, so they probably aren't integers. So
this (nearly) always fails, but as the failure is not reported when a
VM is detected, the failure goes unnoticed.
The bash [[ ]] command can be used instead. Its '>' operator compares
floats.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ifc674d5804b1269cebce82893a08eace9ffd9be4
Reviewed-on: https://review.whamcloud.com/40001
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Jian Yu [Thu, 17 Sep 2020 17:35:00 +0000 (10:35 -0700)]
LU-13919 kernel: kernel update RHEL7.8 [3.10.0-1127.19.1.el7]
Update RHEL7.8 kernel to 3.10.0-1127.19.1.el7.
The extent status tree shrinker patches are removed from the
RHEL 7.8 kernel patch series because the following bug fix has
been included since version 3.10.0-1127.18.2.el7:
ext4: change LRU to round-robin in extent status tree shrinker (BZ#1847343)
Test-Parameters: clientdistro=el7.8 serverdistro=el7.8
Change-Id: I8172e79de239681c540ab0644b63f8172790a027
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39964
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Sun, 30 Aug 2020 16:59:40 +0000 (00:59 +0800)]
LU-13511 obdclass: don't initialize obj for zero FID
Object with zero FID is used in stripe allocation, and it's
meaningless to initialize such object via lu_object_find_at(),
return error early to avoid assertion in lu_object_put().
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia1bda3d01ff7552e94f31a9c928868652937d559
Reviewed-on: https://review.whamcloud.com/39792
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 19 Oct 2018 07:31:41 +0000 (01:31 -0600)]
LU-11548 llite: increase readahead default values
It is commonly recommended to increase the readahead tunables
for clients to increase performance, since the current defaults
are too small, having been set several years ago for slower
networks and servers.
Increase the readahead defaults to better match values that are
recommended today:
- read_ahead_max_mb increased from 64MB to 1GB by default,
or 1/32 RAM, whichever is less
- read_ahead_per_file_max_mb is increased from 64MB to 256MB,
or 1/4 of read_ahead_max_mb, whichever is less
Modify the constant names to better match the variable and /proc
filenames.
Fix sanity test_101g to allow readahead to generate extra read
RPCs, as long as they are the expected size or larger.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iec864788fa1979c27adad42e613d1bf03f3ebbe5
Reviewed-on: https://review.whamcloud.com/33400
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Al Viro [Sun, 6 Sep 2020 21:38:17 +0000 (17:38 -0400)]
LU-13745 llite: switch generic_file_splice_read() to use of ->read_iter()
... and kill the ->splice_read() instances that can be switched to it
* Special Note * Using generic_file_splice_read() breaks Lustre
but in my testing on RHEL7 not provding a ->splice_read, which
results in default_file_splice_read() to be used seems to work.
Linux-commit:
82c156f853840645604acd7c2cebcb75ed1b6652
Change-Id: I28bd2fe43fec25e881c78b41c58c59ac5e74eb49
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Reviewed-on: https://review.whamcloud.com/39272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Fri, 25 Sep 2020 17:27:52 +0000 (13:27 -0400)]
New tag 2.13.56
Change-Id: Iea1ae97f08b59719917e57bc1b2a554a010f76c4
Lee Ochoa [Wed, 16 Sep 2020 21:02:33 +0000 (15:02 -0600)]
LU-13969 - Updates to lustre-release yaml.sh
Updated output of release() function to standarize node.yml file
os_distribution parameter. Changes as follows:
RHEL - use redhat-release first and os-release as backup as the latter may
not include the full version (major/minor)
CENTOS - use centos-release first and os-release as backup, same as RHEL
SUSE - use os-release instead of suse-release as the latter is deprecated
UBUNTU - use os-release
Removed parsing system-release and *-release as neither
option correctly outputs desired info
Removed "lustre_" references in node.yml file attributes, the default in Maloo
is to look for non-lustre prefixes first
Change-Id: Ia011f944aae53f31fcd3a539e846ea5aba7ec7c4
Signed-off-by: Lee Ochoa <lochoa@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39952
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Tue, 15 Sep 2020 21:35:08 +0000 (16:35 -0500)]
LU-13941 test: Restrict create_count to valid range
create_count range is restricted to:
[OST_MIN_PRECREATE=32, OST_MAX_PRECREATE=20000]
Enforce this restriction rather that halt with an error.
Test-Parameters: trivial testlist=parallel-scale
HPE-bug-id: LUS-5960
Fixes:
21501dedf64e ("LU-9780 tests: Testing Round-Robin allocation")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I125f2121a57d16a9031ed21938ce2f0b99b9dea8
Reviewed-on: https://review.whamcloud.com/39798
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Andrew Perepechko [Mon, 31 Aug 2020 08:49:59 +0000 (11:49 +0300)]
LU-13935 ofd: object removal is not handled properly
We should not check for object existence in
ofd_version_get_check() when we haven't got the lock yet.
ofd_attr_set() will check it later.
In ofd_destroy(), we should unlock if the object
does not exist, otherwise oti_w_locks assertion will fail.
Change-Id: I00f67d15c3268bcf55aafa88c088f2dbf55a470c
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
HPE-bug-id: LUS-9063,LUS-9282
Reviewed-on: https://review.whamcloud.com/39765
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Thu, 3 Oct 2019 15:42:18 +0000 (09:42 -0600)]
LU-1538 tests: standardize test script init – full group
Standardize the initial Lustre test script initialization for
clarity and consistency for test suites in the full test group.
The LUSTRE path is already normalized in init_test_env(), so this
doesn't need to be done in the caller. Use $(...) subshells instead
of `...` in the affected lines. Remove NAME, SRCDIR, PATH, MULTIOP,
SETUP, CLEANUP, CHECKSTAT, TMP, SAVE_PWD, variable initialization,
since it is already done in init_test_env() or not needed in the test
scripts.
Move all definitions of ALWAYS_EXCEPT and SLOW to after
init_test_env() and init_logging() and call build_test_filter()
immediately after the ALWAYS_EXCEPT and SLOW definitions.
Test-Parameters: trivial
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 austeroptions=-R iscsi=1 testlist=metadata-updates,performance-sanity,racer
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 austeroptions=-R testlist=obdfilter-survey,sanity-benchmark,sanity-lsnapshot
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I6e21a4a5d9e9215d5b452c4fd30467d9c007b5a5
Reviewed-on: https://review.whamcloud.com/36359
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Serguei Smirnov [Wed, 16 Sep 2020 16:46:48 +0000 (09:46 -0700)]
LU-12233 lnet: deadlock on LNet shutdown
Release ln_api_mutex during LNet shutdown while waiting
for zombie LNI to allow other threads to read the LNet
state updated by the shutdown and fall through, avoiding
the deadlock
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Iaba11624d5b79bd0acb4add39f6153c55770440a
Reviewed-on: https://review.whamcloud.com/39933
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Tue, 15 Sep 2020 18:42:41 +0000 (21:42 +0300)]
LU-13967 quota: change warning to cdebug
Changing CWARN to CDEBUG as it doesn't
point to any known problem.
Change-Id: Ia05a316a6ce1cc61899dd6662a9895946eee46aa
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/39921
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lai Siyao [Sat, 12 Sep 2020 02:23:00 +0000 (10:23 +0800)]
LU-11631 obdclass: nlink is not set in struct obdo
Traditionally the nlink field is not set in obdo_from_la() and vice
versa, because it's used for communication between MDT and OST, and
this field is not used, but when DNE is enabled, this function is
used to getattr between MDTs, which will always see zero nlink.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I2a8e6a7b3f98864ec11bca7f7c2070af90b64ade
Reviewed-on: https://review.whamcloud.com/39896
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
James Simmons [Fri, 11 Sep 2020 13:59:13 +0000 (09:59 -0400)]
LU-13740 ldiskfs: add Ubuntu 20.04 LTS support
The latest Ubuntu LTS is based on a 5.4 Linux kernel. We can use
the previous work of generic 5.4 ldiskfs support with a small
change to the ext-pdirop.patch to make ldiskfs available to
this latest Ubuntu LTS.
Change-Id: I6d1106c3da984b508ccc77364e13adc692bddb60
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/39779
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sergey Cheremencev [Mon, 24 Aug 2020 18:28:04 +0000 (21:28 +0300)]
LU-13840 utils: --pool instead of -o with lfs setquota
The use of "lfs setquota -o" as the short option for
specifying the quota pool is confusing, because "lfs quota -o"
is the short option for getting quota on an OST. Use only
long option --pool for lfs setquota.
Patch also changes -o with --pool for setquota in
sanity-quota.sh
Change-Id: I18a695dcbbcd668611b1301c48f4e10d354b2686
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/39715
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Mon, 25 Mar 2019 22:06:00 +0000 (01:06 +0300)]
LU-12397 osp: always set opd_new_connection
Flag opd_got_disconnected could be set back to 0
due to a race osp_precreate_thread vs osp_import_event.
Next ACTIVE event doesn't set opd_new_connection as
opd_got_disconnected also 0(i.e. import hasn't disconnected).
Such race is causing osp_precreate_thread to infinitly sleep
in wait despite сonnection state is FULL.
The patch always sets opd_new_connection flag on ACTIVE event
regardless value of opd_got_disconnected.
Patch is adding conf-sanity_101b test to race DISCON and ACTIVE
events. Without a fix the test causes to hung osp_precreate_thread
and as a result osp_precreate_reserve threads.
Change-Id: Iff41a2743f108679d5f70aca8e1c2108e979ac09
Cray-bug-id: LUS-7178
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://es-gerrit.dev.cray.com/154883
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-on: https://review.whamcloud.com/35078
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Wed, 3 Apr 2019 14:19:44 +0000 (10:19 -0400)]
LU-11518 osc: Do ELC on locks with no OSC object
Currently, osc_ldlm_weigh_ast weighs locks with no OSC
object in their ast data as "1", meaning the lock is not
considered for ELC.
This doesn't make much sense, since if there is no OSC
object, it's unlikely there's any data under the lock, so
it's actually a good candidate for ELC.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie832afbf2479f3a348e44d2c21992696830000ae
Reviewed-on: https://review.whamcloud.com/34584
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Sat, 29 Aug 2020 21:53:18 +0000 (05:53 +0800)]
LU-13922 osd-ldiskfs: no need to add OI cache in readdir
It's a waste of time to call osd_add_oi_cache() in osd_it_ea_rec(),
because each dirent read will override it.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Iec701bf66153fdf2ba7a3f3b89565381215abf33
Reviewed-on: https://review.whamcloud.com/39782
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Aug 2020 06:30:57 +0000 (16:30 +1000)]
LU-6142 lustre: don't take spinlock to read a 'long'.
Reading a 'long' (or unsigned long) is always an atomic operation.
There is never a need to take a spinlock to just read a single 'long'.
There are several procfs/debugfs/sysfs handlers which needlessly take
a spinlock for this purpose.
This patch:
- removes the taking of the spinlock
- changes the printf to scnprintf() as appropriate
- directly returns the value returned by scnprintf rather than
storing it in a variable
- accesses the 'long' as an arg to the scnprintf(), rather than
introducing a variabe to hold it.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: If4a6454b46844864e1177536a9c7b91e4c97de86
Reviewed-on: https://review.whamcloud.com/39743
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Vitaly Fertman [Tue, 4 Aug 2020 17:45:12 +0000 (20:45 +0300)]
LU-11518 ldlm: pool recalc forceful call
Let pool recalc to be able to be called forcefully independently of
the last recalc time;
Call the pool recalc forcefully on the lock decref instead of LRU
cancel to take into account the fresh SLV obtained from the server.
Call LRU recalc from after_reply if a significant SLV change occurs.
Add a sysfs attribute to control what 'a significant SLV change' is.
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Iffeb8d73effdfc494f412422f285921aa4eb9811
HPE-bug-id: LUS-8678
Reviewed-on: https://es-gerrit.dev.cray.com/157134
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Jenkins Build User <nssreleng@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/39564
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vitaly Fertman [Fri, 31 Jul 2020 18:30:09 +0000 (21:30 +0300)]
LU-11518 ldlm: pool fixes
At the time the client side recalc period was increased up to 10secs
the grant & cancel rates started showing the speed not in seconds but
in tens of seconds.
At the pool initialization time, the server side recalc job should not
be delayed on client's recalc period.
It may happen an NS time is significant and comparable (or even more)
than the recalc period of the next NS (all the following NS's) in the
list. If the time has been already spent on the next NS, it does not
mean we want to double the delay for the original NS and recalc after
next N secs.
Make lock volume factor more fine grained (default is 100 now vs the
original 1): it is likely to cancel locks on clients twice faster than
server requested is too fast.
Protect missed pl_server_lock_volume update by the pool lock.
Replace ktime_get_real_seconds with ktime_get_seconds for the recal
interval.
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Icba73209682a1b1d0d20c087581fad4f73ee3389
HPE-bug-id: LUS-8678
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/39563
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jinshan Xiong [Fri, 31 Jul 2020 18:22:40 +0000 (21:22 +0300)]
LU-11518 ldlm: control lru_size for extent lock
We register ELC for extent locks to be canceled at enqueue time,
but it can't make positive effect to locks that have dirty pages
under it. To keep the semantics of lru_size, the client should
check how many unused locks are cached after adding a lock into
lru list. If it has already exceeded the hard limit
(ns_max_unused), the client will initiate async lock cancellation
process in batch mode (ns->ns_cancel_batch).
To do it, re-use the new batching LRU cancel functionality.
Wherever unlimited LRU cancel is called (not ELC), try to cancel in
batched mode.
And a new field named new sysfs attribute named *lru_cancel_batch*
is introduced into ldlm namespace to control the batch count.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Signed-off-by: Vitaly Fertman <c17818@cray.com>
Change-Id: Ib18b829372da8599ba872b5ac5ab7421661f942d
Reviewed-on: https://es-gerrit.dev.cray.com/157068
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/39562
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Sat, 12 Sep 2020 18:04:02 +0000 (12:04 -0600)]
LU-13960 tests: correct usage of _var variable
In the setmodopts() function in functions.sh, the '_var'
variable is set and used. There is one use of the variable
'var' which should be '_var'. Change the use of 'var' to
'_var'.
Test-Parameters: trivial
Test-Parameters: testlist=conf-sanity env=ONLY=53 clientdistro=ubuntu2004 fstype=ldiskfs
Test-Parameters: testlist=conf-sanity env=ONLY=53 clientdistro=ubuntu1804 fstype=ldiskfs
Test-Parameters: testlist=conf-sanity env=ONLY=53 clientdistro=el7.8 fstype=ldiskfs
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: If524be1f0b4b2170a514a558256a5308c9a5e586
Reviewed-on: https://review.whamcloud.com/39891
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Vikentsi Lapa <vlapa@whamcloud.com>
Andreas Dilger [Fri, 11 Sep 2020 21:31:30 +0000 (15:31 -0600)]
LU-13019 tests: replace usleep in test scripts
The "usleep" command is specific to RHEL, and is not available on
Ubuntu clients. Instead, "sleep" can be used with a fractional
second value to get the same effect more portably.
The "usleep" values are increased to be more reasonable lengths, as
a 100us sleep doesn't make sense after do_facet or other remote call
that may take seconds to complete. Also, in VM testing there may
be unpredictable delays, so it is better to be conservative.
Test-Parameters: trivial testlist=sanity-hsm,replay-single,replay-dual
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic88c37d33ac17f46b9c1fb912112d318d9c3d07f
Reviewed-on: https://review.whamcloud.com/39889
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Elena Gryaznova [Thu, 3 Sep 2020 13:00:09 +0000 (16:00 +0300)]
LU-13944 tests: add ability to set clients parameters
Patch adds the ability to set clients parameters via
CLIENT_LCTL_SETPARAM_PARAM.
Example:
CLIENT_LCTL_SETPARAM_PARAM="lbug_on_eviction=1 debug=-1 "
sets lbug_on_eviction=1 and debug=-1 on all clients.
Test-Parameters: trivial
HPE-bug-id: LUS-9052
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Change-Id: Ifb06e7992c2f1db94f0a1503cfc31e00d4674804
Reviewed-on: https://review.whamcloud.com/39816
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 2 Sep 2020 14:05:10 +0000 (09:05 -0500)]
LU-13940 llite: it_lock_bits should be bit-wise tested
If lock_mode is not set then ensure that it_lock_bits has
the MDS_INODELOCK_OPEN bit set
Fixes:
5db0e2de00 ("LU-7403 llite: flatten struct lookup_intent")
HPE-bug-id: LUS-9198
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Icccaa59cd5b47a7d3f97569a452f34b33e905409
Reviewed-on: https://review.whamcloud.com/39797
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Alex Zhuravlev [Tue, 26 Nov 2019 13:24:55 +0000 (16:24 +0300)]
LU-13013 osd: do not count credits for mapped blocks
this should help to save credits if an application
overwrites using many tiny fragments.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ice6541672339f2ba9c4e3d91403a756be323baab
Reviewed-on: https://review.whamcloud.com/36866
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Tue, 11 Aug 2020 15:17:46 +0000 (23:17 +0800)]
LU-13900 clio: don't call aio_complete() in lustre upon errors
See following codes in aio_ret():
"
static inline ssize_t aio_ret(struct kiocb *req, ssize_t ret)
{
switch (ret) {
case -EIOCBQUEUED:
return ret;
case -ERESTARTSYS:
case -ERESTARTNOINTR:
case -ERESTARTNOHAND:
case -ERESTART_RESTARTBLOCK:
/*
* There's no easy way to restart the
* syscall since other AIO's may be already running.
* Just fail this IO with EINTR.
*/
ret = -EINTR;
/*FALLTHRU*/
default:
aio_complete(req, ret, 0);
return 0;
}
}
"
VFS will call aio_complete() if ret is not -EIOCBQUEUED,
this could happen when we don't pass user buffer as page
alignment or some other errors happen in Lustre.
So in Lustre, we need be careful to handle this case to avoid double
aio_complete() called.
Fixes: d1dde ("LU-4198 clio: AIO support for direct IO")
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I2137844437e91a4e269f7584c3a233e5fc81b0df
Reviewed-on: https://review.whamcloud.com/39636
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 15 Jul 2020 05:32:43 +0000 (15:32 +1000)]
LU-6142 lov: guard against class_exp2obd() returning NULL.
class_exp2obd() can return NULL. Sometimes this is tested for,
sometimes not.
Add appropriate tested so that a NULL return is never dereferenced.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib78514e8e709971ae6d08a11186f77d6c950b1a2
Reviewed-on: https://review.whamcloud.com/39379
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 15 Jul 2020 05:18:29 +0000 (15:18 +1000)]
LU-6142 lov: discard unused lov_dump_lmm* functions
lov_dump_lmm() and lov_dump_lmm_v3() are never used, so remove them.
So mark lov_lsm_pack_v1v3() and lov_lsm_pack_foreign() as static.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I013c265819e7e139a9878e9e0aa18022519bd2fe
Reviewed-on: https://review.whamcloud.com/39378
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 11 Aug 2020 13:36:41 +0000 (22:36 +0900)]
LU-12275 sec: verify dir is empty when setting enc policy
When setting an encryption policy on a directory that we want to
be encrypted, make sure it is empty.
Otherwise return -ENOTEMPTY from server to client doing the ioctl.
Also add test to exercise the ioctl interface to get/set encryption
policies on directories, thanks to the fscrypt userspace tool.
And test the ability to lock an encrypted directory.
Fixes:
40d91eafe257 ("LU-12275 sec: atomicity of encryption context getting/setting")
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: testlist=sanity-sec envdefinitions=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53 54" clientdistro=el8.1 fstype=zfs mdscount=2 mdtcount=4
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iabffd24800314b12908f505d7e66492be7292a16
Reviewed-on: https://review.whamcloud.com/39617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexey Lyashkov [Tue, 18 Aug 2020 13:24:04 +0000 (16:24 +0300)]
LU-13645 ldlm: don't use a locks without l_ast_data
Partially initialized lock (without l_ast_data to be init)
caused a fail with blocking ast, as discard from page cache
skipped, and stale data will read later with fast read.
Slow read have chance to attach this lock to right IO, but
it don’t true always, so that should disabled, until lock
will have l_ast_data set always for DoM and Lock Ahead locks.
HPE-bugid: LUS-8750
Change-Id: I2c5180c8044a12d7bd8f5f1c871447ca8b47a8ff
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/39318
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Mon, 10 Aug 2020 23:09:22 +0000 (16:09 -0700)]
LU-10360 mgc: Use IR for client->MDS/OST connections
When a target registers with the MGS, the MGS sends an IR log
to the client to speed up recovery.
The IR log contains the updated NID information on the target
which just registered.
This patch allows clients to update their imports with the latest
NIDs for the targets reported in the IR log. It also allows clients
to create new connections for targets which were not added via
the config log.
For example if a target reboots and comes up with a new NID, then
the client can continue using it.
This functionality is disabled by default and can be enabled by
setting a new file system specific module parameter, dynamic_nids.
lctl set_param mgc.*.dynamic_nids=1
This parameters will need to be set on the clients and the MGS
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I9e85438c39a46d4950ac50309c1bbae2a96d7d5d
Reviewed-on: https://review.whamcloud.com/39613
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 25 Aug 2020 14:11:07 +0000 (17:11 +0300)]
LU-13921 utils: new output format for ofd_access_log_reader
prefix every field with a tag to make it extensible:
o=l-OST0000 f=[0x1:0x1:0x0] t=
1598363594 b=0 e=4096 s=4096 s=1 n=1 r=w
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iab7d02852a1eb43d2259a4e914bafe87332749a6
Reviewed-on: https://review.whamcloud.com/39728
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mikhail Pershin [Wed, 15 Jul 2020 05:42:49 +0000 (08:42 +0300)]
LU-13763 osc: don't allow negative grants
Add check in the osc_init_grant() to prevent possible
underflow of cl_avail_grant and report error if it happens
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Idcd25ed427c23735e1cdc70359bace43b5b9d886
Reviewed-on: https://review.whamcloud.com/39827
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Elena Gryaznova [Thu, 3 Sep 2020 11:48:32 +0000 (14:48 +0300)]
LU-13943 tests: load modules for list_nids getting
If modules not loaded it does not make sence to call
host_nids_address() -> $LCTL list_nids
Test-Parameters: trivial testlist=lnet-selftest
HPE-bug-id: LUS-9265
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Change-Id: I9eb431cd293e8b4831d433ceda752b03fd78c581
Reviewed-on: https://review.whamcloud.com/39815
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 5 Aug 2020 03:14:27 +0000 (13:14 +1000)]
LU-13902 config: add test for /usr/include/libiberty/
The Makefile currently assumes that -I/usr/include/libiberty
is only needed on UBUNTU. But UBUNTU is only evaludate if
modules are being built (it is based on the base kernel found).
So with --disable-modules, -I/usr/include/liberty is never
selected.
There is no need for this indirection. It is easy to check if
/usr/inclue/libiberty exists, and to use if it if does.
So do that.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6fb482256df3c7ce537dd4d451a9da27b8d8d3ea
Reviewed-on: https://review.whamcloud.com/39749
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 26 Aug 2020 22:17:50 +0000 (16:17 -0600)]
LU-13909 tests: get lustre_inode_cache count from sysfs
If slab cache merging is enabled (which is the default for newer
kernels), then there may not be a dedicated 'lustre_inode_cache'
entry in /proc/slabinfo (it may be aliased with 'signal_cache' and
'ptlrpc_cache' and/or other slabs of the same size). This may result
in num_inodes() returning nothing to the caller, causing sanity
test_76() to not actually check anything useful.
Instead of looking for the number of cached inodes in /proc/slabinfo,
get this count from /sys/kernel/slab/lustre_inode_cache/objects if
available. While it may not hold the actual number of cached inodes
due to slab sharing, sanity test_76 only cares about the relative
increase in cached inodes before and after running the test.
Due to slab sharing, the margin for error in counting cached objects
in the "lustre_inode_cache" slab can be fairly large (it may also be
shared with 'ptlrpc_cache' and increase significantly when sending
many RPCs). The main goal of test_76 is to ensure there are not
persistently more objects cached in this slab compared to before
the create/unlink loop was run.
Test-Parameters: trivial testlist=sanity env=ONLY=76,ONLY_REPEAT=50
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I11066680b7add1ec1e0b4d15ff0ef7a3a666b484
Reviewed-on: https://review.whamcloud.com/39735
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Mon, 17 Aug 2020 16:36:37 +0000 (19:36 +0300)]
LU-10810 osd: implement lseek method in OSD
Introduce lseek support in dt_object methods and at OSD level
This is server support for SEEK_HOLE and SEEK_DATA distributed
request, it returns a resulting offset or error code
ZFS support is added as well but it is not yet available due to
missing export of dmu_offset_next() function.
Patch adds also osd_quasi_file() common method to create fake
file structure to use in inode i_fop calls
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I3802743e31c94d2de219b72a540c5df9c72f1897
Reviewed-on: https://review.whamcloud.com/39706
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 20 Aug 2020 09:45:21 +0000 (12:45 +0300)]
LU-13918 lfs: let lfs mirror extend to use prefer flag
so that lfs mirror extend -N --flags=prefer create new
mirror with prefer flag set and new IO go to the just
created mirror.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I2eb771fce3c14491eb193b053f3e97cc689b29c3
Reviewed-on: https://review.whamcloud.com/39698
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Fri, 7 Aug 2020 04:32:33 +0000 (14:32 +1000)]
LU-13904 tests: don't assume echo_client is a module.
The upstream lustre client can be compiled with all code
in the kernel - no modules.
So testing if the module is loaded gets incorrect results.
So add a test to see if the functionalty is already present,
and don't load a module if none is needed.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Id630107d57f473dfdc21687a4964e1ca4c83efe4
Reviewed-on: https://review.whamcloud.com/39655
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Fri, 3 Jul 2020 05:33:36 +0000 (15:33 +1000)]
LU-13783 osc: handle removal of NR_UNSTABLE_NFS
In Linux 5.8 the NR_UNSTABLE_NFS page counters are go. All pages that
have been writen but are not yet safe are now counted in NR_WRITEBACK.
So change osc_page to count in NR_WRITEBACK, but if NR_UNSTABLE_NFS
still exists in the kernel, use a #define to direct the updates to
that counter.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I49cbc267fafaee949f45b2e559511aedcf4d8fed
Reviewed-on: https://review.whamcloud.com/39260
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Fri, 4 Oct 2019 15:37:03 +0000 (09:37 -0600)]
LU-1538 tests: standardize test script init – misc tests
Standardize the initial Lustre test script initialization for
clarity and consistency test suites are that are not normally
run during patch and branch testing.
The LUSTRE path is already normalized in init_test_env(), so this
doesn't need to be done in the caller. Use $(...) subshells instead
of `...` in the affected lines. Remove NAME, SRCDIR, PATH, MULTIOP,
SETUP, CLEANUP, CHECKSTAT, TMP, SAVE_PWD, variable initialization,
since it is already done in init_test_env() or not needed in the test
scripts.
Remove MACHINEFILE from dom-performance because it is now
included init_test_env().
Move all definitions of ALWAYS_EXCEPT and SLOW to after
init_test_env() and init_logging() and call build_test_filter()
immediately after the ALWAYS_EXCEPT and SLOW definitions.
Test-Parameters: trivial envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 \
austeroptions=-R testlist=lfsck-performance
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 \
austeroptions=-R testlist=sgpdd-survey
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 \
austeroptions=-R clientselinux testlist=sanity-selinux
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I21280eb10de4d70ec3d7b4d6c8b8e976fb0821cc
Reviewed-on: https://review.whamcloud.com/36372
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
James Nunez [Wed, 2 Oct 2019 20:42:24 +0000 (14:42 -0600)]
LU-1538 tests: standardize test script init – parallel-scale
Standardize the initial Lustre test script initialization for
clarity and consistency for all parallel-scale tests.
The LUSTRE path is already normalized in init_test_env(), so this
doesn't need to be done in the caller. Use $(...) subshells instead
of `...` in the affected lines. Remove NAME, SRCDIR, PATH, MULTIOP,
SETUP, CLEANUP, CHECKSTAT, TMP, SAVE_PWD, variable initialization,
since it is already done in init_test_env() or not needed in the test
scripts.
Move all definitions of ALWAYS_EXCEPT and SLOW to after
init_test_env() and init_logging() and call build_test_filter()
immediately after the ALWAYS_EXCEPT and SLOW definitions.
Test-Parameters: trivial
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes clientdistro=el7.8 clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 austeroptions=-R testlist=parallel-scale-nfsv4
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes clientdistro=el7.8 clientcount=2 osscount=1 ostcount=7 mdscount=1 mdtcount=1 austeroptions=-R testlist=parallel-scale,parallel-scale-nfsv3
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I5690f17033ca92e5578c579e81cbcc024ca7f770
Reviewed-on: https://review.whamcloud.com/36355
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 19 Aug 2020 04:58:09 +0000 (14:58 +1000)]
LU-8066 osc: restore cur_dirty_grant_bytes
A previous patch which moved some attrs from /proc to /sys
inadvertently removed 'cur_dirty_grant_bytes'.
This patch restores it in /sys.
Test-Parameters: trivial
Fixes:
32fb31f3bf3d ("LU-8066 osc: move suitable values from procfs to sysfs")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic6cff067d3634d1b1406dad4eb156ad1967042c9
Reviewed-on: https://review.whamcloud.com/39740
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Mr NeilBrown [Tue, 25 Aug 2020 23:17:01 +0000 (09:17 +1000)]
LU-6142 lustre: all rhashtable_params should be static.
pools_hash_params and nid_hash_params are only used in the file where
they are declared - which is best for rhashtable params.
However they are not declared 'static', and one is even EXPORTed.
Change them both to 'static' and remove the EXPORT.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib71f8a7c3bfa38fa54b3653d51786a3d8da6117f
Reviewed-on: https://review.whamcloud.com/39730
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 15 Jul 2020 04:06:10 +0000 (14:06 +1000)]
LU-6142 lov: don't use inline for operations functions.
These functions have their address taken and stored in an 'operations'
structure. So they cannot possibly be compiled "inline".
So remove the "inline" declaration.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie6712048bd40a4518adaa510dd7da0202b394c1f
Reviewed-on: https://review.whamcloud.com/39376
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Wed, 8 Jul 2020 18:54:19 +0000 (14:54 -0400)]
LU-6142 libcfs: remove last use of container_of0
Remove the last use of container_of0 in osd_ldiskfs. Looking at
the code using container_of is safe to use since the handle
shouldn't be NULL or PTR_ERR.
Change-Id: Id018543997884825d1c3cee76de3a520fc3b7c18
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/39321
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Fri, 14 Aug 2020 09:55:06 +0000 (12:55 +0300)]
LU-13127 ptlrpc: avoid ISO C90 mixed declaration
Avoid forbidden mixed declaration in lustre_msg_calc_cksum()
Test-Parameters: trivial
Fixes:
726897c87c42 ("LU-13127 ptlrpc: don't require CONFIG_CRYPTO_CRC32")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I180eefd979331f3fa206aa9b1d54c8e7b17bcbee
Reviewed-on: https://review.whamcloud.com/39677
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr NeilBrown [Fri, 14 Aug 2020 04:02:20 +0000 (14:02 +1000)]
LU-13359 quota: call rhashtable_lookup near params decl
rhashtable_lookup() is an inline function which depends - for
performancs - on the 'rhashtable_params' being visible and
consnt. So it should only be called in the same file that
declared the params.
A recent patch make pools_hash_params an external variable and calls
rhashtable_lookup from a separate file, which will break the
optimisation.
So add lov_pool_find() and use it to maintainer optimization.
Fixes:
6b9f849fd5f4 ("LU-13359 quota: make used for pool correct")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ieeb3b080491f5b2c9c825885fe7a42f4a8599a2a
Reviewed-on: https://review.whamcloud.com/39676
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Wed, 8 Jul 2020 16:19:08 +0000 (16:19 +0000)]
LU-12275 sec: encryption with different client PAGE_SIZE
In order to properly handle encryption/decryption on clients that have
a PAGE_SIZE != LUSTRE_ENCRYPTION_UNIT_SIZE (typically aarch64/ppc64),
a few adjustements are necessary:
- when encrypting, do not proceed with PAGE_SIZE as encryption length.
Instead, round up to a multiple of LUSTRE_ENCRYPTION_UNIT_SIZE.
On aarch64/ppc64, it avoids encrypting way beyond
LUSTRE_ENCRYPTION_UNIT_SIZE when the page is not full.
- when decrypting, do not proceed with PAGE_SIZE as decryption length.
Instead, do LUSTRE_ENCRYPTION_UNIT_SIZE length at a time. It enables
proper detection of 'all 0s' sent by servers for content beyond file
size.
Regarding tests, add sanity-sec test_53 to exercise encryption from
clients with different PAGE_SIZE.
The trick to achieve this with AT is to expect the client to have 64KB
PAGE_SIZE, and the servers to have 4KB PAGE_SIZE, and then mount a
client from the MDS node.
This also means code running on server side needs to have client
encryption support enabled, so CentOS/RHEL 8 at least.
Test-Parameters: trivial
Test-Parameters: clientarch=aarch64 clientcount=1 clientdistro=el8.1 serverdistro=el8.1 testlist=sanity-sec env=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53" fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: clientarch=aarch64 clientcount=1 clientdistro=el8.1 serverdistro=el8.1 testlist=sanity-sec env=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53" fstype=zfs mdscount=2 mdtcount=4
Test-Parameters: clientarch=x86_64 clientcount=1 clientdistro=el8.1 serverdistro=el8.1 testlist=sanity-sec env=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53" fstype=ldiskfs mdscount=2 mdtcount=4
Test-Parameters: clientarch=x86_64 clientcount=1 clientdistro=el8.1 serverdistro=el8.1 testlist=sanity-sec env=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 52 53" fstype=zfs mdscount=2 mdtcount=4
Test-Parameters: clientdistro=el8.1 testlist=sanity-sec env=ONLY="36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52" mdscount=2 mdtcount=4
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iee4b4d9e70c2e8c8e12061c39400cf6a8c03bac3
Reviewed-on: https://review.whamcloud.com/39315
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Lai Siyao [Wed, 26 Aug 2020 14:47:14 +0000 (22:47 +0800)]
LU-12295 mdd: don't LBUG() if dir nlink is wrong
Sometimes dir nlink may not be correctly decreased: subdir is remote,
when it's unlinked, its dirent is removed, but parent nlink decrease
failed.
Don't assert this in osd_destroy(), but print an error message and
continue since we've checked directory is empty.
Add OBD_FAIL_OSD_REF_DEL to simulate the error above.
Add sanity 48f.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I483aaf7a62b7761868b5e2af8dbfa92929fda78c
Reviewed-on: https://review.whamcloud.com/39734
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Chris Horn [Wed, 27 May 2020 17:29:10 +0000 (12:29 -0500)]
LU-13605 lnet: Do not overwrite destination when routing
MR path selection in a routed environment is supposed to allow the
originator of a message to set the final destination NID. On a
multi-hop route, intermediate routers execute the same code path as
the message originator (i.e. the remote send cases). This causes
them to overwrite the destination NID when forwarding the message.
Check the msg_routing flag to determine whether we should set the
final destination NID (i.e. LNet peer NI).
A somewhat related issue is that because intermediate routers are not
selecting a destination lpni, they need to pick the next-hop lpni
based on the destination NID's remote net.
Test-Parameters: trivial
Fixes:
9dfdc2238be ("LU-13035 lnet: fix remote peer ni selection")
HPE-bug-id: LUS-8919
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Id2fbbc5d8da347e971bbb8ad2779e80f75e29dd7
Reviewed-on: https://review.whamcloud.com/38731
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Tue, 5 May 2020 21:13:18 +0000 (16:13 -0500)]
LU-13502 lnet: Add response tracking param to lnetctl
Add support for the lnet_response_tracking parameter to lnetctl.
Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-8827
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I952a415e27582a7a6d920bfeb16618766c0235da
Reviewed-on: https://review.whamcloud.com/38514
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Thu, 3 Sep 2020 23:34:34 +0000 (09:34 +1000)]
LU-13187 osd-ldiskfs: don't enforce max dir size limit on IAM objects
Add ext4-no-max-dir-size-limit-for-iam-objects.patch to introduce new
inode state EXT4_STATE_IAM and use it to mark IAM objects.
Test-Parameter: testlist=sanity env=ONLY=129,ONLY_REPEAT=100
Change-Id: I3bcc5435ea07edb9fa265dcd8e3261d849495f00
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/39823
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Mon, 15 Jun 2020 08:47:44 +0000 (11:47 +0300)]
LU-13827 utils: ofd_access_batch to print top hot files
with -F option one can specify fraction (in %%) of hot
files to be printed.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I131c47bb2252da88c9afce757c5264b8d2011d1a
Reviewed-on: https://review.whamcloud.com/39529
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Mon, 23 Mar 2020 15:11:46 +0000 (10:11 -0500)]
LU-13376 utils: add batching to ofd_access_log_reader
Add interval based batching to ofd_access_log_reader. Add option to
control the batch interval, offset within the interval, and batch
output file.
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I057e9616f4ec198dbf0c7c82a93a1e45907e7a42
Reviewed-on: https://review.whamcloud.com/38035
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
John L. Hammond [Thu, 3 Sep 2020 16:48:42 +0000 (11:48 -0500)]
LU-13945 utils: add includes for copy_file_range()
In lstddef.h include the needed headers to support the compat
definition of copy_file_range().
Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I83544cbbeb6407f4c3bb9fa3bd2a1297f2b2a2dc
Reviewed-on: https://review.whamcloud.com/39821
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Wed, 9 Sep 2020 00:29:06 +0000 (18:29 -0600)]
LU-12661 tests: skip sanity 817 for kernel 4.12+
Skip the NFS exec mode bug for kernels 4.12 and later, since
this is also being hit on SLES kernel 4.12.14+ and not just 4.14.
Test-Parameters: trivial
Fixes:
4fed33473ca2 ("LU-12661 tests: skip sanity 817 if kernel >= 4.14")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibc4ffda72bd7827e250c4583c760505b8f3ebbe5
Reviewed-on: https://review.whamcloud.com/39838
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Fri, 26 Jun 2020 17:19:05 +0000 (20:19 +0300)]
LU-13599 mdt: add test for rs_lock limit exceeding
The check conditions in mdt_link_parents_lock() considers there
can be total 6 local locks but in fact when object has hard links
then it is regular file and it may have maximum 5 local locks.
Patch updates that check conditions for rs_locks as small
improvement and ports test for rs_locks limit from 2.12.
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I98cca84825ce5789094fbceb5d1f7975410d134b
Reviewed-on: https://review.whamcloud.com/39194
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Yang Sheng [Thu, 20 Aug 2020 07:39:40 +0000 (15:39 +0800)]
LU-13915 ldiskfs: Avoid atomic operation while bitmap prefetch
It is expensive since test_and_set_bit is a atomic operation. So
use test_bit while bitmap prefetch to avoid call it frequently.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I2ff2c39f1dd3b351462ed66cbd3ebb36e6af4bea
Reviewed-on: https://review.whamcloud.com/39697
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Mon, 3 Aug 2020 18:21:58 +0000 (02:21 +0800)]
LU-13909 llite: prune invalid dentries
When file LOOKUP lock is canceled on client, mark its dentries
invalid, and also prune them to avoid OOM, to achieve this,
ll_invalidate_aliases() is renamed to ll_prune_aliases(), the latter
calls d_prune_aliases() to prune unused invalid dentries.
The same for negative dentries when parent UPDATE lock is canceled,
rename ll_invalidate_negative_children() to
ll_prune_negative_children().
Since now unused invalid dentries will always be pruned, it's not
necessary to call __d_drop() in d_lustre_invalidate().
It's redundant to take i_lock before d_lustre_invalidate() in
ll_inode_revalidate() because d_lustre_invalidate() takes d_lock,
remove it.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib0ae57537e31ba9269e042b94bc5fbe7cb263a50
Reviewed-on: https://review.whamcloud.com/39685
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Mr NeilBrown [Thu, 13 Aug 2020 00:26:40 +0000 (10:26 +1000)]
LU-4671 tests: give multiop a chance to exit.
If 'multiop' is still running after a test complete, test-framework.sh
reports a failure.
test_43A signals multiop asking it to exit as the last thing it does.
If there is any delay in multip being schedule, test-framework will
see it and report an error - this is a false negative.
So use 'wait' to wait for multiop to respond to the signal.
Test-Parameters: trivial testlist=sanity envdefinitions=ONLY="43A"
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic94d3c89bd98c9f6ef2d5bee22aac1e39116bf11
Reviewed-on: https://review.whamcloud.com/39665
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Chris Horn [Wed, 5 Aug 2020 15:17:14 +0000 (10:17 -0500)]
LU-13896 lnet: Fix reference leak in lnet_select_pathway
We call lnet_nid2peerni_locked() to lookup the peer NI for the message
originator. lnet_nid2peerni_locked() takes a reference on the peer NI
that is never dropped.
Test-Parameters: trivial
Fixes:
b0e8ab1a5f ("LU-13606 lnet: Allow router to forward to healthier NID")
HPE-bug-id: LUS-9185
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ie0e8f215d7becfbf33f905a1806da8513798ee8d
Reviewed-on: https://review.whamcloud.com/39603
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Tue, 21 Jul 2020 12:48:40 +0000 (15:48 +0300)]
LU-13810 tests: OST Pool Quotas with wide striping
All previous tests in sanity-quota except DOM,SEL
and PFL specific tests always use stripe count 1.
Check that block hard limit set to OST Pool Quotas
work properly with wide stripe files.
HPE-bug-id: LUS-8600
Test-Parameters: env=ONLY=1g testlist=sanity-quota
Change-Id: Ia6ebb21adb0fff18c6a9e0a36b1e50cb0f1d212a
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/39469
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log