Whamcloud - gitweb
Fan Yong [Wed, 29 Nov 2017 02:52:30 +0000 (10:52 +0800)]
LU-7585 scrub: general framework for OI scrub
Reconstruct OI scrub code to make some data structure and
functions can be shared between osd-ldiskfs and osd-ZFS.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4c4e3fcbd11078d10592136e9e55de66516a2b16
Reviewed-on: https://review.whamcloud.com/28607
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Amir Shehata [Tue, 28 Nov 2017 00:13:51 +0000 (16:13 -0800)]
LU-10129 lnd: set device capabilities
MLX-4, MLX-5 and OPA support different capabilities. Query the
device and cache the capabilities of the device for future use.
MLX5 can support fast registration and gaps
MLX4 and OPA only support FMR
Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I70d468f8af52d263139e7b51341bf4b5150b89c1
Reviewed-on: https://review.whamcloud.com/30309
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bruno Faccini [Fri, 12 May 2017 21:34:49 +0000 (23:34 +0200)]
LU-7340 mdd: changelogs garbage collection
When changelogs are almost full (few number of
free entries in catalog), try to recover some space
by unregistering users that are idle since too long,
based on new Changelog User record field (in fact,
using previous cur_padding unused field) to keep
track of last user's changelog cancel request time,
or based on gap between user index and current
ChangeLog record, for older registered user.
sanity/test_160[f,g] have been added to verify feature.
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I2100b101772e6d027675e5efa5606d4be24342a0
Reviewed-on: https://review.whamcloud.com/27103
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
John L. Hammond [Thu, 7 Dec 2017 16:11:26 +0000 (10:11 -0600)]
LU-10344 test: create files on local node in sanity-hsm
In sanity-hsm, rewrite create_file() to call dd ... directly rather
than do_facet $SINGLEAGT dd ... Rewrite create_archive_file() to call
do_facet $SINGLEAGT ... directly instead of calling
create_file(). Move the Lustre parent directory creation from
create_archive_file() to import_file().
Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I923475c32fa69b0ac59b95b90b42c147cf361274
Reviewed-on: https://review.whamcloud.com/30433
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Oleg Drokin [Tue, 12 Dec 2017 17:30:53 +0000 (12:30 -0500)]
New tag 2.10.56
Change-Id: I128e691f84fed31ea3b9332be6e254252c6de174
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Alex Zhuravlev [Wed, 29 Nov 2017 16:02:46 +0000 (19:02 +0300)]
LU-10296 obdclass: no errors on missing o_cleanup
it's legal to have o_cleanup undefined.
Change-Id: Ifd80dbd36fabcbfbd9a1609746d79a37e5a01023
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/30314
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Bobi Jam [Fri, 1 Dec 2017 01:56:47 +0000 (09:56 +0800)]
LU-10297 lod: prepare inuse array always
The OST inuse array should be prepared for composite files and for
plain files as well, since creating OST objects in both cases needs
to use it.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ibb37b680c68f8883650cdee6bebebc1c4d844623
Reviewed-on: https://review.whamcloud.com/30334
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Thu, 23 Nov 2017 19:11:36 +0000 (12:11 -0700)]
LU-930 man: better describe conf_parm -d in lctl.8
Add a bit more description to "lctl conf_param -d".
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ib7a71122ea3e77e910d817a0ecdc4e6d1ddfd2eb
Reviewed-on: https://review.whamcloud.com/30241
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Wed, 7 Jun 2017 23:54:56 +0000 (17:54 -0600)]
LU-4761 tests: clean up spurious messages and files
Reduce spurious messages that appear in the test and console logs
that are not useful for debugging, and just clutter things up.
Clean up a few files left over from tests.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ie17888a8f9c5864530dcb455a0cd4e51f83ebbe5
Reviewed-on: https://review.whamcloud.com/29012
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Yang Sheng [Mon, 27 Nov 2017 21:14:49 +0000 (16:14 -0500)]
LU-4134 obdclass: fix double free in failure path
We should just decref for obd if register is failed.
Since class_export_put will invoke class_free_dev,
then we may face a double free.
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ia8d1e487c69c4de1c7c247158cc8615aa6b6093a
Reviewed-on: https://review.whamcloud.com/29967
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Sebastien Buisson [Wed, 19 Jul 2017 14:44:27 +0000 (23:44 +0900)]
LU-9727 lustre: add uid/gid to Changelogs entries
Add a new changelog extension named changelog_ext_uidgid to hold
uid/gid information.
uid/gid info is added to every Changelog entry type except MARK, in
the form 'u=<uid>:<gid>':
5 01CREAT 15:44:32.
385864793 2017.07.18 0x0 t=[0x200000402:0x3:0x0]
ef=0x1 u=500:500 p=[0x200000402:0x2:0x0] file1
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ie09e4cd146dea75985de00b1da58f75c2a5928f2
Reviewed-on: https://review.whamcloud.com/28114
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Bruno Faccini [Tue, 28 Nov 2017 15:15:16 +0000 (10:15 -0500)]
LU-10224 obd: free obd_svc_stats when all users are gone
During object device shutdown obd_svc_stats must only be freed
after all access methods from user-land are no longer possible
to prevent any race and further crash.
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Iea4f5b0486779c1721c90f32538af1a723f76a79
Reviewed-on: https://review.whamcloud.com/30249
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Mon, 27 Nov 2017 00:53:49 +0000 (17:53 -0700)]
LU-5761 tests: re-add test_89 to replay-single ALWAYS_EXCEPT
Re-add replay-single test_89 to the ALWAYS_EXCEPT list, as it
is still failing intermittently.
Test-Parameters: trivial testlist=replay-single
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia98e28a2c4b863fd48cf0cd5d5ca9fe9cbafe9f3
Reviewed-on: https://review.whamcloud.com/30252
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Intel HPDD Gerrit Code Review [Fri, 1 Dec 2017 05:17:10 +0000 (05:17 +0000)]
Merge "LU-9771 flr: Merge branch 'flr'"
James Simmons [Wed, 22 Nov 2017 02:43:14 +0000 (21:43 -0500)]
LU-9897 utils: make liblnetconfig a hard requirment
With the recent changes to LNet lnetctl is now really a hard
requirment. This patch makes lustre require libyaml libraries
in order to build. In turn liblnetconfig and lnetctl are now
always built.
Test-Parameters: trivial
Change-Id: I26ff9397f3d5ba11da5ab4e76658ffd8c27ed035
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30204
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Thomas Stibor [Fri, 17 Nov 2017 14:31:39 +0000 (15:31 +0100)]
LU-10256 hsm: enable setting archive_id in hsm_set
Setting HSM flags in lfs_hsm_change_flags(...,LFS_HSM_SET)
does not allow to specify the archive_id, that is,
in llapi_hsm_state_set(path, mask, 0, 0 /* archive_id */)
archive_id = 0 is always set, which means no identifier change.
For having full flexibility (e.g. for debugging), introduce
the additional option --archive-id in hsm_set. If the option
is not provided, the default behavior (archive_id = 0) is used
and no archive identifier change is done. In addition a
test case is provided to check the functionality and shell
function get_hsm_archive_id() is modified in favor of more
robust grep for contents after pattern approach.
Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: I2145a18ecf32479527bb045140e5e881e58dd115
Reviewed-on: https://review.whamcloud.com/30150
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Tue, 14 Nov 2017 18:23:28 +0000 (12:23 -0600)]
LU-8497 osp: handle remote -ENOMEM from osp_send_update_req()
In osp_send_update_req() detect an unsent request by checking
rq_queued_time == 0 rather than rq_set == NULL, which is always true
after returning from ptlrpc_queue_wait().
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ief959b71600157a9c3521775cc06994326e50c51
Reviewed-on: https://review.whamcloud.com/30083
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Fri, 10 Nov 2017 16:08:44 +0000 (10:08 -0600)]
LU-8402 llite: simplify ll_inode_revalidate()
ll_inode_revalidate() is only called by ll_getattr() so move the
contents of ll_inode_revalidate() to ll_getattr() and rename
__ll_inode_revalidate() to ll_inode_revalidate().
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ieeba1b74551b7a531e1db47ccb0638c72c90adb6
Reviewed-on: https://review.whamcloud.com/30036
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Wed, 8 Nov 2017 16:12:02 +0000 (10:12 -0600)]
LU-8402 llite: assume OBD_CONNECT_ATTRFID
OBD_CONNECT_ATTRFID has been supported by MDSs since before 1.6 so add
it to the client side required flags and remove some code to handle
servers that do not support it.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I841ae21811c0903c10a84e659d2d1f8255c8109f
Reviewed-on: https://review.whamcloud.com/30010
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
John L. Hammond [Wed, 8 Nov 2017 15:44:51 +0000 (09:44 -0600)]
LU-8402 ldlm: assume OBD_CONNECT_IBITS
Clients and MDSs have supported and required OBD_CONNECT_IBITS since
before 1.6 so remove obsolete code to handle clients that do not
support this flag.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I24a623b063c518755b81d02fc48fa0f76aacd318
Reviewed-on: https://review.whamcloud.com/30009
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Lai Siyao [Fri, 27 Oct 2017 03:48:30 +0000 (11:48 +0800)]
LU-10165 llite: disable statahead if starting statahead fail
Once starting statahead thread fails, it should disable statahead.
Current code only does this when "sai != NULL", instead it should
check whether current process is opening this dir, so for cases
like current file is not the first dirent, or sai allocation fail,
it won't retry statahead.
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Iaedddd3659cdffeab51800f45b02f0b39c4a1ec1
Reviewed-on: https://review.whamcloud.com/29817
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Steve Guminski [Wed, 12 Jul 2017 14:46:33 +0000 (10:46 -0400)]
LU-5170 lfs: Standardize error messages in macros
Error messages in ARG2INT() and ARG2ULL() are updated to a standard
format. Messages are prefixed with the name of the utility and the
command that caused the error. User-provided values are delimited
with single quotes.
Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I31a0a30ac15681826e7e25b8a44d56174fb23e08
Reviewed-on: https://review.whamcloud.com/28250
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Matthew Sanderson [Fri, 14 Jul 2017 08:14:23 +0000 (18:14 +1000)]
LU-9727 lustre: Add an additional set of 64 changelog flags.
This adds a new changelog extension containing 64 additional flag
bits, to be used for future changelog extensions.
The presence of the extension is signalled using the last remaining
unused changelog flag bit.
The new extension is present in all changelog records by default, but
will be removed from records read by legacy changelog consumers.
Signed-off-by: Matthew Sanderson <matthew.sanderson@anu.edu.au>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iad05880e82400d7e84d927e84e44d1454e240a80
Reviewed-on: https://review.whamcloud.com/28045
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Alexander Boyko [Thu, 23 Nov 2017 09:51:02 +0000 (04:51 -0500)]
LU-10271 hsm: don't release with wrong size
The result of ll_merge_attr operation was ignored during
ll_hsm_release. For this case the released file could have
the wrong size on MDS, like
File: ‘released.file’
Size: 0 Blocks: 0 IO Block: 4194304 regular empty file
Patch adds test_253 sanity-hsm, to check hsm release
operation when cl_object_attr_get failed. It produces
the wrong size-on-mds for a released file.
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I94df1889265e5b6d03b16745de93e52af95d5b7c
Reviewed-on: https://review.whamcloud.com/30240
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Mikhal Pershin [Tue, 14 Nov 2017 19:40:21 +0000 (22:40 +0300)]
LU-10176 tests: limit DoM tests to new servers only
Run DoM tests only if server has support for DoM by checking
its version is 2.10.55 at least.
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I79f26cd3c80fbfc4bf48ddc3fcc762248d491034
Reviewed-on: https://review.whamcloud.com/30085
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
James Simmons [Mon, 20 Nov 2017 21:16:52 +0000 (16:16 -0500)]
LU-6142 uapi: Get rid of lustre_fid typedef
Replace it with struct lu_fid. Update the userland code and man
pages to reflect this change.
Linux-commit:
d8f6bc9a53f97d1ea4b2b955672904338643308b
Test-Parameters: trivial
Change-Id: I0b7e0770dd9da9bdac55c02c2ec98aea7cea7100
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/29849
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Sebastien Buisson [Thu, 9 Nov 2017 15:48:08 +0000 (00:48 +0900)]
LU-10220 mdd: fix buf alloc in mdd_changelog_data_store_by_fid
Fix allocation of mti_big_buf by call to lu_buf_check_and_alloc()
in mdd_changelog_data_store_by_fid().
reclen must take the header size of struct llog_changelog_rec into
account.
Maybe no memory corruptions were seen before because the buffer size
allocated in a previous call to mdd_declare_changelog_store() was
covering the need. But audit will add more information in changelog
records, provoking memory corruptions without this fix.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id0a06c412b54c0ae12c15d53f3e166e3e5d9ed68
Reviewed-on: https://review.whamcloud.com/30014
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Fan Yong [Fri, 24 Nov 2017 14:12:38 +0000 (22:12 +0800)]
LU-9887 tests: adjust lfsck speek test error range
To make sanity-lfsck test_9{a,b} to be more robust for kinds
of environment.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ie0740f2b205f319cbefe54c86734b889e6d03cc6
Reviewed-on: https://review.whamcloud.com/30247
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Nikolay Borisov [Fri, 17 Nov 2017 14:23:04 +0000 (16:23 +0200)]
LU-10255 tests: Fix wrong comparison operator in sanity 129
Currently sanity 129 uses numerica (-eq) operator to check whether
the has_warning variable is true or false depending on whether
expected warnings have been detected. This is wrong since bash
doesn't really have bolean types and true/false are not evaluated to
1/0. This results in the following bash error when the test is run:
./sanity.sh: line 9244: [: true: integer expression expected
Fix it by using the correct string operator '=' to check for
correct value.
Test-Paramters: clientdistro=sles12sp2 testlist=sanity
Test-Paramters: clientdistro=sles12sp3 testlist=sanity
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: I071dcb3001ea9b2e5806d346dcd48bfcaf3297e9
Reviewed-on: https://review.whamcloud.com/30149
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jim Casper [Thu, 16 Nov 2017 18:48:51 +0000 (13:48 -0500)]
LU-10229 test: Add sanity-sec test_27 to ALWAYS_EXCEPT
Received request to ALWAYS_EXCEPT test_27 since it has been
failing regularly on master. This will keep it from affecting
other patches.
Test-Parameters: trivial testlist=sanity-sec
Change-Id: I630b5cd1f978025e605ecad38cd5e276afdab694
Signed-off-by: Jim Casper <jamesx.casper@intel.com>
Reviewed-on: https://review.whamcloud.com/30135
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Bob Glossman [Tue, 14 Nov 2017 20:44:15 +0000 (12:44 -0800)]
LU-10241 kernel: kernel update RHEL6.9 [2.6.32-696.16.1.el6]
Update RHEL6.9 kernel to 2.6.32-696.16.1.el6
Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib2215b56e647fb8e19bc8b7e344a94564c7a8c7f
Reviewed-on: https://review.whamcloud.com/30130
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Alex Zhuravlev [Thu, 16 Nov 2017 12:43:32 +0000 (15:43 +0300)]
LU-9135 osp: do not fail if last_rcvd update failed
just ignore an error in this case and hope the next time
OST will be able to serve well and OSP cancel the record.
Change-Id: Ib1bbe77132e81322f353909620b6a413981ae342
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/30129
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Wed, 15 Nov 2017 16:05:57 +0000 (10:05 -0600)]
LU-10038 test: improve error reporting in sanity test_133g()
In sanity test_133g() remove the output redirection to /dev/null so
that errors from do_facet or find go to the test logs. Suppress error
messages in badarea_io.c.
Test-Parameters: trivial envdefinitions=ONLY=133 testlist=sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia51ccdacb995ff51a6467cbdd280c8965cfc5f47
Reviewed-on: https://review.whamcloud.com/30105
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nikolay Borisov [Thu, 9 Nov 2017 10:39:19 +0000 (12:39 +0200)]
LU-10217 test: Make sanity 248 work with newer dds
This patch fixes 2 problems when working with value from dd.
Firstly, extraction of the readings from dd rely on them always
being in field number 6. However, that's not true for newer dd
versions where the output has changed and the time reading is now in
field number 10. So to make the code more generic and work with both
old and new outputs switch to using an egrep regular expression to
extract the correct field.
Secondly, dd can output its time in scientific notation, with
decimal and floating point portion being separated by a comma rather
than a dot:
4096 bytes (4,1 kB, 4,0 KiB) copied, 9,1229e-05 s, 44,9 MB/s
Unforuntately this is not understood by bc so it needs to be
processed by substituting the comma for a dot and also expanding
the scientific notation to multiplication of 10 to the correct power.
Test-Parameters: trivial
Test-Parameters: clientdistro=sles12sp2 testlist=sanity
Test-Parameters: clientdistro=sles12sp3 testlist=sanity
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: I83ec659874ffe702c68f94f828827ff93325db72
Reviewed-on: https://review.whamcloud.com/30005
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nikolay Borisov [Tue, 7 Nov 2017 16:44:11 +0000 (18:44 +0200)]
LU-10204 tests: Fix line continuation
There is a simple missing in one of the error paths in
ladvise_willread_performance which results in an unknown command
error. Fix it by adding the missing backslash.
Test-Parameters: trivial
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: Ifff854c6cf5c1add71fd1571cb4de01d56204fde
Reviewed-on: https://review.whamcloud.com/29981
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Quentin Bouget [Thu, 8 Dec 2016 15:30:44 +0000 (16:30 +0100)]
LU-8950 tests: standardize file creation in sanity-hsm
sanity-hsm exposed several functions to create files:
- make_small()
- make_small_sync()
- make_archive()
This patch renames all these functions (respectively):
- create_small_file()
- create_small_sync_file()
- create_archive()
And adds a new one: create_empty_file() which creates an empty file
The actual file_creation is now abstracted in the create_file()
function used by all the aforementionned functions.
Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: Ib4267827eb2e31ed396da1d711419f45f9e0ef16
Reviewed-on: https://review.whamcloud.com/24704
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Mikhal Pershin [Mon, 13 Nov 2017 14:10:52 +0000 (17:10 +0300)]
LU-3285 mdt: remove assertion on mdt_pack_size2body()
The assertion on result of mdt_pack_size2body() is not
always correct, e.g. in case of resend.
Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: Iff2c9723959db591fdd29074b0186d8ddf0752e9
Reviewed-on: https://review.whamcloud.com/30058
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Mikhal Pershin [Mon, 13 Nov 2017 10:18:36 +0000 (13:18 +0300)]
LU-3285 mdt: get lu_object_attr for existent object
In mdt_punch_hdl() lu_object_attr() may be taken for
non-existent object with LBUG() as result.
Take lu_object_attr() after mdt_object_exists()check.
Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: If1181caa94b28cf1f251f57fa6f7102dc40c329d
Reviewed-on: https://review.whamcloud.com/30057
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Mikhail Pershin [Thu, 9 Nov 2017 17:35:43 +0000 (20:35 +0300)]
LU-3285 tests: disable kernel flusher in discard tests
Test 101 checks that client discards data instead of flush,
but it is not reliable because write RPC may happens before
unlink command is executed due to kernel flusher.
Set /proc/sys/vm/dirty_writeback_centisecs to 0 for the test
time and return it back to the default value later.
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I2c4da039a456519e25c6d9ffcc015c5baa479740
Reviewed-on: https://review.whamcloud.com/30016
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Nunez [Mon, 20 Nov 2017 04:22:38 +0000 (21:22 -0700)]
LU-9872 tests: modify check space requirements for NFS test
In parallel-scale-nfs testing, the Lustre clients NFS mounts
the Lustre file system. In some cases, the file system looks
like it has enough space (du) to run NFS testing, but one or
more OSTs is full or is almost full. Since the default striping
for Lustre is a single stripe, some NFS tests will fail because
the file it is writing gets written to that almost full OST.
Increase the default stripe to all OSTs for all
parallel-scale-nfs tests.
Test-Parameters: trivial testlist=parallel-scale,parallel-scale-nfsv3,parallel-scale-nfsv4
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Iff187dc6c158d9063cd36ced0801b88cdc8814ae
Reviewed-on: https://review.whamcloud.com/29786
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Casper <jamesx.casper@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jinshan Xiong [Tue, 28 Nov 2017 16:56:24 +0000 (16:56 +0000)]
LU-9771 flr: Merge branch 'flr'
Merge remote-tracking branch 'origin/flr'.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Idee9297fbcab2bea3bd5987c94e4b4e79c49b3b6
Bobi Jam [Fri, 24 Nov 2017 04:41:59 +0000 (12:41 +0800)]
LU-9771 flr: mirror resync multiple files
mirror resync multiple files in one command line.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I4e301070f678fbad2dd8663d52ded7294ad5dad5
Reviewed-on: https://review.whamcloud.com/30242
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Jinshan Xiong [Fri, 17 Nov 2017 07:11:10 +0000 (23:11 -0800)]
LU-9771 tests: make racer work for flr
Creating flr file in racer tests.
Test-parameters: trivial testlist=racer,racer,racer mdscount=1 mdtcount=1
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I56cef133263a05050ebdb0e3b16ccd0c737fe27f
Reviewed-on: https://review.whamcloud.com/30074
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Tue, 14 Nov 2017 00:12:25 +0000 (00:12 +0000)]
LU-9771 flr: support extending a plain file to mirrored file
Support extending a plain file to mirrored file by
'lfs mirror extend -N <a_plain_file>', so that <a_plain_file>
becomes a mirrored file, and it contins 2 mirrors each of them
has its old layout parameters covering [0, EOF).
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I415cd65effa72a4bde70b02063d8b635866947a8
Reviewed-on: https://review.whamcloud.com/29665
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 17 Nov 2017 07:06:24 +0000 (23:06 -0800)]
LU-9771 flr: to not support dom+flr for phase 1
To exclude the support of dom+flr.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I3b2ff9bf5dd77f2703d8292226b164a9789134b1
Reviewed-on: https://review.whamcloud.com/29609
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Sat, 14 Oct 2017 19:59:06 +0000 (12:59 -0700)]
LU-9771 flr: make sure init size work
Lustre allows to create a file with empty layout, and truncate it
up with non-zero file size. The size should be applied later after
a layout is attached.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie645f8172e98426a531eb2d9ffdcbec8f73f3731
Reviewed-on: https://review.whamcloud.com/29581
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Sat, 14 Oct 2017 07:55:23 +0000 (00:55 -0700)]
LU-9771 flr: inherit mirror options from directory
This patch checks and verifies directory inheritance from parent
works both for regular files and directories.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ief58805ca736acb9a53fd9ad8fb2bb460d46b748
Reviewed-on: https://review.whamcloud.com/29576
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Bobi Jam [Sat, 14 Oct 2017 07:49:27 +0000 (00:49 -0700)]
LU-9771 flr: lfs mirror resync command
This patch adds "lfs mirror resync" command to resynchronize a
mirrored file.
Usage:
lfs mirror resync [--only <mirror_id[,...]>] <mirrored file>
Options:
--only <mirror_id[,...]>
Indicates which mirror(s) specified by mirror_id(s) needs to
be resynchronized. The mirror_id is the numerical unique
identifier for a mirror. Multiple mirror_ids are separated
by comma.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic2aa474d1023a4364aa9d90c59120b1d5c89a269
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: https://review.whamcloud.com/29243
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Jian Yu [Sat, 14 Oct 2017 07:42:39 +0000 (00:42 -0700)]
LU-9771 flr: FLR+PFL test cases
* add some PFLR test cases.
* improve some lfs getstripe output.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Id27ad50456f1c51287df61fc93a2b18a3c6a6361
Reviewed-on: https://review.whamcloud.com/29101
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jian Yu [Fri, 15 Sep 2017 21:43:10 +0000 (21:43 +0000)]
LU-9771 flr: lfs mirror create and extend commands
This patch adds "lfs mirror create" and "lfs mirror extend"
commands to create or extend a mirrored file.
Usage:
lfs mirror create <--mirror-count|-N[mirror_count]>
[setstripe options|--parent]
... <filename|directory>
lfs mirror extend [--no-verify]
<--mirror-count|-N[mirror_count]>
[setstripe options|--parent|-f <victim_file>]
... <filename>
Options:
--mirror-count|-N[mirror_count]
Number of mirrors to be created with the upcoming setstripe
layout options. The option can be repeated multiple times to
separate mirrors that have different layouts. The mirror_count
argument is optional and defaults to 1 if it's not specified;
if specified, it must follow the option without a space.
setstripe options
The layout of one mirror. The options are the same as those for
lfs setstripe command. If setstripe options are not specified,
then the stripe options inherited from the previous component
will be used.
--parent
This option indicates that the default stripe options inherited
from parent directory will be used.
-f <victim_file>
The layout of victim_file will be split and used as a mirror
added to the mirrored file.
--no-verify
This option indicates not to verify the mirror(s) from victim
file(s) in case the victim file(s) contains the same data as
the original mirrored file.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8fb4a367bd169073f194ddee5c26d65b557e8201
Reviewed-on: https://review.whamcloud.com/29100
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Bobi Jam [Mon, 4 Sep 2017 09:28:58 +0000 (17:28 +0800)]
LU-9771 flr: instantiate component for truncate
layout intent for truncate IO needs stale components for non
primary mirror overlapping [trunc_size, eof), while needs to
instantiate components overlapping [0, trunc_size).
Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ibf31e3e1fd68d1a643f2ed3ac5b9e3c778563895
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: https://review.whamcloud.com/29099
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 21:29:58 +0000 (21:29 +0000)]
LU-9771 lod: support setting mirror layout from CLI
Parse and support mirror layout from CLI.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I51d24aecb63af04cb3c1dd1bcfbf5728030b233b
Reviewed-on: https://review.whamcloud.com/29098
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Sat, 26 Aug 2017 06:53:58 +0000 (23:53 -0700)]
LU-9771 flr: randomize mirror pick
Pick mirror randomly for read on client side and write on the MDT.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8dff91e6b7354a9841a2f1595df611717fe51afb
Reviewed-on: https://review.whamcloud.com/29097
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Wed, 8 Nov 2017 23:30:44 +0000 (23:30 +0000)]
LU-9771 osp: no delay commit callback for layout version
OSP batches commit callback by one second interval. However, this
mechanism would cause problem to FLR because the clients has to
wait up to one second before they are able to write OST objects.
This patch registers immediate commit callback if the SETATTR is
for layout version. It also cleans up the code a little bit.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I5fa0fea9a8c8b23379118609bee9a36f412a191d
Reviewed-on: https://review.whamcloud.com/30000
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 21:22:41 +0000 (21:22 +0000)]
LU-9771 flr: resync support and test tool
A tool to resync mirrored file after writing.
It extends the Lustre lease API to support taking file lease and then
sending the MDS_REINT_RESYNC RPC to the MDT so that it can increase
the file's layout version; then the client will start copying
the contents from valid mirror to stale mirrors. At the end of
resync, the copying client will release the lease and revalidate
stale mirrors.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I59f84cd501f6945225c97bdb99a142ae7efbf0fb
Reviewed-on: https://review.whamcloud.com/29096
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 21:13:10 +0000 (21:13 +0000)]
LU-9771 flr: mirror read and write
Support to perform I/O to designated mirror.
Three APIs have been exported for mirror IO:
1. llapi_mirror_read();
2. llapi_mirror_copy();
3. llapi_mirror_{set,clear}().
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iae1d1edea8e72b423f6f46821594a7d7791ff69b
Reviewed-on: https://review.whamcloud.com/29095
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Sebastien Buisson [Fri, 10 Nov 2017 16:16:47 +0000 (17:16 +0100)]
LU-10226 tests: properly check for fileset clear and error
Make sure fileset info on nodemap is cleared everywhere before
continuing test.
Also, properly exit on this error.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1d117ffef6c4a1002d3eba47a301e64830be4b4f
Reviewed-on: https://review.whamcloud.com/30037
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit
eae0f97355cf9a20066815b004716e4f148cec3d)
Reviewed-on: https://review.whamcloud.com/30218
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
John L. Hammond [Wed, 15 Nov 2017 16:05:57 +0000 (10:05 -0600)]
LU-10038 test: improve error reporting in sanity test_133g()
In sanity test_133g() remove the output redirection to /dev/null so
that errors from do_facet or find go to the test logs. Suppress error
messages in badarea_io.c.
Test-Parameters: trivial envdefinitions=ONLY=133 testlist=sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia51ccdacb995ff51a6467cbdd280c8965cfc5f47
Reviewed-on: https://review.whamcloud.com/30219
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Quentin Bouget [Wed, 15 Nov 2017 15:41:53 +0000 (15:41 +0000)]
LU-10234 tests: use wait correctly in sanity-hsm test_3[4-6]
Test 34, 35 and 36 did not properly check the return code of wait
and potentially missed regressions.
Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I21e8b16b55b946a8e5837549c3a3698379132e48
Reviewed-on: https://review.whamcloud.com/30102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Fri, 10 Nov 2017 17:00:58 +0000 (11:00 -0600)]
LU-10218 mdc: propagate changelog errors to readers
Store errors encountered by the changelog llog reader thread in the
crs_err field of struct changelog_reader_state so that they can be
propageted to changelog readers. In chlg_read() improve the error and
EOF reporting. Return -ERESTARTSYS when the blocked reader is
interrupted. Replace uses of l_wait_event() with
wait_event_interruptible().
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I279ec8ca11bfeb67f844acf12e3f7cba8f8aa1e6
Reviewed-on: https://review.whamcloud.com/30040
Tested-by: Jenkins
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Sebastien Buisson [Fri, 10 Nov 2017 16:16:47 +0000 (17:16 +0100)]
LU-10226 tests: properly check for fileset clear and error
Make sure fileset info on nodemap is cleared everywhere before
continuing test.
Also, properly exit on this error.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1d117ffef6c4a1002d3eba47a301e64830be4b4f
Reviewed-on: https://review.whamcloud.com/30037
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Ben Evans [Wed, 1 Nov 2017 17:02:27 +0000 (13:02 -0400)]
LU-10182 utils: change MDT name too long error
Change the error message to something more generic.
fsname-MDT00 failed with "Name too long", which was wrong
Test-parameters: trivial
Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: If7c7de080c75dfdc853ff6d06e98a8ed074279cb
Reviewed-on: https://review.whamcloud.com/29884
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bob Glossman [Fri, 27 Oct 2017 16:37:41 +0000 (09:37 -0700)]
LU-10168 kernel: kernel update [SLES12 SP2 4.4.90-92.45]
Update target and kernel_config files for new version
Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If24eddeeadc825b1b3073439304c0958c0ae1b74
Reviewed-on: https://review.whamcloud.com/29839
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Henri Doreau [Fri, 27 Oct 2017 07:31:58 +0000 (09:31 +0200)]
LU-10166 mdc: invalid free in changelog reader
Use kthread_stop() to instruct the producer thread to exit
when the device is closed, and only then: release the CRS
structure.
The previous implementation left small time windows open,
during which the producer threads could free a structure
before the consumer thread was completely done with it.
Change-Id: Id2038aa9b7fcfd2c3347f628e749f9d2c265ac6e
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-on: https://review.whamcloud.com/29818
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Wed, 25 Oct 2017 17:53:37 +0000 (13:53 -0400)]
LU-10161 lbuild: fix the problem of determine default_iface
The script of determining the default interface is broken.
Remove complexity and replace with hostname
Test-Parameters: trivial
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I0d662e9e24bed17078e31b6b5ca305cff2c0548e
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29776
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Patrick Farrell [Mon, 13 Nov 2017 16:02:21 +0000 (11:02 -0500)]
LU-10136 test: Lockahead test fixes
Two fixes:
1. Add version check for interop testing
2. Reduce count of locks requested in test15. At 5000
locks, we would sometimes get some locks reclaimed during
the test due to memory pressure. Reduce count to 500.
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I693f8543b3579819aa64101ef30453627637ead6
Reviewed-on: https://review.whamcloud.com/29684
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Quentin Bouget [Tue, 17 Oct 2017 19:14:41 +0000 (19:14 +0000)]
LU-9474 tests: add the stack_trap() utility function
This patch implements the stack_trap() function and uses it in
test_24d of sanity-hsm (where the need for it first arose).
The stack_trap() function is meant to be used to register multiple
cleanup actions on a single sigspec (most likely EXIT). Over time,
any call to bash's trap builtin in tests should be replaced by a call
to stack_trap().
Note that cleanup functions do not need to start with "trap - EXIT".
Each test is run in a subshell of its own (in run_one_logged()) and
therefore traps do not leak from one test to another. This is true
with or without this patch.
Test-Parameters: trivial testlist=sanity-hsm,sanity-hsm
Change-Id: I47f92a963f094d3d7055592a748e81ce357bff75
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-on: https://review.whamcloud.com/29653
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Qian Yingjin [Thu, 9 Nov 2017 20:24:58 +0000 (15:24 -0500)]
LU-10028 obdclass: Export Lustre mount flags and options
This patch help to export the information about Lustre special
mount flags and options via '/proc/mounts'.
The output for Lustre mounts by '/proc/mounts' is:
/dev/loop0 /mnt/lustre-mds1 lustre ro,svname=lustre-MDT0000,
mgs,osd=osd-ldiskfs,user_xattr,errors=remount-ro 0 0
/dev/loop1 /mnt/lustre-ost1 lustre ro,svname=lustre-OST0000,
mgsnode=192.168.59.129@tcp,osd=osd-ldiskfs,errors=remount-ro 0 0
/dev/loop2 /mnt/lustre-ost2 lustre ro,svname=lustre-OST0001,
mgsnode=192.168.59.129@tcp,osd=osd-ldiskfs,errors=remount-ro 0 0
192.168.59.129@tcp:/lustre /mnt/lustre lustre rw,flock,
user_xattr,lazystatfs 0 0
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I6cdbcaf4fd5a6a3621ed675b1c4485073e774d8c
Reviewed-on: https://review.whamcloud.com/29228
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Wang Shilong [Tue, 14 Nov 2017 02:12:33 +0000 (10:12 +0800)]
LU-10030 llite: extend inode flags into GET/SETXATTR ioctl
This is very useful for projet quota tools to save RPC call
to change project inherit attribute together with project ID.
Change-Id: I5b545bf2c5c601b552a85322c80994dabbca0b12
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/30072
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Wang Shilong [Fri, 22 Sep 2017 04:38:50 +0000 (12:38 +0800)]
LU-10011 utils: refactor lfs quota codes
'lfs quota' default output logic is a bit hard to
understand, refactor codes to make it eaiser to
read.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I9b2a5867ed0416ccb92856538ef029bc59ac2c4b
Reviewed-on: https://review.whamcloud.com/29165
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Lai Siyao [Fri, 22 Sep 2017 02:42:07 +0000 (10:42 +0800)]
LU-5024 mdc: don't assert on name pack
For old version of MDTs which don't support ATTRFID, open will pack
dentry name in the request, but this name may change if there is
concurrent rename, as a result an assertion will be triggerred.
It's inconvenient to copy name in advance or take dentry lock in the
call chain, so It's better to remove this assert, which is safe.
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I58d8b8fbbd8057aa2e8618e8923d0432f31dac96
Reviewed-on: https://review.whamcloud.com/29161
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Wang Shilong [Thu, 26 Oct 2017 07:03:52 +0000 (15:03 +0800)]
LU-9796 ldiskfs: improve inode allocation performance
Backport following upstream patches:
------
ext4: cleanup goto next group
avoid duplicated codes, also we need goto
next group in case we found reserved inode.
------
ext4: reduce lock contention in __ext4_new_inode
While running number of creating file threads concurrently,
we found heavy lock contention on group spinlock:
FUNC TOTAL_TIME(us) COUNT AVG(us)
ext4_create
1707443399 1440000 1185.72
_raw_spin_lock
1317641501 180899929 7.28
jbd2__journal_start
287821030 1453950 197.96
jbd2_journal_get_write_access
33441470 73077185 0.46
ext4_add_nondir
29435963 1440000 20.44
ext4_add_entry
26015166 1440049 18.07
ext4_dx_add_entry
25729337 1432814 17.96
ext4_mark_inode_dirty
12302433 5774407 2.13
most of cpu time blames to _raw_spin_lock, here is some testing
numbers with/without patch.
Test environment:
Server : SuperMicro Sever (2 x E5-2690 v3@2.60GHz, 128GB 2133MHz
DDR4 Memory, 8GbFC)
Storage : 2 x RAID1 (DDN SFA7700X, 4 x Toshiba PX02SMU020 200GB
Read Intensive SSD)
format command:
mkfs.ext4 -J size=4096
test command:
mpirun -np 48 mdtest -n 30000 -d /ext4/mdtest.out -F -C \
-r -i 1 -v -p 10 -u #first run to load inode
mpirun -np 48 mdtest -n 30000 -d /ext4/mdtest.out -F -C \
-r -i 3 -v -p 10 -u
Kernel version: 4.13.0-rc3
Test 1,440,000 files with 48 directories by 48 processes:
Without patch:
File Creation File removal
79,033 289,569 ops/per second
81,463 285,359
79,875 288,475
With patch:
File Creation File removal
810669 301694
812805 302711
813965 297670
Creation performance is improved more than 10X with large
journal size. The main problem here is we test bitmap
and do some check and journal operations which could be
slept, then we test and set with lock hold, this could
be racy, and make 'inode' steal by other process.
However, after first try, we could confirm handle has
been started and inode bitmap journaled too, then
we could find and set bit with lock hold directly, this
will mostly gurateee success with second try.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I234ff3027c8d96155d374c56b12aab7c4dc0dafd
Reviewed-on: https://review.whamcloud.com/29032
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Mon, 15 May 2017 21:56:18 +0000 (23:56 +0200)]
LU-9201 tests: don't check network for localhost
Don't try to check if the network to the local host is up, since
this just wastes time and adds noise to the test output logs.
Remove ancient 1.8 compatibility check.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I6c1f8df613afd657155da1309525e75b9d1c5bae
Reviewed-on: https://review.whamcloud.com/29011
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Nunez [Thu, 5 Oct 2017 19:51:50 +0000 (13:51 -0600)]
LU-9899 tests: mount client on MGS for tests with pools
When a Lustre file system has the MGS and MDS on separate
nodes, the file system must be mounted on the MGS to allow
OST pools to work properly.
There are several tests that deal with creating, adding
to and deleting OST pools. Create functions to mount and
unmount the file system on the MGS in test-framework and call
these functions when necessary for the Lustre test suites
conf-sanity test 82b, 103
sanity tests 27D, 200, 220, 406 and
replay-single test 85b.
Test-Parameters: combinedmdsmgs=false testlist=sanity,replay-single,conf-sanity
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I4b3e331b8d1ea6c3f8c9ea8a571e26f66f4535f8
Reviewed-on: https://review.whamcloud.com/28806
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Patrick Farrell [Mon, 21 Aug 2017 09:45:05 +0000 (04:45 -0500)]
LU-9894 lfsck: Print correct lfsck start info
When printing which scan type was started, lfsck iterates
through an array of types and compares the given type to
the types in the array. Type numbers are a bit mask.
However, LFSCK_TYPES_SUPPORTED and LFSCK_TYPES_DEF are the
full mask, and they are the first in the array. This means
we always match LFSCK_TYPES_SUPPORTED, which results in
always printing "all" as the type of scan.
Reorder the struct to increasing bit order to fix this.
Test-Parameters: trivial testlist=sanity-scrub,sanity-lfsck
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ibc82369ff6531b56ceda4b1c0a833720dc61f3f4
Reviewed-on: https://review.whamcloud.com/28604
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Simmons [Thu, 9 Nov 2017 00:50:48 +0000 (19:50 -0500)]
LU-8837 utils: make tools lightweight for lustre clients
Currently the lustre tools are built for the server target even
for the case of clients rpms. Rework the tools so server specific
code is not built for the client only case. The UAPI headers
lustre_log_user.h, lustre_barrier_user.h, and lustre_disk.h are
server specific and are not available in the upstream client.
So make our tools not to depend on those headers in the client
case. With this change it is simple step to make our tools
buildable against upstream. Last move user land specific code
from lustre_disk.h to mount_utils.h. Don't install server UAPI
headers on client targets.
Change-Id: I6b2b780ac0e7338a085ee6a8747097f0097efca1
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/28315
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Steve Guminski [Wed, 12 Jul 2017 16:43:22 +0000 (12:43 -0400)]
LU-5170 lfs: Standardize error messages in adjust_first_extent
Error and warning messages in adjust_first_extent() are updated
to a standard format. Messages are prefixed with the name of the
utility and the command that caused the error. User-provided values
are delimited with single quotes.
Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I9240fbce1f8fb065a17eaff1e4bbda70b4a5b0c9
Reviewed-on: https://review.whamcloud.com/28304
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Steve Guminski [Wed, 12 Jul 2017 16:07:44 +0000 (12:07 -0400)]
LU-5170 lfs: Standardize error messages in lfs_component_del()
Error messages in lfs_component_del() are updated to a standard
format. Messages are prefixed with the name of the utility and the
command that caused the error. User-provided values are delimited
with single quotes.
Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I02f8b25226efbbe8bd7a96bd2495a98b8d733183
Reviewed-on: https://review.whamcloud.com/28255
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Hongchao Zhang [Sat, 15 Apr 2017 21:36:19 +0000 (05:36 +0800)]
LU-9343 utils: update the usage of hsm_remove
Update the usage of hsm_remove in lfs to describe it more correctly.
Change-Id: I2934f0d7038f418dbad3a7e7365c7cd655ee28da
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/26882
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Di Wang [Wed, 25 Jan 2017 16:24:28 +0000 (11:24 -0500)]
LU-9052 lod: accept lfs mkdir from old client
MDS should also accept create striped directory request
from old client, but it will only honour stripe index in
this case, since the old client can only create remote
directory.
And also move the magic check from LOD to MDD to validate
LMV as early as possible.
Directory creation for echo client should go with lmv_user_md,
otherwise validation on user md will fail.
For striped directory, echo client should first choose stripe
before MD operation.
And also remove some unecessary lmv audit.
Re-enable 225b tests for DNE to test echo operations on
striped directory.
In mds-survey, if the current environment has more than one MDS,
let's skip the test. Because echo test will attach echo_client
individually on each MDSs(all of MDDs on the MDS), so each echo
client will not be able to access all of MDTs. But the test will
create a striped dir by MAX MDS count internally in mds_survey,
so echo client can not resolve the directory path in this case.
Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I74a3712edfa36fd178f92965f8e35f5ff77ad422
Reviewed-on: https://review.whamcloud.com/25091
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Ben Evans [Tue, 14 Nov 2017 15:28:12 +0000 (10:28 -0500)]
LU-10199 utils: strip // from fid2path
There are a few ways to get a path with a // back from
fid2path, make sure we strip them all out
Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: I30591585ccf62e4d793ee658d2772ce1db242034
Reviewed-on: https://review.whamcloud.com/30078
Tested-by: Jenkins
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
James Simmons [Thu, 9 Nov 2017 20:10:26 +0000 (15:10 -0500)]
LU-6142 lnet: remove last of the typedefs from LNet core
Change the remaining typedefs in LNet core to proper structures.
Several other style changes to fix checkpatch issues with code
impacted by typedef change.
Test-Parameters: trivial
Change-Id: Ic72b31698d1b1e7c612181a50aa7fa921fd417d7
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/29847
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Emoly Liu [Thu, 9 Nov 2017 08:23:48 +0000 (16:23 +0800)]
LU-10040 nodemap: add nodemap idmap correctly
There are 3 situations when adding an idmap {id_client:id_fs} to
a nodemap tree:
- both id_client and id_fs are matched, that means this idmap
already exists, so return -EEXIT;
- neither id_client nor id_fs is matched, that means this is a
new idmap, so insert it;
- only "id_client" or "id_fs" is matched, since idmap uses
"id_client" as index key, we need to delete that old idmap and
its index by its id_client, and then insert it again. In the
original implementation, idmap_insert() calls rb_replace_node()
to replace the old idmap without re-sorting its both id_client
and id_fs, so that this new added idmap can't be found by
idmap_search() due to its wrong left and right nodes, and can't
be deleted either.
Also, this patch improves the following code:
- nm_idmap_lock: use type rw_semaphore instead of rwlock, because
nodemap_idx_idmap_del() may sleep on FS operations while this
lock is being held.
- Add "update_idmaps" to sanity-sec.sh test_15 to verify this fix,
and improve other part of this test case as well to make it run
correctly.
Test-Parameters: testlist=sanity-sec
Signed-off-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Icf777f14c2e1dd56fa5cd0eb56666240e206d199
Reviewed-on: https://review.whamcloud.com/29364
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Simmons [Thu, 9 Nov 2017 00:44:19 +0000 (19:44 -0500)]
LU-9019 mdt: migrate to 64 bit time
Port the remaining time code from 32 bits to 64 bits.
Replace cfs_time_current_sec() to avoid the overflow
issues in 2038 with ktime_get_real_seconds().
Change-Id: Ia97dbfe00cd90dc6ef04583ba2eb512a8feca8c8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/28788
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Di Wang [Tue, 26 Sep 2017 05:33:50 +0000 (01:33 -0400)]
LU-9983 llog: skip non-exist log cancellation
Let's skip the record cancellation if the llog does
not exist at all, which usually happens when remote
log creation failed.
Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: If76d29d8fd72f3c5b619b72a899b18459c608dbb
Reviewed-on: https://review.whamcloud.com/29218
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
James Nunez [Thu, 3 Aug 2017 19:47:27 +0000 (13:47 -0600)]
LU-8998 test: Test 'lfs find' composite file options
The Progressive File Layout feature expanded the options
to find files using the 'lfs find' utility. Files can be
searched for with the component start, end, count and flags
options.
These 'lfs find' options need to be exercised.
Test-Parameters: trivial
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I36a6e774fa77b4089642ff5f2f8d482b3196aa62
Reviewed-on: https://review.whamcloud.com/26926
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Jinshan Xiong [Sat, 26 Aug 2017 19:49:38 +0000 (12:49 -0700)]
LU-9771 lov: skip empty pages in lov_io_submit()
empty page doesn't have corresponding bottem half so it should be
completed in the LOV layer.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Idb4576a8689bdf01638d66fddd123f05ffdd02cf
Reviewed-on: https://review.whamcloud.com/29094
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Jinshan Xiong [Sun, 30 Jul 2017 01:10:58 +0000 (18:10 -0700)]
LU-9771 flr: test script for mirrored file write
Add a test case to verify that writing to mirrored files work.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I98d97cc084554c769f2db5a3ef8a462d8db5dfce
Reviewed-on: https://review.whamcloud.com/29093
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Jinshan Xiong [Fri, 15 Sep 2017 20:49:02 +0000 (20:49 +0000)]
LU-9771 flr: extend DATA_VERSION API to read layout version
The API is extended to fetch layout version from ost objects.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iba852633992f0c52af8837cfed70922c1ed9ae00
Reviewed-on: https://review.whamcloud.com/29092
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Jinshan Xiong [Mon, 13 Nov 2017 23:34:14 +0000 (23:34 +0000)]
LU-9771 flr: Send write intent RPC to mdt
When a mirrored file is going to be written, the client needs
to send a write intent RPC to the MDT. The MDT will pick a mirror
as primary and mark the others as stale. The new md operation
moo_layout_change() is introduced for this purpose. The MDT also
transfers the latest layout version to the OST objects via
do_attr_set().
Once OSTs receive the setattr RPC for layout version change, it
will set the update layout version into extended attribute
XATTR_NAME_FID.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ib0049a78b95895141b0032e8eff526a73a160dcb
Reviewed-on: https://review.whamcloud.com/29091
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Jinshan Xiong [Fri, 29 Sep 2017 19:25:23 +0000 (19:25 +0000)]
LU-10050 lod: suspicious buffer reference in lod replay
In the replay path of lod_declare_layout_change(), it mistakenly uses
the address of buffer pointer where it should use the buffer pointer.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I82b77ad4b12bc7880f7376e13853baaa7dbfe341
Reviewed-on: https://review.whamcloud.com/29269
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 20:37:46 +0000 (20:37 +0000)]
LU-9771 mdt: revise layout_change() to take md_layout_change
This is a preparation patch for FLR write support.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I6323c260a636d31f683c035b45536374faa10746
Reviewed-on: https://review.whamcloud.com/29090
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Jinshan Xiong [Tue, 26 Sep 2017 00:28:57 +0000 (00:28 +0000)]
LU-9771 lod: define ldo_mirrors to manage mirrors
Right now it only supports loading mirror from disk
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Id560070e6147465681ede5a02e1add585281ea89
Reviewed-on: https://review.whamcloud.com/29089
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Sun, 6 Aug 2017 05:27:24 +0000 (22:27 -0700)]
LU-9771 ptlrpc: use lu_extent in layout_intent
This way it will be easier to check and print the extent
of layout.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie99142388310a8e321899c1795bc16bee97657be
Reviewed-on: https://review.whamcloud.com/29087
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Wed, 2 Aug 2017 22:48:32 +0000 (15:48 -0700)]
LU-9771 lov: rework write intent on componect instantiation
It's revised to do in the I/O initialization.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ib5edf6dc3d56c4b2068f434e644396c168e4bc5b
Reviewed-on: https://review.whamcloud.com/29086
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 20:32:06 +0000 (20:32 +0000)]
LU-9771 flr: read support for flr
Avoid stale mirrors to read;
If preferred mirror is inaccessible, try next one, ndelay RPC
is implemented to make the error-out quick. ndelay RPC has
rq_no_delay bit set that can be applied to brw and lock RPC.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I52143079edd1566ecb3a734ed88dab19f882c2fc
Reviewed-on: https://review.whamcloud.com/29085
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 14 Jul 2017 23:22:46 +0000 (16:22 -0700)]
LU-9771 clio: no glimpse for data immutable file
When merging a layout to an existing file as a mirror, client will
report the size and blocks to the MDT, which will be set to the MDT
object. In that case, if the MDT discovers a mirrored file in
READ_ONLY state, it will report the size and blocks. Clients should
take this advantage and skip glimpse.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I60831689116e6c5dd0311dea62b91dd0ae3cfd56
Reviewed-on: https://review.whamcloud.com/29084
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 20:01:58 +0000 (20:01 +0000)]
LU-9771 flr: lfs setstripe to create a new mirror
To create a mirrored file:
1. Create a component file
lfs setstripe -E -1 /mnt/lustre/tf
2. Add a synced mirror - create a mirror with specified layout
lfs setstripe --component-add [--mirror[=victim_file]]
[setstripe options] <file>
if victim_file exists, it will split the layout from that file and
use it as a mirror; otherwise, it will create a new mirror with
the stripe options in 'setstripe options'.
3. [todo] support to create a mirrored file directly by
lfs setstripe --mirror [setstripe options] --mirror [options] <file>
flag 'LCM_USER_FL_MIRROR' is reserved for this purpose.
Test-Parameters: testlist=sanity-flr
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I470feeb1a77554bd2c990e94d8538fd3d03d7b3b
Reviewed-on: https://review.whamcloud.com/29083
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Jinshan Xiong [Fri, 15 Sep 2017 19:53:50 +0000 (19:53 +0000)]
LU-9771 clio: introduce CIT_GLIMPSE for glimpse
It turned out that it's better to have a standaline I/O context
for glimpse request. In that case, it's easier for glimpse to try
a new mirror if the current one is out of access.
Another problem in this patch is to zero cl_io data structure if
the I/O starts from OSC layer.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ieeac36d4af38bc04a01affb9e6d3d129bceeb74e
Reviewed-on: https://review.whamcloud.com/29082
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Jinshan Xiong [Mon, 8 May 2017 20:22:13 +0000 (13:22 -0700)]
LU-9771 lov: pack lsm_flags from layout
Pack lsm_flags from compount layout.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I39cb0fe319c1665ef9d12ff704931c4583d730ff
Reviewed-on: https://review.whamcloud.com/29081
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
git://git.whamcloud.com / fs / lustre-release.git / log