Whamcloud - gitweb
fs/lustre-release.git
2 years agoLU-8119 osd: remove dcb entry from the list prior processing. 02/24202/6
Mikhal Pershin [Wed, 7 Dec 2016 16:23:03 +0000 (19:23 +0300)]
LU-8119 osd: remove dcb entry from the list prior processing.

The osd_trans_commit_cb() processes list of commit callbacks
but there is list_del_init() missed prior calling dcb_func().
The dcb_func() call frees dcb entry itself, so list head become
corrupted. Patch adds this list_del_init().

Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I561add170f408a7d16631424218b2a049d81f4fd
Reviewed-on: https://review.whamcloud.com/24202
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-9339 ldiskfs: Make ldiskfs buildable on kernels with no project quota 47/26647/3
Oleg Drokin [Sun, 16 Apr 2017 04:50:28 +0000 (00:50 -0400)]
LU-9339 ldiskfs: Make ldiskfs buildable on kernels with no project quota

Update ldiskfs patches to ensure buildability on unpatched rhel7.2/7.3
Also update lustre_user.h definition of LL_IOC_FSSETXATTR in the face
of missing define of FS_IOC_FSSETXATTR

Change-Id: I518e46e727086ed5e10b4820c4e69b2ea896fd43
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/26647
Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-9339 quota: Replace MAXQUOTAS usage with LL_MAXQUOTAS 48/26648/3
Oleg Drokin [Sun, 16 Apr 2017 06:29:34 +0000 (02:29 -0400)]
LU-9339 quota: Replace MAXQUOTAS usage with LL_MAXQUOTAS

This is especially important when kernel idea of
MAXQUOTAS is smaller than LL_MAXQUOTAS

Change-Id: I0fb8b15a0bab34993e9570efb8de548955962590
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/26648
Tested-by: Jenkins
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
2 years agoLU-9140 tests: disable sanityn test_77j and test_77k 15/26615/2
Andreas Dilger [Thu, 13 Apr 2017 22:57:56 +0000 (16:57 -0600)]
LU-9140 tests: disable sanityn test_77j and test_77k

These tests are failing regularly, disable until fixed.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I9aa88b0a564e6c74e021627bdc372d40df92c3f6
Reviewed-on: https://review.whamcloud.com/26615
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-4017 quota: add project quota support to lfs quota/setquota 55/23955/8
Wang Shilong [Thu, 30 Mar 2017 09:38:07 +0000 (17:38 +0800)]
LU-4017 quota: add project quota support to lfs quota/setquota

This patch adds project quota support to utils,
lfs quota/setquota now could be aware of project quota.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I274949293e098b94b9af4a1de7dd62c643993743
Reviewed-on: https://review.whamcloud.com/23955
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-4017 quota: enable project quota limits 54/23954/14
Li Xi [Wed, 2 Apr 2014 08:26:32 +0000 (16:26 +0800)]
LU-4017 quota: enable project quota limits

With this patch, lctl conf_param can be used to enable
or disable project quota limits

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I051c1395f909891f7e0ff60fa2f98195914cfa4b
Reviewed-on: https://review.whamcloud.com/23954
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-4017 quota: add setting/getting project id function 02/26202/13
Wang Shilong [Mon, 27 Mar 2017 05:43:30 +0000 (13:43 +0800)]
LU-4017 quota: add setting/getting project id function

Extend Attr RPC to support project ID attribute, new
ioctl is introduced to get/set project id, it is kept
same ioctl number as VFS, you could use:

lsattr -p <file>
chattr -p <file>

Redefine LL_MAXQUOTAS as 3, change
@qsd_acct_failed into qqi, otherwise osd-zfs
could not use USR/GRP quota normally.

Remove osd_quota_migration(), it only output
some messages in case upgrading from 1.8 which
give us extra efforts for project quota, remove
it indead.

also fix a typo bug in osp_attr_set().

Change-Id: I4b537be9efa29fda730592b6d5e41c6b60fb6c0d
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/26202
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-9278 quota: avoid crash when access failed quota proc file 77/26277/4
Wang Shilong [Fri, 31 Mar 2017 02:50:44 +0000 (10:50 +0800)]
LU-9278 quota: avoid crash when access failed quota proc file

If quota dose not have accounting objects because of failing to
load or unsupported type, @lqp_obj will be NULL, we could not access
directly, add proper handling to avoid following crash

[<ffffffffa0c60730>] ? lprocfs_quota_seq_show+0x460/0x460 [lquota]
[<ffffffff8126bd8f>] proc_reg_open+0x6f/0x100
[<ffffffff811fbfb7>] do_dentry_open+0x1a7/0x2e0
[<ffffffff812a8bfc>] ? security_inode_permission+0x1c/0x30
[<ffffffff8126bd20>] ? proc_reg_release+0x70/0x70
[<ffffffff811fc18f>] vfs_open+0x5f/0xe0
[<ffffffff81209b58>] ? may_open+0x68/0x110
[<ffffffff8120ccfd>] do_last+0x1ed/0x12a0
[<ffffffff8120de72>] path_openat+0xc2/0x490
[<ffffffff8118075b>] ? unlock_page+0x2b/0x30
[<ffffffff8120ffeb>] do_filp_open+0x4b/0xb0
[<ffffffff8121cc77>] ? __alloc_fd+0xa7/0x130
[<ffffffff811fd4f3>] do_sys_open+0xf3/0x1f0
[<ffffffff811fd60e>] SyS_open+0x1e/0x20
[<ffffffff81696d49>] system_call_fastpath+0x16/0x1b

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I8dfc2b35c1f38416ed0f933296bdaa455e117926
Reviewed-on: https://review.whamcloud.com/26277
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-4017 quota: add project id support 12/25812/10
Oleg Drokin [Sun, 9 Apr 2017 07:02:01 +0000 (03:02 -0400)]
LU-4017 quota: add project id support

This patch is infrastrcuture change for Lustre project quota,
project ID is considered as file attribute like UID/GID,
MDS will return back project id to client upon open/getattr.

Extend llog_setattr64_rec_v2 to be able to store project
ID, upgrade from old format could be handled properly,
if you want to downgrade again, need to make sure llog
have been handled completely.

Change-Id: Id7d27188f22169d6cffb74ea8b792ad4b133ef1d
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/25812
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
2 years agoLU-4017 ldiskfs: export __ext4_ioctl_setproject for lustre 45/23945/12
Wang Shilong [Wed, 19 Aug 2015 09:56:09 +0000 (05:56 -0400)]
LU-4017 ldiskfs: export __ext4_ioctl_setproject for lustre

Export this helper so that lustre can save project id to
MDS and OST objects

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I5c374e6dc7a509c0cbd46ac383b8d99f7f67d6b6
Reviewed-on: https://review.whamcloud.com/23945
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-4017 quota: add project quota support for Lustre 47/23947/12
Li Xi [Wed, 2 Apr 2014 07:26:59 +0000 (15:26 +0800)]
LU-4017 quota: add project quota support for Lustre

Thsi patch adds necessary support of project quota,
so that OSTs can be mounted successully

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I0c56b14b4ebb0b495497d2f05981a03218f686d7
Reviewed-on: https://review.whamcloud.com/23947
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-4017 quota: add project quota support to system header 46/23946/10
Li Xi [Thu, 3 Apr 2014 07:47:21 +0000 (15:47 +0800)]
LU-4017 quota: add project quota support to system header

System header contains the quota macros. Project quota support
should be added to it so that user space utils can be compiled
successfully

This patch redefines some macros in the system header file,
which avoids to change the sytem header file directly.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I1ed8014b1050684891df43d4c3db3c2e90b7814c
Reviewed-on: https://review.whamcloud.com/23946
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-4017 quota: cleanup codes of quota for new type 94/18894/17
Oleg Drokin [Sat, 8 Apr 2017 18:11:42 +0000 (14:11 -0400)]
LU-4017 quota: cleanup codes of quota for new type

This patch cleanups codes of quota to prepare for
new type other than USRQUOTA/GRPQUOTA.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I5b6fadf4eead957c8c94c91a4993de8798d87d3d
Reviewed-on: https://review.whamcloud.com/18894
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
2 years agoLU-4017 ldiskfs: add project quota support 70/17770/27
Wang Shilong [Thu, 31 Dec 2015 11:51:05 +0000 (19:51 +0800)]
LU-4017 ldiskfs: add project quota support

Pick following patchs from upstream:

commit 847aac644e92e5624f2c153bab409bf713d5ff9a
vfs: Add general support to enforce project quota limits
This patch adds support for a new quota type PRJQUOTA for
project quota enforcement. Also a new method get_projid()
is added into dquot_operations structure.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jan Kara <jack@suse.cz>
commit 334e580a6f97e2e84d1c19a8679603956acaa622
fs: XFS_IOC_FS[SG]SETXATTR to FS_IOC_FS[SG]ETXATTR promotion
Hoist the ioctl definitions for the XFS_IOC_FS[SG]SETXATTR
API from fs/xfs/libxfs/xfs_fs.h to include/uapi/linux/fs.h
so that the ioctls can be used by all filesystems,
not just XFS. This enables (initially) ext4 to use
the ioctl to set project IDs on inodes.

Based-on-patch-from: Li Xi <lixi@ddn.com>
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Above two chages are wrapped into vfs-project-quotas-rhel7.patch

commit 040cb3786d9b25293b8b0b05b90da0f871e1eb9b
ext4: adds project ID support

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Jan Kara <jack@suse.cz>
commit 689c958cbe6be4f211b40747951a3ba2c73b6715
ext4: add project quota support
This patch adds mount options for enabling/disabling project quota
accounting and enforcement. A new specific inode is also used for
project quota accounting.

[Includes fix from Dan Carpenter to correct error checking from dqget(). ]

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Jan Kara <jack@suse.cz>
commit 9b7365fc1c82038faa52d56173b20221cf422cbe
ext4: add FS_IOC_FSSETXATTR/FS_IOC_FSGETXATTR interface support
This patch adds FS_IOC_FSSETXATTR/FS_IOC_FSGETXATTR ioctl interface
support for ext4. The interface is kept consistent with
XFS_IOC_FSGETXATTR/XFS_IOC_FSGETXATTR.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Jan Kara <jack@suse.cz>
This patch add ldiskfs project quota support
Did following adjustment from original patches:

1.adds padding structure to superblock, so that @s_prj_num is
in the same position of superblock from upstream.

2.Patch aslo disable kabi check for rhel7, since
any of vfs patch's include files change will break the check

Also fix conf-sanity: test_55, because now debugfs output
Project ID, we should output file size.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I2392f402d9853af969e4cd37318a83f2c4941c3e
Reviewed-on: https://review.whamcloud.com/17770
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-8998 pfl: PFL feature implementation 23/26423/3
Jinshan Xiong [Sat, 8 Apr 2017 18:03:13 +0000 (14:03 -0400)]
LU-8998 pfl: PFL feature implementation

Merge remote-tracking branch 'origin/pfl'

Please check out the HLD of PFL here:
        http://wiki.lustre.org/PFL_Prototype_High_Level_Design

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ide007c7fffc17dc6a51500b537f19d2391387f9d

2 years agoLU-9300 test: need copytool setup for sanity-hsm 1b 34/26434/3
Bobi Jam [Fri, 7 Apr 2017 03:12:10 +0000 (11:12 +0800)]
LU-9300 test: need copytool setup for sanity-hsm 1b

sanity-hsm test_1b() needs setup/cleanup copytool in the test.

Test-Parameters: trivial testlist=sanity-hsm,sanity-pfl
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I40b083db37eb4e764e06606d0912ac635dc8c2a1
Reviewed-on: https://review.whamcloud.com/26434
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-7659 mdc: expose changelog through char devices 00/18900/15
Henri Doreau [Wed, 13 Jan 2016 14:53:00 +0000 (15:53 +0100)]
LU-7659 mdc: expose changelog through char devices

Register one character device per MDT in order to allow non-llapi to
read them and to make delivery more efficient.

- open() spawns a thread to prefetch records and enqueue them into a
  local buffer (unless the device is open in write-only mode).
- lseek() can be used to jump to a specific record, in which case the
  offset is a record number (with SEEK_SET) or a number of records to
  skip (SEEK_CUR). Movement can only be done forward.
- read() copies records to userland. No truncation happens, so short
  reads are likely.
- write() is used to transmit control commands to the device.
  The only available one is changelog_clear, which is done by writing
  "clear:cl<user>:<recno>" into the device.
- close() terminates the prefetch thread if any, and releases resources.

It is possible to poll() on the device to get notified when new records
are available for read.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I14709fdbac76b5512e58099e4e536cf9c973868c
Reviewed-on: https://review.whamcloud.com/18900
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-9165 pfl: MDS handling of write intent IT_LAYOUT RPC
Bobi Jam [Wed, 1 Mar 2017 09:25:49 +0000 (17:25 +0800)]
LU-9165 pfl: MDS handling of write intent IT_LAYOUT RPC

* Instantiate the OST objects of the 1st component of the PFL when its
  data are being created or its layout being set.

* Hence component_add won't instantiate OST objects, the instantiation
  is relayed to later write/truncate access.

* When the client tries to write beyond OST-object-instantiated extent
  range, it will issue an IT_LAYOUT intent lock RPC, indicating the
  write range, MDS needs to handle this RPC to allocate all OST
  objects of components covering the range.

* MDS also needs to handle the replay of write intent IT_LAYOUT RPC,
  which contains the current layout objects which the client knows, so
  that MDS won't initialize those objects again.

Reviewed-on: https://review.whamcloud.com/25717

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Idcb242af55788dd8d7f01a6ecec4fbe508375512
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-9008 pfl: dynamic layout modification with write/truncate
Bobi Jam [Thu, 6 Apr 2017 00:13:41 +0000 (08:13 +0800)]
LU-9008 pfl: dynamic layout modification with write/truncate

* in lov_init_composite(), skip init sub object without LCME_FL_INIT
  layout component.
* issue layout intent RPC during write/trunc ops when try to write to
  an un-init-ed component (even if at the lock stage).
* After layout intent RPC issued, restart the IO.
* get rid of unused lov_layout_operations::llo_install() interface.
* add an empty mdt_layout_change() interface to handle intent layout
  write RPC.

Reviewed-on: https://review.whamcloud.com/25317

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I2f79482187d2af2660dd86e55da3f5dc0138e94a
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 pfl: test cases for lfsck on PFL
Fan Yong [Thu, 6 Apr 2017 00:11:24 +0000 (08:11 +0800)]
LU-8998 pfl: test cases for lfsck on PFL

New test cases for check and repair inconsistent PFL
files, and related failures injections.

Reviewed-on: https://review.whamcloud.com/24884

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I102079f6a4bbdd4b1ca2739524e0e21ef5f79b98
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 docs: add llapi_ man pages to Makefile.am
Andreas Dilger [Thu, 6 Apr 2017 00:10:32 +0000 (08:10 +0800)]
LU-8998 docs: add llapi_ man pages to Makefile.am

Reformat lustre/doc/Makefile.am to have one entry per line to
match master, so the merge is more straight forward.

Add the llapi_layout_comp_*.3 and llapi_layout_file_comp_*.3 man
pages to lustre/doc/Makefile.am so they are included into the RPMs.

Replace references to the non-existent "liblustreapi.7" man page
with "lustreapi.7" instead, and add all llapi_* man pages to it.

Remove the long-obsolete llbackup.8 man page.

Reviewed-on: https://review.whamcloud.com/26275

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Change-Id: Ic97ea204f9635668fd5a4448687019f84f5cab07
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 docs: add lfs-setripe manpage to dist
Jinshan Xiong [Thu, 6 Apr 2017 00:09:18 +0000 (08:09 +0800)]
LU-8998 docs: add lfs-setripe manpage to dist

It's missing from commit a1e0fd6f.

Reviewed-on: https://review.whamcloud.com/26268

Test-Parameters: trivial
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8212cb1e48faaa6b19cd58d1ea6ab00a6f78644c
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-8998 pfl: layout LFSCK handles PFL file
Fan Yong [Thu, 6 Apr 2017 00:07:12 +0000 (08:07 +0800)]
LU-8998 pfl: layout LFSCK handles PFL file

Enhance the layout LFSCK logic to handle the PFL file,
including all the inconsistent cases corresponding to
the non-PFL file cases. In addition, for PFL file, the
LFSCK can also verify and recover the PFL extent range.

Reviewed-on: https://review.whamcloud.com/24883

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iec5ac57fd556330130e33f187c6fbf8cc24e67b8
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 pfl: enhance PFID EA for PFL
Fan Yong [Thu, 6 Apr 2017 00:05:05 +0000 (08:05 +0800)]
LU-8998 pfl: enhance PFID EA for PFL

This is a misc patch that contains some adjustments to
store more stripe information in the OST-object's PFID
EA (XATTR_NAME_FID), including:

1) For regular file, store the stripe count and stripe
   size information in its OST-object's PFID EA. These
   information are necessary for the LFSCK to recover
   the MDT-object layout EA from orphan OST-object(s).

2) In addition, for PFL file, store the FPL component
   ID and extent information in its OST-object's PFID
   EA also. These information are also necessary for
   the LFSCK to recover the MDT-object PFL layout EA
   from orphan OST-object(s).

3) For ldiskfs backend, there may be not enough space
   in inode body (256 bytes by default) to hold the
   enlarged PFID EA. To avoid performance trouble if
   store the enlarged PFID EA in separated block, we
   make some hack (for the device with 256 bytes or
   smaller inode) inside osd-ldiskfs via merging the
   PFID EA into the LMA EA (XATTR_NAME_LMA) to save
   EA entry space, then allows both the original LMA
   EA and the PFID EA to be filled in the inode body
   without performance trouble. For that, introduce
   two compat flags: LMAC_STRIPE and LMAC_PFL.

4) It is client duty to transfer the stripe and PFL
   information to the OST via the write, setattr and
   punch RPC. Then OST will store these information
   in the PFID EA.

5) On the other hand, to recover the lost or crashed
   MDT-object's layout EA from orphan OST-object(s),
   the LFSCK on the OST needs to return the orphan's
   stripe and PFL (if have) information to the MDT.
   Originally, we only returned the MDT-object's FID
   (and owner) information. Now, we need to return
   related layout information also. For such purpose,
   new structure 'lu_orphan_rec_v2' replaces the old
   'lu_orphan_rec', and the idx_info::ii_recsize is
   used to indicates whether new 'lu_orphan_rec_v2'
   or old ''lu_orphan_rec' is used.

Reviewed-on: https://review.whamcloud.com/24882

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5a69dbe91b1f83eeb394ce54453352d84a285193
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 docs: man pages for composite layout APIs
Niu Yawei [Tue, 23 Feb 2016 09:27:25 +0000 (04:27 -0500)]
LU-8998 docs: man pages for composite layout APIs

Added man pages for the new layout component interfaces:

llapi_layout_comp_add(3)
llapi_layout_comp_del(3)
llapi_layout_comp_extent_get(3)
llapi_layout_comp_extent_set(3)
llapi_layout_comp_flags_get(3)
llapi_layout_comp_flags_set(3)
llapi_layout_comp_flags_clear(3)
llapi_layout_comp_id_get(3)
llapi_layout_comp_move(3)
llapi_layout_comp_move_at(3)
llapi_layout_file_comp_add(3)
llapi_layout_file_comp_del(3)

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I8fd466b9cb0352bb7812f0b748816801f20f5e71
Reviewed-on: https://review.whamcloud.com/25740
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8054 lfsck: enhance lfsck notify API
Fan Yong [Thu, 6 Apr 2017 00:04:04 +0000 (08:04 +0800)]
LU-8054 lfsck: enhance lfsck notify API

Originally, the lfsck_in_notify() contains too much functionalities,
including both the remote LFSCK control and local LFSCK notification.
These LFSCK requirements/callers share the structure "lfsck_request"
to transfer kinds of data, as to it is very difficult to enhance the
"lfsck_request" for more local LFSCK requirements. Because enlarging
or adjusting the "lfsck_request" fields will affect the LFSCK RPC on
wire protocol and cause interoperability issues.

The patch splits the local LFSCK notifications from lfsck_in_notify(),
and introduce new LFSCK API lfsck_in_notify_local() for them. Then
even if we have more local LFSCK requirements in the future, we can
enhance such API freely without affecting LFSCK RPC protocol.

Reviewed-on: https://review.whamcloud.com/24881

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8f285c6680de66a73050e167dc3874a4f6bf4ad0
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-8050 osp: not use obdo::o_lcookie for non-onwire RPC data
Fan Yong [Fri, 22 Apr 2016 23:46:32 +0000 (07:46 +0800)]
LU-8050 osp: not use obdo::o_lcookie for non-onwire RPC data

Originally, obdo::o_lcookie is for destroy with unlink cookie from
MDS, it is obsolete in 2.8. And then, it is reused by the OSP to
avoid RAM allocations. But now, we needs more fields in the obdo
structure to transfer more information from client to OST. The
obdo::o_lcookie is good candidate. So this patch adjust OSP
logic to NOT use such field for non-onwire RPC data usage.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2bff0dce6f00f330657097a9ae9f775396e090cb
Reviewed-on: https://review.whamcloud.com/24876
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
2 years agoLU-8998 utils: enlarge default inode size
Niu Yawei [Tue, 10 May 2016 03:30:24 +0000 (23:30 -0400)]
LU-8998 utils: enlarge default inode size

Composite layout consumes more EA space, to hold a typical 3
components composite layout in inode, the default inode size
on MDT needs be bumped from 512 bytes to 1024 bytes. The default
OST inode size is changed from 256 bytes to 512 bytes as well,
because there will be more composite layout information stored
in OST object EA.

Performance tests shows that increased default inode size doesn't
impact performance, following are few numbers from the mdtest
result:

                  create  stat  removal  tree-create tree-remove
512/256 (origin): 16543  47292  32826    48          22
1024/512 (new)  : 16903  47404  32114    55          20

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ic5c08fd514f1111343f11129b1503a5ba2265ec7
Reviewed-on: https://review.whamcloud.com/24873
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
2 years agoLU-8998 tests: test scripts for PFL
Bobi Jam [Thu, 6 Apr 2017 00:01:31 +0000 (08:01 +0800)]
LU-8998 tests: test scripts for PFL

Added test scripts sanity-pfl.sh for PFL.

Reviewed-on: https://review.whamcloud.com/24872

Test-Parameters: testlist=sanity-pfl
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I2a09282d7ce26fc2b683eba994cfc63cd1ce166d
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 docs: man pages for tools of PFL
Niu Yawei [Wed, 24 Feb 2016 07:28:30 +0000 (02:28 -0500)]
LU-8998 docs: man pages for tools of PFL

Added new options to 'lfs find' and 'lfs getstripe' for composite
files. Moved "lfs setstripe/migrate" into separate man page, and
added new options for the operations to composite file.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9cafde834be5c3d890bf04a3efb2ea973364596a
Reviewed-on: https://review.whamcloud.com/25741
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 lfs: user space tools for PFL
Niu Yawei [Wed, 5 Apr 2017 23:59:49 +0000 (07:59 +0800)]
LU-8998 lfs: user space tools for PFL

* 'lfs getstripe' to support composite file:

   --component-count:
   Display component count only;

   --component-id|-I [comp_id]:
   When 'comp_id' is provided, display components matching the
   provided 'copm_id', otherwise, display component ID only;

   --component-flags [flags]:
   When the 'flags' is provided, dispaly components contains the
   provided 'flags', otherwise, display component flags only;

   --component-start [[+-]start]:
   When '[+-]start' is provided, display components which have
   'at most' or 'at least' or exactly same extent start, otherwise,
   display component extent start only;

   --component-end|-E [[+-]end]:
   When '[+-]end' is provided, display components which have
   'at most' or 'at least' or exactly sme extent end, otherwise,
   display component extent end only;

* 'lfs find' to support composite file:

   --component-count <[+-]count>:
   Find composite files according to component count;

   --component-flags <flags>:
   Find composite files according to component flags;

   --component-start <[+-]start>:
   Find composite files according to component extent start;

   --component-end|-E <[+-]end>:
   Find composite files according to component extent end;

* 'lfs setstripe' to support composite file:

   --component-del [-I <comp_id> | --component-flags <flags>] filename
   Delete specified component(s) from existing composite file;

   --component-add -E <end1> [stripe options]
                              -E <end2> [stripe options] ...:
   Add component(s) to an existing composite file;

   -E <end1> [stripe options] -E <end2> [stripe options] ...:
   Create composite file with specifed layout components;

* 'lfs migrate' to support migrate composite files:
   - composite layout-> plain layout;
   - plain layout -> composite layout;
   - composite layout -> composite layout;

Reviewed-on: https://review.whamcloud.com/24867

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib0d3eeb7927cf83a173099c474df6539d1264370
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 lustreapi: composite llapi_layout
Niu Yawei [Thu, 28 Jan 2016 07:04:02 +0000 (02:04 -0500)]
LU-8998 lustreapi: composite llapi_layout

Extend the llapi_layout API to support composite layout.

- llapi_layout_comp_add():
  Add one component to an existing layout, the layout will
  be turned into composite if it was plain before adding.

- llapi_layout_comp_del():
  Delete current layout component from a composite layout,
  the component to be deleted must be the tail of layout
  component list, and it can't be the last one.

- llapi_layout_comp_extent_get/set():
  Get/Set current component extent of a layout. When set
  component extent, the layout will be turned into composite
  if it was plain.

- llapi_layout_comp_flags_get/set/clear():
  Get/Set/Clear current component flags of a composite layout.

- llapi_layout_comp_id_get():
  Get current component ID of a composite layout.

- llapi_layout_comp_move_at():
  Move the current component to a component with specified ID.

- llapi_layout_comp_move():
  Move the current component to a specified position: FIRST,
  NEXT, or LAST.

- llapi_layout_file_comp_add():
  Add component(s) to an existing composite file.

- llapi_layout_file_comp_del():
  Delete component(s) from an existing composite file.

- llapi_layout_file_comp_set():
  Set certain attribute of the specified layout component(s). Not
  implemented in PFL phase 2.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I44094149636be3162fc7f8b75a90c64abf2c3c02
Reviewed-on: https://review.whamcloud.com/24866
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
2 years agoLU-8998 clio: getstripe support comp layout
Niu Yawei [Thu, 12 Jan 2017 14:48:19 +0000 (09:48 -0500)]
LU-8998 clio: getstripe support comp layout

{get/set}stripe support composite layout

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Iafe5205a79f721be1ec5ec83ab80a86a4b537e18
Reviewed-on: https://review.whamcloud.com/24851
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 clio: Client side implementation for PFL
Bobi Jam [Wed, 5 Apr 2017 23:58:41 +0000 (07:58 +0800)]
LU-8998 clio: Client side implementation for PFL

Make client layer support composite layout.

Plain layout will be stored in LOV layer as a composite layout
containing a single component.

Reviewed-on: https://review.whamcloud.com/24850

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ic3b85a4b10c66745e5c72ff02ea313baa0b12bb5
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
2 years agoLU-8998 lov: add composite layout unpacking
Bobi Jam [Wed, 5 Apr 2017 23:57:54 +0000 (07:57 +0800)]
LU-8998 lov: add composite layout unpacking

Update struct lov_stripe_md to accommodate composite layouts. Add
methods to unpack composite layouts.

Reviewed-on: https://review.whamcloud.com/24849

Change-Id: Ife0b2ae7671dd0ec0020a1da04e7261ef9f7e5f3
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-8998 lod: accomodate to composite layout
Bobi Jam [Wed, 5 Apr 2017 23:56:43 +0000 (07:56 +0800)]
LU-8998 lod: accomodate to composite layout

Modify the LOD to make it support the composite layout:

- Introduced lod_layout_component structure to represent each
  individual layout component, and added ldo_comp_entries and
  ldo_comp_cnt in the lod_object structure to hold the striping
  information for composite layout.

- Object allocation code is adjusted to not only check the used
  OSTs in this round of allocation, but also the used OSTs in
  the existing layout components.

- Support special xattr names to add/set/delete layout component(s).

- Store default layout xattr on the filesystem root inode.

Reviewed-on: https://review.whamcloud.com/24823

Change-Id: I73736df1cd529a3b0e32b2bb6a09b5c436e08c86
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
2 years agoLU-8998 pfl: Basic data structures for composite layout
Niu Yawei [Wed, 11 Jan 2017 02:56:38 +0000 (21:56 -0500)]
LU-8998 pfl: Basic data structures for composite layout

Added basic structures and magic numbers for composite layout.

Test-Parameters: trivial
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I02b4347d4959a935d4120d23f3c7b73511883a3f
Reviewed-on: https://review.whamcloud.com/24822
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-9281 test: stop using invalid ps option 98/26298/4
Bob Glossman [Sat, 1 Apr 2017 20:49:20 +0000 (13:49 -0700)]
LU-9281 test: stop using invalid ps option

Use of the '-q' option in a ps commend was introduced in sanity.sh
test_161d by the recent commit ee25babfe72378f9496a9732742984f26eb7d4a5.
Thst option isn't valid in all versions of ps.  It doesn't exist in any
sles version.

Use the '-p' option which exists everywhere instead.

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I143be0c1ad633889c9ed3ff7f5cb38bf54feb19a
Reviewed-on: https://review.whamcloud.com/26298
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-9245 utils: fix string copy handling within lgss_sk 65/26165/4
Chris Hanna [Thu, 23 Mar 2017 18:20:46 +0000 (14:20 -0400)]
LU-9245 utils: fix string copy handling within lgss_sk

Strings are not copied correctly when the fsname or nodemap field of
a shared key is replaced. Instead of being zero-terminated, the new
string may overlap the original if it is smaller.

Fixed by using the buffer length instead of the string length in
strncpy().

Change-Id: I17bb5aa0ca7c25b2545c17c6f23e69045730a547
Test-Parameters: trivial
Signed-off-by: Chris Hanna <hannac@iu.edu>
Reviewed-on: https://review.whamcloud.com/26165
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Kit Westneat <kit.westneat@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-9236 kernel: new kernel RHEL 6.9 [2.6.32-696.el6] 11/26111/2
Bob Glossman [Tue, 21 Mar 2017 17:51:29 +0000 (10:51 -0700)]
LU-9236 kernel: new kernel RHEL 6.9 [2.6.32-696.el6]

Changes to support new kernel version in new el6.9 release distro
New kernel-config and target files.
Revised base kernel and ldiskfs patches.
Small adjustments to lbuild and autoconf.

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 ossdistro=el6.9

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Idd341e4db98c3e6e91c9d8696a9a5e7f03ded6fe
Reviewed-on: https://review.whamcloud.com/26111
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-8066 obd: make version_show sysfs compliant 32/25632/6
James Simmons [Thu, 9 Mar 2017 18:35:31 +0000 (13:35 -0500)]
LU-8066 obd: make version_show sysfs compliant

A patch was pushed upstream that added "lustre: "
to the lustre version sysfs file. This was
rejected due to the one item per sysfs rule.
Recently a patch was landed to update the
lustre tools to handle "lustre: " missing. Since
this is the case we can bring the version
sysfs file into compliance.

Change-Id: I74568d581b176c802f132cef3c022fe0fdded164
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25632
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
2 years agoLU-8843 build: fix all remaining less than 0 comparison for unsigned values 84/25384/6
James Simmons [Tue, 28 Mar 2017 14:35:32 +0000 (10:35 -0400)]
LU-8843 build: fix all remaining less than 0 comparison for unsigned values

Remove the rest of the test of less than zero for unsigned values
found with -Wtype-limits.

Change-Id: I4ca764f359605a452cf4934e6a93106e91d2cc32
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25384
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-8236 nrs: Add wildcard support for JOBID TBF rule 23/24523/7
Qian Yingjin [Thu, 12 Jan 2017 02:50:10 +0000 (10:50 +0800)]
LU-8236 nrs: Add wildcard support for JOBID TBF rule

This patch adds wildcard support for JOBID TBF rule.
The following kind of wildcard matching rules can be
used:
start ruleName jobid={dd.*} rate=20
start ruleName jobid={*.500} rate=10
start ruleName jobid={prog*.10*} rate=15

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I0a4c44fd15533dfccd9f6ab7374a0a7a24b1403e
Reviewed-on: https://review.whamcloud.com/24523
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
2 years agoLU-8823 ptlrpc: remove obsolete OST/MDT thread parameters 05/23705/3
Andreas Dilger [Thu, 10 Nov 2016 19:11:47 +0000 (12:11 -0700)]
LU-8823 ptlrpc: remove obsolete OST/MDT thread parameters

The mdt_num_threads parameter was only briefly available from 2.0-2.2,
while mds_num_threads has been available since 1.x to control the
number of service threads on the MDS respectively.  A warning message
is printed on startup since 2.3 if mdt_num_threads parameters were in
use, so there shouldn't be any users anymore.

The ost_num_threads parameter has been deprecated since 1.6 or so.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I587a1f3e4629d304412006a0f0e9fc1daeacab07
Reviewed-on: https://review.whamcloud.com/23705
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-8152 utils: improve “lfs df” to show device status 30/23330/9
Jian Yu [Fri, 9 Dec 2016 06:39:37 +0000 (22:39 -0800)]
LU-8152 utils: improve “lfs df” to show device status

This patch improves “lfs df” to check OS_STATE_* flags,
and show the device status as follows:

D stands for OS_STATE_DEGRADED
R stands for OS_STATE_READONLY
S stands for OS_STATE_ENOSPC
I stands for OS_STATE_ENOINO

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ic7389d935f1258bc4217dfe36fcbf2b468b14f20
Reviewed-on: https://review.whamcloud.com/23330
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
2 years agoLU-8458 pacemaker: Script to monitor Server status 97/22297/7
Gabriele Paciucci [Mon, 8 Aug 2016 16:29:24 +0000 (17:29 +0100)]
LU-8458 pacemaker: Script to monitor Server status

A new script to be used in Pacemaker to monitor
the Lustre Servers status compatible with ZFS and
LDISKFS based Lustre server installations.

This RA is able to monitor a Lustre Server
using the Pacemaker's clone technology.

pcs resource create [Resource Name] ocf:lustre:healthLUSTRE
dampen=[seconds 5s]
--clone

where:
* dampen The time to wait (dampening) further changes occur

This script should be located in /usr/lib/ocf/resource.d/lustre/
of both the Lustre servers with permission 755.

Test-Parameters: trivial
Signed-off-by: Gabriele Paciucci <gabriele.paciucci@intel.com>
Change-Id: Ibfbad748e8c1b0c7faecc91984def87002070033
Reviewed-on: https://review.whamcloud.com/22297
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-7470 nrs: extend TBF with NID/JobID/OPCode expression 45/17345/15
Qian Yingjin [Mon, 13 Jun 2016 01:27:00 +0000 (09:27 +0800)]
LU-7470 nrs: extend TBF with NID/JobID/OPCode expression

Extend the NRS TBF policy to support complex rules with
expressions of NID/JobID/OPCode conditions.

Start the generic extended TBF policy:
    lctl set_param ost.OSS.ost_io.nrs_policies="tbf"

Following rule is valid:
    lctl set_param ost.OSS.ost_io.nrs_tbf_rule=
"start $NAME opcode={ost_write}&jobid={dd.0},
 nid={192.168.1.[1-128]@tcp 0@lo} rate=100"
In the rule:
"&" represents the conditional conjunction;
"," represents the conditional disjunction;

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I7541f9d09b2a952853eab97f0aa23f53e2c30a12
Reviewed-on: https://review.whamcloud.com/17345
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-1538 tests: improve sanityn test_51 error messages 56/5256/7
Andreas Dilger [Sat, 2 Feb 2013 09:02:40 +0000 (02:02 -0700)]
LU-1538 tests: improve sanityn test_51 error messages

While changing sanityn.sh it caused test_51[abc] to fail in an
obscure manner, and no error messages were being printed or
reporting test failures.

Improve the messages in these tests and fix some code style.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I798f6a7f5ab40714e867b537f4e32e59ea500c1e
Reviewed-on: https://review.whamcloud.com/5256
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoLU-8900 tests: add snapshot in racer tests 99/26199/3
Fan Yong [Sun, 13 Nov 2016 09:34:10 +0000 (17:34 +0800)]
LU-8900 tests: add snapshot in racer tests

To check whether snapshot may cause trouble or not.

Test-Parameters: trivial envdefinitions=SLOW=yes,RACER_ENABLE_SNAPSHOT=true mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4 testlist=racer
Test-Parameters: trivial envdefinitions=SLOW=yes,RACER_ENABLE_SNAPSHOT=false mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4 testlist=racer
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I09dababec7175c95f1a21519460e75e82ef7bc2f
Reviewed-on: https://review.whamcloud.com/26199
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
2 years agoNew tag 2.9.55 2.9.55 v2_9_55 v2_9_55_0
Oleg Drokin [Wed, 5 Apr 2017 05:59:28 +0000 (01:59 -0400)]
New tag 2.9.55

Change-Id: Ib04394e3a8c49dc5050ee12d8de5e559b031c34a
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9222 build: workaround for sles11sp4 MOFED 45/26045/2
Minh Diep [Thu, 16 Mar 2017 20:26:17 +0000 (13:26 -0700)]
LU-9222 build: workaround for sles11sp4 MOFED

MOFED assume that kernel-source is in the
standard place /usr/src but lbuild is using
different directory

Change-Id: I236659baf344cb3b3dcb62840c576c55febc1523
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/26045
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-2435 osd-zfs: use zfs native dnode accounting 94/15294/28
Jinshan Xiong [Fri, 17 Feb 2017 02:09:33 +0000 (18:09 -0800)]
LU-2435 osd-zfs: use zfs native dnode accounting

If ZFS userobj_accounting feature exists, use it for osd-zfs object
accounting; otherwise estimate object use by block space.

Disable quota check in conf-sanity:32b and sanity-quota:7e if the
underlying zfs version is below 0.7.0. For conf-sanity test, if the
running ZFS is version 0.7.0 or higher, upgrade the zpool image to
enable native dnode accounting. That ensures we also test the upgrade
procedure.

Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=sanity-quota,sanity-quota,sanity-quota,conf-sanity,conf-sanity envdefinitions=SLOW=yes
Change-Id: I0fe47fd0e3d787663a4c72fd708678827fdfb474
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: https://review.whamcloud.com/15294
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9224 fid: race between client_fid_fini and seq_client_flush 79/26079/4
Fan Yong [Tue, 8 Nov 2016 16:47:59 +0000 (00:47 +0800)]
LU-9224 fid: race between client_fid_fini and seq_client_flush

When the client mount failed or umount, the client_fid_fini() will
be called. At that time, the async connection failure will trigger
seq_client_flush() which parameter may have been released by the
client_fid_fini() by race.

Introduce client_obd::cl_seq_rwsem to protect client_obd::cl_seq.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I96b195676b0d1356a8ae7438cdf69d2e258abd23
Reviewed-on: https://review.whamcloud.com/26079
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9019 obd: use 64-bit time for obd_recovery_* fields 04/25604/7
James Simmons [Thu, 23 Mar 2017 13:56:52 +0000 (09:56 -0400)]
LU-9019 obd: use 64-bit time for obd_recovery_* fields

The obd_recovery_* fields stores absolute times in 32-bit
time_t quantities, which will overflow in 2038. This
changes it to use time64_t.

Change-Id: I59e41bc876eeeb563921abb764cafc7e40781aad
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25604
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9129 llite: ignore layout for ll_writepages() 74/25474/3
Jinshan Xiong [Wed, 15 Feb 2017 19:16:59 +0000 (11:16 -0800)]
LU-9129 llite: ignore layout for ll_writepages()

ll_writepages() would be called inside the direct IO context and
if the layout has been changed during this time, the layout_conf()
has to wait for active IO to complete before applying the layout
change, this is a case of deadlock.

It should ignore layout to avoid this problem. This is safe as long
as pages exist, the layout won't be changed on this client.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I457133a4a30215b8513d5eaec449fc129cc43a3e
Reviewed-on: https://review.whamcloud.com/25474
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9088 osd: use od_is_ost check OST FID 30/25330/2
wangdi [Thu, 15 Dec 2016 21:06:43 +0000 (16:06 -0500)]
LU-9088 osd: use od_is_ost check OST FID

Because OST FLDB is not created until 2.6, so
if the DNE system is upgraded from 2.5 to >=
2.6, then the local FLDB will not be loaded
in fld_index_init(), which might cause OSD
can not tell OST FID created by 2.5. So let's
use od_is_ost in this case.

Change-Id: I5da3e95bc24e43aca1b3c840df427cc5a7306194
Signed-off-by: wangdi <di.wang@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/25330
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-6401 uapi: split lustre_disk.h into two headers 94/25194/11
James Simmons [Thu, 23 Mar 2017 15:09:30 +0000 (11:09 -0400)]
LU-6401 uapi: split lustre_disk.h into two headers

The header lustre_disk.h is used by both user and
kernel space but it contains kernel specific material.
Split out the material that is used by both user land
and kernel space into a new header. Change the utilities
over to using this new UAPI header while the lustre
kernel code keeps using the kernel version of
lustre_disk.h.

Change-Id: I2cc75e4e465e17cc45bebb3b61e2e16d4bd615f0
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25194
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7131 utils: add "--erase-param" option to tunefs.lustre 59/16359/17
Nikitas Angelinas [Fri, 30 Oct 2015 04:08:00 +0000 (12:08 +0800)]
LU-7131 utils: add "--erase-param" option to tunefs.lustre

This patch is to fix the following issues:

- A new option "--erase-param" is added to tunefs.lustre, which allows
to remove all instances of a specific parameter.

- The behavior of option "--param" is improved a little. It will erase
all previously stored instances of the parameter, and then store all
new values specified in the command line.

- For zfs only, the patch introduces a "null value" rule in the form
of "<key>=" to mark a property to be removed from dataset by
zfs_write_ldd() finally.

- To keep the right semantics, if "--erase-params" is specified, all
old parameters should be erased before any new changes are done.

- test_89 conf-sanity.sh is added to verify these new features.

Seagate-bug-id: MRP-153
Signed-off-by: Nikitas Angelinas <nikitas.angelinas@seagate.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ibc8d81227045471b8e5c56c7faf1275ad8bbbf86
Tested-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@seagate.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-on: https://review.whamcloud.com/16359
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8900 doc: Lustre snapshot man page 70/24270/13
Fan Yong [Tue, 18 Oct 2016 00:36:57 +0000 (08:36 +0800)]
LU-8900 doc: Lustre snapshot man page

To describe how to use barrier and snapshot commands.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia1833ddd2949aaf5b63411214310c7792b41384d
Reviewed-on: https://review.whamcloud.com/24270
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8900 snapshot: user space snapshot tools 69/24269/19
Fan Yong [Sat, 12 Nov 2016 20:59:11 +0000 (04:59 +0800)]
LU-8900 snapshot: user space snapshot tools

The user space snapshot tools are implemented as part of the
lctl tools set. They use other user-space tools, such as lctl
commands for write barrier on all MDTs, fork/erase Lustre
configuration, zfs/zpool commands for backend zfs snapshot.

1. Create the snapshot
lctl snapshot_create [-b | --barrier [on | off]]
[-c | --comment comment] <-F | --fsname fsname> [-h | --help]
<-n | --name ssname> [-r | --rsh remote_shell]
[-t | --timeout timeout]

2. Destroy the snapshot
lctl snapshot_destroy [-f | --force] <-F | --fsname fsname>
[-h | --help] <-n | --name ssname> [-r | --rsh remote_shell]

3. Modify the snapshot
lctl snapshot_modify [-c | --comment comment]
<-F | --fsname fsname> [-h | --help] <-n | --name ssname>
[-N | --new new_ssname] [-r | --rsh remote_shell]

4. Query the snapshot(s)
lctl snapshot_list [-d | --detail] <-F | --fsname fsname>
[-h | --help] [-n | --name ssname] [-r | --rsh remote_shell]

5. Mount the snapshot
lctl snapshot_mount <-F | --fsname fsname> [-h | --help]
<-n | --name ssname> [-r | --rsh remote_shell]

6. Umount the snapshot
lctl snapshot_umount <-F | --fsname fsname> [-h | --help]
<-n | --name ssname> [-r | --rsh remote_shell]

The system config information, such as each target's hostname,
pool name, local filesystem name, role, index, and so on, will
be written in the snapshot config file "/etc/ldev.conf". To be
compatible with old snapshot usage, it will firstly check whether
the "/etc/ldev.conf" exists or not, it yes, it will load the
configuration from such file preferentially, otherwise, the file
"/etc/lsnapshot/${fsname}.conf" will be used.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4 testlist=sanity-lsnapshot
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I1191c2ea35c355e8554bd2bd022c1b9d32741392
Reviewed-on: https://review.whamcloud.com/24269
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9231 tests: cleanup obdecho before iokit exit 92/26092/2
Fan Yong [Mon, 14 Nov 2016 16:34:53 +0000 (00:34 +0800)]
LU-9231 tests: cleanup obdecho before iokit exit

Otherwise under some failure cases, it may leave the
FID_SEQ_LOCAL_NAME local object which is used by the
obdecho in the system, then the subsequent tests can
NOT exit cleanly.

Lustre-commit: c5675497b411d8b3f0f08fec3271fd528b4a52cf
Lustre-change: http://review.whamcloud.com/23738

Test-Parameters: alwaysuploadlogs envdefinitions=PTLDEBUG=-1,DEBUG_SIZE=150 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs clientdistro=el7 ossdistro=el7 mdsdistro=el7 mdscount=2 mdtcount=4 testlist=mds-survey,mmp,mds-survey,mmp,mds-survey,mmp
Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I741cf20875df679ecdf72d5461cc14eec166e3ae
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-on: https://review.whamcloud.com/26092
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9223: Server on Linux 4.4: XATTR_NAME_POSIX_ACL_ACCESS 76/26076/2
Keith [Fri, 17 Mar 2017 19:11:16 +0000 (15:11 -0400)]
LU-9223: Server on Linux 4.4: XATTR_NAME_POSIX_ACL_ACCESS

Add linux/xattr.h to Lustre_compat.h to allow the server
code to Build in Linux 4.4.

Change-Id: Iccf87acffc04d7c1084a384ae146d1077eed53fd
Signed-off-by: Keith <keith.mannthey@intel.com>
Reviewed-on: https://review.whamcloud.com/26076
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9197 llog: return 0 if llog destroy fails 69/26069/2
Di Wang [Wed, 8 Mar 2017 14:04:42 +0000 (09:04 -0500)]
LU-9197 llog: return 0 if llog destroy fails

In llog_cancel_rec(), let's return 0 if only destroying the
final plain llog fails, because the bitmap has been cleared
successfully, and the orphan supposed to be handled by LFSCK
anyway. So let's not try to destroy again, which will pretty
much complicate the code here.

And also in this case, we do not need restore the llh_count and
bitmap, because the record has been deleted for llog anyway.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I762def0f0de08a2960997ab9463f4e72fbad531d
Reviewed-on: https://review.whamcloud.com/26069
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9213 scripts: check MGT status in lustre init script 95/25995/3
Jian Yu [Wed, 15 Mar 2017 06:44:13 +0000 (23:44 -0700)]
LU-9213 scripts: check MGT status in lustre init script

This patch fixes health_check() in lustre init script to
account for the status of MGT on a dedicated node.

Test-Parameters: trivial

Change-Id: Ifdf2e4a1002e870c25925b6e7fce82b132cc24f2
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: https://review.whamcloud.com/25995
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-6051 utils: improve efficiency of lfs_migrate cmp 78/25878/2
Andreas Dilger [Wed, 8 Mar 2017 00:12:18 +0000 (17:12 -0700)]
LU-6051 utils: improve efficiency of lfs_migrate cmp

The file comparison used by lfs_migrate could be more efficient by
using the "cmp -s" option to avoid printing byte-by-byte differences.
This allows faster comparisons via memcmp() and comparing the file
size.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I424538388d7e31609fa1f875128f03fef33ebbe5
Reviewed-on: https://review.whamcloud.com/25878
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9170 test: fix for hostlist_expand 98/25698/4
Yang Sheng [Wed, 1 Mar 2017 17:38:53 +0000 (01:38 +0800)]
LU-9170 test: fix for hostlist_expand

Fix string substitution in hostlist_expand. It
may produce a wrong result in some case.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Iba33833ec4955dafeeefe5042508a0162c2d477e
Reviewed-on: https://review.whamcloud.com/25698
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-1095 obdclass: improve missing operation message 86/25586/3
Andreas Dilger [Fri, 2 Oct 2015 19:19:05 +0000 (13:19 -0600)]
LU-1095 obdclass: improve missing operation message

Some tests in the past reported missing OBD operations, so improve
the error message to include the device name to make it easier to
debug where this is happening.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id49ab801158690510f9ac191c4e8f7a253bcab07
Reviewed-on: https://review.whamcloud.com/25586
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9085 llstat: don't use a hash as a reference 20/25320/5
Emoly Liu [Wed, 8 Feb 2017 08:07:30 +0000 (16:07 +0800)]
LU-9085 llstat: don't use a hash as a reference

This patch is to fix "Using a hash as a reference is deprecated"
in llstat file. In the newer perl version, we should use "$xxx"
instead of "%xxx" for a hash reference.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I67a1d1e365076b412550de775b115ac373589c60
Reviewed-on: https://review.whamcloud.com/25320
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8907 llite: handle client racy case during create 96/25296/5
Bruno Faccini [Tue, 7 Feb 2017 11:30:35 +0000 (12:30 +0100)]
LU-8907 llite: handle client racy case during create

Some very infrequent situations exists on client side
able to cause a race during create when concurrent access
by fid occurs. The result of the race can allow a d_alias
to be already present when it was not expected when original
code/LBUG has been written.

One of the identified scenario is when a concurrent access of
inode thru the .lustre/fid/<[FID]> method occurs.

New sanity/test_161d has been added to reproduce this scenario.

Final fix is to remove inaccurate
LASSERT(ll_d_hlist_empty(&inode->i_dentry));
in ll_create_node().

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I1ff19883cc5b53831f1d5c577f4152225fa9e0fb
Reviewed-on: https://review.whamcloud.com/25296
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8930 lfsck: sync failure with others no check LF_INCOMPLETE 00/24900/2
Fan Yong [Wed, 28 Sep 2016 02:32:43 +0000 (10:32 +0800)]
LU-8930 lfsck: sync failure with others no check LF_INCOMPLETE

During the first stage scanning, if the MDT has ever failed to
verfiy some OST-objects (related OSTs inaccessable temporarily
because of network trouble or OSTs restart), it needs to sync
failures with other MDTs at the end of the first stage scanning
to avoid improper repairing in the second stage scanning. Such
sync failure should be done even if the "LF_INCOMPLETE" is set
locally, otherwise, other MDTs may have no chance to know that.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8466d8560b5d5070873f10746189fd82bf6f0172
Reviewed-on: https://review.whamcloud.com/24900
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8403 obd: remove unused data parameter from obd_notify() 28/24428/2
John L. Hammond [Mon, 19 Dec 2016 16:42:44 +0000 (10:42 -0600)]
LU-8403 obd: remove unused data parameter from obd_notify()

Remove the unused data parameter from obd_notify() and related
functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1ae72de176ae71441cf486ad6a10c45d57b32aad
Reviewed-on: https://review.whamcloud.com/24428
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8808 mdt: avoid out of order ChangeLog during restore 38/23638/6
Bruno Faccini [Mon, 7 Nov 2016 23:49:42 +0000 (00:49 +0100)]
LU-8808 mdt: avoid out of order ChangeLog during restore

In hsm_cdt_request_completed(), upon restore completion, layout
lock is returned before associated ChangeLog entry has been created.
This allows for a timing window where concurrent file updater can
take control and create its own ChangeLog out of order (before
restore ChangeLog).
This can puzzle, ChangeLog readers (like RobinHood), and have them
think of a wrong file state.
This patch fixes this hole by posting restore ChangeLog before
giving back layout lock.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I958cb61aab34a6a0edc97f4d1390eed623631c21
Reviewed-on: https://review.whamcloud.com/23638
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8701 tests: scrub-performance with separated MGS and MDS 24/23124/4
Fan Yong [Sun, 6 Nov 2016 14:12:57 +0000 (22:12 +0800)]
LU-8701 tests: scrub-performance with separated MGS and MDS

Support to run scrub-performance under the environment of MGS and
MDS on separated devices.

Test-Parameters: trivial envdefinitions=SLOW=yes combinedmdsmgs=false testlist=scrub-performance
Test-Parameters: trivial envdefinitions=SLOW=yes testlist=scrub-performance
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5b01097f3e73c2e0eca6071ecb597ac84893c77d
Reviewed-on: https://review.whamcloud.com/23124
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-137 obdclass: add dt_object_put() and use it 92/23092/8
Andreas Dilger [Tue, 11 Oct 2016 21:18:45 +0000 (15:18 -0600)]
LU-137 obdclass: add dt_object_put() and use it

Introduce dt_object_put() and dt_object_put_nocache() and use them
instead of lu_object_put() and lu_object_put_nocache(), to balance
the use of dt_object_locate() to make a symmetrical interface.

Move the existing ofd_object_put() and osd_object_put() wrappers
into common header functions to use instead of dt_object_put()
or lu_object_put() calls in the respective code, when possible.

Functionally, this patch is a no-op.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7ef657438441de4a91ee08edb4bc097ac97b7418
Reviewed-on: https://review.whamcloud.com/23092
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5969 tests: allow "version" without "lustre:" prefix 47/26047/2
Andreas Dilger [Thu, 16 Mar 2017 21:09:04 +0000 (15:09 -0600)]
LU-5969 tests: allow "version" without "lustre:" prefix

Fix lustre_build_version() to handle if "lctl get_param version"
returns a bare version number like "2.9.54" as done with the
upstream kernel client and once https://review.whamcloud.com/25632
lands, instead of the old prefixed "lustre: 2.8.53" version string.

This was already fixed in "lctl lustre_build_version" in patch
https://review.whamcloud.com/25324 but due to the order of checks
in lustre_build_version() the fallback of using "lctl --version"
took precedence, which returns only the build version of the lctl
tool itself and not the Lustre module version.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1b0d87d186d234b62aa59bcadde42452323ebbe5
Reviewed-on: https://review.whamcloud.com/26047
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8872 quota: incorrect LASSERT 24/24024/5
Niu Yawei [Wed, 30 Nov 2016 03:43:56 +0000 (22:43 -0500)]
LU-8872 quota: incorrect LASSERT

Remove an incorrect LASSERT in qsd_add_deferred().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I7ea0274032ac67e3a039a0a0982d5945498fd434
Reviewed-on: https://review.whamcloud.com/24024
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9040 scrub: handle group boundary properly (2) 59/26059/2
Fan Yong [Sun, 6 Nov 2016 13:57:55 +0000 (21:57 +0800)]
LU-9040 scrub: handle group boundary properly (2)

In the osd_inode_iteration() logic, if the next/exec method
sets the iteration @offset variable just as group boundary:
the LDISKFS_INODES_PER_GROUP(sb), the original logic cannot
incrase the group#, as to the iteration repeats in the same
groupc for ever.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ie55d8ffaed2257baed9761a30c09c62e23105434
Reviewed-on: https://review.whamcloud.com/26059
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
3 years agoLU-9216 config: move config types into lustre_idl.h 22/26022/5
Niu Yawei [Thu, 16 Mar 2017 02:28:27 +0000 (22:28 -0400)]
LU-9216 config: move config types into lustre_idl.h

Move config type values CONFIG_T_XXX into lustre_idl.h since they
will be put on wire when reading config log for IR and nodemap.

Add missing wire checks for mgs_nidtbl_entry, mgs_config_body and
mgs_config_res.

Redefine CONFIG_SUB_XXX for the sub clds attached on config log.

Test-Parameters: clientjob=lustre-b2_7_fe clientbuildno=133 testlist=runtests

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9eb7a89c0fee0c11666bc8f587d3fd5e2ad4fc8d
Reviewed-on: https://review.whamcloud.com/26022
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7497 tests: Fix test failure in conf-sanity 32b 40/25940/4
Jinshan Xiong [Sat, 11 Mar 2017 00:57:10 +0000 (16:57 -0800)]
LU-7497 tests: Fix test failure in conf-sanity 32b

It turned out that the MGS svc was wrongly turned on in disk image
'disk2_4-zfs.tar.bz2', which caused the OST mount failure with
EALREADY because the MGS service has already been started by the MDS.

This is just a temporary solution and the final fix would be to clean
the disk image in question.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I912d39c08ba1437078fe7126d4498950ebeadaf1
Reviewed-on: https://review.whamcloud.com/25940
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9102 test: Include specific headers needed by Posix suite 06/25606/8
Wei Liu [Thu, 23 Feb 2017 17:07:41 +0000 (09:07 -0800)]
LU-9102 test: Include specific headers needed by Posix suite

Change prefix to the missing headers on el7

Change-Id: I79d59104a224ccf51795072e2a12da1a113cdaa0
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/25606
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9026 o2iblnd: Adapt to the removal of ib_get_dma_mr() 77/25277/10
Dmitry Eremin [Thu, 2 Mar 2017 18:32:47 +0000 (21:32 +0300)]
LU-9026 o2iblnd: Adapt to the removal of ib_get_dma_mr()

In Linux kernel 4.9-rc1, the function ib_get_dma_mr()
was removed and a second parameter was added to ib_alloc_pd().

This patch fixes this breakage by:

- Detect if ib_get_dma_mr() has been removed, if so, do the
  following:
- Make it so the module parameter map_on_demand can no longer be
  zero (we have to configure FMR/FastReg pools; it can no longer be
  off).
- No longer try to use the global DMA memory region, but make use
  of the FMR/FastReg pool for all RDMA Tx operations.
- Everywhere we are using the device DMA mr to derive the
  L-key for non-registered memory regions, use the
  pd->local_dma_lkey value instead.
- Make the default map_on_demand = 256.  This will allow nodes with
  this patch to still connected to older nodes without this patch
  and FMR/FastReg turned off.  When FMR/FastReg is turned off, we
  use 256 as the max frags so the two sides will still be able to
  communicate and work.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: Iab967d1fdff760c2b06aed395152772fe71bc26d
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25277
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8366 test: Remove PIOS tests 14/26014/2
James Nunez [Wed, 15 Mar 2017 21:25:30 +0000 (15:25 -0600)]
LU-8366 test: Remove PIOS tests

PIOS is an old benchmark and is not open source. PIOS should
be removed from sanity-benchmark.

Test-Parameters: trivial testlist=sanity-benchmark
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Id89ab1107e4d7e444f6149d04bd90b4ea8c7828b
Reviewed-on: https://review.whamcloud.com/26014
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
3 years agoLU-9169 kernel: kernel update [SLES12 SP2 4.4.49-92.11] 88/25688/3
Bob Glossman [Tue, 28 Feb 2017 19:39:37 +0000 (11:39 -0800)]
LU-9169 kernel: kernel update [SLES12 SP2 4.4.49-92.11]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I210b548c1f27a9e444626a29dfd61953850d60ff
Reviewed-on: https://review.whamcloud.com/25688
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-4423 llite: get rid of lustre_dump_dentry() 39/25639/4
Al Viro [Thu, 9 Mar 2017 18:11:35 +0000 (13:11 -0500)]
LU-4423 llite: get rid of lustre_dump_dentry()

it's not only badly racy, it's actually dead code - the call in
ll_invalidate_aliases() is unreachable.  For an alias of our inode
to be root dentry, the inode would have to be its ->d_inode, aka
the root inode, and we never call ll_invalidate_aliases() for that.

Linux-commit: 5e9e2040247ac19a46d6627736ece726c46f1fdf

Change-Id: Ic5e475102793424396bb8b173eb8d9a3b020bd82
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25639
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9153 utils: improve llog name parsing 68/25468/7
Andreas Dilger [Wed, 15 Feb 2017 06:28:39 +0000 (23:28 -0700)]
LU-9153 utils: improve llog name parsing

Current llog name specification is totally non-standard for
lctl llog_print, llog_info, and related commands.  Change this
to accept llog names without the leading '$', since adding '$'
is non-intuitive and the shell will think this is a variable
and "$fsname-client" becomes "-client" unless it is escaped.

Similarly, llog identifiers were formatted like "#0x4:10#0000000"
instead of the standard FID format [0x4:0x10:0x0].  Update the
code to accept and print FIDs in normal format.

Add lctl-llog_print.8, lctl-llog_info.8, lctl-llog_catlist.8 man
pages.

Add a test case for parsing "lctl llog_print" with both old
and new input formats.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Idc00308162eb40b121b49bae3a6bc1d9326927fb
Reviewed-on: https://review.whamcloud.com/25468
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9117 tests: SKIP sanity/129 on older MDS versions 40/25440/3
Parinay Kondekar [Tue, 14 Feb 2017 11:48:14 +0000 (17:18 +0530)]
LU-9117 tests: SKIP sanity/129 on older MDS versions

LU-3531 esp the change,
"LU-3531 mdt: delete striped directory" is not present
on 2.5.x esp on tags less 2.5.56.

Thus sanity/129 test needs to be skipped in interop
with such MDS versions.

Test-Parameters: testlist=sanity
Signed-off-by: Parinay Kondekar <Parinay.Kondekar@seagate.com>
Change-Id: I3f1b92ed5ef5b8011bee8b2ef77c7c5b0db197b2
Reviewed-on: https://review.whamcloud.com/25440
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-1095 mdt: quiet mdt_mfd_set_mode() debugging 99/25399/2
Andreas Dilger [Sat, 11 Feb 2017 02:12:50 +0000 (19:12 -0700)]
LU-1095 mdt: quiet mdt_mfd_set_mode() debugging

Quiet overly verbose debugging in mdt_mfd_set_mode().

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ibdd77b70bfdd430fb9e3f976564780f5d1877613
Reviewed-on: https://review.whamcloud.com/25399
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7092 tests: generate sanity proc_dirs on facet 63/25363/9
Andreas Dilger [Fri, 10 Feb 2017 01:09:35 +0000 (18:09 -0700)]
LU-7092 tests: generate sanity proc_dirs on facet

For sanity test_133f and test_133g generate the list of available
/proc and /sys directories for lnet and lustre on the remote facet
instead of only the client, so that the tests work during interop.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0125e20656a712eb620ba4f07d1eaeeb0b9cab07
Reviewed-on: https://review.whamcloud.com/25363
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9014 test: handle separate MGS in conf-sanity test_72() 52/24852/3
John L. Hammond [Thu, 12 Jan 2017 14:56:22 +0000 (08:56 -0600)]
LU-9014 test: handle separate MGS in conf-sanity test_72()

In conf-sanity test_72() when using a separate MGS, add --replace to
the mkfs options to prevent failures from already registered targets.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I06a266a8ca954c4b7c62fee3957145854d48dd38
Reviewed-on: https://review.whamcloud.com/24852
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
3 years agoLU-9010 obdclass: use static initializer macros where possible 27/24827/3
John L. Hammond [Wed, 11 Jan 2017 17:03:45 +0000 (11:03 -0600)]
LU-9010 obdclass: use static initializer macros where possible

In lustre/ldlm/ replace module load time initialization of several
atomics, lists, locks, mutexes, wait queues, etc with static
initialization using the kernel provided macros.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5514992762b95fb23a154970abebd34fb65d9be2
Reviewed-on: https://review.whamcloud.com/24827
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9066 ldlm: don't evict client on umount if AST fails 21/23921/10
Mikhal Pershin [Thu, 3 Nov 2016 22:17:30 +0000 (06:17 +0800)]
LU-9066 ldlm: don't evict client on umount if AST fails

When server is being stopped the AST send is denied and
that AST failure will lead to the unexpected client
eviction.
Patch adds extra checks in ldlm_handle_ast_error() to
distinguish that type of error and proceed without
eviction.

Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I57c7f417376ac64c8df4c6d5f68bea5083c0d22f
Reviewed-on: https://review.whamcloud.com/23921
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8735 llite: Return -ERESTARTSYS in range_lock() 59/23259/3
Chris Horn [Tue, 6 Sep 2016 15:48:28 +0000 (10:48 -0500)]
LU-8735 llite: Return -ERESTARTSYS in range_lock()

If we return -ERESTARTSYS rather than -EINTR then the syscall can be
retried rather than failing with -EINTR.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I20d1a552448e567f0400a14a71a00d0d25a9164d
Reviewed-on: https://review.whamcloud.com/23259
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7790 test: skip test_130a and test_130b if server is less than 2.7.2 82/19682/8
Wei Liu [Wed, 20 Apr 2016 21:08:09 +0000 (14:08 -0700)]
LU-7790 test: skip test_130a and test_130b if server is less than 2.7.2

Skip test_130a and test_130b if server version is less than 2.7.2

Test-Parameters: trivial testlist=recovery-small

Change-Id: I4377792111e973cad793a56221a7bc8fe8ddfe68
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/19682
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5964 tests: open a large number of files at once 85/12885/10
Andreas Dilger [Fri, 28 Nov 2014 20:54:37 +0000 (13:54 -0700)]
LU-5964 tests: open a large number of files at once

Modify createmany to allow keeping files open after creation to test
the total number of files that can be open at one time.  Also add a
sanity.sh test for this functionality.

Fix handling of last "-seconds" argument, but deprecate positional
parameters in favor of proper named argument parsing, since that is
more flexible and sustainable in the long term.

Order the include files alphabetically to simplify coding.
Order the argument parsing to be alphabetical to simplify coding.

Use booleans for do_* flags, and don't treat multiple of the same
argument as an error, that doesn't really improve usability at all.

Rename the "-r" option to "-u", to better match "unlinkmany", but
allow both to be accepted for the time being.  Allow "-u" (or "-r")
to be used in isolation to only delete files created in a prior run.
Print stats properly if only "-u" is specified, instead of assuming
that files are always being created in the same pass as deletion.

Print only the short program name instead of all of argv[0].  This
will also simplify multiplexing "createmany" as "unlinkmany" by name
at a later time.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0e8c1af65dcf0b25065f26731c694f4beaab75d7
Reviewed-on: https://review.whamcloud.com/12885
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9019 socklnd: use 64-bit incarnation time stamp 90/25690/3
James Simmons [Thu, 9 Mar 2017 18:06:29 +0000 (13:06 -0500)]
LU-9019 socklnd: use 64-bit incarnation time stamp

ksnn_incarnation is a 64-bit value, but using timeval to compute
it will cause an overflow in 2038. This changes it to use
ktime_get_real_ns() instead.

Change-Id: I9a0b1f2c35d04d8dc2c8fd5e17e19583c8af5ff0
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25690
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9019 msg : migrate IR stats to 64 bit time 19/24919/17
James Simmons [Thu, 9 Mar 2017 17:59:33 +0000 (12:59 -0500)]
LU-9019 msg : migrate IR stats to 64 bit time

Replace the struct timeval usage due to the overflow
issues in 2038 on 32-bit systems. Use of struct
timeval upstream is deprecated as well. Replace its
use with timespec64 and use ktime to store the
fsdb_notify_* fields.

Change-Id: I7771710723d6717cbff93e64abdf794957c42be5
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24919
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8900 snapshot: rename filesysetem fsname 68/24268/16
Fan Yong [Fri, 4 Nov 2016 12:28:31 +0000 (20:28 +0800)]
LU-8900 snapshot: rename filesysetem fsname

Introuduce new tunefs options "--rename" for renaming the existing
Lustre filesystem fsname with configuration/parameters preserved.
Usage: tunefs.lustre --fsname $NEW_FSNAME --rename $OLD_FSNAME ...

NEW_FSNAME is the new fsname, must be specified.
OLD_FSNAME is the old fsname, it is optional if the MGS and the MDT
are combined together.

It shares the functionality of fork/erase Lustre configuration logs
in kernel.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic9aaf6ed6209c7580bfabd4b660a6a7667576484
Reviewed-on: https://review.whamcloud.com/24268
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8900 snapshot: simulate readonly device 67/24267/15
Fan Yong [Fri, 4 Nov 2016 12:10:12 +0000 (20:10 +0800)]
LU-8900 snapshot: simulate readonly device

Introduce new server-side mount option: rdonly_dev. Under the
device readonly mode, the Lustre kernel threads that may cause
system modification, such as lfsck, scrub, osp_precreate_thread,
osp_sync_thread, and so on, will be disabled when mount. And the
system modification for recovery and client connect/disconnect,
will be ignored also.

On the other hand, if the server is mounted as "rdonly_dev", then
the client must specify "-o ro" option when mount; otherwise, it
will get -EACCES.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iafc02bf53c41b9d7f08120ba7639b91ae527bbbc
Reviewed-on: https://review.whamcloud.com/24267
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8900 snapshot: fork/erase configuration 66/24266/16
Fan Yong [Fri, 4 Nov 2016 10:30:33 +0000 (18:30 +0800)]
LU-8900 snapshot: fork/erase configuration

Introuduce new lctl command to fork/erase the configuration for
the specified system. It is used for mount the snapshot together
with the snapshot's original system MGS. Usage:

lctl fork_lcfg <fsname> <newname>
lctl erase_lcfg <fsname>

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I55d420b54215b09d42b2496b5c1271211c1fb5d7
Reviewed-on: https://review.whamcloud.com/24266
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8900 snapshot: user interface for write barrier on MDT 65/24265/19
Fan Yong [Fri, 4 Nov 2016 10:29:14 +0000 (18:29 +0800)]
LU-8900 snapshot: user interface for write barrier on MDT

The user can control the write barrier on MDTs via lctl commands:

Freeze barrier:
lctl barrier_freeze <fsname> [timeout (in second)]
NOTE: the default timeout value is 30 (seconds).

Thaw barrier:
lctl barrier_thaw <fsname>

Query barrier:
lctl barrier_stat <fsname>

Rescan barrier bitmap:
lctl barrier_rescan <fsname> [timeout (in second)]
NOTE: the default timeout value is 30 (seconds).

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Id953203fc3ce6ebbce9f1ae0511fbe2b3813bb9f
Reviewed-on: https://review.whamcloud.com/24265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>