Whamcloud - gitweb
fs/lustre-release.git
6 years agoNew tag 2.10.51 2.10.51 v2_10_51 v2_10_51_0
Oleg Drokin [Thu, 3 Aug 2017 23:57:30 +0000 (19:57 -0400)]
New tag 2.10.51

Change-Id: I87e70d18c01fe26e283014193463286f0d725cd1
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-7544 doc: improve setquota man page 63/28163/4
Niu Yawei [Fri, 21 Jul 2017 03:01:50 +0000 (23:01 -0400)]
LU-7544 doc: improve setquota man page

Move 'lfs setquota' to a separate man page and change the man page to
use "K, M, G, T, P" suffixes for block limits.

Test-Parameters: trivial
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I8de81ebfed86151dbb1ac16690f3b4e377f95366
Reviewed-on: https://review.whamcloud.com/28163
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-9778 llite: Read ahead should return pages read 52/28052/3
Patrick Farrell [Fri, 14 Jul 2017 14:07:52 +0000 (09:07 -0500)]
LU-9778 llite: Read ahead should return pages read

ll_read_ahead_pages was modified by:
LU-7990 clio: revise readahead to support 16MB IO
d8467ab8a2ca15fbbd5be3429c9cf9ceb0fa78b8

And returning the count of pages read was removed.

This only affects debug, but it's very nice to have it
printed out, and several messages still try to print out
pages read ahead, but print 0.

Restore this functionality.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I80fe66b5195629e0c46d5d19c76e3bcc0030a22a
Reviewed-on: https://review.whamcloud.com/28052
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
6 years agoLU-9733 obd: fix fsname parsing 16/27916/5
Wang Shilong [Mon, 3 Jul 2017 11:55:30 +0000 (19:55 +0800)]
LU-9733 obd: fix fsname parsing

Lustre support Lustre fsname started with numbers.
So here we should try fsname firstly, if it failed,
fall back to number parsing.

Change-Id: Idd98b9fdc8375729faa61f51251caa101035dd93
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/27916
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializer in mount_utils_zfs.c 25/27825/2
Steve Guminski [Mon, 26 Jun 2017 17:53:13 +0000 (13:53 -0400)]
LU-6210 utils: Use C99 struct initializer in mount_utils_zfs.c

This patch makes no functional changes.  The ZLB_INIT macro is
updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ieaf53059782254935a13f0efd3f72785dacc0b8b
Reviewed-on: https://review.whamcloud.com/27825
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
6 years agoLU-9710 utils: adjust barrier_stat input/output 10/27810/5
Fan Yong [Wed, 19 Jul 2017 05:58:06 +0000 (13:58 +0800)]
LU-9710 utils: adjust barrier_stat input/output

The command format will be:
lctl barrier_stat [--state|-s] [--timeout|-t] <fsname>

If no option is specified, or both "state" and "timeout"
options are specified, then the output format will be:
state: xxx
timeout: nnn seconds

Otherwise, only the value ('xxx' or 'nnn') corresponding
to the given option will be printed.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If39f95ef984be3ab709b1366fdefe8eedb4b2453
Reviewed-on: https://review.whamcloud.com/27810
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
6 years agoLU-9558 llite: user enhanced getattr functionality in newer kernels 15/27715/3
James Simmons [Wed, 21 Jun 2017 21:37:04 +0000 (17:37 -0400)]
LU-9558 llite: user enhanced getattr functionality in newer kernels

Starting with the linux kernel 4.11 a new system call was added
to enhance the file info available. This new functionality has
changed the getattr functions in the linux kernel. This patch
only provides the most basic support for now.

Linux-commit: a528d35e8bfcc521d7cb70aaf03e1bd296c8493f

Change-Id: Iee1b02bb94dab05a3197866af9eedd6f35cac7c5
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27715
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
6 years agoLU-9558 llite: handle struct vm_operations changes 51/27651/8
James Simmons [Thu, 27 Jul 2017 18:09:17 +0000 (14:09 -0400)]
LU-9558 llite: handle struct vm_operations changes

For the linux 4.11 kernel passing in struct vm_area_struct
to struct vm_operations members has been removed since
struct vm_area_struct has been merged into struct vm_fault.
Handle these changes in the llite layer.

Linux-commit: 11bac80004499ea59f361ef2a5516c84b6eab675

Test-Parameters: trivial

Change-Id: Ie44dd50de487044e3e2205978b7a36b15f7b0983
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27651
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9203 lnet: fix lnet_cpt_of_md() 65/28165/7
Amir Shehata [Fri, 21 Jul 2017 04:17:50 +0000 (21:17 -0700)]
LU-9203 lnet: fix lnet_cpt_of_md()

The intent of this function is to get the cpt nearest to the
memory described by the MD.

There are three scenarios that must be handled:
1. The memory is described by an lnet_kiov_t structure
 -> this describes kernel pages
2. The memory is described by a struct kvec
 -> this describes kernel logical addresses
3. The memory is a contiguous buffer allocated via vmalloc

For case 1 and 2 we look at the first vector which contains
the data to be DMAed, taking into consideration the msg offset.

For case 2 we have to take the extra step of translating the kernel
logical address to a physical page using virt_to_page() macro.

For case 3 we need to use is_vmalloc_addr() and vmalloc_to_page to
get the associated page to be able to identify the CPT.

o2iblnd uses the same strategy when it's mapping the memory into
a scatter/gather list. Therefore, lnet_kvaddr_to_page() common
function was created to be used by both the o2iblnd and
lnet_cpt_of_md()

kmap_to_page() performs the high memory check which
lnet_kvaddr_to_page() does. However, unlike the latter it handles
the highmem case properly instead of calling LBUG. It's not
100% clear why the code was written that way. Since the legacy
code will need to still be maintained, adding kmap_to_page() will
not simplify the code. Furthermore, the behavior for kernels
which export kmap_to_page() will be different from kernels which
do not. At worst calling kmap_to_page() might mask some problems
which would've been caught by the LBUG earlier on. However, at
the time of this fix, that LBUG has never been observed.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I2c67e5df77d60112bf27f900e0325d189f193aed
Reviewed-on: https://review.whamcloud.com/28165
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4536 tests: Re-enable sanity/65ic 03/27803/3
Nathaniel Clark [Wed, 19 Jul 2017 05:34:22 +0000 (01:34 -0400)]
LU-4536 tests: Re-enable sanity/65ic

Because of work done for LU-9019 to bring timing to be universally
64bit, re-enable this test.

Test-Parameters: trivial
Test-Parameters: testlist=sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=2 ostcount=2 mdscount=1 mdtcount=1 clientcount=2
Test-Parameters: testlist=sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=2 ostcount=2 mdscount=1 mdtcount=1 clientcount=2
Test-Parameters: testlist=sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=2 ostcount=2 mdscount=1 mdtcount=1 clientcount=2
Test-Parameters: testlist=sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=2 ostcount=2 mdscount=1 mdtcount=1 clientcount=2

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ifae4b2f6d3ad1e8c02eab8b1ca97f9924c0bf360
Reviewed-on: https://review.whamcloud.com/27803
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-7991 osd-zfs: remove obsolete quota code 61/27661/6
Fan Yong [Tue, 25 Jul 2017 06:09:21 +0000 (14:09 +0800)]
LU-7991 osd-zfs: remove obsolete quota code

Directly use ZFS backend quota accounting objects,
no need be mapped from Lustre own account objects.
Related obsolete logic is removed totally.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2c6aeb1ccac52348d8d163017f73ef0fd9133551
Reviewed-on: https://review.whamcloud.com/27661
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-7988 hsm: run HSM coordinator once per second at most 41/19341/41
Frank Zago [Wed, 6 Apr 2016 21:03:14 +0000 (16:03 -0500)]
LU-7988 hsm: run HSM coordinator once per second at most

When there is heavy HSM usage, each new HSM request can trigger the
HSM coordinator, which may run many times per seconds. When it is
running it locks the HSM catalog (using cdt_llog_lock) preventing any
other HSM operation to happen, such as insertion, removal or dumping
of the requests.

Limit the coordinator to run once per second, and only if there is
work to do. It will still execute the loop once every 10 seconds (or
as defined by the procfs loop_period parameter) to do housekeeping.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ide3f061f8943a3088ea713993521897fb74e5d99
Reviewed-on: https://review.whamcloud.com/19341
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoRevert "LU-9564 build: Add server-build for Ubuntu with Kernel 4.4.0" 93/28293/2
Oleg Drokin [Tue, 1 Aug 2017 05:05:28 +0000 (05:05 +0000)]
Revert "LU-9564 build: Add server-build for Ubuntu with Kernel 4.4.0"

This actually breaks our Ubuntu builds which prevents any sort of full testing, so I am reverting this now.

This reverts commit 86c3f90d3ab8dbd21dc6fa325aa3a0556fb94035.

Change-Id: I14d242bfde1efb0144080b882e63542fc2190465
Reviewed-on: https://review.whamcloud.com/28293
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Jenkins
6 years agoLU-9439 scripts: add lnet script in .gitignore 06/28106/2
Dmitry Eremin [Wed, 19 Jul 2017 14:00:04 +0000 (17:00 +0300)]
LU-9439 scripts: add lnet script in .gitignore

Ignore generated script.

Test-Parameters: trivial
Change-Id: Id1c67749958b98dc10610514e11fcecbe6ffe3c5
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/28106
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-7589 build: update checkpatch to ~4.13-rc1 kernel 55/28055/3
Andreas Dilger [Sat, 15 Jul 2017 05:57:23 +0000 (23:57 -0600)]
LU-7589 build: update checkpatch to ~4.13-rc1 kernel

Update checkpatch.pl to v4.12-11743-g96d0d83 kernel version.

This version includes new functionality that allows us to
replace the Lustre specific dep_functions and dep_includes
checks added in our local version with the "spelling.txt"
file.

Allow CDEBUG/CERROR/LCONSOLE/DEBUG_REQ text strings to exceed
80 columns, as with upstream printk messages.

Test-Parameters: trivial

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2a16bbd85722b15eb9188eb504bac3a2f63ebbe5
Reviewed-on: https://review.whamcloud.com/28055
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9773 kernel: kernel update [SLES12 SP2 4.4.74-92.29] 43/28043/2
Bob Glossman [Thu, 13 Jul 2017 19:32:30 +0000 (12:32 -0700)]
LU-9773 kernel: kernel update [SLES12 SP2 4.4.74-92.29]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I42107acb5a12f5200d3cb58121d10ffdc1dbc6d2
Reviewed-on: https://review.whamcloud.com/28043
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9763 kernel: kernel update RHEL6.9 [2.6.32-696.6.3.el6] 29/28029/4
Bob Glossman [Tue, 11 Jul 2017 18:47:57 +0000 (11:47 -0700)]
LU-9763 kernel: kernel update RHEL6.9 [2.6.32-696.6.3.el6]

Update RHEL6.9 kernel to 2.6.32-696.6.3.el6
kernel patch added as a workaround for LU-9698

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
  ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4c3d2faa35d70b5aa981e8dc9bc630275c1c61f1
Reviewed-on: https://review.whamcloud.com/28029
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9683 ptlrpc: fix argument misorder 27/28027/4
Alex Zhuravlev [Wed, 19 Jul 2017 04:59:13 +0000 (00:59 -0400)]
LU-9683 ptlrpc: fix argument misorder

involved in timediffs calculation.

Change-Id: Ib4a45dddb3866824b696aaeaa190f2ab9b1c71ac
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/28027
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-8346 obdclass: Set lc_version 94/27994/3
Patrick Farrell [Tue, 11 Jul 2017 14:31:46 +0000 (09:31 -0500)]
LU-8346 obdclass: Set lc_version

The patch LU-8346 obdclass: guarantee all keys filled
removed the setting of lc_version, which makes us always
refill cached envs.  This is very expensive, particularly
for fast reads.

Original commit e58f8d609a81576eaf5bc9d0fa53bef274a01bf,
https://review.whamcloud.com/26099

Change-Id: I13ba7d19185b899d1f68d244365160539e881b8e
Signed-off-by: Patrick Farrell <paf@cray.com>
Reviewed-on: https://review.whamcloud.com/27994
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 initializer in jt_nodemap_del_idmap() 91/27891/2
Steve Guminski [Mon, 22 May 2017 13:04:01 +0000 (09:04 -0400)]
LU-6210 utils: Use C99 initializer in jt_nodemap_del_idmap()

This patch makes no functional changes.  The long_options struct
initializer in jt_nodemap_del_idmap() is updated to C99 syntax.
It is renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ic97d308286ce03507bf3f928d9a2b23d6d39947f
Reviewed-on: https://review.whamcloud.com/27891
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: Use C99 initializer in jt_nodemap_add_idmap() 90/27890/2
Steve Guminski [Mon, 22 May 2017 13:06:09 +0000 (09:06 -0400)]
LU-6210 utils: Use C99 initializer in jt_nodemap_add_idmap()

This patch makes no functional changes.  The long_options struct
initializer in jt_nodemap_add_idmap() is updated to C99 syntax.
It is renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Iafeb7e76a943d8f8118623c532da1e8e2880aba2
Reviewed-on: https://review.whamcloud.com/27890
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: Use C99 initializer in jt_nodemap_modify() 89/27889/2
Steve Guminski [Mon, 22 May 2017 12:54:11 +0000 (08:54 -0400)]
LU-6210 utils: Use C99 initializer in jt_nodemap_modify()

This patch makes no functional changes.  The long_options struct
initializer in jt_nodemap_modify() is updated to C99 syntax.
It is renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I4c1bd5a3aa26c486436ae68d698fd89f8735a7fb
Reviewed-on: https://review.whamcloud.com/27889
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: Use C99 initializer in jt_nodemap_set_fileset() 88/27888/2
Steve Guminski [Mon, 22 May 2017 12:50:48 +0000 (08:50 -0400)]
LU-6210 utils: Use C99 initializer in jt_nodemap_set_fileset()

This patch makes no functional changes.  The long_options struct
initializer in jt_nodemap_set_fileset() is updated to C99 syntax.
It is renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I8c0e90e04dd407246e95e59c5b856b9d11b78500
Reviewed-on: https://review.whamcloud.com/27888
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: Use C99 initializer in jt_nodemap_add_range() 82/27882/2
Steve Guminski [Mon, 22 May 2017 12:42:50 +0000 (08:42 -0400)]
LU-6210 utils: Use C99 initializer in jt_nodemap_add_range()

This patch makes no functional changes.  The long_options struct
initializer in jt_nodemap_add_range() is updated to C99 syntax.
It is renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Iba46b68e1c19f89e770f234ab1149926d64f7efb
Reviewed-on: https://review.whamcloud.com/27882
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: C99 initializer in llog_cancel_parse_optional() 80/27880/2
Steve Guminski [Mon, 22 May 2017 12:23:27 +0000 (08:23 -0400)]
LU-6210 utils: C99 initializer in llog_cancel_parse_optional()

This patch makes no functional changes.  The long_options struct
initializer in llog_cancel_parse_optional() is updated to C99 syntax.
The option struct and option string are renamed to long_opts and
short_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ie6e1c823ebc59c3791c7c6b0360f33d1d0184c1a
Reviewed-on: https://review.whamcloud.com/27880
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: Use C99 initializer in jt_obd_md_common() 79/27879/2
Steve Guminski [Mon, 22 May 2017 12:13:56 +0000 (08:13 -0400)]
LU-6210 utils: Use C99 initializer in jt_obd_md_common()

This patch makes no functional changes.  The long_opts struct
initializer in jt_obd_md_common() is updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ifa7140e9980570940f8dfd1200307d02242a8c45
Reviewed-on: https://review.whamcloud.com/27879
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-9712 kernel: kernel update [SLES11 SP4 3.0.101-107] 41/27841/2
Bob Glossman [Mon, 26 Jun 2017 17:05:26 +0000 (10:05 -0700)]
LU-9712 kernel: kernel update [SLES11 SP4 3.0.101-107]

Update SLES11 SP4 kernel to 3.0.101-107

Test-Parameters: mdsdistro=sles11sp4 ossdistro=sles11sp4 \
  clientdistro=sles11sp4 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ic62b288becc936d54ee88539ab1635ca3a78cf7f
Reviewed-on: https://review.whamcloud.com/27841
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6051 utils: allow lfs_migrate to migrate links 53/27553/2
Andreas Dilger [Sat, 10 Jun 2017 02:39:11 +0000 (20:39 -0600)]
LU-6051 utils: allow lfs_migrate to migrate links

When running on a server that supports "lfs migrate", it is possible
to migrate hard links transparently.  This is still not possible for
servers that do not support "lfs migrate" (layout swap) so we still
need to keep the hard-link check if "lfs migrate" fails.

Add test for migrating many hardlinks on supported kernels.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7fa1c70611aa7dd61f822844d5fe339f730640e5
Reviewed-on: https://review.whamcloud.com/27553
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lfs_hsm_request() 26/27526/3
Steve Guminski [Fri, 14 Apr 2017 19:47:25 +0000 (15:47 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_hsm_request()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_hsm_request() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I62b07e59a3f81cf8ee936ae46b1cd8f99913e53c
Reviewed-on: https://review.whamcloud.com/27526
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lfs_setquota() 21/27521/3
Steve Guminski [Fri, 14 Apr 2017 19:31:40 +0000 (15:31 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_setquota()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_setquota() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I4bae3ccf40380d662b151b8ac53976f38c63daf6
Reviewed-on: https://review.whamcloud.com/27521
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 initializers in lfs_getstripe_internal 14/27514/4
Steve Guminski [Fri, 14 Apr 2017 19:10:56 +0000 (15:10 -0400)]
LU-6210 utils: Use C99 initializers in lfs_getstripe_internal

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_getstripe_internal() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I865ee0200b58f649810452000d91fa455da4d7ba
Reviewed-on: https://review.whamcloud.com/27514
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9019 obdclass: migrate jobstats to 64 bit time 11/27911/3
James Simmons [Sat, 8 Jul 2017 21:23:13 +0000 (17:23 -0400)]
LU-9019 obdclass: migrate jobstats to 64 bit time

Change all time_t fields for jobstats to time64_t.
Replace cfs_time_current_sec() to avoid the overflow
issues in 2038 with ktime_get_real_seconds().

Change-Id: Icf3af60b9b373acbf5d921add9b4c4d33c77aa61
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27911
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9728 osd: use GFP_HIGHUSER for non-local IO 08/27908/3
Andreas Dilger [Fri, 30 Jun 2017 20:37:07 +0000 (14:37 -0600)]
LU-9728 osd: use GFP_HIGHUSER for non-local IO

When the obdfilter code was split into separate OFD and OSD modules,
the bulk IO page allocation was implemented to use GFP_NOFS to avoid
allocations recursing into the filesystem and causing deadlocks.

However, this is only possible if the RPC is coming from a local
client, as we might end up waiting on a page sent in the request we're
serving. Local RPCs use __GFP_HIGHMEM so that the pages can use all of
the available memory on the OSS on 32-bit machines.

It is possible to use more aggressive GFP_HIGHUSER flags for non-local
clients to be able to generate more memory pressure on the OSS and
allow inactive pages to be reclaimed, since the OSS doesn't have any
other processes or allocations that generate memory reclaim pressure.

See also b=17576 (bdf50dc9) and b=19529 (3dcf18d3) for details.

The patch also implements an LNet function to determine if a client NID
is local or not.  This becomes more complex in the LNet Multi-Rail world
and it is really LNet's job to handle NIDs, not that of Lustre.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2806c9c5c2fe269669eafdafaf2310924c3ebbe5
Reviewed-on: https://review.whamcloud.com/27908
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-8935 ptlrpc: missing barrier before wake_up 83/26583/4
Lai Siyao [Wed, 12 Apr 2017 21:56:50 +0000 (05:56 +0800)]
LU-8935 ptlrpc: missing barrier before wake_up

ptlrpc_client_wake_req() misses a memory barrier, which may cause
strange errors.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ic8e9cbaf8c07f503798b95c608477508204d9614
Reviewed-on: https://review.whamcloud.com/26583
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9019 lfsck: migrate to 64 bit time 66/26466/7
James Simmons [Wed, 19 Jul 2017 04:49:42 +0000 (00:49 -0400)]
LU-9019 lfsck: migrate to 64 bit time

Replace cfs_time_current_sec() to avoid the overflow
issues in 2038 with ktime_get_real_seconds(). The
reset of the lfsck code can move to time64_t as
well. Handle older lfsck on disk data formats that
contain 32 bit time formats. Newly generated on disk
data will always be 64 bit timestamps.

Change-Id: I1db47af3b9f462c31375f53a6216a095ab4767ad
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/26466
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-7372 mgs: reprocess all locks at device fini 53/17853/25
Jinshan Xiong [Wed, 10 May 2017 17:39:25 +0000 (10:39 -0700)]
LU-7372 mgs: reprocess all locks at device fini

This is to avoid a case that IR lock revocation is going on when
the obd is being stopped, an extra ldlm_reprocess_recovery_done() is
required to make revocation process move forward.

Turn off 'set -e' in rundbench. Otherwise killing dbench process will
return an error to wait(1) in rundbench. Since test-framework has
turned on error on exit, it will set test result as failure, which
is actually a false alarm.

Test-Parameters: envdefinitions=SLOW=yes,ONLY=26 testlist=replay-dual,replay-dual,replay-dual,replay-dual
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I43ab1df9c8fe5aea15da6c90175fd08a0b099ea2
Reviewed-on: https://review.whamcloud.com/17853
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-8760 lib: avoid unexpected out of order execution 64/23564/3
Fan Yong [Fri, 4 Nov 2016 01:04:39 +0000 (09:04 +0800)]
LU-8760 lib: avoid unexpected out of order execution

There is race condtion in __l_wait_event() because of the
out-of-order execution between changing thread state and
checking condition. It may block the thread (to be waken)
for ever. Consider the following real execution order:

1. Thread1 checks condition on CPU1, gets false.
2. Thread2 sets condition on CPU2.
3. Thread2 calls wake_up() on CPU2 to wake the threads with
   state TASK_INTERRUPTIBLE | TASK_UNINTERRUPTIBLE. But the
   Thread1'sstate is TASK_RUNNING at that time.
4. Thread1 sets its state as TASK_INTERRUPTIBLE on CPU1,
   then schedule.

If the '__timeout' variable is zero, the Thread1 will have
no chance to check the condition again.

Generally, the interval between out-of-ordered step1 and step4
is very tiny, as to above step2 and step3 cannot happen. On some
degree, it can explain why we seldom hit related trouble. But
such race really exists, especially consider that the step1 and
step4 can be interruptible.

The patch adds barrier between changing thread's state and
checking condition to avoid out-of-order execution.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I32caee6b332f037d864419ea8728112da563cce0
Reviewed-on: https://review.whamcloud.com/23564
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9776 lustre-client cannot be installed if both client and server repos are configured 65/28065/4
Brian J. Murrell [Fri, 14 Jul 2017 12:53:27 +0000 (08:53 -0400)]
LU-9776 lustre-client cannot be installed if both client and server repos are configured

Due to current Obsoletes: tags on lustre-client, the client cannot be
installed on nodes where the client and server repos are configured.

Update the Obsoletes: to only obsolete previous lustre-client versions,
not the current one.

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Change-Id: Ie2f022967fd6f65030feeb23ea9637dce505054a
Reviewed-on: https://review.whamcloud.com/28065
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-9731 kmods need to be limited to EL minor release kernel 66/28066/4
Brian J. Murrell [Mon, 17 Jul 2017 16:19:04 +0000 (12:19 -0400)]
LU-9731 kmods need to be limited to EL minor release kernel

Due to upstream RHBZ#1467319 kmods are not being populated with the full
kabi information needed to find a matching kernel for the kmod.

Until this is fixed, we need to apply a workaround to achieve the same
result.

Test-Parameters: trivial

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Change-Id: Ib2eab09719c75be8928eaf607efaa2d814baf5f2
Reviewed-on: https://review.whamcloud.com/28066
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9775 Look for kernel-devel in /usr/src/kernels 64/28064/4
Brian J. Murrell [Fri, 14 Jul 2017 12:40:55 +0000 (08:40 -0400)]
LU-9775 Look for kernel-devel in /usr/src/kernels

If one is building in a build [ch]root such as mock provides, one may
not have the kernel installed which corresponds to $(uname -r).  In such
a case, also try to look for the kernel-devel in /usr/src/kernels/ and
just build for the latest one.  Ideally there is only one installed in any case.

Test-Parameters: trivial

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Change-Id: I7704c6ce7078a507fd6b5f9178b07f750dc03789
Reviewed-on: https://review.whamcloud.com/28064
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9500 lnd: Don't Page Align remote_addr with FastReg 49/27149/6
Doug Oucharek [Tue, 16 May 2017 23:00:53 +0000 (16:00 -0700)]
LU-9500 lnd: Don't Page Align remote_addr with FastReg

Trying to page align the remote_addr for IB_RDMA_WRITE work
requests is triggering "dump_cqe" errors from MOFED 4.x + mlx5.

This patch removes the address masking we were doing with FastReg
which was trying to page align remote_addr values. I am also
removing the setting of "mr->iova" with FastReg as this is being
done in the call to ib_map_mr_sg() and could cause problems.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: If35baa467d8d60866f709b5feea7f619063c6da4
Reviewed-on: https://review.whamcloud.com/27149
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9749 llite: Reduce overhead for ll_do_fast_read 70/27970/12
Patrick Farrell [Mon, 17 Jul 2017 14:03:07 +0000 (09:03 -0500)]
LU-9749 llite: Reduce overhead for ll_do_fast_read

In ll_do_fast_read, looking up a cl_env adds some overhead,
and can also cause spinlock contention on older kernels.

Fast read can safely use the preallocated percpu cl_env, so
do that to reduce overhead.

SLES numbers on recent Xeon, CentOS numbers on VMs on
older hardware.  SLES has queued spinlocks and scales
perfectly with multiple threads, with or without this
patch.  CentOS scales poorly at small I/O sizes without
this patch.

SLES is SLES12SP2, CentOS is CentOS 7.3.

SLES:
1 thread
         8b   1K   1M
Without: 23   2200 6800
With:    27.5 2500 7200

4 threads
         8b   1K    1M
Without: 90   8700  27000
With:    108  10000 28000

Earlier kernel (CentOS 7.3):
1 thread
         8b  1K   1M
Without: 9   1000 5100
with:    12  1300 5800

4 threads
         8b  1K   1M
Without: 22  2400 17000
With:    48  4900 20000

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ice5d653ace5ce76bc8911501a9b15c11b7a3234a
Reviewed-on: https://review.whamcloud.com/27970
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-7129 tests: fsx with directio 57/16357/6
Parinay Kondekar [Thu, 6 Oct 2016 17:00:35 +0000 (22:30 +0530)]
LU-7129 tests: fsx with directio

This change adds O_DIRECT support to the existing fsx in lustre/tests.
It also add sanityn test_16 to excercise this option to fsx.

Upstream Xfstest-commit cad234da750aceb563992d95a0b14379740aedc6

Test-Parameters: trivial testlist=sanityn
Signed-off-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Signed-off-by: Parinay Kondekar <parinay.kondekar@seagate.com>
Change-Id: Ibd6a8dfeff8afaf08eeb3f2f8bddd7cdb688e8cf
Seagate-bug-id: MRP-852
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/16357
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9772 utils: Enable new ZFS MMP on mkfs 51/28051/2
Nathaniel Clark [Fri, 14 Jul 2017 17:39:10 +0000 (13:39 -0400)]
LU-9772 utils: Enable new ZFS MMP on mkfs

ZFS 0.7.0 come with new multi-modifier protection, this patch
enables it by default, on mkfs.

This also ensures canmount is off for pools that were not just
created.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: If9b87e9786e0eaefe5ac9a536edcdca3d1012585
Reviewed-on: https://review.whamcloud.com/28051
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9769 lnet: Fix lost lock 39/28039/2
Dmitry Eremin [Thu, 13 Jul 2017 20:56:41 +0000 (23:56 +0300)]
LU-9769 lnet: Fix lost lock

Unlock lnet_net_lock in case of error in function lnet_dyn_del_net().

Change-Id: Ie3fc8e27abc72077b9c0c330b5d325a47e5236e1
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/28039
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-9019 target: migrate to 64 bit time 09/27909/5
James Simmons [Thu, 13 Jul 2017 16:53:28 +0000 (12:53 -0400)]
LU-9019 target: migrate to 64 bit time

Replace cfs_time_current_sec() to avoid the overflow
issues in 2038 with ktime_get_real_seconds(). Change
time_t bi_deadline to time64_t. Do a direction compare
instead of cfs_time_beforeeq() wrapper.

Change-Id: I93d1cab9ebdcc1f910096e8a229d6c37b244f41d
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27909
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-8849 ofd: Client hanges on ladvise with large start values 09/27209/3
Qian Yingjin [Fri, 19 May 2017 08:49:08 +0000 (16:49 +0800)]
LU-8849 ofd: Client hanges on ladvise with large start values

when the ladvise end is larger than file size, we don't judge
the ladvise start should be smaller than the ladvise end after
capped the end vaule to file size. This patch fixes the problem.

Change-Id: I30346c7ce69ad0d7ceca5ffc0d21d900afd3d2f1
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/27209
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializer for long_opt_start 89/27789/2
Steve Guminski [Fri, 19 May 2017 18:58:59 +0000 (14:58 -0400)]
LU-6210 utils: Use C99 struct initializer for long_opt_start

This patch makes no functional changes.  The long_opt_start struct
initializer in lustre_lfsck.c is updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Id14efb47d29a5f16382422df6a3ad19cdf3156bf
Reviewed-on: https://review.whamcloud.com/27789
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-7760 ptlrpc: remove incorrect pid printing 36/27336/6
Sebastien Buisson [Tue, 30 May 2017 21:39:49 +0000 (17:39 -0400)]
LU-7760 ptlrpc: remove incorrect pid printing

Remove incorrect pid printing in ptlrpc_send_new_req().
It was always printing 0, and current pid is part of the message
header anyway.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia786bc108c4fecb87d5cb16dab88253dc16ce28c
Reviewed-on: https://review.whamcloud.com/27336
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-9663 tests: replace “ls -s” with “stat -c '%s'” to get file size 61/27961/3
Jian Yu [Fri, 7 Jul 2017 06:16:06 +0000 (23:16 -0700)]
LU-9663 tests: replace “ls -s” with “stat -c '%s'” to get file size

This patch improves acl/2561{,_zfs}.test to use “stat -c '%s'” to
get file size instead of using “ls -s”, which prints the allocated
size of the file in blocks.

Test-Parameters: trivial \
envdefinitions=ONLY=103 \
mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanity

Change-Id: I8bfea5197102baeece244e20731c90172c5cc2f1
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: https://review.whamcloud.com/27961
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-7653 tests: replay-single/110f fails for mdts on same MDS 40/27940/2
Parinay Kondekar [Thu, 6 Jul 2017 03:18:00 +0000 (08:48 +0530)]
LU-7653 tests: replay-single/110f fails for mdts on same MDS

replay-single/110f fails when multiple mdts are present on same MDS
node. Changing/reversing the failover order of mdts in the script
replay-singl/110f helps fix this failure in such setup.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Parinay Kondekar <parinay.kondekar@seagate.com>
Change-Id: I8b528c422f0a90dd3174a5ff6c7f8d938ed68a05
Reviewed-on: https://review.whamcloud.com/27940
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9738 kernel: kernel update RHEL7.3 [3.10.0-514.26.2.el7] 34/27934/3
Bob Glossman [Wed, 5 Jul 2017 15:08:06 +0000 (08:08 -0700)]
LU-9738 kernel: kernel update RHEL7.3 [3.10.0-514.26.2.el7]

update RHEL 7.3 kernel to 3.10.0-514.26.2.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I8eff2a9d0c732d97dac1df0c9233d88dbe564a4f
Reviewed-on: https://review.whamcloud.com/27934
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9729 lnet: correct locking in legacy add net 07/27907/3
Amir Shehata [Sat, 1 Jul 2017 01:06:40 +0000 (18:06 -0700)]
LU-9729 lnet: correct locking in legacy add net

Make sure to unlock the api mutex properly
in lnet_dyn_add_net()

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I786545de690ea5966771be3e84d3561b794d55ec
Reviewed-on: https://review.whamcloud.com/27907
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9716 osc: osc_extent_tree_dump0() implementation is suboptimal 66/27866/3
Andrew Perepechko [Wed, 28 Jun 2017 09:24:26 +0000 (12:24 +0300)]
LU-9716 osc: osc_extent_tree_dump0() implementation is suboptimal

Avoid looping in osc_extent_tree_dump() if debugging is disabled.
This helps us save some cpu ticks.

Change-Id: I492429d8a6de79f67b5923895ffa58b7fe3a100d
Seagate-bug-id: MRP-4469
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: https://review.whamcloud.com/27866
Reviewed-by: Alexander Boyko <alexander.boyko@seagate.com>
Tested-by: Jenkins
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializer in parse_opts() 17/27817/2
Steve Guminski [Mon, 22 May 2017 12:00:35 +0000 (08:00 -0400)]
LU-6210 utils: Use C99 struct initializer in parse_opts()

This patch makes no functional changes.  The long_opt initializer in
parse_opts() is updated to C99 syntax, and renamed to long_opts for
consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Id3f1540c33cbd5db33a4a0eb69a89672eac7f713
Reviewed-on: https://review.whamcloud.com/27817
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
6 years agoLU-6210 utils: Use C99 struct initializer for opt_map 16/27816/4
Steve Guminski [Mon, 22 May 2017 11:53:21 +0000 (07:53 -0400)]
LU-6210 utils: Use C99 struct initializer for opt_map

This patch makes no functional changes.  The opt_map struct
initializer in mount_lustre.c is updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I42532a6f6b3506e1893e137c054139f971241f61
Reviewed-on: https://review.whamcloud.com/27816
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
6 years agoLU-6210 utils: Use C99 struct initializer in mkfs_lustre.c 15/27815/2
Steve Guminski [Fri, 19 May 2017 19:59:17 +0000 (15:59 -0400)]
LU-6210 utils: Use C99 struct initializer in mkfs_lustre.c

This patch makes no functional changes.  The long_opt_start
struct initializer in mkfs_lustre.c is updated to C99 syntax.
The option struct and option string have been renamed to long_opts
and short_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Icbc6be0eeaf6bf369a1b593917115378b2e3e159
Reviewed-on: https://review.whamcloud.com/27815
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
6 years agoLU-6210 utils: Use C99 struct initializer in lustre_rsync.c 14/27814/2
Steve Guminski [Fri, 19 May 2017 19:51:36 +0000 (15:51 -0400)]
LU-6210 utils: Use C99 struct initializer in lustre_c

This patch makes no functional changes.  The long_opts struct
initializer in lustre_rsync.c is updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ic89ee776d5c2578759554446cdc33f5861316130
Reviewed-on: https://review.whamcloud.com/27814
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
6 years agoLU-6210 utils: Use consistent naming for option strings 92/27792/4
Steve Guminski [Fri, 19 May 2017 19:35:45 +0000 (15:35 -0400)]
LU-6210 utils: Use consistent naming for option strings

This patch makes no functional changes.  The option strings in
jt_lfsck_start() and jt_lfsck_stop() are renamed to short_opts
for consistency.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Id29ef63aa2ef62223d72f31374eb0a93badc280e
Reviewed-on: https://review.whamcloud.com/27792
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-6210 utils: Use C99 initializer for lfsck_types_names 91/27791/2
Steve Guminski [Fri, 19 May 2017 19:25:37 +0000 (15:25 -0400)]
LU-6210 utils: Use C99 initializer for lfsck_types_names

This patch makes no functional changes.  The lfsck_types_names
struct initializer in lustre_lfsck.c is updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I8a3b44b56263b00a926550e4122193300c0c399f
Reviewed-on: https://review.whamcloud.com/27791
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lustre_fsck.c 90/27790/2
Steve Guminski [Fri, 19 May 2017 19:17:03 +0000 (15:17 -0400)]
LU-6210 utils: Use C99 struct initializers in lustre_fsck.c

This patch makes no functional changes.  The long_opt_stop and
long_opt_query struct initializers in lustre_lfsck.c are updated
to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ie713904f31208447e4d5741023356aa1a37c0ea3
Reviewed-on: https://review.whamcloud.com/27790
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-6436 llite: NULL pointer dereference in cl_object_top() 77/27777/2
Andriy Skulysh [Thu, 22 Jun 2017 08:08:45 +0000 (11:08 +0300)]
LU-6436 llite: NULL pointer dereference in cl_object_top()

During race condition iget5_locked() returns inode without
I_NEW bit for the second thread. If cl_file_inode_init()
failed for the first thread lli_clob isn't valid for
corresponding fid.

But failed inode is marked bad, check it.

Change-Id: I02e477486e51d2653c8b047a18c54a5528b440c5
Seagate-bug-id: MRP-4035
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: https://review.whamcloud.com/27777
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9558 lnet: kernel socket accept takes new bool agrument 42/27642/2
James Simmons [Wed, 14 Jun 2017 16:42:52 +0000 (12:42 -0400)]
LU-9558 lnet: kernel socket accept takes new bool agrument

During the development of the linux 4.11 kernel it was discovered
that the kernel socket layer could get into lockdep situation. To
handle this a new bool argument was added to the accept member
of struct socket. For LNet we can always pass false.

Change-Id: I420cda95b70cf927b1a6e3493b631bc5a3585d74
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27642
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug@cadentcomputing.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lshowmount.c 32/27532/2
Steve Guminski [Fri, 14 Apr 2017 19:57:55 +0000 (15:57 -0400)]
LU-6210 utils: Use C99 struct initializers in lshowmount.c

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lshowmount.c to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: If30db4ee6a825c326eaf246201f9173fc291a436
Reviewed-on: https://review.whamcloud.com/27532
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lr_reader.c 31/27531/2
Steve Guminski [Fri, 14 Apr 2017 19:56:00 +0000 (15:56 -0400)]
LU-6210 utils: Use C99 struct initializers in lr_reader.c

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lr_reader.c to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ie073206d6049fb63b23a30092b11672f8e7e9fd4
Reviewed-on: https://review.whamcloud.com/27531
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in llverfs.c 30/27530/2
Steve Guminski [Fri, 14 Apr 2017 19:54:46 +0000 (15:54 -0400)]
LU-6210 utils: Use C99 struct initializers in llverfs.c

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates llverfs.c to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ifa3882f435f546715c7a22c31934a9f17b1bfc01
Reviewed-on: https://review.whamcloud.com/27530
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 initializers in lfs_get_ladvice() 27/27527/2
Steve Guminski [Fri, 14 Apr 2017 19:49:11 +0000 (15:49 -0400)]
LU-6210 utils: Use C99 initializers in lfs_get_ladvice()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_get_ladvice() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ida06e903a53aae37c878eebb8707b8062bef59dc
Reviewed-on: https://review.whamcloud.com/27527
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lfs_hsm_state() 25/27525/2
Steve Guminski [Fri, 14 Apr 2017 19:45:59 +0000 (15:45 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_hsm_state()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_hsm_state() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I32f427396e99f1b86d329eb8457883a9aaaed181
Reviewed-on: https://review.whamcloud.com/27525
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lfs_fid2path() 24/27524/2
Steve Guminski [Fri, 14 Apr 2017 19:41:51 +0000 (15:41 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_fid2path()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_fid2path() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I65fb184a54aad142a87dd8f244e99a09a6482b71
Reviewed-on: https://review.whamcloud.com/27524
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in mntdf() 17/27517/2
Steve Guminski [Fri, 14 Apr 2017 19:23:03 +0000 (15:23 -0400)]
LU-6210 utils: Use C99 struct initializers in mntdf()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates mntdf() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ic4e446ecb03695b2526b0890f7bfeb76b1bfee88
Reviewed-on: https://review.whamcloud.com/27517
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lfs_mv() 16/27516/2
Steve Guminski [Fri, 14 Apr 2017 19:19:41 +0000 (15:19 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_mv()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_mv() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I81b9ace65b7f26766173199f6a14d947ccc86158
Reviewed-on: https://review.whamcloud.com/27516
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-6210 utils: Use C99 struct initializers in lfs_find() 13/27513/2
Steve Guminski [Fri, 14 Apr 2017 19:06:45 +0000 (15:06 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_find()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_find() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I646e42ac5173f927c670a41cc56c13a7caa19a1e
Reviewed-on: https://review.whamcloud.com/27513
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-9620 tests: test race between pool_list and pool_remove 05/27505/4
Jadhav Vikram [Thu, 8 Jun 2017 17:41:38 +0000 (23:11 +0530)]
LU-9620 tests: test race between pool_list and pool_remove

Test ost-pools/test_27 added to test race between
pool_list and pool_remove.

Change-Id: I8497836dd6e8b2c20ade3626b7a1f84b98f3a94f
Seagate-bug-id: MRP-3629
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/27505
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9620 lod: protected ost pool count updation 06/27506/6
Jadhav Vikram [Thu, 12 Jan 2017 12:11:14 +0000 (17:41 +0530)]
LU-9620 lod: protected ost pool count updation

ASSERTION(iter->lpi_idx <= ((iter->lpi_pool)->pool_obds.op_count)
caused due to reading of ost pool count is not protected in
pool_proc_next and pool_proc_show, pool_proc_show get called when
op_count was zero.

Fix to protect ost pool count by taking lock at start sequence
function pool_proc_start and released lock in pool_proc_stop.
Rather than using down_read / up_read pairs around pool_proc_next
and pool_proc_show, this changes make sure ost pool data gets
protected throughout sequence operation.

Change-Id: I2954496d52dd703b600e69a494898c6266f026bd
Seagate-bug-id: MRP-3629
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-on: https://review.whamcloud.com/27506
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9564 build: Add server-build for Ubuntu with Kernel 4.4.0 23/27323/11
Martin Schroeder [Tue, 13 Jun 2017 14:42:57 +0000 (10:42 -0400)]
LU-9564 build: Add server-build for Ubuntu with Kernel 4.4.0

This enables compatibility with the current LTS flavours of Ubuntu.
Do note that you need the Xenial HWE Kernel for Ubuntu 14.04.5, as
that distribution originally used a 3.x series Kernel.

The patches have been developed to apply cleanly to the kernel versions
4.4.0-45.66 to 4.4.0-85.108 from the Ubuntu Xenial (and its Trusty backports).

This change also adjusts the Debian scripting to produce the
ldiskfs modules and the server utilities.

To create the server modules run "./configure" with "--enable-server"
and specify "--enable-ldiskfs" and "--with-zfs/-spl" as appropriate.

The call to "make debs" will then produce the server modules and
utils instead of their client versions.

Test-Parameters: trivial
Signed-off-by: Martin Schroeder <martin.h.schroeder@intel.com>
Change-Id: Ia78702da304f735bb932738784f1346be0f39026
Reviewed-on: https://review.whamcloud.com/27323
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-282 tests: remove extra logging from sanity 27 22/27322/3
Andreas Dilger [Mon, 29 May 2017 00:54:37 +0000 (20:54 -0400)]
LU-282 tests: remove extra logging from sanity 27

Remove extra logging from sanity.sh test_27b to avoid confusing
autotest log parsing.

Replace some uses of $SETSTRIPE and $GETSTRIPE with $LFS in these
functions since this was only needed during the ancient transition
from the standalone "lstripe" binary.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I270d2f75cd803bede5776117c9d5aaaa5b3ebbe5
Reviewed-on: https://review.whamcloud.com/27322
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9509 tests: fix compilebench timeout 31/27131/3
Jadhav Vikram [Thu, 9 Feb 2017 06:25:57 +0000 (11:55 +0530)]
LU-9509 tests: fix compilebench timeout

With ext4 and lustre found compilebench always takes more time in
create dir operation whenever previous same test aborted without
deleting directory. As per test board result, found compilebench
test ran from suite parallel-scale-nfs.sh timedout before running
from suite parallel-scale.sh, so 1st timedout compilebench not
clear data, rerunning compilebench again causes timeout.

Fix is creating different directory on each run of compilebench.

Seagate-bug-id: MRP-3753
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Tested-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I9b47e28fe5c638f4167c05fb68e46ec5e90de09e
Reviewed-on: https://review.whamcloud.com/27131
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
6 years agoLU-9511 utils: fix parallel_grouplock test timeout 27/27127/4
Jadhav Vikram [Wed, 15 Feb 2017 04:22:12 +0000 (09:52 +0530)]
LU-9511 utils: fix parallel_grouplock test timeout

In parallel_grouplock/subtest12, when process 2 acquire
grouplock before writing by process 0 there is deadlock
situation.
- Process 0 write will wait till process 2 release GR lock
  (gid=1).
- Process 0 will wait for process 2 to get its lock after write
  completed by process 0 using MPI_RECV and here write already
  waiting for process 2 to release grouplock.
- Process 2 release lock when process 0 notify process 2.
  This issue occured once on local setup, analysed issue through
  coredump and concluded following solution which helps to
  resolve this timeout.

Increased sleep time from microseconds to seconds for process 1
and 2 to make sure process 0 starts writing and get PW lock.

Seagate-bug-id: MRP-3751
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Tested-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I1100b86dba55b2eb01c9ae61ba8f1c8e376090c8
Reviewed-on: https://review.whamcloud.com/27127
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9345 tests: use hsm_remove with --mntpath for deleted files 06/27006/4
Quentin Bouget [Tue, 9 May 2017 12:05:36 +0000 (14:05 +0200)]
LU-9345 tests: use hsm_remove with --mntpath for deleted files

In test_29d of sanity-hsm, to run "lfs hsm_remove" on a file deleted
from Lustre, one has to use the --mntpath option.

Test-Parameters: trivial testlist=sanity-hsm clientcount=3 envdefinitions="ONLY=29d"
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I35865c059e498e1a0ced0cebeac22a8491231e00
Reviewed-on: https://review.whamcloud.com/27006
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9545 lfsck: report "inconsistent" under dryrun mode 06/27606/3
Fan Yong [Tue, 13 Jun 2017 10:52:43 +0000 (18:52 +0800)]
LU-9545 lfsck: report "inconsistent" under dryrun mode

It is confused to report the item as "fixed" under dryrun
mode LFSCK. Instead, report them as "inconsistent".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I22e056d7143a55e0dc06d9a891f4126522b466c9
Reviewed-on: https://review.whamcloud.com/27606
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-9153 llog: consolidate common error checking 11/26211/4
Andreas Dilger [Thu, 9 Mar 2017 08:20:04 +0000 (01:20 -0700)]
LU-9153 llog: consolidate common error checking

Consolidate error checking for corrupted and empty llog files
detected when the llog file is being opened into a single
routine llog_cat_process_common().

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I575343f486e7420c51466f9923b841c37f3ebbe5
Reviewed-on: https://review.whamcloud.com/26211
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9220 gss: support Kerberos auth from unprivileged container 35/26035/6
Sebastien Buisson [Tue, 14 Mar 2017 10:27:10 +0000 (19:27 +0900)]
LU-9220 gss: support Kerberos auth from unprivileged container

When a container runs unprivileged, it cannot get access to /proc.
So, to be able to do the required ioctl to
/proc/fs/lustre/sptlrpc/gss/init_channel in order to negotiate
credentials, delegate this ioctl to a parent thread that does not run
in the container's namespace.
The rest of the authentication process is still carried out in the
container's namespace if relevant.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I445ccc82d9b853775bc0de73323699638f688dab
Reviewed-on: https://review.whamcloud.com/26035
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Kit Westneat <kit.westneat@gmail.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9219 tests: add missing mgs reformat to conf-sanity/56 29/26029/3
Jadhav Vikram [Wed, 15 Mar 2017 17:50:56 +0000 (23:20 +0530)]
LU-9219 tests: add missing mgs reformat to conf-sanity/56

conf-sanity/test_56 timedout while mounting client
the reason of timeout is mounting mds failed with -EADDRINUSE.
Registering mdt to MGS while mounting mds index of server was
already present in MGS configuration database so mds mount
failed with -EADDRINUSE and further test stuck while client
mount so test timedout.

This change adds missing mgs reformat when mgs and mds
separate before starting mdt and osts, This will make sure
index of mdt will not be present in MGS config database.

Seagate-bug-id: MRP-3806
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-by: Alexandr Boyko <alexander.boyko@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Change-Id: Ie7d897534197af7e01d92d29613123a0290ffc4c
Reviewed-on: https://review.whamcloud.com/26029
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alexander Boyko <alexander.boyko@seagate.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9183 llite: add support set_acl method in inode operations 65/25965/13
Dmitry Eremin [Sat, 10 Jun 2017 21:36:52 +0000 (17:36 -0400)]
LU-9183 llite: add support set_acl method in inode operations

Linux kernel v3.14 adds set_acl method to inode operations.
See kernel commit 893d46e443346370cd4ea81d9d35f72952c62a37

Change-Id: Ia40d55364016fafa8633fdaecd317910505f8ad4
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25965
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9183 llite: handle xattr with the xattr_handler infrastructure 40/27240/25
James Simmons [Sun, 18 Jun 2017 22:21:17 +0000 (18:21 -0400)]
LU-9183 llite: handle xattr with the xattr_handler infrastructure

In commit fd50ecaddf8372a1d96e0daeaac0f93cf04e4d42 for the linux
4.9 kernel the {get,set,remove}xattr inode operations were removed
and all xattr operations are now handled by xattr_handlers. For
the upsteam lustre client a port was already done with:

Linux-commit: 1e1f9ff406fd5f6003a5dab2ab5a26c4c5bb8cbd
Linux-commit: 2c563880ea8fdc900693ae372fa07b3894f8ff63

This patch brings this work to the OpenSFS/Intel branch. The
difference is that we also have to support older kernels which
means we need to handle the following changes to the
struct xattr_handler:

Linux-commit: e409de992e3ea3674393465f07cc71c948edd87a
Linux-commit: b296821a7c42fa58baa17513b2b7b30ae66f3336

Lastly the xattr_handler api for RHEL6 is too old for proper
support, lacks the flags in struct xattr_handler. Since this
is the case we have to carry around the pre xattr handler
code. Once RHEL6 support is dropped we can remove that code.

Test-Parameters: testgroup=review-ldiskfs
Test-Parameters: testgroup=review-zfs-part-1
Test-Parameters: testgroup=review-zfs-part-2
Test-Parameters: testgroup=review-dne-part-1
Test-Parameters: testgroup=review-dne-part-2
Test-Parameters: trivial clientselinux testlist=sanity-selinux

Change-Id: I7bdeb57c09f4a252f61737dbdbfa76939df7b5eb
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27240
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9104 obd: Ignore unknown config param while mounting 68/25368/15
Rahul Deshmkuh [Mon, 8 May 2017 17:03:41 +0000 (22:33 +0530)]
LU-9104 obd: Ignore unknown config param while mounting

class_process_proc_param() returns positive value when it encounters
unknown parameters in order to have the below levels process them.
At the very bottom layer the positive value returned by
class_process_proc_param() needs to be dropped.osd_process_config()
missed that, which resulted in target mount failure.

Make sure that osd_process_config() does not return positive value
returned by class_process_proc_param().

Test case has been added to check processing of unknown config
parameters.

Test-Parameters: testlist=conf-sanity
Signed-off-by: Parinay Kondekar <parinay.kondekar@seagate.com>
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Seagate-bug-id: MRP-4162
Change-Id: I068b8f2aee4cee69629efc83745d7cb88aea268c
Reviewed-on: https://review.whamcloud.com/25368
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
6 years agoLU-9750 nrs: some code cleanup in NRS policies 19/25319/3
Emoly Liu [Wed, 8 Feb 2017 06:21:09 +0000 (14:21 +0800)]
LU-9750 nrs: some code cleanup in NRS policies

This patch does some code cleanup in NRS CRR and ORR polices,
including:
- remove the useless NULL checks
- handle errors properly by multiple labels instead of a single one

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Iafe86ac94042547e83c69e4b46ff7bf1ca31f073
Reviewed-on: https://review.whamcloud.com/25319
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9184 ldlm: selective IBITS lock trying 62/25262/11
Mikhal Pershin [Sun, 5 Feb 2017 08:58:28 +0000 (11:58 +0300)]
LU-9184 ldlm: selective IBITS lock trying

There is code to get lock without blocking, e.g. 'try'
to get it. It enqueues lock with all wanted bits and
get it only when all bits are not conflicting.
The new code allows to get as much 'tried' bits as
possible. It also doesn't require extra locking attempt
for that.
Patch intoduces new inodebits field try_bits which contains
optinal bits to get if possible without blocking. Those
bits are checked along with the required bits and only
non-blocked are added to the new IBITS lock.

Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I78d181243556e65d3bd702d332307d0723d5b665
Reviewed-on: https://review.whamcloud.com/25262
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-8066 ptlrpc: migrate ptlrpc proc files to sysfs 73/25073/11
Oleg Drokin [Sat, 8 Apr 2017 22:00:10 +0000 (18:00 -0400)]
LU-8066 ptlrpc: migrate ptlrpc proc files to sysfs

move procfs high_priority_ratio file to sysfs

Move ptlrpc service high_priority_ratio entry from procfs to sysfs.
Currently in use only by ldlm callback service only in
/sys/fs/lustre/ldlm/services/ldlm_cbd

Linux-commit: b40881e579588360242bd3f8f79f978d2a871e0e

move procfs threads* files to sysfs

Move ptlrpc service threads_min, threads_max and threads_running
entries from procfs to sysfs. Currently in use only by ldlm
callback service only in /sys/fs/lustre/ldlm/services/ldlm_cbd

Linux-commit: 673a6796f290fe8079af6a688f20c87e7416bba5

Add infrastructure for sysfs migration

Added necessary plumbing for ptlrpc sysfs integration for registered
services, sysfs directory registration.

Linux-commit: 328676f823b49741675f8ae47643669a31c79558

Add support for server side for this change.

Change-Id: Ia82a11dabd672a0c36083d06303aff2a81f3c48f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/25073
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-8066 obdclass : Add infrastructure for procfs to sysfs migration 20/26020/10
Oleg Drokin [Tue, 2 May 2017 16:37:12 +0000 (12:37 -0400)]
LU-8066 obdclass : Add infrastructure for procfs to sysfs migration

Prepare for procfs to sysfs migration

Add necessary plumbing to register obd types and instances
under /sys/fs/lustre

Linux-commit : 9b8013023cb62360b56c04313687e93a1c2bf3d6

Move common obd proc files to sysfs

This moves uuid display and also underlying fs statistics.

Linux-commit : 0bc36cb06e4bc5e936f2f442535daf557c7a4200

Change-Id: Ieb4e2f60c7003d658126beac071c05f94fa8eab8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/26020
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-8703 libcfs: change CPT estimate algorithm 04/24304/14
Dmitry Eremin [Tue, 13 Jun 2017 00:50:03 +0000 (20:50 -0400)]
LU-8703 libcfs: change CPT estimate algorithm

The main idea to have more CPU partitions is based on KNL experience.
When a thread submit IO for network communication one of threads from
current CPT is used for network stack. Whith high parallelization many
threads become involved in network submission but having less CPU
partitions they will wait until single thread process them from network
queue. So, the bottleneck just moves into network layer in case of
small amount of CPU partitions. My experiments showed that the best
performance was when for each IO thread we have one network thread.
This condition can be provided having 2 real HW cores (without hyper
threads) per CPT. This is exactly what implemented in this patch.

Change CPT estimate algorithm from 2 * (N - 1)^2 < NCPUS <= 2 * N^2
to 2 HW cores per CPT. This is critical for machines with number of
cores different from 2^N.

Current algorithm splits CPTs in KNL:
LNet: HW CPU cores: 272, npartitions: 16
cpu_partition_table=
0       : 0-4,68-71,136-139,204-207
1       : 5-9,73-76,141-144,209-212
2       : 10-14,78-81,146-149,214-217
3       : 15-17,72,77,83-85,140,145,151-153,208,219-221
4       : 18-21,82,86-88,150,154-156,213,218,222-224
5       : 22-26,90-93,158-161,226-229
6       : 27-31,95-98,163-166,231-234
7       : 32-35,89,100-103,168-171,236-239
8       : 36-38,94,99,104-105,157,162,167,172-173,225,230,235,240-241
9       : 39-43,107-110,175-178,243-246
10      : 44-48,112-115,180-183,248-251
11      : 49-51,106,111,117-119,174,179,185-187,242,253-255
12      : 52-55,116,120-122,184,188-190,247,252,256-258
13      : 56-60,124-127,192-195,260-263
14      : 61-65,129-132,197-200,265-268
15      : 66-67,123,128,133-135,191,196,201-203,259,264,269-271

New algorithm will split CPTs in KNL:
LNet: HW CPU cores: 272, npartitions: 34
cpu_partition_table=
0       : 0-1,68-69,136-137,204-205
1       : 2-3,70-71,138-139,206-207
2       : 4-5,72-73,140-141,208-209
3       : 6-7,74-75,142-143,210-211
4       : 8-9,76-77,144-145,212-213
5       : 10-11,78-79,146-147,214-215
6       : 12-13,80-81,148-149,216-217
7       : 14-15,82-83,150-151,218-219
8       : 16-17,84-85,152-153,220-221
9       : 18-19,86-87,154-155,222-223
10      : 20-21,88-89,156-157,224-225
11      : 22-23,90-91,158-159,226-227
12      : 24-25,92-93,160-161,228-229
13      : 26-27,94-95,162-163,230-231
14      : 28-29,96-97,164-165,232-233
15      : 30-31,98-99,166-167,234-235
16      : 32-33,100-101,168-169,236-237
17      : 34-35,102-103,170-171,238-239
18      : 36-37,104-105,172-173,240-241
19      : 38-39,106-107,174-175,242-243
20      : 40-41,108-109,176-177,244-245
21      : 42-43,110-111,178-179,246-247
22      : 44-45,112-113,180-181,248-249
23      : 46-47,114-115,182-183,250-251
24      : 48-49,116-117,184-185,252-253
25      : 50-51,118-119,186-187,254-255
26      : 52-53,120-121,188-189,256-257
27      : 54-55,122-123,190-191,258-259
28      : 56-57,124-125,192-193,260-261
29      : 58-59,126-127,194-195,262-263
30      : 60-61,128-129,196-197,264-265
31      : 62-63,130-131,198-199,266-267
32      : 64-65,132-133,200-201,268-269
33      : 66-67,134-135,202-203,270-271

'N' pattern in KNL works is not always good.
in flat mode it will be one CPT with all CPUs inside.

in SNC-4 mode:
cpu_partition_table=
0       : 0-17,68-85,136-153,204-221
1       : 18-35,86-103,154-171,222-239
2       : 36-51,104-119,172-187,240-255
3       : 52-67,120-135,188-203,256-271

Change-Id: I07d1fa2e490dd5720497d438027f128df5f01773
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/24304
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-6671 wireshark: Process mds_md entries 61/18061/6
Nathaniel Clark [Wed, 20 Jan 2016 20:33:07 +0000 (15:33 -0500)]
LU-6671 wireshark: Process mds_md entries

In LDLM_ENQUEUE and MDS_GETATTR responses, process the v1 and v3
versions of the lov_mds_md structure.

Test-Parameters: trivial
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I7bc75d6ae0eb7fe14c805bd0488369dd605521eb
Reviewed-on: https://review.whamcloud.com/18061
Reviewed-by: Doug Oucharek <doug@cadentcomputing.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9725 lwp: wait on deregister 87/27987/2
Niu Yawei [Tue, 11 Jul 2017 02:26:12 +0000 (22:26 -0400)]
LU-9725 lwp: wait on deregister

When lustre_deregister_lwp_item() is being called, it should wait
for the inflight callback done before moving on to free the data
used by the callback.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9c27a0ae4c765147fd183b78bf3693a66e7511dc
Reviewed-on: https://review.whamcloud.com/27987
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9753 ofd: 64-bits diff variable to avoid overflow 75/27975/2
Fan Yong [Fri, 7 Jul 2017 15:24:14 +0000 (23:24 +0800)]
LU-9753 ofd: 64-bits diff variable to avoid overflow

In ofd_create_hdl(), the logic will compare the OST stroed LAST_ID
with the MDT given one: if the difference exceeds some threshold,
then it will trust the OST LAST_ID directly and reset the MDT side
value with the OST one. Otherwise, the orphan OST-objects will be
destroyed.

Unfortunately, both the OST stored LAST_ID and MDT given one are
64 bits, but the @diff variable is only 32 bits, and if the OST
side value is too larger than the MDT side, then the @diff will
overflow. That will misguide the OST to destroy useful OST-objects
by wrong. This patch change the @diff as 64 bits variable.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If75899cbab5754be4ede226e0463ba5f69d70e3d
Reviewed-on: https://review.whamcloud.com/27975
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-9740 ldiskfs: more credits for non-append write 47/27947/2
Fan Yong [Thu, 6 Jul 2017 00:49:32 +0000 (08:49 +0800)]
LU-9740 ldiskfs: more credits for non-append write

As code comment explained: for not append write, the split
may need to modify existing blocks moving entries into the
new ones. That needs more journal credits. The old logic
handled it as append case by typo.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I277f144ec056bb2f07ffd5e5ce19d9a6eee8e0ef
Reviewed-on: https://review.whamcloud.com/27947
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-9657 llapi: check if the file layout is composite 52/27752/4
Emoly Liu [Wed, 28 Jun 2017 09:28:34 +0000 (17:28 +0800)]
LU-9657 llapi: check if the file layout is composite

Add a new API llapi_layout_is_composite() to check if the file
layout is composite.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ie7d60f95feb9dfde5fbf52ea65ebd26b925bd33d
Reviewed-on: https://review.whamcloud.com/27752
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-3719 ldiskfs: adjust s_mb_prealloc_table_size correctly 48/27748/3
Bobi Jam [Tue, 20 Jun 2017 09:44:01 +0000 (17:44 +0800)]
LU-3719 ldiskfs: adjust s_mb_prealloc_table_size correctly

When mb prealloc table item value is not valid, the
s_mb_prealloc_table_size should be adjust accordingly.
Port patch to rhel7 & sles12.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5aa3a32a6a3aedc70160409c6443746fd2ccbbc9
Reviewed-on: https://review.whamcloud.com/27748
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5106 readdir: improve striped readdir 63/27663/6
Lai Siyao [Thu, 15 Jun 2017 16:03:10 +0000 (00:03 +0800)]
LU-5106 readdir: improve striped readdir

Striped directory needs to build its directory page from sub-stripe
directory pages, current code iterate sub stripe directory pages
to search dirent for each hash, this is inefficient, as may cause
statahead fail because statahead thread is slow in readdir, while
'ls' is faster and can't find cached statahead entries, and finally
cause statahead fail.

This patch introduces a struct lmv_dir_ctxt which saves dir page and
current dirent for all stripes, to find the dirent which has the
closest hash value it only needs to compare the dirent of all stripes,
and then pop this dirent from its stripe, until all stripes reache
the end.

This patch contains another fix: previously LDP_COLLIDE is set by
default, change to set dir page end hash 'ldp_hash_end' to hash
of next dirent, and only set LDP_COLLIDE when 'ldp_hash_end' equals
last dirent hash 'lde_hash'. We should avoid dir hash collision.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I99c33ea3c55772d3bb77571dcdc67312d386fa27
Reviewed-on: https://review.whamcloud.com/27663
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5106 statahead: support striped directory 83/27683/3
Lai Siyao [Thu, 15 Jun 2017 17:09:12 +0000 (01:09 +0800)]
LU-5106 statahead: support striped directory

Current statahead will getattr by fid, but for striped directory,
it needs to fix its parent fid to stripe fid, which needs file
name, so pack file name in md_op_data.

Remove check for remote dir check in lmv_intent_getattr_async(),
which is not necessary, and not checking it won't cause anything
wrong, while the code is simpler.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I275d866cb1ea157ef2b594ded0cc342264f0382c
Reviewed-on: https://review.whamcloud.com/27683
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
6 years agoLU-6142 selftest: remove typedefs from lnet selftest 98/27298/8
James Simmons [Mon, 19 Jun 2017 22:41:11 +0000 (18:41 -0400)]
LU-6142 selftest: remove typedefs from lnet selftest

Change the typedefs in lnet selftest to proper structures.
Several other style changes to fix checkpatch issues with
code impacted by tyepdef change.

Test-Parameters: testlist=lnet-selftest

Change-Id: I941d460566673ccb5bccad9f3506f90ede47ed67
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27298
Reviewed-by: Doug Oucharek <doug@cadentcomputing.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>