Whamcloud - gitweb
fs/lustre-release.git
3 years agoLU-6210 utils: Use C99 initializers in lfs_get_ladvice() 27/27527/2
Steve Guminski [Fri, 14 Apr 2017 19:49:11 +0000 (15:49 -0400)]
LU-6210 utils: Use C99 initializers in lfs_get_ladvice()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_get_ladvice() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ida06e903a53aae37c878eebb8707b8062bef59dc
Reviewed-on: https://review.whamcloud.com/27527
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in lfs_hsm_state() 25/27525/2
Steve Guminski [Fri, 14 Apr 2017 19:45:59 +0000 (15:45 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_hsm_state()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_hsm_state() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I32f427396e99f1b86d329eb8457883a9aaaed181
Reviewed-on: https://review.whamcloud.com/27525
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in lfs_fid2path() 24/27524/2
Steve Guminski [Fri, 14 Apr 2017 19:41:51 +0000 (15:41 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_fid2path()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_fid2path() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I65fb184a54aad142a87dd8f244e99a09a6482b71
Reviewed-on: https://review.whamcloud.com/27524
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in mntdf() 17/27517/2
Steve Guminski [Fri, 14 Apr 2017 19:23:03 +0000 (15:23 -0400)]
LU-6210 utils: Use C99 struct initializers in mntdf()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates mntdf() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ic4e446ecb03695b2526b0890f7bfeb76b1bfee88
Reviewed-on: https://review.whamcloud.com/27517
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in lfs_mv() 16/27516/2
Steve Guminski [Fri, 14 Apr 2017 19:19:41 +0000 (15:19 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_mv()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_mv() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I81b9ace65b7f26766173199f6a14d947ccc86158
Reviewed-on: https://review.whamcloud.com/27516
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in lfs_find() 13/27513/2
Steve Guminski [Fri, 14 Apr 2017 19:06:45 +0000 (15:06 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_find()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_find() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I646e42ac5173f927c670a41cc56c13a7caa19a1e
Reviewed-on: https://review.whamcloud.com/27513
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-9620 tests: test race between pool_list and pool_remove 05/27505/4
Jadhav Vikram [Thu, 8 Jun 2017 17:41:38 +0000 (23:11 +0530)]
LU-9620 tests: test race between pool_list and pool_remove

Test ost-pools/test_27 added to test race between
pool_list and pool_remove.

Change-Id: I8497836dd6e8b2c20ade3626b7a1f84b98f3a94f
Seagate-bug-id: MRP-3629
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/27505
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9620 lod: protected ost pool count updation 06/27506/6
Jadhav Vikram [Thu, 12 Jan 2017 12:11:14 +0000 (17:41 +0530)]
LU-9620 lod: protected ost pool count updation

ASSERTION(iter->lpi_idx <= ((iter->lpi_pool)->pool_obds.op_count)
caused due to reading of ost pool count is not protected in
pool_proc_next and pool_proc_show, pool_proc_show get called when
op_count was zero.

Fix to protect ost pool count by taking lock at start sequence
function pool_proc_start and released lock in pool_proc_stop.
Rather than using down_read / up_read pairs around pool_proc_next
and pool_proc_show, this changes make sure ost pool data gets
protected throughout sequence operation.

Change-Id: I2954496d52dd703b600e69a494898c6266f026bd
Seagate-bug-id: MRP-3629
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-on: https://review.whamcloud.com/27506
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9564 build: Add server-build for Ubuntu with Kernel 4.4.0 23/27323/11
Martin Schroeder [Tue, 13 Jun 2017 14:42:57 +0000 (10:42 -0400)]
LU-9564 build: Add server-build for Ubuntu with Kernel 4.4.0

This enables compatibility with the current LTS flavours of Ubuntu.
Do note that you need the Xenial HWE Kernel for Ubuntu 14.04.5, as
that distribution originally used a 3.x series Kernel.

The patches have been developed to apply cleanly to the kernel versions
4.4.0-45.66 to 4.4.0-85.108 from the Ubuntu Xenial (and its Trusty backports).

This change also adjusts the Debian scripting to produce the
ldiskfs modules and the server utilities.

To create the server modules run "./configure" with "--enable-server"
and specify "--enable-ldiskfs" and "--with-zfs/-spl" as appropriate.

The call to "make debs" will then produce the server modules and
utils instead of their client versions.

Test-Parameters: trivial
Signed-off-by: Martin Schroeder <martin.h.schroeder@intel.com>
Change-Id: Ia78702da304f735bb932738784f1346be0f39026
Reviewed-on: https://review.whamcloud.com/27323
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-282 tests: remove extra logging from sanity 27 22/27322/3
Andreas Dilger [Mon, 29 May 2017 00:54:37 +0000 (20:54 -0400)]
LU-282 tests: remove extra logging from sanity 27

Remove extra logging from sanity.sh test_27b to avoid confusing
autotest log parsing.

Replace some uses of $SETSTRIPE and $GETSTRIPE with $LFS in these
functions since this was only needed during the ancient transition
from the standalone "lstripe" binary.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I270d2f75cd803bede5776117c9d5aaaa5b3ebbe5
Reviewed-on: https://review.whamcloud.com/27322
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9509 tests: fix compilebench timeout 31/27131/3
Jadhav Vikram [Thu, 9 Feb 2017 06:25:57 +0000 (11:55 +0530)]
LU-9509 tests: fix compilebench timeout

With ext4 and lustre found compilebench always takes more time in
create dir operation whenever previous same test aborted without
deleting directory. As per test board result, found compilebench
test ran from suite parallel-scale-nfs.sh timedout before running
from suite parallel-scale.sh, so 1st timedout compilebench not
clear data, rerunning compilebench again causes timeout.

Fix is creating different directory on each run of compilebench.

Seagate-bug-id: MRP-3753
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Tested-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I9b47e28fe5c638f4167c05fb68e46ec5e90de09e
Reviewed-on: https://review.whamcloud.com/27131
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
3 years agoLU-9511 utils: fix parallel_grouplock test timeout 27/27127/4
Jadhav Vikram [Wed, 15 Feb 2017 04:22:12 +0000 (09:52 +0530)]
LU-9511 utils: fix parallel_grouplock test timeout

In parallel_grouplock/subtest12, when process 2 acquire
grouplock before writing by process 0 there is deadlock
situation.
- Process 0 write will wait till process 2 release GR lock
  (gid=1).
- Process 0 will wait for process 2 to get its lock after write
  completed by process 0 using MPI_RECV and here write already
  waiting for process 2 to release grouplock.
- Process 2 release lock when process 0 notify process 2.
  This issue occured once on local setup, analysed issue through
  coredump and concluded following solution which helps to
  resolve this timeout.

Increased sleep time from microseconds to seconds for process 1
and 2 to make sure process 0 starts writing and get PW lock.

Seagate-bug-id: MRP-3751
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Tested-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I1100b86dba55b2eb01c9ae61ba8f1c8e376090c8
Reviewed-on: https://review.whamcloud.com/27127
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9345 tests: use hsm_remove with --mntpath for deleted files 06/27006/4
Quentin Bouget [Tue, 9 May 2017 12:05:36 +0000 (14:05 +0200)]
LU-9345 tests: use hsm_remove with --mntpath for deleted files

In test_29d of sanity-hsm, to run "lfs hsm_remove" on a file deleted
from Lustre, one has to use the --mntpath option.

Test-Parameters: trivial testlist=sanity-hsm clientcount=3 envdefinitions="ONLY=29d"
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I35865c059e498e1a0ced0cebeac22a8491231e00
Reviewed-on: https://review.whamcloud.com/27006
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9545 lfsck: report "inconsistent" under dryrun mode 06/27606/3
Fan Yong [Tue, 13 Jun 2017 10:52:43 +0000 (18:52 +0800)]
LU-9545 lfsck: report "inconsistent" under dryrun mode

It is confused to report the item as "fixed" under dryrun
mode LFSCK. Instead, report them as "inconsistent".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I22e056d7143a55e0dc06d9a891f4126522b466c9
Reviewed-on: https://review.whamcloud.com/27606
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
3 years agoLU-9153 llog: consolidate common error checking 11/26211/4
Andreas Dilger [Thu, 9 Mar 2017 08:20:04 +0000 (01:20 -0700)]
LU-9153 llog: consolidate common error checking

Consolidate error checking for corrupted and empty llog files
detected when the llog file is being opened into a single
routine llog_cat_process_common().

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I575343f486e7420c51466f9923b841c37f3ebbe5
Reviewed-on: https://review.whamcloud.com/26211
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9220 gss: support Kerberos auth from unprivileged container 35/26035/6
Sebastien Buisson [Tue, 14 Mar 2017 10:27:10 +0000 (19:27 +0900)]
LU-9220 gss: support Kerberos auth from unprivileged container

When a container runs unprivileged, it cannot get access to /proc.
So, to be able to do the required ioctl to
/proc/fs/lustre/sptlrpc/gss/init_channel in order to negotiate
credentials, delegate this ioctl to a parent thread that does not run
in the container's namespace.
The rest of the authentication process is still carried out in the
container's namespace if relevant.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I445ccc82d9b853775bc0de73323699638f688dab
Reviewed-on: https://review.whamcloud.com/26035
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Kit Westneat <kit.westneat@gmail.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9219 tests: add missing mgs reformat to conf-sanity/56 29/26029/3
Jadhav Vikram [Wed, 15 Mar 2017 17:50:56 +0000 (23:20 +0530)]
LU-9219 tests: add missing mgs reformat to conf-sanity/56

conf-sanity/test_56 timedout while mounting client
the reason of timeout is mounting mds failed with -EADDRINUSE.
Registering mdt to MGS while mounting mds index of server was
already present in MGS configuration database so mds mount
failed with -EADDRINUSE and further test stuck while client
mount so test timedout.

This change adds missing mgs reformat when mgs and mds
separate before starting mdt and osts, This will make sure
index of mdt will not be present in MGS config database.

Seagate-bug-id: MRP-3806
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-by: Alexandr Boyko <alexander.boyko@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Change-Id: Ie7d897534197af7e01d92d29613123a0290ffc4c
Reviewed-on: https://review.whamcloud.com/26029
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alexander Boyko <alexander.boyko@seagate.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9183 llite: add support set_acl method in inode operations 65/25965/13
Dmitry Eremin [Sat, 10 Jun 2017 21:36:52 +0000 (17:36 -0400)]
LU-9183 llite: add support set_acl method in inode operations

Linux kernel v3.14 adds set_acl method to inode operations.
See kernel commit 893d46e443346370cd4ea81d9d35f72952c62a37

Change-Id: Ia40d55364016fafa8633fdaecd317910505f8ad4
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25965
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9183 llite: handle xattr with the xattr_handler infrastructure 40/27240/25
James Simmons [Sun, 18 Jun 2017 22:21:17 +0000 (18:21 -0400)]
LU-9183 llite: handle xattr with the xattr_handler infrastructure

In commit fd50ecaddf8372a1d96e0daeaac0f93cf04e4d42 for the linux
4.9 kernel the {get,set,remove}xattr inode operations were removed
and all xattr operations are now handled by xattr_handlers. For
the upsteam lustre client a port was already done with:

Linux-commit: 1e1f9ff406fd5f6003a5dab2ab5a26c4c5bb8cbd
Linux-commit: 2c563880ea8fdc900693ae372fa07b3894f8ff63

This patch brings this work to the OpenSFS/Intel branch. The
difference is that we also have to support older kernels which
means we need to handle the following changes to the
struct xattr_handler:

Linux-commit: e409de992e3ea3674393465f07cc71c948edd87a
Linux-commit: b296821a7c42fa58baa17513b2b7b30ae66f3336

Lastly the xattr_handler api for RHEL6 is too old for proper
support, lacks the flags in struct xattr_handler. Since this
is the case we have to carry around the pre xattr handler
code. Once RHEL6 support is dropped we can remove that code.

Test-Parameters: testgroup=review-ldiskfs
Test-Parameters: testgroup=review-zfs-part-1
Test-Parameters: testgroup=review-zfs-part-2
Test-Parameters: testgroup=review-dne-part-1
Test-Parameters: testgroup=review-dne-part-2
Test-Parameters: trivial clientselinux testlist=sanity-selinux

Change-Id: I7bdeb57c09f4a252f61737dbdbfa76939df7b5eb
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27240
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9104 obd: Ignore unknown config param while mounting 68/25368/15
Rahul Deshmkuh [Mon, 8 May 2017 17:03:41 +0000 (22:33 +0530)]
LU-9104 obd: Ignore unknown config param while mounting

class_process_proc_param() returns positive value when it encounters
unknown parameters in order to have the below levels process them.
At the very bottom layer the positive value returned by
class_process_proc_param() needs to be dropped.osd_process_config()
missed that, which resulted in target mount failure.

Make sure that osd_process_config() does not return positive value
returned by class_process_proc_param().

Test case has been added to check processing of unknown config
parameters.

Test-Parameters: testlist=conf-sanity
Signed-off-by: Parinay Kondekar <parinay.kondekar@seagate.com>
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Seagate-bug-id: MRP-4162
Change-Id: I068b8f2aee4cee69629efc83745d7cb88aea268c
Reviewed-on: https://review.whamcloud.com/25368
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
3 years agoLU-9750 nrs: some code cleanup in NRS policies 19/25319/3
Emoly Liu [Wed, 8 Feb 2017 06:21:09 +0000 (14:21 +0800)]
LU-9750 nrs: some code cleanup in NRS policies

This patch does some code cleanup in NRS CRR and ORR polices,
including:
- remove the useless NULL checks
- handle errors properly by multiple labels instead of a single one

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Iafe86ac94042547e83c69e4b46ff7bf1ca31f073
Reviewed-on: https://review.whamcloud.com/25319
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9184 ldlm: selective IBITS lock trying 62/25262/11
Mikhal Pershin [Sun, 5 Feb 2017 08:58:28 +0000 (11:58 +0300)]
LU-9184 ldlm: selective IBITS lock trying

There is code to get lock without blocking, e.g. 'try'
to get it. It enqueues lock with all wanted bits and
get it only when all bits are not conflicting.
The new code allows to get as much 'tried' bits as
possible. It also doesn't require extra locking attempt
for that.
Patch intoduces new inodebits field try_bits which contains
optinal bits to get if possible without blocking. Those
bits are checked along with the required bits and only
non-blocked are added to the new IBITS lock.

Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I78d181243556e65d3bd702d332307d0723d5b665
Reviewed-on: https://review.whamcloud.com/25262
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8066 ptlrpc: migrate ptlrpc proc files to sysfs 73/25073/11
Oleg Drokin [Sat, 8 Apr 2017 22:00:10 +0000 (18:00 -0400)]
LU-8066 ptlrpc: migrate ptlrpc proc files to sysfs

move procfs high_priority_ratio file to sysfs

Move ptlrpc service high_priority_ratio entry from procfs to sysfs.
Currently in use only by ldlm callback service only in
/sys/fs/lustre/ldlm/services/ldlm_cbd

Linux-commit: b40881e579588360242bd3f8f79f978d2a871e0e

move procfs threads* files to sysfs

Move ptlrpc service threads_min, threads_max and threads_running
entries from procfs to sysfs. Currently in use only by ldlm
callback service only in /sys/fs/lustre/ldlm/services/ldlm_cbd

Linux-commit: 673a6796f290fe8079af6a688f20c87e7416bba5

Add infrastructure for sysfs migration

Added necessary plumbing for ptlrpc sysfs integration for registered
services, sysfs directory registration.

Linux-commit: 328676f823b49741675f8ae47643669a31c79558

Add support for server side for this change.

Change-Id: Ia82a11dabd672a0c36083d06303aff2a81f3c48f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/25073
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
3 years agoLU-8066 obdclass : Add infrastructure for procfs to sysfs migration 20/26020/10
Oleg Drokin [Tue, 2 May 2017 16:37:12 +0000 (12:37 -0400)]
LU-8066 obdclass : Add infrastructure for procfs to sysfs migration

Prepare for procfs to sysfs migration

Add necessary plumbing to register obd types and instances
under /sys/fs/lustre

Linux-commit : 9b8013023cb62360b56c04313687e93a1c2bf3d6

Move common obd proc files to sysfs

This moves uuid display and also underlying fs statistics.

Linux-commit : 0bc36cb06e4bc5e936f2f442535daf557c7a4200

Change-Id: Ieb4e2f60c7003d658126beac071c05f94fa8eab8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/26020
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
3 years agoLU-8703 libcfs: change CPT estimate algorithm 04/24304/14
Dmitry Eremin [Tue, 13 Jun 2017 00:50:03 +0000 (20:50 -0400)]
LU-8703 libcfs: change CPT estimate algorithm

The main idea to have more CPU partitions is based on KNL experience.
When a thread submit IO for network communication one of threads from
current CPT is used for network stack. Whith high parallelization many
threads become involved in network submission but having less CPU
partitions they will wait until single thread process them from network
queue. So, the bottleneck just moves into network layer in case of
small amount of CPU partitions. My experiments showed that the best
performance was when for each IO thread we have one network thread.
This condition can be provided having 2 real HW cores (without hyper
threads) per CPT. This is exactly what implemented in this patch.

Change CPT estimate algorithm from 2 * (N - 1)^2 < NCPUS <= 2 * N^2
to 2 HW cores per CPT. This is critical for machines with number of
cores different from 2^N.

Current algorithm splits CPTs in KNL:
LNet: HW CPU cores: 272, npartitions: 16
cpu_partition_table=
0       : 0-4,68-71,136-139,204-207
1       : 5-9,73-76,141-144,209-212
2       : 10-14,78-81,146-149,214-217
3       : 15-17,72,77,83-85,140,145,151-153,208,219-221
4       : 18-21,82,86-88,150,154-156,213,218,222-224
5       : 22-26,90-93,158-161,226-229
6       : 27-31,95-98,163-166,231-234
7       : 32-35,89,100-103,168-171,236-239
8       : 36-38,94,99,104-105,157,162,167,172-173,225,230,235,240-241
9       : 39-43,107-110,175-178,243-246
10      : 44-48,112-115,180-183,248-251
11      : 49-51,106,111,117-119,174,179,185-187,242,253-255
12      : 52-55,116,120-122,184,188-190,247,252,256-258
13      : 56-60,124-127,192-195,260-263
14      : 61-65,129-132,197-200,265-268
15      : 66-67,123,128,133-135,191,196,201-203,259,264,269-271

New algorithm will split CPTs in KNL:
LNet: HW CPU cores: 272, npartitions: 34
cpu_partition_table=
0       : 0-1,68-69,136-137,204-205
1       : 2-3,70-71,138-139,206-207
2       : 4-5,72-73,140-141,208-209
3       : 6-7,74-75,142-143,210-211
4       : 8-9,76-77,144-145,212-213
5       : 10-11,78-79,146-147,214-215
6       : 12-13,80-81,148-149,216-217
7       : 14-15,82-83,150-151,218-219
8       : 16-17,84-85,152-153,220-221
9       : 18-19,86-87,154-155,222-223
10      : 20-21,88-89,156-157,224-225
11      : 22-23,90-91,158-159,226-227
12      : 24-25,92-93,160-161,228-229
13      : 26-27,94-95,162-163,230-231
14      : 28-29,96-97,164-165,232-233
15      : 30-31,98-99,166-167,234-235
16      : 32-33,100-101,168-169,236-237
17      : 34-35,102-103,170-171,238-239
18      : 36-37,104-105,172-173,240-241
19      : 38-39,106-107,174-175,242-243
20      : 40-41,108-109,176-177,244-245
21      : 42-43,110-111,178-179,246-247
22      : 44-45,112-113,180-181,248-249
23      : 46-47,114-115,182-183,250-251
24      : 48-49,116-117,184-185,252-253
25      : 50-51,118-119,186-187,254-255
26      : 52-53,120-121,188-189,256-257
27      : 54-55,122-123,190-191,258-259
28      : 56-57,124-125,192-193,260-261
29      : 58-59,126-127,194-195,262-263
30      : 60-61,128-129,196-197,264-265
31      : 62-63,130-131,198-199,266-267
32      : 64-65,132-133,200-201,268-269
33      : 66-67,134-135,202-203,270-271

'N' pattern in KNL works is not always good.
in flat mode it will be one CPT with all CPUs inside.

in SNC-4 mode:
cpu_partition_table=
0       : 0-17,68-85,136-153,204-221
1       : 18-35,86-103,154-171,222-239
2       : 36-51,104-119,172-187,240-255
3       : 52-67,120-135,188-203,256-271

Change-Id: I07d1fa2e490dd5720497d438027f128df5f01773
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/24304
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-6671 wireshark: Process mds_md entries 61/18061/6
Nathaniel Clark [Wed, 20 Jan 2016 20:33:07 +0000 (15:33 -0500)]
LU-6671 wireshark: Process mds_md entries

In LDLM_ENQUEUE and MDS_GETATTR responses, process the v1 and v3
versions of the lov_mds_md structure.

Test-Parameters: trivial
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I7bc75d6ae0eb7fe14c805bd0488369dd605521eb
Reviewed-on: https://review.whamcloud.com/18061
Reviewed-by: Doug Oucharek <doug@cadentcomputing.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9725 lwp: wait on deregister 87/27987/2
Niu Yawei [Tue, 11 Jul 2017 02:26:12 +0000 (22:26 -0400)]
LU-9725 lwp: wait on deregister

When lustre_deregister_lwp_item() is being called, it should wait
for the inflight callback done before moving on to free the data
used by the callback.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9c27a0ae4c765147fd183b78bf3693a66e7511dc
Reviewed-on: https://review.whamcloud.com/27987
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9753 ofd: 64-bits diff variable to avoid overflow 75/27975/2
Fan Yong [Fri, 7 Jul 2017 15:24:14 +0000 (23:24 +0800)]
LU-9753 ofd: 64-bits diff variable to avoid overflow

In ofd_create_hdl(), the logic will compare the OST stroed LAST_ID
with the MDT given one: if the difference exceeds some threshold,
then it will trust the OST LAST_ID directly and reset the MDT side
value with the OST one. Otherwise, the orphan OST-objects will be
destroyed.

Unfortunately, both the OST stored LAST_ID and MDT given one are
64 bits, but the @diff variable is only 32 bits, and if the OST
side value is too larger than the MDT side, then the @diff will
overflow. That will misguide the OST to destroy useful OST-objects
by wrong. This patch change the @diff as 64 bits variable.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If75899cbab5754be4ede226e0463ba5f69d70e3d
Reviewed-on: https://review.whamcloud.com/27975
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9740 ldiskfs: more credits for non-append write 47/27947/2
Fan Yong [Thu, 6 Jul 2017 00:49:32 +0000 (08:49 +0800)]
LU-9740 ldiskfs: more credits for non-append write

As code comment explained: for not append write, the split
may need to modify existing blocks moving entries into the
new ones. That needs more journal credits. The old logic
handled it as append case by typo.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I277f144ec056bb2f07ffd5e5ce19d9a6eee8e0ef
Reviewed-on: https://review.whamcloud.com/27947
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-9657 llapi: check if the file layout is composite 52/27752/4
Emoly Liu [Wed, 28 Jun 2017 09:28:34 +0000 (17:28 +0800)]
LU-9657 llapi: check if the file layout is composite

Add a new API llapi_layout_is_composite() to check if the file
layout is composite.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ie7d60f95feb9dfde5fbf52ea65ebd26b925bd33d
Reviewed-on: https://review.whamcloud.com/27752
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-3719 ldiskfs: adjust s_mb_prealloc_table_size correctly 48/27748/3
Bobi Jam [Tue, 20 Jun 2017 09:44:01 +0000 (17:44 +0800)]
LU-3719 ldiskfs: adjust s_mb_prealloc_table_size correctly

When mb prealloc table item value is not valid, the
s_mb_prealloc_table_size should be adjust accordingly.
Port patch to rhel7 & sles12.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5aa3a32a6a3aedc70160409c6443746fd2ccbbc9
Reviewed-on: https://review.whamcloud.com/27748
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5106 readdir: improve striped readdir 63/27663/6
Lai Siyao [Thu, 15 Jun 2017 16:03:10 +0000 (00:03 +0800)]
LU-5106 readdir: improve striped readdir

Striped directory needs to build its directory page from sub-stripe
directory pages, current code iterate sub stripe directory pages
to search dirent for each hash, this is inefficient, as may cause
statahead fail because statahead thread is slow in readdir, while
'ls' is faster and can't find cached statahead entries, and finally
cause statahead fail.

This patch introduces a struct lmv_dir_ctxt which saves dir page and
current dirent for all stripes, to find the dirent which has the
closest hash value it only needs to compare the dirent of all stripes,
and then pop this dirent from its stripe, until all stripes reache
the end.

This patch contains another fix: previously LDP_COLLIDE is set by
default, change to set dir page end hash 'ldp_hash_end' to hash
of next dirent, and only set LDP_COLLIDE when 'ldp_hash_end' equals
last dirent hash 'lde_hash'. We should avoid dir hash collision.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I99c33ea3c55772d3bb77571dcdc67312d386fa27
Reviewed-on: https://review.whamcloud.com/27663
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5106 statahead: support striped directory 83/27683/3
Lai Siyao [Thu, 15 Jun 2017 17:09:12 +0000 (01:09 +0800)]
LU-5106 statahead: support striped directory

Current statahead will getattr by fid, but for striped directory,
it needs to fix its parent fid to stripe fid, which needs file
name, so pack file name in md_op_data.

Remove check for remote dir check in lmv_intent_getattr_async(),
which is not necessary, and not checking it won't cause anything
wrong, while the code is simpler.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I275d866cb1ea157ef2b594ded0cc342264f0382c
Reviewed-on: https://review.whamcloud.com/27683
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
3 years agoLU-6142 selftest: remove typedefs from lnet selftest 98/27298/8
James Simmons [Mon, 19 Jun 2017 22:41:11 +0000 (18:41 -0400)]
LU-6142 selftest: remove typedefs from lnet selftest

Change the typedefs in lnet selftest to proper structures.
Several other style changes to fix checkpatch issues with
code impacted by tyepdef change.

Test-Parameters: testlist=lnet-selftest

Change-Id: I941d460566673ccb5bccad9f3506f90ede47ed67
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27298
Reviewed-by: Doug Oucharek <doug@cadentcomputing.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9442 osp: can't create IDIF fid number > 0xFFFFFFFF 25/27225/6
Sergey Cheremencev [Thu, 15 Jun 2017 05:14:07 +0000 (08:14 +0300)]
LU-9442 osp: can't create IDIF fid number > 0xFFFFFFFF

fid_is_last_id didn't recognize IDIFes. This caused OST
to allocate new sequence despite of MDT still used
initial FID_SEQ_IDIF. Finally allocation failed with -115
returned from osd_check_lma:
osd_check_lma()) lustre-OST0000-osd: FID [0x100000001:0x0:0x0] != self_fid [0x100000000:0x0:0x0]

Patch has several typical "IDIF" fixes. Also it has lov_objid
fix to store all 48 IDIF bits instead of 32.
Finally it changes union fields order in ost_id.
Before the fix oi_fid.f_seq addressed oi_id instead of oi_seq.

Change-Id: Ifbda97a5b228254aedcb050c3d94d2ecb3a9590c
Seagate-bug-id: MRP-4392
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Reviewed-on: https://review.whamcloud.com/27225
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9505 llapi: treat MDT index as a hex number 56/27156/4
Emoly Liu [Wed, 17 May 2017 09:04:44 +0000 (17:04 +0800)]
LU-9505 llapi: treat MDT index as a hex number

Since MDT index is a hex number, "base" in strtol() should be 16.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I50922f9eb5d1095f06a493628ef521d34969a59f
Reviewed-on: https://review.whamcloud.com/27156
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9417 mdc: excessive memory consumption by the xattr cache 87/26887/7
Andrew Perepechko [Fri, 28 Apr 2017 17:20:06 +0000 (20:20 +0300)]
LU-9417 mdc: excessive memory consumption by the xattr cache

The refill operation of the xattr cache does not know the
reply size in advance, so it makes a guess based on
the maxeasize value returned by the MDS.

In practice, it allocates 16 KiB for the common case and
4 MiB for the large xattr case. However, a typical reply
is just a few hundred bytes. Even worse, RHEL6 has a very
bad vmalloc()/vfree() design with global locks.

If we follow the conservative approach, we can prepare a
single memory page for the reply. It is large enough for
any reasonable xattr set and, at the same time, it does
not require multiple page memory reclaim, which can be
costly.

If, for a specific file, the reply is larger than a single
page, the client is prepared to handle that and will fall back
to non-cached xattr code. Indeed, if this happens often and
xattrs are often used to store large values, it makes sense to
disable the xattr cache at all since it wasn't designed for
such [mis]use.

Change-Id: I98d97ffea5c83bccbc9f254717af8d2c0ac4d77f
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: https://review.whamcloud.com/26887
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8062 ptlrpc: increase sleep time in ptlrpc_request_bufs_pack() 15/26815/6
Vitaly Fertman [Wed, 21 Jun 2017 02:52:19 +0000 (22:52 -0400)]
LU-8062 ptlrpc: increase sleep time in ptlrpc_request_bufs_pack()

schedule_timeout() does not necessarily expire. Increase the sleeping
time in ptlrpc_request_bufs_pack() as 2 seconds is too short, given
the 1 second sleep used for recovery-small test_115_write().

Test-Parameters: envdefinitions=ONLY=115 testlist=recovery-small,recovery-small,recovery-small,recovery-small
Test-Parameters: envdefinitions=ONLY=115 testlist=recovery-small,recovery-small,recovery-small,recovery-small
Test-Parameters: envdefinitions=ONLY=115 testlist=recovery-small,recovery-small,recovery-small,recovery-small
Test-Parameters: envdefinitions=ONLY=115 testlist=recovery-small,recovery-small,recovery-small,recovery-small
Signed-off-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: If1b6b147095e01663c0fa9a5b8f93c445d2061ee
Reviewed-on: https://review.whamcloud.com/26815
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
3 years agoLU-8885 tests: zconf_mount_clients() defect 54/24054/4
Arshad Hussain [Fri, 23 Sep 2016 03:12:09 +0000 (23:12 -0400)]
LU-8885 tests: zconf_mount_clients() defect

Presently zconf_mount_clients() returns success(0)
in cases where the 'NFS' mount is already mounted
on a mount point which it is trying to mount again.
In this case, it silently ignores the mount, leading
to testcase failure. This patch addresses this defect
by allowing zconf_mount_clients() to compare that
nothing unexpected is mounted by comparing the result
of mount count with mount count of "type lustre". If
they are unequal the function exists with an error.

Test-Parameters: envdefinitions=PARALLEL_SCALE_EXCEPT=parallel_grouplock \
testlist=sanity,parallel-scale,parallel-scale-nfsv3
Signed-off-by: Arshad Hussain <arshad.hussain@seagate.com>
Change-Id: I55e4b2ef2a18985be4833fca017cc6c6b0c5410f
Seagate-bug-id: MRP-3773
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-by: Ujjwal Lanjewar <ujjwal.lanjewar@seagate.com>
Reviewed-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/24054
Reviewed-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-3285 osc: prepare OSC code to be used from MDC 96/22996/18
Mikhal Pershin [Fri, 4 Dec 2015 20:05:10 +0000 (23:05 +0300)]
LU-3285 osc: prepare OSC code to be used from MDC

Move osc/osc_cl_include.h to include/lustre_osc.h
to be included from both OSC and MDC.

Change-Id: Ic04995d8793cd64b998463aee59c6bf30445b64b
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Reviewed-on: https://review.whamcloud.com/22996
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-9671 nodemap: restore client's IDs for OST_WRITE 80/27680/4
Niu Yawei [Fri, 16 Jun 2017 04:52:50 +0000 (00:52 -0400)]
LU-9671 nodemap: restore client's IDs for OST_WRITE

Client sets overquota flags for certain UID/GID based on the
IDs & flags in OST_WRITE reply, so we need to reply client IDs
instead of mapped IDs.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I375847fa734237f9bcea10fa676e09c471a0fcfb
Reviewed-on: https://review.whamcloud.com/27680
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9654 mdt: fix problem of RAoLU HSM policy 64/27564/4
Li Xi [Mon, 12 Jun 2017 07:32:29 +0000 (15:32 +0800)]
LU-9654 mdt: fix problem of RAoLU HSM policy

mdt_attr_get_complex() clears all known attributes even they are
already valid. So in mdt_handle_last_unlink(), the valid attributes
of HSM or nlink should be checked before calling that function.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I9ba561cadcc40baf5e28172cfda699cdecce7ea8
Reviewed-on: https://review.whamcloud.com/27564
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7631 tests: wait_osts_up waits for MDS precreates 41/27441/2
Andreas Dilger [Mon, 5 Jun 2017 19:22:17 +0000 (13:22 -0600)]
LU-7631 tests: wait_osts_up waits for MDS precreates

Fix wait_osts_up() to wait for the MDS to finish orphan cleanup and
precreate some OST objects so that there isn't a race to get all of
the OSTs available for conf-sanity test_82a.

Test-Parameters: trivial testlist=replay-single,replay-single
Test-Parameters: testlist=conf-sanity,conf-sanity,conf-sanity
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I534f0a1f36c3d00f702684041bfa991a4a3ebbe5
Reviewed-on: https://review.whamcloud.com/27441
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9514 ptlrpc: free reply buffer earlier for open RPC 08/27208/17
Fan Yong [Wed, 14 Jun 2017 09:03:46 +0000 (17:03 +0800)]
LU-9514 ptlrpc: free reply buffer earlier for open RPC

It is unnecessary to keep the reply buffer for open RPC. Replay
related data has already been saved in the request buffer when
the RPC replied. If the open replay really happen, the replay
logic will alloc the reply buffer when needed.

On the other hand, the client always tries to alloc big enough
space to hold the open RPC reply since the client does not exactly
know how much data the server will reply to the client. So the reply
buffer may be quite larger than the real needed. Under such case,
keeping the large reply buffer for the open RPC will occupy a lot
of RAM as to OOM if the are too many open RPCs to be replayed.

This patch frees the reply buffer for the open RPC when only
the replay logic holds the last reference of the RPC.

Test-Parameters: envdefinitions=ONLY=51f testlist=sanity ostfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs
Test-Parameters: envdefinitions=ONLY=51f testlist=sanity ostfilesystemtype=zfs mdtfilesystemtype=zfs
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I1bea2456b8aa4e53a0b65143a48e617f836181a0
Reviewed-on: https://review.whamcloud.com/27208
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-7988 hsm: mark the cdt as stopped when its thread exits 62/22762/28
Frank Zago [Tue, 27 Sep 2016 19:13:29 +0000 (15:13 -0400)]
LU-7988 hsm: mark the cdt as stopped when its thread exits

Use kthread_stop() to stop and join the coordinator thread. Only after
that step can the coordinator state be set to CDT_STOPPED. As a
coordinator doesn't stop instantly, this closes a race if the
coordinator is being restarted at the same time, leaving one thread
shutting down and a new one starting up.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I0a21d0d22403a56a8965441e1b57118073b6f210
Signed-off-by: Ben Evans <bevans@cray.com>
Reviewed-on: https://review.whamcloud.com/22762
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9274 ptlrpc: add replay request into unreplied list 20/27920/2
Niu Yawei [Thu, 22 Jun 2017 07:03:38 +0000 (03:03 -0400)]
LU-9274 ptlrpc: add replay request into unreplied list

ptlrpc_prepare_replay() may fail to add replay request into unreplied
list if the request hasn't been on replay list yet, so in
ptlrpc_replay_next() before sending replay, we'd always make sure the
replay request is on unreplied list.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I91757cd4fde1d85d146475e078db125acc2c821f
Reviewed-on: https://review.whamcloud.com/27920
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9305 osd: do not release pages twice 50/27950/8
Alex Zhuravlev [Sun, 9 Jul 2017 16:56:53 +0000 (12:56 -0400)]
LU-9305 osd: do not release pages twice

in case of blocksize mismatch dmu_assign_arcbuf() releases passed
abuf internally, including the pages. osd_bufs_put() can't detect
this and may call __free_page() on inappropriate pages (which can
be allocated to someone else already).

Change-Id: I454e56ee3de3d201a14e6ba7b4beabaad42d82ae
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/27950
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9693 tests: Adding sanity test_42a & 42c to always_except 40/27740/2
Saurabh Tandan [Tue, 20 Jun 2017 18:57:35 +0000 (11:57 -0700)]
LU-9693 tests: Adding sanity test_42a & 42c to always_except

Sanity test_42a and test_42c were removed from Always_Except
list earlier. But it seems the issues were not completely
resolved.

Hence, adding them back to Always_Except list.

Test-Parameters: trivial
Signed-off-by: Saurabh Tandan <saurabh.tandan@intel.com>
Change-Id: Ib82e0d788054f3ede58c2dbdf5af21227fb4e7f3
Reviewed-on: https://review.whamcloud.com/27740
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
3 years agoLU-9715 libcfs: crash in cpu_pattern parsing code 72/27872/3
Andreas Dilger [Wed, 28 Jun 2017 17:12:28 +0000 (11:12 -0600)]
LU-9715 libcfs: crash in cpu_pattern parsing code

The for loop in cfs_cpt_table_create_pattern() that scans
for brackets to count the number of cpts is broken. It will
increment bracket beyond NULL and it will increment ncpt
beyond the number of available cpts. This has been fixed.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I87bc9de4c531c42c421e8e62edd881417dbcab07
Reviewed-on: https://review.whamcloud.com/27872
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoBranch of 2.11 development branch new tag 2.10.50 2.10.50 v2_10_50 v2_10_50_0
Oleg Drokin [Tue, 27 Jun 2017 19:20:54 +0000 (15:20 -0400)]
Branch of 2.11 development branch new tag 2.10.50

Change-Id: Iea32c681312885f577421e1bf194c569a6b0da65
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9073 gss: remove newer kernel support 23/27823/2
James Simmons [Mon, 26 Jun 2017 17:54:25 +0000 (13:54 -0400)]
LU-9073 gss: remove newer kernel support

Revert the work to support newer kernels for GSS. For now
disable GSS support for kernels newer than 4.6 so this
doesn't block people on newer distros. Even is very basic
support for GSS is restored I wouldn't recommend this for
production systems at this time.

Change-Id: I7e1636bf695e1686bbdf968d088fcfc5a8f8f062
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27823
Tested-by: Jenkins
Reviewed-by: Chris Hanna <hannac@iu.edu>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9678 osd-zfs: register arc_prune_func() after site init 08/27708/3
Lai Siyao [Fri, 16 Jun 2017 14:34:33 +0000 (22:34 +0800)]
LU-9678 osd-zfs: register arc_prune_func() after site init

Register arc_prune_func() after site init, otherwise it may be
called while object cache is not initialized yet.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I822252da906f03899386fb0941cc11c1c3366fbf
Reviewed-on: https://review.whamcloud.com/27708
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9049 obdclass: change object lookup to no wait mode 65/26965/9
Lai Siyao [Tue, 23 May 2017 07:56:06 +0000 (15:56 +0800)]
LU-9049 obdclass: change object lookup to no wait mode

Currently we set LU_OBJECT_HEARD_BANSHEE on object when we want
to remove object from cache, but this may lead to deadlock, because
when other process lookup such object, it needs to wait for this
object until release (done at last refcount put), while that process
maybe already hold an LDLM lock.

Now that current code can handle dying object correctly, we can just
return such object in lookup, thus the above deadlock can be avoided.

There is another case we need to make some changes:
objects created in OUT doesn't set dt_body_ops for LOD layer because
originally it's set by lod_create(), change to set dt_body_ops in
lod_object_init() so objects created in OUT is no different from
those created in MDT. To achieve this, functions in lod_body_ops
should check file type inside to avoid misuse.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ia31ab5f09f9bf80a9aa8fd7e7b60348b02400b25
Reviewed-on: https://review.whamcloud.com/26965
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9485 llite: check the return value of cl_file_inode_init() 58/27658/7
Bobi Jam [Thu, 15 Jun 2017 08:01:13 +0000 (16:01 +0800)]
LU-9485 llite: check the return value of cl_file_inode_init()

ll_update_inode() does not check the return value of
cl_file_inode_init(), and it should check.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I4174e4f8166d7834a1d619aa8d0191d1f428c62c
Reviewed-on: https://review.whamcloud.com/27658
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8703 libcfs: rework CPU pattern parsing code 06/23306/14
Dmitry Eremin [Tue, 13 Jun 2017 00:59:16 +0000 (20:59 -0400)]
LU-8703 libcfs: rework CPU pattern parsing code

Rewrite CPU pattern parsing code to avoid passed buffer change
add real errors propogation to caller function.

Change-Id: I8dfee2c0013fcfccd3d99c361d3ec626594689bd
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/23306
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9597 test: wait setattr finished before checking accounting 25/27425/11
Wang Shilong [Wed, 7 Jun 2017 08:10:12 +0000 (16:10 +0800)]
LU-9597 test: wait setattr finished before checking accounting

Need wait setattr finished for ost objects, otherwise, accounting
for projects won't be accurate, also add failure check for setattr
ioctl.

Test-Parameters: testlist=sanity-quota,sanity-quota,sanity-quota,sanity-quota clientdistro=el7 serverdistro=el7 \
        ostfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs
Change-Id: I106689c224997f79eb779fdc6843704ae7e9ffe6
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/27425
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9558 llite: use struct vma_area_struct address field 81/27281/8
James Simmons [Wed, 14 Jun 2017 17:54:13 +0000 (13:54 -0400)]
LU-9558 llite: use struct vma_area_struct address field

The field virtual_address of struct vma_area_struct was
removed since it provided no better benefit than just
using the address field directly.

Linux-commit: 1a29d85eb0f19b7d8271923d8917d7b4f5540b3e

Change-Id: I05068cdf27c93c5b3201c76ec043bc6c0e66df1f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27281
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9210 statahead: missing barrier before wake_up 30/27330/2
Lai Siyao [Tue, 7 Mar 2017 05:56:04 +0000 (13:56 +0800)]
LU-9210 statahead: missing barrier before wake_up

A barrier is missing before wake_up() in ll_statahead_interpret(),
which may cause 'ls' hang.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I391d6222e353fb27761ffd5412b52ce08f7a0592
Reviewed-on: https://review.whamcloud.com/27330
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9074 llite: Stop file creation for ro bind mnt 04/25204/12
Noopur Maheshwari [Thu, 2 Feb 2017 09:09:31 +0000 (14:39 +0530)]
LU-9074 llite: Stop file creation for ro bind mnt

While remounting the bind mount of lustre with read-only, vfs
sets MNT_READONLY in the mnt flags and does not make a call
to lustre. Hence, the change in mnt flags is not reflected in
lustre.

Therefore, file creation goes ahead in lookup operation of
lustre with LOOKUP_CREATE intent set and converted to IT_CREAT.

The fix is to disallow file creation by not setting IT_CREAT
intent when bind mnt pt is readonly and unsetting O_CREAT.

Added a test case to test that files are not created in ro
bind mount point. Files must be created after bind mount point
is converted from ro to rw.

Signed-off-by: Noopur Maheshwari <noopur.maheshwari@seagate.com>
Change-Id: Ic60fb18f539159715049515e264afdf51a00378e
Reviewed-on: https://review.whamcloud.com/25204
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9504 ptlrpc: REP-ACK hr may race with trans commit 07/27207/9
Lai Siyao [Thu, 18 May 2017 16:27:31 +0000 (00:27 +0800)]
LU-9504 ptlrpc: REP-ACK hr may race with trans commit

REP-ACK hr may race with transaction commit, and the latter will
release saved locks, so in REP-ACK hr we need to get locks early to
convert them to COS mode safely.

But the locks got may be decrefed and canceled, in this case it
can't be converted to COS mode, remove an assert in
ldlm_lock_downgrade() for this.

Also protect mdt_steal_ack_locks() with rs_lock because it may also
race with REP-ACK hr. And move ldlm_lock_decref() outside of locks
because it may sleep.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ia9a3ba6a83689c0552ae8aaf2eb735c3f06b62e2
Reviewed-on: https://review.whamcloud.com/27207
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9567 mgc: set cfg_instance to NULL for sptlrpc case 20/27320/6
James Simmons [Thu, 8 Jun 2017 16:40:09 +0000 (12:40 -0400)]
LU-9567 mgc: set cfg_instance to NULL for sptlrpc case

The patch for LU-9034 mapped config logs to separate mgc
devices. This change broke sptlrpc by preventing the
ablitity to pass along config log change with
lctl conf_param. The reason for this is due to the
cfg_instance field being set to an obd device pointer.
This patch sets cfg_instance back to NULL to restore
the functionality. This is only a temporary work
around so GSS can function for the 2.10 release. A
proper fix will be worked out in 2.11.

Change-Id: Ic60e6516802d3617b38df3371f52a7f27aeae8cd
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Reviewed-on: https://review.whamcloud.com/27320
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9399 llite: register mountpoint before process llog 41/27241/6
Emoly Liu [Sat, 27 May 2017 04:05:44 +0000 (12:05 +0800)]
LU-9399 llite: register mountpoint before process llog

In ll_fill_super(), lprocfs_ll_register_mountpoint() should be
called before lustre_process_log(), otherwise the directory
/proc/fs/lustre/llite/* can't be created in time and the params
"llite.*.*" won't be set correctly.

Also, this patch adds sbi->ll_xattr_cache_set to mark the flag
LL_SBI_XATTR_CACHE already set during lustre_process_log(),
in case that it will be overwritten in client_common_fill_super().

conf-sanity.sh test_76d is added to verify this fix.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ie1b4d2ef23bbe48a22d48f92f7d4efe64042eec4
Reviewed-on: https://review.whamcloud.com/27241
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9655 lnet: Remove redirect in lnet unit file 78/27578/2
Giuseppe Di Natale [Mon, 12 Jun 2017 15:49:06 +0000 (08:49 -0700)]
LU-9655 lnet: Remove redirect in lnet unit file

Remove the file redirect in the lnet systemd unit file
because it causes a startup failure.

Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: I2962babccee708f80b716cacfe21014ee9bfc442
Reviewed-on: https://review.whamcloud.com/27578
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
3 years agoLU-9558 libcfs: include new sched directory headers 57/27557/2
James Simmons [Sat, 10 Jun 2017 18:47:40 +0000 (14:47 -0400)]
LU-9558 libcfs: include new sched directory headers

Starting in linux kernel version 4.11 the very large
sched.h header is not being broken up into smaller
headers that are placed in linux/sched/*. This patch
handles this small change.

Test-Parameters: trivial

Change-Id: I82ba8ced17f7570fe052777d970a769f1dd366da
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27557
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9558 libcfs: replace set_task_state() with set_current_state() 56/27556/3
James Simmons [Tue, 13 Jun 2017 01:48:21 +0000 (21:48 -0400)]
LU-9558 libcfs: replace set_task_state() with set_current_state()

Linxu commit 642fa448ae6b3a4e5e8737054a094173405b7643 removed the
set_task_state() for reasons stated in the commit. The solution
is to move to set_current_state().

Change-Id: I8e5024fdf7e80f85ad20a5b9343abf6c40a9f339
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27556
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9604 tests: fix version check in sanity/test_77c 46/27446/2
Bruno Faccini [Tue, 6 Jun 2017 06:56:31 +0000 (08:56 +0200)]
LU-9604 tests: fix version check in sanity/test_77c

Version check had not been updated in original and
upstreamed master patch version from LU-8376.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Iaa3c042f5058006b01f71932e3ed5baa4b12999e
Reviewed-on: https://review.whamcloud.com/27446
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-9611 lod: allow -1 for default stripe count/offset 73/27473/3
Andreas Dilger [Tue, 6 Jun 2017 21:38:15 +0000 (15:38 -0600)]
LU-9611 lod: allow -1 for default stripe count/offset

Since LU-7344 patch http://review.whamcloud.com/16930 was landed,
lod_stripeoffset_seq_write() and lod_stripecount_seq_write() have
incorrectly checked that lod.*.stripecount and lod.*.stripeoffset
are not negative.  In fact they can both be "-1" to indicate that
the filesystem-wide default value should be used. These parameters
can also be set internally if using "lfs setstripe -c -1 $MOUNT"
or "lfs setstripe -i -1 $MOUNT" to set the system wide default,
generating console errors on the MDS from class_process_proc_param():

  lov.: error writing proc entry 'stripecount': rc = -34
  lov.: error writing proc entry 'stripeoffset': rc = -34

Add comment blocks to lod_fix_desc_stripe*() to indicate their use.
Fix these functions to allow "-1" as a valid value.
Fix the error message to properly indicate which device is being
modified, in case multiple filesystems are mounted.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I295d2591d535b039634689524a29725e96ce2c01
Reviewed-on: https://review.whamcloud.com/27473
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9607 lnet: Fix lost lock 55/27455/2
Dmitry Eremin [Tue, 6 Jun 2017 14:02:03 +0000 (17:02 +0300)]
LU-9607 lnet: Fix lost lock

Unlock lnet_net_lock in case of error in function lnet_select_pathway().

Change-Id: Ib48fb3aebdc60bafff80f5c52b90301830ca4afa
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/27455
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-20 osd-ldiskfs: Make readonly patches optional 49/27549/3
Oleg Drokin [Fri, 9 Jun 2017 20:51:56 +0000 (16:51 -0400)]
LU-20 osd-ldiskfs: Make readonly patches optional

If CONFIG_KALLSYMS is enabled, see if there are dev_set/check_rdonly
functions present and use them if so.
Otherwise assume the functionality is unavailable.

This should allow using osd_ldiskfs module on both patched and
unpatched kernels with no modifications thanks to weak updates.

Change-Id: I274db77cf76abc63ad6a5643a98c521778f294b3
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/27549
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in lhsmtool_posix.c 28/27528/2
Steve Guminski [Fri, 14 Apr 2017 19:50:51 +0000 (15:50 -0400)]
LU-6210 utils: Use C99 struct initializers in lhsmtool_posix.c

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lhsmtool_posix.c to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I93e9e7dc10e30bfc81eed3a8d868d2400f50761e
Reviewed-on: https://review.whamcloud.com/27528
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in llverdev.c 29/27529/2
Steve Guminski [Fri, 14 Apr 2017 19:53:14 +0000 (15:53 -0400)]
LU-6210 utils: Use C99 struct initializers in llverdev.c

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates llverdev.c to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Icad1529704118fa6b0a0a4be4c9e6203d9534469
Reviewed-on: https://review.whamcloud.com/27529
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-6210 utils: Use C99 initializers in lfs_changelog_clear() 23/27523/2
Steve Guminski [Fri, 14 Apr 2017 19:39:39 +0000 (15:39 -0400)]
LU-6210 utils: Use C99 initializers in lfs_changelog_clear()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.  Variables of
type struct option are renamed to long_opts for consistency.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_changelog_clear() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ib9847af80b7eb76bf81b9692baa5c6deb4cdad85
Reviewed-on: https://review.whamcloud.com/27523
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 initializers in lfs_setquota_times() 19/27519/2
Steve Guminski [Fri, 14 Apr 2017 19:27:22 +0000 (15:27 -0400)]
LU-6210 utils: Use C99 initializers in lfs_setquota_times()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_setquota_times() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I50c6260246e0ac345d8b981dd624791bab9220a5
Reviewed-on: https://review.whamcloud.com/27519
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 struct initializers in lfs_df() 18/27518/2
Steve Guminski [Fri, 14 Apr 2017 19:25:15 +0000 (15:25 -0400)]
LU-6210 utils: Use C99 struct initializers in lfs_df()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_df() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I006857f76e2a0bd89fea41fef158931953b21aa8
Reviewed-on: https://review.whamcloud.com/27518
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-6210 utils: Use C99 initializers in lfs_setdirstripe() 15/27515/2
Steve Guminski [Fri, 14 Apr 2017 19:16:23 +0000 (15:16 -0400)]
LU-6210 utils: Use C99 initializers in lfs_setdirstripe()

This patch makes no functional changes.  Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_setdirstripe() to use the C99 syntax.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I0335cae79bc5f57da3d73b40c17e83f87853ef5f
Reviewed-on: https://review.whamcloud.com/27515
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-8661 libcfs: handle padata_alloc() rename 91/27491/3
Dmitry Eremin [Thu, 8 Jun 2017 01:27:44 +0000 (21:27 -0400)]
LU-8661 libcfs: handle padata_alloc() rename

In upstream commit e6cc11707661770ca2bd4db4b0256d28f48e7541
the function padata_alloc() was renamed to padata_alloc_possible().

Change-Id: I6ad6a503af3c52fa3c64ef9b3452bd35d8490cd5
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/27491
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9615 mdt: use write llog lock for pending restores 84/27484/2
John L. Hammond [Wed, 7 Jun 2017 13:11:14 +0000 (08:11 -0500)]
LU-9615 mdt: use write llog lock for pending restores

In mdt_hsm_pending_restore(), ask for a write llog lock when
processing the llog.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib93a3a3b0f9472afae50c240b7dcb14fce4a8501
Reviewed-on: https://review.whamcloud.com/27484
Tested-by: Jenkins
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8941 tests: remove wrong redirection syntaxes 51/27351/3
Gregoire Pichon [Wed, 31 May 2017 14:50:50 +0000 (16:50 +0200)]
LU-8941 tests: remove wrong redirection syntaxes

The "2&>" redirection syntax is wrong. Replace it with
what was probably intended by the test writer.

Test-Parameters: trivial testlist=replay-single,sanityn

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I8322bc853316a2580572c415fc1af4b73377fbc2
Reviewed-on: https://review.whamcloud.com/27351
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
3 years agoLU-9468 llite: fix for stat under kthread and X86_X32 92/26992/14
Frank Zago [Fri, 5 May 2017 21:10:47 +0000 (17:10 -0400)]
LU-9468 llite: fix for stat under kthread and X86_X32

Under the following conditions, ll_getattr will flatten the inode
number when it shouldn't:

  - the X86_X32 architecture is defined CONFIG_X86_X32, and not even
    used,
  - ll_getattr is called from a kernel thread (though vfs_getattr for
    instance.)

This has the result that inode numbers are different whether the same
file is stat'ed from a kernel thread, or from a syscall. For instance,
4198401 vs. 144115205272502273.

ll_getattr calls ll_need_32bit_api to determine whether the task is 32
bits. When the combination is kthread+X86_X32, that function returns
that the task is 32 bits, which is incorrect, as the kernel is 64
bits.

The solution is to check whether the call is from a kernel thread
(which is 64 bits) and act consequently.

Added test_410 to test the condition. A failed run will have a message like this:

  lustre_kinode_54354: CONFIG_X86_X32 is set
  lustre_kinode_54354: inode is 144115205272502273
  lustre_kinode_54354: inode is 4198401
  lustre_kinode_54354: inode numbers are different: 144115205272502273 4198401

while a successfull one will be:

  lustre_kinode_10519: CONFIG_X86_X32 is set
  lustre_kinode_10519: inode is 144115205272502275
  lustre_kinode_10519: inode is 144115205272502275
  lustre_kinode_10519: inode numbers are identical: 144115205272502275

Signed-off-by: Frank Zago <fzago@cray.com>
Change-Id: Ib8f07de47eaa29046a61c488368d466f9096a994
Reviewed-on: https://review.whamcloud.com/26992
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-8891 llite: rcu-walk check should not depend on statahead 95/24195/3
Steve Guminski [Wed, 7 Dec 2016 13:02:52 +0000 (08:02 -0500)]
LU-8891 llite: rcu-walk check should not depend on statahead

Moves the check for the LOOKUP_RCU flag, so that it does not depend
on the statahead setting.  The caller is now informed if rcu-walk
was requested but the filesystem does not support it, regardless
of whether statahead is enabled or disabled.

Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I0dcfd594c1b44cba1741c60a24ed623ef635b79c
Reviewed-on: https://review.whamcloud.com/24195
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8817 tests: Update 'min_ost_size' 85/23685/2
Arshad Hussain [Thu, 14 Jul 2016 10:19:58 +0000 (15:49 +0530)]
LU-8817 tests: Update 'min_ost_size'

On setup where MDS and client are on same node
lctl get_param osc.*.kbytesavail returns the
value from OSP. This patch updates function
min_ost_size to use "lfs df" output instead of
reading it from proc.

Output of 'lfs df'. 'Available' field only.
$ lfs df | grep OST | awk '{print $4}' | sort -n | head -n1
151276

Output with current code:
$ lctl get_param -n osc.*.kbytesavail | sort -n | head -n1
151004

Test-Parameters: trivial testlist=sanity-benchmark,obdfilter-survey
Signed-off-by: Arshad Hussain <arshad.hussain@seagate.com>
Change-Id: I07ddff43d7fc4b39b89bd998f96bf36776c5c048
Seagate-bug-id: MRP-3683
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/23685
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Ujjwal Lanjewar <ujjwal.lanjewar@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-2189 tests: Ensure quiescence for sanityn/36 73/27373/3
Nathaniel Clark [Thu, 1 Jun 2017 12:00:01 +0000 (08:00 -0400)]
LU-2189 tests: Ensure quiescence for sanityn/36

Because sanityn/36 measures space used/free, ensure that all previous
operations have completed prior to getting statfs info.

Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=1 ostcount=2 mdscount=1 mdtcount=1 testlist=sanityn
Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=1 ostcount=2 mdscount=1 mdtcount=1 testlist=sanityn
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I684d428bbb6fca1a004bba5acf6a17a1636fb0dd
Reviewed-on: https://review.whamcloud.com/27373
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9579 osc: adds radix_tree_preload 72/27372/4
Alexander Boyko [Thu, 1 Jun 2017 11:31:37 +0000 (14:31 +0300)]
LU-9579 osc: adds radix_tree_preload

The client fail with next error
osc_page_init()) ASSERTION( result == 0 )
in low memory conditions.

The patch fixes the problem above by adding
radix_tree_preload.

Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Seagate-bug-id: MRP-4424
Change-Id: I0a87366f8483fce4d2b3bcca449becce2a8262cb
Reviewed-on: https://review.whamcloud.com/27372
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-5969 obdclass: deprecate OBD_GET_VERSION ioctl 40/26440/5
Andreas Dilger [Tue, 20 Jan 2015 09:51:20 +0000 (02:51 -0700)]
LU-5969 obdclass: deprecate OBD_GET_VERSION ioctl

Mark OBD_GET_VERSION ioctl deprecated, disable before 3.1 release.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I681a353e6f65fd1ec2d1434fe4a1fa55ed3ebbe5
Reviewed-on: https://review.whamcloud.com/26440
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-4825 utils: improve lfs_migrate usage message 61/25661/5
Andreas Dilger [Tue, 28 Feb 2017 04:06:56 +0000 (21:06 -0700)]
LU-4825 utils: improve lfs_migrate usage message

Improve the warning message printed by lfs_migrate to make it match
the OST device names used in Lustre 2.x instead of 1.x device names.
Also reference the new max_create_count parameter instead of the old
deactivate command, which prevented OST objects from being deleted.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I114006afb93d8c8d78923a874f3b914202500c1e
Reviewed-on: https://review.whamcloud.com/25661
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9498 osp: revert till a fix 47/27447/3
Alex Zhuravlev [Tue, 6 Jun 2017 08:05:57 +0000 (12:05 +0400)]
LU-9498 osp: revert till a fix

Revert "LU-8367 osp: orphan cleanup do not wait for reserved"

LU-9498 was discovered with soak testing, the root cause is unknown
at the moment.

This reverts commit 1b3028ab142a1f605e6274a6019bb39d89ae8d25.

Change-Id: I65016876f3e42345a06194771c9731e28a850adc
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/27447
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoNew tag 2.9.59 2.9.59 v2_9_59 v2_9_59_0
Oleg Drokin [Mon, 12 Jun 2017 22:37:00 +0000 (18:37 -0400)]
New tag 2.9.59

Change-Id: I900f72cc0137b71ee0dfd6f4ced59e9e55e0c76e
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9367 llite: restore ll_file_getstripe in ll_lov_setstripe 15/26915/13
Bobi Jam [Tue, 2 May 2017 12:30:27 +0000 (20:30 +0800)]
LU-9367 llite: restore ll_file_getstripe in ll_lov_setstripe

Commit fafe6b4d4a6fa63cedff3bd44e6578009578b3d7 has get rid of
the call to ll_file_getstripe in ll_lov_setstripe.

Add a @size parameter for series of xxx_getstripe interfaces,
indicating the max buffer size that user provides to hold the
stripe information. It is mainly for the ll_lov_setstripe, which
will call ll_file_getstripe to fetch basic stripe inforation.

Add LL_IOC_LOV_SETSTRIPE_NEW/LL_IOC_LOV_GETSTRIPE_NEW ioctl interface
which defines the interface correctly, which could be used in later
Lustre versions.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic2f541fa3c74133eb40d230c8a836434043d7691
Reviewed-on: https://review.whamcloud.com/26915
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-9572 kernel: kernel update RHEL6.9 [2.6.32-696.3.1.el6] 58/27358/4
Bob Glossman [Tue, 30 May 2017 15:28:54 +0000 (08:28 -0700)]
LU-9572 kernel: kernel update RHEL6.9 [2.6.32-696.3.1.el6]

Update RHEL6.9 kernel to 2.6.32-696.3.1.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
  ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Change-Id: Ie61a7a8c521acd40ebf5106db17dcfb5d25f7a7f
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/27358
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8661 libcfs: handle pdata_set_cpumasks() removal 22/27422/2
James Simmons [Sun, 4 Jun 2017 15:34:11 +0000 (11:34 -0400)]
LU-8661 libcfs: handle pdata_set_cpumasks() removal

In Linux commit 815613da6a67c196d7458d0e6c278ea88e21933f the
function padata_set_cpumasks was removed since no one was
using it upstream. In order to support newer kernels we need
to migrate to pdata_set_cpumask() instead.

Change-Id: Ia2e5143d9a2649bfbf9e1df1fb7f2d1b3c807884
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27422
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-930 doc: remove llapi_layout_t from man pages 98/26698/4
Andreas Dilger [Tue, 18 Apr 2017 06:31:01 +0000 (00:31 -0600)]
LU-930 doc: remove llapi_layout_t from man pages

Remove the use of the non-existent llapi_layout_t from llapi_layout.7
and llapi_layout_alloc.3 and replace with struct llapi_layout.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Icb3753038677ce95a35ed8f3b8266315553ebbe5
Reviewed-on: https://review.whamcloud.com/26698
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8703 libcfs: make tolerant to offline CPUs and empty NUMA nodes 22/23222/20
Dmitry Eremin [Fri, 21 Oct 2016 12:32:28 +0000 (15:32 +0300)]
LU-8703 libcfs: make tolerant to offline CPUs and empty NUMA nodes

Rework CPU partition code in the way of make it more tolerant to
offline CPUs and empty nodes.

For example, in KNL:
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221
node 0 size: 24472 MB
node 0 free: 12409 MB
node 1 cpus: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239
node 1 size: 24576 MB
node 1 free: 20388 MB
node 2 cpus: 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255
node 2 size: 24576 MB
node 2 free: 20621 MB
node 3 cpus: 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271
node 3 size: 24576 MB
node 3 free: 21183 MB
node 4 cpus:
node 4 size: 4096 MB
node 4 free: 3982 MB
node 5 cpus:
node 5 size: 4096 MB
node 5 free: 3982 MB
node 6 cpus:
node 6 size: 4096 MB
node 6 free: 3982 MB
node 7 cpus:
node 7 size: 4096 MB
node 7 free: 3981 MB
node distances:
node   0   1   2   3   4   5   6   7
  0:  10  21  21  21  31  41  41  41
  1:  21  10  21  21  41  31  41  41
  2:  21  21  10  21  41  41  31  41
  3:  21  21  21  10  41  41  41  31
  4:  31  41  41  41  10  41  41  41
  5:  41  31  41  41  41  10  41  41
  6:  41  41  31  41  41  41  10  41
  7:  41  41  41  31  41  41  41  10

Contain the fix for LU-8492 ptlrpc: Correctly calculate hrp->hrp_nthrs
Fix an error code return which was introduced in
commit def25e9c7eff57cdaf8a6ee5e8e7db005bab6525

Change-Id: I7f64a20ee009a88e836f592ce044400f07ffbcdd
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/23222
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-8656 vvp: Add S_NOSEC support 53/22853/15
Patrick Farrell [Sat, 3 Jun 2017 15:20:10 +0000 (11:20 -0400)]
LU-8656 vvp: Add S_NOSEC support

We must use the i_mutex to protect permission changes,
which means we need to take it when we write to a file with
the setuid or setgid bit set (as this removes those bits).

LU-8025 attempted to use IS_NOSEC to check for this case,
but did not actually add support for the S_NOSEC flag to
Lustre.

S_NOSEC was added in upstream kernel commit:
69b4573296469fd3f70cf7044693074980517067
But a key change requiring parallel filesystems to opt in
with a superblock flag was added in:
9e1f1de02c2275d7172e18dc4e7c2065777611bf

This patch adds the required support.

Specifically, Lustre should set S_NOSEC correctly when an
inode is created (ll_iget), but only for new inodes.
Setting it for existing inodes requires taking the i_mutex,
creating an unacceptable metadata performance impact in the
lookup code.

Existing inodes get S_NOSEC set either by a setattr call
(see below), or by the first writer to write to the node,
in file_remove_privs/file_remove_suid.  So it's OK not to
set S_NOSEC on all inodes in ll_iget.

Also, Lustre must clear S_NOSEC when it gets a metadata
update because another node could have changed permissions.
i_flags is already cleared in ll_update_inode, but we would
prefer to have S_NOSEC set whenever possible, so we want to
re-do the check after the update.

This requires holding the i_mutex to avoid a check/set race
with permissions changes.

We cannot easily take the i_mutex in ll_update_inode (it is
called from too many places, some of which already hold the
i_mutex).

It is acceptable to sometimes not have S_NOSEC set (since
occasionally taking the i_mutex when not needed is OK), so
we look at getting it set most of the time.

It looks like the primary concern is ll_md_setattr.  The
caller (ll_setattr_raw) takes the i_mutex before returning,
so we do the relevant call there.

This opens a window during which S_NOSEC is not set, but
again, this merely creates a situation where we take the
i_mutex unnecessarily, and should be rare in practice.

Testing with multiple writers shows that we only very
rarely attempt to take the i_mutex, so the performance
impact is minimal.

This commit also adds a test to verify the S_NOSEC/i_mutex
behavior.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: If481366283ceaeb247e50cc4d536fb10e9c329ee
Reviewed-on: https://review.whamcloud.com/22853
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-9019 libcfs: remove no longer use time abstraction 20/27420/3
James Simmons [Sat, 3 Jun 2017 17:31:04 +0000 (13:31 -0400)]
LU-9019 libcfs: remove no longer use time abstraction

With the cleanup from 64 bit time handling we can remove
several no longer used wrapper functions in libcfs. Also
add test to checkpatch to warn patch submitters to use
the proper linux kernel timer functions for functions
that will soon go away.

Test-Parameters: trivial

Change-Id: Ie23d74d1faea6589e7def5225321ddc52c7e65b7
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27420
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9433 osd-ldiskfs: fix inode reference leak 12/27212/5
Fan Yong [Fri, 26 May 2017 17:20:30 +0000 (01:20 +0800)]
LU-9433 osd-ldiskfs: fix inode reference leak

There are several corner cases in the osd_consistency_check(),
osd_iget_check() and osd_fid_lookup(), if the found inode with
the given FID does not match each other, then the logic may
miss to release current inode before "return" or "goto" for
further check. That will cause inode refrence leak.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I53997dd7f321ae34f951b9e3aac754ac33214da8
Reviewed-on: https://review.whamcloud.com/27212
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9488 scrub: reset noslot for full speed scrub 28/27228/3
Fan Yong [Thu, 25 May 2017 19:45:49 +0000 (03:45 +0800)]
LU-9488 scrub: reset noslot for full speed scrub

If there are too much inconsistect OI mappings detected,
then OI scrub will run under full speed mode for fixing
OI inconsistencies as fast as possible. Under such case,
the @noslot variable should be reset to avoid misguiding
the osd_scrub_next() to return SCRUB_NEXT_WAIT by wrong.

This patch also dumps more information for oi_scrub proc
output, those are helpful to understand OI scurb status.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0f10d5ff703bbe5460eb77703b67fb066ad4d58b
Reviewed-on: https://review.whamcloud.com/27228
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
3 years agoLU-9440 osd-ldiskfs: efficient way to get ldiskfs backend name 77/27177/5
Fan Yong [Thu, 25 May 2017 19:35:20 +0000 (03:35 +0800)]
LU-9440 osd-ldiskfs: efficient way to get ldiskfs backend name

In the OSD debug message, we originally used the code similar as
"LDISKFS_SB(osd_sb(osd))>s_es>s_volume_name" to get the ldiskfs
backend name information. That is inefficient, it can be replaced
by the "osd_sb(osd)->s_id" or "inode->i_sb->s_id".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I077452796065cc924e9ca3b9cfaec4b1a2a4bebe
Reviewed-on: https://review.whamcloud.com/27177
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
3 years agoLU-9484 llite: eat -EEXIST on setting trunsted.lov 26/27126/7
Bobi Jam [Tue, 16 May 2017 12:37:20 +0000 (20:37 +0800)]
LU-9484 llite: eat -EEXIST on setting trunsted.lov

Tools like rsync, tar, cp may copy and restore the xattrs on a file.
The client previously ignored the setting of trusted.lov/lustre.lov
if the layout had already been specified, to avoid causing these
tools to fail for no reason.

For PFL files we still need to silently eat -EEXIST on setting these
attributes to avoid problems.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic6d359c0f3557d4a21e950a53f57e38ae97a40da
Reviewed-on: https://review.whamcloud.com/27126
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
3 years agoLU-9603 test: Fix test-framework.sh 38/27438/3
Sonia [Mon, 5 Jun 2017 22:34:00 +0000 (15:34 -0700)]
LU-9603 test: Fix  test-framework.sh

In test-frameowrk.sh, h2nettype(), h2o2ib() and h2tcp()
uses if [[ -v NETTYPE ]] where -v option is recognized
only in bash version 4.2 and above.

This patch replaces if [[ -v NETTYPE ]] with
if [[ -n "$NETTYPE" ]].

Test-Parameters: trivial testlist=conf-sanity, sanity, sanity-sec
Change-Id: Ife584f27c7a4caf6f405b701e85d389804ed33ca
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/27438
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
3 years agoLU-9575 obdclass: remove cl_for_each defines 85/27385/2
Ben Evans [Thu, 1 Jun 2017 20:14:00 +0000 (15:14 -0500)]
LU-9575 obdclass: remove cl_for_each defines

cl_for_each and cl_for_each_reverse are simply aliases
for list_for_each_entry and list_for_each_entry_reverse
There is no point to them so just get rid of them and
eliminate any confusion.

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: I4f21cc5020142c82999324f0ae7ccb57704a98bd
Reviewed-on: https://review.whamcloud.com/27385
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>