Whamcloud - gitweb
fs/lustre-release.git
13 months agoLU-11771 ldlm: use hrtimer for recovery to fix timeout messages 83/33883/12
James Simmons [Mon, 25 Mar 2019 15:42:17 +0000 (11:42 -0400)]
LU-11771 ldlm: use hrtimer for recovery to fix timeout messages

Currently the functions target_handle_connect/reconnect show
incorrect timeout to the end of recovery:

fs1-OST0000: Recovery already passed deadline 71578:57.
If you do not want to wait more, please abort the recovery by force.
...
fs1-OST0000: Denying connection for new client ...
(1 recovered, 11 in progress, and 1 evicted) to recover in 71578:57

This is due to the assumption that the time returned by the
monotonic clock and jiffies was initialized at the same time but
that is not the case. So a compare between ktime_get_seconds()
and jiffies converted to seconds is invalid.

We solve this by replacing the recovery timer with a hrtimer based
one. Their are many benefits to using a hrtimer over jiffies like
better scaling, power profile, and better handling on tickless
system. This also makes the code clear by using just the real wall
clock in all cases.

Change-Id: I50442605686382f7afb9a1f49eb336c0ee637cdc
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33883
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
13 months agoLU-12068 test: compare position for ZFS dot entry 25/34525/2
Hongchao Zhang [Wed, 27 Mar 2019 17:08:03 +0000 (13:08 -0400)]
LU-12068 test: compare position for ZFS dot entry

in test_6b of sanity-lfsck.sh, the position will be zero for
special entries "." and "..", which should not be used to
determine whether the LFSCK process is forward or not, in this
case, the otable position should be used.

Change-Id: I98aee1ae92fa5ea742a8001b58e092111d646477
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34525
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
13 months agoLU-8760 lfsck: fix bit operations lfsck_assistant_data 02/34502/2
Alexander Boyko [Tue, 26 Mar 2019 11:43:14 +0000 (07:43 -0400)]
LU-8760 lfsck: fix bit operations lfsck_assistant_data

Race between lfsck_master_engine->lfsck_double_scan_generic()
and  lfsck_layout->lfsck_assistant_engine() take a place.
Both threads were sleeping and waiting for each other.
lad_to_double_scan was set before sleep, but lfsck_assistant_data had
zeros
  lad_post_result = 1,
  lad_to_post = 0,
  lad_to_double_scan = 0,
  lad_in_double_scan = 0,
  lad_exit = 0,
  lad_incomplete = 0,

Using
 unsigned int a:1,
           b:1;
  f1() {a = 1;}
  f2() {b = 0;}
is racy for multithread execution. These type of logic requires
atomic bit operations.
The race is lad_to_double_scan vs lad_to_post.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-7076
Change-Id: I4f971ce2acb244f32ae2e108b96995dc2f27e7a3
Reviewed-on: https://review.whamcloud.com/34502
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
13 months agoLU-6142 ldlm: Fix style issues for ldlm_pool.c 97/34497/3
Arshad Hussain [Thu, 21 Mar 2019 07:17:30 +0000 (12:47 +0530)]
LU-6142 ldlm: Fix style issues for ldlm_pool.c

This patch fixes issues reported by checkpatch
for file lustre/ldlm/ldlm_pool.c

Change-Id: Iee850badeced8ad4edcb4a75bfd2daca0f508c2a
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/34497
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
13 months agoLU-12069 mdt: add missing argument to enable compilation 20/34420/5
Alex Zhuravlev [Thu, 14 Mar 2019 12:59:09 +0000 (15:59 +0300)]
LU-12069 mdt: add missing argument to enable compilation

with gcc8 which is very strict about missing arguments.

Change-Id: I08cd4a876c4cb49e15ada7458a637c98ea4d83c0
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34420
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
13 months agoLU-10073 tests: stop running smoke test 43/34543/4
James Nunez [Fri, 29 Mar 2019 19:01:54 +0000 (13:01 -0600)]
LU-10073 tests: stop running smoke test

lnet-selftest test smoke is failing at a high rate
when tested with ARM clients and when run with Ubuntu
clients. Stop running this test for ARM and Ubuntu
clients until we find a solution.

Test-Parameters: trivial clientarch=aarch64 testlist=lnet-selftest
Test-Parameters: clientdistro=ubuntu1804 testlist=lnet-selftest
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I5c59b3a5dd42c9b6afcf5e0d1ce17e49efc1b44a
Reviewed-on: https://review.whamcloud.com/34543
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-10602 llite: add file heat support 99/34399/10
Li Xi [Mon, 5 Feb 2018 03:57:54 +0000 (22:57 -0500)]
LU-10602 llite: add file heat support

File heat is a special attribute fo files/objects which reflects
the access frequency of the files/objects.
File heat is mainly desinged for cache management. Caches like
PCC can use file heat to determine which files to be removed from
the cache or which files to fetch into cache.
This patch adds file heat support on llite level.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I168fc657f0c859311e5114191b60047646909be0
Reviewed-on: https://review.whamcloud.com/34399
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-9859 libcfs: remove wi_data from cfs_workitem 66/34466/2
NeilBrown [Tue, 19 Mar 2019 18:12:53 +0000 (14:12 -0400)]
LU-9859 libcfs: remove wi_data from cfs_workitem

In every case, the value passed via wi_data can be determined
from the cfs_workitem pointer using container_of().

So use container_of(), and discard wi_data.

Linux-commit: 19ae89d32503493315dec77919815d3add851389

Change-Id: Iefc4b6ebf40b48bd60a5820de05eb44746a041c0
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/34466
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
14 months agoLU-12081 mdt: rename shouldn't PDO lock if parent is remote 40/34440/2
Lai Siyao [Tue, 5 Mar 2019 04:30:15 +0000 (12:30 +0800)]
LU-12081 mdt: rename shouldn't PDO lock if parent is remote

In rename parent locking, if target parent is source parent, but
it's remote, rename shouldn't PDO lock on it because PDO lock is
for local lock.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib6ee1f70a50ddec3182c04c38a10ebbf2c384ccd
Reviewed-on: https://review.whamcloud.com/34440
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-12059 build: Update ZFS version to 0.7.13 93/34393/2
Nathaniel Clark [Mon, 11 Mar 2019 17:46:20 +0000 (13:46 -0400)]
LU-12059 build: Update ZFS version to 0.7.13

ZFS
 * test-runner: python3 support #8096
 * Fix flake 8 style warnings #7925 #7952
 * GCC 9.0: Fix ztest "directive argument is not a nul-terminated
   string" #8330
 * Linux 5.0 compat: Fix bio_set_dev() #8287
 * Linux 5.0 compat: Disable vector instructions on 5.0+ kernels
   #8259
 * Linux 5.0 compat: Fix SUBDIRs #8257
 * Linux 5.0 compat: Convert MS_* macros to SB_* #8264
 * Linux 5.0 compat: Use totalram_pages() #8263
 * Linux 5.0 compat: access_ok() drops 'type' parameter #8261
 * deadlock between mm_sem and tx assign in zfs_write() and page
   fault #7939
 * dkms: Enable debuginfo option to be set with zfs sysconfig file
   #8304
 * Bump commit subject length to 72 characters #8250
 * zfs.8 uses wrong snapshot names in Example 15 #8241
 * Add enclosure_symlinks option to vdev_id #8194
 * vdev_id: new slot type ses #6956
 * vdev_id: extension for new scsi topology #6592
 * Rename macro ZFS_MINOR due to Lustre conflict #8195
 * Add kernel module auto-loading #7287
 * Use autoconf variable for C preprocessor #8180
 * OpenZFS 9577 - remove zfs_dbuf_evict_key tsd #7602
 * Honor --with-mounthelperdir where applicable #6962
 * contrib/initramfs: switch to automake #6761

SPL
 * Linux 4.20 compat: Fix current_kernel_time() #8258
 * Linux 5.0 compat: Fix timespec_sub() tonyhutter/spl@a333b28
 * Linux 5.0 compat: Fix SUBDIRs #8257
 * Linux 5.0 compat: Use totalram_pages() #8263
 * deadlock between mm_sem and tx assign in zfs_write() and page
   fault #7939
 * Linux 4.18 compat: Use ktime_get_coarse_real_ts64() #8258
 * Use autoconf variable for C preprocessor #8180

Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I73fa5c2a9ddcf19683229d6fb9e61ff0835639ff
Reviewed-on: https://review.whamcloud.com/34393
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
14 months agoLU-8066 llite: don't use class_setup_tunables() 92/34292/8
James Simmons [Wed, 13 Mar 2019 14:03:28 +0000 (10:03 -0400)]
LU-8066 llite: don't use class_setup_tunables()

llite is very different from the other types of lustre devices.
Since this is the case llite should register independently. Doing
this allows us to cleanup the debugfs registering in the release
function of struct kobj_type. Also fix the improper handling to
linking it to the lustre_kset. Use kset_get() and kset_put() to
properly keep the reference count for the lustre_kset. This
lastly provides flexibility for making changes to
class_setup_tunables().

Change-Id: I676159caa7885485fdd57c908123214f68514227
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34292
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-6202 misc: remove LIBCFS_IOC_DEBUG_MASK ioctl 92/33692/5
Andreas Dilger [Tue, 20 Nov 2018 21:27:54 +0000 (14:27 -0700)]
LU-6202 misc: remove LIBCFS_IOC_DEBUG_MASK ioctl

Remove the LIBCFS_IOC_DEBUG_MASK ioctl, since the debug and subsystem
mask can be modified via /proc for a long time, and tools have not
used this ioctl since 2.6.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Idfcf85b2d4317bb4baac7ee9af55158f7b3ebbe5
Reviewed-on: https://review.whamcloud.com/33692
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-6202 misc: delete OBD_IOC_PING_TARGET ioctl 91/33691/5
Andreas Dilger [Tue, 20 Nov 2018 21:24:22 +0000 (14:24 -0700)]
LU-6202 misc: delete OBD_IOC_PING_TARGET ioctl

The OBD_IOC_PING_TARGET ioctl was removed from tool usage in
Lustre v2_5_60_0-27-g122aadd and replaced with a /proc interface.
It is no longer needed and can be removed.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7e4881edc50526e28b1a1aa039a4c986593ebbe5
Reviewed-on: https://review.whamcloud.com/33691
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-8130 nrs: convert NRS CRR to rhashtable 85/33485/6
James Simmons [Wed, 27 Feb 2019 04:17:47 +0000 (23:17 -0500)]
LU-8130 nrs: convert NRS CRR to rhashtable

Move away for the cfs hash implementation to rhashtable
for NRS CRR handling. Since rhashtable is lockless it
should also increase performance.

Test-Parameters:trivial testlist=sanityn envdefinitions=ONLY=77b

Change-Id: I817d35c8e36d7cb3397ffe8d00eee225245559b8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33485
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Nikitas Angelinas <nangelinas@cray.com>
14 months agoLU-8066 obd: make health_check sysfs compliant 31/25631/7
James Simmons [Fri, 15 Mar 2019 18:10:42 +0000 (14:10 -0400)]
LU-8066 obd: make health_check sysfs compliant

The patch http://review.whamcloud.com/16721 was
ported to the upstream client but was rejected
since it violating the sysfs one item rule. Change
the reporting of LBUG plus unhealthy to just
reporting LBUG. Move the reporting of which device
is unhealthy to a new debugfs file that mirrors
the sysfs file.

Change-Id: Ie1640399e97902272000313bb7ccdcbd2be6daf6
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25631
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-8955 ptlrpc: manage SELinux policy info for metadata ops 24/24424/38
Sebastien Buisson [Tue, 16 Aug 2016 08:17:40 +0000 (17:17 +0900)]
LU-8955 ptlrpc: manage SELinux policy info for metadata ops

Add SELinux policy info for following metedata operations:
- create
- open
- unlink
- rename
- getxattr
- setxattr
- setattr
- getattr
- symlink
- hardlink

On server side, get SELinux policy info from nodemap and compare
it with the one received from client.

Test-Parameters: serverbuildno=62488 serverjob=lustre-reviews testlist=sanity,sanity-selinux clientselinux
Test-Parameters: clientbuildno=4033 clientjob=lustre-reviews-patchless testlist=sanity,sanity-selinux clientselinux
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I16493d7c5713180fb065623b735d7348fc3f9140
Reviewed-on: https://review.whamcloud.com/24424
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-8955 ptlrpc: manage SELinux policy info at connect time 22/24422/34
Sebastien Buisson [Tue, 16 Aug 2016 12:53:03 +0000 (21:53 +0900)]
LU-8955 ptlrpc: manage SELinux policy info at connect time

At connect time, compute SELinux policy info on client side, and
send it over the wire.
On server side, get SELinux policy info from nodemap and compare
it with the one received from client.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9b4a206455f2c0b451f6b3ed7e3a85285592758e
Reviewed-on: https://review.whamcloud.com/24422
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-6142 utils: Fix style issues for l_getidentity.c 37/34437/3
Arshad Hussain [Sat, 9 Mar 2019 16:35:49 +0000 (22:05 +0530)]
LU-6142 utils: Fix style issues for l_getidentity.c

This patch fixes issues reported by checkpatch
for file lustre/utils/l_getidentity.c

Change-Id: If49272725e663c1e3ddb75acd0eadc58b79be35a
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/34437
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
14 months agoLU-6142 utils: Fix style issues for create_iam.c 36/34436/2
Arshad Hussain [Sat, 9 Mar 2019 14:27:54 +0000 (19:57 +0530)]
LU-6142 utils: Fix style issues for create_iam.c

This patch fixes issues reported by checkpatch for
file lustre/utils/create_iam.c

Test-parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ibb221c3e7fa53d6d3b87027e8604426bf416211c
Reviewed-on: https://review.whamcloud.com/34436
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
14 months agoLU-11798 grant: prevent overflow of o_undirty 48/33948/10
Alex Zhuravlev [Wed, 2 Jan 2019 18:11:29 +0000 (10:11 -0800)]
LU-11798 grant: prevent overflow of o_undirty

tgt_grant_inflate() returns a u64, and if tgd_blockbits and val are
large enough, can return a value >= 2^32.  tgt_grant_incoming()
assigns oa->o_undirty the returned value.  Since o_undirty is u32, it
can overflow.

This occurs with Lustre clients < 2.10 and a ZFS backend when the zfs
"recordsize" > 128k (the default).

In tgt_grant_inflate(), check the returned value and prevent o_undirty
from being assigned a value greater than 2^30.

Change-Id: I75b9065a524238df2d582e640418fdfa2f1e9a72
Signed-off-by: Alexey Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/33948
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11742 test: have libtool execute the test binaries 47/33947/6
James Simmons [Fri, 15 Mar 2019 00:38:31 +0000 (20:38 -0400)]
LU-11742 test: have libtool execute the test binaries

With the move to libtools the ability to run all the lustre
utilities form the source tree was lost. To work around this
the libtool -no-install flag was used to prevent the creation
of the libtool wrappers. While this worked to restore the
source tree sand box development new package breakage is showing.
This is due to the rpath being hard coded into the utilies when
-no-install is used and some platforms disable fixed rpaths.

A very similar problem exist for people who want to use gdb to
debug their projects application. gdb does not work on libtool
wrappers as well so the recommended approach to this type of
problem is to use the libtool execute command. This command
allows the execution of an external non project binary, like
gdb, with the projects real binary application. Apply this
approach to the lustre test suite so commands like kill can
be used to shutdown lustre utilies that are not installed into
the testing environment.

Change-Id: I74112f7250f1c43313d868c0edc7c8815d373002
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33947
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-12037 mdt: call mdt_dom_discard_data() after rename unlock 01/34401/2
Lai Siyao [Mon, 4 Mar 2019 12:20:14 +0000 (20:20 +0800)]
LU-12037 mdt: call mdt_dom_discard_data() after rename unlock

mdt_reint_rename() should drop all locks including global rename
lock, and then call mdt_dom_discard_data(), otherwise it may
cause deadlock.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I72cad3ee589c98c54f1e1281c39faa8779e562e8
Reviewed-on: https://review.whamcloud.com/34401
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-9193 security: return security context for metadata ops 31/26831/20
Bruno Faccini [Wed, 26 Apr 2017 10:35:28 +0000 (12:35 +0200)]
LU-9193 security: return security context for metadata ops

Security layer needs to fetch security context of files/dirs
upon metadata ops like lookup, getattr, open, truncate, and
layout, for its own purpose and control checks.
Retrieving the security context consists in a getxattr operation
at the file system level. The fact that the requested metadata
operation and the getxattr are not atomic can create a window
for a dead-lock situation where, based on some access patterns,
all MDT service threads can become stuck waiting for lookup lock
to be released and thus unable to serve getxattr for security context.
Another problem is that sending an additional getxattr request for
every metadata op hurts performance.

This patch introduces a way to get atomicity by having
the MDT return security context upon granted lock reply,
sparing the client an additional getxattr request.

Test-Parameters: serverbuildno=62488 serverjob=lustre-reviews testlist=sanity,sanity-selinux clientselinux
Test-Parameters: clientbuildno=4033 clientjob=lustre-reviews-patchless testlist=sanity,sanity-selinux clientselinux
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaaf4d93f8d3bf31b5a2c23e7db36b3cb3feb31ba
Reviewed-on: https://review.whamcloud.com/26831
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11596 tests: skip sanity test_64d for ARM 32/34432/2
Andreas Dilger [Fri, 15 Mar 2019 21:21:58 +0000 (15:21 -0600)]
LU-11596 tests: skip sanity test_64d for ARM

Add sanity.sh test_64d to the ALWAYS_EXCEPT list for this bug for
ARM, since it is also intermittently failing.

Test-Parameters: trivial clientarch=aarch64 testlist=sanity
Test-Parameters: testgroup=review-dne-part-1 testlist=sanity
Test-Parameters: testgroup=review-dne-zfs-part-1 testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifd52aa33d9bbf27303341ff0314322765b3ebbe5
Reviewed-on: https://review.whamcloud.com/34432
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-12068 tests: add debug for sanity-lfsck test_6b 17/34417/4
Andreas Dilger [Thu, 14 Mar 2019 07:54:41 +0000 (01:54 -0600)]
LU-12068 tests: add debug for sanity-lfsck test_6b

Dump the lfsck_namespace stats file on error to see if it provides
any more information about why the test is failing.

Test-Parameters: trivial testgroup=review-dne-zfs-part-2 testlist=sanity-lfsck
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If93202101f33f013b9f5ef56022f76c86b3ebbe5
Reviewed-on: https://review.whamcloud.com/34417
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-12065 lnd: increase CQ entries 73/34473/3
Amir Shehata [Wed, 20 Mar 2019 18:10:34 +0000 (11:10 -0700)]
LU-12065 lnd: increase CQ entries

Several sites have reported RDMA timeouts. Most of the timeouts
are occurring for transmits on the active_tx queue. Transmits are
placed on the active_tx queue until a completion is received. If
there isn't enough CQ entries available, it's possible for a
completions events to be delayed, causing these timeouts.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I9edad734b5860ce20af4977b4c1cdc07f25f078e
Reviewed-on: https://review.whamcloud.com/34473
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
14 months agoLU-10496 tgt: move FMD handling from OFD to target 90/34190/7
Mikhail Pershin [Fri, 1 Feb 2019 12:13:38 +0000 (15:13 +0300)]
LU-10496 tgt: move FMD handling from OFD to target

- move ofd/ofd_fmd.c to target/tgt_fmd.c with corresponding
  changes
- add FMD calls to the MDT for Data-on-MDT files
- per-target tunable parameters init/fini
- update related tests to be correctly used with DOM
- make sanity.sh test_36 to work again
- remove target_handle_ping() along with o_ping method in
  obd operations because it is not used anymore. Ping is
  fully handled in tgt_obd_ping()

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I24280a2a9610d05eb9655c73bb067f94ff251980
Reviewed-on: https://review.whamcloud.com/34190
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
14 months agoLU-10929 tests: skip sanity/315 if IO accounting is disabled 72/32072/7
Alex Zhuravlev [Thu, 19 Apr 2018 13:23:03 +0000 (16:23 +0300)]
LU-10929 tests: skip sanity/315 if IO accounting is disabled

just to avoid false fails

Test-Parameters: trivial

Change-Id: I1f7820f03fd3487aa2bd6187362d06934b12ad8e
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/32072
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
14 months agoLU-12044 ptlrpc: Translate HABEO_ macros 75/34375/5
Ben Evans [Tue, 5 Mar 2019 22:28:11 +0000 (17:28 -0500)]
LU-12044 ptlrpc: Translate HABEO_ macros

HABEO_CORPUS -> HAS_BODY
HABEO_CLAVIS -> HAS_KEY
HABEO_REFERO -> HAS_REPLY
MUTABOR -> IS_MUTABLE

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: I341cb899aeebd9cccf6ee8111112016c5e6dee53
Reviewed-on: https://review.whamcloud.com/34375
Reviewed-by: Ann Koehler <amk@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-12030 tests: Properly detect debug kernel use on rhel7.6 42/34342/4
Oleg Drokin [Thu, 28 Feb 2019 04:40:37 +0000 (23:40 -0500)]
LU-12030 tests: Properly detect debug kernel use on rhel7.6

kmalloc-128 slab seems to be gone, so let's use dma-kmalloc-128
instead.

Change-Id: Ice7f350ba2bc6cc733c0a98b0037e6f0980216c9
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34342
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
14 months agoLU-11926 ldlm: Lost lease lock on migrate error 82/34182/4
Andriy Skulysh [Tue, 4 Dec 2018 13:27:58 +0000 (15:27 +0200)]
LU-11926 ldlm: Lost lease lock on migrate error

All the file operations have the following locking order - parent,
child. If a lock for a child is returned to the client, the following
operations on this file are done by the child fid.

However, the migrate is an exception - it takes the lease lock first and
takes the PW parent lock next during the MDS_REINT.

At the same time, if there is a parallel racing operation (open) which
has taken a lock on parent (conflicting with the next MDS_REINT) and
is trying to take a lock on child - it is blocked until
the lease cancel comes.

The lease cancel is piggy-backed on the MDS_REINT RPC and is handled
at the end of the operation, trying to take the conflicting parent lock
first - thus a deadlock occurs.

At the same time, the lease lock is not supposed to block anything, it
is just an indicator on the server there is no other conflicting
operation has occurred during the migration - thus
set LDLM_FL_CANCEL_ON_BLOCK on it and the conflicting operation
will not be blocked.

In this case, the MDS_REINT will return -EAGAIN as the lease
is cancelled and the client will retry its migration.

Change-Id: Ib6cdc24ffe4ecb99d314a5466bcbb066a1d04dc1
Cray-bug-id: LUS-6811
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-on: https://review.whamcloud.com/34182
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11902 build: Remove obsolete lustre-utils files and entries 37/34137/2
Thomas Stibor [Wed, 30 Jan 2019 13:47:46 +0000 (14:47 +0100)]
LU-11902 build: Remove obsolete lustre-utils files and entries

DEB packages can be build for server or client, where naming
convention for server is:
* lustre-server-modules-*
* lustre-server-utils-*
and for client:
* lustre-client-modules-*
* lustre-client-utils-*
Previously the util package was named lustre-utils which
is now obsolete due to the client and server separation
and thus can be removed.

Test-Parameters: clientdistro=ubuntu1604 trivial
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: Id0a945d5b76e5c35cf858e3ab224efa342cde28d
Reviewed-on: https://review.whamcloud.com/34137
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11898 libcfs: do not calculate debug_mb if it is set 28/34128/2
Vladimir Saveliev [Tue, 29 Jan 2019 03:30:28 +0000 (06:30 +0300)]
LU-11898 libcfs: do not calculate debug_mb if it is set

debug_mb is libcfs module parameter. It should be possible to set it
via

modprobe libcfs libcfs_debug_mb=800

or via adding

options libcfs libcfs_debug_mb=800

to modules configuration.

Fixes: 7092309f32 ("LU-8066 libcfs: migrate to debugfs")
Test-Parameters: trivial
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Cray-bug-id: LUS-6936
Change-Id: I9da51e44a938a312e43b8a0781b49efc197f7ca9
Reviewed-on: https://review.whamcloud.com/34128
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
14 months agoLU-11885 test: reset quota upon test failed 08/34108/3
Wang Shilong [Fri, 25 Jan 2019 02:05:41 +0000 (10:05 +0800)]
LU-11885 test: reset quota upon test failed

Currently if test fail, EXIT will be trapped and
cleanup_quota_test() will be called to cleanup dirs.

However, we didn't reset quota in this case, this might
make following test fail because of this.

Also cleanup duplicated lines sine we have done reseting
inside cleanup_quota_test().

Test-Parameters: trivial testlist=sanity-quota
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ic70f59cdc1181473721e4f87b806b8203857fca8
Reviewed-on: https://review.whamcloud.com/34108
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-9010 ptlrpc: Change static defines to use macro for sec_gc.c 37/33937/4
Arshad Hussain [Fri, 28 Dec 2018 04:07:39 +0000 (23:07 -0500)]
LU-9010 ptlrpc: Change static defines to use macro for sec_gc.c

This patch replaces all mutex, locks, and wait qeueues
which are defined statically in file lustre/ptlrpc/sec_gc.c
with kernel provided macro.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: Ifc978e6d7b83a9d41078a98a0bcf5a2606ea7360
Reviewed-on: https://review.whamcloud.com/33937
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11821 lfsck: do not compare negatives to sizeof() 08/33908/7
Alex Zhuravlev [Fri, 21 Dec 2018 12:03:07 +0000 (15:03 +0300)]
LU-11821 lfsck: do not compare negatives to sizeof()

as sizeof() is unsigned.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib8cf8026a4e18ac9704a0294eaf7c57ecf678e03
Reviewed-on: https://review.whamcloud.com/33908
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
14 months agoLU-11752 osc: pass client page size during reconnect too 47/33847/8
Mikhail Pershin [Thu, 13 Dec 2018 10:11:05 +0000 (13:11 +0300)]
LU-11752 osc: pass client page size during reconnect too

Client page size is reported to the server in ocd_grant_blkbits
and server returns back device blocksize. During reconnect that
ocd_grant_blkbits contains server device blocksize which is used
by server as client page size wrongly.

Patch sets ocd_grant_blkbits to the client page size again during
reconnect so server will get expected information.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I14bba1d025e4e9fb99fd4bae4002463439ac265c
Reviewed-on: https://review.whamcloud.com/33847
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11678 quota: protect quota flags at OSC 47/33747/3
Hongchao Zhang [Tue, 22 Jan 2019 08:39:21 +0000 (16:39 +0800)]
LU-11678 quota: protect quota flags at OSC

There is no protection in OSC quota hash tracking the quota flags of
different qid, which could cause the previous request to modify the
quota flags which was set by the current request because the replies
could be out of order.

This patch also adds a lock to protect the operations on the quota
hash from different requests.

Test-Parameters: testlist=sanity-quota,sanity-quota,sanity-quota,sanity-quota

Change-Id: Ia6e5141265beacb9401dd533081fa0b85fd5ea6a
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33747
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-10258 lfs: lfs mirror copy command 20/33220/6
Bobi Jam [Sat, 22 Sep 2018 16:42:37 +0000 (00:42 +0800)]
LU-10258 lfs: lfs mirror copy command

Add "lfs mirror copy" command to copy a mirror's content to other
mirror(s) of a mirrored file.

Usage:

lfs mirror copy {--read-mirror|-i <id0>}
{--write-mirror|-o <id1>[,<id2>,...]} <mirrored_file>

Options:

--read-mirror|-i <id0>
  This option indicates the content of which mirror specified by id0
  needs to be read. The id0 is the numerical unique identifier for a
  mirror.

--write-mirror|-o <id1>[,<id2>,...]
  This option indicates the content of which mirror(s) specified by
  mirror IDs needs to be written. The mirror IDs are separated with
  comma.  If the mirror id -1 is used here, it means that all mirrors
  other than the read mirror are to be written.

Note:

Be ware that the written mirror(s) will be marked as non-stale
mirror(s), be careful that after using this command, you could get a
file with non-stale mirrors while containing different contents.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Id138368cdb29ec14b7c03a5db3b2dd1e0db5ea37
Reviewed-on: https://review.whamcloud.com/33220
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11409 osc: grant shrink shouldn't account skipped OSC 06/33206/3
Alex Zhuravlev [Thu, 20 Sep 2018 14:15:42 +0000 (17:15 +0300)]
LU-11409 osc: grant shrink shouldn't account skipped OSC

otherwise only the first 100 OSCs are subject to grant shrink procedure.

Change-Id: I65ed247b91422effb8f278d1991d4a5ba1c24814
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33206
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
14 months agoLU-11408 osc: propagate grant shrink interval immediately 04/33204/2
Alex Zhuravlev [Thu, 20 Sep 2018 07:47:03 +0000 (10:47 +0300)]
LU-11408 osc: propagate grant shrink interval immediately

currently the new interval (updated with lctl) will be used
only when the next shrink happens. with default interval it
will take at least 20 minutes. instead we should refresh it
immediately.

Change-Id: Id22824e48fbc50c1f464316ab5b574d1189bb0c5
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33204
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-8384 scripts: Add scripts to systemd for EL7 57/21457/6
Dmitry Eremin [Fri, 8 Jul 2016 21:15:37 +0000 (00:15 +0300)]
LU-8384 scripts: Add scripts to systemd for EL7

When rebooting a lustre client where Lustre filesystem is still
mounted, the shutdown hangs. This patch create a systemd service
that unmount the Lustre filesystems and unload the Lustre modules
when system is shutdown.

Test-Parameters: trivial
Change-Id: I1cfe84684e23b8861743241dfbc4d6e320ace4a6
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Gregoire Pichon <gregoire.pichon@atos.net>
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/21457
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-8047 llite: optimizations for not granted lock processing 65/19665/13
Andrew Perepechko [Thu, 7 Mar 2019 20:18:45 +0000 (12:18 -0800)]
LU-8047 llite: optimizations for not granted lock processing

This patch removes ll_md_blocking_ast() processing for
not granted locks. The reason is ll_invalidate_negative_children()
can slow down I/O significantly without a reason if there
are thousands or millions of files in the directory
cache.

Change-Id: Ic69c5f02f71c14db4b9609677d102dd2993f4feb
Seagate-bug-id: MRP-3409
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/19665
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-6836 test: re-add test 4a to sanity-quota for ZFS 43/34143/4
Hongchao Zhang [Thu, 24 Jan 2019 19:45:05 +0000 (14:45 -0500)]
LU-6836 test: re-add test 4a to sanity-quota for ZFS

The ZFS sync performance has been improved, it's time to add test
4a back into sanity-quota for ZFS, and also increase the grace time
a little for ZFS.

Test-Parameters: trivial
Test-Parameters: envdefinitions=ONLY=4a fstype=zfs testlist=sanity-quota,sanity-quota,sanity-quota

Change-Id: I32dd76686cdd289b49e36efff3abd6691e76ef57
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34143
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
14 months agoLU-11835 mdt: return DOM size on open resend 44/34044/5
Mikhail Pershin [Wed, 16 Jan 2019 13:24:58 +0000 (16:24 +0300)]
LU-11835 mdt: return DOM size on open resend

DOM size is returned along with DOM lock always, but it is
not true with open resend.

Patch fixes that issue and adds test case.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I73d43933f781f192e9aa8c6ee388a043dab5bde9
Reviewed-on: https://review.whamcloud.com/34044
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-8260 osd-ldiskfs: osd_fiemap_get() fix address space mismatch 78/33878/6
Arshad Hussain [Tue, 4 Dec 2018 18:20:59 +0000 (23:50 +0530)]
LU-8260 osd-ldiskfs: osd_fiemap_get() fix address space mismatch

There was an address space mismatch in function
osd_fiemap_get() as this uses "__user" qualifier
in fiemap_extent buffer. Since this buffer is created
under kernel and again passed to another call, this
may fail under some configuration.

This patch address this issue by modifying the
address space limit by using get_fs() and set_fs()
call suggesting that the pointers are intact and
secure.

Change-Id: I25048faecd3475d5e91e25e6a47e065e49e36b26
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33878
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-6142 obdclass: Fix style issues for obd_config.c 82/33082/9
Arshad Hussain [Sat, 25 Aug 2018 23:59:42 +0000 (05:29 +0530)]
LU-6142 obdclass: Fix style issues for obd_config.c

This patch fixes issues reported by checkpatch
for file lustre/obdclass/obd_config.c

Change-Id: If97513fe594ee76c9e153c33d644cd94c48f82c0
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33082
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11999 dne: performance improvement for file creation 91/34291/4
Jinshan Xiong [Sun, 24 Feb 2019 22:32:41 +0000 (14:32 -0800)]
LU-11999 dne: performance improvement for file creation

This is to remove an obsoleted code where it causes drastic
performance degradation. This code is written before PERM lock
is introduced, and it requests UPDATE lock at path walk for
remote directory, which will be cancelled at later file creation.

Tests result before and after this patch is applied:

Test case:
rm -rf /mnt/lustre_purple/testdir
lfs mkdir -i 0 /mnt/lustre_purple/testdir
lfs mkdir -i 2 /mnt/lustre_purple/testdir/dir2
./lustre-release/lustre/tests/createmany -o \
/mnt/lustre_purple/testdir/dir2/f 10000

Before the patch is applied:
total: 10000 open/close in 12.82 seconds: 780.22 ops/second

After the patch is applied:
total: 10000 open/close in 4.89 seconds: 2044.75 ops/second

Signed-off-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Change-Id: Ib474dc28d6edc7d15801b6821edc0e1d108bb4b6
Reviewed-on: https://review.whamcloud.com/34291
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11964 mdc: prevent glimpse lock count grow 61/34261/8
Mikhail Pershin [Thu, 14 Feb 2019 21:51:00 +0000 (00:51 +0300)]
LU-11964 mdc: prevent glimpse lock count grow

DOM locks matching tries to ignore locks with
LDLM_FL_KMS_IGNORE flag during ldlm_lock_match() but
checks that after ldlm_lock_match() call. Therefore if
there is any lock with such flag in queue then all other
locks after it are ignored and new lock is created causing
big amount of locks on single resource in some access
patterns.
Patch extends lock_matches() function to check flags to
exclude and adds ldlm_lock_match_with_skip()p to use that
when needed.
Corresponding test was added in sanity-dom.sh

Test-Parameters: testlist=sanity-dom
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic45ca10f0e603e79a3a00e4fde13a5fae15ea5fc
Reviewed-on: https://review.whamcloud.com/34261
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-10949 mdt: lost reference on mdt_md_root 81/34181/3
Andriy Skulysh [Wed, 20 Feb 2019 10:48:03 +0000 (12:48 +0200)]
LU-10949 mdt: lost reference on mdt_md_root

mdt_remote_object_lock_try() drops object
reference in case of an error but if the
request was sent to a server it is decreased
again via failed_lock_cleanup()

Add ldlm_created_callback. It is called after
lock creation, so we can safely add a reference
to l_ast_data and drop it only in BL AST handler.

Cray-bug-id: LUS-7013
Change-Id: Iaf98c620804f2de4528689e44e957a9fb0073162
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-on: https://review.whamcloud.com/34181
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-10496 ofd: move FMD to the target code 76/34176/7
Mikhail Pershin [Thu, 31 Jan 2019 13:15:28 +0000 (16:15 +0300)]
LU-10496 ofd: move FMD to the target code

- make FMD structures common for all targets
- adapt FMD functionality to be isolated from OFD for
  further move to the target code.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I9f67f14e4132205cca67aa778b990bb3b45c30be
Reviewed-on: https://review.whamcloud.com/34176
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
14 months agoLU-8066 fid: use LDEBUGFS_SEQ_* macro 72/34372/3
James Simmons [Mon, 4 Mar 2019 16:06:52 +0000 (11:06 -0500)]
LU-8066 fid: use LDEBUGFS_SEQ_* macro

Lustre has LPROC_SEQ_* for proc handling and LDEBUGFS_SEQ_* macros
for debugfs handling. While similar using the wrong macro can
break things. To avoid that chance lets move the fid subsystem to
the LDBEUGFS_SEQ_* macro since its already moved to debugfs.

Change-Id: I3936c72f9fb58a38847822dad21c4b6f5e1d7a78
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34372
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11721 lod: limit statfs ffree if less than OST ffree 67/34167/8
Andreas Dilger [Sun, 3 Feb 2019 00:11:00 +0000 (17:11 -0700)]
LU-11721 lod: limit statfs ffree if less than OST ffree

If the OSTs report fewer total free objects than the MDTs, then
use the free files count reported by the OSTs, since it represents
the minimum number of files that can be created in the filesystem
(creating more may be possible, but this depends on other factors).
This has always been what ll_statfs_internal() reports, but the
statfs aggregation via the MDT missed this step in lod_statfs().

Fix a minor defect in sanity test_418() that would let it loop
forever until the test was killed due to timeout if the "df -i"
and "lfs df -i" output did not converge.

Fixes: b500d5193360 ("LU-10018 protocol: MDT as a statfs proxy")
Fixes: 263e80f4572b ("LU-11721 tests: wait for statfs to update ...")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id8d7b7edfd854f1ec30bfbbb85f04b0c973ebbe5
Reviewed-on: https://review.whamcloud.com/34167
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Nikitas Angelinas <nangelinas@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11860 lnet: support config of LNDs with numeric intf name 28/34028/3
Gregoire Pichon [Mon, 14 Jan 2019 19:52:58 +0000 (20:52 +0100)]
LU-11860 lnet: support config of LNDs with numeric intf name

This patch adds support for the net configuration of LNDs that
have numeric interface name (PTL4LND, GNILND). The GNILND case has
already been treated with a specific fix (see patch a29eb587).

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I10556f9c78ec332bca3344990e509434f904ffc0
Reviewed-on: https://review.whamcloud.com/34028
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-12039 tests: fix 'lfs mkdir' in sanity-selinux 68/34368/2
Sebastien Buisson [Mon, 4 Mar 2019 07:43:42 +0000 (08:43 +0100)]
LU-12039 tests: fix 'lfs mkdir' in sanity-selinux

sanity-selinux test_2b and test_20c assume that directory created
with 'lfs mkdir' will be on MDT0, but they have to use the '-i 0'
flag to make sure.

Test-Parameters: trivial testlist=sanity-selinux clientselinux
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I57df7aed18673b9f7f1301b45e621bb76ebb9845
Reviewed-on: https://review.whamcloud.com/34368
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
14 months agoLU-6142 obdclass: Fix style issues for obd_mount.c 66/34366/3
Arshad Hussain [Sun, 12 Aug 2018 09:18:29 +0000 (14:48 +0530)]
LU-6142 obdclass: Fix style issues for obd_mount.c

This patch fixes issues reported by checkpatch
for file lustre/obdclass/obd_mount.c

Change-Id: Icd0ccdf25d69f01b690b9381864a96c8dc45dd96
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/34366
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
14 months agoLU-11836 ldlm: don't convert wrong resource 64/34264/4
Mikhail Pershin [Fri, 15 Feb 2019 09:14:30 +0000 (12:14 +0300)]
LU-11836 ldlm: don't convert wrong resource

During enqueue the returned lock may have different resource
and local client lock replaces resource too. But there is
a valid race with bl_ast and reply from server, so BL AST
may come earlier and find client lock with old resource.
In that case ldlm_handle_bl_callback() should proceed with
normal cancel and don't use cancel_bits for lock convert.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ib7fd98ce73821b1e3207e7f2bfba0e0acfdc2380
Reviewed-on: https://review.whamcloud.com/34264
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11922 ldiskfs: make dirdata work with metadata_csum 19/34219/4
Li Dongyang [Sat, 9 Feb 2019 05:37:29 +0000 (16:37 +1100)]
LU-11922 ldiskfs: make dirdata work with metadata_csum

Handle ext4_dir_entry_tail correctly, which is a bogus dir entry
contains the checksum at the end of dir leaf block.

Fix how we get to the limit on the dx_root, we can't assume the
rec_len as 12 as . and .. in front of the dx_root have dirdata.

Also includes another fix for large_dir, where we should update
checksum for dx_node calling ext4_handle_dirty_dx_node(). The change
also makes large_dir patch more consistent with the upstream version.

With this we can enable metadata_csum on the targets.

Test-Parameters: fstype=ldiskfs envdefinitions=LDISKFS_MKFS_OPTS="-O metadata_csum"
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I04df8c30d9d423111e2b4031a7e4b9058101016f
Reviewed-on: https://review.whamcloud.com/34219
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
14 months agoNew tag 2.12.52 2.12.52 v2_12_52
Oleg Drokin [Fri, 15 Mar 2019 23:15:38 +0000 (19:15 -0400)]
New tag 2.12.52

Change-Id: I7de1c524f2cc413f4028851406ccf50d43bc8d43
Signed-off-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11765 ofd: return EAGAIN during 1st CLEANUP_ORPHAN 36/33836/6
Sergey Cheremencev [Wed, 24 Oct 2018 10:23:43 +0000 (13:23 +0300)]
LU-11765 ofd: return EAGAIN during 1st CLEANUP_ORPHAN

During the 1st CLEANUP_ORPHAN after failover some objects
could absent - they haven't been recreated yet. Issue exists
when MDS last_id much grater than OST last_id and ofd should
recreate thousands of objects. Some of these objects could
be assigned to a FID and requested by client through
glimpse RPC. Thus if object is not found return EAGAIN instead
of ENOENT during the 1st CLEANUP_ORPHAN.

Patch is also adding a test to reproduce the issue.
Test adds a delay to osd_trans_commit_cb() causing
large number OST objects not written to the disk
after failover. And checks that all objects have been
successfully recreated after failover.
The test works only with FAILURE_MODE=HARD option.

Cray-bug-id: LUS-6414
Change-Id: Ia6899b4c1c35e1681f49faf1cb93a501ad159ec2
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://es-gerrit.dev.cray.com/154151
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/33836
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11855 utils: move declarations out of local blocks 26/34026/6
Alex Zhuravlev [Tue, 5 Feb 2019 16:45:19 +0000 (19:45 +0300)]
LU-11855 utils: move declarations out of local blocks

few variables were declared within local blocks, but used by
pointer when blocks are closed. this schema used to work with
older GCC (due to trivial stack management), but not with GCC8.

Change-Id: Ibd02a72264d50609ccf3c5bc5252e45de4160b9e
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34026
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11943 llog: Reset current log on ENOSPC 47/34347/4
Patrick Farrell [Thu, 28 Feb 2019 18:02:13 +0000 (13:02 -0500)]
LU-11943 llog: Reset current log on ENOSPC

The original LU-10527 patch:
"LU-10527 obdclass: don't recycle loghandle upon ENOSPC"
https://review.whamcloud.com/#/c/30897/

Kept the current log on ENOSPC.

This appears to cause llog corruption on failover, and the
other part of the original patch (removing an incorrect
assert) should be sufficient to fix the original issue.

Fixes: 5761b9576d39 ("LU-10527 obdclass: don't recycle loghandle upon ENOSPC")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie5c0ab77940c1be0ec1f166e4d38080b254bed5c
Reviewed-on: https://review.whamcloud.com/34347
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-12033 build: Git ignore generated *.install files 44/34344/2
Thomas Stibor [Thu, 28 Feb 2019 13:37:46 +0000 (14:37 +0100)]
LU-12033 build: Git ignore generated *.install files

Building client or server DEB packages generates files:
* lustre-server-utils.install
* lustre-client-utils.install
from description files
* lustre-server-utils.install.in
* lustre-client-utils.install.in
Any modifications take place in *.install.in files,
thus gitignore the generated *.install files.

Test-Parameters: trivial
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: Ib7812f3407b35a74ef8bd63e630bda24fcbd5b48
Reviewed-on: https://review.whamcloud.com/34344
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
14 months agoLU-11329 misc: reorder Git .mailmap file 09/34109/3
Andreas Dilger [Tue, 22 Jan 2019 23:09:41 +0000 (16:09 -0700)]
LU-11329 misc: reorder Git .mailmap file

If there are multiple remappings of the same person (e.g. CFS to
Sun to Oracle to Whamcloud to Intel to Whamcloud2), the entries
in the .mailmap file need to be in chronological order, so that
older entries are mapped to the next email, which is mapped to
the next one, etc.  Otherwise, only a single remapping is done.

Add additional mapping for Patrick Farrell.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6d7f2977da5c24707e3d54fddfdabf94c28e3f0b
Reviewed-on: https://review.whamcloud.com/34109
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11880 build: allow Debian build on Raspbian 85/34085/4
Andreas Dilger [Wed, 26 Dec 2018 09:03:55 +0000 (02:03 -0700)]
LU-11880 build: allow Debian build on Raspbian

Allow the Debian build to work on Raspbian and other Debian and
Ubuntu variants that use the same build machinery.  Add armhf
to the list of allowed CPU architectures.

Test-Parameters: trivial clientdistro=ubuntu1804
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iedc90ad32754f81a3d387a66409f07aaa305a5b1
Reviewed-on: https://review.whamcloud.com/34085
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
14 months agoLU-11974 llapi: improve llapi_layout_get_by_xattr(3) API 76/34276/8
Li Xi [Tue, 19 Feb 2019 02:33:20 +0000 (10:33 +0800)]
LU-11974 llapi: improve llapi_layout_get_by_xattr(3) API

llapi_layout_get_by_xattr() assumes that the lum has already
been properly swapped by llapi_layout_swab_lov_user_md().
However, llapi_layout_swab_lov_user_md() function is not
exported, so external tool won't be able to use it.

Instead of exporting a lot of APIs, this patch include the
swab functions into llapi_layout_get_by_xattr() and add an
argument flags to the API.

Change-Id: I9fbf0f0ba66660d2f382fb20b03f069c1a7afad5
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/34276
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
14 months agoLU-11243 lod: fix assertion and hang upon lod_add_device failure 94/32994/7
Wang Shilong [Mon, 10 Dec 2018 05:45:33 +0000 (13:45 +0800)]
LU-11243 lod: fix assertion and hang upon lod_add_device failure

There are two problems:

See following assertion:

    lod_add_device() lustre-OSTe42a-osc-MDT0000:
                     can't set up pool, failed with -12
    osp_disconnect() ASSERTION( imp != ((void *)0) ) failed:
    osp_disconnect() LBUG
    CPU: 1 PID: 10059 Comm: llog_process_th

Problem is obd_disconnect() will cleanup @imp and set NULL.
 ->osp_obd_disconnect
    ->class_manual_cleanup
       ->class_process_config
          ->class_cleanup
             ->obd_precleanup
                ->osp_device_fini
                   ->client_obd_cleanup

While ldo_process_config() will try to access @imp again:
 ->ldo_process_config
    ->osp_shutdown
       ->osp_disconnect
          ->LASSERT(imp != NULL)

Another problem is if we failed before obd_connect().
we will hang on with mount:
 ->ldo_process_config
    ->osp_shutdown
       ->osp_disconnect
          ->ptlrpc_disconnect_import
             ->rc = l_wait_event(imp->imp_recovery_waitq,
                                 !ptlrpc_import_in_recovery(imp), &lwi);

Since connect is not called, imp state will stay LUSTRE_IMP_NEW.
Fix this by check whether we are in recovery properly, only consider
we are in recovery if we are in following states:

 LUSTRE_IMP_CONNECTING = 4,
 LUSTRE_IMP_REPLAY     = 5,
 LUSTRE_IMP_REPLAY_LOCKS = 6,
 LUSTRE_IMP_REPLAY_WAIT  = 7,
 LUSTRE_IMP_RECOVER    = 8,

Change-Id: I2113b95a421bae7117f3057d5f0fdf78db95caa3
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/32994
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-12042 utils: Remove stray waring from lfs mkdir 74/34374/2
Nathaniel Clark [Tue, 5 Mar 2019 14:22:09 +0000 (09:22 -0500)]
LU-12042 utils: Remove stray waring from lfs mkdir

lfs mkdir -i shouldn't issue a warning about -m being depricated.

Test-Parameters: trivial mdtcount=2 mdscount=2
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I7012ff8ec7499b07d6463c2637c8bedcb3976fe2
Reviewed-on: https://review.whamcloud.com/34374
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
15 months agoLU-12020 llite: make sure name pack atomic 30/34330/3
Wang Shilong [Tue, 26 Feb 2019 14:38:29 +0000 (22:38 +0800)]
LU-12020 llite: make sure name pack atomic

We are trying to access dentry name directly and pass it
down without holding @d_lock, this is racy and possibly
make us trigger assertions:

(mdc_lib.c:137:mdc_pack_name()) ASSERTION( lu_name_is_valid_2(buf, cpy_len) ) failed:

Fix the problem by allocting memory and copy name with @d_lock
held.

Change-Id: Iae0066661f42e8fca9358cbedd9cb21828779bbb
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34330
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-12018 quota: do not start a thread under memory pressure 28/34328/2
Alex Zhuravlev [Tue, 26 Feb 2019 07:31:53 +0000 (10:31 +0300)]
LU-12018 quota: do not start a thread under memory pressure

this leads to a deadlock as kthreadd creating new threads
can get stuck waiting for memory as well:

PID: 2 TASK: ffff88015d1e0fb0 CPU: 3 COMMAND: "kthreadd"

Change-Id: I88f14da24ea64dcc02a9fd1f4a9c03f5771f8fda
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34328
Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-10143 tests: Add version check for interop 85/34285/5
Patrick Farrell [Wed, 20 Feb 2019 22:46:20 +0000 (17:46 -0500)]
LU-10143 tests: Add version check for interop

Without the fix for LU-10143, these tests will fail, so add
a version check for interop.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I65eead5d5252ec6abf5d8de68d73e4f9b690d030
Reviewed-on: https://review.whamcloud.com/34285
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
15 months agoLU-11982 utils: Correct lfs migrate help 84/34284/3
Patrick Farrell [Wed, 20 Feb 2019 20:04:41 +0000 (15:04 -0500)]
LU-11982 utils: Correct lfs migrate help

Correct lfs migrate help to correctly describe
block/non-block behavior.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id7c01df08514ecc1b3968ef92a466bd8f5d9e656
Reviewed-on: https://review.whamcloud.com/34284
Tested-by: Jenkins
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11944 llite: Lock inode on tiny write if setuid/setgid set 18/34218/4
Ann Koehler [Fri, 8 Feb 2019 21:41:37 +0000 (15:41 -0600)]
LU-11944 llite: Lock inode on tiny write if setuid/setgid set

During a write, the setuid/setgid bits must be reset if they are
enabled and the user does not have the correct permissions. Setting
any file attributes, including setuid and setgid, requires the inode
to be locked. Writes became lockless with the introduction of
LU-1669. Locking the inode in the setuid/setgid case was added to
vvp_io_write_start() as a special case. The inode locking was not
included when support for tiny writes was added with LU-9409. This
mod adds the necessary inode lock/unlock calls to ll_do_tiny_write().

If the inode is not locked when setuid/setgid are reset, the kernel
will issue a one time warning and Lustre may hang trying to get the
inode lock in ll_setattr_raw().

Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I5e8a98789828de52dbff4226958741320aba92e6
Reviewed-on: https://review.whamcloud.com/34218
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-9859 libcfs: use a workqueue for rehash work. 69/34169/3
NeilBrown [Mon, 11 Feb 2019 15:46:14 +0000 (10:46 -0500)]
LU-9859 libcfs: use a workqueue for rehash work.

lustre has a work-item queuing scheme that provides the
same functionality as linux work_queues.
To make the code easier for linux devs to follow, change
to use work_queues.

Linux-commit: 0aa211e39857f17e24126c47f6e3fe3b971344b3

Change-Id: I1600ea1ef8769f1f6489b81fd578685ea58f9cb6
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/34169
Tested-by: Jenkins
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11894 lnet: check for asymmetrical route messages 19/34119/4
Sebastien Buisson [Mon, 28 Jan 2019 15:16:42 +0000 (00:16 +0900)]
LU-11894 lnet: check for asymmetrical route messages

Asymmetrical routes can be an issue when debugging network,
and allowing them also opens the door to attacks where hostile
clients inject data to the servers.

In order to prevent asymmetrical routes, add a new lnet kernel
module option named 'lnet_drop_asym_route'. When set to non-zero,
lnet_parse() will check if the message received from a remote peer
is coming through a router that would normally be used by this node
to reach the remote peer. If it is not the case, then it means we
are dealing with an asymmetrical route message, and the message will
be dropped.

The check for asymmetrical route can also be switched on/off with
the command 'lnetctl set drop_asym_route 0|1'. And this parameter is
exported/imported in Yaml.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I06fb23d9e46984d79c14fa9b53b2fa04ce3c50c5
Reviewed-on: https://review.whamcloud.com/34119
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11891 utils: getstripe use --mdt-index consistently 16/34116/2
Andreas Dilger [Sun, 27 Jan 2019 18:23:48 +0000 (11:23 -0700)]
LU-11891 utils: getstripe use --mdt-index consistently

LU-10856 fixed most usages of "warning: '-M' deprecated,
use '--mdt-index' or '-m' instead" but missed a few in
cases in sanity test_271d, test_271e, and test_271f.
Fix those tests to use "--mdt-index".

Also, lfs has a few places were the usage of "--mdt-index"
and "--mdt" is inconsistent.  Fix those options to be used
consistently across all commands.

Fixes: 6c617a3d56 ("LU-10856 tests: remove deprecated lfs ...")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I013b2198f3a39533da9a0067a0bf5846604b3052
Reviewed-on: https://review.whamcloud.com/34116
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-9855 obd: use ldo_process_config for mdc and osc layer 06/34106/7
James Simmons [Thu, 24 Jan 2019 16:59:32 +0000 (11:59 -0500)]
LU-9855 obd: use ldo_process_config for mdc and osc layer

Both the mdc and osc layer use the lu_device infrastructure but
we don't use ldo_process_config() which is preferred over the
currently used obd_process_config() handling. Migrate to the
lu_device ldo_process_config() for both mdc and osc layer.

Change-Id: I4d5e84c48377283148133e6338ef7257c44b89a3
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34106
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11849 utils: fix to make exclude projid works 05/34005/4
Wang Shilong [Thu, 10 Jan 2019 15:32:14 +0000 (23:32 +0800)]
LU-11849 utils: fix to make exclude projid works

We intended to use projid not uid here, fix it.
Also add ! --projid options test to cover this.

Test-Parameters: trivial testlist=sanity-quota
Change-Id: I64c3f1c68885947d0e91626525ee037756e1d7d8
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34005
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
15 months agoLU-11830 lov: avoid signed vs. unsigned comparison 21/33921/8
Andreas Dilger [Wed, 26 Dec 2018 09:29:55 +0000 (02:29 -0700)]
LU-11830 lov: avoid signed vs. unsigned comparison

In the expansion of do_div64() GCC complains about pointer comparison
because loff_t is not a u64 variable as it should be.  lov_do_div64()
also has signed vs. unsigned comparisons due to a signed loff_t.
Change lov_do_div() to use a 64-bit variable for do_div() instead of
loff_t to avoid these warnings.

Change OST_MAXREQSIZE and friends to be consistently unsigned values
to avoid compiler warnings.

Fix "lfs mirror resync" to avoid comparing signed and unsigned valued.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6df31a2c0d75e5777f471fe8cb252715dd85a5b1
Reviewed-on: https://review.whamcloud.com/33921
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-8066 utils: have llapi_target_iterate use sysfs tree 99/33799/16
James Simmons [Fri, 22 Feb 2019 15:41:34 +0000 (10:41 -0500)]
LU-8066 utils: have llapi_target_iterate use sysfs tree

Update llapi_target_iterate() to not use 'devices' but collect the
data from the lustre sysfs tree itself.

Change-Id: If100b4918bdcc8b24e72f37127048a32a808310f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33799
Tested-by: Jenkins
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11010 tests: remove return after skip for conf-sanity 35/32735/7
James Nunez [Wed, 27 Jun 2018 15:52:22 +0000 (09:52 -0600)]
LU-11010 tests: remove return after skip for conf-sanity

The skip() routine now contains a call to exit. All calls
to skip() and skip_env() should be reviewed and calls to
return() that followed skip() should be removed.

This is the fifth patch in a series of patches that
remove calls to return() after skip() in the Lustre test
suites.

A comment is added to skip_env() defining when it should
be used.

Calls to return() after skip() are removed for:
conf-sanity.sh
functions.sh

Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Idbcdffda38aaac07f128ae42a2ffcda8986afc33
Reviewed-on: https://review.whamcloud.com/32735
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-8207 scripts: add auto-stripe option to lfs_migrate 52/20552/18
Nathan Dauchy [Mon, 2 Jul 2018 14:21:35 +0000 (10:21 -0400)]
LU-8207 scripts: add auto-stripe option to lfs_migrate

Add a "-A" flag to lfs_migrate, which will automatically select the
stripe count as the file is rewritten. Initial algorithm to
determine stripe count is sqrt(size_in_GB)+1, with an additional cap
on object size, though the algorithm or thresholds could conceivably
change in the future.  The primary intent for this feature is to be
able to give users a tool to fix stripe settings on existing files
based on file size.

A new "-C" flag specifies the object size cap.  On each OST, the
amount of space available for migration is capped by dividing the
free space of the smallest OST by the specified value.

A new "-M" flag allows OSTs with free space less than the specified
value to be considered unavailable for migration.

A new "-v" flag increases verbosity to help debug what is being done.

A new "-X" flag limits the amount of free space on each OST that
can be used for migration to the specified value.  This flag is
useful for testing by simulating OSTs that are nearly full.

A new sanity test verifies the operation of the new "-A" flag.

Test-Parameters: trivial
Signed-off-by: Nathan Dauchy <nathan.dauchy@nasa.gov>
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I9ce8b64e028d9abb66b6b49cf7675263fd7202f0
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/20552
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
15 months agoLU-11502 migrate: link parents lock may deadlock 25/33325/3
Lai Siyao [Fri, 31 Aug 2018 20:23:11 +0000 (04:23 +0800)]
LU-11502 migrate: link parents lock may deadlock

To cancel link parent lock, it should cancel all locks taken including
source parent locks, otherwise it may cause deadlock, so lock retry
should start from beginning.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I820d0e1664dbb405d6ed8245bb4ca2137140c323
Reviewed-on: https://review.whamcloud.com/33325
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-8365 ldiskfs: procfs entries for mballoc 42/21142/14
Lokesh Nagappa Jaliminche [Mon, 4 Jul 2016 09:04:20 +0000 (14:34 +0530)]
LU-8365 ldiskfs: procfs entries for mballoc

Export mballoc streaming block allocator variables
mb_last_group and mb_last_start through procfs.

Test-Parameters: testgroup=review-ldiskfs
Change-Id: I5dd00503a81c6819751c9f99b64615b497ef4e28
Cray-bug-id: LUS-3176
Signed-off-by: Lokesh Nagappa Jaliminche <lokesh.jaliminche@seagate.com>
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/21142
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
15 months agoLU-11413 lnet: use right address for routing message 32/34032/3
Alexey Lyashkov [Tue, 22 Jan 2019 08:41:00 +0000 (11:41 +0300)]
LU-11413 lnet: use right address for routing message

msg_initiator is real sender address, so use this address as
hash source to better distribution against CPT on server side.

Cray-bug-id: LUS-6841
Test-Parameters: trivial
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: Ie4487ea29d9db458564c66518270ad82b5ffae49
Reviewed-on: https://review.whamcloud.com/34032
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11413 lnet: use right rtr address 31/34031/3
Alexey Lyashkov [Tue, 22 Jan 2019 08:40:59 +0000 (11:40 +0300)]
LU-11413 lnet: use right rtr address

use a sender router to avoid credits distribution problem.
Sender is preferable rtr now.

Cray-bug-id: LUS-6490
Test-Parameters: trivial
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: Ic7cf57820176979a52675dcc74342c2e26335e73
Reviewed-on: https://review.whamcloud.com/34031
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11775 osc: check if opg is in lru list without locking 60/33860/4
Li Dongyang [Fri, 14 Dec 2018 01:36:22 +0000 (12:36 +1100)]
LU-11775 osc: check if opg is in lru list without locking

osc_lru_use is called for every page queued for io,
we can just check if the osc_page is in the lru list
without taking the cl_lru_list_lock and return if not
as a fast path.
Note we still need to do the check again after locking
as it could be removed from the lru list by another thread.

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I1587b6b5547ae5a7a8bfe32a78361bb888c85d5b
Reviewed-on: https://review.whamcloud.com/33860
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11951 ptlrpc: reset generation for old requests 21/34221/9
Alex Zhuravlev [Mon, 11 Feb 2019 11:27:54 +0000 (14:27 +0300)]
LU-11951 ptlrpc: reset generation for old requests

all requests generated while the import is changing from
FULL to IDLE need to be moved to the new generation.

Change-Id: I59d9b92680c724132dba9c7315c26e9851c5d5d2
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34221
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-8299 llite: ll_fault should fail for insane file offsets 42/34242/5
Alexander Zarochentsev [Tue, 12 Feb 2019 15:28:37 +0000 (18:28 +0300)]
LU-8299 llite: ll_fault should fail for insane file offsets

A page fault for a mmapped lustre file at offset large than
2^63 cause Lustre client to hang due to wrong page index
calculations from signed loff_t.
There is no need to do such calclulations but perform
page offset sanity checks in ll_fault().

Cray-bug-id: LUS-1392
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Change-Id: Ia492083ee4bdc23edfcbf88cb6d7e9726b2ca80c
Reviewed-on: https://review.whamcloud.com/34242
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
15 months agoLU-11838 llite: remove assert for acl refcount 36/34236/3
James Simmons [Tue, 12 Feb 2019 23:59:19 +0000 (18:59 -0500)]
LU-11838 llite: remove assert for acl refcount

The purpose of this asssert to was to ensure lustre
was properly managing its posix_acl access. This test
is invalid due to the VFS layer also taking references
on the posix_acl. In reality their is no simple way to
detect this class of mistakes.

* lastest kernels remove this refcount *

Linux-commit: 6a42e615a28bad49f2e04829486e94190c066390

Change-Id: I167f2de449a2e8357517f33c2e81a25b25104d57
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/34236
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11960 build: Add missing libssl-dev DEB package 33/34233/3
Thomas Stibor [Tue, 12 Feb 2019 13:30:51 +0000 (14:30 +0100)]
LU-11960 build: Add missing libssl-dev DEB package

Building Lustre client DEB packages on Debian fails due to missing
package libssl-dev and results in error:
"No such file or directory #include <openssl/evp.h>"
Add required package libssl-dev into "make debs" chain.

Test-Parameters: clientdistro=ubuntu1804 envdefinitions=SANITY_EXCEPT=802a trivial
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: Ib99cd744b2d44d3f6c1915e2f2d2da7d83e07cae
Reviewed-on: https://review.whamcloud.com/34233
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11206 tests: Use import_ready to check IDLE 25/34225/2
Patrick Farrell [Mon, 11 Feb 2019 17:47:09 +0000 (12:47 -0500)]
LU-11206 tests: Use import_ready to check IDLE

When checking if a client/OST import is up, we have to
check for IDLE as well as FULL.

wait_osc_import_ready is provided for this, but a few spots
don't use it, so they occasionally fail.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I826659a7f5953dee4e4551c1177479ef742b5589
Reviewed-on: https://review.whamcloud.com/34225
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-9706 dt: remove dt_txn_hook_commit() 12/34212/3
Alex Zhuravlev [Thu, 7 Feb 2019 09:33:12 +0000 (12:33 +0300)]
LU-9706 dt: remove dt_txn_hook_commit()

it's not used and it's not safe as dt_txn_callback_del()
and dt_txn_callback_add() can race with commit callbacks.

Change-Id: Ib80b0f69be008b4f895586dde35d1a5833a1a861
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34212
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
15 months agoLU-11927 kernel: new kernel [SLES12 SP4 4.12.14-95.6.1] 91/34191/4
Jian Yu [Mon, 11 Feb 2019 23:53:13 +0000 (15:53 -0800)]
LU-11927 kernel: new kernel [SLES12 SP4 4.12.14-95.6.1]

This patch makes changes to support new SLES12 SP4 release
for Lustre client.

Test-Parameters: trivial clientdistro=sles12sp4 \
envdefinitions=LNET_SELFTEST_EXCEPT=smoke,SANITY_EXCEPT=103a

Change-Id: Ibe59ebc30c25f2cab771ac4c2c9b7a9b974732d5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34191
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11914 build: add a configure check for l_getsepol 83/34183/5
Sebastien Buisson [Tue, 5 Feb 2019 14:09:45 +0000 (23:09 +0900)]
LU-11914 build: add a configure check for l_getsepol

l_getsepol requires openssl-devel, so add a configure check for
openssl/evp.h header and EVP_MD_CTX_create function, and disable
building l_getsepol in case they are missing.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I31ddbc2f5300e9e38db9e00e2b7fbcac7f83d9e5
Reviewed-on: https://review.whamcloud.com/34183
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11923 utils: fix mkfs.lustre meta_bg handling 78/34178/2
Andreas Dilger [Tue, 5 Feb 2019 10:06:43 +0000 (03:06 -0700)]
LU-11923 utils: fix mkfs.lustre meta_bg handling

If meta_bg is specified as a formatting option, mke2fs reports an
error because it conflicts with the default resize_inode feature,
which is enabled by default for filesystems under 2^32 blocks.

Enable meta_bg by default if the filesystem is over 2^36 blocks
(256TiB) or the group descriptor table grows too large.

Disable the resize_inode feature if meta_bg is explicitly specified
or if the filesystem is over 2^32 blocks.

Fix is_e2fsprogs_feature_supp() to return a boolean in the proper
sense for the function, rather than a 0 or -ve error number, since
that doesn't make sense for the name of the function.

Drop a level of indent in ldiskfs_make_lustre() by returning an
error directly if the ldd_mount_type is unknown.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8886e6850aad5868155b2208043dbbc4873ebbe5
Reviewed-on: https://review.whamcloud.com/34178
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-9859 libcfs: use strim instead of cfs_trimwhite. 68/34168/4
NeilBrown [Mon, 11 Feb 2019 15:56:11 +0000 (10:56 -0500)]
LU-9859 libcfs: use strim instead of cfs_trimwhite.

Linux lib provides identical functionality to cfs_trimwhite,
so discard that code and use the standard.

Linux-commit: 213b14b1fa55790f55b180ed5121b07f037c7ddd

Change-Id: Ide9d829aef554541a3dfb65ecb305e89c7ddf74a
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/34168
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Jenkins
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11913 utils: allow "mq-deadline" as scheduler 63/34163/2
Andreas Dilger [Fri, 1 Feb 2019 20:10:40 +0000 (13:10 -0700)]
LU-11913 utils: allow "mq-deadline" as scheduler

Allow the "mq-deadline" scheduler for multi-queue block devices, in
addition to just "noop" and "deadline".  Explicitly add "deadline"
as a valid option, in case the default scheduler is changed.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2cb0878188aea43f88c503ea70a699be083ebbe5
Reviewed-on: https://review.whamcloud.com/34163
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11905 mdc: Add RETURN to mdc_intent_open_pack 40/34140/3
Patrick Farrell [Wed, 30 Jan 2019 21:28:08 +0000 (16:28 -0500)]
LU-11905 mdc: Add RETURN to mdc_intent_open_pack

mdc_intent_open_pack has ENTRY, but not RETURN.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I443dad13db19b6ee8fa4102c97ec93e16f9dd008
Reviewed-on: https://review.whamcloud.com/34140
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 months agoLU-11777 tests: add version check sanityn 102 53/33953/6
James Nunez [Tue, 15 Jan 2019 16:08:17 +0000 (09:08 -0700)]
LU-11777 tests: add version check sanityn 102

sanityn test 102 was added to Lustre tag 2.11.57. Thus,
we need to check that the server is 2.11.57 or later
before running test 102.

Test-Parameters: trivial serverjob=lustre-b2_10 serverbuildno=152 testlist=sanityn
Test-Parameters: testlist=sanityn
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ieefd6f0f3dc0051646f07c309fb59dc6124c2975
Reviewed-on: https://review.whamcloud.com/33953
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>