Whamcloud - gitweb
fs/lustre-release.git
4 years agoLU-5568 lnet: fix kernel crash when network failed to start 12/12512/5
Amir Shehata [Fri, 31 Oct 2014 00:50:15 +0000 (17:50 -0700)]
LU-5568 lnet: fix kernel crash when network failed to start

When loading Lustre modules without proper network configuration,
it always hit the following kernel panic:
LNetError: 105-4: Error -100 starting up LNI tcp
LNetError: 2145:0:(api-ni.c:823:lnet_unprepare())
 ASSERTION( list_empty(&the_lnet.ln_nis) ) failed:
LNetError: 2145:0:(api-ni.c:823:lnet_unprepare()) LBUG
Pid: 2145, comm: modprobe
x0aCall Trace:
[<ffffffffa044f853>] libcfs_debug_dumpstack+0x53/0x80 [libcfs]
[<ffffffffa044fdf5>] lbug_with_loc+0x45/0xc0 [libcfs]
[<ffffffffa04f3267>] lnet_unprepare+0x297/0x340 [lnet]
[<ffffffffa04f3b5c>] LNetNIInit+0x25c/0x3e0 [lnet]
[<ffffffff81061bc6>] ? put_online_cpus+0x56/0x80
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa081310c>] ptlrpc_ni_init+0x2c/0x1a0 [ptlrpc]
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa0813291>] ptlrpc_init_portals+0x11/0xf0 [ptlrpc]
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa09831c4>] init_module+0x1c4/0x1000 [ptlrpc]
[<ffffffff810020e2>] do_one_initcall+0xe2/0x190
[<ffffffff810ca7fb>] load_module+0x129b/0x1a90
[<ffffffff812da590>] ? ddebug_dyndbg_module_param_cb+0x0/0x60
[<ffffffff810c7133>] ? copy_module_from_fd.isra.43+0x53/0x150
[<ffffffff810cb1a6>] SyS_finit_module+0xa6/0xd0
[<ffffffff815f2119>] system_call_fastpath+0x16/0x1b
...
This is because in lnet_startup_lndnis(), we may add list items to
@the_lnet.ln_nis and @the_lnet.ln_nis_cpt before it failed. But in
lnet_startup_lndis() failure path,it did not cleanup list thus
causing assertion in lnet_unprepare().

Fix the assertion by cleaning up using lnet_shutdown_lndnis()
if the startup fails.

In a future enahancement the ni startup API will be modified to
cleanup after itself in case of failure.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ia344fd7c0f24c87b654554dda9e57bf5525edc85
Reviewed-on: http://review.whamcloud.com/12512
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5731 osp: flush async updates for osp_sync 59/12359/2
Fan Yong [Thu, 21 Aug 2014 04:19:25 +0000 (12:19 +0800)]
LU-5731 osp: flush async updates for osp_sync

Current osp_sync() only considers the async requests that are
handled by the osp_sync_thread, but ignores the async updates
that are handled directly by the background ptlrpcd threads.
Usually, such async updates are for LFSCK remote repairing.
This patch will flush all of them when dt_sync() is called.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0e6d54120acbd8ab82cf776222277ae3b805812d
Reviewed-on: http://review.whamcloud.com/12359
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-4839 tests: Give copytool more time to start 82/12682/6
Nathaniel Clark [Wed, 12 Nov 2014 01:56:28 +0000 (20:56 -0500)]
LU-4839 tests: Give copytool more time to start

Copytool can take some time to start, and if the HSM archive directory
is on a busy NFS server, it can take a bit of time for the initial
opens to occur.  This allows those actions more time to complete which
should give this test a better chance of passing correctly.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes,ONLY=60 \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
mdtcount=4 testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I28bc57b92c34b4eee07ba34a2d976f2c39dc70dc
Reviewed-on: http://review.whamcloud.com/12682
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Michael MacDonald <michael.macdonald@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5707 lfsck: store namespace LFSCK statistics info in new EA 21/12321/5
Fan Yong [Tue, 9 Sep 2014 03:23:04 +0000 (11:23 +0800)]
LU-5707 lfsck: store namespace LFSCK statistics info in new EA

For Lustre-2.6 or older release, the namespace LFSCK statistics info
was stored as XATTR_NAME_LFSCK_NAMESPACE EA, but in Lustre-2.7, the
namespace LFSCK will introduce more statistics information that will
cause the XATTR_NAME_LFSCK_NAMESPACE EA to be extended. If it still
uses the old XATTR_NAME_LFSCK_NAMESPACE EA, then when downgrade, the
old LFSCK will get -ERANGE when load the new trace file from disk,
and then the LFSCK cannot be started after downgrade.

To avoid such trouble, Lustre-2.7 will use new EA to store the
namespace LFSCK statistics info: XATTR_NAME_LFSCK_NAMESPACE_V2,
and keep a dummy XATTR_NAME_LFSCK_NAMESPACE EA in the trace file
to be compatible with old LFSCK.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I55b5adb962434013b00e3938a67b671010ecc206
Reviewed-on: http://review.whamcloud.com/12321
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
4 years agoLU-5740 build: add RHEL6.6 [2.6.32-504.el6] to build selections 09/12609/4
Bob Glossman [Tue, 28 Oct 2014 17:25:04 +0000 (10:25 -0700)]
LU-5740 build: add RHEL6.6 [2.6.32-504.el6] to build selections

Add support for building with RHEL6.6 kernel version 2.6.32-504.el6
while retaining the ability to build with older RHEL 6.5 kernels.
New ldiskfs patch series for el6.6 is included.

Test-Parameters: clientdistro=el6.6 mdsdistro=el6.6\
  ossdistro=el6.6 mdsfilesystemtype=ldiskfs\
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I780feefbbc179607762c0d2997fd608830f3db8b
Reviewed-on: http://review.whamcloud.com/12609
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5941 build: build dkms build at installed source tree 02/12802/2
Minh Diep [Thu, 20 Nov 2014 16:10:53 +0000 (08:10 -0800)]
LU-5941 build: build dkms build at installed source tree

Port from:
https://github.com/
zfsonlinux/zfs/commit/46bf86a9635266dd399443f5bf5c5f8d0f280aa2

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: If0c8543d955594b4f9dc305c35271a9cc94e1bbd
Reviewed-on: http://review.whamcloud.com/12802
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5941 dkms: make lustre-dkms require 2.2.0.3-28.git.7c3e7c5 01/12801/2
Minh Diep [Thu, 20 Nov 2014 16:05:25 +0000 (08:05 -0800)]
LU-5941 dkms: make lustre-dkms require 2.2.0.3-28.git.7c3e7c5

Due to a bug in dkms, we need to enfore the use of
dkms-2.2.0.3-28.git.7c3e7c5 version.

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I9ad8ccaa5106b221f41a50c520d8bdfef160c065
Reviewed-on: http://review.whamcloud.com/12801
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-2524 test: Code clean up for conf-sanity 30/10530/7
James Nunez [Fri, 30 May 2014 19:20:21 +0000 (13:20 -0600)]
LU-2524 test: Code clean up for conf-sanity

The patch modifying the tdir variable to a single directory
has landed; http://review.whamcloud.com/#/c/8123/. We can
now conduct miscellaneous cleanup including:

Remove the `-p` (parents) option from many calls to mkdir
Replace `lfs setstripe` with $SETSTRIPE
Replace `lfs getstripe` with $GETSTRIPE
Replace `lctl` with $LCTL
Added check for and call `error` and/or added error messages
for a variety of common functions.
Replace `…` with $(...)
Remove linefeed escape after |, ||, & and && operators.
Modify directory and file names to use $tdir and $tfile
Remove 'mkdir -p $MOUNT' when 'mount_client $MOUNT' is
called right before or after mkdir

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes testlist=conf-sanity

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I94bd51ce2d2f225736e12c4f9ac1a86a3d8a23d8
Reviewed-on: http://review.whamcloud.com/10530
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
4 years agoLU-5814 llite: remove ll_objects_destroy() 18/12618/2
John L. Hammond [Fri, 7 Nov 2014 15:00:09 +0000 (09:00 -0600)]
LU-5814 llite: remove ll_objects_destroy()

Remove ll_objects_destroy(). This function is not needed for
interoperability with servers of version 2.4 or higher (after lustre
commit 5165cdd4).

Remove the then unused function lov_destroy() and its supporting
functions. Remove the lsm_destroy method of struct lsm_operations.

Remove the unused struct lov_stripe_md, MD export, and capa parameters
from obd_destroy() and its implementations.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If8634b3d88a660d00891219c348622ec45361316
Reviewed-on: http://review.whamcloud.com/12618
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5418 echo: replace lov_stripe_md with lov_oinfo 47/12447/3
John L. Hammond [Wed, 29 Oct 2014 17:15:06 +0000 (12:15 -0500)]
LU-5418 echo: replace lov_stripe_md with lov_oinfo

In echo_client replace uses of struct lov_stripe_md with struct
lov_oinfo (since the instances of the former really only contained a
single instance of the latter). Remove the then unneccessary functions
echo_alloc_memmd(), echo_free_memmd(), osc_unpackmd(), and
obd_alloc_memmd(). Remove the struct lov_stripe_md * parameter from
obd_create(). Flatten osc_create() and osc_real_create() into a single
function.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5fe276bcc56e1fa8138a4d3f20b9d5297cf74f3f
Reviewed-on: http://review.whamcloud.com/12447
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
4 years agoLU-3962 iokit: fix whitespace in scripts 56/10456/6
Andreas Dilger [Tue, 27 May 2014 17:53:01 +0000 (11:53 -0600)]
LU-3962 iokit: fix whitespace in scripts

Fix the whitespace in mds-survey and obdfilter-survey to use tabs
instead of 4-space indentation.  Fix coding style in several places.

Remove the use of a python script just to get the page size.  Instead,
use "getconf PAGE_SIZE" to do this.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
testlist=mds-survey,obdfilter-survey

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I921007043c360b45d45fc03a8237edea9a3ebbe5
Reviewed-on: http://review.whamcloud.com/10456
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5537 ptlrpc: Fix an rq_no_reply assertion failure 40/11740/3
Li Wei [Wed, 3 Sep 2014 09:02:22 +0000 (17:02 +0800)]
LU-5537 ptlrpc: Fix an rq_no_reply assertion failure

An OSS had an assertion failure:

  LustreError: 5366:0:(ldlm_lib.c:2689:target_bulk_io()) @@@ timeout
  on bulk GET after 0+0s  req@ffff88083a61b400
  x1476486691018500/t0(4300509964)
  o4->8dda3382-83f8-6445-5eea-828fd59e4a06@192.168.1.116@o2ib1:0/0
  lens 504/448 e 391470 to 0 dl 1408494729 ref 2 fl Complete:/4/0 rc
  0/0
  LustreError: 5432:0:(niobuf.c:550:ptlrpc_send_reply()) ASSERTION(
  req->rq_no_reply == 0 ) failed:
  Lustre: soaked-OST0000: Bulk IO write error with
  8dda3382-83f8-6445-5eea-828fd59e4a06 (at 192.168.1.116@o2ib1),
  client will retry: rc -110
  LustreError: 5432:0:(niobuf.c:550:ptlrpc_send_reply()) LBUG
  Pid: 5432, comm: ll_ost_io03_003

  Call Trace:
  [<ffffffffa0641895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
  [<ffffffffa0641e97>] lbug_with_loc+0x47/0xb0 [libcfs]
  [<ffffffffa09cda4c>] ptlrpc_send_reply+0x4ec/0x7f0 [ptlrpc]
  [<ffffffffa09d4aae>] ? lustre_pack_reply_flags+0xae/0x1f0 [ptlrpc]
  [<ffffffffa09e4d75>] ptlrpc_at_check_timed+0xcd5/0x1370 [ptlrpc]
  [<ffffffffa09dc1e9>] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc]
  [<ffffffffa09e66f8>] ptlrpc_main+0x12e8/0x1990 [ptlrpc]
  [<ffffffff81069290>] ? pick_next_task_fair+0xd0/0x130
  [<ffffffff81529246>] ? schedule+0x176/0x3b0
  [<ffffffffa09e5410>] ? ptlrpc_main+0x0/0x1990 [ptlrpc]
  [<ffffffff8109abf6>] kthread+0x96/0xa0
  [<ffffffff8100c20a>] child_rip+0xa/0x20
  [<ffffffff8109ab60>] ? kthread+0x0/0xa0
  [<ffffffff8100c200>] ? child_rip+0x0/0x20

The thread in tgt_brw_write() had decided not to reply by setting
rq_no_reply, right before another thread tried to send an early reply
for the request.

Change-Id: I9096a098621a38610c0d0d2dff016c012fc4b7f2
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/11740
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
4 years agoLU-20 kernel: increase BH_LRU_SIZE to 16 77/12577/2
Sebastien Buisson [Wed, 5 Nov 2014 15:34:14 +0000 (16:34 +0100)]
LU-20 kernel: increase BH_LRU_SIZE to 16

As kernel community did not want a complicated way of
modifying BH_LRU_SIZE, it was proposed to directly set it
to 16. This has been accepted.
This patch is merged in the upstream kernel:
http://git.kernel.org/cgit/linux/kernel/git/torvalds/
linux.git/commit/?id=86cf78d73de8c6bfa89804b91ee0ace71a459961

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I71fb455de9ec70ed90f86d402ae76ecfba1e1e61
Reviewed-on: http://review.whamcloud.com/12577
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
4 years agoLU-5729 osd: iput in case of error in osd_scrub_setup 25/12325/4
Sergey Cheremencev [Fri, 26 Sep 2014 13:00:56 +0000 (17:00 +0400)]
LU-5729 osd: iput in case of error in osd_scrub_setup

In case of ENOSPACE from osd_scrub_file_store iput is needed.
Otherwise there is a message in dmesg: "VFS: Busy inodes after
unmount of vdb. Self-destruct in 5 seconds. Have a nice day..."
Also added osd_oi_fini for case of error from osd_initial_OI_scrub
or osd_scrub_start.

Change-Id: Ibc6f487c9bd5b07f09cb3f7e3b5fc2bf1e329fb0
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Xyratex-bug-id: MRP-2109
Reviewed-on: http://review.whamcloud.com/12325
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5855 lfsck: misc fixes for zfs-based backend 52/12552/5
Fan Yong [Wed, 3 Sep 2014 16:25:33 +0000 (00:25 +0800)]
LU-5855 lfsck: misc fixes for zfs-based backend

It contains several fixes to make the LFSCK to work under DNE mode
for zfs-based backend.

Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=2 testlist=sanity-lfsck
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8e8758336d4ce67667f7e3586475ddd72db2d419
Reviewed-on: http://review.whamcloud.com/12552
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5833 lfsck: handle lfsck_open_dir() return-value properly 33/12533/3
Fan Yong [Tue, 2 Sep 2014 11:06:03 +0000 (19:06 +0800)]
LU-5833 lfsck: handle lfsck_open_dir() return-value properly

Inside the lfsck_prep(), the returned value from lfsck_open_dir()
should be handled properly before returning to the caller directly.
For example: positive number from lfsck_open_dir() means the end of
current directory, but if continuously return such value to the
lfsck_prep()'s caller, then the whole LFSCK first-stage scanning
will be regarded as done by wrong.

Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=sanity-lfsck
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I9e5c32b8594a65f1b605196373034ace6c9d1881
Reviewed-on: http://review.whamcloud.com/12533
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
4 years agoLU-5817 clio: Do not allow group locks with gid 0 59/12459/4
Patrick Farrell [Mon, 10 Nov 2014 07:39:29 +0000 (01:39 -0600)]
LU-5817 clio: Do not allow group locks with gid 0

When a group lock with GID=0 is released (put_grouplock is
called), an assertion in cl_put_grouplock is hit.

We should not allow group lock requests with GID=0, instead
we should return -EINVAL.

Also fix random_group_id so it never returns gid==0.

Change-Id: I56e58791742809da5353a4d8dfbf3b80a22f3814
Signed-off-by: Patrick Farrell <paf@cray.com>
Reviewed-on: http://review.whamcloud.com/12459
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
4 years agoLU-5732 hsm: sanity check for progress input 85/12285/4
Frank Zago [Sun, 12 Oct 2014 18:57:05 +0000 (13:57 -0500)]
LU-5732 hsm: sanity check for progress input

During an HSM archive or restore, the progress is reported by the
copytool, in userspace. That value may be bogus. For instance, this
will crash the MDS in interval_set():

he.offset = -1;
he.length = 10;
rc = llapi_hsm_action_progress(hcp, &he, length, 0);

So check that userspace is giving a sane progress extent value.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I0eb3fa9a66400a4ff3cee2f256c08e1d84744111
Reviewed-on: http://review.whamcloud.com/12285
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-3270 statahead: small fixes and cleanup 67/9667/12
Lai Siyao [Fri, 14 Mar 2014 12:10:32 +0000 (20:10 +0800)]
LU-3270 statahead: small fixes and cleanup

small fixes:
* when 'unplug' is set for ll_statahead(), sa_put() shouldn't kill
  the entry found, because its inflight RPC may not finish yet.
* remove 'sai_generation', add 'lli_sa_generation' because the
  former one is not safe to access without lock.
* revalidate_statahead_dentry() may fail to wait for statahead
  entry to become ready, in this case it should not release this
  entry, because it may be used by inflight statahead RPC.

cleanups:
* rename ll_statahead_enter() to ll_statahead().
* move dentry 'lld_sa_generation' update to ll_statahead() to
  simplify code and logic.
* other small cleanups.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I65759c7dfcbe879b42f14152dbfe5949e3d37ea0
Reviewed-on: http://review.whamcloud.com/9667
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
4 years agoLU-5473 tests: print space usage in sanity test_51b 85/12185/7
Andreas Dilger [Fri, 24 Oct 2014 07:23:14 +0000 (00:23 -0700)]
LU-5473 tests: print space usage in sanity test_51b

In sanity.sh test_51b print out the space usage before and after
the test so that the failure can be debugged.

Skip test_51b and test_51ba for ZFS when running regular review
tests, since there isn't a limit of 60000 subdirectories (ZFS
nlink is a 64-bit number), and they take a long time to run in
a VM (20 minutes combined).

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=51 \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 \
mdtcount=1 mdssizegb=2 ostcount=2 ostsizegb=8 \
testlist=sanity

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=51 \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 \
mdtcount=1 mdssizegb=2 ostcount=2 ostsizegb=8 nettypes=o2ib \
testlist=sanity

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I21b072fbcb05dea3fd7803bf3353de11ffbcab07
Reviewed-on: http://review.whamcloud.com/12185
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5778 lod: Fix lod_qos_statfs_update() 17/12617/2
Aurelien Degremont [Fri, 7 Nov 2014 13:10:03 +0000 (14:10 +0100)]
LU-5778 lod: Fix lod_qos_statfs_update()

When an OST is sick, or unactivate, lod cannot fetch its statfs
information. In lod_qos_statfs_update() this was preventing lod
to get information for other OST because refresh was stopped at
first error.
This patch fixes this behaviour.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: Id0217f228381ef7a41fdbfd99f5499dcc97ace0e
Reviewed-on: http://review.whamcloud.com/12617
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5865 lfsck: avoid NULL pointer 97/12597/2
Fan Yong [Thu, 4 Sep 2014 01:37:08 +0000 (09:37 +0800)]
LU-5865 lfsck: avoid NULL pointer

NOT pass "NULL" as the parameter of @lmv for lfsck_record_lmv(),
then the subsequent handling inside lfsck_record_lmv() needs NOT
to worry about the case of "lmv == NULL".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5f308818edd5ded2c4ccc7d59fb0908791b8aae3
Reviewed-on: http://review.whamcloud.com/12597
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-181 ptlrpc: reorganize ptlrpc_request 06/8806/16
Liang Zhen [Sun, 5 Jan 2014 15:00:26 +0000 (23:00 +0800)]
LU-181 ptlrpc: reorganize ptlrpc_request

ptlrpc_request has some structure members are only for client side,
and some others are only for server side, this patch moved these
members to different structure then putting into an union.

By doing this, size of ptlrpc_request is decreased about 300 bytes,
besides saving memory, it also can reduce memory footprint while
processing.

Another change in this patch is, osp will not use rq_exp_list anymore
because it is a server only member now.
osp will use ptlrpc_req_async_args to store commit_cb parameters in
this patch.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Id910ac225b8e9d33a0cae40b3124ce55f1a3fbc9
Reviewed-on: http://review.whamcloud.com/8806
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
5 years agoLU-5654 osp: Call obd_fid_fini() on osp_init0() error path 37/12037/2
Li Wei [Wed, 24 Sep 2014 09:09:48 +0000 (17:09 +0800)]
LU-5654 osp: Call obd_fid_fini() on osp_init0() error path

osp_init0() should call obd_fid_fini() on its error path to avoid
leaks.

Change-Id: I1a679db172ae60c74049d2dd3e111c93cfcbeda2
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/12037
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
5 years agoLU-5854 lnet: make YAML output/input consistent 56/12556/6
Amir Shehata [Tue, 4 Nov 2014 20:49:51 +0000 (12:49 -0800)]
LU-5854 lnet: make YAML output/input consistent

The YAML format used for configuring and showing networks was not
consistent. This patch makes both formats consistent.

EX:
net:
    - net: tcp
      nid: 192.168.206.130@tcp
      status: up
      interfaces:
          0: eth0
      tunables:
          peer_timeout: 180
          peer_credits: 8
          peer_buffer_credits: 0
          credits: 256
      CPT: [0,1]

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Id4314679930709ac43104f1ba544bb6d1ca8cb0a
Reviewed-on: http://review.whamcloud.com/12556
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5831 lfsck: extend lfsck_request::lr_pool_name 34/12534/3
Fan Yong [Mon, 1 Sep 2014 12:11:30 +0000 (20:11 +0800)]
LU-5831 lfsck: extend lfsck_request::lr_pool_name

Fix some issues found during Lustre source static analysis:

1) Extend lfsck_request::lr_pool_name size to match the
   lmv_mds_md_v1::lmv_pool_name.
2) Check lfsck->li_obj_dir inside lfsck_close_dir() before using.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I84443089135c5de1b9fa89eb76e5cd623412a01f
Reviewed-on: http://review.whamcloud.com/12534
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
5 years agoLU-4217 build: Remove ACL mount options 54/12154/3
James Nunez [Wed, 1 Oct 2014 14:06:08 +0000 (08:06 -0600)]
LU-4217 build: Remove ACL mount options

The "acl" mount option on the client has been
depricated since Lustre 1.8. The "acl" mount
option code is now obsolete and is removed.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ib765476a71ebb732d9ffda60b336530e0a758943
Reviewed-on: http://review.whamcloud.com/12154
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5006 mdd: don't call attr_set on object create 43/12243/4
Niu Yawei [Thu, 9 Oct 2014 08:11:31 +0000 (04:11 -0400)]
LU-5006 mdd: don't call attr_set on object create

The object attr has been initialzed in OSD layer when create
object, it's not necessary to initialize it again in MDD layer.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I6f4094d4384b2c153d4dad2666d64281c0450059
Reviewed-on: http://review.whamcloud.com/12243
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5848 lnet: fix inconsistent seq_no names 62/12562/2
Amir Shehata [Tue, 4 Nov 2014 21:29:06 +0000 (13:29 -0800)]
LU-5848 lnet: fix inconsistent seq_no names

When YAML output is printed the literal "seqno" is used,
when it's parsed, the literal "seq_no" is expected.
This patch makes it consistent.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Iabf5394e858007c7f6e87c7baf892887da88f8e3
Reviewed-on: http://review.whamcloud.com/12562
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
5 years agoLU-4730 utils: lctl get_param, set_param cleanup 45/9545/10
Andreas Dilger [Mon, 3 Nov 2014 15:42:18 +0000 (10:42 -0500)]
LU-4730 utils: lctl get_param, set_param cleanup

Cleanup "lctl get_param" and "lctl set_param" code and error handling.
Deny "parameters" with embedded relative paths to avoid strangeness.
Return an error consistently if multiple parameters are set but the
last one did not fail.  Remove deprecated full-path handling.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1004b5b4da4dc9b5825ef498758e248ed52f4141
Reviewed-on: http://review.whamcloud.com/9545
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Chao Wang <chao.ornl@gmail.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5329 mgs: Remove nibtbl swab code for 2.2 clients 10/12010/3
James Nunez [Mon, 22 Sep 2014 22:50:59 +0000 (16:50 -0600)]
LU-5329 mgs: Remove nibtbl swab code for 2.2 clients

Remove obsolete code that allows compatibility with
Lustre 2.2 clients.

Due to a bug, Lustre 2.2 clients always swab nidtbl
entries even if the server and client are using the
same endian. The fix was to allow the servers to do
the swabbing for the client.

Now, clients will do the swabbing.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I420ca986c0a68343be07272bb419cbdb1cebf148
Reviewed-on: http://review.whamcloud.com/12010
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
5 years agoLU-2675 libcfs: remove libcfs posix headers 87/11987/5
John L. Hammond [Mon, 3 Nov 2014 19:42:49 +0000 (14:42 -0500)]
LU-2675 libcfs: remove libcfs posix headers

Remove libcfs/include/libcfs/posix/. Include what was needed from
libcfs/posix/libcfs.h into libcfs/libcfs.h or in the appropriate .c
file.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia3016c83f13554b617c5f4a6dcc86adf222d4e49
Reviewed-on: http://review.whamcloud.com/11987
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-5551 lustre: fix messages with missisng newlines 96/11996/5
John L. Hammond [Fri, 19 Sep 2014 15:09:51 +0000 (10:09 -0500)]
LU-5551 lustre: fix messages with missisng newlines

Add missisng newlines to four CERROR() messages. Restore the trailing
newline in the definition of OSC_DUMP_GRANT(). Remove an unnecessary
CDEBUG() from ldlm_pool_recalc().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I549de59dd9cd205e1a6d0fbcd70ccd1cbf5e389b
Reviewed-on: http://review.whamcloud.com/11996
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-5871 lod: Do not return EAGAIN in lod_object_init 86/12586/4
Wang Di [Wed, 5 Nov 2014 18:46:59 +0000 (10:46 -0800)]
LU-5871 lod: Do not return EAGAIN in lod_object_init

Convert EAGAIN to EIO if fld_client_rpc() fails in
lod_object_init(), otherwise it will confuse
lu_object_find_at(), and make it wait there for no
reason, which should only wait if the object is dying.
See call chain lu_object_find_at()-> lu_object_find_try()
->lu_object_alloc()->lod_object_init()->lod_fld_lookup()
->fld_client_rpc(), and even worse waitq is not being
initialized yet when the failure happened here.

Change-Id: Ieae434b34c239efea86a4a471fb01e397336a31c
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/12586
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-3573 osd-zfs: Only advance zap cursor as needed 82/12582/3
Nathaniel Clark [Wed, 5 Nov 2014 18:05:22 +0000 (13:05 -0500)]
LU-3573 osd-zfs: Only advance zap cursor as needed

Only advance the zap cursor when ozi_pos is not advanced, otherwise
occasionally the a file could get "lost" because the zap_cursor would
advance over it before the retrieve happened.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Iad560e2ffb4cfe2c74a1cf9197be7c2537538822
Reviewed-on: http://review.whamcloud.com/12582
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5565 osd-ldiskfs: separate LASSERT() into two lines 98/12398/2
Andreas Dilger [Wed, 22 Oct 2014 18:35:31 +0000 (12:35 -0600)]
LU-5565 osd-ldiskfs: separate LASSERT() into two lines

Separate the compound assertions in osd-ldiskfs into two lines:

    LASSERT(dt_object_exists(dt) && !dt_object_remote(dt));
to
    LASSERT(dt_object_exists(dt));
    LASSERT(!dt_object_remote(dt));

so that it is possible to distinguish which of the two is being hit.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I3ff4fc28bffe955ab051ece665faa4c8a6500c1e
Reviewed-on: http://review.whamcloud.com/12398
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
5 years agoLU-5626 ldiskfs: update non-htree dotdot in rename 85/12585/2
Bob Glossman [Wed, 5 Nov 2014 17:40:42 +0000 (09:40 -0800)]
LU-5626 ldiskfs: update non-htree dotdot in rename

This mod duplicates changes previously committed only for el6
for sles11sp3.

In 2.4+, when renaming a directory, its old dotdot entry will
be removed firstly, then the new dotdot entry is inserted, and
ldiskfs tries to append FID-in-dirent to the new entry.
But the space for dotdot entry may not be enough to hold
the new dotdot with FID-in-dirent, such as an MDT device
restored from file-level backup, or a device upgraded from 1.8.

In that case, for non-HTree directories, the ".." entry
will be written in the next available space in the directory
block.  This is invalid, as the ".." entry must be the
second entry in the block.

The same bug was fixed for HTree directories in LU-2638.
As Fan Yong said then: we do not want to introduce
complex logic to handle directory data moving, instead, in
such case, ignore the FID-in-dirent for the new dotdot entry,
and just insert the new dotdot entry.

There is one known flaw: This patch, like the one for
LU-2638, skips the entire data section rather than just
the FID.  This could cause trouble if something else ever
uses this section with ".." entries.

Test-Parameters: mdsdistro=sles11sp3 ossdistro=sles11sp3 \
 mdsfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs \
 ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iaba11ac19ab7f802925af7a562ad7f739e6ed5c8
Reviewed-on: http://review.whamcloud.com/12585
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-5830 scripts: use lustre_rmmod in lnet start/stop script 13/12513/4
Bruno Faccini [Fri, 31 Oct 2014 00:59:51 +0000 (01:59 +0100)]
LU-5830 scripts: use lustre_rmmod in lnet start/stop script

In lnet's start/stop script stop phase, use lustre_rmmod instead
to try to unload a static list/sequence of modules.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ie8584b32e4d7cd21de0ed18954aa38124485964d
Reviewed-on: http://review.whamcloud.com/12513
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoNew tag 2.6.90 2.6.90 v2_6_90 v2_6_90_0
Oleg Drokin [Thu, 6 Nov 2014 18:50:28 +0000 (13:50 -0500)]
New tag 2.6.90

Change-Id: I9fcc98e0df6a44f5836c6a038fd59d8614200bd8

5 years agoLU-5863 utils: Handle the special case of ldd_svname for mgs 64/12564/3
James Simmons [Wed, 5 Nov 2014 00:43:44 +0000 (19:43 -0500)]
LU-5863 utils: Handle the special case of ldd_svname for mgs

Currently parse_ldd checks to see if ldd_svname is 8 or more
characters in length. This is true for all servers except
the mgs which is always labeled as "MGS". This can prevent
the mounting of the MGT. The solution is to see if we are
handling a MGT which doesn't require any extra type of
special handling that other OSD need.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I68582471e2b6ce47473a4fefb21e589c8c5b3730
Reviewed-on: http://review.whamcloud.com/12564
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5814 lov: remove unused {get,set}_info handlers 45/12445/4
John L. Hammond [Mon, 27 Oct 2014 22:26:04 +0000 (17:26 -0500)]
LU-5814 lov: remove unused {get,set}_info handlers

In LOV and OSC remove handlers for the obsolete get and set info keys:
KEY_CAPA_KEY, KEY_CONNECT_FLAG, KEY_EVICT_BY_NID, KEY_LAST_ID,
KEY_LOCK_TO_STRIPE, KEY_MDS_CONN, KEY_NEXT_ID.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iab1adaffc4ea860ea6ce2a2614b5ab6f6444e34b
Reviewed-on: http://review.whamcloud.com/12445
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoLU-5691 hsm: remove a request from the index if not present in the store 42/12142/2
Frank Zago [Tue, 30 Sep 2014 23:05:48 +0000 (18:05 -0500)]
LU-5691 hsm: remove a request from the index if not present in the store

When processing the list of requests that have aged out, if the
request cannot be found in the store, removing it from the index. If
that is not done, Lustre will try again to remove it, leading to an
endless cycle of cancellation.

This fixes the repetition of these messages:

  LustreError:
  2028:0:(mdt_coordinator.c:1465:mdt_hsm_update_request_state())
  tas01-MDT0000: Cannot find running request for cookie 0x54249515 on
  fid=[0x200000404:0x15caa:0x0]
  LustreError:
  2028:0:(mdt_coordinator.c:1465:mdt_hsm_update_request_state())
  Skipped 15979999 previous similar messages
  LustreError: 2028:0:(mdt_coordinator.c:339:mdt_coordinator_cb())
  tas01-MDT0000: Cannot cleanup timeouted request:
  [0x200000404:0x15caa:0x0] for cookie 0x54249515 action=CANCEL
  LustreError: 2028:0:(mdt_coordinator.c:339:mdt_coordinator_cb())
  Skipped 15979999 previous similar messages

Change-Id: Ie7a2a98be8cc97db9af7a64476c06fc7321544eb
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/12142
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5853 build: fix el7 build regression 46/12546/2
Bob Glossman [Mon, 3 Nov 2014 22:43:27 +0000 (14:43 -0800)]
LU-5853 build: fix el7 build regression

Correct the build failure caused by recent master landing for
LU-4647 nodemap: add mapping functionality by using PDE_DATA()
instead of PDE() in new code.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5b43e485cf5ba25e8473ed5783848aca77b96048
Reviewed-on: http://review.whamcloud.com/12546
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5383 utils: fix array index out of bounds 24/12524/2
Dmitry Eremin [Fri, 31 Oct 2014 15:33:59 +0000 (18:33 +0300)]
LU-5383 utils: fix array index out of bounds

Possible attempt to access element -8..-1 of array 'ldd_svname'.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib4ec6a6d74ff6e805725d0ff4487868b7cbffa2f
Reviewed-on: http://review.whamcloud.com/12524
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5577 changelog: fix comparison between signed and unsigned 74/12474/2
Dmitry Eremin [Wed, 29 Oct 2014 12:36:25 +0000 (15:36 +0300)]
LU-5577 changelog: fix comparison between signed and unsigned

Change type of changelog_*{namelen,size}() to size_t.
Fixed string specifier for unsigned types.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie24c87242328d14ee608ad38b530a6e581db93b9
Reviewed-on: http://review.whamcloud.com/12474
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
5 years agoLU-5814 echo: remove userspace LSM handling 46/12446/4
John L. Hammond [Mon, 27 Oct 2014 22:53:04 +0000 (17:53 -0500)]
LU-5814 echo: remove userspace LSM handling

In lustre/obdecho/echo_client.c, remove handling of lov_stripe_md
passed from userspace (since userspace never passes it). Remove the
LOV specific code (ed_next_islov) from the echo client (since it
doesn't work).

Remove echo_get_stripe_off_id() and all calls to it since the stripe
count of the passed in lsm is always 0 and the funciton does nothing
in this case. Remove the then unused lsm parameters of
echo_client_page_debug_setup() and echo_client_page_debug_check().

In the OBD_IOC_GETATTR and OBD_IOC_SETATTR cases of
echo_client_iocontrol() do not set the oi_md member of struct obd_info
since only LOV OBD methods access it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If5d31ca3bf798d2e4f6c4f63c2012160e50f8cd7
Reviewed-on: http://review.whamcloud.com/12446
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-5814 lov: remove LL_IOC_RECREATE_{FID,OBJ} 42/12442/4
John L. Hammond [Mon, 27 Oct 2014 21:13:21 +0000 (16:13 -0500)]
LU-5814 lov: remove LL_IOC_RECREATE_{FID,OBJ}

Remove the obsolete ioctls LL_IOC_RECREATE_FID and LL_IOC_RECREATE_OBJ
along with their handlers in llite. Remove the then unused OBD method
lov_create(). Remove OBD_FL_RECREATE_OBJS handling from osc_create().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib7183235d9eb761d2dfa2072dbeb8dd4d918e4ad
Reviewed-on: http://review.whamcloud.com/12442
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
5 years agoLU-2675 utils: remove loadgen 95/12395/2
John L. Hammond [Wed, 22 Oct 2014 18:33:01 +0000 (13:33 -0500)]
LU-2675 utils: remove loadgen

Remove lustre/utils/loadgen.c. It doesn't work and is not being
maintained.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I214cbbde5a3f18dd2050e852d33cca2bc2998b6a
Reviewed-on: http://review.whamcloud.com/12395
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
5 years agoLU-5577 obdclass: change uuid_unpack arg to size_t 89/12389/2
Dmitry Eremin [Mon, 13 Oct 2014 17:28:08 +0000 (21:28 +0400)]
LU-5577 obdclass: change uuid_unpack arg to size_t

Cleanup warnings about comparison between signed and unsigned.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib577a36879a1a57f20f13a5c4c697ba404e113fa
Reviewed-on: http://review.whamcloud.com/12389
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5577 obdclass: change lu_site->ls_purge_start to unsigned 84/12384/2
Dmitry Eremin [Fri, 10 Oct 2014 19:26:30 +0000 (23:26 +0400)]
LU-5577 obdclass: change lu_site->ls_purge_start to unsigned

Change the type accordant usage.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ic2d6906eff21ab1fe964899f0da9732e68c193f7
Reviewed-on: http://review.whamcloud.com/12384
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5577 mdd: lu_dirent_calc_size() return type to size_t 83/12383/2
Dmitry Eremin [Fri, 10 Oct 2014 19:15:24 +0000 (23:15 +0400)]
LU-5577 mdd: lu_dirent_calc_size() return type to size_t

Change the type accordant usage.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I2451940225dd30015928cf85a2e0cc0e6cc8dfeb
Reviewed-on: http://review.whamcloud.com/12383
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5577 ldlm: count of pools is unsigned long 04/12304/3
Dmitry Eremin [Wed, 15 Oct 2014 19:12:13 +0000 (23:12 +0400)]
LU-5577 ldlm: count of pools is unsigned long

Function ldlm_pools_count() return unsigned long but counter is int.
Use ldlm_pool_granted() everywhere.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I6ee7bb1ba7f9590a776465b00e63584aada5f7dc
Reviewed-on: http://review.whamcloud.com/12304
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5387 test: Skip sanity test_239 if MDS version older than 2.5.60 41/12241/2
Wei Liu [Wed, 8 Oct 2014 22:37:41 +0000 (15:37 -0700)]
LU-5387 test: Skip sanity test_239 if MDS version older than 2.5.60

Skip sanity test_239 if MDS version older than 2.5.60

Change-Id: Ic2c71235a0f05ecb4a8b111a6044efe51c5270c8
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/12241
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
5 years agoLU-5668 test: enable ior data consistency check 58/12058/9
Johann Lombardi [Thu, 25 Sep 2014 14:04:13 +0000 (16:04 +0200)]
LU-5668 test: enable ior data consistency check

Check file content (-W) is consistent during read phase of IOR
and also change task ordering to n+1 ordering for readback (-C).

Test-Parameters: testlist=parallel-scale

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Ic9b1c3cda48ebd08907a6251f62cf2c845d00476
Reviewed-on: http://review.whamcloud.com/12058
Tested-by: Jenkins
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5443 lustre: replace direct HZ access with kernel APIs 52/12052/8
Jian Yu [Fri, 31 Oct 2014 14:15:10 +0000 (10:15 -0400)]
LU-5443 lustre: replace direct HZ access with kernel APIs

On some customer's systems, kernel was compiled with HZ defined to
100, instead of 1000. This improves performance for HPC applications.
However, to use these systems with Lustre, customers have to re-build
Lustre for the kernel because Lustre directly uses the defined
constant HZ.

Since kernel 2.6.21, some non-HZ dependent timing APIs become non-
inline functions, which can be used in Lustre codes to replace the
direct HZ access.

These kernel APIs include:
 jiffies_to_msecs()
 jiffies_to_usecs()
 jiffies_to_timespec()
 msecs_to_jiffies()
 usecs_to_jiffies()
 timespec_to_jiffies()

And here are some samples of the replacement:
 HZ            -> msecs_to_jiffies(MSEC_PER_SEC)
 n * HZ        -> msecs_to_jiffies(n * MSEC_PER_SEC)
 HZ / n        -> msecs_to_jiffies(MSEC_PER_SEC / n)
 n / HZ        -> jiffies_to_msecs(n) / MSEC_PER_SEC
 n / HZ * 1000 -> jiffies_to_msecs(n)

This patch replaces the direct HZ access in lustre modules.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib0ed9b5faf6ed311ff5423873d1c125b02ec4ab5
Reviewed-on: http://review.whamcloud.com/12052
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5545 ptlrpc: false alarm in AT network latency measuring 18/12018/5
Liang Zhen [Tue, 23 Sep 2014 13:00:27 +0000 (21:00 +0800)]
LU-5545 ptlrpc: false alarm in AT network latency measuring

If early reply of client RPC is lost and client RPC is expired and
resent, server will drop the resent RPC because it's already in
processing, server may also send reply or early reply to client,
which can still match reply buffer of the original request.
In this case, client is measuring time from resent time, but server
is reporting service time of original RPC, which is longer than
the time measured by client.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I772fb054480a3212e28dc018a6f592f3da7a87b5
Reviewed-on: http://review.whamcloud.com/12018
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
5 years agoLU-5651: ptlrpc: fix import state during replay 15/12015/4
Andriy Skulysh [Tue, 21 Oct 2014 20:53:11 +0000 (16:53 -0400)]
LU-5651: ptlrpc: fix import state during replay

Client doesn't restore import state correctly
on reconnect during replay. It resends lock replay
when final ping was queued by server.
Server fails with "target_queue_recovery_request())
ASSERTION( req->rq_export->exp_lock_replay_needed ) failed"

Add imp_replay_state to store last replay state.
imp_state is restored from imp_replay_state
during reconnect.

Xyratex-bug-id: MRP-2022
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Change-Id: Iaa14fe968cc31f266b605785df4fa676083fbca4
Reviewed-on: http://review.whamcloud.com/12015
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5591 lod: fix Null pointer dereference in lod_ah_init() 70/11770/8
Dmitry Eremin [Fri, 5 Sep 2014 13:35:55 +0000 (17:35 +0400)]
LU-5591 lod: fix Null pointer dereference in lod_ah_init()

Null pointer may be dereferenced in lod_ah_init()

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I76567d222ac7eb74440c74692aaa79f7078bca61
Reviewed-on: http://review.whamcloud.com/11770
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5589 obdclass: fix NULL pointer dereference 69/11769/5
Dmitry Eremin [Fri, 5 Sep 2014 13:13:34 +0000 (17:13 +0400)]
LU-5589 obdclass: fix NULL pointer dereference

NULL pointer dereferenced without check in lsi_prepare()

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I144c9bcd3739c68563c7460799efa4897489a1a7
Reviewed-on: http://review.whamcloud.com/11769
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-2675 obd: cleanup struct md_op_data and uses 34/11734/4
John L. Hammond [Tue, 2 Sep 2014 19:30:07 +0000 (14:30 -0500)]
LU-2675 obd: cleanup struct md_op_data and uses

Make the following changes in or around struct md_op_data:

* Move the definition of enum op_cli_flags from lclient.h to obd.h and
  rename it to enum md_cli_flags.

* Change to type of the op_flags member from __u32 to enum
  md_op_flags.

* Remove the used but never set member op_npages.

* Remove the set but never used member op_stripe_offset (an alias for
  op_ioepoch).

* Remove the op_max_pages alias for op_valid and add a op_max_pages
  member.

* Add a new member op_attr_flags.

* Remove the definition and all uses of struct ll_iattr. This
  structure was only used in expressions of the form
  ((struct ll_iattr *)&op_data->op_attr)->ia_attr_flags which can all
  be rewritten as op_data->op_attr_flags.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I17aabfcecdfd1a02dbee04362b033ef404a2cb27
Reviewed-on: http://review.whamcloud.com/11734
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4167 tests: correct version check to enable ff_convert 56/11556/6
Emoly Liu [Sat, 30 Aug 2014 21:12:48 +0000 (05:12 +0800)]
LU-4167 tests: correct version check to enable ff_convert

In conf-sanity.sh test_32d, filter fid convertion can be made only
when both of the following conditions are satisfied:
- ost device img version < 2.3.64 (struct filter_fid_old has been
introduced since 2.3.64)
- ost server version >= 2.5

Also, this patch fix the ofd_iocontrol() message to print the ioctl()
CMD argument as hex instead of a signed integer.

Test-Parameters: testlist=conf-sanity envdefinitions=SLOW=yes,ONLY=32 ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73  ossdistro=el6 mdsdistro=el6
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I4045fe6b7504a3ed30436da5a1319c09331fc261
Reviewed-on: http://review.whamcloud.com/11556
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
5 years agoLU-5577 mdc: fix comparison between signed and unsigned 79/11379/17
Dmitry Eremin [Wed, 29 Oct 2014 11:37:08 +0000 (14:37 +0300)]
LU-5577 mdc: fix comparison between signed and unsigned

Change type of client_obd->*_mds_*size from int to __u32 and
argumanets of related create/rename/setattr functions.
Change type of op_data->op_namelen to size_t.
Change type of argument size for all mdc_*_pack() to size_t.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I97f4fa6928c24fa416c334206c75f9885266b1ae
Reviewed-on: http://review.whamcloud.com/11379
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4176 tests: re-enable sanity-hsm/test_31a 77/9577/5
Bruno Faccini [Sun, 2 Nov 2014 20:06:46 +0000 (15:06 -0500)]
LU-4176 tests: re-enable sanity-hsm/test_31a

Due to frequent failures and despite first patch/fix for LU-4176 has
landed, sanity-hsm/test_31a has been disabled like other sub-tests
in a patch for LU-4178. Since both test_31[b,c], which also triggered
the same issue, are running fine now and with original fix for
LU-4176, it is time to re-enable it.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I5cc6c23b7cb48e190438eb3b84fa55ffdb198739
Reviewed-on: http://review.whamcloud.com/9577
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
5 years agoLU-5807 qos: enable QOS_DEBUG() 34/12434/3
Niu Yawei [Mon, 27 Oct 2014 12:43:07 +0000 (08:43 -0400)]
LU-5807 qos: enable QOS_DEBUG()

Enable QOS_DEBUG() by default.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: If2b0e1388a5f12edd5b03ffc1081709b9efb1c13
Reviewed-on: http://review.whamcloud.com/12434
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
5 years agoLU-4856 obdclass: check val in proc_max_dirty_pages_in_mb() 69/12269/4
Jian Yu [Fri, 10 Oct 2014 19:39:26 +0000 (12:39 -0700)]
LU-4856 obdclass: check val in proc_max_dirty_pages_in_mb()

In proc_max_dirty_pages_in_mb(), assigning "__u64 val" to
"unsigned long obd_max_dirty_pages" will cause values over
2^32 to be truncated on a 32-bit client (where "unsigned long"
is 32 bits, and not 64 bits).

This patch fixes the above issue by checking "val" in an
acceptable range before assigning it to obd_max_dirty_pages.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I838b5bab283a0068f72a86f5a990d909c892a9d9
Reviewed-on: http://review.whamcloud.com/12269
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
5 years agoLU-5380 at: net AT after connect 55/11155/2
Alexander.Boyko [Mon, 21 Jul 2014 10:18:23 +0000 (14:18 +0400)]
LU-5380 at: net AT after connect

Once connected, the previously gathered AT statistics is not valid
anymore because may reflect other routing, etc. The connect by itself
could take a long time due to different reasons (e.g. server was not
ready) and net latency got very high (see import_select_connection())
what does not reflect the current situation.

Take into account only the current (re-)CONNECT rpc latency.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Xyratex-bug-id: MRP-1285
Change-Id: I6edc0e232a92319e7c8535aced28fe1ad3436c54
Reviewed-on: http://review.whamcloud.com/11155
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
5 years agoLU-4810 utils: print messages when set tunables 65/9865/5
Niu Yawei [Mon, 28 Apr 2014 04:39:38 +0000 (00:39 -0400)]
LU-4810 utils: print messages when set tunables

When set scheduler and max_sectors_kb on mount, error messages should
be printed to notify user that these tunables are changed by lustre.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I6a3618b7e5eb0127e9aea18631397cd4dcbde546
Reviewed-on: http://review.whamcloud.com/9865
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Blake Caldwell <blakec@ornl.gov>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-3259 clio: cl_lock simplification 58/10858/15
Jinshan Xiong [Fri, 26 Sep 2014 21:46:30 +0000 (14:46 -0700)]
LU-3259 clio: cl_lock simplification

In this patch, the cl_lock cache is eliminated. cl_lock is turned
into a cacheless data container for the requirements of locks to
complete the IO. cl_lock is created before I/O starts and destroyed
when the I/O is complete.

cl_lock depends on LDLM lock to fulfill lock semantics. LDLM lock
is attached to cl_lock at OSC layer. LDLM lock is still cacheable.

Two major methods are supported for cl_lock: clo_enqueue and
clo_cancel.  A cl_lock is enqueued by cl_lock_request(), which will
call clo_enqueue() methods for each layer to enqueue the lock.
At the LOV layer, if a cl_lock consists of multiple sub cl_locks,
each sub locks will be enqueued correspondingly. At OSC layer, the
lock enqueue request will tend to reuse cached LDLM lock; otherwise
a new LDLM lock will have to be requested from OST side.

cl_lock_cancel() must be called to release a cl_lock after use.
clo_cancel() method will be called for each layer to release the
resource held by this lock. At OSC layer, the reference count of LDLM
lock, which is held at clo_enqueue time, is released.

LDLM lock can only be canceled if there is no cl_lock using it.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I6a61250549cfbc28070fe4bb7789ba7429eaf089
Reviewed-on: http://review.whamcloud.com/10858
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4181 tests: cleanup lustre before starting lnet-selftest.sh 69/12469/3
Amir Shehata [Tue, 28 Oct 2014 23:56:41 +0000 (16:56 -0700)]
LU-4181 tests: cleanup lustre before starting lnet-selftest.sh

Cleanup lustre before starting lnet-selftest.sh to attempt and
identify why RPC requests are timing out.

The approach is to land this patch and monitor the failure rate
for LU-4181 to see if it's been reduced.  The current failure rate
is approximately 1 failure per day.

Added a debug message to print out rpc stats when an RPC
error occurs.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I86cbc00af30b165918a57e6665ad725a9926c089
Reviewed-on: http://review.whamcloud.com/12469
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5079 tests: decrease at_max value in replay-vbr test_7_cycle 90/12490/2
Jian Yu [Wed, 29 Oct 2014 20:52:09 +0000 (13:52 -0700)]
LU-5079 tests: decrease at_max value in replay-vbr test_7_cycle

After commit 1d88909 (http://review.whamcloud.com/11213) was landed,
replay-vbr test 7* took long time to run due to more sleep time in
ptlrpc_set_wait(). This patch speeds up the tests by decreasing the
at_max value in test_7_cycle().

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 \
mdtcount=1 mdssizegb=2 ostcount=7 ostsizegb=2 \
testlist=replay-vbr,replay-vbr

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: If24443955290b091fd22905dfb74b0d6a6d1b4e8
Reviewed-on: http://review.whamcloud.com/12490
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5825 kernel: kernel update [RHEL7 3.10.0-123.9.2.el7] 78/12478/3
Bob Glossman [Wed, 29 Oct 2014 16:35:26 +0000 (09:35 -0700)]
LU-5825 kernel: kernel update [RHEL7 3.10.0-123.9.2.el7]

update RHEL7 kernel to 3.10.0-123.9.2.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5618ae37b14f65d146696dcf0fb1034a1d0bfef0
Reviewed-on: http://review.whamcloud.com/12478
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5795 kernel: kernel update [SLES11 SP3 3.0.101-0.40] 01/12401/2
Bob Glossman [Thu, 23 Oct 2014 16:35:01 +0000 (09:35 -0700)]
LU-5795 kernel: kernel update [SLES11 SP3 3.0.101-0.40]

Update target and config files for new version

Test-Parameters: mdsdistro=sles11sp3 ossdistro=sles11sp3\
 clientdistro=sles11sp3 mdsfilesystemtype=ldiskfs\
 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iac6e9dc76a6d2bdeb5615bffaeaa5e69652f31ba
Reviewed-on: http://review.whamcloud.com/12401
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5842 tests: reduce time to run sanity-sec tests 13 and 14 32/12532/2
Kit Westneat [Mon, 3 Nov 2014 03:32:39 +0000 (22:32 -0500)]
LU-5842 tests: reduce time to run sanity-sec tests 13 and 14

For nodemap testing, instead of running through all possible IPs in
a NID range, just test a subset in order to verify functionality.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I4b3ec0b47b81b2d3b5ac6f4b716a13ab34c9a4c3
Reviewed-on: http://review.whamcloud.com/12532
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-3270 statahead: race in start/stop statahead 66/9666/8
Lai Siyao [Mon, 20 Oct 2014 14:47:41 +0000 (10:47 -0400)]
LU-3270 statahead: race in start/stop statahead

When starting statahead thread, it should check whether current
lli_opendir_key was deauthorized in the mean time by another
process.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I5bc00ddd1a501b94cdfe7509ddb0e7d922bef624
Reviewed-on: http://review.whamcloud.com/9666
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-2272 statahead: ll_intent_drop_lock() called in spinlock 65/9665/9
Lai Siyao [Sun, 19 Oct 2014 16:19:21 +0000 (12:19 -0400)]
LU-2272 statahead: ll_intent_drop_lock() called in spinlock

ll_intent_drop_lock() may sleep, which should not be called inside
spinlock.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ife50396dc16beaad921751b4bdd7da721dfc15c9
Reviewed-on: http://review.whamcloud.com/9665
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-3270 statahead: use dcache-like interface for sa entry 64/9664/11
Lai Siyao [Thu, 16 Oct 2014 18:16:18 +0000 (14:16 -0400)]
LU-3270 statahead: use dcache-like interface for sa entry

Rename ll_sa_entry to sa_entry, and manage sa_entry cache with
dcache-like interfaces.

sa_entry is not needed to be refcounted, because only scanner
can free it, so after it's put in stat list, statahead thread
shouldn't access it any longer.

ll_statahead_interpret() doesn't need to take sai refcount,
because statahead thread will wait for all inflight RPC to
finish.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ibd55227f4c0710139bd039a4e7232dc1cb5041be
Reviewed-on: http://review.whamcloud.com/9664
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4647 nodemap: add mapping functionality 99/9299/44
Joshua Walgenbach [Mon, 3 Mar 2014 08:10:20 +0000 (09:10 +0100)]
LU-4647 nodemap: add mapping functionality

Add functions to allow the MDT to use the nodemap module
functions to map UIDs and GIDs. When a client connects to the MDT,
the NID of the client is classified by nodemap and a pointer
to the nodemap is added to the export structure.
In old_init_ucred and old_init_ucred_reint, the body ids and fs
ids are mapped according to the nodemap state (enabled, maps,
squashed ids, trust_client_ids and allow_root_access). ACLs are mapped
in mdt_getattr_internal. Squashed ACLs are discarded before the ACLs
are sent to clients. Trusted administrative nodes are allowed to
use setfacl on all users. Mapped nodes can only use setfacl for users
also mapped into the node.

Change-Id: I9648f6837f8bb4ff145c8a0d647fc934c08aaffe
Signed-off-by: Joshua Walgenbach <jjw@iu.edu>
Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Reviewed-on: http://review.whamcloud.com/9299
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoLU-5435 lnet: lustre network latency simulation 09/11409/14
Liang Zhen [Wed, 6 Aug 2014 02:49:19 +0000 (10:49 +0800)]
LU-5435 lnet: lustre network latency simulation

Incoming lnet message can be delayed for seconds if it can match
any of LNet Delay Rules.

User can add/remove/list Delay Rule by lctl commands:
- lctl net_delay_add
  Add a new Delay Rule to LNet, options
  <-s | --source SRC_NID>
  <-d | --dest DST_NID>
  <<-r | --rate RATE_NUMBER>
   <-i | --interlval SECONDS>>
  <-l | --latency DELAY_LATENCY>

- lctl net_delay_del
  Remove matched Delay Rule from LNet, options:
  <[-a | --all] |
   <-s | --source SRC_NID>
   <-d | --dest DST_NID>>

- lctl net_delay_list
  List all Delay Rules in LNet

- lctl net_delay_reset
  Reset statistic counters for all Delay Rules

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Iba1234908918ad1619ae85aa2f0ad77a992374aa
Reviewed-on: http://review.whamcloud.com/11409
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4665 utils: lfs setstripe to specify OSTs 83/9383/29
Jinshan Xiong [Sat, 18 Oct 2014 03:29:33 +0000 (20:29 -0700)]
LU-4665 utils: lfs setstripe to specify OSTs

Extend lfs setstripe command to support specifying individual
OSTs. [--ost|-o] option is invented for this purpose. Support
specifying OSTs for regular files only. Directory support will
be implemented later in a separate project.

OSTs can be spcified by lfs setstripe [--ost|-o] as follows:
  lfs setstripe [[--ost-list|-o] <ost_1>,<ost_i>-<ost_j>,<ost_n>]

For example, -o 1,2,4-6,8 will be expanded as OSTs:
  1,2,4,5,6,8

In addition, duplicate indices will be eliminated automatically.

Calculate the max easize by ld_active_tgt_count instead of
ld_tgt_count. However this may introduce problems when the OSTs
are in recovery because non sufficient buffer may be allocated
to store EA.

This patch adds the following test cases into conf-sanity.sh:
test 81: sparse OST indexing
test 82a: specify OSTs for file (succeed) or directory (fail)
test 82b: specify OSTs for file with --pool and --ost-list options

Signed-off-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I055ea492e909d1073e08d46e00cfb5389e958ad2
Reviewed-on: http://review.whamcloud.com/9383
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5435 lnet: LNet drop rule implementation 14/11314/10
Liang Zhen [Mon, 4 Aug 2014 09:16:56 +0000 (17:16 +0800)]
LU-5435 lnet: LNet drop rule implementation

This is implementation of LNet Drop Rule, which can randomly drop
LNet messages at specified rate.

LNet Drop Rule can only be applied to receive side of message. User
can add drop_rule either on end point of cluster (client/server) or
on LNet routers.

Here are lctl command to control LNet Drop Rules:
- net_drop_add -s SRC_NID -d DEST_NID --rate VALUE
  drop 1/@VALUE of messages from @SRC_NID to @DEST_NID

- net_drop_del -s SRC_NID -d DEST_NID
  remove all drop rules from @SRC_NID to @DEST_NID

- net_drop_list
  list all drop rules on current node

Examples:
- lctl net_drop_add -s *@o2ib0 -d 192.168.1.102@tcp 1000
  add new drop rule, it will drop 1/1000 messages from network o2ib0
  to 192.168.1.102@tcp

- lctl net_drop_add -s 10.8.6.123@o2ib1 -d * 500
  add new drop rule, it will drop 1/500 messages from 10.8.6.123@o2ib1
  to all nodes

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Ib4cc384a8a191e88bd5f293cf1a5d7f524970d01
Reviewed-on: http://review.whamcloud.com/11314
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4198 clio: generalize cl_sync_io 56/8656/18
Jinshan Xiong [Fri, 26 Sep 2014 21:45:17 +0000 (14:45 -0700)]
LU-4198 clio: generalize cl_sync_io

To make cl_sync_io interfaces not just wait for pages, but to be
a generic synchronization mechanism.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iea43cb56c463efa7a6c6d17d84dacf4bdad8b377
Reviewed-on: http://review.whamcloud.com/8656
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5435 libcfs: copy out ioctl inline buffer 13/11313/14
Liang Zhen [Mon, 4 Aug 2014 08:53:15 +0000 (16:53 +0800)]
LU-5435 libcfs: copy out ioctl inline buffer

- libcfs_ioctl_popdata should copy out inline buffers.
- code cleanup for libcfs ioctl handler
- error number fix for obd_ioctl_getdata
- add new function libcfs_ioctl_unpack for upcoming patches

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I3969e4e96b1dc0c2f6c3e86039b97af5482edc74
Reviewed-on: http://review.whamcloud.com/11313
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5519 lfsck: repair slave LMV for striped directory 48/11848/15
Fan Yong [Fri, 29 Aug 2014 09:04:23 +0000 (17:04 +0800)]
LU-5519 lfsck: repair slave LMV for striped directory

Before all shards of the striped directory scanned, the LFSCK
cannot know whether the master LMV EA is valid or not, and also
cannot know how to repair an invalid shard exactly. For example,
the stripe index stored in the shard's name does not match the
stripe index stored in the slave LMV EA, then the LFSCK cannot
know which one is correct. If the LFSCK just assumed one is
correct, and fixed the other, then as the LFSCK processing, it
may find that the former reparation is wrong and have to roll
back. Unfortunately, if some applications saw the changes and
made further modification based on such changes, then the roll
back is almost impossible.

To avoid above trouble, the LFSCK will double scan the master
object of the striped directory, that is NOT the same as normal
two-stages scanning, the double scanning the striped directory
will happen both during the first-stage scanning:

1) When the striped directory is opened for scanning, the LFSCK
   will iterate each shard in turn, and records its slave LMV EA
   in the lfsck_lmv::ll_lslr. In this step, if the 'shard' (may
   be fake shard) name does not match the shard naming rule, for
   example, it does not contain the shard's FID, or not contain
   index, then we can remove the bad name entry directly. But if
   the name is valid, but the shard has no slave LMV EA or the
   slave LMV EA does not match its name, then we just record
   related information in the lfsck_lmv::ll_lslr in RAM.

2) When all the known shards have been scanned, then the engine will
   generate a dummy request (via lfsck_namespace_close_dir) to tell
   the assistant thread that all the known shards have been scanned.
   Since the assistant has got the global knowledge about the index
   conflict, stripe count, hash type, and so on. Then the assistant
   thread will scan the lfsck_lmv::ll_lslr, and for every shard in
   the record, check and repair inconsistency.
2.1) If the shard lost its slave LMV EA, then regenerate the slave
     LMV EA.
2.2) If the index stored in the shard's name entry is wrong, then
     rename the shard.
2.3) If the shard's slave LMV EA contains bad index/stripe_count/
     hash type, then update the slave LMV EA.
2.4) If the shard object is missing, then re-create the lost shard.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I92c0bd569e5a2711b8c7e93904d050b3fdf35377
Reviewed-on: http://review.whamcloud.com/11848
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-3613 llite: Add ioctl to get parent fids from link EA. 69/7069/17
Henri Doreau [Wed, 15 Oct 2014 07:56:04 +0000 (09:56 +0200)]
LU-3613 llite: Add ioctl to get parent fids from link EA.

Added LL_IOC_GETPARENT to retrieve the <parent_fid>/name(s) of a given
entry, based on its link EA. This saves multiple calls to
path2fid/fid2path.

Added llapi_path2parent() and llapi_fd2parent() wrappers.

Added '--parents' option to lfs path2fid to call this function.

New sanity test 154f to validate the behavior.

Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I10f9316da605050be8fc709fe88d87abe86e22a4
Reviewed-on: http://review.whamcloud.com/7069
Tested-by: Jenkins
Reviewed-by: Ned Bass <bass6@llnl.gov>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5519 lfsck: repair master LMV for striped directory 47/11847/12
Fan Yong [Wed, 27 Aug 2014 21:35:00 +0000 (05:35 +0800)]
LU-5519 lfsck: repair master LMV for striped directory

If the master MDT-object of a striped directory lost its LMV EA,
then there may be some users have created some files under the
master MDT-object directly. Under such case, the LFSCK cannot
re-generate LMV EA for the master MDT-object, because we should
keep the existing files to be visible to client. Then the LFSCK
will mark the striped directory as read-only and keep it there
to be handled by administrator manually.

If nobody has created files under the master MDT-object of the
striped directory, then we will set the master LMV EA and
generate a new rescan (the striped directory) request that will
be handled later by the LFSCK instance on the MDT later.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4d604cbd346c2cb044503f193ab4745e7fd2c2a2
Reviewed-on: http://review.whamcloud.com/11847
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5519 lfsck: repair bad name hash for striped directory 46/11846/13
Fan Yong [Wed, 27 Aug 2014 17:30:27 +0000 (01:30 +0800)]
LU-5519 lfsck: repair bad name hash for striped directory

If the name hash of some name entry under the striped directory
does not match the shard of the striped directory, then the LFSCK
will repair the inconsistency. Ideally, the LFSCK should migrate
the name entry from the current MDT to the right MDT (another one),
but before the async commit finished, the LFSCK will change the
striped directory's hash type as LMV_HASH_TYPE_UNKNOWN and mark
the lmv flags as LMV_HASH_FLAG_BAD_TYPE.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I665a77473be1fead09c12cb7fb5d0ffa739ffe87
Reviewed-on: http://review.whamcloud.com/11846
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5584 llite: ensure all data flush out when umount 03/12103/10
Yang Sheng [Sun, 28 Sep 2014 22:14:44 +0000 (06:14 +0800)]
LU-5584 llite: ensure all data flush out when umount

Write out all extents when clear inode. Otherwise we
may lose data while umount.

Change-Id: I199217c96a7ba49c2f3577a65750e7956a9af63a
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/12103
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoRevert "LU-5568 lnet: fix kernel crash when network failed to start" 02/12502/2
Oleg Drokin [Thu, 30 Oct 2014 16:27:14 +0000 (16:27 +0000)]
Revert "LU-5568 lnet: fix kernel crash when network failed to start"

Isaac has found many issues with the patch and advise to revert it.

This reverts commit 8fab48a8ef0bad6961c2ca1e2959726252ba43ac.

Change-Id: I9d8422e018d9dc809eed53dcfe376cb5154181af
Reviewed-on: http://review.whamcloud.com/12502
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5568 lnet: fix kernel crash when network failed to start 18/11718/11
Wang Shilong [Mon, 1 Sep 2014 20:44:38 +0000 (16:44 -0400)]
LU-5568 lnet: fix kernel crash when network failed to start

When loading Lustre modules without proper network configuration,
it always hit the following kernel panic:

LNetError: 105-4: Error -100 starting up LNI tcp
LNetError: 2145:0:(api-ni.c:823:lnet_unprepare())
 ASSERTION( list_empty(&the_lnet.ln_nis) ) failed:
LNetError: 2145:0:(api-ni.c:823:lnet_unprepare()) LBUG
Pid: 2145, comm: modprobe
x0aCall Trace:
[<ffffffffa044f853>] libcfs_debug_dumpstack+0x53/0x80 [libcfs]
[<ffffffffa044fdf5>] lbug_with_loc+0x45/0xc0 [libcfs]
[<ffffffffa04f3267>] lnet_unprepare+0x297/0x340 [lnet]
[<ffffffffa04f3b5c>] LNetNIInit+0x25c/0x3e0 [lnet]
[<ffffffff81061bc6>] ? put_online_cpus+0x56/0x80
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa081310c>] ptlrpc_ni_init+0x2c/0x1a0 [ptlrpc]
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa0813291>] ptlrpc_init_portals+0x11/0xf0 [ptlrpc]
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa09831c4>] init_module+0x1c4/0x1000 [ptlrpc]
[<ffffffff810020e2>] do_one_initcall+0xe2/0x190
[<ffffffff810ca7fb>] load_module+0x129b/0x1a90
[<ffffffff812da590>] ? ddebug_dyndbg_module_param_cb+0x0/0x60
[<ffffffff810c7133>] ? copy_module_from_fd.isra.43+0x53/0x150
[<ffffffff810cb1a6>] SyS_finit_module+0xa6/0xd0
[<ffffffff815f2119>] system_call_fastpath+0x16/0x1b
...

This is because in lnet_startup_lndnis(), we may add list items to
@the_lnet.ln_nis and @the_lnet.ln_nis_cpt before it failed. But in
lnet_startup_lndis() failure path,it did not cleanup list thus
causing assertion in lnet_unprepare().

Fix this problem by:
1) move lnet_shutdown_lndnis() back to lnet_startup_lndnis() so
that lnet_startup_lndnis() will cleanup itself.

2) move codes in lnet_startup_lndnis() that starts a single
NI into a new function called lnet_startup_lndni().

3)make lnet_dyn_add_ni() call lnet_startup_lndni() instead of
lnet_startup_lndnis().

This patch also fix problem LU-5734 addressed since they are
closely related.

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I1082361626881e798fca49981fe92b4082769ecf
Reviewed-on: http://review.whamcloud.com/11718
Tested-by: Jenkins
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5756 hsm: add missing return code in llapi_hsm_copytool_register 14/12314/6
Frank Zago [Thu, 16 Oct 2014 16:49:49 +0000 (11:49 -0500)]
LU-5756 hsm: add missing return code in llapi_hsm_copytool_register

llapi_hsm_copytool_register() returns a bogus error code on one error
path.

Removed a never true "if" case.

If the LL_IOC_HSM_CT_START fails, then we have to go to out_kuc,
not out_err.

Change-Id: I738078392f98d4616f6451beece4f0d97bc8051d
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/12314
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5743 build: Update to zfs/spl 0.6.3-1.1 73/12273/3
Nathaniel Clark [Fri, 10 Oct 2014 21:19:18 +0000 (17:19 -0400)]
LU-5743 build: Update to zfs/spl 0.6.3-1.1

Update ZFS and SPL version to the latest released patch version of
0.6.3-1.1

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I19ec6da5c431b41e741b469e43f5373502d43130
Reviewed-on: http://review.whamcloud.com/12273
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5641 tests: ensure user daemon is in group bin 44/12044/4
Bob Glossman [Wed, 24 Sep 2014 19:09:53 +0000 (12:09 -0700)]
LU-5641 tests: ensure user daemon is in group bin

sanity test_103a assumes user daemon is a member of group bin.
This addition makes that so even in el7 where it's
not true by default.

Test-Parameters: clientdistro=el7 mdsfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I180479e80d4da5f1bfec98c25676feaae85c29d7
Reviewed-on: http://review.whamcloud.com/12044
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
5 years agoLU-5287 export: hold exp_lock when modify exp_flags 71/11871/3
Niu Yawei [Thu, 11 Sep 2014 10:57:52 +0000 (06:57 -0400)]
LU-5287 export: hold exp_lock when modify exp_flags

There are few places that changing the exp_flags without holding
exp_lock, that could overwrite other concurrent exp_flags updating.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9c0421ffeacd6bf6feaa31dca9f9fc0e7963bc8b
Reviewed-on: http://review.whamcloud.com/11871
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5674 test: print spl debug info 80/11580/18
Minh Diep [Thu, 25 Sep 2014 05:30:54 +0000 (22:30 -0700)]
LU-5674 test: print spl debug info

Print /proc/spl files in case of tests fail

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I33890eedb9e5a2f87ae94cddd1db346c3c17c06d
Reviewed-on: http://review.whamcloud.com/11580
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4942 at: per-export lock callback timeout 36/9336/9
Vitaly Fertman [Fri, 10 Oct 2014 14:45:45 +0000 (18:45 +0400)]
LU-4942 at: per-export lock callback timeout

The lock callback timeout is calculated as an average per namespace.
This does not reflect individual client behavior.
Instead, we should calculate it on a per-export basis.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I12e3fc5f8d261cce252fcf13f22193273dc054ee
Tested-by: Elena Gryaznova <Elena_Gryaznova@xyratex.com>
Reviewed-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-by: Alexey Lyashkov <Alexey_Lyashkov@xyratex.com>
Xyratex-bug-id: MRP-417
Reviewed-on: http://review.whamcloud.com/9336
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
5 years agoLU-5626 ldiskfs: update non-htree dotdot in rename 39/11939/11
Patrick Farrell [Tue, 23 Sep 2014 15:14:24 +0000 (10:14 -0500)]
LU-5626 ldiskfs: update non-htree dotdot in rename

In 2.4+, when renaming a directory, its old dotdot entry will
be removed firstly, then the new dotdot entry is inserted, and
ldiskfs tries to append FID-in-dirent to the new entry.
But the space for dotdot entry may not be enough to hold
the new dotdot with FID-in-dirent, such as an MDT device
restored from file-level backup, or a device upgraded from 1.8.

In that case, for non-HTree directories, the ".." entry
will be written in the next available space in the directory
block.  This is invalid, as the ".." entry must be the
second entry in the block.

The same bug was fixed for HTree directories in LU-2638.
As Fan Yong said then: we do not want to introduce
complex logic to handle directory data moving, instead, in
such case, ignore the FID-in-dirent for the new dotdot entry,
and just insert the new dotdot entry.

There is one known flaw: This patch, like the one for
LU-2638, skips the entire data section rather than just
the FID.  This could cause trouble if something else ever
uses this section with ".." entries.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I57fc492e694973f5020191e4e2c79c74c7c4f18c
Reviewed-on: http://review.whamcloud.com/11939
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoLU-5675 quota: correctly set II_FL_NONUNQ in dt_index_read() 74/12074/3
Johann Lombardi [Fri, 26 Sep 2014 19:19:13 +0000 (21:19 +0200)]
LU-5675 quota: correctly set II_FL_NONUNQ in dt_index_read()

Fix regression introduced by 60e07b972114df24105a3a1bfa7365892f72a4a7
II_FL_NONUNQ should be set in the index information only when the
backend index supports DT_IND_NONUNQ.
This mistake triggers the following warning in the qsd code:

Lustre: 25029:0:(qsd_reint.c:237:qsd_reint_index()) lustre-MDT0000:
II_FL_NONUNQ is set on index transfer for fid
[0x200000005:0x1011:0x0], it shouldn't be

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I42a9c70666c741064b356ffd253b49e5ed7b3e93
Reviewed-on: http://review.whamcloud.com/12074
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5519 lfsck: LFSCK code framework adjustment (2) 45/11845/13
Fan Yong [Wed, 27 Aug 2014 15:44:02 +0000 (23:44 +0800)]
LU-5519 lfsck: LFSCK code framework adjustment (2)

Enhance LFSCK code framework for striped directory verification,
including:

1) New lfsck component APIs: lfsck_operations::lfsck_open_dir()
   and lfsck_operations::lfsck_close_dir().
1.1) The namespace LFSCK uses lfsck_open_dir() to initialize the
     LMV EA verification environment.
1.2) The namespace LFSCK uses lfsck_close_dir() to indicate the
     end of the scanning the striped directory.

2) When create orphan parent, the LMV EA should be considered.

3) Data structure definition and extention.

4) Other code cleanup.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I01e14645796c3fa3269c5f3cdcbb755ccb455d7e
Reviewed-on: http://review.whamcloud.com/11845
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5518 lfsck: recover orphans from backend lost+found 36/11536/25
Fan Yong [Wed, 27 Aug 2014 15:38:30 +0000 (23:38 +0800)]
LU-5518 lfsck: recover orphans from backend lost+found

Some local filesystem consistency verification tools, such as
e2fsck for ldiskfs, will add the orphan objects under backend
special /lost+found directory. Such directory is invisible to
clients. The namespace LFSCK will scan such directory, and for
the objects that were visible to clients originally (according
to the FID), the namespace LFSCK will move them back to the
normal namespace (according to the linkEA) or to the global
visible .lustre/lost+found/MDTxxxx/ directory.

The namespace LFSCK will insert the orphan (that is under the
backend /lost+found directory) FID into the namespace LFSCK
tracing file for further processing (via the subsequent namespace
LFSCK second-stage scanning). At the same time, remove the orphan
name entry from backend /lost+found directory. There is an interval
between the orphan name entry removed from the backend /lost+found
directory and the orphan FID in the LFSCK tracing file handled. In
such interval, the LFSCK can be reset, then all the FIDs recorded
in the namespace LFSCK tracing file will be dropped. To guarantee
that the orphans can be found when LFSCK run next time without
e2fsck again, when remove the orphan name entry, the LFSCK will
set the orphan's ctime attribute as 1. Since normal applications
cannot change the object's ctime attribute as 1. Then when LFSCK
run next time, it can record the object (that ctime is 1) in the
namespace LFSCK tracing file during the first-stage scanning. Once
the FID in the LFSCK tracing file has been handled successfully,
then the object's ctime attribute will be changed to normal time.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2f8e23801a868f2c99c630face4bbfe08ece8844
Reviewed-on: http://review.whamcloud.com/11536
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-5517 lfsck: repair invalid nlink count 16/11516/29
Fan Yong [Wed, 27 Aug 2014 15:12:44 +0000 (23:12 +0800)]
LU-5517 lfsck: repair invalid nlink count

If the namespace LFSCK has verified all the known name entries during
the first-stage scanning, then the MDT-object's linkEA is trustable.
So if the non-directory MDT-object's nlink attribute does not match
the MDT-object linkEA entries count, then the LFSCK will repair the
MDT-object's nlink attribute according to its linkEA entries count.

One exception is that: the linkEA space is limited, if there are too
much hard links on the MDT-object and exceeds the object's linkEA
space limitation, then some name entries cannot be recorded in the
linkEA. Under such case, we will add some flags (LLF_SKIP_NLINK)
in the LFSCK tracing file for related MDT-objects. Then the LFSCK
can skip the nlink attribute verification for the marked MDT-objects
during the second-stage scanning.

This patch also cleanup the LFSCK environment when current LFSCK
scanning exits (completed/stopped/failed) to avoid some stale to
misguide the next LFSCK scanning.

This patch also makes some code adjustment for the former landed
LFSCK patches according to the inspection feedback.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iedc676e8cc06a52f55e82372e6dc8b30008e20f4
Reviewed-on: http://review.whamcloud.com/11516
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>