LU-4343 tests: mkdir failing in sanity-hsm test 228

sanity-hsm test 228 calls mkdir on $tdir. Currently, the tdir
variable is two directories. This is changed in LU-2524. Until
LU-2524 lands, any call to mkdir with the tdir variable needs
the "-p" flag.

Also added removal of two files that the test creates and a new
routine to create small files with dd using the sync flag.

Lustre-commit: 2f253abab679c21b41197379b23e36943e5995a7
Lustre-change: http://review.whamcloud.com/8542

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Idd4354d6012032563d41c10238f619251e885e65
Reviewed-on: http://review.whamcloud.com/9280
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4505 quota: race of edquot updating

The slave edquot flag could be set mistakenly as following:

- slave A acquires quota from master, master found that the
  user is running out of quota, set edquot in reply;
- another slave deletes files and release quota to master,
  master clears edquot and notify all slaves by glimpse;
- glimpse reaches slave A before the reply of dqacq, so
  edquot flag will be set on slave A at the end.

Given that edquot can't be fully trusted, it should only be
revalidated every 5 seconds on the sync acquire path.

Lustre-commit: 109fef5b053490549726f7b5abc9ba840d3a4ae0
Lustre-change: http://review.whamcloud.com/8954

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I553bd1bc3aa6df6c449341e56564073043afd3da
Reviewed-on: http://review.whamcloud.com/9315
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4577 lnet: Dropped messages are not accounted correctly

LNET messages that are dropped are not accounted for correctly in
/proc/sys/lnet/stats. What I assume to be a simple typo is causing
drop_length to be double-counted and drop_count to never be
incremented.

Lustre-change: http://review.whamcloud.com/9096
Lustre-commit: 3abb0bb5f82559f2f5349dca763cf6edc7f6754b

Change-Id: Ia8454221885a1d765a3f7fadbcf5582fcfe7cf09
Signed-off-by: Matt Ezell <ezellma@ornl.gov>
Reviewed-on: http://review.whamcloud.com/9311
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4620 kernel: kernel update [RHEL6.5 2.6.32-431.5.1.el6]

update RHEL6.5 kernel to 2.6.32-431.5.1.el6

Lustre-commit: 6fa2299177e8749e63a88047aee49b5a9af6c3ef
Lustre-change: http://review.whamcloud.com/9253

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I54c6476c84650d7419d70fb89efc2680af6ecabe
Reviewed-on: http://review.whamcloud.com/9318
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2687 test: add b2_4 zfs image for conf-sanity test_32a

In order to ensure that we do not break ZFS upgrades
in the future, add 2.4.0 zfs filesystem test image for
conf-sanity.sh test_32a.

Test-Parameters: mdtfilesystemtype=zfs \
ostfilesystemtype=zfs mdsfilesystemtype=zfs \
envdefinitions=SLOW=yes testlist=conf-sanity

Change-Id: Iae560e05b428907409dc7069d30b601b52750cca
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/8824
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4484 lbuild: add support for fresh versions of MPSS 3.x.x

* Adopt lbuild script for new version of MPSS with x.x.x notation.
* Remove dependency from MPSS package to avoid renaming issue in
  the future. The name of package which was used for dependency
  was renamed in MPSS.
* Use new server with MPSS released packages for download.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie4407ad00177ad6d22770230a4dc6bde967d91ef
Reviewed-on: http://review.whamcloud.com/8836
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3968 lbuild: Extend script with build for Xeon Phi card

Automatically download, compile and produce Lustre client RPMs
for Xeon Phi(TM) card if "--mpss-version" option is specified
for contrib/lbuild/lbuild script.

Also try to compile with Xeon Phi(TM) OFED if it's available.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ida07d764dc824c13f22ffb53d24e2c6f79ce3573
Reviewed-on: http://review.whamcloud.com/7066
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4613 tests: purge older request result in test_12o

sanity-hsm/test_12o sub-test, which has been introduced as part
of LU-3834, submits 2 RESTORE requests for the same FID and thus
needs to purge 1st result from log before to check 2nd.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ia2a0ead487b29a68c8a920bae2aa1d654eac4051
Reviewed-on: http://review.whamcloud.com/9295
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4442 test: add version check for replay-vbr.sh test_7g

In replay-vbr.sh test_7g.3, because mdt_object_exists() was added
in http://review.whamcloud.com/#/c/8371, client will not be evicted
without object version check.

The patch also fixes the wrong usage of wait_mds_ost_sync() in
replay_vbr.sh test_7_cycle(). The first parameter should be a timeout
in seconds, not a facet.

Test-Parameters: envdefinitions=SLOW=yes,ONLY=7 testlist=replay-vbr

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I4a960fc53451fc717370bc96f926f067bbb2946a
Reviewed-on: http://review.whamcloud.com/9290
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4386 osc: don't activate deactivated obd_import

In ptlrpc_activate_import(), obd_import->imp_deactive should
be checked if it is deactivated, otherwise it will trigger an
LBUG in ptlrpc_invalidate_import():

  ptlrpc_invalidate_import() ASSERTION(imp->imp_invalid) failed

Change-Id: I4c16f166c0c2cf60664119bf438dfd8606d71a2f
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/9283
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4293 mdd: Allow layout swap for IGIF FIDs

Patch to also allow layout swap for pre-2.x migrated
files (ie, IGIF FID with linkEA).

Root user special case has also been added to lfs/migrate
command to map owner/group of original file to
volatile, in order to comply with other layout_swap rules.

Lustre-commit: bd5ba50502bec5786c9a2f05c29f7b99a35147fb
Lustre-change: http://review.whamcloud.com/8737

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iad6194c6050fa2ba066d2051871a10a60ddae995
Reviewed-on: http://review.whamcloud.com/9278
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4208 osd-zfs: hold pool config lock to register property

- Hold the DSL pool configuration lock when calling
  dsl_prop_register().  Failure to do so will panic the node if
  assertions are enabled in ZFS.  This change requires a build of ZFS
  on Linux that exports symbols dsl_pool_config_enter and
  dsl_pool_config_exit, which was done in commit 40a806d [1], and will
  appear in ZFS release 0.6.3.

- Fix up variable declaration alignment in osd_mount().

- Add check for exported symbols in autoconf

[1] https://github.com/zfsonlinux/zfs/commit/40a806d

Lustre-change: http://review.whamcloud.com/8172
Lustre-commit: f8bc2f7fc03d3f86eef434cf644191e689ee57ec

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Ice673efb5501456d1a4f423ec08dfb4f571f8221
Reviewed-on: http://review.whamcloud.com/9256
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4429 llite: fix open lock matching in ll_md_blocking_ast()

In ll_md_blocking_ast() match open locks before all others, ensuring
that MDS_INODELOCK_OPEN is not cleared from bits by another open lock
with a different mode. Change the int flags parameter of
ll_md_real_close() to fmode_t fmode. Clean up verious style issues in
both functions.

Lustre-commit: 2b23ad0d183141dc25377f2d37de6e6e36ba1169
Lustre-change: http://review.whamcloud.com/8718

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Ic44ac8ac8c07b71d4c929d7d359bee881c6b05b0
Reviewed-on: http://review.whamcloud.com/9260
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4287 kernel: kernel update RHEL6.5 [2.6.32-431.3.1.el6]

Add RHEL6.5 support [2.6.32-431.3.1.el6]

ext4 in RHEL6.5's kernel version 2.6.32-431.3.1.el6 no longer contains
the required function ext4_ext_walk_space(). We start a new rhel6.5
ldiskfs patch series and reintroduce ext4_ext_walk_space() through an
new patch, copying ext4_ext_walk_space() from older kernel rhel6.4
2.6.32-358.23.2.el6.

Lustre-commit: efa8fa578d2f7eeeaea11522dd311dddaa715a03
Lustre-change: http://review.whamcloud.com/8549

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I5cff1860c43d06a6399b43f92ef90283c4600c8e
Reviewed-on: http://review.whamcloud.com/9103
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4154 lfsck: skip old lfsck test in DNE mode

The old e2fsck/lfsck tool will not be allowed to run on a DNE
filesystem. This patch updates generate_db() to pass master MDS
parameters only, so that the old lfsck does not corrupt it or
delete all of the files on other MDTs.
This patch also fixes a typo in run_lfsck_remote().

This patch is back-ported from the following one:
Lustre-commit: b5f3d6db9200e369a68284a8ef85a1205e5905e1
Lustre-change: http://review.whamcloud.com/8206

Test-Parameters: alwaysuploadlogs mdtcount=4 testlist=lfsck
Test-Parameters: alwaysuploadlogs mdtcount=1 testlist=lfsck

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I5590b20c1c0003dbb1975a254093724de22497d4
Reviewed-on: http://review.whamcloud.com/9176
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-946 lprocfs: List open files in filesystem

Added lprocfs file on MDT to list open files in per-export
directory for mdt.

Test-Parameters: testlist=sanity,sanityn
Signed-off-by: Girish Shilamkar <gshilamkar@ddn.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: If8f233d95dca4cd4c4044d85bd117a027dabd80e
Reviewed-on: http://review.whamcloud.com/6386
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Swapnil Pimpale <spimpale@ddn.com>

LU-3528 mdt: check object exists for remote directory

Check whether the remote object exists before enqueue and
getattr to avoid LBUG.

Remove unnecssary remote object exist check in mdd_object_lock.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia634a8c7b9cd2810515e854163c5fdd6bdf8716f
Reviewed-on: http://review.whamcloud.com/8371
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4430 mdt: check for MDS_FMODE_EXEC in mdt_mfd_open()

In the error path of mdt_mfd_open() check for MDS_FMODE_EXEC rather
than FMODE_EXEC in the open flags.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I04c53eb1af0fdeeb2c2b0c2f2ef1340b247921d8
Reviewed-on: http://review.whamcloud.com/8719
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4260 lod: free striping if striping initialization fails

It should free striping if striping information initialization is
faild, otherwise the later object find will pick up this wrong lod
object, and hit LBUG

ASSERTION( lc->ldo_stripenr == 0 ) failed:

[<ffffffffa0349895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
[<ffffffffa0349e97>] lbug_with_loc+0x47/0xb0 [libcfs]
[<ffffffffa0e3f78f>] lod_ah_init+0x57f/0x5c0 [lod]
[<ffffffffa0b73a83>] mdd_object_make_hint+0x83/0xa0 [mdd]
[<ffffffffa0b7feb2>] mdd_create_data+0x332/0x7d0 [mdd]
[<ffffffffa0d9cc2c>] mdt_finish_open+0x125c/0x18a0 [mdt]
[<ffffffffa0d984f8>] ? mdt_object_open_lock+0x1c8/0x510 [mdt]
[<ffffffffa0d9ee8d>] mdt_reint_open+0x115d/0x20c0 [mdt]

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I67b2bd0e013b860767d19eda986fdcff7e16c486
Reviewed-on: http://review.whamcloud.com/8324
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3772 ptlrpc: fix nrs cleanup

When service start failed due to short of memory, the cleanup code
could operate on uninitialized structure and cause crash at the end.

This patch fix the nrs_svcpt_cleanup_locked() to perform cleanup only
on the nrs which has been properly initialized.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ieafa5b144133490b662f5a80a7b99311a9970de3
Reviewed-on: http://review.whamcloud.com/7410
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3857 osd: cleanup procfs after osd_shutdown

Since osd_procfs_fini will try cleanup all proc entries,
and osd_shutdown/qsd_fini will try to cleanup procfs
itself, so osd_procfs_fini should be done after qsd_fini,
otherwise the qsd entries will be destoryed twice, and
caused panic

Call Trace:
 [<ffffffffa081cc45>] lprocfs_remove+0x25/0x40 [obdclass]
 [<ffffffffa0b23dd0>] qsd_fini+0x80/0x450 [lquota]
 [<ffffffffa0d2ec78>] osd_shutdown+0x38/0xe0 [osd_ldiskfs]
 [<ffffffffa0d36bf9>] osd_device_fini+0x129/0x190 [osd_ldiskfs]
 [<ffffffffa0834913>] class_cleanup+0x573/0xd30 [obdclass]
 [<ffffffffa081233c>] ? class_name2dev+0x7c/0xe0 [obdclass]
 [<ffffffffa083663a>] class_process_config+0x156a/0x1ad0 [obdclass]
 [<ffffffffa06be9b8>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffffa082f202>] ? lustre_cfg_new+0x312/0x6e0 [obdclass]
 [<ffffffffa0836d19>] class_manual_cleanup+0x179/0x6e0 [obdclass]
 [<ffffffffa06be9b8>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffffa0d378b4>] osd_obd_disconnect+0x174/0x1e0 [osd_ldiskfs]
 [<ffffffffa083926b>] lustre_put_lsi+0x1ab/0xeb0 [obdclass]
 [<ffffffffa08414d8>] lustre_common_put_super+0x5c8/0xbe0 [obdclass]
 [<ffffffffa087081d>] server_put_super+0x1bd/0xed0 [obdclass]
 [<ffffffffa0871bbb>] server_fill_super+0x68b/0x1630 [obdclass]
 [<ffffffffa0840bb8>] lustre_fill_super+0x1d8/0x530 [obdclass]
 [<ffffffffa08409e0>] ? lustre_fill_super+0x0/0x530 [obdclass]

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: If12cebf971583afeeaa031bd24f69bb0fe0cdf1a
Reviewed-on: http://review.whamcloud.com/8506
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2818 mdt: Properly handle ENOMEM

When osd_keys_init fails in mdt_lvbo_fill, properly bail out with
error instead of asserting.

Change-Id: I832742ed49cc7740d8e709bc4b87e5d5aa100d39
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8947
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3618 ptlrpc: rq_commit_cb is called for twice

If a ptlrpc_request is already on imp::imp_replay_list, when it's
replayed and replied, after_reply() will call req::rq_commit_cb
for the request, then call it again in ptlrpc_free_committed.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I796c3351ad896aa3e1d0c2147ca7f775b7c14bfc
Reviewed-on: http://review.whamcloud.com/8815
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4454 libcfs: warn if all HTs in a core are gone

libcfs cpu partition can't support CPU hotplug, but it is safe
when plug-in new CPU or enabling/disabling hyper-threading.
It has potential risk only if plug-out CPU because it may break CPU
affinity of Lustre threads.

Current libcfs will print warning for all CPU notification, this
patch changed this behavior and only output warning when we lost all
HTs in a CPU core which may have broken affinity of Lustre threads.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I62267b62871c129beeb1593c4f69e7b81a79999d
Reviewed-on: http://review.whamcloud.com/8770
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3601 Do not create layout in lease-open

leases are not real opens so it makes no sense to create layouts
when the lease is taken.

Change-Id: Ica2d6a348c360bd20bb7bd27061839df84dae84b
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/8084
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4152 mdt: Don't enqueue two locks on the same resource

Due to mechanics of ldlm internals, enqueueing two different ibits
lock on the same resource is deadlock prone.
As such change mdt_object_open_lock to release open lock if it becomes
necessary to get exclusive layout lock (to create objects).
It's ok to release the open lock right away as it's never guaranteed to
be issued anyway.

Change-Id: Ib669e68323ea72c75a0a8bea289d8bea079309b0
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8083
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3834 mdt: handle swap_layouts failures during restore

Actually nothing occur after swap_layouts failures during restore,
this can lead to file being left in incoherent state and thus be
unavailable because HS_RELEASED is clear but LOV_PATTERN_F_RELEASED
is still set.
This patch will allow original layout to be recovered by the use of
SWAP_LAYOUTS_MDS_HSM flag. Additionaly this requires HSM xattr of
the data FID to be set.
Also adds layout-swap failure injection and related test.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Id0e9a005362e4a3854b33f6ce1888197d20e7dbf
Reviewed-on: http://review.whamcloud.com/7631
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4336 quota: improper assert in osc_quota_chkdq()

In osc_quota_chkdq(), we should never try to access oqi found
from hash, since it could have been freed by osc_quota_setdq().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ia73cf89cb5bbd730fa6f0a00e44771f733b2baa6
Reviewed-on: http://review.whamcloud.com/8460
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-4253 osc: Don't flush active extents.

The extent is active so we need to abort and let the caller
re-dirty the page. If we continued on here, and we were the
one making the extent active, we could deadlock waiting for
the page writeback to clear but it won't because the extent
is active and won't be written out.

Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: Iba646d8185b12ab227fe0bbee1c6602ccdc32ad6
Reviewed-on: http://review.whamcloud.com/8278
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3373 misc: small changes for 3.10 server support

--quota use struct kqid as parameter
--export ext4_dec/inc_count for nlink count
--ext4_find_entry & ext4_journal_start_sb changes
--iop->truncate removed
--other trival changes to calm compiler warning

Lustre-commit: 9bd7e40d2934cd0162eeff5388f054444a982ac9
Lustre-change: http://review.whamcloud.com/7794

Signed-off-by: yang sheng <yang.sheng@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia2670d925ecbbfcc1ed3abb1a15a8d91fa27bd32
Reviewed-on: http://review.whamcloud.com/9013
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4106 scrub: Trigger OI scrub properly

There is the following race case between osd_fid_lookup() and object
unlink/detroy:

Both RPC service thread_1 and RPC service thread_2 try to find the
same obj_A at the same time. At the beginning, the obj_A is not in
cache. The thread_1 is in osd_fid_lookup() and finds the OI mapping
for obj_A. But before the thread_1 finding out related inode_A, the
thread_2 moves faster and finds the inode_A and unlinks the inode_A.
So the thread_1 will fail to find the inode_A. Under such case, the
thread_1 will try to check OI again to make sure whether related OI
mapping is still there or not. If no OI mapping, then it is normal
becuase someone has unlinked the file by race; otherwise, it may be
caused by file-level backup/restore, then thread_1 will trigger OI
scrub to rebuild OI files.

But we ignored a corner case that the thread_1 recheck the OI files
may just between the thread_2 has dropped the inode_A's referene to
zero and will remove related OI mapping from the OI file. Then the
thread_1 is misguided, and will trigger OI scrbu unexpectedly.

More initial OI scrub for the /ROOT/.lustre directory to make sure
the necessary files/directories for mount are ready before used.

This patch also enhances the ls_locate()/dt_locate_at() interface
to allow the caller to pass some hints to low layer, such as flag
LOC_F_NEW for create, to help the low layer to handle efficiently
and properly.

This patch is back-ported from the following ones:
Lustre-commit: 8931d9070415e808e09bb4befd7cd38ef2431149
Lustre-change: http://review.whamcloud.com/8002
and
Lustre-commit: bab8a7dd5597014ee68e52bd39bde0ed40711777
Lustre-change: http://review.whamcloud.com/8101

Test-Parameters: mdtcount=4 testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5259549340f97a2f9118ab1db081f2ab2cfd8933
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/9006
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4178 tests: disable HSM sanity subtests

Turn off high failure rate subtests in sanity-hsm.
If fixes for these failures land tests may be turned on again later.

Lustre-commit: d85d724d9a3a503718f6df840be67e5f6f5af78c
Lustre-change: http://review.whamcloud.com/8122

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2e3f3822492a3398ebcbd2ba4565455986515764
Reviewed-on: http://review.whamcloud.com/9134
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4554 lfsck: old single-OI MDT always scrubbed

Old ldiskfs MDT's that contain a single OI container named "oi.16"
trigger an automatic OI scrub on each restart. This is because
osd_oi_table_open() gets ENOENT opening "oi.16.0" and consequently
sets bit 0 in scrub_file::sf_oi_bitmap. This bit indicates the OI
container 0 needs to be recreated, and it triggers a scrub in
osd_fid_lookup() for lookups that fail with ENOENT. Fix this by
clearing the bit in osd_oi_init() after a successful open of
"oi.16".

Lustre-change: http://review.whamcloud.com/#/c/9067
Lustre-commit: b4159f5d722bc43cff82b4c45336b01fd769e1db

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I3f19b15b51fce85bf791df76389f0b28951356c3
Reviewed-on: http://review.whamcloud.com/9139
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4512 hsm: Fix lhsmtool_posix --report option

The --report option is intended to allow an override of the
default copytool progress reporting interval, but it doesn't
work. This commit implements the intended functionality and
renames the option to "--update-progress", or "-u" for short.

Also fixes the progress display in hsm/active_requests to
reflect the change from percentage complete to bytes moved.

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Id6ead1b33868e3454f00053165944bc3900cabb4
Reviewed-on: http://review.whamcloud.com/8934
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4589 kernel: kernel update [SLES11 SP3 3.0.101-0.15]

update target and config files for new kernel version

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I28496e7dc9322dcf7fec4493602042cc89db6fec
Reviewed-on: http://review.whamcloud.com/9150
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4263 osd-zfs: Avoid converting last ID FIDs to OST IDs

When obdfilter-survey first creates an object on a fresh ZFS OST, the
last ID object for FID_SEQ_ECHO has to be created in the first place.
The last ID FID, [FID_SEQ_ECHO:0:0], can not be converted to an OST ID
because the resulting OST ID would be indistinguishable from an
FID_SEQ_OST_MDT0 OST ID and would confuse ostid_id().  This patch
checks for last ID FIDs before converting them to OST IDs in
osd_get_idx_for_ost_obj().

Change-Id: I96cdf85b4725e4882cecabaf90466c7f77a5e0a6
Intel-bug-id: FF-182
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/8301
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4293 utils: handle lfs migrate failure in lfs_migrate

If "lfs migrate" returns an error, possibly because it is refusing
to migrate an IGIF FID, fall back to using rsync to copy the file
and rename it.  Print a message in this case so the user knows it
is not a fatal error yet.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I114006afb93d8c8d78923a874f3b914200500c1e
Reviewed-on: http://review.whamcloud.com/8616
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3806 obdclass: add LCT_SERVER_SESSION for server session

Add LCT_SERVER_SESSION for server session, and separate the
server session flag from LCT_SESSION, so to avoid allocating
session info for client stack for each server request, if
client and server are on the same node.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I808c3f58cd7a03ebc166e51fe1e32ea34ae0e3e8
Reviewed-on: http://review.whamcloud.com/7412
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4194 ldlm: Make OBD_[ALLOC|FREE]_LARGE use consistent

struct ldlm_lock's l_lvb_data field is freed in ldlm_lock_put()
using OBD_FREE.  However, some other code paths can attach
a buffer to l_lvb_data that was allocated using OBD_ALLOC_LARGE.
This can lead to a kfree() of a vmalloc()ed buffer, which can
trigger a kernel Oops.

Change-Id: Ic75a67530862eeb4d065c14bbbac80939bff5731
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/8298
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4194 ldlm: set l_lvb_type coherent when layout is returned

In case layout has been packed into server reply when not
requested, lock l_lvb_type must be set accordingly.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Iaf54c9ba27785e529f4f2fb967d2fad4fc1dfbcb
Reviewed-on: http://review.whamcloud.com/8270
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-4196 build: Build support for OFED-3.5 and SLES 11

CONFIG_COMPATE_SLES_11_SP* needed in EXTRA_LNET_INCLUDE to allow
building against OFED-3.5

Lustre-commit: 369e02e84f39565195e08f043ab0421d2d3bd185
Lustre-change: http://review.whamcloud.com/8140

Test-Parameters: clientdistro=sles11sp3 ossdistro=sles11sp3 \
mdsdistro=sles11sp3 nettypes=o2ib clientibstack=inkernel \
ossibstack=inkernel mdsibstack=inkernel testlist=sanity

Signed-off-by: Chris Horn <hornc@cray.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib26c757044aff828c3bbbd3adfd5fb709cca9cf0
Reviewed-on: http://review.whamcloud.com/8882
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4124 build: make module installation directory flexible

Add --with-kmp-moddir option to configure.

Distributions vary in the installation directory for kernel modules.

The RHEL standard installation directory is
        /lib/modules/$(uname -r)/extra
while the SLES standard is
        /lib/modules/$(uname -r)/updates

Adding the option --with-kmp-moddir option to configure allows users
to select the appropriate installation target.  With this change, it
is neccessary to support both options in the test framework.

Signed-off-by: Stephen Champion <schamp@sgi.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/8065
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Iec3137e0e5039dd43622c2e285030a5339fa6fd3
Reviewed-on: http://review.whamcloud.com/8315
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>

LU-3679 lnet: reflect down routes in /proc/sys/lnet/routes

We consider routes "down" if the router is down or the router
NI for the target network is down. This should be reflected
in the output of /proc/sys/lnet/routes

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I82ee769d88aec92f1690ad9c095e32c9a9f9e282
Reviewed-on: http://review.whamcloud.com/7857
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8195
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-4231 llite: proper support of NFS anonymous dentries

NFS can ask to encode dentries that are not connected to the root.
The fix check for parent is NULL and encode a file handle accordingly.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Idba91fd4bca4f26a37fd9bc76a340d2fbf557c9e
Reviewed-on: http://review.whamcloud.com/8347
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8498

LU-4444 tests: Skip conf-sanity/69 on zfs

Because file creates happen slowly on ZFS and the number of files
required to run the test is 100K, this test cannot run in a
reasonable amount of time.

Also bail out of test if createmany fails (possible if MDS or OST is
too small), this prevents the test from just timing out instead.

This patch is back-ported from the following one:
Lustre-commit: eb38c458c868d5389e2641189218f22ad1272aef
Lustre-change: http://review.whamcloud.com/8841

Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity
Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity \
mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I6d9daad3239b576935190a121a2aa818441ec97b
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8854
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3189 tests: add version check code into sanity test 53

This patch adds Lustre version check codes into sanity test
53 to make the test work with servers that do not have the
following patch:

Lustre-commit: 6c4c51e3079e6c257fbf86536e4739110c166e3b
Lustre-change: http://review.whamcloud.com/4789

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=53 \
ossjob=lustre-b2_3 mdsjob=lustre-b2_3 ossbuildno=41 mdsbuildno=41 \
testlist=sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ie6eaeee31780f4ea4077805f52efda279ff96670
Reviewed-on: http://review.whamcloud.com/8834
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4222 mdt: extra checking for getattr RPC.

Check whether getattr RPC can hold layout MD(RMF_MDT_MD),
in case the client sends some invalid RPC, which can
cause panic on MDT.

Client will retrieve cl_max_md_size/cl_default_md_size
from MDS during mount process, so it will initialize
cl_max_md_size/cl_default_md_size before sending getattr
to MDS.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I43bbe54c37360242bb7a3cd2aa8d90c2b9e0baf1
Reviewed-on: http://review.whamcloud.com/8599
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4360 Fix use after free in ksocknal_send

Call to ksocknal_launch_packet might schedule a callback that
might free the just sent message, and so subsequent access to it
via lntmsg->msg_vmflush goes to freed memory.

Instead we'll just remember if we are in the vmflush thread and
only restore if we happened to set mempressure flag.

Change-Id: I2f0f8b27e26e11b37ad60fde4c98e86c39768349
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3680 ptlrpc: Fix assertion failure of null_alloc_rs()

lustre_get_emerg_rs() set the size of the reply buffer to zero
by mistake, which will cause LBUG in null_alloc_rs() when memory
pressure is high. This patch fix this problem and adds a size
check to avoid the problem of insufficient buffer size.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I9fbd4f14e8e1263de2af564c4f2e420f5f2b43bc
Reviewed-on: http://review.whamcloud.com/8200
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4221 osd: add case LCFG_PARAM to osd_process_config

Some proc parameters were moved from ofd to osd module and only
their symlinks were kept in ofd for interoperability/compatibility.
To process this kind of config params passed by ofd, this patch is
to do the following fixes:
- add case LCFG_PARAM to osd_process_config() to process parameters
  with prefix both PARAM_OSD and PARAM_OST.
- since these parameters are not included by the static lprocfs var
  list, a pre-check is added for them to avoid "unknown param" error
  message confuses the uses. If they are matched in this check, they
  will be passed to the osd directly.
- get rid of lprocfs_osd_init_vars() and use struct lprocfs_vars
  lprocfs_osd_{obd,module}_vars[] instead.
- improve the error messages in class_process_proc_param() and
  class_process_proc_seq_param() a little.
- add conf-sanity.sh test_28a to verify the patch and skip this test
  for ZFS OSTs since ZFS has no such kind of parameters.

This is a backport of commit b1491d26271f074dc6f99cde037403337c0b2151
in http://review.whamcloud.com/8238 .

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Signed-off-by: Michael MacDonald <mjmac@whamcloud.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I8b8d4244f90bd9e16acdccedd09da73fbb5e501b
Reviewed-on: http://review.whamcloud.com/8618
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Michael MacDonald <michael.macdonald@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4276 ldiskfs: enable read/write access by default

Add build time config option to allow read/write access by default.
While the new CONFIG_LDISKFS_FS_RW only matters in SLES11 builds,
it's easiest to just add the flag to all builds unconditionally.
It will be ignored in builds where it doesn't matter.

Lustre-commit: 14c94c20c3447584e81d720c2b2a17888716709e
Lustre-change: http://review.whamcloud.com/8335

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I4fcf0b2f884b1442db0aac5788bf62f07537c5d4
Reviewed-on: http://review.whamcloud.com/8779
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4030 tests: use free_fd() to allocate file descriptor

free_fd() lists /proc/self/fd to find the smallest unused file
descriptor
sanity test_31n is changed to use free_fd() instead of using hardcoded
173
sanity test_236 is changed to use free_fd() instead of using "{FD}<>"
which is not available on eariler bash

Since test_31n now uses the function free_fd to find an unused file
descriptor, it no longer depends on fd 173 being free.  This change
also removes that test on whether fd 173 is in use.

This patch is back-ported from the following ones:
Lustre-commit: 1f9235152b2f44c7bd64c5c021066f1984f341e6
Lustre-change: http://review.whamcloud.com/8181
and
Lustre-commit: 73e816e57167eb92425b6cf29fc570e56c88f6bd
Lustre-change: http://review.whamcloud.com/8622

Signed-off-by: Vladmir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I0c9c04787d45dfe6ba5ed01adb0a8ee265c6b3c5
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8811
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-3939 tests: sanity-hsm/test_40 needs a local HSM_ARCHIVE

sanity-hsm/test_40 suffers frequent failures during auto-test due
to remote/NFS-mounted HSM_ARCHIVE causing the 400 archive requests
to take more than 100s to be drained from copytool requests queue.
This patch allows copytool_setup func to allow each sub-test to
specify a non-default hsm-root/HSM_ARCHIVE dir and test_40 uses it.

This patch is back-ported from the following one:
Lustre-commit: 8484f1c51c701141237e98a1467c75364766f357
Lustre-change: http://review.whamcloud.com/7703

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I733b267991faa3b8c9415fea116d2086575333bb
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8771
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3920 tests: check MDS version before testing HSM feature

Sanity tests 56y and 229 fail when MDS does not have HSM
support. Check MDS version and skip the tests in that case.

This patch is back-ported from the following one:
Lustre-commit: b635ddd7f6ebe04681fae34da3b26e3b6b5301f0
Lustre-change: http://review.whamcloud.com/8121

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=229 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=70 mdsbuildno=70 \
testlist=sanity

Xyratex-bug-id: MRP-1417

Signed-off-by: Vladimir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I6bf3bffd45ad8a2a7c72424447a4d486389c8e8d
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8734
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4299 kernel: kernel update [SLES11 SP3 3.0.101-0.8]

update target and config files for new kernel version

Lustre-commit: a6bf2c1ee73a217df8e0b44fb0d5cea15a3bd874
Lustre-change: http://review.whamcloud.com/8762

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I1970bc9657286b57746e3f0a18ca9d22f134189e
Reviewed-on: http://review.whamcloud.com/8766
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4165 tests: skip sanity-lfsck test_2c for 2.4 or older

It makes no sense to run sanity-lfsck test_2c against 2.4 or older.

Test-Parameters: mdsjob=lustre-b2_4 ossjob=lustre-b2_4 mdsbuildno=58 ossbuildno=58 testlist=sanity-lfsck
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I96792b4325a69f880e326dc8963cf3e6bd09bf87
Reviewed-on: http://review.whamcloud.com/8386
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8728
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4072 tests: Decrease load on MDT for ZFS in sanity/24v

Due to performance of ZFS, reduce the number of file creates until
LU-2887/LU-4072 are resolved.

This patch is back-ported from the following one:
Lustre-commit: ee009f3b3e7bd467df3da3d0b53777db65790062
Lustre-change: http://review.whamcloud.com/7870

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I6271a7892c02885855b9e5b750438087e7875c5b
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8769
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4223 utils: fixing loop leaking in utils

1. If the file is being opened by popen, it should use
pclose instead of fclose to close the file, to make sure
the process created by popen is closed after pclose, then
to avoid loop device is being hold on release.

2. Give another try in loop_cleanup in case there are still
some process going on with the loop.

3. wait loop device to release before continue conf-sanity
32c.

4. Add losetup -a to list loop dev information when the
test(conf-sanity 32) fails.

This patch is back-ported from the following one:
Lustre-commit: 98ac0fe3a45dde62759ecaa4c84e6250ac2067f8
Lustre-change: http://review.whamcloud.com/8409

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdscount=4 mdtcount=4 testlist=conf-sanity

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ic1ebc2a6b2ce4280c2123080171e203e99267b28
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8723
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4270 test: fix sanity test_209

Fix the connect_flags checking in test_209 of sanity.sh

This patch is back-ported from the following one:
Lustre-commit: b498499104af17da081f1c22b9c07951104846a3
Lustre-change: http://review.whamcloud.com/8326

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=209 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=70 mdsbuildno=70 \
testlist=sanity

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I6c34b9dbe6d3b7475d85588e7adb3acb762fab32
Reviewed-on: http://review.whamcloud.com/8729
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3971 hsm: Copytool code cleanup

Minor refactoring of the bandwidth controling code.
Deletion of a superfluous select() call on regular files.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: Iae550bb69c1524865b38a92d9b7674fce2f58258
Reviewed-on: http://review.whamcloud.com/7583
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4046 build: fix 'uninitialized variables' errors

Fix 'uninitialized variables' defects found by Coverity version 6.5.1:
Uninitialized scalar variable (UNINIT)
Using uninitialized value.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I4b5a3293e7d01436a82b6f7cea50b84c60660b3a
Reviewed-on: http://review.whamcloud.com/7821
Tested-by: Hudson
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4022 hsm: Set agent_actions start index from -1

aai_index was set to start from 0 which will cause the first entry
to be skipped. Set it to -1 instead.

Also take llog lock in proc_show() to avoid it from being changed
during the iteration.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I09013dbf8a85d198496d8470746e773b388d9bee
Reviewed-on: http://review.whamcloud.com/7879
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3726 tests: DNE specific tests for HSM.

Add tests in sanity-hsm.sh to check DNE specific
features in HSM code: copytool registration to
several MDTs, HSM request shunting to the right MDT,
and coumpound request splitting.

Test-Parameters: mdtcount=2 mdscount=2 testlist=sanity-hsm
Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I6706571c92e4c3e978b2143dd5bdef4b73d2c1e2
Reviewed-on: http://review.whamcloud.com/7571
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3726 tests: make sanity-hsm compatible with MDSCOUNT >= 2.

Add generic functions to perform set_param and check_param
on all MDTs, and persistent set_param on the mgs.
CL_USER variable replaced by CL_USERS array (1 value per MDT).
MDT0 variable replaced by MDT array (1 value per MDT).

Test-Parameters: mdtcount=2 mdscount=2 testlist=sanity-hsm
Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I0d8610a74427423dda25a6defd85f5c52c535850
Reviewed-on: http://review.whamcloud.com/7437
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3973 tests: use -P option of df to avoid line breaks

Sometimes df command output requires to break lines to
allow for columns alignement. This causes fields parsing
to fail. Use -P option to select compatibility output
and ensure one single line format per file-system is used.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ibc9d4b8f50857e680a5590dbbfe312fce137bcb6
Reviewed-on: http://review.whamcloud.com/7915
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-3985 tests: Archive access must be done from an agent

In sanity-hsm, all access to the archive backend must be
done from an agent. Tests 111a and 111b call the
copytool_remove_backend routine that calls find on the
archive backend. With this patch, the call to find is
done on the agent with a call to "do_facet $SINGLEAGT".

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I537fe5e019861908612140b3c96ea2018a738e46
Reviewed-on: http://review.whamcloud.com/8076
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3815 tests: HSM sanity test suites

Reevaluate the exception list.
Enable all sanity-hsm tests except #34, #35, #36 and #40.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I66dbf6f5119fc75b379752a33a8d5e44562ff985
Reviewed-on: http://review.whamcloud.com/7374
Tested-by: Hudson
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4064 hsm: create files to be imported on agent node

In sanity-hsm test 13 create the files to be imported on the agent
node rather than the local node.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3f50cc5269512d3b99644119ea18272d3a80f8a7
Reviewed-on: http://review.whamcloud.com/7885
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3692 api: More flexible logging API.

Allow applications to register their own logging callback instead of
writing every log message to stderr/stdout. Registered callbacks are
global to the process, just like the log level.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I5ccfd1e24f1786408ce67b2cd87d53443bcd028a
Reviewed-on: http://review.whamcloud.com/7572
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3909 tests: sanity test 129 fix

Patch from LU-2479 changed max dir size limit behaviour. Before the
patch last block of a directory which reached its limit was mostly
empty so that creation of new file after limit cancelation caused no
dir size grow. With the patch last block of limited directory gets
full when limit is reached. Therefore dir size increases on file
creation after limit cancellation.
The patch makes sanity test_129 to distinguish these cases.

This patch is back-ported from the following one:
Lustre-commit: 5cb9ac9c09f25234babc239ed949d85cd8489bef
Lustre-change: http://review.whamcloud.com/8143

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=129 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=70 mdsbuildno=70 \
testlist=sanity

Xyratex-bug-id: MRP-1444

Signed-off-by: Vladmir Saveliev <vladimir_saveliev@xyratex.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I691d7bac3dffe4eea98085ef2f9be3736e970869
Reviewed-on: http://review.whamcloud.com/8727
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3929 lwp: don't connect LWP to old MDT

When rolling upgrade Lustre from old version without LWP support
(< 2.3.60) to newer version, the OSTs will be upgraded firstly,
then LWP will try to connect the old MDT, and old MDT will regard
the LWP client as normal client and save client export on disk for
recovery, which will break recovery at the end since LWP
connection isn't recoverable.

Change-Id: Ib58c11a9d06b78ac9617563ae1baa4844e0efa45
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-on: http://review.whamcloud.com/8701
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4060 test: skip sanityn.sh test_74 on older than 2.4.93 MDS

2.5 client 2.4 server interop testing shows that sanityn.sh
test_74 needs to be skipped for Lustre MDS versions older than
2.4.93 servers.

This patch is for the b2_5 branch.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I8fd414adcf6bb6b7f8b390f1c52e3fea42701d1b
Reviewed-on: http://review.whamcloud.com/8700
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3952 nfs: don't panic NFS server if MDS fails to find FID

When MDS fails to retrive the parent's fid, we'd handle it without
crashing the NFS server.

Lustre-commit: 57649535fee8e92412db6806a30d2fa0752f762f
Lustre-change: http://review.whamcloud.com/8459

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I2ad36aa1ac2b7ae131cb21bf955a8ec6933d1dc3
Reviewed-on: http://review.whamcloud.com/8706
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4175 tests: fix obdecho_test() to calculate write count

This patch fixes obdecho_test() in sanity.sh to calculate
write count so as to avoid out of space failure while the
page number is large but the obd target size is small.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I5cbffb64387a62562eb85dd0208f9b1d7846ae31
Reviewed-on: http://review.whamcloud.com/8698
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4285 kernel: fix uninitialized access of quota got[] array

The pointer maybe uninitialized when dqput be called.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I94c138664c2f0a2404f2dafda25f3ee968a981fb
Reviewed-on: http://review.whamcloud.com/8418
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8716

LU-4176 tests: do not wait for deleted HSM request record

restore_and_check_size() function of sanity-hsm was systematically
waiting for restore request status, even if already flagged as
succeed. This extra wait can become forever if request record has
already been deleted.
Also fix wrong restore timing msg.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I9709d9e665b66fb5d16e93c7139fa8d95fa92589
Reviewed-on: http://review.whamcloud.com/8154
Tested-by: Jenkins
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8720

LU-3889 osc: Allow lock to be canceled at ENQ time

A cl_lock can be canceled when it's in CLS_ENQUEUED state.
We can't unuse this kind of lock in lov_lock_unuse() because
it will bring this lock into CLS_NEW state and then confuse
osc_lock_upcall().

Add a regression test case by Alexander Boyko.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: Ieb9209203a363c9098b9ea4637a9d12e12ef1aa4
Reviewed-on: http://review.whamcloud.com/8717
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-14 tests: add version check code to conf-sanity test 69

This patch adds Lustre version check codes into conf-sanity
test 69 to make the test work with servers which do not have
the following patch:

Lustre-commit: db6613f5bed1606cc8f97b46d1b298746af03a75
Lustre-change: http://review.whamcloud.com/7443

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=69 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=45 mdsbuildno=45 \
testlist=conf-sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I4e736fd5d760872f88364f30a39430f961419c64
Reviewed-on: http://review.whamcloud.com/8411
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-4204 tests: fix test_69 in conf-sanity

Recent commit added new test 69 to conf-sanity,
but calls test_68 by mistake. This patch fixes that typo.
In addition the test in its original form could never have worked.
There were several functional problems in it.
This patch fixes these up too.

Test-Parmeters: testlist=conf-sanity envdefinitions=SLOW=yes,ONLY=69

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I44d1e5f620a3e4cc14b59842abdd2f56ed409872
Reviewed-on: http://review.whamcloud.com/8158
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-4379 procfs: dont always check max_pages_per_rpc alignement

If connection is not established yet, cli->cl_chunkbits is
uninitilized so we cannot use it to check max_pages_per_rpc
alignment.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I4c782e2eeb5a2757d79194ee6ab6bff831a283fc
Reviewed-on: http://review.whamcloud.com/8558
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>

LU-4365 quota: wait for global lock cancel

In qsd_qtype_fini(), we'd wait for the global lock cancel done.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes  mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs  testlist=recovery-small
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib51dba4e4e4b13693c49560162d53cb28937e253
Reviewed-on: http://review.whamcloud.com/8583
Tested-by: Jenkins
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4362 libcfs: wrong parameter to cfs_hash_keycpy

cfs_hash_rehash_key() passed wrong parameter to cfs_hash_keycpy,
hnode should be the second parameter not the third one.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I44f73e4d2ffdeab5b021d49642598e939ebacee5
Reviewed-on: http://review.whamcloud.com/8509
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4313 osp: continue waiting for -ENOTCONN

in osp_precreate_reserve, -ENOTCONN should also be treated
as recoverable errors, such as -ENODEV, and continue waiting
the object to be created by OSP once it completes recovery.

Change-Id: I39b600eaacdb68f54c6ad7ecb417532bc0ad5e7b
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/8415
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4310 kernel: missed unlock in quota patch

The quota-replace-dqptr-sem.patch introduced a regression which
could miss unlock inode->i_lock in __dquot_alloc_space().

Change-Id: If0f240fe76d661909c13b11e3d384e77da3ebbe5
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: http://review.whamcloud.com/8438
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-4310 kernel: missed unlock in quota patch

The quota-replace-dqptr-sem.patch introduced a regression which
could miss unlock inode->i_lock in __dquot_alloc_space().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: If0f240fe76d661909c13b11e3d384e77da1662b1
Reviewed-on: http://review.whamcloud.com/8400
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Jenkins
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4243 mgs: use proper uuid in mgs_write_log_failnids()

This patch revert the changes to mgs_write_log_failnids() in the
commit d9d27cad, which introduced a regression of that when multiple
'--servicenode' or '--failnode' option specified, the uuid from
first option will be wrongly used for all other connections.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ibd989de67fe6851cc75569be3f64dadaec5e6f8c
Reviewed-on: http://review.whamcloud.com/8372
Tested-by: Jenkins
Reviewed-by: Erich Focht <efocht@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4219 mdd: limit os_namelen to the max of NAME_MAX

While running pathconf(".", _PC_NAME_MAX) on ldiskfs-based
Lustre filesystem, it returns 255 (NAME_MAX). However, on
zfs-based Lustre filesystem, it returns 256 (MAXNAMELEN).
Since Lustre can not handle the name length large than
NAME_MAX, this patch limits the value of os_namelen in
mdd_statfs() to the max of NAME_MAX.

Test-Parameters: envdefinitions=SLOW=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
ostcount=2 testlist=posix

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I13b5398eff5b15c74b0e08e2a3394b79a3ff6a44
Reviewed-on: http://review.whamcloud.com/8217
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4179 mdt: skip open lock enqueue during resent

Skip open lock enqueue, if the open lock has been
acquired(mdt_intent_fixup_resent) during resent.

Signed-off-by: wang di <di.wang@intel.com>
Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I78228112b0ba8f93e05930fdd042763d77700626
Reviewed-on: http://review.whamcloud.com/8142
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4139 quota: improve write performance when over softlimit

To improve the write performance when over softlimit, we choose
to lose some grace time accuracy: shrinking the qunit size to
(4 * least_qunit_size) instead of least_qunit_size.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I20e341d53979ac42227197ec6fa04dd38974073f
Reviewed-on: http://review.whamcloud.com/8078
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-4136 obdclass: use obd_get_info to report network status

Use obd_get_info() to report MDT-OST connection status instead of
health_check(), so that /proc/fs/lustre/health_check does not
check MDT's connection status.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I8b4afcfdce3712fff893475b9c5cd44654f3762e
Reviewed-on: http://review.whamcloud.com/8408
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4123 tests: use nodename in check_write_access

Check for a file create using the remote nodename.

When the node name of a host does not exactly match the host name
passed via xxx_HOST parameters, check_write_access can check for
creation of a filename different from what the remote node created.

By querying and using the nodename from the remote host, the node
initiating the tests can check for the same file name that the remote
node creates.

Test-Parameters: testlist=lfsck

Signed-off-by: Stephen Champion <schamp@sgi.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I57f081597395578c1a37e16531477e6fa4867c8d
Reviewed-on: http://review.whamcloud.com/8009
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4093 tests: fix typo and inverted test in original patch

1st/original patch for LU-4093 ticket, patch-set #3 of Change at
http://review.whamcloud.com/8157/ with Id
I3274e5864469251e6f2040377b84429e4f218f47, contains a typo and
an inverted test that prevent it to work as expected but rather
with reversed logic.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I3f8031b85db116711312b8f7decf2c33761b89e6
Reviewed-on: http://review.whamcloud.com/8329
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4093 tests: prevent zombie requests when stopping CT

Fix to prevent any current requests to become zombie when stopping
copytool and thus not to sometimes to induce failures during next
sanity-hsm sub-tests.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I3274e5864469251e6f2040377b84429e4f218f47
Reviewed-on: http://review.whamcloud.com/8157
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4067 osd: apply PTR_ERR() to right pointer

In ofd_precreate_objects(), osd_index_ea_insert(), and
osd_object_create() ensure that PTR_ERR() is applied to the right
pointer. Also convert some explicit casts of ERR_PTRs to ERR_CAST()
calls.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I846b10543ae426652ad72c277d2c0f0bc259fd6e
Reviewed-on: http://review.whamcloud.com/7887
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4058 tests: skip sanity-scrub test_15 for 2.4.1 or older

Lustre-2.4.1 or older does not support dry-run mode OI scrub yet.

Test-Parameters: mdsjob=lustre-b2_4 ossjob=lustre-b2_4 \
mdsbuildno=54 ossbuildno=54 testlist=sanity-scrub
Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I1a97f850ab39d7d42ef7ed6e191ce66b3b093415
Reviewed-on: http://review.whamcloud.com/7868
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4016 tests: use user and group names in metadata-updates

pdsh eats %u passed to stat executed on remote nodes.  Avoid this
problem by using the user and group names instead of numerical
identifiers in metadata-updates ownership checks.

Test-Parameters: testlist=metadata-updates
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Stephen Champion <schamp@sgi.com>
Change-Id: Id0a283d58873bb64772f28cbc28c5a6abad9a48e
Reviewed-on: http://review.whamcloud.com/8052
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4015 ldiskfs: access external journal using device path

Currently ldiskfs when configured with an external journal
uses the device number to access it. The problem with this
approach is that the device number can change across
reboots. This patch adds a new mount option -o journal_path
to tell ldiskfs where the external journal is instead.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I43fff89c2bebeff4f0305cdc24cefa534fb9fe67
Reviewed-on: http://review.whamcloud.com/7777
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-3906 tests: use df instead of lfs_df in run_compilebench()

Since run_compilebench() will be run on NFS client which has no
Lustre filesystem, we need change to use df instead of lfs_df to
get the space usage information.

Test-Parameters: envdefinitions=SLOW=yes,ONLY=compilebench \
testlist=parallel-scale,parallel-scale-nfsv3,parallel-scale-nfsv4

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ia1a9586586b888231952830180157a9e37f08843
Reviewed-on: http://review.whamcloud.com/8429
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3906 tests: fix space estimation codes in run_compilebench()

While running compilebench test, the actual space required for each
kernel directory is about 1GB instead of 680MB. This patch fixes this
number in run_compilebench() so as to make the space estimation
more correct.

Test-Parameters: envdefinitions=SLOW=yes,ONLY=compilebench \
testlist=parallel-scale,parallel-scale-nfsv3,parallel-scale-nfsv4

Test-Parameters: envdefinitions=SLOW=yes,ONLY=compilebench \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
ostcount=2 mdtcount=1 \
testlist=parallel-scale,parallel-scale-nfsv3,parallel-scale-nfsv4

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I8aa14673b8df99f6b50980ee091acda76da7cd9a
Reviewed-on: http://review.whamcloud.com/8258
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3879 test: Check for e2fsprogs on server

sanity-quota checks if e2fsprogs is loaded on the client,
but should be checking for e2fsprogs on the server. This
patch changes checking for dumpe2fs from the client to the MDS.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I83b0b5b99deec376f4ee2d59e30810373e330174
Reviewed-on: http://review.whamcloud.com/8346
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>