LU-4724 hsm: Safe copytool event logging

Protect against concurrent event log writes by multiple
threads within a copytool process. Fixes sanity-hsm test_71
failures.

Lustre-commit: f28e754dcbf36e159386038bdebfaef50135715f
Lustre-change: http://review.whamcloud.com/#/c/9553/

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Ic83b162c60074bde7235bb9ae44f8f28797c1dfd
Reviewed-on: http://review.whamcloud.com/9568
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.5.1-RC2

Change-Id: Ie7c87fe160c2cdd0bc4ca225ea07dc4cbdb9b7e2

LU-4703 mdd: do not skip xattr sanity check for all cases

xattr sanity check should be done at all times.
Otherwise we are risking letting a non root user setting
access acl on any file.

Lustre-commit: 6d7066989c06bdf6c86f5b8b61d542fda8e5739e
Lustre-change: http://review.whamcloud.com/9469

Signed-off-by: Li Dongyang <dongyang.li@anu.edu.au>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia669a20139abc5f4b64688cccd4b07a5bcc6910b
Reviewed-on: http://review.whamcloud.com/9513
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4020 hsm: allow copytool event monitoring with JSON

Adds hooks into various llapi_hsm_* functions to emit JSON-formatted
event messages for consumption by a monitoring agent. The copytool
needs to be supplied with an optional --event-fifo argument to
enable this feature.

Incorporates the following work done by Bruno Faccini:
* Put all JSON routines in a separate file/lib to allow for
  LGPL licensing.
* Major code cleanup to follow best practices and address review
  concerns.

Lustre-commit: 45c9ef1dfff207086665c764c7d500c00aa03c7f
Lustre-change: http://review.whamcloud.com/7790

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Icfad0f2093f24fcfec22408d9819d438c6a5e7f1
Reviewed-on: http://review.whamcloud.com/9512
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4704 acl: fix permission problem of setfacl

Setxattr does not check the permission when setting ACL xattrs. This
will cause security problem because any user can walk around
permission checking by changing ACL rules.

Lustre-commit: 42f504ecada81d7a2a8e2244f345e8dbf73fd157
Lustre-change: http://review.whamcloud.com/9473

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If6bda8ec6b3d91f31bef812acb9f9f636ed56291
Reviewed-on: http://review.whamcloud.com/9514
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4083 mdt: Take lov_mutex in mdt_reint_unlink and _rename

Take the mot_lov_mutex lock around mdo_unlink and mdo_rename
to synchronize around striping.

Test-Parameters: testlist=racer
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Iaa00b81fd7cfb25ce5f3dcea3c2d6289d133134f
Reviewed-on: http://review.whamcloud.com/7919
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.5.1-RC1

Change-Id: I9a98cdff4049373d4931e234c032c7e577739319

LU-4101 mdt: protect internal xattrs

In mdt_reint_setxattr() require CAP_SYS_ADMIN to modify a trusted
xattr and silently disallow modification those trusted xattrs used by
Lustre internally.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic616dca74a90da0aedb0ec2624618f91ac6fcaf4
Reviewed-on: http://review.whamcloud.com/7943
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4639 hsm: HSM requests not delivered

The total size of an HSM archive request may exceed the
desired (LNET) message. When this happens, it can hang
the client and not allow the archive request to succeed.

Before we know the total size of the hsm_action_items, we
need to limit the size of the reguest. Doing this limits
the number of items that can be sent in one archive request.
We’ve reduced the size allowed for the user archive request
to MDS_MAXREQSIZE/3.

sanity-hsm test 90 forms a list of files to archive. The
number of files in the list needed to be decreased to
match the limit described above.

Lustre-change: http://review.whamcloud.com/#/c/9393/
Lustre-commit: a23e0fe42f26cd54384058d927bcf42330174e7b

Change-Id: I84c36ba318a6ed424248a0567c33e824de3e8021
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Reviewed-on: http://review.whamcloud.com/9422
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4470 build: wrong linux symbol file search

Long standing build flaw just discovered.  The autoconf function
LB_CHECK_SYMBOL_EXPORT looks for the linux symbol table in the wrong place.
In most builds this doesn't matter as the wrong path being used exactly
matches the correct path.  In SLES builds it does matter a lot.
Failing to find the linux symbol table can lead to incorrect autoconf results.

Change-Id: I9068f2b9aace740b67d88598844c0b8fc1d95b49
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: http://review.whamcloud.com/9288
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4357 libcfs: restore __GFP_WAIT flag to memalloc calls

In 2.4, the flags passed to the memory allocation functions are
translated from CFS enumeration values types to the kernel GFP
values by calling cfs_alloc_flags_to_gfp(). This function adds
__GFP_WAIT to all flags except CFS_ALLOC_ATOMIC. In 2.5, when
the cfs wrappers were dropped, cfs_alloc_flags_to_gfp() was
removed and the CFS_ALLOC_xxxx was simply replaced with __GFP_xxxx.
This means that most memory allocation calls are missing the
__GFP_WAIT flag. The result is that Lustre experiences more ENOMEM
errors, many of which the higher levels of Lustre do not handle
robustly.
Notes GFP_NOFS = __GFP_WAIT | __GFP_IO. So the patch replaces
__GFP_IO with GFP_NOFS.
Patch does not add __GFP_WAIT to GFP_IOFS. GFP_IOFS was not used in
2.4 so it has never been used with __GFP_WAIT.

Lustre-commit: e6b3d5de62b43a8702aa2dbda4a0bc6b6b10b5c2
Lustre-change: http://review.whamcloud.com/9223

Signed-off-by: Ann Koehler <amk@cray.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I8f0341c34e595d437c06a4564a11e4c52bd4363a
Reviewed-on: http://review.whamcloud.com/9382
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4471 mdd: mdd_unlink: do trans_start after sanity check

Currently, mdd_trans_start is called before
mdd_unlink_sanity_check. This means a remote directory
which has files in it can be removed on MDT0 before the
sanity check on MDT1 finds the files and errors, which
orphans the files on MDT1. This patch moves the sanity
check before mdd_trans_create and mdd_trans_start.

Lustre-commit: 754a8ac8bf4b907bb579fe441eed1ba3447d3e0f
Lustre-change: http://review.whamcloud.com/8827

Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iac3cb214f16cb8e342be6691013d1cf262775ff0
Reviewed-on: http://review.whamcloud.com/9379
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4597 clio: clear nowait flag agl lock re-enqueue

The LDLM_FL_BLOCK_NOWAIT flag should be cleared when re-enqueue
the agl lock as normal glimpse, otherwise, it won't get size back
if there is conflicting locks on other client.

Lustre-change: http://review.whamcloud.com/9249
Lustre-commit: 85c352274b3435a41649e9a5089da83b54893d37

Change-Id: I421d033496b57fb3d24635587112eaab3ba2ea32
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-on: http://review.whamcloud.com/9328
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4382 ldiskfs: add quota credit for ldiskfs_delete_inode

In ldiskfs_delete_inode() we missed possible journal credits
for journaled quota change, this patch makes it up.

Lustre-commit: c6041da30a189e2bd2f2dd85b6032cb9b99a769d
Lustre-change: http://review.whamcloud.com/9187

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I29d5e2bfa471a08a01b4c316e447cf2fa3df6b82
Reviewed-on: http://review.whamcloud.com/9334
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4460 mount: fix lmd_parse() to handle comma-separated NIDs

This patch reverts commit 3917e62018878dfffac59ceed70f20b0419945d3,
which cannot handle the upgrade situation that old mountdata already
contains comma-separated NIDs. The correct way to fix the original
issue is to parse comma-separated NIDs in lmd_parse().

The patch also updates disk2_4-ldiskfs.tar.bz2 to make the mountdata
of ost contain comma-separated NIDs so as to verify the patch under
upgrade situation.

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes testlist=conf-sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I58d6f4f1b74c78fc2198652677571bfd4a57d785
Reviewed-on: http://review.whamcloud.com/9029
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4269 ldlm: Hold lock when clearing flag

This patch moves lock's skip flag clearing from lru-delete to
lru-add code to prevent clearing lock's flag without resource lock
proection.

Lustre-commit: 98c2e6b446166ba8f89e60a0d4f38683b920f506
Lustre-change: http://review.whamcloud.com/8772

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I6f449ba0e03b936435ffa8a1b158a4987c636862
Reviewed-on: http://review.whamcloud.com/9346
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>

LU-4098 lmv: kernel crash due to misconfigured MDT

There are few places with access to lmv->tgts[] without check for NULL.
Usually it may happens when MDT configured starting from index 1
instead of 0. For example:
  mkfs.lustre --reformat --mgs --mdt --index=1 /dev/sdd1

Lustre-commit: caa70178a413c84c400e875866cdbbd368df4758
Lustre-change: http://review.whamcloud.com/7941

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Ieccedcfe7a3a18cb649ef739a077338a8a1b8855
Reviewed-on: http://review.whamcloud.com/9347
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4343 tests: mkdir failing in sanity-hsm test 228

sanity-hsm test 228 calls mkdir on $tdir. Currently, the tdir
variable is two directories. This is changed in LU-2524. Until
LU-2524 lands, any call to mkdir with the tdir variable needs
the "-p" flag.

Also added removal of two files that the test creates and a new
routine to create small files with dd using the sync flag.

Lustre-commit: 2f253abab679c21b41197379b23e36943e5995a7
Lustre-change: http://review.whamcloud.com/8542

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Idd4354d6012032563d41c10238f619251e885e65
Reviewed-on: http://review.whamcloud.com/9280
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4505 quota: race of edquot updating

The slave edquot flag could be set mistakenly as following:

- slave A acquires quota from master, master found that the
  user is running out of quota, set edquot in reply;
- another slave deletes files and release quota to master,
  master clears edquot and notify all slaves by glimpse;
- glimpse reaches slave A before the reply of dqacq, so
  edquot flag will be set on slave A at the end.

Given that edquot can't be fully trusted, it should only be
revalidated every 5 seconds on the sync acquire path.

Lustre-commit: 109fef5b053490549726f7b5abc9ba840d3a4ae0
Lustre-change: http://review.whamcloud.com/8954

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I553bd1bc3aa6df6c449341e56564073043afd3da
Reviewed-on: http://review.whamcloud.com/9315
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4577 lnet: Dropped messages are not accounted correctly

LNET messages that are dropped are not accounted for correctly in
/proc/sys/lnet/stats. What I assume to be a simple typo is causing
drop_length to be double-counted and drop_count to never be
incremented.

Lustre-change: http://review.whamcloud.com/9096
Lustre-commit: 3abb0bb5f82559f2f5349dca763cf6edc7f6754b

Change-Id: Ia8454221885a1d765a3f7fadbcf5582fcfe7cf09
Signed-off-by: Matt Ezell <ezellma@ornl.gov>
Reviewed-on: http://review.whamcloud.com/9311
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4620 kernel: kernel update [RHEL6.5 2.6.32-431.5.1.el6]

update RHEL6.5 kernel to 2.6.32-431.5.1.el6

Lustre-commit: 6fa2299177e8749e63a88047aee49b5a9af6c3ef
Lustre-change: http://review.whamcloud.com/9253

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I54c6476c84650d7419d70fb89efc2680af6ecabe
Reviewed-on: http://review.whamcloud.com/9318
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2687 test: add b2_4 zfs image for conf-sanity test_32a

In order to ensure that we do not break ZFS upgrades
in the future, add 2.4.0 zfs filesystem test image for
conf-sanity.sh test_32a.

Test-Parameters: mdtfilesystemtype=zfs \
ostfilesystemtype=zfs mdsfilesystemtype=zfs \
envdefinitions=SLOW=yes testlist=conf-sanity

Change-Id: Iae560e05b428907409dc7069d30b601b52750cca
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/8824
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4484 lbuild: add support for fresh versions of MPSS 3.x.x

* Adopt lbuild script for new version of MPSS with x.x.x notation.
* Remove dependency from MPSS package to avoid renaming issue in
  the future. The name of package which was used for dependency
  was renamed in MPSS.
* Use new server with MPSS released packages for download.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie4407ad00177ad6d22770230a4dc6bde967d91ef
Reviewed-on: http://review.whamcloud.com/8836
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3968 lbuild: Extend script with build for Xeon Phi card

Automatically download, compile and produce Lustre client RPMs
for Xeon Phi(TM) card if "--mpss-version" option is specified
for contrib/lbuild/lbuild script.

Also try to compile with Xeon Phi(TM) OFED if it's available.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ida07d764dc824c13f22ffb53d24e2c6f79ce3573
Reviewed-on: http://review.whamcloud.com/7066
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4613 tests: purge older request result in test_12o

sanity-hsm/test_12o sub-test, which has been introduced as part
of LU-3834, submits 2 RESTORE requests for the same FID and thus
needs to purge 1st result from log before to check 2nd.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ia2a0ead487b29a68c8a920bae2aa1d654eac4051
Reviewed-on: http://review.whamcloud.com/9295
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4442 test: add version check for replay-vbr.sh test_7g

In replay-vbr.sh test_7g.3, because mdt_object_exists() was added
in http://review.whamcloud.com/#/c/8371, client will not be evicted
without object version check.

The patch also fixes the wrong usage of wait_mds_ost_sync() in
replay_vbr.sh test_7_cycle(). The first parameter should be a timeout
in seconds, not a facet.

Test-Parameters: envdefinitions=SLOW=yes,ONLY=7 testlist=replay-vbr

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I4a960fc53451fc717370bc96f926f067bbb2946a
Reviewed-on: http://review.whamcloud.com/9290
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4386 osc: don't activate deactivated obd_import

In ptlrpc_activate_import(), obd_import->imp_deactive should
be checked if it is deactivated, otherwise it will trigger an
LBUG in ptlrpc_invalidate_import():

  ptlrpc_invalidate_import() ASSERTION(imp->imp_invalid) failed

Change-Id: I4c16f166c0c2cf60664119bf438dfd8606d71a2f
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/9283
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4293 mdd: Allow layout swap for IGIF FIDs

Patch to also allow layout swap for pre-2.x migrated
files (ie, IGIF FID with linkEA).

Root user special case has also been added to lfs/migrate
command to map owner/group of original file to
volatile, in order to comply with other layout_swap rules.

Lustre-commit: bd5ba50502bec5786c9a2f05c29f7b99a35147fb
Lustre-change: http://review.whamcloud.com/8737

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iad6194c6050fa2ba066d2051871a10a60ddae995
Reviewed-on: http://review.whamcloud.com/9278
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4208 osd-zfs: hold pool config lock to register property

- Hold the DSL pool configuration lock when calling
  dsl_prop_register().  Failure to do so will panic the node if
  assertions are enabled in ZFS.  This change requires a build of ZFS
  on Linux that exports symbols dsl_pool_config_enter and
  dsl_pool_config_exit, which was done in commit 40a806d [1], and will
  appear in ZFS release 0.6.3.

- Fix up variable declaration alignment in osd_mount().

- Add check for exported symbols in autoconf

[1] https://github.com/zfsonlinux/zfs/commit/40a806d

Lustre-change: http://review.whamcloud.com/8172
Lustre-commit: f8bc2f7fc03d3f86eef434cf644191e689ee57ec

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Ice673efb5501456d1a4f423ec08dfb4f571f8221
Reviewed-on: http://review.whamcloud.com/9256
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4429 llite: fix open lock matching in ll_md_blocking_ast()

In ll_md_blocking_ast() match open locks before all others, ensuring
that MDS_INODELOCK_OPEN is not cleared from bits by another open lock
with a different mode. Change the int flags parameter of
ll_md_real_close() to fmode_t fmode. Clean up verious style issues in
both functions.

Lustre-commit: 2b23ad0d183141dc25377f2d37de6e6e36ba1169
Lustre-change: http://review.whamcloud.com/8718

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Ic44ac8ac8c07b71d4c929d7d359bee881c6b05b0
Reviewed-on: http://review.whamcloud.com/9260
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4287 kernel: kernel update RHEL6.5 [2.6.32-431.3.1.el6]

Add RHEL6.5 support [2.6.32-431.3.1.el6]

ext4 in RHEL6.5's kernel version 2.6.32-431.3.1.el6 no longer contains
the required function ext4_ext_walk_space(). We start a new rhel6.5
ldiskfs patch series and reintroduce ext4_ext_walk_space() through an
new patch, copying ext4_ext_walk_space() from older kernel rhel6.4
2.6.32-358.23.2.el6.

Lustre-commit: efa8fa578d2f7eeeaea11522dd311dddaa715a03
Lustre-change: http://review.whamcloud.com/8549

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I5cff1860c43d06a6399b43f92ef90283c4600c8e
Reviewed-on: http://review.whamcloud.com/9103
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4154 lfsck: skip old lfsck test in DNE mode

The old e2fsck/lfsck tool will not be allowed to run on a DNE
filesystem. This patch updates generate_db() to pass master MDS
parameters only, so that the old lfsck does not corrupt it or
delete all of the files on other MDTs.
This patch also fixes a typo in run_lfsck_remote().

This patch is back-ported from the following one:
Lustre-commit: b5f3d6db9200e369a68284a8ef85a1205e5905e1
Lustre-change: http://review.whamcloud.com/8206

Test-Parameters: alwaysuploadlogs mdtcount=4 testlist=lfsck
Test-Parameters: alwaysuploadlogs mdtcount=1 testlist=lfsck

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I5590b20c1c0003dbb1975a254093724de22497d4
Reviewed-on: http://review.whamcloud.com/9176
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-946 lprocfs: List open files in filesystem

Added lprocfs file on MDT to list open files in per-export
directory for mdt.

Test-Parameters: testlist=sanity,sanityn
Signed-off-by: Girish Shilamkar <gshilamkar@ddn.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: If8f233d95dca4cd4c4044d85bd117a027dabd80e
Reviewed-on: http://review.whamcloud.com/6386
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Swapnil Pimpale <spimpale@ddn.com>

LU-3528 mdt: check object exists for remote directory

Check whether the remote object exists before enqueue and
getattr to avoid LBUG.

Remove unnecssary remote object exist check in mdd_object_lock.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia634a8c7b9cd2810515e854163c5fdd6bdf8716f
Reviewed-on: http://review.whamcloud.com/8371
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4430 mdt: check for MDS_FMODE_EXEC in mdt_mfd_open()

In the error path of mdt_mfd_open() check for MDS_FMODE_EXEC rather
than FMODE_EXEC in the open flags.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I04c53eb1af0fdeeb2c2b0c2f2ef1340b247921d8
Reviewed-on: http://review.whamcloud.com/8719
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4260 lod: free striping if striping initialization fails

It should free striping if striping information initialization is
faild, otherwise the later object find will pick up this wrong lod
object, and hit LBUG

ASSERTION( lc->ldo_stripenr == 0 ) failed:

[<ffffffffa0349895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
[<ffffffffa0349e97>] lbug_with_loc+0x47/0xb0 [libcfs]
[<ffffffffa0e3f78f>] lod_ah_init+0x57f/0x5c0 [lod]
[<ffffffffa0b73a83>] mdd_object_make_hint+0x83/0xa0 [mdd]
[<ffffffffa0b7feb2>] mdd_create_data+0x332/0x7d0 [mdd]
[<ffffffffa0d9cc2c>] mdt_finish_open+0x125c/0x18a0 [mdt]
[<ffffffffa0d984f8>] ? mdt_object_open_lock+0x1c8/0x510 [mdt]
[<ffffffffa0d9ee8d>] mdt_reint_open+0x115d/0x20c0 [mdt]

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I67b2bd0e013b860767d19eda986fdcff7e16c486
Reviewed-on: http://review.whamcloud.com/8324
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3772 ptlrpc: fix nrs cleanup

When service start failed due to short of memory, the cleanup code
could operate on uninitialized structure and cause crash at the end.

This patch fix the nrs_svcpt_cleanup_locked() to perform cleanup only
on the nrs which has been properly initialized.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ieafa5b144133490b662f5a80a7b99311a9970de3
Reviewed-on: http://review.whamcloud.com/7410
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3857 osd: cleanup procfs after osd_shutdown

Since osd_procfs_fini will try cleanup all proc entries,
and osd_shutdown/qsd_fini will try to cleanup procfs
itself, so osd_procfs_fini should be done after qsd_fini,
otherwise the qsd entries will be destoryed twice, and
caused panic

Call Trace:
 [<ffffffffa081cc45>] lprocfs_remove+0x25/0x40 [obdclass]
 [<ffffffffa0b23dd0>] qsd_fini+0x80/0x450 [lquota]
 [<ffffffffa0d2ec78>] osd_shutdown+0x38/0xe0 [osd_ldiskfs]
 [<ffffffffa0d36bf9>] osd_device_fini+0x129/0x190 [osd_ldiskfs]
 [<ffffffffa0834913>] class_cleanup+0x573/0xd30 [obdclass]
 [<ffffffffa081233c>] ? class_name2dev+0x7c/0xe0 [obdclass]
 [<ffffffffa083663a>] class_process_config+0x156a/0x1ad0 [obdclass]
 [<ffffffffa06be9b8>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffffa082f202>] ? lustre_cfg_new+0x312/0x6e0 [obdclass]
 [<ffffffffa0836d19>] class_manual_cleanup+0x179/0x6e0 [obdclass]
 [<ffffffffa06be9b8>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffffa0d378b4>] osd_obd_disconnect+0x174/0x1e0 [osd_ldiskfs]
 [<ffffffffa083926b>] lustre_put_lsi+0x1ab/0xeb0 [obdclass]
 [<ffffffffa08414d8>] lustre_common_put_super+0x5c8/0xbe0 [obdclass]
 [<ffffffffa087081d>] server_put_super+0x1bd/0xed0 [obdclass]
 [<ffffffffa0871bbb>] server_fill_super+0x68b/0x1630 [obdclass]
 [<ffffffffa0840bb8>] lustre_fill_super+0x1d8/0x530 [obdclass]
 [<ffffffffa08409e0>] ? lustre_fill_super+0x0/0x530 [obdclass]

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: If12cebf971583afeeaa031bd24f69bb0fe0cdf1a
Reviewed-on: http://review.whamcloud.com/8506
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2818 mdt: Properly handle ENOMEM

When osd_keys_init fails in mdt_lvbo_fill, properly bail out with
error instead of asserting.

Change-Id: I832742ed49cc7740d8e709bc4b87e5d5aa100d39
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8947
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3618 ptlrpc: rq_commit_cb is called for twice

If a ptlrpc_request is already on imp::imp_replay_list, when it's
replayed and replied, after_reply() will call req::rq_commit_cb
for the request, then call it again in ptlrpc_free_committed.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I796c3351ad896aa3e1d0c2147ca7f775b7c14bfc
Reviewed-on: http://review.whamcloud.com/8815
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4454 libcfs: warn if all HTs in a core are gone

libcfs cpu partition can't support CPU hotplug, but it is safe
when plug-in new CPU or enabling/disabling hyper-threading.
It has potential risk only if plug-out CPU because it may break CPU
affinity of Lustre threads.

Current libcfs will print warning for all CPU notification, this
patch changed this behavior and only output warning when we lost all
HTs in a CPU core which may have broken affinity of Lustre threads.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I62267b62871c129beeb1593c4f69e7b81a79999d
Reviewed-on: http://review.whamcloud.com/8770
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3601 Do not create layout in lease-open

leases are not real opens so it makes no sense to create layouts
when the lease is taken.

Change-Id: Ica2d6a348c360bd20bb7bd27061839df84dae84b
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/8084
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4152 mdt: Don't enqueue two locks on the same resource

Due to mechanics of ldlm internals, enqueueing two different ibits
lock on the same resource is deadlock prone.
As such change mdt_object_open_lock to release open lock if it becomes
necessary to get exclusive layout lock (to create objects).
It's ok to release the open lock right away as it's never guaranteed to
be issued anyway.

Change-Id: Ib669e68323ea72c75a0a8bea289d8bea079309b0
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8083
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3834 mdt: handle swap_layouts failures during restore

Actually nothing occur after swap_layouts failures during restore,
this can lead to file being left in incoherent state and thus be
unavailable because HS_RELEASED is clear but LOV_PATTERN_F_RELEASED
is still set.
This patch will allow original layout to be recovered by the use of
SWAP_LAYOUTS_MDS_HSM flag. Additionaly this requires HSM xattr of
the data FID to be set.
Also adds layout-swap failure injection and related test.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Id0e9a005362e4a3854b33f6ce1888197d20e7dbf
Reviewed-on: http://review.whamcloud.com/7631
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4336 quota: improper assert in osc_quota_chkdq()

In osc_quota_chkdq(), we should never try to access oqi found
from hash, since it could have been freed by osc_quota_setdq().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ia73cf89cb5bbd730fa6f0a00e44771f733b2baa6
Reviewed-on: http://review.whamcloud.com/8460
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-4253 osc: Don't flush active extents.

The extent is active so we need to abort and let the caller
re-dirty the page. If we continued on here, and we were the
one making the extent active, we could deadlock waiting for
the page writeback to clear but it won't because the extent
is active and won't be written out.

Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: Iba646d8185b12ab227fe0bbee1c6602ccdc32ad6
Reviewed-on: http://review.whamcloud.com/8278
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3373 misc: small changes for 3.10 server support

--quota use struct kqid as parameter
--export ext4_dec/inc_count for nlink count
--ext4_find_entry & ext4_journal_start_sb changes
--iop->truncate removed
--other trival changes to calm compiler warning

Lustre-commit: 9bd7e40d2934cd0162eeff5388f054444a982ac9
Lustre-change: http://review.whamcloud.com/7794

Signed-off-by: yang sheng <yang.sheng@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia2670d925ecbbfcc1ed3abb1a15a8d91fa27bd32
Reviewed-on: http://review.whamcloud.com/9013
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4106 scrub: Trigger OI scrub properly

There is the following race case between osd_fid_lookup() and object
unlink/detroy:

Both RPC service thread_1 and RPC service thread_2 try to find the
same obj_A at the same time. At the beginning, the obj_A is not in
cache. The thread_1 is in osd_fid_lookup() and finds the OI mapping
for obj_A. But before the thread_1 finding out related inode_A, the
thread_2 moves faster and finds the inode_A and unlinks the inode_A.
So the thread_1 will fail to find the inode_A. Under such case, the
thread_1 will try to check OI again to make sure whether related OI
mapping is still there or not. If no OI mapping, then it is normal
becuase someone has unlinked the file by race; otherwise, it may be
caused by file-level backup/restore, then thread_1 will trigger OI
scrub to rebuild OI files.

But we ignored a corner case that the thread_1 recheck the OI files
may just between the thread_2 has dropped the inode_A's referene to
zero and will remove related OI mapping from the OI file. Then the
thread_1 is misguided, and will trigger OI scrbu unexpectedly.

More initial OI scrub for the /ROOT/.lustre directory to make sure
the necessary files/directories for mount are ready before used.

This patch also enhances the ls_locate()/dt_locate_at() interface
to allow the caller to pass some hints to low layer, such as flag
LOC_F_NEW for create, to help the low layer to handle efficiently
and properly.

This patch is back-ported from the following ones:
Lustre-commit: 8931d9070415e808e09bb4befd7cd38ef2431149
Lustre-change: http://review.whamcloud.com/8002
and
Lustre-commit: bab8a7dd5597014ee68e52bd39bde0ed40711777
Lustre-change: http://review.whamcloud.com/8101

Test-Parameters: mdtcount=4 testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5259549340f97a2f9118ab1db081f2ab2cfd8933
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/9006
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4178 tests: disable HSM sanity subtests

Turn off high failure rate subtests in sanity-hsm.
If fixes for these failures land tests may be turned on again later.

Lustre-commit: d85d724d9a3a503718f6df840be67e5f6f5af78c
Lustre-change: http://review.whamcloud.com/8122

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2e3f3822492a3398ebcbd2ba4565455986515764
Reviewed-on: http://review.whamcloud.com/9134
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4554 lfsck: old single-OI MDT always scrubbed

Old ldiskfs MDT's that contain a single OI container named "oi.16"
trigger an automatic OI scrub on each restart. This is because
osd_oi_table_open() gets ENOENT opening "oi.16.0" and consequently
sets bit 0 in scrub_file::sf_oi_bitmap. This bit indicates the OI
container 0 needs to be recreated, and it triggers a scrub in
osd_fid_lookup() for lookups that fail with ENOENT. Fix this by
clearing the bit in osd_oi_init() after a successful open of
"oi.16".

Lustre-change: http://review.whamcloud.com/#/c/9067
Lustre-commit: b4159f5d722bc43cff82b4c45336b01fd769e1db

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I3f19b15b51fce85bf791df76389f0b28951356c3
Reviewed-on: http://review.whamcloud.com/9139
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4512 hsm: Fix lhsmtool_posix --report option

The --report option is intended to allow an override of the
default copytool progress reporting interval, but it doesn't
work. This commit implements the intended functionality and
renames the option to "--update-progress", or "-u" for short.

Also fixes the progress display in hsm/active_requests to
reflect the change from percentage complete to bytes moved.

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Id6ead1b33868e3454f00053165944bc3900cabb4
Reviewed-on: http://review.whamcloud.com/8934
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4589 kernel: kernel update [SLES11 SP3 3.0.101-0.15]

update target and config files for new kernel version

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I28496e7dc9322dcf7fec4493602042cc89db6fec
Reviewed-on: http://review.whamcloud.com/9150
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4263 osd-zfs: Avoid converting last ID FIDs to OST IDs

When obdfilter-survey first creates an object on a fresh ZFS OST, the
last ID object for FID_SEQ_ECHO has to be created in the first place.
The last ID FID, [FID_SEQ_ECHO:0:0], can not be converted to an OST ID
because the resulting OST ID would be indistinguishable from an
FID_SEQ_OST_MDT0 OST ID and would confuse ostid_id().  This patch
checks for last ID FIDs before converting them to OST IDs in
osd_get_idx_for_ost_obj().

Change-Id: I96cdf85b4725e4882cecabaf90466c7f77a5e0a6
Intel-bug-id: FF-182
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/8301
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4293 utils: handle lfs migrate failure in lfs_migrate

If "lfs migrate" returns an error, possibly because it is refusing
to migrate an IGIF FID, fall back to using rsync to copy the file
and rename it.  Print a message in this case so the user knows it
is not a fatal error yet.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I114006afb93d8c8d78923a874f3b914200500c1e
Reviewed-on: http://review.whamcloud.com/8616
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3806 obdclass: add LCT_SERVER_SESSION for server session

Add LCT_SERVER_SESSION for server session, and separate the
server session flag from LCT_SESSION, so to avoid allocating
session info for client stack for each server request, if
client and server are on the same node.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I808c3f58cd7a03ebc166e51fe1e32ea34ae0e3e8
Reviewed-on: http://review.whamcloud.com/7412
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4194 ldlm: Make OBD_[ALLOC|FREE]_LARGE use consistent

struct ldlm_lock's l_lvb_data field is freed in ldlm_lock_put()
using OBD_FREE.  However, some other code paths can attach
a buffer to l_lvb_data that was allocated using OBD_ALLOC_LARGE.
This can lead to a kfree() of a vmalloc()ed buffer, which can
trigger a kernel Oops.

Change-Id: Ic75a67530862eeb4d065c14bbbac80939bff5731
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/8298
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4194 ldlm: set l_lvb_type coherent when layout is returned

In case layout has been packed into server reply when not
requested, lock l_lvb_type must be set accordingly.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Iaf54c9ba27785e529f4f2fb967d2fad4fc1dfbcb
Reviewed-on: http://review.whamcloud.com/8270
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-4196 build: Build support for OFED-3.5 and SLES 11

CONFIG_COMPATE_SLES_11_SP* needed in EXTRA_LNET_INCLUDE to allow
building against OFED-3.5

Lustre-commit: 369e02e84f39565195e08f043ab0421d2d3bd185
Lustre-change: http://review.whamcloud.com/8140

Test-Parameters: clientdistro=sles11sp3 ossdistro=sles11sp3 \
mdsdistro=sles11sp3 nettypes=o2ib clientibstack=inkernel \
ossibstack=inkernel mdsibstack=inkernel testlist=sanity

Signed-off-by: Chris Horn <hornc@cray.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib26c757044aff828c3bbbd3adfd5fb709cca9cf0
Reviewed-on: http://review.whamcloud.com/8882
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4124 build: make module installation directory flexible

Add --with-kmp-moddir option to configure.

Distributions vary in the installation directory for kernel modules.

The RHEL standard installation directory is
        /lib/modules/$(uname -r)/extra
while the SLES standard is
        /lib/modules/$(uname -r)/updates

Adding the option --with-kmp-moddir option to configure allows users
to select the appropriate installation target.  With this change, it
is neccessary to support both options in the test framework.

Signed-off-by: Stephen Champion <schamp@sgi.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/8065
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Iec3137e0e5039dd43622c2e285030a5339fa6fd3
Reviewed-on: http://review.whamcloud.com/8315
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>

LU-3679 lnet: reflect down routes in /proc/sys/lnet/routes

We consider routes "down" if the router is down or the router
NI for the target network is down. This should be reflected
in the output of /proc/sys/lnet/routes

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I82ee769d88aec92f1690ad9c095e32c9a9f9e282
Reviewed-on: http://review.whamcloud.com/7857
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8195
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-4231 llite: proper support of NFS anonymous dentries

NFS can ask to encode dentries that are not connected to the root.
The fix check for parent is NULL and encode a file handle accordingly.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Idba91fd4bca4f26a37fd9bc76a340d2fbf557c9e
Reviewed-on: http://review.whamcloud.com/8347
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8498

LU-4444 tests: Skip conf-sanity/69 on zfs

Because file creates happen slowly on ZFS and the number of files
required to run the test is 100K, this test cannot run in a
reasonable amount of time.

Also bail out of test if createmany fails (possible if MDS or OST is
too small), this prevents the test from just timing out instead.

This patch is back-ported from the following one:
Lustre-commit: eb38c458c868d5389e2641189218f22ad1272aef
Lustre-change: http://review.whamcloud.com/8841

Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity
Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity \
mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I6d9daad3239b576935190a121a2aa818441ec97b
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8854
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3189 tests: add version check code into sanity test 53

This patch adds Lustre version check codes into sanity test
53 to make the test work with servers that do not have the
following patch:

Lustre-commit: 6c4c51e3079e6c257fbf86536e4739110c166e3b
Lustre-change: http://review.whamcloud.com/4789

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=53 \
ossjob=lustre-b2_3 mdsjob=lustre-b2_3 ossbuildno=41 mdsbuildno=41 \
testlist=sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ie6eaeee31780f4ea4077805f52efda279ff96670
Reviewed-on: http://review.whamcloud.com/8834
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4222 mdt: extra checking for getattr RPC.

Check whether getattr RPC can hold layout MD(RMF_MDT_MD),
in case the client sends some invalid RPC, which can
cause panic on MDT.

Client will retrieve cl_max_md_size/cl_default_md_size
from MDS during mount process, so it will initialize
cl_max_md_size/cl_default_md_size before sending getattr
to MDS.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I43bbe54c37360242bb7a3cd2aa8d90c2b9e0baf1
Reviewed-on: http://review.whamcloud.com/8599
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4360 Fix use after free in ksocknal_send

Call to ksocknal_launch_packet might schedule a callback that
might free the just sent message, and so subsequent access to it
via lntmsg->msg_vmflush goes to freed memory.

Instead we'll just remember if we are in the vmflush thread and
only restore if we happened to set mempressure flag.

Change-Id: I2f0f8b27e26e11b37ad60fde4c98e86c39768349
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3680 ptlrpc: Fix assertion failure of null_alloc_rs()

lustre_get_emerg_rs() set the size of the reply buffer to zero
by mistake, which will cause LBUG in null_alloc_rs() when memory
pressure is high. This patch fix this problem and adds a size
check to avoid the problem of insufficient buffer size.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I9fbd4f14e8e1263de2af564c4f2e420f5f2b43bc
Reviewed-on: http://review.whamcloud.com/8200
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4221 osd: add case LCFG_PARAM to osd_process_config

Some proc parameters were moved from ofd to osd module and only
their symlinks were kept in ofd for interoperability/compatibility.
To process this kind of config params passed by ofd, this patch is
to do the following fixes:
- add case LCFG_PARAM to osd_process_config() to process parameters
  with prefix both PARAM_OSD and PARAM_OST.
- since these parameters are not included by the static lprocfs var
  list, a pre-check is added for them to avoid "unknown param" error
  message confuses the uses. If they are matched in this check, they
  will be passed to the osd directly.
- get rid of lprocfs_osd_init_vars() and use struct lprocfs_vars
  lprocfs_osd_{obd,module}_vars[] instead.
- improve the error messages in class_process_proc_param() and
  class_process_proc_seq_param() a little.
- add conf-sanity.sh test_28a to verify the patch and skip this test
  for ZFS OSTs since ZFS has no such kind of parameters.

This is a backport of commit b1491d26271f074dc6f99cde037403337c0b2151
in http://review.whamcloud.com/8238 .

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Signed-off-by: Michael MacDonald <mjmac@whamcloud.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I8b8d4244f90bd9e16acdccedd09da73fbb5e501b
Reviewed-on: http://review.whamcloud.com/8618
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Michael MacDonald <michael.macdonald@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4276 ldiskfs: enable read/write access by default

Add build time config option to allow read/write access by default.
While the new CONFIG_LDISKFS_FS_RW only matters in SLES11 builds,
it's easiest to just add the flag to all builds unconditionally.
It will be ignored in builds where it doesn't matter.

Lustre-commit: 14c94c20c3447584e81d720c2b2a17888716709e
Lustre-change: http://review.whamcloud.com/8335

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I4fcf0b2f884b1442db0aac5788bf62f07537c5d4
Reviewed-on: http://review.whamcloud.com/8779
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4030 tests: use free_fd() to allocate file descriptor

free_fd() lists /proc/self/fd to find the smallest unused file
descriptor
sanity test_31n is changed to use free_fd() instead of using hardcoded
173
sanity test_236 is changed to use free_fd() instead of using "{FD}<>"
which is not available on eariler bash

Since test_31n now uses the function free_fd to find an unused file
descriptor, it no longer depends on fd 173 being free.  This change
also removes that test on whether fd 173 is in use.

This patch is back-ported from the following ones:
Lustre-commit: 1f9235152b2f44c7bd64c5c021066f1984f341e6
Lustre-change: http://review.whamcloud.com/8181
and
Lustre-commit: 73e816e57167eb92425b6cf29fc570e56c88f6bd
Lustre-change: http://review.whamcloud.com/8622

Signed-off-by: Vladmir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I0c9c04787d45dfe6ba5ed01adb0a8ee265c6b3c5
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8811
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-3939 tests: sanity-hsm/test_40 needs a local HSM_ARCHIVE

sanity-hsm/test_40 suffers frequent failures during auto-test due
to remote/NFS-mounted HSM_ARCHIVE causing the 400 archive requests
to take more than 100s to be drained from copytool requests queue.
This patch allows copytool_setup func to allow each sub-test to
specify a non-default hsm-root/HSM_ARCHIVE dir and test_40 uses it.

This patch is back-ported from the following one:
Lustre-commit: 8484f1c51c701141237e98a1467c75364766f357
Lustre-change: http://review.whamcloud.com/7703

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I733b267991faa3b8c9415fea116d2086575333bb
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8771
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3920 tests: check MDS version before testing HSM feature

Sanity tests 56y and 229 fail when MDS does not have HSM
support. Check MDS version and skip the tests in that case.

This patch is back-ported from the following one:
Lustre-commit: b635ddd7f6ebe04681fae34da3b26e3b6b5301f0
Lustre-change: http://review.whamcloud.com/8121

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=229 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=70 mdsbuildno=70 \
testlist=sanity

Xyratex-bug-id: MRP-1417

Signed-off-by: Vladimir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I6bf3bffd45ad8a2a7c72424447a4d486389c8e8d
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8734
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4299 kernel: kernel update [SLES11 SP3 3.0.101-0.8]

update target and config files for new kernel version

Lustre-commit: a6bf2c1ee73a217df8e0b44fb0d5cea15a3bd874
Lustre-change: http://review.whamcloud.com/8762

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I1970bc9657286b57746e3f0a18ca9d22f134189e
Reviewed-on: http://review.whamcloud.com/8766
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4165 tests: skip sanity-lfsck test_2c for 2.4 or older

It makes no sense to run sanity-lfsck test_2c against 2.4 or older.

Test-Parameters: mdsjob=lustre-b2_4 ossjob=lustre-b2_4 mdsbuildno=58 ossbuildno=58 testlist=sanity-lfsck
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I96792b4325a69f880e326dc8963cf3e6bd09bf87
Reviewed-on: http://review.whamcloud.com/8386
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8728
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4072 tests: Decrease load on MDT for ZFS in sanity/24v

Due to performance of ZFS, reduce the number of file creates until
LU-2887/LU-4072 are resolved.

This patch is back-ported from the following one:
Lustre-commit: ee009f3b3e7bd467df3da3d0b53777db65790062
Lustre-change: http://review.whamcloud.com/7870

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I6271a7892c02885855b9e5b750438087e7875c5b
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8769
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4223 utils: fixing loop leaking in utils

1. If the file is being opened by popen, it should use
pclose instead of fclose to close the file, to make sure
the process created by popen is closed after pclose, then
to avoid loop device is being hold on release.

2. Give another try in loop_cleanup in case there are still
some process going on with the loop.

3. wait loop device to release before continue conf-sanity
32c.

4. Add losetup -a to list loop dev information when the
test(conf-sanity 32) fails.

This patch is back-ported from the following one:
Lustre-commit: 98ac0fe3a45dde62759ecaa4c84e6250ac2067f8
Lustre-change: http://review.whamcloud.com/8409

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdscount=4 mdtcount=4 testlist=conf-sanity

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ic1ebc2a6b2ce4280c2123080171e203e99267b28
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/8723
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4270 test: fix sanity test_209

Fix the connect_flags checking in test_209 of sanity.sh

This patch is back-ported from the following one:
Lustre-commit: b498499104af17da081f1c22b9c07951104846a3
Lustre-change: http://review.whamcloud.com/8326

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=209 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=70 mdsbuildno=70 \
testlist=sanity

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I6c34b9dbe6d3b7475d85588e7adb3acb762fab32
Reviewed-on: http://review.whamcloud.com/8729
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3971 hsm: Copytool code cleanup

Minor refactoring of the bandwidth controling code.
Deletion of a superfluous select() call on regular files.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: Iae550bb69c1524865b38a92d9b7674fce2f58258
Reviewed-on: http://review.whamcloud.com/7583
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4046 build: fix 'uninitialized variables' errors

Fix 'uninitialized variables' defects found by Coverity version 6.5.1:
Uninitialized scalar variable (UNINIT)
Using uninitialized value.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I4b5a3293e7d01436a82b6f7cea50b84c60660b3a
Reviewed-on: http://review.whamcloud.com/7821
Tested-by: Hudson
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4022 hsm: Set agent_actions start index from -1

aai_index was set to start from 0 which will cause the first entry
to be skipped. Set it to -1 instead.

Also take llog lock in proc_show() to avoid it from being changed
during the iteration.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I09013dbf8a85d198496d8470746e773b388d9bee
Reviewed-on: http://review.whamcloud.com/7879
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3726 tests: DNE specific tests for HSM.

Add tests in sanity-hsm.sh to check DNE specific
features in HSM code: copytool registration to
several MDTs, HSM request shunting to the right MDT,
and coumpound request splitting.

Test-Parameters: mdtcount=2 mdscount=2 testlist=sanity-hsm
Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I6706571c92e4c3e978b2143dd5bdef4b73d2c1e2
Reviewed-on: http://review.whamcloud.com/7571
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3726 tests: make sanity-hsm compatible with MDSCOUNT >= 2.

Add generic functions to perform set_param and check_param
on all MDTs, and persistent set_param on the mgs.
CL_USER variable replaced by CL_USERS array (1 value per MDT).
MDT0 variable replaced by MDT array (1 value per MDT).

Test-Parameters: mdtcount=2 mdscount=2 testlist=sanity-hsm
Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I0d8610a74427423dda25a6defd85f5c52c535850
Reviewed-on: http://review.whamcloud.com/7437
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3973 tests: use -P option of df to avoid line breaks

Sometimes df command output requires to break lines to
allow for columns alignement. This causes fields parsing
to fail. Use -P option to select compatibility output
and ensure one single line format per file-system is used.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ibc9d4b8f50857e680a5590dbbfe312fce137bcb6
Reviewed-on: http://review.whamcloud.com/7915
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-3985 tests: Archive access must be done from an agent

In sanity-hsm, all access to the archive backend must be
done from an agent. Tests 111a and 111b call the
copytool_remove_backend routine that calls find on the
archive backend. With this patch, the call to find is
done on the agent with a call to "do_facet $SINGLEAGT".

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I537fe5e019861908612140b3c96ea2018a738e46
Reviewed-on: http://review.whamcloud.com/8076
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3815 tests: HSM sanity test suites

Reevaluate the exception list.
Enable all sanity-hsm tests except #34, #35, #36 and #40.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I66dbf6f5119fc75b379752a33a8d5e44562ff985
Reviewed-on: http://review.whamcloud.com/7374
Tested-by: Hudson
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4064 hsm: create files to be imported on agent node

In sanity-hsm test 13 create the files to be imported on the agent
node rather than the local node.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3f50cc5269512d3b99644119ea18272d3a80f8a7
Reviewed-on: http://review.whamcloud.com/7885
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3692 api: More flexible logging API.

Allow applications to register their own logging callback instead of
writing every log message to stderr/stdout. Registered callbacks are
global to the process, just like the log level.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I5ccfd1e24f1786408ce67b2cd87d53443bcd028a
Reviewed-on: http://review.whamcloud.com/7572
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3909 tests: sanity test 129 fix

Patch from LU-2479 changed max dir size limit behaviour. Before the
patch last block of a directory which reached its limit was mostly
empty so that creation of new file after limit cancelation caused no
dir size grow. With the patch last block of limited directory gets
full when limit is reached. Therefore dir size increases on file
creation after limit cancellation.
The patch makes sanity test_129 to distinguish these cases.

This patch is back-ported from the following one:
Lustre-commit: 5cb9ac9c09f25234babc239ed949d85cd8489bef
Lustre-change: http://review.whamcloud.com/8143

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=129 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=70 mdsbuildno=70 \
testlist=sanity

Xyratex-bug-id: MRP-1444

Signed-off-by: Vladmir Saveliev <vladimir_saveliev@xyratex.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I691d7bac3dffe4eea98085ef2f9be3736e970869
Reviewed-on: http://review.whamcloud.com/8727
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3929 lwp: don't connect LWP to old MDT

When rolling upgrade Lustre from old version without LWP support
(< 2.3.60) to newer version, the OSTs will be upgraded firstly,
then LWP will try to connect the old MDT, and old MDT will regard
the LWP client as normal client and save client export on disk for
recovery, which will break recovery at the end since LWP
connection isn't recoverable.

Change-Id: Ib58c11a9d06b78ac9617563ae1baa4844e0efa45
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-on: http://review.whamcloud.com/8701
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4060 test: skip sanityn.sh test_74 on older than 2.4.93 MDS

2.5 client 2.4 server interop testing shows that sanityn.sh
test_74 needs to be skipped for Lustre MDS versions older than
2.4.93 servers.

This patch is for the b2_5 branch.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I8fd414adcf6bb6b7f8b390f1c52e3fea42701d1b
Reviewed-on: http://review.whamcloud.com/8700
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3952 nfs: don't panic NFS server if MDS fails to find FID

When MDS fails to retrive the parent's fid, we'd handle it without
crashing the NFS server.

Lustre-commit: 57649535fee8e92412db6806a30d2fa0752f762f
Lustre-change: http://review.whamcloud.com/8459

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I2ad36aa1ac2b7ae131cb21bf955a8ec6933d1dc3
Reviewed-on: http://review.whamcloud.com/8706
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4175 tests: fix obdecho_test() to calculate write count

This patch fixes obdecho_test() in sanity.sh to calculate
write count so as to avoid out of space failure while the
page number is large but the obd target size is small.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I5cbffb64387a62562eb85dd0208f9b1d7846ae31
Reviewed-on: http://review.whamcloud.com/8698
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4285 kernel: fix uninitialized access of quota got[] array

The pointer maybe uninitialized when dqput be called.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I94c138664c2f0a2404f2dafda25f3ee968a981fb
Reviewed-on: http://review.whamcloud.com/8418
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8716

LU-4176 tests: do not wait for deleted HSM request record

restore_and_check_size() function of sanity-hsm was systematically
waiting for restore request status, even if already flagged as
succeed. This extra wait can become forever if request record has
already been deleted.
Also fix wrong restore timing msg.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I9709d9e665b66fb5d16e93c7139fa8d95fa92589
Reviewed-on: http://review.whamcloud.com/8154
Tested-by: Jenkins
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8720

LU-3889 osc: Allow lock to be canceled at ENQ time

A cl_lock can be canceled when it's in CLS_ENQUEUED state.
We can't unuse this kind of lock in lov_lock_unuse() because
it will bring this lock into CLS_NEW state and then confuse
osc_lock_upcall().

Add a regression test case by Alexander Boyko.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: Ieb9209203a363c9098b9ea4637a9d12e12ef1aa4
Reviewed-on: http://review.whamcloud.com/8717
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-14 tests: add version check code to conf-sanity test 69

This patch adds Lustre version check codes into conf-sanity
test 69 to make the test work with servers which do not have
the following patch:

Lustre-commit: db6613f5bed1606cc8f97b46d1b298746af03a75
Lustre-change: http://review.whamcloud.com/7443

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=69 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=45 mdsbuildno=45 \
testlist=conf-sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I4e736fd5d760872f88364f30a39430f961419c64
Reviewed-on: http://review.whamcloud.com/8411
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-4204 tests: fix test_69 in conf-sanity

Recent commit added new test 69 to conf-sanity,
but calls test_68 by mistake. This patch fixes that typo.
In addition the test in its original form could never have worked.
There were several functional problems in it.
This patch fixes these up too.

Test-Parmeters: testlist=conf-sanity envdefinitions=SLOW=yes,ONLY=69

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I44d1e5f620a3e4cc14b59842abdd2f56ed409872
Reviewed-on: http://review.whamcloud.com/8158
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-4379 procfs: dont always check max_pages_per_rpc alignement

If connection is not established yet, cli->cl_chunkbits is
uninitilized so we cannot use it to check max_pages_per_rpc
alignment.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I4c782e2eeb5a2757d79194ee6ab6bff831a283fc
Reviewed-on: http://review.whamcloud.com/8558
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>

LU-4365 quota: wait for global lock cancel

In qsd_qtype_fini(), we'd wait for the global lock cancel done.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes  mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs  testlist=recovery-small
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib51dba4e4e4b13693c49560162d53cb28937e253
Reviewed-on: http://review.whamcloud.com/8583
Tested-by: Jenkins
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4362 libcfs: wrong parameter to cfs_hash_keycpy

cfs_hash_rehash_key() passed wrong parameter to cfs_hash_keycpy,
hnode should be the second parameter not the third one.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I44f73e4d2ffdeab5b021d49642598e939ebacee5
Reviewed-on: http://review.whamcloud.com/8509
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4313 osp: continue waiting for -ENOTCONN

in osp_precreate_reserve, -ENOTCONN should also be treated
as recoverable errors, such as -ENODEV, and continue waiting
the object to be created by OSP once it completes recovery.

Change-Id: I39b600eaacdb68f54c6ad7ecb417532bc0ad5e7b
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/8415
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>