LU-5319 tests: testcases for multiple modify RPCs feature

This patch creates new testcases in the Auster test suite to
test the support of multiple modify RPCs in flight feature.

Two new OBD_FAIL codes are added to allow several failure occurences
of the reint requests or replies. This is needed because the current
fail checks impose the OBD_FAIL_ONCE flag.

Added testcases are :
- sanity         245
- conf-sanity    90a 90b 90c 90d
- replay-single  102a 102b 102c 102d

This patch also removes replay-single tests 53a and 53b
from the ALWAYS_EXCEPT list.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I3cd233d5ef67a696e830c77b154dcc01520bb3d2
Reviewed-on: http://review.whamcloud.com/14861
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6480 o2iblnd: leak cmid in kiblnd_dev_need_failover

cmid created by kiblnd_dev_need_failover should always be destroyed,
however it is not the case in current implementation and we will leak
cmid when this function detected a device failover.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Icc567593f50fd557d5a7d1642a65d4fe49e8b4c2
Reviewed-on: http://review.whamcloud.com/14603
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6213 llite: add LL_IOC_FUTIMES_3

Add a new regular file ioctl LL_IOC_FUTIMES_3 similar to futimes() but
which allows setting of all three inode timestamps. Use this ioctl
during HSM restore to ensure that the volatile file has the same
timestamps as the file to be restored. Strengthen sanity-hsm test_24a
to check that archive, release, and restore do not change a file's
ctime. Add sanity-hsm test_24e to check that tar will succeed when it
encounters a HSM released file.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I98a4913048f902892fdafc514547bbe4d51523d7
Reviewed-on: http://review.whamcloud.com/13665
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5149 utils: Create debug upcall to dump ldlm namespaces

Created a script to be used as the debug_log_upcall which will dump
ldlm namespace information in raw/binary format and append it to the
given debug log file.

Signed-off-by: Ryan Haasken <haasken@cray.com>
Change-Id: I69f08f1b9563024c962f0213fd47a95b33b7da23
Reviewed-on: http://review.whamcloud.com/10617
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6600 obdclass: race lustre_profile_list

running multiple mounts at the same time results in
lustre_profile_list corruption when adding a new profile.

This patch adds a new spin_lock to protect the list and
avoid the bug

Signed-off-by: Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com>
Change-Id: I0d6ef47e9fa3dd9de8bd33d426112242136a9e26
Reviewed-on: http://review.whamcloud.com/14896
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6819 update: update transno of replay properly

Do not update master transno when the update request
in the replay list, otherwise it will affect the order
of the replay list.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I73709be714724c36f34ff64f8eabf4c2bec1112d
Reviewed-on: http://review.whamcloud.com/15594
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-589 auster: Export 'CONFIG' environment variable.

The 'CONFIG' variable defines the accsm configuration file. This
variable needs to be exported so that test-suite scripts use the
appropriate accsm configuration.

Change-Id: Ia0fdb4014d6eeefc3e08dced3d843427756c8587
Signed-off-by: Chris Horn <hornc@cray.com>
Reviewed-on: http://review.whamcloud.com/2051
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-4499 nrs: adjust the order of REQ NRS initilization

NRS block of ptlrpc_request must be initialized before it is added
into exp_{hp,reg}_list; otherwise ptlrpc_nrs_req_hp_move() may access
uninitialized NRS block. Code cleanup.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ide1d7edfa5e09bf498baaa60023dbccf3fb152dc
Reviewed-on: http://review.whamcloud.com/15943
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6757 ldiskfs: acquire i_mutex lock for ext4_update_i_disksize

For RHEL 7.1 kernel version 3.10.0, i_mutex lock needs to be acquired
before calling ext4_update_i_disksize() to avoid races with truncate.

i_size_write() also needs locking around it (normally i_mutex), otherwise
on 32bit/SMP an update of i_size_seqcount can be lost, resulting in
subsequent i_size_read() calls spinning forever.

For large EA support, ext4_xattr_check_names() will return -EIO. This
patch fixes that by checking whether the large EA value is saved in an
external EA inode or not.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I095246cc1f7457defe87ca246edf98bbc051bc29
Reviewed-on: http://review.whamcloud.com/16012
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-6894 kernel: new kernel RHEL6.7 [2.6.32-573.3.1.el6]

Add target and config files for new kernel in RHEL 6.7
Revise lbuild to add RHEL 6.7
Add new ldiskfs patch series for RHEL 6.7,
similar to 6.6 but without the
ext4-drop-inode-from-orphan-list-if-ext4_delete_inode-fails.patch
That change is now already present in the 6.7 version of ext4.

Test-Parameters: mdsdistro=el6.7 ossdistro=el6.7 \
  clientdistro=el6.7 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
  testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I09496b03b030fc678422a8d027ee185b8a660611
Reviewed-on: http://review.whamcloud.com/15701
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7033 utils: Add missing calls to wirecheck

The following functions were defined but never called:
    check_update_params
    check_update_op
    check_update_ops
    check_update_records
    check_llog_update_record

Change-Id: Ic1d12f5f422a01cf886432348d5c6c80ab19d1a5
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-on: http://review.whamcloud.com/16059
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7025 lfsck: Invalid return value assignment

In lfsck_layout_scan_stripes(), the following code section:
                if (lso == NULL) {
                        struct lu_attr *attr = &info->lti_la;

                        rc = dt_attr_get(env, parent, attr);
                        if (rc != 0) {
                                rc = PTR_ERR(lso);
                                goto next;
                        }
                        ...
                }
When dt_attr_get() returns failure, we should NOT assign the
"rc" as "PTR_ERR(lso)", instead, just use the current "rc".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8237e1f79588b54c73398f20bd833e81787bf4a6
Reviewed-on: http://review.whamcloud.com/16042
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6997 mdt: enqueue nonblocking locks in migration

During migration, it needs to lock all of parent objects
in the linkea of the migrating object, so these parent
objects can change the name entry after the migration.

But it is not easy to order these objects(in linkEA) in
parent-child order. To avoid deadlock, let's use
mdt_object_lock_try(), which will enqueue nonblocking
locks, i.e. once there are conflicting locks, the
migration will release all of locks then return -EBUSY.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id657cfd9d9caefaa7c699d5a5ca90604cf707ac0
Reviewed-on: http://review.whamcloud.com/15970
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6981 target: update obd_last_committed

In tgt_reply_data_init(), it should update
last_committed after analyzing those reply_data,
similar as tgt_server_data_init(), otherwise
obd_last_committed might be smaller than the real
committed transno, and then cause recovery_thread
stuck in check_for_next_transno(), which rely on
the correct last committed transno.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id036c41028c5bc5bf45f6ad300756e0653929bb3
Reviewed-on: http://review.whamcloud.com/15971
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Grégoire Pichon <gregoire.pichon@bull.net>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6977 lod: do_index_try should be called first

In lod_striped_it_next do_index_try should be called first
to initialize do_index_ops, then checking do_index_ops.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I477fb3c9ccd65c4e7721d78a3746f64acf8733ce
Reviewed-on: http://review.whamcloud.com/15972
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6475 mdt: race between open and migrate

During intent open, if it founds the parent has been
migrated to another MDT, it should retry the open
request with the new object, so it needs to keep the
old object in the orphan list, which will be cleanup
during next recovery. Note: if the client still using
the old FID after next recovery, it will return -ENOENT
for the application.

Reset parent FID on OST object after migration, which
also make sanity-lfsck.sh 15c obsolete, because that
test suppose to fix the parent FID of the OST object
after migration.

And also enqueue the lease lock of the migrating file,
then compare the lease before migration to make sure
no other clients open the file at the same time.

Add sanityn.sh 80b to test race between open/stat and
migration.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Iad3e5006079e9f72b338680e17448c39c6d1af98
Reviewed-on: http://review.whamcloud.com/14497
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7029 osc: fix debug log message formatting

Corrected newline specifier in debug log message.

Signed-off-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: Id7928648bbe6d8e00d36a9f05a90aaf732b0595a
Reviewed-on: http://review.whamcloud.com/16046
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2524 test: Clean up replay-ost-single test suite

Miscellaneous cleanup of replay-ost-single including:
Replace `lfs setstripe` with $SETSTRIPE
Replace `lfs getstripe` with $GETSTRIPE
Replaced beginning of line space indent with tabs
Replaced ‘return’ with a call to ‘error’
Added call to `error` with error messages for a variety of common
routines ,like mkdir, or for functions that return a value.
Replace `…` with $(...)
Removed linefeed escape after |, ||, & and && operators.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I92bebb5db0b42d8e0fd23aa6c783d546a8fd6507
Reviewed-on: http://review.whamcloud.com/14685
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4932 doc: update design doc for LFSCK

Update the document for MDT-MDT consistency verification.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Richard Henwood <richard.henwood@intel.com>
Change-Id: Idf36e9b9d1189ddf4062d96bcc647590d24e948f
Reviewed-on: http://review.whamcloud.com/10040
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6846 llog: combine cancel rec and destroy

Merge cancel llog record and llog object destroy in
a single transaction. Otherwise it will cause the
race between cancel record and llog object destroy,
especally when deleting remote record, i.e. if delete
record RPC arrives after destroy, then delete record
will not find the object, cause LBUG.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I33773eeb859fffa871ce01689c5bf1b4196a6658
Reviewed-on: http://review.whamcloud.com/15730
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 llite: handle backing_dev_info removal

For kernels before 3.20 the struct address_space contained
a pointer to a struct backing_dev_info. This backing_dev_info
had to setup with the backing_dev_info that was contained
in the super block. Now in newer kernels that backing_dev_info
pointer has been removed from the address_space. If you want
to use the backing_dev_info you need to access it from the
super block data structure directly. Lustre upstream removed
all the backing_dev_info setup for the inode i_mapping which
is no longer needed. This patch makes Lustre handle the bdi
inode backing_dev_info only for per 3.20 kernels now. This
change was due to:

Linux-commit: 6bec0035286119eefc32a5b1102127e6a4032cb2

Another change in the 3.20 kernels is the removal of the mmap
functionaly for the backing device. This was removed in with

Linux-commit: b4caecd48005fbed3949dde6c1cb233142fd69e9

With this change the flag BDI_CAP_MAP_COPY was removed.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ic2cd9227c99e3f66001ea9470cdc6bfdbf4f1c2c
Reviewed-on: http://review.whamcloud.com/15182
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6119 test: Add testing for Lustre exported via Samba

Create a parallel-scale-cifs.sh test which exports
Lustre via Samba and runs concurrent load compilebench, dbench,
fsx, iozone on Samba clients.

The test assumes that Samba server and clients have necessary
packages installed.

Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Xyratex-bug-id: MRP-1374
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Change-Id: Iace3b786bd34e42198a2ffdb35c436595fe2b8a5
Reviewed-on: http://review.whamcloud.com/13402
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6727 osd: auto locate BH_DXLock bit

BH_DXLock use a fix code bit before. It may occupied by other one
silently. From lustre view, jdb2 & ext4 also use this defines, So
we use ext4's define to determine last value.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I0fde8b461afec6acafe2708211761c3c60c529d7
Reviewed-on: http://review.whamcloud.com/15961
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6911 ldiskfs: mds crash kernel at fs/inode.c:1358

A glitch exists in LU-4648 patch (commit
ed69a331b1e502643fe074c121f9a8b9d041fc49) which does not put ea
inodes to orphan list if no external ea is used.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I1c28b989d73d06317e4064b739d69fa9453c5deb
Reviewed-on: http://review.whamcloud.com/15768
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6234 util: check fsname and pool name

The pool name length check should not include its fsname. This patch
get rid of the possibly added fsname in the pool name argument.

Check lustre filesystem name (fsname) in mkfs.lustre, it can only
contains alphabetic, digital char and '_'/'-'.

Check pool name in lctl/lfs, its character also abides by the
constraint of the fsname.

Add test cases to check the pool names for length, with and without
a filesystem name prefix, and with invalid names.

Also fix pool_add in the test-framework so that it greps for the
poolname plus the filesystem name, so it doesn't match more lines than
necessary.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I4f542e1bc7b3ea97b35ec60e51aa971e73da91eb
Reviewed-on: http://review.whamcloud.com/13742
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6714 llog: test plain llog more throughly

Modify llog test 3 to check more aspects of its
functionality:
- search the record by index
- check all records are processed
- check modification in-place
- check that processing is seeing concurrently added
  new records
- check the correctness of lgh_cur_offset and lgh_cur_idx

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I11a7c6474a7420393987a964e9112d61263bdc81
Reviewed-on: http://review.whamcloud.com/15835
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6817 llite: lower message level for ll_setattr_raw()

truncate and write can happen at the same time, so that a file can
be set modified even though the file is not restored from released
state, and ll_hsm_state_set() is not applicable for the file, and
it will return error in this case, we'd lower the error message level
in this case.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Iff5bc0a6cec990e4e9f6f8f4ef0faacc229b828b
Reviewed-on: http://review.whamcloud.com/15541
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6516 ofd: fix double spin_unlock

In ofd_inconsistency_verification_main(), spin_unlock() would
be performed twice if OBD_ALLOC_PTR(lr) failed. This patch fixes
this by adding a new label to gain lock again before unlocking.

Signed-off-by: Ulka Vaze <ulka.vaze@yahoo.in>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: If9838d75b9bf6936b99c9d92e8ff0c9b36d783c3
Reviewed-on: http://review.whamcloud.com/14875
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6385 tests: Sync OSTs between tests in obdfilter-survey

osd-ldiskfs and osd-zfs operate differently with regards to writeback
cache.  This adds a force sync between test runs so that a test list
of "write read" will have all writes finished before reads starts for
both ldiskfs and zfs.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I3414a595220d9eb65145f8246ed5e243c112b866
Reviewed-on: http://review.whamcloud.com/14143
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6305 osd-ldiskfs: buffer head leak in osd

iam_new_leaf() should release new_leaf
in case of an error

Change-Id: I48d358cb823d83b5412a1e4b88ab3ea13a0bee35
Xyratex-bug-id: MRP-2338
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: http://review.whamcloud.com/13909
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: remove atomic cpt allocations

libcfs contains functions to perform atomic memory
operations. These functions have never been used
so remove them.

Change-Id: I48dff483c4560700acbddff6268735837ed12b79
Signed-off-by: frank zago <fzago@cray.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15913
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: remove err.h

With the cleanup of userland with libcfs we no longer
need the special error handling macros.

Change-Id: I5a7a2e1df3beef548b74703b45052ce85166f3aa
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15909
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: remove unused cfs_timer_done

Remove the cfs_timer_done function in the libcfs
kernel module since it is not used anywhere.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ieb521b5b2ce4d3a66dfa0bafc3a866fc38fcd65f
Reviewed-on: http://review.whamcloud.com/13917
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5108 osc: Performance tune for LRU

Early launch page LRU work in osc_io_rw_iter_init();
Change the page LRU shrinking policy by OSC attributes;
Delete the contented lock osc_object::oo_seatbelt
Other tiny changes for LRU management

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I688c29a99a469ef74f929a0689596170c665b2ee
Reviewed-on: http://review.whamcloud.com/10458
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3031 ldlm: disconnect speedup

disconnect takes too long time if there are many locks to cancel.
besides the amount of time spent on each lock cancel, there is a
resched() in cfs_hash_for_each_relax(), i.e. disconnect or eviction
may take unexpectedly long time.
- do not cancel locks on disconnect_export;
- export will be left in obd_unlinked_exports list pinned by live
  locks;
- new re-connects will created other non-conflicting exports;
- new locks will cancel obsolete locks on conflicts;
- once all the locks on the disconnected export will be cancelled,
  the export will be destroyed on the last ref put;
- do not cancel in small portions, cancel all together in just 1
  dedicated thread - use server side blocking thread for that;
- cancel blocked locks first so that waiting locks could proceed;
- take care about blocked waiting locks, so that they would get
  cancelled quickly too;
- do not remove lock from waiting list on AST error before moving
  it to elt_expired_locks list, because it removes it from export
  list too; otherwise this blocked lock will not be cancelled
  immediately on failed export;
- cancel lock instead of just destroy for failed export, to make
  full cleanup, i.e. remove it from export list.

also make the proper order of events on umount:
- disconnect export;
- cleanup namespace, to cancel all the locks before export barrier;
- exports barrier;
- lprocfs_free_per_client_stats (requires nid_exp_ref_count == 0);
- namespace_free_post is left in cleanup ensure will not get and
  segfault on an absent namespace.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: Ia39b09ce967237ed5078c8a71e760b1e103c6f55
Xyratex-bug-id: MRP-395 MRP-1366 MRP-1366
Reviewed-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-by: Alexey Lyashkov <Alexey_Lyashkov@xyratex.com>
Tested-by: Elena Gryaznova <Elena_Gryaznova@xyratex.com>
Reviewed-on: http://review.whamcloud.com/5843
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6802 recovery: don't skip open replay on reconnect

Once reconnect happened during replay, we'd continue the open
replay with the last failed replay, but not the next.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I164c40db143ca860ab59f60582942614d5fb7925
Reviewed-on: http://review.whamcloud.com/15871
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6929 libcfs: typo in cfs_hash_for_each_relax()

In cfs_hash_for_each_relax(), rc != 0 means caller want to break
the iteration, so we should only continue the iteration when rc
is zero.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9666cd26ebe93627009bd03b7bbd341a65beaddf
Reviewed-on: http://review.whamcloud.com/15813
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6122 lnet: Allocate the correct number of rtr buffers

This patch ensures that the correct number of router buffers are
allocated.  It keeps a count that keeps track of the number of
buffers allocated.  Another count keeps the number of buffers
requested. The number of buffers allocated is set when creating
new buffers and reduced when buffers are freed.

The number of requested buffer is set when the buffers are
allocated and is checked when credits are returned to determine
whether the buffer should be freed or kept.

In lnet_rtrpool_adjust_bufs() grab lnet_net_lock() before using
rbp_nbuffers to ensure that it doesn't change by
lnet_return_rx_credits_locked() during the process of allocating
new buffers.  All other access to rbp_nbuffers is already being
protected by lnet_net_lock().

This avoids the case where we allocate less than the desired
number of buffers.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I96627cc8ba3d3d70a0bf581b21ccd3c9b2de327f
Reviewed-on: http://review.whamcloud.com/13519
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6455 mdt: disable IMA support

in IMA (Integrity Measurement Architecture), there are two xattr
"security.ima" and "security.evm" to protect the file to be modified
accidentally or maliciously, the two xattr are not compatible with
VBR, then disable it to workaround the problem currently and enable
it when the conditions are ready.

Change-Id: Ie3e30dcb0d4d605a17d301c6cda14818af40d7b0
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/14928
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.7.58

Change-Id: Ia373f99ab02b0ed713e7cc762782fd7ef2d6405d

LU-6846 osd: reset do_body_ops after creation

Reset the do_body_ops from osd_body_ops_new to
osd_body_ops after object creation succeeds.
Otherwise in OUT handler, if creation the llog object
and write records to the llog object are in the same
transaction(one updates handling process), the object
will not have create do_body_ops in write update.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id45f1a29572faadc381194f45a980eb58d193e93
Reviewed-on: http://review.whamcloud.com/15595
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6742 osd: remove legacy code

osd_convert_root_to_new_seq() was introduced to convert
the filesystems created with pre-production Orion code.
we don't need this anymore.

Change-Id: Ib5657fc7a905bebacd4074878cc76365e1e3d8d9
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/15335
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-7004 utils: defer conf_param deprecation

Since there has been relatively little testing of "lctl set_param -P"
and the test scripts and documentation have not been converted from
using "lctl conf_param" yet, I'm deferring the conf_param deprecation
warning to 2.8.53 (essentially 2.9.0) to give time for this change.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I6e879aaf5f9c9e1434f52a7274c663096ceb35b5
Reviewed-on: http://review.whamcloud.com/15979
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@seagate.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-6905 osp: rename ost_server(conn)_uuid

For OSP to MDT, rename ost_server(conn)_uuid(under /proc)
to mdt_server(conn)_uuid.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Iae794b3fd9fd9c21e5b6f04f406ab5e504aab1eb
Reviewed-on: http://review.whamcloud.com/15725
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7005 tests: disable conf-sanity test_50i

Skip new test for permanently disabling MDT because it is failing
about 50% of the time.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I39f71aa5fa880f95f024f74fbca6ef261a6e4dcd
Reviewed-on: http://review.whamcloud.com/15980

LU-6924 ptlrpc: replay bulk request

Even though the server might already got the bulk
replay request, but bulk transfer timeout, let's
replay the bulk request, i.e. treat such replay as
same as no replied replay request (See
ptlrpc_replay_interpret()).

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1f71eacc3a68941c00f16c9628342c662e7fe181
Reviewed-on: http://review.whamcloud.com/15793
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6881 update: add lock to protect commit check

In sub_trans_commit_cb(), the commit check should
be protected by lock, otherwise in some racy
scenarios, all committed will never be true,
even though all sub transaction has been committed.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I8f43ca8083753ab6eef4f2be56ef77bb8640bb79
Reviewed-on: http://review.whamcloud.com/15690
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6586 mgs: deactive MDT permanently

Deactivate/activate MDT in the config log, so
the user can permanently deactivate one MDT.

Add active proc entry for MDC, and mark MDC to
be inactive once the MDC is deactivated.

Add conf-sanity.sh 50i to verify activate
and deactivate MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia8d18f48f12fc180c32da777bde0902a774ddf93
Reviewed-on: http://review.whamcloud.com/14747
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6770 osc: use global osc_rq_pool to reduce memory usage

The per-osc request pools consume a lot of memory if there are
hundreds of OSCs on one client. This will be a critical problem
if the client doesn't have sufficient memory for both OSCs and
applications.

This patch replaces per-osc request pools with a global pool
osc_rq_pool. The total memory usage is 5MB by default. And it
can be set by a module parameter of OSC:
"options osc osc_reqpool_mem_max=POOL_SIZE". The unit of POOL_SIZE
is MB. If cl_max_rpcs_in_flight is the same for all OSCs, the
memory usage of the OSC pool can be calculated as:
Min(POOL_SIZE * 1M,
    (cl_max_rpcs_in_flight + 2) * OSC number * OST_IO_MAXREQSIZE)

Also, this patch changes the allocation logic of OSC write requests.
The allocation from osc_rq_pool will only be tried after normal
allocation failed.

Signed-off-by: Wu Libin <lwu@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I1b0c522ade01dba11d860ab57f83af53619ce4ba
Reviewed-on: http://review.whamcloud.com/15422
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6765 obdecho: don't copy lu_site

While creating an echo device, echo_device_alloc() copies the lu_site
from MD stack, such kind of copy result in uninitialized mutex and
other potential issues.

Instead of copying the lu_site, we'd use the lu_site by pointer directly.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
clientdistro=el6.6 ossdistro=el6.6 mdsdistro=el6.6 \
mdtcount=1 testlist=mds-survey

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
clientdistro=el6.6 ossdistro=el6.6 mdsdistro=el6.6 \
mdtcount=1 testlist=mds-survey

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: I00bd1a211cf0d556e427ba2f281fbcb1940d41f3
Reviewed-on: http://review.whamcloud.com/15657
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6714 llog: fix the llog_cat_set_first_idx()

The bug was introduced in previous commit causing
wrong catalog index to be set.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs clientdistro=el7 ossdistro=el6.6 mdsdistro=el6.6 mdtcount=1 testlist=performance-sanity
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Iaa0747c5f504109f3a265c7cdda91ebe7ba83b4f
Reviewed-on: http://review.whamcloud.com/15841
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6090 lnet: save errno between function calls

errno can be overwritten by functions like snprintf even if
it succeeded, so save the errno when needed.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I9a47267851f7c102727f8653a3a3727047f33186
Reviewed-on: http://review.whamcloud.com/15783
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6339 lnet: correct lnet script Usage help

Changed Usage message to use lnet rather than Lustre

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I9357fded89e9e33ccac63ee49620bd1c7e69f454
Reviewed-on: http://review.whamcloud.com/15780
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6891 test: fix grow_xattr() defect

string needs to be quoted to avoid syntax error on
DNE setup

Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Seagate-bug-id: MRP-2762
Reviewed-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Change-Id: Ibdb383f2581fbe4bd9e92f9d89a98bfe14209f23
Reviewed-on: http://review.whamcloud.com/15672
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6791 build: Update ZFS/SPL version to 0.6.4.2

Updates ZFS and SPL to latest maintence version.  Includes the
following:

Bug Fixes:
* Fix panic due to corrupt nvlist when running utilities
(zfsonlinux/zfs#3335)
* Fix hard lockup due to infinite loop in zfs_zget()
(zfsonlinux/zfs#3349)
* Fix panic on unmount due to iput taskq (zfsonlinux/zfs#3281)
* Improve metadata shrinker performance on pre-3.1 kernels
(zfsonlinux/zfs#3501)
* Linux 4.1 compat: use read_iter() / write_iter()
* Linux 3.12 compat: NUMA-aware per-superblock shrinker
* Fix spurious hung task watchdog stack traces (zfsonlinux/zfs#3402)
* Fix module loading in zfs import systemd service
(zfsonlinux/zfs#3440)
* Fix intermittent libzfs_init() failure to open /dev/zfs
(zfsonlinux/zfs#2556)

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I053087317ff9e5bedc1671bb46062e96bfe6f074
Reviewed-on: http://review.whamcloud.com/15481
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6450 mdd: Remove unused MAY_ constants

Remove unused MAY_ constants from lustre_idl.h
Remove dead code from mdd_permission()

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: I722f55eac88c7143555e680f72971b96791b00ae
Reviewed-on: http://review.whamcloud.com/15398
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 utils: update lr_reader to display additional data

The lr_reader command currently displays only the server area
of the LAST_RCVD file of a Lustre ldiskfs target device.

This patch enhances the command to allow diplaying :
- the per-client area of the LAST_RCVD file
  when --client or -c option is specified
- the reply data of the REPLY_DATA file
  when --reply or -r option is specified
It still only support ldiskfs target device.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ifc2b7f9591eef0da5e4a5dc00f37e58484d4ea97
Reviewed-on: http://review.whamcloud.com/14862
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2261 test: Re-enable sanity test 156 for zfs

With the landing of LU-4259 zfs now supports brw
stats like ldiskfs. Earlier sanity test 156 was
disabled for zfs but now it can be re-enabled so
we can actually verify brw stats on zfs works.

Apply additional clean up to test 156: Spaces to tabs.

Change-Id: I0a0014bee4c666faadce96177c44953573d85ffd
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/13535
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-5246 tests: create OST objects on correct MDT

The sanity.sh test_220 is checking that precreated objects on the MDT
can be used even when the OST has subsequently run out of inodes.
The test was not correctly working if the directory was striped over
multiple MDTs since the number of precreated objects on other MDTs
may be fewer than the number of precreated objects $SINGLEMDS and
run out of objects.

Change the test to only create files on $SINGLEMDS so to avoid this.

Also fix the return code of lod_alloc_specific() to be -ENOSPC if
the stripe count is 1, since -EFBIG doesn't make sense in that case.
Fix other code comments for lod_alloc_specific() and lod_alloc_qos().

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I788706c3b212b426aa15424d02913aedcdb98b2a
Reviewed-on: http://review.whamcloud.com/12937
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 llite: handle new_sync_[read|write] removal

In newer linux kernel version the file read/write
API have been moving to using struct iov_iter. To
continue supporting the old API new wrapper
functions were created, new_sync_*, which allowed
the use of the new API's to support the old API's.
For the linux 4.1 kernel those new_sync_* are used
internally so they are no longer exposed to any
file systems. If a file system wants to use the
"old" api they need to call vfs_[write|read].
Update lustre to handle this change. The upstream
commit that made new_sync_* internal was:

Linux commit: 5d5d568975307877e9195f5305f4240e506a2807

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ibf69d25af8a7d5cda00c5b1be4757ec369e8e814
Reviewed-on: http://review.whamcloud.com/15165
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6365 obd: Eliminate hash bucket scans in lu_cache_shrink

The lu_cache_shrink slab shrinker is too slow, accounting for > 90% of
the time spent in shrink_slab when allocating huge pages. Most of its
time is spent iterating over the buckets in each site's object hash
table to compute the number of freeable objects. This iteration is
eliminated by adding an lru length count to the lu_site struct. A
percpu counter is used to maintain the lru length, so that the
lu_site does not need to be locked when an object is accessed through
the hash table. A counter is updated whenever an object is added to
or deleted from any of the hash table buckets. The number of freeable
objects is the sum of the counter values across all cpus.

Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I5c70dd8e8cc1fe024292076e118774d9299bf40b
Reviewed-on: http://review.whamcloud.com/14066
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5554 ptlrpc: race at req processing

First commit 56e5a6bb4fc0329092f85016b8664da0b6ffa8a8
decrease race window, but does not remove it. Disable rq_resend
right after MSG_REPLAY flag set. Import lock protects two threads
from race between set/clear MSG_REPLAY and rq_resend flags.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Xyratex-bug-id: MRP-1888
Change-Id: I64d7c997976ff73956ebe97d7ea0775e0a54bc50
Reviewed-on: http://review.whamcloud.com/10735
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6896 llog: update log initialization needs to be sync

During initialization, update llog catlist and cat file creation
needs to be synchronized, because the following update records
will be written to these files, if these files are missing during
update recovery, then update recovery will fail.

Because update llog is initialized in a separate thread, it should
set loc_handle until the catlog is written to catlist to make sure
the udpate llog is fully initialized because MDT can handle cross
MDT operation. see lod_obd_get_info() to check the healthy of the
target.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Iccd09749bedde46c01ff89b5be4fa6f4327ec9b9
Reviewed-on: http://review.whamcloud.com/15721
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6972 llite: get rid of unused ll_super_blocks list

ll_super_blocks became unused quite a while ago with switch
to the new CLIO code.
So this patch removes the list, ll_sb_lock spinlock that guards it
and superblock info ll_list linkage.

Change-Id: If0af4f4c05f45feaedc54ca57f0d1106ad7cabe3
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/15922
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6831 lmv: revalidate the dentry for striped dir

If there are bad stripe during striped dir revalidation,
most likely due the race between close(unlink) and
getattr, then let's revalidate the dentry, instead of
return error, like normal directory.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I410940473d123796f75607d39153240b7d3f737b
Reviewed-on: http://review.whamcloud.com/15720
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6124 test: skip tests require remote server with nodsh set

Patch fixes the following tests to be skipped for remote
servers with nodsh set:
- sanity 116a, 116b, 160c, 205, 220, 225a, 225b, 230c;
- sanityn 34, 40a, 40b, 40c, 40d, 40e.

Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Xyratex-bug-id: MRP-1720
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Change-Id: I5567e752f0d05c2447de595cb72c754e06e2fe83
Reviewed-on: http://review.whamcloud.com/13406
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3163 tests: Fix test_61 from conf-sanity test

When large xattr is enabled and fstype is ldiskfs,
use tune2fs to enable/disable large_xattr.

Also replaced $TUNE2FS in all places instead of tune2fs.

Seagate-bug-id: MRP-1191
Signed-off-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I86605d980343a234cd6f4e2101cac409e017af62
Reviewed-on: http://review.whamcloud.com/15486
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6126 test: skip sanity test_187a, test_187b on old MDS

Don't run sanity test_187a, test_187b in interop mode for
MDS's versions older than 2.3.0 (per file data_version
implementation was included in 2.3, LU-827).

Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Xyratex-bug-id: MRP-1826
Reviewed-by: Vladimir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I6d42b51c769be80486c84fe436f6f3f71616e1cb
Reviewed-on: http://review.whamcloud.com/13408
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6490 gss: handle struct key_type match replacement

Starting with the 3.17 kernel the function match
used by struct key_type was replaced by a new
function match_preparse. This patch added support
for this change.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I2db0860cf089b0ba8f70bad9155f48e52ec8c4b5
Reviewed-on: http://review.whamcloud.com/15804
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6096 ldiskfs: mark dir's inode dirty

mark directory's inode dirty in ldiskfs_add_dot_dotdot()
as ldiskfs_init_new_dir() doesn't do this internally.
otherwise, the attributes set by ldiskfs_append() aren't
transferred to the buffer cache.

Change-Id: Ice93b75e291736f83c0d9b1047f7a627fef0141e
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/15581
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6938 osd: fix error handling in osd_xattr_list()

in case of buffer overflow, osd_xattr_list() should leave
properly: release the semaphore and the cursor.

Change-Id: I750183cb083cb87b3d8adae8fb0f41b61e6689e5
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/15828
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6542 lnet: handle cYAML out of memory error

in cYAML_build_error() there is a potential on run out of memory
scenario, that it is ignored and no error is built.
In this case print out a properly formated YAML error message
indicating fatal error

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Id9159044acafb38bd022f421a997fa2d2f1841e8
Reviewed-on: http://review.whamcloud.com/15778
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6906 mgc: IR log failure should not stop mount

If clients or other targets can not get IR config lock
or lock, the mount should continue, instead of failing.
Because timeout mechanism will handle the recovery anyway.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie717cb363180907b510593ee4a6caaec6f07a5f3
Reviewed-on: http://review.whamcloud.com/15728
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6544 mkfs: Improve MDT inode size calculation

This patch reduces mkfs.lustre "--stripe-count-hint" limits when
calculating MDT inode size, so that there is a proper amount of
space reserved for different kinds of EAs and ACL in the inode.

This would allow files with N stripes to fit the lov, lma, link
EAs into the inode rather than storing it in an external xattr
block, which reduces performance and significantly increases the
space used per file on the MDT.

Also, this patch adds conf-sanity.sh test_87 to verfiy this
calculation with different stripe count.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Idfdd9a064f0ac07c383a3af79c61c1ff973fb3f7
Reviewed-on: http://review.whamcloud.com/15643
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6904 mdd: prepare linkea before declare

In mdd_rename, it should prepare linkea before journal
start, otherwise, it may send RPC to other MDT while
holding the journal(see mdd_prepare_linkea). So if the
other MDT is in recovery, then this thread will be
blocked with the started journal handle, then it will
block other journal thread.

For the same reason, mdd_rename_order, which will send
RPC to the remote MDT (mdd_is_parent()), should also
move ahead of mdd_trans_start().

Because mdd_sobj can not be null in mdd_rename, this
patch also cleanup this API a bit.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id2426ef5d0d83c704403aa3f762596284576df6d
Reviewed-on: http://review.whamcloud.com/15724
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6874 out: missing noop in out_update_ops

NOOP is missing in out_update_ops, though this NOOP
operation will only exit inside the update log, i.e.
out_handler should never handle noop update. But
if we add new update in out_update_ops table, missing
noop there might cause problems.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I967ffefcd1499801fa1890ed8d97ce70190c1902
Reviewed-on: http://review.whamcloud.com/15692
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6490 build: enable gss build on sles12

Now that the build failures of gss on sles12 are fixed
change lbuild to stop disabling the gss build there.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Icf25c038c28b8e230550de0bbe5b0b2fbe0c0b78
Reviewed-on: http://review.whamcloud.com/15354
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6490 gss: 3.1x kernels adjustments for gssapi code

There are a number of changes in 3.1x kernels concerning the GSSAPI:
- libgssapi and libgssglue do not exist anymore, so call krb5
  primitives directly, and remove associated config checks;
- struct cred has no tgcred member anymore, so use cred directly;
- struct key_type instantiate and update function prototypes
  have changed;
- add new config checks on struct cred and struct key_type;
- u_int is BSD specific, so it is replaced with unsigned int.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I3b13c2afcb4b800bdcffb3b8713048f8e39f6866
Reviewed-on: http://review.whamcloud.com/15342
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6356 ptlrpc: add a 'enc_pool_max_memory_mb' module param

Create a new 'enc_pool_max_memory_mb' module param for ptlrpc.
It is used to set max memory used by encoding pools, in MB.
By default encoding pools will use at max 1/8 of total physical
memory, but it appears to be too few under some circumstances.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: If431ba082eec4bc2c4f2131d4b6b5b7028a3a569
Reviewed-on: http://review.whamcloud.com/15069
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5888 utils: limit max_sectors_kb tunable setting

Properly limit the max_sectors_kb tunable if larger than 32MB, as
added in commit 73bd456e896 (http://review.whamcloud.com/12723).
It was using the old value in "buf" instead of the updated "newval".

Change write_file() to use a file descriptor directly instead of
streams, since it only writes a single value before closing again.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I43a5b4c9d0e45649e0666fd58634286de53ebbe5
Reviewed-on: http://review.whamcloud.com/13240
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6916 target: fix ted_lcd_lock init

Move ted_lcd_lock init from tgt_client_new()
to tgt_client_alloc(), otherwise if error happens
in client_new_export(), the error handler path
will cause panic like this

<1>BUG: unable to handle kernel NULL pointer dereference at (null)
<1>IP: [<ffffffff812957f6>] __list_add+0x26/0xa0
<4> [<ffffffff8152adef>] __mutex_lock_slowpath+0xcf/0x180
<4> [<ffffffffa047e087>] ? cfs_hash_putref+0x2e7/0x480 [libcfs]
<4> [<ffffffff8152acfb>] mutex_lock+0x2b/0x50
<4> [<ffffffffa083db72>] tgt_client_free+0x62/0x610 [ptlrpc]
<4> [<ffffffffa0f59f84>] mdt_destroy_export+0x74/0x220 [mdt]
<4> [<ffffffffa05862c5>] class_new_export+0x315/0x9c0 [obdclass]
<4> [<ffffffffa058745e>] class_connect+0xae/0x250 [obdclass]
<4> [<ffffffffa0f57fd1>] mdt_obd_connect+0xb1/0x720 [mdt]
<4> [<ffffffffa07aaf78>] target_handle_connect+0xe58/0x2d30 [ptlrpc]
<4> [<ffffffff8128c1f0>] ? string+0x40/0x100
<4> [<ffffffffa084edf2>] tgt_request_handle+0x5b2/0x1230 [ptlrpc]
<4> [<ffffffffa07f7611>] ptlrpc_main+0xe41/0x1920 [ptlrpc]
<4> [<ffffffffa07f67d0>] ? ptlrpc_main+0x0/0x1920 [ptlrpc]
<4> [<ffffffff8109abf6>] kthread+0x96/0xa0
<4> [<ffffffff8100c20a>] child_rip+0xa/0x20
<4> [<ffffffff8109ab60>] ? kthread+0x0/0xa0
<4> [<ffffffff8100c200>] ? child_rip+0x0/0x20

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: If09c019384f53a63107aeaa293ef93d6415f81b5
Reviewed-on: http://review.whamcloud.com/15770
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6827 osd: trans credit insufficient for EA object create

EA object consumes more credits than regular object: osd_mk_index vs.
osd_mkreg, this patch reserves more credits for OSD_OT_CREATE
operation.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I40ec58ca467474ec4d96e5a1d24164fae03fc227
Reviewed-on: http://review.whamcloud.com/15694
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6768 osd: unmap reallocated blocks

call unmap_underlying_metadata() on every allocated
block. otherwise metadata blocks released in a previous
transaction can be written by the kernel corrupting
user's data. pblock != 0 should be a good sign of that.

Change-Id: I0d06611feb384a3f7ef5d8e5b34951822369ed0f
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/15593
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6427 osd: race between destroy and load LMV

Do not get LMVEA if the object has been destroyed,
because it needs to iterate the master object, which
is dangerous for ZFS OSD.

Add oo_destroyed flags to avoid accessing destroyed
directory.

Add sanityn.sh 83 to verify accessing destroyed striped
directory will not cause panic.

Signed-off-by: wang di <di.wang@intel.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1ef6bf197eac0917fd649868235c96af2f64207c
Reviewed-on: http://review.whamcloud.com/14346
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6502 lnet: remove unnecessary NULL check

In LNetCtl():IOC_LIBCFS_GET_NET there is a check for config == NULL
This is not necessary as it'll never be NULL.  That's ensured before
the call to LNetCtl.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ib4d1ca5ca4a342f134e3102bcaf167e60f978b5f
Reviewed-on: http://review.whamcloud.com/15779
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6833 build: gerrit_checkpatch.py improvements

In contrib/scripts/gerrit_checkpatch.py:
  Add LASSERT and LCONSOLE, to the list of ignored types.
  Use a 60 second HTTP request timeout to avoid hangs.
  Catch any Exception around HTTP methods.
  Refactor some methods for to facilitate unit testing.
  Add timestamps to log messages.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic8ddf815e58744af1cfa5a03986d869b790d91ae
Reviewed-on: http://review.whamcloud.com/15560
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5092 nodemap: add structure to hold nodemap config

This patch moves global state variables into a configuration
structure so that new configurations can be more easily loaded and
swapped into the active role.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: Ib0d51d56154d5e831b13f2935feab9bd73944bcc
Reviewed-on: http://review.whamcloud.com/14254
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6908 lfsck: initalize name before use

lfsck_create_lpf() should initialize name before using
that to enqueue a hashed lock.

Change-Id: I32ade3231111e5ce687eff348a0454a98c34d101
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/15727
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6803 gss: add printf format checking to printerr()

Add printf format checking to the gss error reporting functions
printerr(). Fixup all errors exposed by this change.

Change-Id: I2b7a2d65cb3ee81b11eb6af45297dad2e6cbb796
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15751
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6857 tests: TF_FAIL is not initialised

TF_FAIL is initialised in auster only,
this leads to t-f failure "touch: missing file operand"
if the suites run not under auster control.
DEFAULT_SUITES is initialised in acceptance-small only,
this leads to garbage summary message printed.

Patch adds export TF_FAIL to init_test_env() and
skips useless summary printing for the case when suites run
not under acceptance-small control.

Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Xyratex-bug-id: MRP-2116
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Change-Id: I047aae3b7c6f7d75e98e7862a7f00fd2d75d8b1f
Reviewed-on: http://review.whamcloud.com/15617
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6261 gnilnd: kgnilnd_check_rdma_cq race in error path.

It is possible for multiple threads to both be processing a
transaction error that causes the connection to close.
If the close path stalls after changing the conn state to CLOSING,
the tx_done path will put a tx in purgatory since the conn state is
not ESTABLISHED anymore. When the close path continues, it expects to
find that the conn->gnc_mdd_list is empty and assert if it isn't
empty.
Change the error code for the tx_done path so that we don't put this
tx in purgatory since we are closing the connection anyway.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I4458522f16508eb53d380f62320c65c7bf84657a
Reviewed-on: http://review.whamcloud.com/15439
Tested-by: Jenkins
Reviewed-by: James Shimek <jshimek@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6261 gnilnd: Use kgnilnd_vzalloc to allocate fma blocks.

In low memory situations we may not be able to allocate memory.
vmalloc tries forever to get memory.
Use kgnilnd_vzalloc which uses the GFP_NOFS flag for allocating
memory for fma blocks.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I18f8564424abfc9c63e675ac98ca61487b5a2b34
Reviewed-on: http://review.whamcloud.com/15438
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6460 llite: clear LLIF_FILE_RESTORING when done

Clear LLIF_FILE_RESTORING if restore done to ensure to start again
to glimpse new attrs.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I0f2f9532e06965273efa10e3f69e26d00676b8a6
Reviewed-on: http://review.whamcloud.com/14609
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6260 llite: add support for new iter functionality

For the 3.16+ kernels struct file_operations added new read
and write methods; read_iter and write_iter; to use struct
iov_iter as a parameter since it contains all the iovec
data needed. This avoid having the file system managing
iovec data like transversing the iovec page list. Now we
can use supplied iov_iter helper functions that does this
work for us. In later kernels only this back end is
supported. Backported from the upstream lustre client.

-----------------------------------------------------------------------
Linux-commit: b42b15fdad3ebb790250041d1517acebb9bd56d9

lustre: get rid of messing with iovecs

* switch to ->read_iter/->write_iter
* keep a pointer to iov_iter instead of iov/nr_segs
* do not modify iovecs; use iov_iter_truncate()/iov_iter_advance() and
  a new primitive - iov_iter_reexpand() (expand previously truncated
  iterator) istead.
* (racy) check for lustre VMAs intersecting with iovecs kept for now as
  for_each_iov() loop.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
-----------------------------------------------------------------------
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I71928beaa7b1c87f3e2c689e1dee96052eaef872
Reviewed-on: http://review.whamcloud.com/15028
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6341 llite: Use ll_dir_getstripe to get default LMVEA

Use ll_dir_getstripe to get default stripeEA in ll_new_node(),
Because ll_getxattr_common requires admin rights for retrieving
default LMVEA (because of trusted- prefix), which might cause
mkdir (from normal user) failure.

If parent does not have default stripeEA, then child should always
be in the same MDT for mkdir. Otherwise MDT should return -EREMOTE,
then client will refresh the default stripe index, and recreate
the object.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I0d8884b78fbc8b8d1930b1133150686e65d20c54
Reviewed-on: http://review.whamcloud.com/13990
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6052 utils: change "lfs mv" to "lfs migrate"

"lfs mv" causes some confusion between mv and migration, change
"lfs mv" to "lfs migrate -m", and mark "lfs mv" deprecated.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I72ead14854eb6038ba13fddec81b867bc0542b46
Reviewed-on: http://review.whamcloud.com/13754
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>

LU-6836 tests: Skip sanity-quota test 4a for ZFS

sanity-quota test 4a is failing frequently in
review-zfs-part-1 test groups. We are temporarily disabling
this test until a fix is submitted.

Note that LU-6836 is thought to be caused by bad ZFS sync
performance. We should reenable test 4a when ZIL support
is ready (LU-4009).

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I07867458e4365c0b1d31654be08e0fef9092c9eb
Reviewed-on: http://review.whamcloud.com/15679
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6769 build: Test for kthread_worker support

The OFED compatibility layer will backport kthread_worker unless
CONFIG_COMPAT_IS_KTHREAD is defined. kthread_worker is available in
the SLES 11 SP3 kernel, but not RHEL 6.5 or RHEL 6.6. This mod adds a
test to check if kthread is available in the kernel and defines
CONFIG_COMPAT_IS_KTHREAD appropriately.

Later versions of MLNX_OFED (2.4, 3.0) require that we similarly
define HAVE_KTHREAD_WORK.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ia89b73e4c8a3ad8549efdea70f0ebd96a1151dba
Reviewed-on: http://review.whamcloud.com/15428
Tested-by: Jenkins
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6828 lod: fix memory leak in lod_connect_to_osd

We would replace '$fsname-mdtlov' to '$fsname-MDT0000-osd'
when upgrade from 1.x. But buffer alloced as former length.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I8c92e3cb4472527fa5ad5da7f0e4cb498eaccb6c
Reviewed-on: http://review.whamcloud.com/15635
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5597 build: Ensure MOFED Module symvers are used

Change way extra symbols are included in build.  Use
KBUILD_EXTRA_SYMBOLS to directly included Modules.symvers
files instead of buildling Modules.symvers in buildroot
directory.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I4a6ee59f4e4eed9f878ad52993b8b17426f19d4a
Reviewed-on: http://review.whamcloud.com/15498
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>