LU-6356 tgt: handle sec context requests properly

When dealing with security context initialization requests,
no export is associated with the request yet, because these
requests are sent before *_CONNECT requests.
So the normal *_common_slice handlers will not be called,
given there is no reference to the target.

So add a special case for handling of SEC_CTX_INIT, SEC_CTX_INIT_CONT
and SEC_CTX_FINI requests in tgt_handle_request0() and
tgt_request_handle(), in order to directly send the reply.

Also remove tgt_sec_ctx_handlers because they become useless.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Ic0947082c7acc188ebbf2e9cb9ff57e999b5ed3e
Reviewed-on: http://review.whamcloud.com/14041
Tested-by: Jenkins
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 update: change sync updates to async updates

1. change sync updates to async updates.

2. add tests(110,111,112,115) in replay-single to test
async update recovery.

3. Add replay-barrier for those old DNE replay tests,
so all of MDTs can be recoveried correctly.

4. enable DNE failover to dbench test (replay-single.sh 70b).

Change-Id: Ibe1ca42d60c7a0ec870b9cf4a70a71adc663b951
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/12450
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6695 jobstats: skip too long env variables

The PATH env variable can be very large, it may exceed the page
size sometimes. cfs_get_envrion() should skip such too long env
variables instead of returning error.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I43c103a647cde3afa499dec3a11312155c3cf924
Reviewed-on: http://review.whamcloud.com/15177
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 lov: remove LSM from struct lustre_md

In struct lustre_md replace the stuct lov_stripe_md *lsm member with
an opaque struct lu_buf layout which holds the layout metadata
returned by the server. Refactor the LOV object initialization and
layout change code to accommodate this. Simplify lov_unpackmd() and
supporting functions according to the reduced number of use cases.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iac44ba479556693932bae0b9878565155b609e07
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/13722
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6718 build: force update of lustre-osd-ldiskfs-mount

Change the .spec file to make lustre-osd-ldiskfs require a
specific version of lustre-osd-ldiskfs-mount rather than
accepting just any version.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If12f8cb6d1a9d7105b7bfb04503acf437e189d46
Reviewed-on: http://review.whamcloud.com/15227
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6455 test: disable IMA related tests

Disable tests where IMA xattrs cause test failures.
This is a short term workaround to avoid known test problems
until a good fix for the IMA issues is found.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I7622135218facb8ed9ed9c9056204bba389cf370
Reviewed-on: http://review.whamcloud.com/15164
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-6030 ldiskfs: support pdirop split for RHEL6.5

Lustre commit 953e97987426bcc43a74be5b14f3a71319b33825
which split the parallel directory patch into two parts
was not applied to the RHEL6.5 platform which now results
in a build failure. The simple fix is to add the new
patches into the RHEL6.5 ldiskfs series file.

Change-Id: I12c407ac00e5471945217656e4c7a203ef34e99c
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15196
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6486 ptlrpc: export lustre_swab_lmv_mds_md

For ppc64 clients lustre would not mount due to an Unknown
symbol error. This is due to lustre_swab_lmv_mds_md not
being exported. A simple export of this function resolves
this issue.

Change-Id: I8db142e91f376fa0727b9de774ae0ca1cb0d9b5a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15159
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5835 ptlrpc: Introduce iovec to bulk descriptor

Added a union to ptlrpc_bulk_desc for KVEC and KIOV buffers.
bd_type changed to be a bit mask.
Bits are set in bd_type to specify {put,get}{source,sink}{kvec,kiov}
changed all instances in the code to access the union properly
ASSUMPTION: all current code only works with KIOV and new DNE code
to be added will introduce a different code path that uses IOVEC

As part of the implementation buffer operations are added
as callbacks to be defined by users of ptlrpc.  This enables
buffer users to define how to allocate and release buffers.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I86d718a668e7c0b422b96b393a6a69e5b8e89291
Reviewed-on: http://review.whamcloud.com/12525
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3537 mdt: allow cross-MDT rename and link

Remove checks for cross-MDT operation, so cross-MDT
rename and link will be allowed.

Remove obsolete locality parameters in MDT lock API after all of
cross-MDT operations are allowed.

Change-Id: I29874d929f98593d00f5cbd836c7ce681d51add7
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/12282
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6602 obdclass: variable llog chunk size

Do not use fix LLOG_CHUNK_SIZE (8192 bytes), and
it will get the llog_chunk_size from llog_log_hdr.
Accordingly llog header will be variable too, so
we can enlarge the bitmap in the header, then
have more records in each llog file.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1d18c4a502c5ab51d574bb026c94752ddb7dff8a
Reviewed-on: http://review.whamcloud.com/14883
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 osp: send updates by separate thread

Sending updates to other MDT by separate thread, so

1. The thread will sends the update by version number, i.e.
lower version of update will be sent first.

2. If one operation includes updates on several MDTs, these
updates can be sent on different MDTs in parallel. Though in
transaction stop (top_trans_stop()), the top transaction needs
to wait all sub transaction stop their transaction (i.e. updates
have been executed on all MDTs), then return.

3. Move dt_update_request to osp layer, since it is only OSP layer
thing, and each OSP transaction will create the osp_update_request
and attached to the list of OSP sending thread.

4. Change lgh_hdr_lock from spinlock to rw_semaphore, because if
the llog object is remote object, it will go into osp_md_write_obj,
where it might be blocked by memory allocation.

5. a few cleanups for the pervious DNE patches
http://review.whamcloud.com/11572
http://review.whamcloud.com/11737
see LU-6691, LU-6692, LU-6693.

Change-Id: I2dde892e4ca4bf74c2cd56e16bb1125920365334
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/12825
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6566 mdt: optimize sub_stripe lock/unlock

In lod_object_unlock, since it already know the lock
handle, it can release the lock directly, instead of
load and holding the sub-stripe object.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id9267f3619b036ce943a1df6ae163ea49a0fbc25
Reviewed-on: http://review.whamcloud.com/14679
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5781 ldlm: Solve a race for LRU lock cancel

This patch solves a race condition that the lock may be used again
after LRU cancellation policy check. In that case, the lock may have
locked or dirty pages that makes the policy check totally useless.
The problem is solved by checking l_last_used at cancellation time
therefore it can make sure that the lock has not been used.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I3c7e2a3cc84fa9f549cf002e751018c022493606
Reviewed-on: http://review.whamcloud.com/12603
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

new tag 2.7.55

Change-Id: I109819bf3f0d4a8c5c51b06e8cd58ff397e53f9b

LU-6697 build: remove SRPM dependency from kernel version

If machine don't have default kernel sources installed
the following error will be when you "make srpm".

error: line 144: Version required: Requires: kernel =
make: *** [srpm] Error 1

Change-Id: I1ab6cf1650e97b86ca5f25723bf7244e3ca08a03
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/15181
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 lnet: fix return value check in lnet_sock_ioctl()

Linux-commit: b3c1afd1fa366ef733a9f16c493a03396d7e4ab5

In case of error, the function sock_alloc_file() returns ERR_PTR()
and never returns NULL. The NULL test in the return value check
should be replaced with IS_ERR().

Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ib1e04da4ab2d637a632f0b4e84f38ed6b114e27e
Reviewed-on: http://review.whamcloud.com/15125
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 lnet: check for null pointer at line 1569 of router.c

Linux commit: d36175e9d2c986ee653277928bfcbab6ddf6bb85

staging/lustre/lnet: fix potential null pointer dereference

Pointer 'ni' checked for NULL at line 1569 may be passed to
function and may be dereferenced there by passing argument 1 to
function 'lnet_ni_notify_locked' at line 1621.
found by Klocwork Insight tool

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
CC: Dmitry Eremin <dmitry.eremin@intel.com>
CC: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I00c8f9f1d4ebd55d6c6f27943e59db7fb1adb525
Reviewed-on: http://review.whamcloud.com/15093
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>

LU-4423 lnet: Avoid forward declarations in nidstring.c

Backport of a patch from upstream  to remove some
forward declarations in nidstring.c. It appears
forward declarations are frowned on.

--------------------------------------------------------------
Linux commit: fd17846d8d6acf991c660ed3498031c6451db61d

staging: lustre: libcfs/nidstrings: Avoid forward declarations

Avoid forward declarations by moving functions to a location
before they are referenced.

Signed-off-by: Sören Brinkmann <soeren.brinkmann@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---------------------------------------------------------------
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ifeb5c6d61736b9d166966e76bb2a0224d77b602e
Reviewed-on: http://review.whamcloud.com/15084
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6636 libcfs: minor fix in cfs_hash_for_each_relax()

cfs_hash_for_each_relax() should break iteration when callback
returns non-zero value.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I92d35be4b11931cae252646ab2d1f8e5599037f1
Reviewed-on: http://review.whamcloud.com/14927
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6524 llog: Incorrect use of ERR_PTR

This issue is raised by smatch as below description
llog_osd_destroy incorrectly tried to use ERR_PTR pointer
Resolution is changed rc value to ENOENT

Incorporated code review comments.
Removed ENOENT of earlier commit and updated ot path.

Signed-off-by: Ulka Vaze <ulka.vaze@yahoo.in>
Change-Id: I010c21763ac7fc3664214eda93ea18e1c2d35de6
Signed-off-by: Ulka Vaze <ulka.vaze@yahoo.in>
Reviewed-on: http://review.whamcloud.com/14876
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6030 ldiskfs: further cleanup patches of ldiskfs

Remove some unused export symbol and correct patch
style as results of checkpatch.pl

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I84676b019189d1df113ae621297c60db860cd155
Reviewed-on: http://review.whamcloud.com/14490
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6030 ldiskfs: split pdirop patch

Split pdirop patch as two parts. One for Nlevel-htree and Largedir,
other one for pdirop and htree-lock. Also doing some cleanup
work to reduce the patch size.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I08b65d9098be95994f44748dbf14afa9f6d5b372
Reviewed-on: http://review.whamcloud.com/14264
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: cleanup up libcfs hash code for upstream

This patch brings the libcfs hash handling up to linux
kernel coding style. Various typedefs and macros have
been removed. Also support for the change of in 3.18
kernels of hlist_add_behind are also handled.

Change-Id: If50e34cfe4aaa593f755c9214f350ce21da56f20
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14624
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6652 tests: replay-dual test 18 stats wrong file

replay-dual test 18 touches file $MOUNT1/$tdir/$tfile and
then calls statmany on $MOUNT1/$tdir/f. File 'f' does not
exist and statmany fails. This is corrected by touching
$MOUNT1/$tdir/${tfile}0.

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=18 \
testlist=replay-dual

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I27b51a1bdff17c4bbc4e9d261d01686612099069
Reviewed-on: http://review.whamcloud.com/14960
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Justin Miller <jmiller@cray.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-5407 tests: Error message for replay-single 58b and 58c

In the replay-single test suite, tests 58b and 58c set an attribute,
retrieve the attribute and check that the original message matches
what was set. If large extended attributes is enabled, the attribute
written can be very long. In case of error, we only print the first
few charaters of the message that was set.

Also, call error if there is a problem when the client mounts the
file system on $MOUNT2.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I8c51ab04d906d74d6eac5a65420203936184ffe4
Reviewed-on: http://review.whamcloud.com/14766
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6675 tests: Improve error detection replay-single test 80

The replay-single tests 80a - 80d call remote_dir_check_80. This
subroutine does not check the return code of 'lfs getstripe' which
results in bash errors in when checking the return of getstrip.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I6c7e974aff38c13afc38594327d4a11ac5efd60b
Reviewed-on: http://review.whamcloud.com/15111
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Justin Miller <jmiller@cray.com>

LU-5710 all: third batch of corrected typos and grammar errors

All the typos are in comments or logs.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I3af7baa3cda6774b48e5651fd0591b2dade1131e
Reviewed-on: http://review.whamcloud.com/14424
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5278 echo: request pages in batches

rather than fetch them one by one.

Change-Id: I55dc5066402d1a6393f8edfe8cf71892a397339c
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/13612
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6528 mdt: keep FS capability for intent_getattr

When "no_subtree_check" is set for NFS export, nfsd_set_fh_dentry()
doesn't set correct fsuid explicitely, but raise capability to allow
exportfs_decode_fh() to reconnect disconnected dentry into dcache.
So for lookup (i.e. intent_getattr), we should keep FS capability on
MDS, otherwise it will fail permission check.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I99e2caeec06239a1df39918cbd6e9213e37ec758
Reviewed-on: http://review.whamcloud.com/14978
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3540 lod: update recovery thread

start recovery thread for each sub device in LOD, and
these threads will retrieve the update log from remote
MDT and attach them to recovery list.

Add update records to the recovery list, which
is attached to lu_target, and the recovery thread
(target_recover_thread()) will choose update or request
by transno during recovery.

Add update_recovery.c to handle updates replay during
recovery.

Change-Id: I367e9ef88f010143df50f7d06bd4242db1105db7
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/11737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6473 mdt: enqueue lookup lock on the parent MDT

Because the parent MDT will control the LOOKUP lock,
so Unlink and migration should enqueue the LOOKUP lock
on the parent MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I68fa328dc0e5f255b6a5b99a22db90a722c415fa
Reviewed-on: http://review.whamcloud.com/14495
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6581 mdt: Transfer layout only if layout lock is granted

Make sure that only valid layout is transferred;
Client also checks if lock is granted before trusting the layout;
Restore change LU-3299 commit 03a6a8b1 because it breaks the
assumption that l_lvb_data is immutable once assigned;
Comment fixups.

Signed-off-by: wang di <di.wang@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: If18e022131c5015e40e0408cdd0168df237e4338
Reviewed-on: http://review.whamcloud.com/14726
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6535 utils: fix updating index and fsname

We should do whether we can update index and fsname checks after
we have done all options parsing, otherwise, we failed this command:
tunefs.lustre --fsname=lustre --writeconf /dev/mgs
but succeed with following command:
tunefs.lustre --writeconf --fsname=lustre /dev/mgs

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Iadeaad1a6245c58214330c544df245b532f42372
Reviewed-on: http://review.whamcloud.com/14632
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6464 ldiskfs: 64bit pointers in ext4_map_inode_page()

To support >128TB OST ldiskfs should operates with large
blocks numbers. There is function ext4_map_inode_page() with
parameter “unsigned long *blocks”. An overflow is possible
for large blocks numbers. The problem is actual for x86_32
systems only.

This patch changes parameter type to sector_t that is 64 bit.

Xyratex-bug-id: MRP-2020
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Change-Id: If36544f39d3d31df9ff5ed644dad20f4910f3b39
Reviewed-on: http://review.whamcloud.com/14463
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6458 mdd: try linkEA first in mdd_parent_fid

It should try to get parent FID from linkEA first, if that
fails, then do lookup dotdot. Two benefits

1. read EA is likely cheaper than lookup.

2. for striped directory, lookup might cause load sub stripes,
which is only necessary for certain operation (like unlink),
and for other cases, we should avoid it, because it will pin
the sub_stripe object with the master objet in the memory, i.e.
sub stripe object will only be freed when the master object
is freed, see lod_object_free()->lod_object_free_striping().

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia54a2d36b6ab640c533c0fb864431db062646299
Reviewed-on: http://review.whamcloud.com/14444
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6030 ldiskfs: Remove alloc-policy patch

Remove alloc_policy & wantedi two patches since they
are no longer used.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ia283e16afe9d35d667c48e38db83d2a9cacb152c
Reviewed-on: http://review.whamcloud.com/14299
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-5814 lov: move LSM to LOV layer

Move the definition of struct lov_stripe_md along with supporting
functions from obd.h to lov_internal.h. Remove the unused functions
obd_packmd() and obd_free_diskmd(). Simplify lov_obd_packmd()
according to the reduced use cases and rename it lov_packmd().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I873a08f20bcb2e0919695325e51d74b2b97f4f0c
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/13696
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3105 osc: remove capa support

Remove capa support from the OSC layer. Convert the OBD getattr and
setattr paths to use struct obdo rather than struct obd_info. Remove
the oi_policy, oi_oa, and oi_capa members from struct obd_info.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3c7b3a8b5a5f330291e0096f65782f028fd4b5d8
Reviewed-on: http://review.whamcloud.com/14640
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6245 libcfs: move struct netstrfns to nidstr.h

The reason struct netstrfns exist in nidstrings.c
was to avoid forward decleration errors. The best
way to handle this instead is to move this structure
to a header file. Since this structure is used in
the userland utilities as well so we place it in
nidstr.h which is exposed to userland.

Change-Id: I02610e90a8d252defc9e8778f9f86be5ab2de9fe
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15083
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 client: handle dirty page cleanup api changes

In linux 4.0.0 the cancel_dirty_page function was replaced
by account_page_cleaned function. We handle this new case.

Change-Id: I956fa6b436bb2b01da0f9cbdfb2db707370e504e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14669
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 kernel: use f_path for inode dentry

Starting with linux kernel 2.6.17 the f_dentry field of
struct inode was moved to f_path from the top level. To
prevent breakage a backwards compatible macro was provided.
The linux kernel 3.18 removed that  macro which causes
lustre to fail to compile. This patch migrates the code
over to using the proper field.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I1497fa6667dfe1e72db0a6c7e2da78af9acdb1ee
Reviewed-on: http://review.whamcloud.com/14598
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6624 osc: LBUG in osc_lru_reclaim

LASSERT touches cl_client_cache->ccc_lru without any protection.
So this patch the LASSERT moves to the section protected by
cl_client_cache->ccc_lru_lock

Signed-off-by: Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com>
Change-Id: I1bc7829e4114a986a8299d652b978da1842829a5
Reviewed-on: http://review.whamcloud.com/14901
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4416 build: add build files for SLES 12 client build

minimum changes to allow building SLES 12 clients
target and lbuild

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I9e3ab4a597f3e48658de01b1eeee150eeddadf2d
Reviewed-on: http://review.whamcloud.com/14532
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: cancel update log after all committed

Cancel the log cookies after all of updates have been
committed to disk.

In DNE2, the distributed operation will write update log in all
of involve MDTs, when these the updates are committed on all of
MDT, these update records will be cancelled. If one of these MDTs
restart during the operation, it will replay the updates based
on these update records.

The distribute operation will be represent as distribute transaction
(top_multiple_thandle), and all of distribute transaction will be
linked into a list, which is attached to lu_target (see
struct target_distribute_txn_data), after distribute thandle is
committed, the commit thread (distribute_txn_commit_thread) will
update the commit batchid, after it is committed, it will then
cancel the update records.

Change-Id: I3023f95515b2c15307479f7f85c3f13926d4bcc9
Signed-off-by: Wang Di <di.wang@intel.com>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/11572
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: write updates to update log

For cross-MDT operation, LOD will write updates into the
update log on all of MDTs.

1. In transaction start, LOD perpare the update records
   buffer for cross-MDT operation.
2. Sub LOD collects all updates in execution phase.
3. In transaction stop, LOD will write thse updates as
   llog record on all of MDTs.

4. Disable sanity-scrub.sh 4 until LU6380 is fixed.

Change-Id: Ibba79267393db00ba05e0aa2df9865f88149eaa4
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/11408
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 mdt: move last_rcvd obj update to LOD

Move last_rcvd/version_obj updating from OSD to LOD, so
the update can be recorded in the update log, and last_rcvd
and version obj can be updated correctly in the recovery
process.

Change-Id: I2492864a66fab0028370d2d27255f6f458066197
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/11909
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: record update for cross-MDT operation

Packing updates for cross-MDT operation in the buffer, and
the maximum update record size is about 1M, which is enough
to create 2k stripes for now. To save the save, these update
records will use different packing format with OSP RPC,
see lustre/target/update_records.c

These updates will be stored in all of MDTs(in the later patch).
During the recovery, master MDT will redo the operation according
to these updates records.

Change-Id: Ic1919ab1c3d2eeca9ef027e2309c42201e3f7a74
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10939
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 mdc: add max modify RPCs in flight variable

This patch introduces the maximum modify RPCs in flight variable of
a mdc client obd device. Its value is set from connection flag and
and connection data. It can later be tuned through the
max_mod_rpcs_in_flight procfs file.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I51600f151fd508450d1898647a94856f16d2b245
Reviewed-on: http://review.whamcloud.com/14153
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6530 lod: do index_try for sub stripe

In lod_striped_it_next(), for each sub_stripe, it should call
index_try to initialize index operation, then access it.

Set lit_it to NULL in lod_striped_it_next(), so once failure
happens, lod_striped_it_fini will not finish the sub_it twice.
see (LU-5337).

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I83b45866f1d048530782262f7502cf9f51491741
Reviewed-on: http://review.whamcloud.com/14621
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6606 tests: delete memory limit from obdfilter-survey

This limit prevents to do enough load on newer HW, and have no
relation to current Lustre code. It is safe to remove this limit.

For disk usage the echo_client_brw_ioctl() limit the npages to
PTLRPC_MAX_BRW_SIZE>>PAGE_SHIFT. The total transfer is divided on
small portion with ptlrpc transfer size. Echo client will return
ENOMEM for bad memory allocation. And obdfilter-survey .detail file
will have the next error message
error: test_brw-1: #1 - Cannot allocate memory on write

Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Xyratex-bug-id: MRP-1642
Change-Id: I83d5b811cfa34dd3ec60fb670e595af193495b1c
Reviewed-on: http://review.whamcloud.com/14823
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6420 lfsck: misc patch to avoid unexpected repairing

It contains several fixes, including:

1) Generate D_LFSCK debug log only when the inconsistency needs
   to be repaired (or failed to be repaired).

2) If the OST-object's PFID EA has not been initialized, then
   not handle it as inconsistency.

3) Other fixes.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I3124ec6fa66c242b2cd8ebbf0fa99b2dc6f022a6
Reviewed-on: http://review.whamcloud.com/14332
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4615 lfsck: verify index before locating target descriptor

Sometimes, the OST index stored in the file layout EA may be crashed,
or maybe the target server has not registered. LFSCK should has the
ability to detect that to avoid invalid accessing.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I83ead7077885b6fc73bc56361725724d087c1957
Reviewed-on: http://review.whamcloud.com/14189
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6361 lfsck: measure LFSCK 4 performance

Measure the performance of namespace LFSCK for routine-check
cases and repairing cases under kinds of DNE configurations.

It also measures how much the namespace LFSCK will affect the
performance of creating small files.

Test-Parameters: mdtcount=4 mdscount=2 mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs envdefinitions=SLOW=yes testlist=lfsck-performance
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4f510b946d9d779fabcdec0ecd8e6c65278eb875
Reviewed-on: http://review.whamcloud.com/12109
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4745 tests: Skip dry-run for OST version older than 2.5.56

Skip the dry-run testing in test_52 for OST version older than
2.5.56

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=52 \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs clientdistro=el6.6 ossdistro=el6 \
mdsdistro=el6 clientarch=x86_64 ossarch=x86_64 mdsarch=x86_64 \
ossjob=lustre-b2_5 mdsjob=lustre-b2_5 \
ossbuildno=86 mdsbuildno=86 mdtcount=1 mdssizegb=3 \
ostcount=7 ostsizegb=3 testlist=conf-sanity

Change-Id: If3b45f6d29b83bce21ed40c67c5e985a8a9fd6db
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/9739
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6184 test: Skip sanity test_102q and test_102r

Skip sanity test_102q if MDS version older than 2.6.92;
skip sanity test_102r if MDS version older than 2.6.93.

Change-Id: I8a58446db453f55a2928d38091d6b5d7f8274f59
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/14886
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6302 lov: copy_to_user uses wrong casting

While testing on newer kernels lov_obd.c failes to compile
with the following warning.

In function copy_to_user,
inlined from lov_iocontrol at
lustre-2.7.52/lustre/lov/lov_obd.c:1168:
./arch/x86/include/asm/uaccess.h:735: error: call to
__copy_to_user_overflow declared with attribute warning:
copy_to_user() buffer size is not probably correct

In lov_iocontrol the data was being casted to int instead
of the required unsigned long. This patch changes the cast
to what is needed for copy_to_user.

Change-Id: I75b9f4f935d57dc3df56a2c0869c7d2506217d39
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14613
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6491 llog: fix wrong return value for too long fsname

If users passed a fsname longer than 9 bytes, caller
will handle it as ENOMEM which output following results:

error: conf_param: Cannot allocate memory

which is really unfriendly for common users.

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I51d0cf28a88d828cea4f8be9977ed274b822ea01
Reviewed-on: http://review.whamcloud.com/14568
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6626 utils: lfs_getstripe prints empty pool name

When the stripe version is V3, yet pool name is empty, the
lfs getstripe command will prints an empty pool name. That is
confusing since the user might think there is a pool for this
directory.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I4338362331bc958686464bfdfea647c90244f818
Reviewed-on: http://review.whamcloud.com/14906
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6507 obdclass: Fix possible NULL pointer dereference

The imp->imp_connection really could be NULL, we better check for it
before dereferencing it in taht call to libcfs_nid2str_r.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I16212eb6eb195eb197683606a4e4709810ff708f
Reviewed-on: http://review.whamcloud.com/14808
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6260 llite: add support for direct IO api changes

For the 3.16 kernels the direct_IO api changed which
breaks the lustre build. This patch adds support back
for direct IO on newer kernels. Backported from the
upstream lustre client.

------------------------------------------------------
Linux-commit: d8d3d94b80aa1a1c0ca75c58b8abdc7356f38418
pass iov_iter to ->direct_IO()

unmodified, for now

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: a6cbcd4a4a85e2fdb0b3344b88df2e8b3d526b9e
get rid of pointless iov_length() in ->direct_IO()

all callers have iov_length(iter->iov, iter->nr_segs) == iov_iter_count(iter)

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 886a39115005ced8b15ab067c9c2a8d546b40a5e

new primitive: iov_iter_alignment()

returns the value aligned as badly as the worst remaining segment
in iov_iter is.  Use instead of open-coded equivalents.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 91f79c43d1b54d7154b118860d81b39bad07dfff

new helper: iov_iter_get_pages_alloc()

same as iov_iter_get_pages(), except that pages array is allocated
(kmalloc if possible, vmalloc if that fails) and left for caller to
free.  Lustre and NFS ->direct_IO() switched to it.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 1d8cb70c7bdda47125ed551fc390aa9597c5f264
drivers: staging: lustre: Fix space required after that ',' errors

Fix checkpatch.pl space required after that ',' errors

Signed-off-by: Greg Donald <gdonald@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
------------------------------------------------------
Linux-commit: ef96fdddcd386c88fee1f2078a174943472b615e
staging: lustre: lustre: llite: use DIV_ROUND_UP

    The kernel.h macro DIV_ROUND_UP performs the computation (((n) + (d) - 1) /
    (d)) but is perhaps more readable.

    Coccinelle script used :

    // <smpl>
    @haskernel@
    @@
    @depends on haskernel@
    expression n,d;
    @@
    (
    - (n + d - 1) / d
    + DIV_ROUND_UP(n,d)
    |
    - (n + (d - 1)) / d
    + DIV_ROUND_UP(n,d)
    )
    @depends on haskernel@
    expression n,d;
    expression n,d;
    @@
    - DIV_ROUND_UP(n,(d))
    + DIV_ROUND_UP(n,d)
    // </smpl>

Signed-off-by: Tapasweni Pathak <tapaswenipathak@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
------------------------------------------------------
Linux-commit: 9fb186cf6907ba4e873d8396d7e5abfa4f22ca4e

Staging: Lustre: rw26.c: include according to checkpatch.pl

Signed-off-by: Georges-Axel Jaloyan <georges-axel.jaloyan@ens.fr>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
------------------------------------------------------
Linux-commit: 6f67376318abea58589ebe6d69dffeabb6f6c26a

direct_IO: use iov_iter_rw() instead of rw everywhere

The rw parameter to direct_IO is redundant with iov_iter->type, and
treated slightly differently just about everywhere it's used: some users
do rw & WRITE, and others do rw == WRITE where they should be doing a
bitwise check. Simplify this with the new iov_iter_rw() helper, which
always returns either READ or WRITE.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 22c6186ecea0be9eff1c399298ad36e94a59995f

direct_IO: remove rw from a_ops->direct_IO()

Now that no one is using rw, remove it completely.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------

Change-Id: I683e02a0952aec2d8fe387cf3ddaf75010858c7c
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14665
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6494 hsm: add support to lfs to remove by fid

Once an archived file has been deleted from Lustre, the only way to
remove it from an archive is to access the archive and delete it.

Enhance lfs hsm_remove so it can take an archive id and a fid to
delete that file. The FID has to be known somehow before the file is
deleted.

The lfs hsm_remove command gains a mntpath option to point to the
Lustre mountpoint, and an archive ID option.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ib7c5ab07d66754c71038147aa4524fd3a737a88f
Reviewed-on: http://review.whamcloud.com/14581
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6484 build: autoconf checks cleanup

Some of autoconf checks are not used any more and can be removed.
Some of them should be moved according new place of code.
Rrename LN_CONFIG_CDEBUG to LIBCFS_CONFIG_CDEBUG
fix AC_CHECK_HEADERS([netdb.h]) and AC_CHECK_FUNCS([gethostbyname])
fix caching for check LB_CONFIG_MPITESTS

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I988434d97051972b76b611e7b5a54835e8044734
Reviewed-on: http://review.whamcloud.com/14541
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6354 mdd: use cr_tfid on error in mdd_changelog_ns_store()

In mdd_changelog_ns_store() use the target FID in the the changelog
record for the CERROR() message rather than the sfid parameter (which
may be NULL).

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If493f982c4ec6ae36dc9703353c35a9f7b6b0d43
Reviewed-on: http://review.whamcloud.com/14027
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 llite: cleanup inode_dio_XXX functions

With the latest kernels inode_dio_done has been removed
so remove it from the autoconf test for the presence of
of inode_dio_wait. Besides inode_dio_done we can remove
inode_dio_read which is no longer used.

Change-Id: I97b7d926c246aebc19cc7747ba8846575f2aa06f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14664
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5739 ldlm: interval tree search in ldlm_lock_match()

replace the linear search by interval_tree one for granted list
in ldlm_lock_match()

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I3932521b279253ac130835d53d489477be7598ef
Xyratex-bug-id: MRP-2089
Reviewed-on: http://review.whamcloud.com/12294
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6201 llite: remove duplicate fiemap defines

* replace struct ll_user_fiemap with struct fiemap
* replace struct ll_fiemap_extent with struct fiemap_extent
* remove kernel defined FIEMAP_EXTENT_* constants
* remove kernel defined FIEMAP_FLAG_* flags
* add member prefix for struct ll_fiemap_info_key

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ib43276fa0c62242d82649c23afce9f7ec707a74a
Reviewed-on: http://review.whamcloud.com/13608
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4840 lfs: Use file lease to implement migration

Implement non-blocking migration based on exclusive open instead of
group lock. Implemented exclusive close operation to atomically put
a lease, swap two layouts and close a file. This allows race-free
migrations.

Make the caller responsible for retrying on failure (EBUSY, EAGAIN)
in non-blocking mode.

In blocking mode, allow applications to trigger layout swaps using a
grouplock they already own, to prevent race conditions between the
actual data copy and the layout swap. Updated lfs accordingly. File
leases are also taken in blocking mode, so that lfs migrate can issue
a warning if an application attempts to open a file that is being
migrated and gets blocked.

Timestamps (atime/mtime) are set from userland, after the layout swap
is performed, to prevent conflicts with the grouplock.

lli_trunc_sem is taken/released in the vvp_io layer, under the DLM
lock. This re-ordering fixes the original issue between truncate and
migrate.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie420e1998cae03928ae24834070c16642e8cd3b9
Reviewed-on: http://review.whamcloud.com/10013
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6299 osp: zero-length EA xattr_set for striped_dir

OSP/OUT should not assume xattr values always have
non-zero length, and OSP should be able to pack empty
xattr updates and sends to OUT.

Add test case in sanity 102r to verify empty value
xattr_set for striped dir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I14e0172038ef4c5671ec470228edd7cc4533a89c
Reviewed-on: http://review.whamcloud.com/14787
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6565 llog: Do not update the whole llog header

During adding llog record ( see llog_osd_write_rec()),
do not update the whole llog header, but only update
the byte where the bit locates and the counter.

So cross-MDT operation, which needs to write all of
updates of the operation in the llog record, does not
have to write the whole llog header(8K bytes) into
the update record, instead only writing a few bytes.
It will save a lot space for update logs and
ptlrpc request, and help to pack more updates into
the update log.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I777afbb111da6c1bc291735f9c4f105c48b3966b
Reviewed-on: http://review.whamcloud.com/14678
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4647 tests: properly cleanup after test failure

If test_16 fails all subsequent tests will fail also because of
improper cleanup after errors. Add cleanup in case of failure.

Change-Id: If828ad0e908c4f65be6f41f57fe14fb23801b2ed
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/14933
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6096 osd: mark directory's inode dirty

ldiskfs_init_new_dir() doesn't mark the inode
dirty, so i_size isn't transferred when ./..
are being inserted.

Change-Id: I1a80a80f6646984fbdec03d65579e735f6fac7c5
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/14772
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>

LU-5407 test: wait MGC import to finish recovery

in test_58c of replay_single.sh, the MGS is failed over with MDS
by the above subtest "test_58b", and the import of MGC could not
detect it yet, then causes the second Lustre mount to be failed.

this patch also remove test 58c from ALWAYS_EXCEPT list of ZFS

Change-Id: I7311e122964d6d8e14fa4327ed3ad936ac4ef59b
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/14792
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-790 tests: remove old get_lustre_version() calls

Replace old get_lustre_version() calls with lustre_build_version(),
since the former was deleted in commit 2a223541d2.  The few places
using get_lustre_version() were non-critical (just advisory messages)
but may generate errors messages in the test output.

Fix sanity-sec.sh to check the version on the mgs facet instead of
the mds1 facet, in case they are on separate nodes at some point.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I448bc13abbd36e86d0aa8cd6e22bacc9b6def27c
Reviewed-on: http://review.whamcloud.com/14768
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-6599 build: warn on likely uses of GPLv3

Add a warning to checkpatch.pl to warn on likely uses of GPLv3
headers.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5fc48c9aa7495d949734db432b6a0b413379e99f
Reviewed-on: http://review.whamcloud.com/14863
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6517 lmv: fix potential null pointer dereference

In lmv_disconnect_mdc do procfs removal only if we actually know the name.

Change-Id: Icb658a8a2007eea254ea5ce70b95c5d73df89914
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14605
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5919 build: Make "make rpms" use "make srpm" output

We rework the rpms and srpm Make targets.  The srpm
target generates the source rpm package much as it did
before, but it puts the results in a known temporary
rpm build tree.  This allows the "make rpms" target to
simply depend on the srpm target, and then build the
binary rpms from the source rpm.

Previously the rpm target _also_ generated a source rpm
package, which could be redudant, and failed to verify that
the source rpm package was itself buildable.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I9b2e83e000aca62b5d4841e2ebe9e91fa1db300f
Reviewed-on: http://review.whamcloud.com/12961
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6529 ldlm: cancel aged locks for LRUR

It doesn't make sense to keep the very aged lock even with the
LRUR policy. This patch decreased the default ns_max_age from 10
hours to 65 minutes and changed LRUR policy to cancel very aged
locks.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Iabf6f50a7837821afcf2a75b6c17c7484b34667f
Reviewed-on: http://review.whamcloud.com/14856
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 lov: use obd_get_info() to get def/max LOV EA sizes

Use obd_get_info() to get the default and maximum LOV EA sizes (along
with maximum cookiesize) from LOV. Remove the then unused function
obd_size_diskmd() and the unused get info key KEY_LOVDESC. When
computing the maximum LOV EA size use the active OST count
(ld_active_tgt_count) rather than the OST count (ld_tgt_count).

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I4a7eb158d9cb8f4828f5cb272df8d1c843f2eb6b
Reviewed-on: http://review.whamcloud.com/13695
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6573 test: fix sanity test_224c

It should always wait for the changing to the at_max/obd_timeout
taking effect.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I681ecb4a72c0b5f4f69e92341e7bde1b554c12a5
Reviewed-on: http://review.whamcloud.com/14844
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6593 kernel: kernel update [RHEL7.1 3.10.0-229.4.2.el7]

kernel update [RHEL7.1 3.10.0-229.4.2.el7]

Test-Parameters: clientdistro=el7 testgroup=review-ldiskfs \
  mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Id2e221c70d591f6ca852f87a9bdb317c7bd6a2c8
Reviewed-on: http://review.whamcloud.com/14804
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4772 mgs: free MGS fsdb before export barrier

The mgs_fsc keeps export reference and should be cleaned up
before the waiting for exports cleanup. Patch moves FSDB cleanup
functions which cleanups also mgs_fsc ahead of obd_export_barried()

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I98aa2e669a7ed2cba3b31df44fd3e96dcbbb6807
Reviewed-on: http://review.whamcloud.com/14443
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6557 llite: remove unused ll_max_rw_chunk

ll_max_rw_chunk seems to be unused ever since we implemented CLIO in 2.0,
so remove it and all supporting infrastructure.

Change-Id: Ie4d4142c3b70aa4fa757770b03b220de2253a681
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14657
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-6557 tests: ll_max_rw_chunk is going away, remove from test framework

ll_max_rw_chunk is being removed already, but it's used by
parallel group lock test too, so need to remove it from there.

The parameter itself was a noop since Lustre 2.0

Change-Id: Ib4df4d916348b408fd09ec09fba7ae2d9ae8e2d1
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14746
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-6505 libcfs: Use after free in __proc_cpt_table

Only free the too small buffer if it's really too small, not always.

Change-Id: I3fbdcb74dd0091c43aeb3e5b98460c203595bc19
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14602
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: frank zago <fzago@cray.com>

LU-6215 libcfs: shrinker code needs atomic.h

In the latest linux kernel code struct shrinker now
includes a new atomic_long_t so atomic.h is required
for the libcfs autoconf test will pass.

Change-Id: I7b5a5cc030bc8e8fc4cbd0c47bc75f73b07742f7
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14662
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5407 tests: Disable replay-single test 58c

Replay-single test 58c is failing review-zfs with enough
frequency that is it interfering with landing patches.

Test 58c is temporarily disabled for zfs only.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ief1b4d5c9ab6bf163bdae9737c1bffcdbef763cf
Reviewed-on: http://review.whamcloud.com/14864
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5683 clio: add CIT_DATA_VERSION

Add a new cl_io type CIT_DATA_VERSION to get file data version. Remove
the old data version code (cl_object_obd_info_get() and
cl_object_data_version()). Add test_37() to sanity-hsm.sh to ensure
that data version is computed under the correct layout.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If371c8e8c18e91d476f94de2e0170199252c088d
Reviewed-on: http://review.whamcloud.com/14649
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.7.54

Change-Id: I43dc353a8463eae5a2cc0d21c335e8fb74a9d575

LU-6599 header: Change erroneous GPLv3 header to GPLv2

The template for lustre_dlm_flags.h was including default
gpl definition which happened to be v3, and should have been v2.

Update the header to reflect that.
Also while we are at it, drop the whole autogeneration completely,
it was not really used and is no longer valid.

Change-Id: Id249566b71f77ecbf79a13c711f8d3ab72ca3679
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14797
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6068 misc: update old URLs to hpdd.intel.com

Update a URLs in several places to more modern equivalents.  Some
were in patches landed after the last mass URL update, others are
so old they were missed on the last update (CVS URL, sigh).

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I18418078ba8923b71c78f3de87aac997dda2eecc
Reviewed-on: http://review.whamcloud.com/13193
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6389 llite: restart short read/write for normal IO

If normal IO got short read/write, we'd restart the IO from where
we've accomplished until we meet EOF or error happens.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I4582affe8d2030df669b6e22b598b0c321e81466
Reviewed-on: http://review.whamcloud.com/14123
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5657 man: manpages for Lustre HSM user API

Added 2 man pages describing the following HSM user API functions:

  llapi_hsm_action_begin
  llapi_hsm_action_end
  llapi_hsm_action_get_dfid
  llapi_hsm_action_get_fd
  llapi_hsm_action_progress
  llapi_hsm_copytool_get_fd
  llapi_hsm_copytool_recv
  llapi_hsm_copytool_register
  llapi_hsm_copytool_unregister

Change-Id: Icd9c738c7b5287b5cfac2db4b697acb5a9e758ba
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/12061
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6017 obd: remove destroy cookie handling

Clients no longer need to track the max and default MDS cookiesizes
so remove
  the obsolete obod o_valid flag OBD_MD_FLCOOKIE,
  the struct client_obd members cl_{default,max}_mds_cookiesize,
  the struct obd_trans_info and parameters of this type,
  the get info keys KEY_{DEFAULT,MAX}_COOKIESIZE,
  the cookiesize parameters from md_init_ea_size(),
  the files llite/*/{default,max}_cookiesize, and
  any code that needlessly handled these values.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I95f0dae370998006ec366a252891db531759ea7d
Reviewed-on: http://review.whamcloud.com/12922
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-6503 lnet: info leak in lnet_ping()

This patch is a backport from upstream
751a624adbe47145d3667c4b504858ffd3e8f175

On 64 bit systems there is a 4 byte hole after
the last member of the struct. We should clear
it to avoid disclosing stack information.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I15849b3b35034d6c0c2a0f4c5cd0210ec453084e
Reviewed-on: http://review.whamcloud.com/14706
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5242 osd-zfs: umount hang in sanity 133g

It's not optimal to free large/sparse object directly by
dmu_object_free(). The dmu_tx_count_free() may over estimate the
memory overhead and cause the TX to fail in arc_tempreserve_space()
and eventually hang.

The correct way is to truncate the object with dmu_free_long_range()
before freeing it. This patch moves large object to ZPL unlinked set
and then later truncates and frees it, with as many TXs as needed.

Change-Id: I077dbcafff5abaea4a42dc9bc3f9ae980bdcdc54
Signed-off-by: Isaac Huang <he.huang@intel.com>
Test-Parameters: envdefinitions=SLOW=yes mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs
Reviewed-on: http://review.whamcloud.com/13630
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5975 quota: remove obsolete quota code

Remove the obsolete quotacheck, quotaon and quotaoff, which
were retained for the interoprability with old (< 2.4) client
and server.

Some other obsolete quota code related to LL_IOC_QUOTACTL_18,
Q_INVLIDATE and Q_FINVALIDATE are removed as well.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I21cd14d9fb783c1bfa09668b318189a24bf72224
Reviewed-on: http://review.whamcloud.com/14705
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6534 tests: Fix build issue, missing mode on an open with O_CREAT

When building with gcc 4.9.2, we hit the following build
error:

In function ‘open’,
    inlined from ‘sendfile_copy’ at sendfile_grouplock.c:229:9:
/usr/include/x86_64-linux-gnu/bits/fcntl2.h:50:4: error: call
to ‘__open_missing_mode’ declared with attribute error: open
with O_CREAT in second argument needs 3 arguments
    __open_missing_mode ();

..fix this with a mode of 0644

Change-Id: Ia06392228c960584e4a8262fac775078152d78d8
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Reviewed-on: http://review.whamcloud.com/14631
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4672 test: use large files in sanity-hsm tests 54 and 55

In sanity-hsm test_54() and test_55() use make_large_for_progress()
rather than make_small() so that the archives do not complete too
quickly for the tests to function correctly.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic46ff5e2f92497808be791201759e250ebbbfe7c
Reviewed-on: http://review.whamcloud.com/14759
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6468 tests: skip conf-sanity.sh test_83 for 2.6.90 or older

Skip conf-sanity.sh test_83 if OST server version is older than
2.6.91.

Test-Parameters: alwaysuploadlogs testlist=conf-sanity envdefinitions=ONLY=83 ossjob=lustre-b2_5 mdsjob=lustre-b2_5 ossbuildno=86 mdsbuildno=86
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I38fe3a52586dcb4d5706b4d7c379685e09e031cc
Reviewed-on: http://review.whamcloud.com/14534
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>