LU-5710 all: third batch of corrected typos and grammar errors

All the typos are in comments or logs.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I3af7baa3cda6774b48e5651fd0591b2dade1131e
Reviewed-on: http://review.whamcloud.com/14424
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5278 echo: request pages in batches

rather than fetch them one by one.

Change-Id: I55dc5066402d1a6393f8edfe8cf71892a397339c
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/13612
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6528 mdt: keep FS capability for intent_getattr

When "no_subtree_check" is set for NFS export, nfsd_set_fh_dentry()
doesn't set correct fsuid explicitely, but raise capability to allow
exportfs_decode_fh() to reconnect disconnected dentry into dcache.
So for lookup (i.e. intent_getattr), we should keep FS capability on
MDS, otherwise it will fail permission check.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I99e2caeec06239a1df39918cbd6e9213e37ec758
Reviewed-on: http://review.whamcloud.com/14978
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3540 lod: update recovery thread

start recovery thread for each sub device in LOD, and
these threads will retrieve the update log from remote
MDT and attach them to recovery list.

Add update records to the recovery list, which
is attached to lu_target, and the recovery thread
(target_recover_thread()) will choose update or request
by transno during recovery.

Add update_recovery.c to handle updates replay during
recovery.

Change-Id: I367e9ef88f010143df50f7d06bd4242db1105db7
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/11737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6473 mdt: enqueue lookup lock on the parent MDT

Because the parent MDT will control the LOOKUP lock,
so Unlink and migration should enqueue the LOOKUP lock
on the parent MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I68fa328dc0e5f255b6a5b99a22db90a722c415fa
Reviewed-on: http://review.whamcloud.com/14495
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6581 mdt: Transfer layout only if layout lock is granted

Make sure that only valid layout is transferred;
Client also checks if lock is granted before trusting the layout;
Restore change LU-3299 commit 03a6a8b1 because it breaks the
assumption that l_lvb_data is immutable once assigned;
Comment fixups.

Signed-off-by: wang di <di.wang@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: If18e022131c5015e40e0408cdd0168df237e4338
Reviewed-on: http://review.whamcloud.com/14726
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6535 utils: fix updating index and fsname

We should do whether we can update index and fsname checks after
we have done all options parsing, otherwise, we failed this command:
tunefs.lustre --fsname=lustre --writeconf /dev/mgs
but succeed with following command:
tunefs.lustre --writeconf --fsname=lustre /dev/mgs

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Iadeaad1a6245c58214330c544df245b532f42372
Reviewed-on: http://review.whamcloud.com/14632
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6464 ldiskfs: 64bit pointers in ext4_map_inode_page()

To support >128TB OST ldiskfs should operates with large
blocks numbers. There is function ext4_map_inode_page() with
parameter “unsigned long *blocks”. An overflow is possible
for large blocks numbers. The problem is actual for x86_32
systems only.

This patch changes parameter type to sector_t that is 64 bit.

Xyratex-bug-id: MRP-2020
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Change-Id: If36544f39d3d31df9ff5ed644dad20f4910f3b39
Reviewed-on: http://review.whamcloud.com/14463
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6458 mdd: try linkEA first in mdd_parent_fid

It should try to get parent FID from linkEA first, if that
fails, then do lookup dotdot. Two benefits

1. read EA is likely cheaper than lookup.

2. for striped directory, lookup might cause load sub stripes,
which is only necessary for certain operation (like unlink),
and for other cases, we should avoid it, because it will pin
the sub_stripe object with the master objet in the memory, i.e.
sub stripe object will only be freed when the master object
is freed, see lod_object_free()->lod_object_free_striping().

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia54a2d36b6ab640c533c0fb864431db062646299
Reviewed-on: http://review.whamcloud.com/14444
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6030 ldiskfs: Remove alloc-policy patch

Remove alloc_policy & wantedi two patches since they
are no longer used.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ia283e16afe9d35d667c48e38db83d2a9cacb152c
Reviewed-on: http://review.whamcloud.com/14299
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-5814 lov: move LSM to LOV layer

Move the definition of struct lov_stripe_md along with supporting
functions from obd.h to lov_internal.h. Remove the unused functions
obd_packmd() and obd_free_diskmd(). Simplify lov_obd_packmd()
according to the reduced use cases and rename it lov_packmd().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I873a08f20bcb2e0919695325e51d74b2b97f4f0c
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/13696
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3105 osc: remove capa support

Remove capa support from the OSC layer. Convert the OBD getattr and
setattr paths to use struct obdo rather than struct obd_info. Remove
the oi_policy, oi_oa, and oi_capa members from struct obd_info.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3c7b3a8b5a5f330291e0096f65782f028fd4b5d8
Reviewed-on: http://review.whamcloud.com/14640
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6245 libcfs: move struct netstrfns to nidstr.h

The reason struct netstrfns exist in nidstrings.c
was to avoid forward decleration errors. The best
way to handle this instead is to move this structure
to a header file. Since this structure is used in
the userland utilities as well so we place it in
nidstr.h which is exposed to userland.

Change-Id: I02610e90a8d252defc9e8778f9f86be5ab2de9fe
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15083
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 client: handle dirty page cleanup api changes

In linux 4.0.0 the cancel_dirty_page function was replaced
by account_page_cleaned function. We handle this new case.

Change-Id: I956fa6b436bb2b01da0f9cbdfb2db707370e504e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14669
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 kernel: use f_path for inode dentry

Starting with linux kernel 2.6.17 the f_dentry field of
struct inode was moved to f_path from the top level. To
prevent breakage a backwards compatible macro was provided.
The linux kernel 3.18 removed that  macro which causes
lustre to fail to compile. This patch migrates the code
over to using the proper field.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I1497fa6667dfe1e72db0a6c7e2da78af9acdb1ee
Reviewed-on: http://review.whamcloud.com/14598
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6624 osc: LBUG in osc_lru_reclaim

LASSERT touches cl_client_cache->ccc_lru without any protection.
So this patch the LASSERT moves to the section protected by
cl_client_cache->ccc_lru_lock

Signed-off-by: Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com>
Change-Id: I1bc7829e4114a986a8299d652b978da1842829a5
Reviewed-on: http://review.whamcloud.com/14901
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4416 build: add build files for SLES 12 client build

minimum changes to allow building SLES 12 clients
target and lbuild

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I9e3ab4a597f3e48658de01b1eeee150eeddadf2d
Reviewed-on: http://review.whamcloud.com/14532
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: cancel update log after all committed

Cancel the log cookies after all of updates have been
committed to disk.

In DNE2, the distributed operation will write update log in all
of involve MDTs, when these the updates are committed on all of
MDT, these update records will be cancelled. If one of these MDTs
restart during the operation, it will replay the updates based
on these update records.

The distribute operation will be represent as distribute transaction
(top_multiple_thandle), and all of distribute transaction will be
linked into a list, which is attached to lu_target (see
struct target_distribute_txn_data), after distribute thandle is
committed, the commit thread (distribute_txn_commit_thread) will
update the commit batchid, after it is committed, it will then
cancel the update records.

Change-Id: I3023f95515b2c15307479f7f85c3f13926d4bcc9
Signed-off-by: Wang Di <di.wang@intel.com>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/11572
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: write updates to update log

For cross-MDT operation, LOD will write updates into the
update log on all of MDTs.

1. In transaction start, LOD perpare the update records
   buffer for cross-MDT operation.
2. Sub LOD collects all updates in execution phase.
3. In transaction stop, LOD will write thse updates as
   llog record on all of MDTs.

4. Disable sanity-scrub.sh 4 until LU6380 is fixed.

Change-Id: Ibba79267393db00ba05e0aa2df9865f88149eaa4
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/11408
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 mdt: move last_rcvd obj update to LOD

Move last_rcvd/version_obj updating from OSD to LOD, so
the update can be recorded in the update log, and last_rcvd
and version obj can be updated correctly in the recovery
process.

Change-Id: I2492864a66fab0028370d2d27255f6f458066197
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/11909
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: record update for cross-MDT operation

Packing updates for cross-MDT operation in the buffer, and
the maximum update record size is about 1M, which is enough
to create 2k stripes for now. To save the save, these update
records will use different packing format with OSP RPC,
see lustre/target/update_records.c

These updates will be stored in all of MDTs(in the later patch).
During the recovery, master MDT will redo the operation according
to these updates records.

Change-Id: Ic1919ab1c3d2eeca9ef027e2309c42201e3f7a74
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10939
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 mdc: add max modify RPCs in flight variable

This patch introduces the maximum modify RPCs in flight variable of
a mdc client obd device. Its value is set from connection flag and
and connection data. It can later be tuned through the
max_mod_rpcs_in_flight procfs file.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I51600f151fd508450d1898647a94856f16d2b245
Reviewed-on: http://review.whamcloud.com/14153
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6530 lod: do index_try for sub stripe

In lod_striped_it_next(), for each sub_stripe, it should call
index_try to initialize index operation, then access it.

Set lit_it to NULL in lod_striped_it_next(), so once failure
happens, lod_striped_it_fini will not finish the sub_it twice.
see (LU-5337).

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I83b45866f1d048530782262f7502cf9f51491741
Reviewed-on: http://review.whamcloud.com/14621
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6606 tests: delete memory limit from obdfilter-survey

This limit prevents to do enough load on newer HW, and have no
relation to current Lustre code. It is safe to remove this limit.

For disk usage the echo_client_brw_ioctl() limit the npages to
PTLRPC_MAX_BRW_SIZE>>PAGE_SHIFT. The total transfer is divided on
small portion with ptlrpc transfer size. Echo client will return
ENOMEM for bad memory allocation. And obdfilter-survey .detail file
will have the next error message
error: test_brw-1: #1 - Cannot allocate memory on write

Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Xyratex-bug-id: MRP-1642
Change-Id: I83d5b811cfa34dd3ec60fb670e595af193495b1c
Reviewed-on: http://review.whamcloud.com/14823
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6420 lfsck: misc patch to avoid unexpected repairing

It contains several fixes, including:

1) Generate D_LFSCK debug log only when the inconsistency needs
   to be repaired (or failed to be repaired).

2) If the OST-object's PFID EA has not been initialized, then
   not handle it as inconsistency.

3) Other fixes.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I3124ec6fa66c242b2cd8ebbf0fa99b2dc6f022a6
Reviewed-on: http://review.whamcloud.com/14332
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4615 lfsck: verify index before locating target descriptor

Sometimes, the OST index stored in the file layout EA may be crashed,
or maybe the target server has not registered. LFSCK should has the
ability to detect that to avoid invalid accessing.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I83ead7077885b6fc73bc56361725724d087c1957
Reviewed-on: http://review.whamcloud.com/14189
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6361 lfsck: measure LFSCK 4 performance

Measure the performance of namespace LFSCK for routine-check
cases and repairing cases under kinds of DNE configurations.

It also measures how much the namespace LFSCK will affect the
performance of creating small files.

Test-Parameters: mdtcount=4 mdscount=2 mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs envdefinitions=SLOW=yes testlist=lfsck-performance
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4f510b946d9d779fabcdec0ecd8e6c65278eb875
Reviewed-on: http://review.whamcloud.com/12109
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4745 tests: Skip dry-run for OST version older than 2.5.56

Skip the dry-run testing in test_52 for OST version older than
2.5.56

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=52 \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs clientdistro=el6.6 ossdistro=el6 \
mdsdistro=el6 clientarch=x86_64 ossarch=x86_64 mdsarch=x86_64 \
ossjob=lustre-b2_5 mdsjob=lustre-b2_5 \
ossbuildno=86 mdsbuildno=86 mdtcount=1 mdssizegb=3 \
ostcount=7 ostsizegb=3 testlist=conf-sanity

Change-Id: If3b45f6d29b83bce21ed40c67c5e985a8a9fd6db
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/9739
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6184 test: Skip sanity test_102q and test_102r

Skip sanity test_102q if MDS version older than 2.6.92;
skip sanity test_102r if MDS version older than 2.6.93.

Change-Id: I8a58446db453f55a2928d38091d6b5d7f8274f59
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/14886
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6302 lov: copy_to_user uses wrong casting

While testing on newer kernels lov_obd.c failes to compile
with the following warning.

In function copy_to_user,
inlined from lov_iocontrol at
lustre-2.7.52/lustre/lov/lov_obd.c:1168:
./arch/x86/include/asm/uaccess.h:735: error: call to
__copy_to_user_overflow declared with attribute warning:
copy_to_user() buffer size is not probably correct

In lov_iocontrol the data was being casted to int instead
of the required unsigned long. This patch changes the cast
to what is needed for copy_to_user.

Change-Id: I75b9f4f935d57dc3df56a2c0869c7d2506217d39
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14613
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6491 llog: fix wrong return value for too long fsname

If users passed a fsname longer than 9 bytes, caller
will handle it as ENOMEM which output following results:

error: conf_param: Cannot allocate memory

which is really unfriendly for common users.

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I51d0cf28a88d828cea4f8be9977ed274b822ea01
Reviewed-on: http://review.whamcloud.com/14568
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6626 utils: lfs_getstripe prints empty pool name

When the stripe version is V3, yet pool name is empty, the
lfs getstripe command will prints an empty pool name. That is
confusing since the user might think there is a pool for this
directory.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I4338362331bc958686464bfdfea647c90244f818
Reviewed-on: http://review.whamcloud.com/14906
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6507 obdclass: Fix possible NULL pointer dereference

The imp->imp_connection really could be NULL, we better check for it
before dereferencing it in taht call to libcfs_nid2str_r.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I16212eb6eb195eb197683606a4e4709810ff708f
Reviewed-on: http://review.whamcloud.com/14808
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6260 llite: add support for direct IO api changes

For the 3.16 kernels the direct_IO api changed which
breaks the lustre build. This patch adds support back
for direct IO on newer kernels. Backported from the
upstream lustre client.

------------------------------------------------------
Linux-commit: d8d3d94b80aa1a1c0ca75c58b8abdc7356f38418
pass iov_iter to ->direct_IO()

unmodified, for now

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: a6cbcd4a4a85e2fdb0b3344b88df2e8b3d526b9e
get rid of pointless iov_length() in ->direct_IO()

all callers have iov_length(iter->iov, iter->nr_segs) == iov_iter_count(iter)

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 886a39115005ced8b15ab067c9c2a8d546b40a5e

new primitive: iov_iter_alignment()

returns the value aligned as badly as the worst remaining segment
in iov_iter is.  Use instead of open-coded equivalents.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 91f79c43d1b54d7154b118860d81b39bad07dfff

new helper: iov_iter_get_pages_alloc()

same as iov_iter_get_pages(), except that pages array is allocated
(kmalloc if possible, vmalloc if that fails) and left for caller to
free.  Lustre and NFS ->direct_IO() switched to it.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 1d8cb70c7bdda47125ed551fc390aa9597c5f264
drivers: staging: lustre: Fix space required after that ',' errors

Fix checkpatch.pl space required after that ',' errors

Signed-off-by: Greg Donald <gdonald@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
------------------------------------------------------
Linux-commit: ef96fdddcd386c88fee1f2078a174943472b615e
staging: lustre: lustre: llite: use DIV_ROUND_UP

    The kernel.h macro DIV_ROUND_UP performs the computation (((n) + (d) - 1) /
    (d)) but is perhaps more readable.

    Coccinelle script used :

    // <smpl>
    @haskernel@
    @@
    @depends on haskernel@
    expression n,d;
    @@
    (
    - (n + d - 1) / d
    + DIV_ROUND_UP(n,d)
    |
    - (n + (d - 1)) / d
    + DIV_ROUND_UP(n,d)
    )
    @depends on haskernel@
    expression n,d;
    expression n,d;
    @@
    - DIV_ROUND_UP(n,(d))
    + DIV_ROUND_UP(n,d)
    // </smpl>

Signed-off-by: Tapasweni Pathak <tapaswenipathak@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
------------------------------------------------------
Linux-commit: 9fb186cf6907ba4e873d8396d7e5abfa4f22ca4e

Staging: Lustre: rw26.c: include according to checkpatch.pl

Signed-off-by: Georges-Axel Jaloyan <georges-axel.jaloyan@ens.fr>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
------------------------------------------------------
Linux-commit: 6f67376318abea58589ebe6d69dffeabb6f6c26a

direct_IO: use iov_iter_rw() instead of rw everywhere

The rw parameter to direct_IO is redundant with iov_iter->type, and
treated slightly differently just about everywhere it's used: some users
do rw & WRITE, and others do rw == WRITE where they should be doing a
bitwise check. Simplify this with the new iov_iter_rw() helper, which
always returns either READ or WRITE.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------
Linux-commit: 22c6186ecea0be9eff1c399298ad36e94a59995f

direct_IO: remove rw from a_ops->direct_IO()

Now that no one is using rw, remove it completely.

Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
------------------------------------------------------

Change-Id: I683e02a0952aec2d8fe387cf3ddaf75010858c7c
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14665
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6494 hsm: add support to lfs to remove by fid

Once an archived file has been deleted from Lustre, the only way to
remove it from an archive is to access the archive and delete it.

Enhance lfs hsm_remove so it can take an archive id and a fid to
delete that file. The FID has to be known somehow before the file is
deleted.

The lfs hsm_remove command gains a mntpath option to point to the
Lustre mountpoint, and an archive ID option.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ib7c5ab07d66754c71038147aa4524fd3a737a88f
Reviewed-on: http://review.whamcloud.com/14581
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6484 build: autoconf checks cleanup

Some of autoconf checks are not used any more and can be removed.
Some of them should be moved according new place of code.
Rrename LN_CONFIG_CDEBUG to LIBCFS_CONFIG_CDEBUG
fix AC_CHECK_HEADERS([netdb.h]) and AC_CHECK_FUNCS([gethostbyname])
fix caching for check LB_CONFIG_MPITESTS

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I988434d97051972b76b611e7b5a54835e8044734
Reviewed-on: http://review.whamcloud.com/14541
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6354 mdd: use cr_tfid on error in mdd_changelog_ns_store()

In mdd_changelog_ns_store() use the target FID in the the changelog
record for the CERROR() message rather than the sfid parameter (which
may be NULL).

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If493f982c4ec6ae36dc9703353c35a9f7b6b0d43
Reviewed-on: http://review.whamcloud.com/14027
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 llite: cleanup inode_dio_XXX functions

With the latest kernels inode_dio_done has been removed
so remove it from the autoconf test for the presence of
of inode_dio_wait. Besides inode_dio_done we can remove
inode_dio_read which is no longer used.

Change-Id: I97b7d926c246aebc19cc7747ba8846575f2aa06f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14664
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5739 ldlm: interval tree search in ldlm_lock_match()

replace the linear search by interval_tree one for granted list
in ldlm_lock_match()

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I3932521b279253ac130835d53d489477be7598ef
Xyratex-bug-id: MRP-2089
Reviewed-on: http://review.whamcloud.com/12294
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6201 llite: remove duplicate fiemap defines

* replace struct ll_user_fiemap with struct fiemap
* replace struct ll_fiemap_extent with struct fiemap_extent
* remove kernel defined FIEMAP_EXTENT_* constants
* remove kernel defined FIEMAP_FLAG_* flags
* add member prefix for struct ll_fiemap_info_key

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ib43276fa0c62242d82649c23afce9f7ec707a74a
Reviewed-on: http://review.whamcloud.com/13608
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4840 lfs: Use file lease to implement migration

Implement non-blocking migration based on exclusive open instead of
group lock. Implemented exclusive close operation to atomically put
a lease, swap two layouts and close a file. This allows race-free
migrations.

Make the caller responsible for retrying on failure (EBUSY, EAGAIN)
in non-blocking mode.

In blocking mode, allow applications to trigger layout swaps using a
grouplock they already own, to prevent race conditions between the
actual data copy and the layout swap. Updated lfs accordingly. File
leases are also taken in blocking mode, so that lfs migrate can issue
a warning if an application attempts to open a file that is being
migrated and gets blocked.

Timestamps (atime/mtime) are set from userland, after the layout swap
is performed, to prevent conflicts with the grouplock.

lli_trunc_sem is taken/released in the vvp_io layer, under the DLM
lock. This re-ordering fixes the original issue between truncate and
migrate.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie420e1998cae03928ae24834070c16642e8cd3b9
Reviewed-on: http://review.whamcloud.com/10013
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6299 osp: zero-length EA xattr_set for striped_dir

OSP/OUT should not assume xattr values always have
non-zero length, and OSP should be able to pack empty
xattr updates and sends to OUT.

Add test case in sanity 102r to verify empty value
xattr_set for striped dir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I14e0172038ef4c5671ec470228edd7cc4533a89c
Reviewed-on: http://review.whamcloud.com/14787
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6565 llog: Do not update the whole llog header

During adding llog record ( see llog_osd_write_rec()),
do not update the whole llog header, but only update
the byte where the bit locates and the counter.

So cross-MDT operation, which needs to write all of
updates of the operation in the llog record, does not
have to write the whole llog header(8K bytes) into
the update record, instead only writing a few bytes.
It will save a lot space for update logs and
ptlrpc request, and help to pack more updates into
the update log.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I777afbb111da6c1bc291735f9c4f105c48b3966b
Reviewed-on: http://review.whamcloud.com/14678
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4647 tests: properly cleanup after test failure

If test_16 fails all subsequent tests will fail also because of
improper cleanup after errors. Add cleanup in case of failure.

Change-Id: If828ad0e908c4f65be6f41f57fe14fb23801b2ed
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/14933
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6096 osd: mark directory's inode dirty

ldiskfs_init_new_dir() doesn't mark the inode
dirty, so i_size isn't transferred when ./..
are being inserted.

Change-Id: I1a80a80f6646984fbdec03d65579e735f6fac7c5
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/14772
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>

LU-5407 test: wait MGC import to finish recovery

in test_58c of replay_single.sh, the MGS is failed over with MDS
by the above subtest "test_58b", and the import of MGC could not
detect it yet, then causes the second Lustre mount to be failed.

this patch also remove test 58c from ALWAYS_EXCEPT list of ZFS

Change-Id: I7311e122964d6d8e14fa4327ed3ad936ac4ef59b
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/14792
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-790 tests: remove old get_lustre_version() calls

Replace old get_lustre_version() calls with lustre_build_version(),
since the former was deleted in commit 2a223541d2.  The few places
using get_lustre_version() were non-critical (just advisory messages)
but may generate errors messages in the test output.

Fix sanity-sec.sh to check the version on the mgs facet instead of
the mds1 facet, in case they are on separate nodes at some point.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I448bc13abbd36e86d0aa8cd6e22bacc9b6def27c
Reviewed-on: http://review.whamcloud.com/14768
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-6599 build: warn on likely uses of GPLv3

Add a warning to checkpatch.pl to warn on likely uses of GPLv3
headers.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5fc48c9aa7495d949734db432b6a0b413379e99f
Reviewed-on: http://review.whamcloud.com/14863
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6517 lmv: fix potential null pointer dereference

In lmv_disconnect_mdc do procfs removal only if we actually know the name.

Change-Id: Icb658a8a2007eea254ea5ce70b95c5d73df89914
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14605
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5919 build: Make "make rpms" use "make srpm" output

We rework the rpms and srpm Make targets.  The srpm
target generates the source rpm package much as it did
before, but it puts the results in a known temporary
rpm build tree.  This allows the "make rpms" target to
simply depend on the srpm target, and then build the
binary rpms from the source rpm.

Previously the rpm target _also_ generated a source rpm
package, which could be redudant, and failed to verify that
the source rpm package was itself buildable.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I9b2e83e000aca62b5d4841e2ebe9e91fa1db300f
Reviewed-on: http://review.whamcloud.com/12961
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6529 ldlm: cancel aged locks for LRUR

It doesn't make sense to keep the very aged lock even with the
LRUR policy. This patch decreased the default ns_max_age from 10
hours to 65 minutes and changed LRUR policy to cancel very aged
locks.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Iabf6f50a7837821afcf2a75b6c17c7484b34667f
Reviewed-on: http://review.whamcloud.com/14856
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 lov: use obd_get_info() to get def/max LOV EA sizes

Use obd_get_info() to get the default and maximum LOV EA sizes (along
with maximum cookiesize) from LOV. Remove the then unused function
obd_size_diskmd() and the unused get info key KEY_LOVDESC. When
computing the maximum LOV EA size use the active OST count
(ld_active_tgt_count) rather than the OST count (ld_tgt_count).

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I4a7eb158d9cb8f4828f5cb272df8d1c843f2eb6b
Reviewed-on: http://review.whamcloud.com/13695
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6573 test: fix sanity test_224c

It should always wait for the changing to the at_max/obd_timeout
taking effect.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I681ecb4a72c0b5f4f69e92341e7bde1b554c12a5
Reviewed-on: http://review.whamcloud.com/14844
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6593 kernel: kernel update [RHEL7.1 3.10.0-229.4.2.el7]

kernel update [RHEL7.1 3.10.0-229.4.2.el7]

Test-Parameters: clientdistro=el7 testgroup=review-ldiskfs \
  mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Id2e221c70d591f6ca852f87a9bdb317c7bd6a2c8
Reviewed-on: http://review.whamcloud.com/14804
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4772 mgs: free MGS fsdb before export barrier

The mgs_fsc keeps export reference and should be cleaned up
before the waiting for exports cleanup. Patch moves FSDB cleanup
functions which cleanups also mgs_fsc ahead of obd_export_barried()

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I98aa2e669a7ed2cba3b31df44fd3e96dcbbb6807
Reviewed-on: http://review.whamcloud.com/14443
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6557 llite: remove unused ll_max_rw_chunk

ll_max_rw_chunk seems to be unused ever since we implemented CLIO in 2.0,
so remove it and all supporting infrastructure.

Change-Id: Ie4d4142c3b70aa4fa757770b03b220de2253a681
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14657
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-6557 tests: ll_max_rw_chunk is going away, remove from test framework

ll_max_rw_chunk is being removed already, but it's used by
parallel group lock test too, so need to remove it from there.

The parameter itself was a noop since Lustre 2.0

Change-Id: Ib4df4d916348b408fd09ec09fba7ae2d9ae8e2d1
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14746
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-6505 libcfs: Use after free in __proc_cpt_table

Only free the too small buffer if it's really too small, not always.

Change-Id: I3fbdcb74dd0091c43aeb3e5b98460c203595bc19
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14602
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: frank zago <fzago@cray.com>

LU-6215 libcfs: shrinker code needs atomic.h

In the latest linux kernel code struct shrinker now
includes a new atomic_long_t so atomic.h is required
for the libcfs autoconf test will pass.

Change-Id: I7b5a5cc030bc8e8fc4cbd0c47bc75f73b07742f7
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14662
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5407 tests: Disable replay-single test 58c

Replay-single test 58c is failing review-zfs with enough
frequency that is it interfering with landing patches.

Test 58c is temporarily disabled for zfs only.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ief1b4d5c9ab6bf163bdae9737c1bffcdbef763cf
Reviewed-on: http://review.whamcloud.com/14864
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5683 clio: add CIT_DATA_VERSION

Add a new cl_io type CIT_DATA_VERSION to get file data version. Remove
the old data version code (cl_object_obd_info_get() and
cl_object_data_version()). Add test_37() to sanity-hsm.sh to ensure
that data version is computed under the correct layout.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If371c8e8c18e91d476f94de2e0170199252c088d
Reviewed-on: http://review.whamcloud.com/14649
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.7.54

Change-Id: I43dc353a8463eae5a2cc0d21c335e8fb74a9d575

LU-6599 header: Change erroneous GPLv3 header to GPLv2

The template for lustre_dlm_flags.h was including default
gpl definition which happened to be v3, and should have been v2.

Update the header to reflect that.
Also while we are at it, drop the whole autogeneration completely,
it was not really used and is no longer valid.

Change-Id: Id249566b71f77ecbf79a13c711f8d3ab72ca3679
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14797
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6068 misc: update old URLs to hpdd.intel.com

Update a URLs in several places to more modern equivalents.  Some
were in patches landed after the last mass URL update, others are
so old they were missed on the last update (CVS URL, sigh).

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I18418078ba8923b71c78f3de87aac997dda2eecc
Reviewed-on: http://review.whamcloud.com/13193
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6389 llite: restart short read/write for normal IO

If normal IO got short read/write, we'd restart the IO from where
we've accomplished until we meet EOF or error happens.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I4582affe8d2030df669b6e22b598b0c321e81466
Reviewed-on: http://review.whamcloud.com/14123
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5657 man: manpages for Lustre HSM user API

Added 2 man pages describing the following HSM user API functions:

  llapi_hsm_action_begin
  llapi_hsm_action_end
  llapi_hsm_action_get_dfid
  llapi_hsm_action_get_fd
  llapi_hsm_action_progress
  llapi_hsm_copytool_get_fd
  llapi_hsm_copytool_recv
  llapi_hsm_copytool_register
  llapi_hsm_copytool_unregister

Change-Id: Icd9c738c7b5287b5cfac2db4b697acb5a9e758ba
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/12061
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6017 obd: remove destroy cookie handling

Clients no longer need to track the max and default MDS cookiesizes
so remove
  the obsolete obod o_valid flag OBD_MD_FLCOOKIE,
  the struct client_obd members cl_{default,max}_mds_cookiesize,
  the struct obd_trans_info and parameters of this type,
  the get info keys KEY_{DEFAULT,MAX}_COOKIESIZE,
  the cookiesize parameters from md_init_ea_size(),
  the files llite/*/{default,max}_cookiesize, and
  any code that needlessly handled these values.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I95f0dae370998006ec366a252891db531759ea7d
Reviewed-on: http://review.whamcloud.com/12922
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-6503 lnet: info leak in lnet_ping()

This patch is a backport from upstream
751a624adbe47145d3667c4b504858ffd3e8f175

On 64 bit systems there is a 4 byte hole after
the last member of the struct. We should clear
it to avoid disclosing stack information.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I15849b3b35034d6c0c2a0f4c5cd0210ec453084e
Reviewed-on: http://review.whamcloud.com/14706
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5242 osd-zfs: umount hang in sanity 133g

It's not optimal to free large/sparse object directly by
dmu_object_free(). The dmu_tx_count_free() may over estimate the
memory overhead and cause the TX to fail in arc_tempreserve_space()
and eventually hang.

The correct way is to truncate the object with dmu_free_long_range()
before freeing it. This patch moves large object to ZPL unlinked set
and then later truncates and frees it, with as many TXs as needed.

Change-Id: I077dbcafff5abaea4a42dc9bc3f9ae980bdcdc54
Signed-off-by: Isaac Huang <he.huang@intel.com>
Test-Parameters: envdefinitions=SLOW=yes mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs
Reviewed-on: http://review.whamcloud.com/13630
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5975 quota: remove obsolete quota code

Remove the obsolete quotacheck, quotaon and quotaoff, which
were retained for the interoprability with old (< 2.4) client
and server.

Some other obsolete quota code related to LL_IOC_QUOTACTL_18,
Q_INVLIDATE and Q_FINVALIDATE are removed as well.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I21cd14d9fb783c1bfa09668b318189a24bf72224
Reviewed-on: http://review.whamcloud.com/14705
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6534 tests: Fix build issue, missing mode on an open with O_CREAT

When building with gcc 4.9.2, we hit the following build
error:

In function ‘open’,
    inlined from ‘sendfile_copy’ at sendfile_grouplock.c:229:9:
/usr/include/x86_64-linux-gnu/bits/fcntl2.h:50:4: error: call
to ‘__open_missing_mode’ declared with attribute error: open
with O_CREAT in second argument needs 3 arguments
    __open_missing_mode ();

..fix this with a mode of 0644

Change-Id: Ia06392228c960584e4a8262fac775078152d78d8
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Reviewed-on: http://review.whamcloud.com/14631
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4672 test: use large files in sanity-hsm tests 54 and 55

In sanity-hsm test_54() and test_55() use make_large_for_progress()
rather than make_small() so that the archives do not complete too
quickly for the tests to function correctly.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic46ff5e2f92497808be791201759e250ebbbfe7c
Reviewed-on: http://review.whamcloud.com/14759
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6468 tests: skip conf-sanity.sh test_83 for 2.6.90 or older

Skip conf-sanity.sh test_83 if OST server version is older than
2.6.91.

Test-Parameters: alwaysuploadlogs testlist=conf-sanity envdefinitions=ONLY=83 ossjob=lustre-b2_5 mdsjob=lustre-b2_5 ossbuildno=86 mdsbuildno=86
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I38fe3a52586dcb4d5706b4d7c379685e09e031cc
Reviewed-on: http://review.whamcloud.com/14534
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-6558 ptlrpc: handle NULL export in nrs_orr_key_fill()

In nrs_orr_key_fill() the request export may be NULL so avoid
dereferencing it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I7f0a3a898586eb49ae7c1fadd9118759179280b9
Reviewed-on: http://review.whamcloud.com/14737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6406 tests: fix undersized fsname and FID strings

A Lustre filesystem name is up to 8 characters, plus 1 for the NUL
terminator. If a filesystem name is 8 characters, then the
llapi_layout_test, llapi_hsm_test, group_lock_test and llapi_fid_test
may have their stack scribbled or overran when they call
llapi_search_mounts(). Add one extra byte for the NUL character.

Same issue with a FID string, which also needs an extra character for
NUL termination.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ic7757ebce051e12f4723e27b5b00d4db1637b384
Reviewed-on: http://review.whamcloud.com/14213
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6484 build: remove unused LIBLUSTRE stuff

- Remove unused LIBLUSTRE stuff
- Fix packaging for ptlrpc module
- Remove liblustre.h
- Remove LIBLUSTRE_TEST
- Remove LLCPPFLAGS and LLCFLAGS
- Remove AC_LUSTRE_LIB_VER_OFFSET_WARN

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib1735e75a8e1b043eacca861448cecdfc84283cd
Reviewed-on: http://review.whamcloud.com/14543
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2667 ptlrpc: Move NRS structures out of lustre_net.h

NRS specific structures are not needed in the rest of the PtlRPC code.
It is more appropriate for these structures to be defined in a
separate header. This commit creates a lustre_nrs.h header for the
generic NRS structures, and policy-specific headers for the various
NRS policies.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: If7ef5e7eebf629a8102c8a199926cbb394a374bd
Reviewed-on: http://review.whamcloud.com/13966
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6152 osd-zfs: ZFS large block compat

This patch adds the required compatibility code to leverage the
new ZFS large block feature.  This includes supporting older versions
of ZFS which not contain large block support (ZFS v0.6.3 and earlier).

At the heart of this change is the SPA_MAXBLOCKSIZE macro which has
been increased from 128K to 16M.  This macro has always defined the
maximum possible blocksize support by ZFS.  However, just because
the ZFS code may now support this new large block it doesn't mean
that the feature is enabled on the pool.  Therefore, all consumers
of SPA_MAXBLOCKSIZE have been changed to use spa_maxblocksize() which
consults the pools feature flags to determine the supported maximum.

In addition, the ZFS OSD has been updated to to properly integrate
with a datasets recordsize property.  This allows the maximum block
size of a dataset to be changed with the 'zfs set' command.  This
has no effect on existing objects but new objects will be written
using the larger block size.  For example:

  zfs set recordsize=1M <dataset>

Finally, all references to SPA_MAXBLOCKSIZE and SPA_MAXBLOCKSHIFT
were removed and replaced with portable equivilants.  Going forward
the OSD layer needs to take case to no longer make any assumtions
about the maximum block size.

Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs mdsfilesystemtype=zfs
Change-Id: Iae782ff80ff185ca8c0f38e1002ff90e457c3ca1
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/13544
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5868 nodemap: Add nodemap_info to lctl

This patch adds nodemap_info to lctl, which can list all defined
nodemaps and display information about them, such as idmaps, ranges,
and flags. It is implemented as a wrapper to get_param/list_param,
and adds a -D parameter to list_param to only show directories.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I5d5ac44ec30bad49878be76168954dd231656421
Reviewed-on: http://review.whamcloud.com/13044
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6546 rpm: exclude duplicate binaries

Exclude /usr/sbin/wirecheck and /usr/sbin/wiretest binaries from lustre
package to avoid duplicate binaries in lustre and lustre-tests packages.

Change-Id: I59674fa5d1263a7e8022e6008556bf35a2de477b
Signed-off-by: Arnaud Guignard <arnaud.guignard@cea.fr>
Reviewed-on: http://review.whamcloud.com/14646
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6533 utils: Include unistd.h in wirecheck

Include unistd.h which is needed due to the use of
pipe, fork, dup2 and close.

Remove libcfs.h that became unneeded.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I123e0051b6988445e2e56c17ad582ece6758e3a2
Reviewed-on: http://review.whamcloud.com/14626
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6426 lustre: remove EIOCBRETRY handling

With linux commit 41003a7bcfed1255032e1e7c7b487e505b22e298
AIO retry handling was removed due to it being buggy and no
one using it, including lustre. Since this is the case
remove EIOCBRETRY since it no longer in the kernel starting
with version 3.18.

Change-Id: Icdc2de3fe252a6315914b906ef29c80343aa93e8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14507
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Blake Caldwell <blakec@ornl.gov>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <thomas@stibor.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6476 lnet: avoid race during acceptor thread termination

This patch will avoid potential race, around socket sleepers
wait list, during acceptor thread termination and using
sk_callback_lock RW-Lock protection.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ib0a06bd8b3947648cb3ab7e1148b5073f68316fe
Reviewed-on: http://review.whamcloud.com/14503
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5939 hsm: make HSM modification requests replayable

There are several HSM requests which modify data on server and
reply on Lustre recovery, e.g. they should replay changes in
case of recovery.

Patch allows such requests to be replayed in recovery time and
they are issued from client using mdc_rpc_lock to serialize them
and avoid concurrent last_rcvd update on server.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I39810f99e99fadb76a96facf903405f6c85cc71f
Reviewed-on: http://review.whamcloud.com/13684
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 llite: add cl_object_maxbytes()

Add cl_object_maxbytes() to return the maximum supported size of a
cl_object. Remove the lli_maxbytes member from struct
ll_inode_info. Change the lsm_maxbytes member of struct lov_stripe_md
from __u64 to loff_t. Correct the computation of lsm_maxbytes in the
released layout case.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Idfa6d38ed1192d562f7d99bf21b924fd711f77dc
Reviewed-on: http://review.whamcloud.com/13694
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6536 llapi: lmm_stripe_count used unswabbed

The function llapi_layout_get_by_fd() uses the value
lum->lmm_stripe_count without first checking if it needs
to be swabbed.  This causes a false error on PowerPC clients
because the lov_user_md data appears corrupted.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I5aacbd5804c784ec8231d51f8c93645fda077102
Reviewed-on: http://review.whamcloud.com/14633
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6559 test: use local tmp for HSM archive

In sanity-hsm.sh place the HSM archive in the tmp directory local to
the copytool.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ied319cfca2481f19f726f91f065148c5675e006d
Reviewed-on: http://review.whamcloud.com/14659
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6245 tests: remove libcfs.h from test applications

Since user land libcfs is mostly gone libcfs.h can be
removed in the appilcations used for the test suite.

Change-Id: I8104655a750533ece9c74b2ee06cd197abfd3354
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14629
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6527 ext4: journal_commit_callback optimization

Remove spin lock/unlock s_md_lock - it gives too
much overhead with huge transactions. This place
is not racy because txn is closed to the moment
when callbacks are called.

Change-Id: Ieaf76cce52854b9e5481c729bbb539eb7f97213f
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Xyratex-bug-id: MRP-2466
Reviewed-on: http://es-gerrit.xyus.xyratex.com:8080/5690
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com>
Tested-by: Jenkins
Tested-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: http://review.whamcloud.com/14610
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4698 target: check for NULL tgt before deref

In the case of and echo server during obdfilter-survey tests a target
can be missing from the obd_device.
Check for a valid pointer before dereferencing it, return error if
it's NULL.
Check for valid obt in class_exp2tgt otherwise return NULL target.
Add checks for all places that call class_exp2tgt for NULL return
value.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I27a3020dc7ef9943fb600c3ddb6432f7f67f8b39
Reviewed-on: http://review.whamcloud.com/9936
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6419 fld: fld client lookup should retry

If FLD client lookup fails because of the remote target
is shutdown (or deactive), it should retry another target,
otherwise it will cause the application failure.

And FLD client should stop retry if the import has
been deactive.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ic5934054bf0aff3158b6fe040fcb26699336f038
Reviewed-on: http://review.whamcloud.com/14313
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4366 test: sync didn't return ENOMEM

The fail_loc has been intercepted by earlier write, this patch add
a sync before setting the fail_loc.

Test-Parameters: alwaysuploadlogs testlist=sanity
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I0517ccc591651e275dc4d4cd2a3de179c2c2b3f1
Reviewed-on: http://review.whamcloud.com/14658
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-5420 mgc: MGC should retry for invalid import

After http://review.whamcloud.com/#/c/9967/ is landed, mgc does
not wait the import connected(state = FULL), then enqueue and
retrieve config log, which will cause the mount process to fail,
especially if the mgc is shared by multiple targets.
So once mgc enqueue is failed, it will give another chance to
wait the import to recover, if the import comes back in time,
it will try to enqueue again. Otherwise it will use local config log.

Signed-off-by: Wang Di <di.wang@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I8dbcafbda362ebbd3370e7527a3c14c594500c1e
Reviewed-on: http://review.whamcloud.com/11258
Tested-by: Jenkins

Revert "LU-6206 ptlrpc: start minimum service threads"

This is causing test failures in conf-sanity test_53b,
as seen in LU-6563.

This reverts commit cafa669a43062c5097d40803b9ba14e66edbae25.

Change-Id: I9935bb4d84fda7a9219818c20daa683954dcd0d7
Reviewed-on: http://review.whamcloud.com/14682
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

Revert "New tag 2.7.53"

Actually some quota code causes us build failures here.
revert for now, as this is gettign fixed.

This reverts commit 984ee697332660a9e1d67965db26060e35a9ab32.

New tag 2.7.53

Change-Id: I503736234daf8af75cf4e3cbdac32535ef01b7f4

LU-6465 osd: NO OI scrub because of cached invalid OI mapping

On server side, the RPC service thread may cache one OI mapping
on its stack, such OI mapping will become invalid if some other
removed the object by race. If the RPC service thread uses the
cached OI mapping and finds some inconsistent object, it should
re-lookup the OI files to verify whether the OI mapping that is
cached on the stack is valid or not. If the OI mapping has been
removed from the OI file, then means the OI mapping is invalid,
should NOT trigger OI scrub under such case.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifbc653fcc635d77447747f91de79bcc16f2dcbbc
Reviewed-on: http://review.whamcloud.com/14499
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 obd: remove unused LSM parameters

Remove unused struct lov_stripe_md * parameters from obd_get_info(),
mgc_enqueue(), and osc_brw_prep_request().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iee0be72393c609967ee8dad06f878026ca09821d
Reviewed-on: http://review.whamcloud.com/13426
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6319 tests: Clean up replay-single ALWAYS_EXCEPT list

Several tests in replay-single were added to the ALWAYS_EXCEPT list
when a bug was making the test fail repeatedly. For tests 33a, 33b,
and 62, the tickets were closed, but the tests were never
re-activated.

Remove replay-single tests 33a, 33b, and 62 from the
ALWAYS_EXCEPT list.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I872af58ca6fb7b362b4ffe146132d98262a280a4
Reviewed-on: http://review.whamcloud.com/14149
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6477 build: Update SPL/ZFS to 0.6.4.1

Version 0.6.4 Relase Notes
https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.4

New Functionality

* Compatible with kernels up to Linux 4.0.
* New feature flags (additional details below):
        spacemap_histogram
        extensible_dataset
        bookmarks
        enabled_txg
        hole_birth
        embedded_data
* New asynchronous I/O (AIO) support.
* New fallocate() FALLOC_FL_PUNCH_HOLE support.
* New fragmentation metric in 'zpool list'.
* New LZ4 compression of meta data.
* New "redundant_metadata" property controls desired redundancy
level.
* New "overlay" property controls behavior for non-empty mount
points.
* New 'zpool list -v' shows individual disk capacity.
* New 'zpool get -H' (scripted mode) support.
* New 'zpool create -t' creates a pool with a temporary name.
* New arc_summary.py script from FreeNAS.
* New bash completion support.
* New DTRACE_PROBES integrated with Linux tracepoints.
* New compressed block histograms with zdb.
* New verbatim pool imports with zdb.

New Feature Flags

*   spacemap_histogram

    This features allows ZFS to maintain more information about how
free space is organized within the pool. If this feature is enabled,
ZFS will set this feature to active when a new space map object is
created or an existing space map is upgraded to the new format.

*   extensible_dataset

    This feature allows more flexible use of internal ZFS data
structures, and exists for other features to depend on. This feature
will be active when the first dependent feature uses it.

*   bookmarks

    This feature enables use of the zfs bookmark subcommand. Bookmarks
mark the point in time when a snapshot was created, they can be used
as the incremental source for a zfs send command. All bookmarks in the
pool can be listed by running zfs list -t bookmark -r poolname.

*   enabled_txg

    Once this feature is enabled ZFS records the transaction group
number in which new features are enabled. This has no user-visible
impact, but other features may depend on this feature.

*   hole_birth

    This feature improves performance of incremental sends ("zfs send
-i") and receives for objects with many holes. The most common case of
hole-filled objects is zvols.

*   embedded_data

    This feature improves the performance and compression ratio of
highly-compressible blocks. Blocks whose contents can compress to 112
bytes or smaller can take advantage of this feature.

Bug Fixes

    Fixed I/O error on fs/vol delete corrupting space map.
    Fixed corruption during spacemap reallocation.
    Fixed corruption due to faulty logic when undirtying spill block.
    Fixed stale bonus buffer in recycled dnode_t data corruption.
    Fixed SA header size accounting.
    Fixed O_APPEND flag for open(2).
    Fixed deadlocks caused by direct reclaim by setting PF_FSTRANS.
    Fixed deadlocks on suspended pools.
    Fixed deadlock in zio pipeline caused by mutex_exit() race.
    Fixed deadlock between 'zpool export' and 'zfs list'.
    Fixed deadlock related to 'zfs rename'.
    Fixed deadlock related to z_teardown_inactive_lock.
    Fixed deadlock related to zfs_putpage().
    Fixed deadlock for meta data intensive workloads.
    Fixed panic when removing log device.
    Fixed panic in metaslab init when space_map_open returned ENXIO.
    Fixed panic due to dirtying inodes in a snapshot.
    Fixed panic in dbufstat.py.
    Fixed panic when performing ACL-to-mode translation on empty ACL.
    Fixed SEEK_HOLE misreporting hole at end of file.
    Fixed discrepancies in futimens() timestamps.
    Fixed pool free space leak.
    Fixed L2ARC compressed buffer leak.
    Fixed 'zpool history -i' hang.
    Fixed 'zpool import -t' it should not update the cache file.
    Fixed multiple 'zfs send/recv' failure modes.
    Fixed dracut to export ZFS root pool on shutdown.
    Fixed restore_object now performed in a single transaction.
    Fixed zvol symbolic link handling.
    Fixed per-filesystem memory reclaim.
    Fixed removal of SA in sa_modify_attrs().
    Fixed readdir for .zfs/snapshot directory.
    Fixed maximum zvol transfer size.
    Fixed dmu_sync'ed holes should retain birth time.
    Fixed ctor/dtor called on each alloc/free not once per slab.
    Fixed ZED io-spare.sh script.
    Fixed spurious timeouts when create large pools.
    Improved 'zpool add' dry-run mode.
    Improved 'zfs send -p' to only send properties for sent snapshots.
    Improved 'zpool import' hostid behavior.
    Improved 'zpool import -XF' behavior.
    Improved 'zpool import' when multiple duplicate labels are found.
    Improved 'zfs receive' performance by increasing pipe buffer size.
    Improved 'zfs send' for small blocks by increasing prefetch.
    Improved SPL kmem implementation.
    Improved zvol_get_stats() performance.
    Improved ashift auto-detect and management.
    Improved documentation in man pages.
    Improved handling of damaged block pointers.
    Improved ZED logging
    Rate limited debug backtraces to avoid impacting performance.
    Assorted performance improvements.
    Substantial changes to realign code base with illumos.
    Over 200 additional bug fixes.

Version 0.6.4.1 Relase Notes
https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.4.1

Bug Fixes:
  Fixed io-spare.sh script for ZED.
  Fixed multiple deadlocks which might occur when reclaiming memory.
  Fixed excessive CPU usage for meta data heavy workloads when
reclaiming the ARC.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ifebb473ba5508cc9a105a5dd08a6598bbda17db3
Reviewed-on: http://review.whamcloud.com/14505
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>