LU-2778 tests: UNEXCEPT conf-sanity/57b

Due to fixes for LU-2059 this test should now pass consistantly on
ZFS.

Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=conf-sanity
Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=conf-sanity
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ia9e567a5692109458b8cbf9fc034d571c52ea130
Reviewed-on: http://review.whamcloud.com/10357
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6541 utils: fix potential memory leaks with realloc()

The semantics of realloc are to return NULL when the allocation fails,
but won't free the original allocation:

"The realloc() function returns a pointer to the newly allocated memory,
which is suitably aligned for any built-in type and may be different
from ptr, or NULL if the request fails. If size was equal to 0, either
NULL or a pointer suitable to be passed to free() is returned. If
realloc() fails, the original block is left untouched; it is not freed
or moved."

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib348acad13c866a2904e023ac5822f23d13b38ca
Reviewed-on: http://review.whamcloud.com/14814
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6539 quota: fix incorrect format string used for short type

The %u in format string requires 'unsigned int' but the argument type
is 'short'.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie84b5f2d46809eb1a840d07ffa0b45f6ada1379e
Reviewed-on: http://review.whamcloud.com/14810
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: remove last of user land code in libcfs module

Currently user land code still exist in libcfs source for
workitems and some of the linux specific debug code. We
can now safely remove the last bit of this code.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Id137071f52cbce3a4c6de323b5412976fb687c54
Reviewed-on: http://review.whamcloud.com/15197
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5829 libcfs: remove unnecessary EXPORT_SYMBOL

A lot of symbols don't need to be exported at all because they are
only used in the module they belong to.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ie386a75bb8d9b5b77360ea2c714aba886dd8f780
Reviewed-on: http://review.whamcloud.com/13319
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6800 obdclass: change spinlock of key to rwlock

Most of the time, keys are never changed. So rwlock might be
better for the concurrency of key read.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I5c86c7333884a44ec8e9017d532d6eea5a119cfa
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: http://review.whamcloud.com/15558
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6837 update: re-lookup the dtrq in the replay list.

In insert_update_records_to_replay_list(), if the
dtrq request has been added to the list by another
thread, it should lookup the request in the list
again, otherwise it will cause the following sub req
lookup panic, because the current dtrq was just freed.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I766cddbe5abcd96e97a611985032208585573b43
Reviewed-on: http://review.whamcloud.com/15573
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4340 tests: Fix test_69 of conf-sanity test

Since 100% inodes were used with default value of MDSSIZE and OSTSIZE,
test_69 of conf-sanity test was failing with ENOSPC error.

Use decreased value of num_create to create 20000*3 files.

Also added umount of client before OST replace operation so that
eviction/recovery of client is not seen.

Seagate-bug-id: MRP-1698
Signed-off-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I4568cfb3d259ae248f26d5463cd3291db60f2d5b
Reviewed-on: http://review.whamcloud.com/15487
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6660 rename: DNE2 should return -EXDEV upon remote rename

DNE2 MDS should return -EXDEV upon remote rename, so that old
client can do rename with copy and delete, instead of fail
with -EREMOTE.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I68e8e99259065922f31bee5343be309380715674
Reviewed-on: http://review.whamcloud.com/15323
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-5946 lprocfs: free expired jobstats after /proc read

Currently, jobstats for expired jobids are only freed when a new
jobid arrives.  If no new jobids have arrived in a long time, the
expired jobstats won't be freed even though the /proc/.../job_stats
file is read repeatedly.  This shouldn't be a problem for steady job
submission patterns, but if there is a big gap in new jobid arrival
then expired jobids shouldn't be left around such a long time.

Free expired stats in lprocfs_jobstats_seq_release() after job_stats
is read, and move normal expiry to the arrival of new jobids instead
of on every stat to reduce overhead.  If jobs arrive frequently then
expired jobs will also be freed frequently, and if jobs arrive
slowly then there wan't be many expired jobs accumulating.

Rename job_iter_callback() to job_cleanup_iter_callback() for name
consistency with its caller and to make it more clear what it is for.

Instead of just passing a "bool force" param to lprocfs_job_cleanup(),
pass the number of seconds before which stats should be expired.
That allows lprocfs_job_cleanup() to be used at all callers instead
of having to have custom iterators in several places.

Avoid having multiple threads do cleanup of the jobstats concurrently.
This is not itself dangerous, since hash locking avoids races, but it
is inefficient to have multiple threads contending on the hash locks.

Add comment blocks for modified functions.

Remove some LASSERT() calls and replace with checks and error returns,
preferably at initialization time rather than during later usage.

Fix miscellaneous coding style issues.

Add a test for jobstats expiry after /proc read, and add a DNE test.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Icfd65ed7cd7beb26a9b566667e2686afe23ebbe5
Reviewed-on: http://review.whamcloud.com/12821
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-5638 tests: Skip sanity-quota tests for ZFS

sanity-quota tests 11, 33, 34 and 35 are failing frequently
in review-zfs-part-1 test groups. We are temporarily disabling
these tests until a fix is submitted.

Note that LU-5638 is thought to be caused by
LU-2435 inode accounting in osd-zfs is racy.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I3eaa3f89ca82123f8578b54006451143ce005349
Reviewed-on: http://review.whamcloud.com/15590
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6325 ptlrpc: make ptlrpcd threads cpt-aware

On NUMA systems, the placement of worker threads relative to the
memory they use greatly affects performance. The CPT mechanism can be
used to constrain a number of Lustre thread types, and this change
makes it possible to configure the placement of ptlrpcd threads in a
similar manner.

To simplify the code changes, the global structures used to manage
ptlrpcd threads are changed to one per CPT. In particular this means
there will be one ptlrpcd recovery thread per CPT.

To prevent ptlrpcd threads from wandering all over the system, all
ptlrpcd thread are bound to a CPT. Note that some CPT configuration
is always created, but the defaults are not likely to be correct for
a NUMA system. After discussing the options with Liang Zhen we
decided that we would not bind ptlrpcd threads to specific CPUs,
and rather trust the kernel scheduler to migrate ptlrpcd threads.

With all ptlrpcd threads bound to a CPT, but not to specific CPUs,
the load policy mechanism can be radically simplified:

- PDL_POLICY_LOCAL and PDL_POLICY_ROUND are currently identical.
- PDL_POLICY_ROUND, if fully implemented, would cost us the locality
  we are trying to achieve, so most or all calls using this policy
  would have to be changed to PDL_POLICY_LOCAL.
- PDL_POLICY_PREFERRED is not used, and cannot be implemented without
  binding ptlrpcd threads to individual CPUs.
- PDL_POLICY_SAME is rarely used, and cannot be implemented without
  binding ptlrpcd threads to individual CPUs.

The partner mechanism is also updated, because now all ptlrpcd
threads are "bound" threads. The only difference between the various
bind policies, PDB_POLICY_NONE, PDB_POLICY_FULL, PDB_POLICY_PAIR, and
PDB_POLICY_NEIGHBOR, is the number of partner threads. The bind
policy is replaced with a tunable that directly specifies the size of
the groups of ptlrpcd partner threads.

Ensure that the ptlrpc_request_set for a ptlrpcd thread is created on
the same CPT that the thread will work on. When threads are bound to
specific nodes and/or CPUs in a NUMA system, it pays to ensure that
the datastructures used by these threads are also on the same node.

Visible changes:

* ptlrpcd thread names include the CPT number, for example
  "ptlrpcd_02_07". In this case the "07" is relative to the CPT, and
  not a CPU number.

Tunables added:

* ptlrpcd_cpts (string): A CPT string describing the CPU partitions
  that ptlrpcd threads should run on. Used to make ptlrpcd threads
  run on a subset of all CPTs.

* ptlrpcd_per_cpt_max (int): The maximum number of ptlrpcd threads
  to run in a CPT.

* ptlrpcd_partner_group_size (int): The desired number of threads
  in each ptlrpcd partner thread group. Default is 2, corresponding
  to the old PDB_POLICY_PAIR. A negative value makes all ptlrpcd
  threads in a CPT partners of each other.

Tunables obsoleted:

* max_ptlrpcds: The new ptlrcpd_per_cpt_max can be used to obtain the
  same effect.

* ptlrpcd_bind_policy: The new ptlrpcd_partner_group_size can be used
  to obtain the same effect.

Internal interface changes:

* pdb_policy_t and related code have been removed. Groups of partner
  ptlrpcd threads are still created, and all threads in a partner
  group are bound on the same CPT. The ptlrpcd threads bound to a
  CPT are typically divided into several partner groups. The partner
  groups on a CPT all have an equal number of ptlrpcd threads.

* pdl_policy_t and related code have been removed. Since ptlrpcd
  threads are not bound to a specific CPU, all the code that avoids
  scheduling on the current CPU (or attempts to do so) has been
  removed as non-functional. A simplified form of PDL_POLICY_LOCAL
  is kept as the only load policy.

* LIOD_BIND and related code have been removed. All ptlrpcd threads
  are now bound to a CPT, and no additional binding policy is
  implemented.

* ptlrpc_prep_set(): Changed to allocate a ptlrpc_request_set
  on the current CPT.

* ptlrpcd(): If an error is encountered before entering the main loop
  store the error in pc_error before exiting.

* ptlrpcd_start(): Check pc_error to verify that the ptlrpcd thread
  has successfully entered its main loop.

* ptlrpcd_init(): Initialize the struct ptlrpcd_ctl for all threads
  for a CPT before starting any of them. This closes a race during
  startup where a partner thread could reference a non-initialized
  struct ptlrpcd_ctl.

Signed-off-by: Olaf Weber <olaf@sgi.com>
Change-Id: I3ac40ea56f9c792c3e7c36967e2e1f20105c566c
Reviewed-on: http://review.whamcloud.com/13972
Tested-by: Jenkins
Reviewed-by: Grégoire Pichon <gregoire.pichon@bull.net>
Reviewed-by: Stephen Champion <schamp@sgi.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6816 utils: remove libzfs_load_module() call

Remove libzfs_load_module() call from the OSD.  The function is no
longer exported by libzfs2 in ZFS master, as it was intended as an
internal function to begin with.  libzfs_init(), which is already
being used to set up the OSD, loads the module if necessary.

The intent of the call to libzfs_load_module apparently was to make
zfs_init() fail silently if ZFS was improperly installed (e.g. the
userspace libraries were missing), but fail with an error message if
ZFS initialization failed.  However it's not clear how this could ever
work.

Handling such error conditions will be dealt with in a separate ticket
as it's a larger concern.

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: I97a2de7cd1a32c8251d805192aaf557f48840584
Reviewed-on: http://review.whamcloud.com/15539
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-6602 osp: change lgh_hdr_lock to mutex

Change lgh_hdr_lock from spinlock to mutex, because in
osp_md_write_obj(), it might be blocked by memory
allocation.

Reorganize the llog_cache_rec process, so lgh_hdr_mutex
will protect both setbit and update llog header, to avoid
conflicts between add records and cancel llog records.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ic12d7d768546c67a858cb235929de8ae6aeaa1aa
Reviewed-on: http://review.whamcloud.com/15274
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6467 test: skip s-l test_9 for interop with b2_5

Master uses layout LFSCK for sanity-lfsck test_9, that is not
support old version, should be skipped.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I41497b27d669813d321208e291792f200f7ac863
Reviewed-on: http://review.whamcloud.com/14500
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4045 mdd: use same attr to declare setattr

in mdd_attr_set, different lu_attr ("la" and "la_copy") is used
for declaring "setattr" and the "setattr" itself, the "LA_MODE"
could be added to "la_copy" by "mdd_fix_attr" in some case and
cause the "setxattr" in "mdd_attr_set_internal" is processed
without declaration.

Change-Id: Ic4441e0c72e8d8209d8f5e60f71ab09dc7987163
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/14566
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6794 nfs: ASSERTION( atomic_read(&d->ld_ref) == 0 ) failed

ll_iget_for_nfs() can call unbalanced iput() causing memory
leaks. This patch removes this iput() call.

Also, avoid unhashing disconnected dentries in
d_lustre_invalidate(), which is another source of memory
leaks.

One of the symptoms of the leak is the following crash pattern:
<0>LustreError: 14812:0:(lu_object.c:1251:lu_device_fini())
ASSERTION( atomic_read(&d->ld_ref) == 0 ) failed: Refcount is 1
<0>LustreError: 14812:0:(lu_object.c:1251:lu_device_fini()) LBUG
<4>Pid: 14812, comm: umount
<4>
<4>Call Trace:
<4> [<ffffffffa11bc895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
<4> [<ffffffffa11bce97>] lbug_with_loc+0x47/0xb0 [libcfs]
<4> [<ffffffffa1458a48>] lu_device_fini+0xb8/0xc0 [obdclass]
<4> [<ffffffffa08e9ab2>] lovsub_device_free+0x52/0x220 [lov]
<4> [<ffffffffa145c64e>] lu_stack_fini+0x7e/0xc0 [obdclass]
<4> [<ffffffffa146356e>] cl_stack_fini+0xe/0x10 [obdclass]
<4> [<ffffffffa08bc1a8>] lov_device_fini+0x58/0x120 [lov]
<4> [<ffffffffa145c619>] lu_stack_fini+0x49/0xc0 [obdclass]
<4> [<ffffffffa146356e>] cl_stack_fini+0xe/0x10 [obdclass]
<4> [<ffffffffa0e1279d>] cl_sb_fini+0x6d/0x190 [lustre]
<4> [<ffffffffa0dd34bc>] ll_put_super+0x1bc/0x11e0 [lustre]
<4> [<ffffffff811cd0f2>] ? fsnotify_clear_marks_by_inode+0x32/0xf0
<4> [<ffffffff811a59df>] ? destroy_inode+0x2f/0x60
<4> [<ffffffff811a5eac>] ? dispose_list+0xfc/0x120
<4> [<ffffffff811a62a6>] ? invalidate_inodes+0xf6/0x190
<4> [<ffffffff8118b35b>] generic_shutdown_super+0x5b/0xe0
<4> [<ffffffff8118b446>] kill_anon_super+0x16/0x60
<4> [<ffffffffa144e7ba>] lustre_kill_super+0x4a/0x60 [obdclass]
<4> [<ffffffff8118bbe7>] deactivate_super+0x57/0x80
<4> [<ffffffff811aabef>] mntput_no_expire+0xbf/0x110
<4> [<ffffffff811ab73b>] sys_umount+0x7b/0x3a0
<4> [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b

Xyratex-bug-id: MRP-2414
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Change-Id: Ie8bc428014f3897dc108568f55db9a9ed93c932b
Reviewed-on: http://review.whamcloud.com/15480
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4416 ldiskfs: enable support for SLES12

This patch adds support for the 3.12-based SLE12 kernel. Additonally
this work separates out the basic needed patches for the upstream
3.12 kernel which can be shared with other distros as well as future
upstream kerrnels. This will help with book keeping ldiskfs patches
that need to be pushed upstream.

kernel version SLES12 [3.12.43-52.6]

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I2b8d83cddf8fd27617657348ea6d7c3a48e9472c
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-on: http://review.whamcloud.com/10165
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6805 ptlrpc: use smp unsafe at_init only for initialization

at_init() is not smp safe, so it is not supposed to be used anywhere
but in at initialization.
Add at_reinit() - safe version of at_init().

Xyratex-bug-id: MRP-2726
Signed-off-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Change-Id: I90f872bd217d1e8008f8d35bb4606d168c9bf2ad
Reviewed-on: http://review.whamcloud.com/15522
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>

LU-6747 osd-zfs: initialize obd_statfs in osd_statfs()

The osd-zfs osd_statfs() method wasn't initializing all of the fields
of struct obd_statfs, and in some cases could be returning random
data to the client.  For most of the fields this doesn't matter, but
the os_state field could contain OS_STATE_READONLY and prevent the
MDS from allocating objects there.

Reorder the assignment of struct obd_statfs fields to be in struct
order so that it is more clear which fields have already been set.

Also fixup osd-ldiskfs obd_statfs() to initialize os_maxbytes as
osd-zfs does, even though it currently isn't used.

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: If4ec6b5fc9c31676267e63be34d00275b4ac8dfb
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-on: http://review.whamcloud.com/15346
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5092 nodemap: convert member hash to a list

The members of a nodemap were stored in a hash, but they were only
iterated against, not accessed by key. This patch converts the hash
to a list in order to simplify things.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: Ic1df4035d1c53a7c7966c4c528d20cefc7258d0e
Reviewed-on: http://review.whamcloud.com/14885
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6775 ldlm: reduce mem footprint of ldlm_resource

- Removing the unused lr_most_restr, reduced 4 bytes;
- Allocating lr_itree only for LDLM_EXTENT resource, reduced
  120 bytes;
- Moving fields around to eliminate holes, eliminated 3 holes,
  reduced 4 bytes;
- Making lr_contention_time and lr_lvb_inode into union,
  reduced 8 bytes;

Reduced 136 bytes in total.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I232a2669d9f5eaee35eb9ff8313d86d4ac297bc1
Reviewed-on: http://review.whamcloud.com/15485
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6766 utils: remove llapi_quotachown() and cb_quotachown()

The SYS_chown syscall is not available on arm64, but the functions
llapi_quotachown() and cb_quotachown() are not needed anymore and
should have been removed in the patch
http://review.whamcloud.com/4022

Change-Id: I11fa4ab901327c1c8bcfb7e6dfe14a955e2f2618
Signed-off-by: Arnaud Guignard <arnaud.guignard@cea.fr>
Reviewed-on: http://review.whamcloud.com/15396
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6755 kernel: kernel update RHEL7.1 [3.10.0-229.7.2.el7]

Update RHEL7.1 kernel to 3.10.0-229.7.2.el7.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ie922ed7c04394d4ffb02d28fd213c8a1da884bdd
Reviewed-on: http://review.whamcloud.com/15388
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6763 lnet: test for sk_sleep presence in compact-2.6.h

Like Lustre external infiniband stacks create a compability
layer to handle various distributions and kernel versions.
Due to this besides the linux kernel sk_sleep can also be
defined by the external infiniband stack. We need to examine
the infiniband stack's headers to see if sk_sleep is available
there as well.

Change-Id: I0317349cfe13f281c56a8aec17dc330a0e3c759d
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15386
Tested-by: Jenkins
Reviewed-by: Chris Horn <hornc@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6714 llog: fix wrong offset in llog_process_thread()

- llh_cat_idx may become bigger than llog bitmap size in
  llog_cat_set_first_idx() function
- it is wrong to use previous cur_offset as new buffer offset,
  new offset should be calculated from value returned by
  llog_next_block().
- optimize llog_skip_over() to find llog entry offset by index
  for llog with fixed-size records.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I0207ddd7e09161e25a2f5d56d69db4c993e650c3
Reviewed-on: http://review.whamcloud.com/15316
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5988 obdclass: drop malloc fault and page accounting

Remove the OBD_ALLOC() memory allocation fault injection since this
is also possible with existing kernel mechanisms and it is no longer
useful to have Lustre-specific code to do this.

Remove the page allocation counters and OBD_PAGE_{ALLOC,FREE} helpers
that are almost entirely unused since the move to CLIO.

Update the max memory allocation counter on /proc read, so that it
doesn't show as lower than the current allocation, in case it hasn't
been updated since the previous ping.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I024919d1e81730a9388e7999938a4cca1e1cab07
Reviewed-on: http://review.whamcloud.com/15110
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 lnet: fixed space coding style issues for lib-msg.c

Linux commit: a1caff6de83acca44faecea14b72724cebb9972e

staging: lustre: lnet: lnet: fixed space coding style issues

Fixed a coding style issue with spaces between function names and arguments:

func (x) -> func(x)

Signed-off-by: Samuel Dominguez <samuel.dominguez.lorenzo@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I0952bcf15e93705e77abe5a3fe0d181581df9973
Reviewed-on: http://review.whamcloud.com/15092
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6543 utils: fix memory leak in lfs_setstripe

Allocated buffer param should be freed before return.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I22d21fb12c0f62330a7c2ca6296f571db71084f2
Reviewed-on: http://review.whamcloud.com/14809
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5956 o2iblnd: wrong uses of kib_tx_t::tx_nfrags

The kib_tx_t::tx_nfrags field is the # entries in
the kib_tx_t::tx_frags array, rather than # DMA
mapped entries. So kiblnd_send/kiblnd_recv should
use kib_rdma_desc_t::rd_nfrags instead.

Signed-off-by: Isaac Huang <he.huang@intel.com>
Change-Id: Ib395636af8bcb0b448f4bb3338b0ddd9c5cfa903
Reviewed-on: http://review.whamcloud.com/12857
Tested-by: Jenkins
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5794 osd: additional checks to verify credits calculation

every operation counts credits consumed, so we can check when
credits are exceeded. now the number of declarations doesn't
matter for the checks, it's only credits. i.e. it's OK to
declare a single write for a specific range and then make many
writes to that range within a single transaction.

the patch fixes few issues on osd-zfs making it impossible to
declare change to an object being created in a same transaction.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I714ccae7ce957e639938df3bed0c4e8fb49d3454
Reviewed-on: http://review.whamcloud.com/12169
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5580 ptlrpc: policy switch directly in tbf

Former version can't change nid to jobid(or reversely) directly in
tbf, it need to change to FIFO(or any other policy) first. This patch
fix this problem. It save the policy argument(like "nid" in tbf),
and if the policy start with a new argument(like "jobid"), it will
stop the former policy and start a new one.

This is also adapt to any other nrs policies if they have arguments.

Signed-off-by: Wu Libin <lwu@ddn.com>
Change-Id: I645af94d774f60e2081e3716b2f249fcab4930d4
Reviewed-on: http://review.whamcloud.com/11749
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4306 tests: improve test_4a of s-q

test_4a of s-q used "sync; sleep 1; sync" to sync all data, that
could stall for an unexpected long time, this patch changes it
to sync_all_data for better time-efficiency.

This patch also modified the script to report proper error when
the grace has already expired before trying create new file.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I4aa2f69933be37d5f98af8172ed6a20781fe6e38
Reviewed-on: http://review.whamcloud.com/15483
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 gss: remove libcfs.h from gss utilities code

The gss utility code has nearly all the user land headers
it needs. No need to include libcfs.h anymore.

Change-Id: I2d08d8bf7d4f6165134eb8d445563497db3bf65d
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15368
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6735 lnet: select appropriate optimization options for ko2iblnd

In order to support different IB cards in the best way we need to have
ability to select appropriate optimization options for ko2iblnd
according IB cards installed on the system.

The different defaults for different IB cards are selected by aliases
from /etc/modprobe.d/ko2iblnd.conf and customer can change them manually.

In this patch the general mechanism of selecting tunables is proposed.
If TrueScale IB cards are inserted a new optimal tunables are selected.

With proposed tunables the performance with TrueScale IB cards gets up
from 900 MB/sec to 2.2 GB/sec.

Change-Id: Ie296599f8b0c923100aac41994cbb9b49f99decf
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/15313
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>

LU-6275 obdclass: don't forget to remove LWP mgc config

We may take an MGC lock, but failed with take a config from MGS.
mgc will cancel a lock in that situation and but lock added to
re-enqueue list.

Shutdown (cleanup) started as result lustre_start_lwp failed, but
cleanup forget to call lustre_end_log as lsi_lwp_started don't set
so - lock/cld still on enqueue list, but super block is destroyed.

Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Change-Id: I9e72d2526f5a21e66a326532d4770c07e2b7cca2
Reviewed-on: http://review.whamcloud.com/13863
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6433 quota: handle QUOTA_DQACQ in READPAGE portal

MDS sends quota request to quota master, which can be
the same host. Deadlock is possible if all other MDS
threads are waiting for lock taken by first thread.

Lets use MDS_READPAGE portal to avoid the deadlock.

Xyratex-bug-id: MRP-2433
Change-Id: Ib2266a6771cfc5a70bae23206b1144bfd45f9ff3
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: http://review.whamcloud.com/14369
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ann Koehler <amk@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6570 tests: fix 40a in sanityn.sh

In sanityn.sh 40a, it should cache the parent lookup LOCK
Before check parallel directory lock, otherwise parent
revalidation might cause lock conflicts on MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ib4c7c667d4e1dd821df96497e06459db0d5c33f9
Reviewed-on: http://review.whamcloud.com/15366
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6825 ofd: statfs needs NOT check layout LFSCK status

The ofd_statfs() needs NOT to check whether the layout LFSCK is
rebuilding the LAST_ID file(s) on the OST or not, because:
1) Such checking will be done when the MDT wants to pre-allocate
   OST-objects via ofd_create_hdl().
2) The STATFS request may be from client, if ofd_statfs() returns
   -ENOSPC to the client under the case of layout LFSCK rebuilding
   the LAST_ID file(s), the client may be confused.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iae9eb8485f3b971fda3bcd299df28db7ad4d80a0
Reviewed-on: http://review.whamcloud.com/15349
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6415 utils: deny non-root user for changelog operations

To avoid potential security problems, non-privileged users should
have no permission to run 'lfs changelog' & 'lfs changelog_clear'.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I5f38ba5b139f2f3b6495d3c97d82a47daecf8187
Reviewed-on: http://review.whamcloud.com/14280
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6073 tests: use sys/xattr.h in tests/mpi/*.c

In lustre/tests/mpi/mdsrate.c include sys/xattr.h rather than
attr/xattr.h since the former is more likely to be installed that the
latter and provides the needed declarations.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3b3f6a2cc2a5a7153bb3dbff535d7c1b3472f9b8
Reviewed-on: http://review.whamcloud.com/15391
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6245 utils: remove libcfs.h from lustre utilities code

Since user land libcfs is mostly gone libcfs.h can be
removed from the code that is used to create the lustre
utilities such as lfs and mount.lustre.

Change-Id: I5e0dd7d9ad44e988b2045906d2179dc3e2538d38
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15217
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6705 lod: re-order lodsub recovery cleanup

Before MDT stop recovery thread, it should disconnect
the OSP and stop the sub_recovery thread, otherwise
the recovery thread will be blocked by sub_recovery
thread.

The the right steps to stop the MDT recovery thread is
(see mdt_device_fini())

1. Do stack pre_cleanup (by LCFG_PRE_CLEANUP), to disconnect
OSP and stop sub_recovery thread (used for collecting update
records).
2. Stop the recovery thread.
3. Cleanup the update llog ctxt in mdt_stack_fini()
(by LCFG_CLEANUP).

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I4e938cb87ab02d388bce16bd15d181fd7fcc783b
Reviewed-on: http://review.whamcloud.com/15210
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 tests: remove libcfs.h from lctl code

Since user land libcfs is mostly gone libcfs.h can be
removed from the code that is used by lctl.

Change-Id: Ieb2f3873177dfc2c7fa3036568e263f4ba27e1ce
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15208
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 lnet: Avoid nid range related forward declarations in nidstring.c

Since forward declarations are frowned on upstream we move
the NID range handling to near the start of the nidstring.c
file.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I03961e1f5a89a4bf60873fcc3fc1522542e4931a
Reviewed-on: http://review.whamcloud.com/15086
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6489 osd-ldiskfs: check max_page_index correctly

This patch fixes a issue that original code check
max_page_index against incorrect variable.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I4b164d6187ef4a1c343d4b5803b8915283394037
Reviewed-on: http://review.whamcloud.com/14731
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-977 lod: Patch to protect lqr_start_idx

Protect lqr_start_idx to avoid the imblance in allocating objects
on OSTs with round-robin algorithm

Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Change-Id: I689ba1c4e8c9224cc67badba7fd6cf45e64dd7b6
Seagate-bug-id: MRP-2471
Reviewed-on: http://review.whamcloud.com/14636
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6750 zfs: add stop trans callback

Add stop transaction callback in OSD-ZFS, so when the
osd transaction is stopped, the top transaction
will be notified.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I56b8d964b3c4dd24e3c56429b7f1a8960d3202cb
Reviewed-on: http://review.whamcloud.com/15367
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6717 llog: Create update llog synchronously

Create remote update llog object synchronously, so
it will make sure the llog object exists, before
the cross-MDT operations will write any update llog
records.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I690e94e68b8b64dfe94412f58448f3867f61ff3e
Reviewed-on: http://review.whamcloud.com/15278
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6674 utils: fix of uninitilized lmm structure usage

Fill with "fake" the "use the default" values since the lmm struct
is zeroed out at this point and not filled by ioctl.

Change-Id: I9df005446c5d03bdf561823842afe98ab8d352c1
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: http://review.whamcloud.com/15109
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6109 osd-ldiskfs: handle no fid-in-dirent correctly

The commit: 4d408c9aed9adaf1f4e2ea87851728a1cf662594
introduced a issue in '..' no fid-in-dirent case.
Since it just deal with remote case. So the '..'
would disappeared if it is a local object and
no fid-in-dirent.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ib2b0be1c43986472cb674dd86d32f023a4b26a34
Reviewed-on: http://review.whamcloud.com/13987
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6207 osd: add osd_ost_fini in osd_obj_map_init

In case of ENOSPC in osd_mdt_init->simple_mkdir("REMOTE_PARENT_DIR")
missed dput(for "O") causes kernel panic with msg:
BUG: Dentry ffff88005d6b3e40{i=b2,n=O} still in use (1)

Change-Id: Ibd5391e781e19efe48e9a9700ee4565b46c2f0b5
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Xyratex-bug-id: MRP-2059
Reviewed-on: http://es-gerrit.xyus.xyratex.com:8080/4431
Tested-by: Jenkins
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Reviewed-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Tested-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-on: http://review.whamcloud.com/13649
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>

New tag 2.7.56

Change-Id: Ib5d0eb7a06314f347f87eb0ed103435c61fc7033

LU-6722 ldiskfs: declare credits for quota when destroy inode

When destroy the inode, if it has large EA that occupies another
separated block, then it calls ldiskfs_xattr_delete_inode(). But
such function does not consider the quota modification as to that
before making the group descriptor dirty, the persevered credits
becomes zero, so jbd2_journal_dirty_metadata() failed as "-ENOSPC",
then the subsequent journal abort makes the system as RDONLY.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0567d6b6d54cd57b25524c3d58190926e3505867
Reviewed-on: http://review.whamcloud.com/15334
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6454 test: double default OSTSIZE

Double the default test OST size from 200000KB to 400000KB.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I76a19086e789fe60519b301ee2b1d8f186e13062
Reviewed-on: http://review.whamcloud.com/14650
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6380 scrub: handle update_log during initial scrub

The normal OI scrub will not run until recovery finished. But the
update logs (update_log, and sub-items under update_log_dir) will
be accessed during the recovery. So if the the OI mapping for the
update log cannot be handled before the recovery, then the normal
OI scrub cannot be started/resumed. To resolve above dependency,
the initial OI scrub (that will be triggered by the mount process
with sync mode before the recovery) will handle the update logs'
OI mappings in advance.

In DNE2, each MDT will have a special FID for update log, during
recovery, the failover MDT can retrieve update logs on remote MDT
by this FID, that said the FID is special and it also has to be
located globally (not local FID). So the MDT index is packed into
the FID (as f_oid), then FLD can locate the special FID. To handle
such special FIDs, OI scrub needs to know the target index.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iac71c20c1f1631efbe19a03e380822cead6c00d1
Reviewed-on: http://review.whamcloud.com/15192
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6780 ptlrpc: Do not resend req with allow_replay

If the request is allowed to be sent during recovery,
and it is not timeout yet, then we do not need to
resend it in the final stage of recovery.

Unnecessary resend will cause the bulk request to resend the
request with different mbit, but same xid, and on the remote
server side, it will drop such request with same xid, so it
will never get the bulk in this case.

re-enable the multiple MDT failover cases in replay-single.sh

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I96900448fc16b0e2a336a42c8ff977ec56941427
Reviewed-on: http://review.whamcloud.com/15458
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6602 update: split update llog record

If the update llog record size exceeds the limit (llog
chunk size), which usually happens when creating striped
directory with large stripe count , then it will be split
into mulitple records, and these update records will have
the same batchid.

During recovery, these records will be combined and
constructed into the updates of one operation, then
do update replay.

Allow multiples stripes in a single MDT, so it can verify
creating large stripes in small scale test environment.

Add sanity 300j/300k and replay-single 116 to verify it.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I86ca2594fe62d5b921e794de4cd88981d91f7677
Reviewed-on: http://review.whamcloud.com/15162
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6602 llog: increase update llog chunk size

Increase DNE update llog chunk size from 8KB to 32KB, so
one cross-MDT operation can includes more update records,
and it can create striped directory with more stripe count.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I01d1435f1249c3048ac0a412bdb53f710e6e98b3
Reviewed-on: http://review.whamcloud.com/15161
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 osp: transfer updates with bulk RPC

Send update request with bulk RPC(iovec), and update request
will be split into multiple buffers, so OSP does not need to
allocate big contingous buffer to store the update request.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I70995a552ee6fdd77f4c0a84d2435e5963e4fd6d
Reviewed-on: http://review.whamcloud.com/13786
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6744 lov: init LOV stripe type beforehand

When lu_object_alloc() reaches to LOV object init, we need initialize
its stripe type beforehand, so that if something wrong in the conf
buffer, the object chain need to be traversed to free what has been
allocated, with LOV object type be set as LLT_EMPTY, and when the LOV
part is reached, it won't panic without knowing what stripe type it
is.

This patch also improves debug messages in lsm_unpackmd_common(), and
does not return error if the LOV device is still processing config
log while trying to verify a layout buffer.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic814f89ebc960cc4c04835141d915f86c35ca39d
Reviewed-on: http://review.whamcloud.com/15362
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3881 tests: single-node minimal config in sanity-hsm

Patch to allow for a minimal configuration to be
automatically setup in sanity-hsm.sh, and for example
to permit the single-node sessions runs like :
./lustre/tests/llmount.sh
./lustre/tests/sanity-hsm.sh

Also changed default mount-point to be monitored by agent
to become $MOUNT2 if $MOUNT_2==y.

Created get_agent_uuid() function to dynamically discover
agent's monitored mount-point.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I194d4a321ea3bcc169c3811e19b1069749b99e72
Reviewed-on: http://review.whamcloud.com/12982
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6356 mgs: fix security flavor setting for connection to mgs

Security flavor setting for connections to MGS is done with special
target '_mgs'. If set, only allow connections to MGS with the given
flavor. If not set, all flavors must be allowed.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Id47aae25f6d5591870656d9926a64e52ab789c56
Reviewed-on: http://review.whamcloud.com/14349
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3778 sptlrpc: OSP and LWP don't know sptlrpc

The sptlrpc subsystem must be initialized for OSP and LWP
connections, by calling sptlrpc_lprocfs_cliobd_attach().
Moreover, GSS related functions must not return an LBUG when
dealing with OSP and LWP OBDs.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I3d4bc635a7076addea7bbb52ac13f74a25f625ad
Reviewed-on: http://review.whamcloud.com/14040
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 mdt: support multiple modify RCPs in parallel

This patch implements the server part of the feature that allows
support of multiple modify RPCs in parallel on MDT targets.

Each target export is able to store several in-memory reply
data so that it can reconstruct several requests issued by
the client in parallel.

Additionally, a new internal file REPLY_DATA is created on
the target to store on-disk reply data. The reply data slots
in that file are managed by a bitmap (lut_reply_bitmap) and
can be used to store reply data of any client.

When target recovers, the on-disk reply data is used to
restore the in-memory reply data and ensure reconstruction
of committed operations.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I8f91666f5b0b4f7b9445a01c520d73f56d059ff3
Reviewed-on: http://review.whamcloud.com/14860
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 mdc: manage number of modify RPCs in flight

This patch is the main client part of a new feature that supports
multiple modify metadata RPCs in parallel. Its goal is to improve
metadata operations performance of a single client, while maintening
the consistency of MDT reply reconstruction and MDT recovery
mecanisms.

It allows to manage the number of modify RPCs in flight within
the client obd structure and to assign a virtual index (the tag) to
each modify RPC to help server side cleaning of reply data.

The mdc component uses this feature to send multiple modify RPCs
in parallel.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ia707e39770e479648627611a99d0724e7070baec
Reviewed-on: http://review.whamcloud.com/14374
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5560 llite: basic support of SELinux in CLIO

Bring the ability to properly initiate security context
on SELinux-enabled client and store it on server side via
extended attribute.

Security context initialization is not atomic, but that would
require a wire protocol change to send security label in the
creation request.

Filter out security.selinux from xattr cache as it is
already cached in system slab.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Ie6db97be6f78b0b33da2ae54883a4051a2576af5
Reviewed-on: http://review.whamcloud.com/11648
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 ptlrpc: mbits is sent within ptlrpc_body

ptlrpc is using rq_xid as matchbits of bulk data, which means it
has to change rq_xid for bulk resend to avoid several bulk data
landing into the same buffer from different resends.

This patch uses one of reserved __u64 of ptlrpc_body to transfer
mbits to peer, matchbits is now separated from xid. With this change,
ptlrpc can keep rq_xid unchanged on resend, it only updates matchbits
for bulk data.

This protocol change is only applied if both sides of connection have
OBD_CONNECT_BULK_MBITS, otherwise, ptlrpc still uses old approach and
update xid while resending bulk.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Ide0b3f490368babeb6ac1b4ea6953f8f9aacf81a
Reviewed-on: http://review.whamcloud.com/15421
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5823 llite: Remove access of stripe in ll_setattr_raw

In ll_setattr_raw(), it needs to know if a file is released
when the file is being truncated. It used to get this information
by accessing lov_stripe_md. This turns out not necessary. This
patch removes the access of lov_stripe_md and solves the problem
in lov_io_init_released().

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I05b532fd137889d13e8ecb35585f4d1b09958583
Reviewed-on: http://review.whamcloud.com/13514
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6377 llite: clear LLIF_DATA_MODIFIED in atomic

This flag should be cleared atomically after the op_data flag
MDS_DATA_MODIFIED is packed. Otherwise, if there exists an
operation to dirty the file again, the state may be missed on
the MDT.

Stop using spin lock lli_lock to protect operations of changing
file flags; using bit operations instead.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I3a3129e24cea89b0ebaae1b7816d892b5623b3b4
Reviewed-on: http://review.whamcloud.com/14100
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6409 nodemap: don't destroy nodemaps in cfs_hash_for_each

nodemap_destroy might sleep, but cfs_hash_for_each iterators aren't
allowed to sleep. This patch changes nodemap_cleanup_all to avoid
destroying nodemaps in its cfs_hash_for_each iterator. Instead, the
iterator creats a list of nodemaps, and then all the nodemaps in the
list are destroyed.

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I1d16f61ba7739af0494944f6515508b3f0d19ba1
Reviewed-on: http://review.whamcloud.com/14275
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-5319 ptlrpc: embed highest XID in each request

Atomically assign XIDs and put request and sending list so
we can learn the lowest unreplied XID at any point.

This allows to embed in every resquests the highest XID for
which a reply has been received and does not have an unreplied
lower-numbered XID.

This will be used by the MDT target to release in-memory
reply data corresponding to XIDs of reply received by the client.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ic88fb6db704d8e9a78a34fe16f64abb2cdffc4c4
Reviewed-on: http://review.whamcloud.com/14793
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6587 mem: refactor OBD_ALLOC_LARGE marco

Always try kmalloc first if that fails then retry
with vmalloc. Change checkpach.pl to indicate the
OBD_FREE_LARGE has deprecated. since it hasn't any
special meaning anymore and just using OBD_FREE
instead of.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I44a002fb855956bb0c4bea0f328679a5f56e4d49
Reviewed-on: http://review.whamcloud.com/14994
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6514 fid: remove misleading rc check in lprocfs_client_fid_width_seq_write

At the very end of lprocfs_client_fid_width_seq_write in the success path
there is a very misleading rc check thta not only is out of place,
but also causes the reader to think what was the goal there.
It's after we already assign the requested variable too and only
controls the debug print.

Change-Id: I89b020b85bd8948c859fc3703f716ad8def387e1
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/14604
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: frank zago <fzago@cray.com>

LU-6483 utils: add setxattr/smallwrite for mdsrate

Add --setxattr option for mdsrate, the XATTR value length is
4000.

--setxattr can be set along with --create or --mknod, which will
setxattr after file creation, and it can also be set alone which
will setxattr on existed files.

Add --smallwrite option for mdsrate --create, which will write
4000 bytes after creation, this is used to compare with
data-on-mds.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I47b13c619a17d5764c85feee8d46c2b87eab33f6
Reviewed-on: http://review.whamcloud.com/14536
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3105 mdt: do not announce support for capabilities

this is not supported yet. we shouldn't confuse the clients.

Change-Id: I9ea203935d90335f8babf5f977e9a8a39158bd70
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/14422
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6661 dne: setdirstripe should fail if not supported

If MDS doesn't support striped directory, creating striped directory
with 'lfs setdirstripe' should fail.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I3e1ea394d3d1055230859ff46277179f518692b3
Reviewed-on: http://review.whamcloud.com/15123
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 obd: rename obd_unpackmd() to md_unpackmd()

obd_unpackmd() is only implemented by LMV so move it from OBD
operations to OBD MD operations and update the prototype to refectl
the actual usage. Remove the unused function obd_free_memmd().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I0cbbb003a94f1e5a04ba24c2a49223ff8fa16da9
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/13737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6698 kernel: kernel update RHEL 6.6 [2.6.32-504.23.4.el6]

Update RHEL6.6 kernel to 2.6.32-504.23.4.el6

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iaedc1b81b74cee213650940a3e1791eb73ef40e3
Reviewed-on: http://review.whamcloud.com/15198
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5433 doc: update llapi_hsm_state_get manpage

Add more verbose descriptions of HSM status flags, and
general editing.

Signed-off-by: Robert Read <robert.read@intel.com>
Change-Id: I463adf01ac1698b1fe7b1f6f1b92bfc937aef75a
Reviewed-on: http://review.whamcloud.com/11283
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 lnet: fix deadloop in ksocknal_push

ksocknal_push() should break the loop if it can't find matching peer

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I538b1904fb3bc90572007d3c68f81258fb3d405b
Reviewed-on: http://review.whamcloud.com/10128
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: move cfs_ip_addr_* function from kernel libcfs to LNet

Both of cfs_ip_addr_parse and cfs_ip_addr_match which are located in
libcfs kernel module are used only for LNet so move this into the
nidstring handling code where it belongs. Also create user land
versions of these functions in the libcfs user land library.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Idb1be9c6ab3328f2f85e92847b5a367d91a903da
Reviewed-on: http://review.whamcloud.com/15085
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 llite: filldir_t api changed for 3.18 kernels

The filldir_t callback went from passing in a void *cookie
to struct dir_context instead. Here we enable handling this
api change in newer kernels. This functionality is used for
nfs over lustre support.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I5de025306ace61b8defdce227fe12b854caf1b3b
Reviewed-on: http://review.whamcloud.com/14681
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6215 llite: handle removal of ki_nbytes in struct kiocb

Starting in kernel version 3.19 the ki_nbytes field was
removed from struct kiocb since now that data can be
extracted from the struct bio_iter data.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Iaaf0c6dac19cffb4f4b6fc7a55cbc98cd09678a5
Reviewed-on: http://review.whamcloud.com/14683
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4389 clio: update file attributes after sync

This really only affects ZFS, because zfs does not update # of blocks
until the blocks are flushed to disk.

This patch update file's blocks attribute after OST_SYNC completes.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I2b62dff5acda2984f60d5ed5df4ad19144cd6829
Reviewed-on: http://review.whamcloud.com/12915
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6718 build: force update of lustre-osd-zfs-mount

Previous change fixed the lustre-osd-ldiskfs dependency,
but neglected lustre-osd-zfs.

Change the .spec file to make lustre-osd-zfs require a
specific version of lustre-osd-zfs-mount rather than
accepting just any version.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ice17e23fdf778962a1a988b563bfc426c5d41c29
Reviewed-on: http://review.whamcloud.com/15285
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5423 tests: add version check to sanity-sec test 4

This patch adds Lustre version check codes into sanity-sec
test 4 to make the test interoperate with servers that do not
have the following change:

Lustre-commit: 2bc5bcb7efa247fcd8cc65d013ffc9f6c33dd788
Lustre-change: http://review.whamcloud.com/12476

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Idbbf82fb4fb580b3a695b3512b54aae3602eca60
Reviewed-on: http://review.whamcloud.com/15153
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6390 ldlm: restore the ELC for enqueue

after LU-4300 enqueue does not ELC anymore, however if enqueue is
agressive (ls -la of a large dir) we may exceed lru-resize limit
quickly because LRUR shrinker and recalc are called not so often.

ELC is to be restored in enqueue.
ELC also should check for the lock weight, in addition to LRUR.
ELC can also keep "skipped" locks, i.e. once checked for the weight
and left in the lru - let LRUR take care about them later.
LRUR is to be left untouched, no weight logic, otherwise LU-5727
appears and OPEN locks do not get canceled.

Xyratex-bug-id: MRP-2550
Signed-off-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Change-Id: I19fb7c367d9d38107a4fb6d100a47c2e7b7ad640
Reviewed-on: http://review.whamcloud.com/14342
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6207 osd: add dput in osd_ost_init

In case of error(from osd_ea_fid_set) in osd_ost_init
missed dput causes kernel panic with message:
BUG: Dentry ffff880038452f00{i=280,n=O} still in use

Change-Id: I8fee0b99f5784b644a4d8773e8e358a2a4b5187d
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Xyratex-bug-id: MRP-2059
Reviewed-on: http://es-gerrit.xyus.xyratex.com:8080/3787
Reviewed-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Tested-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Tested-by: Jenkins
Reviewed-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-on: http://review.whamcloud.com/13648
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-6617 quota: remove quota migration code

The quota migration code was used to migrate old quota admin
files into new format when upgrading system from old 2.1 system.

It can be removed now, anyone want to upgrade from such an old
system, and want to keep the quota limits settings at the same
time, one has to upgrade to an intermediate release to do the
migration first.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I41699157f07eba94453a86fe77a02189807ecbe8
Reviewed-on: http://review.whamcloud.com/14854
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6445 test: display test distribution accurately

Each Linux distribution has different file and format
We will read from less common
/etc/SuSE-release for SUSE
/etc/os-release for Ubuntu, and EL6.6/EL7
/etc/system-release for EL6.5./EL6.6/EL7

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I1a4fb8418aca303ea0dbe603e58fbfa916b8e20b
Reviewed-on: http://review.whamcloud.com/14409
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6728 lfs: check if pool_name is NULL

It should check if the pool_name is NULL before
dumping pool information in lmv_dump_user_lmm().

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I2f3c016be919d099437bfdb0cf78a2a1ca88d57d
Reviewed-on: http://review.whamcloud.com/15296
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3105 mdc: remove capa support

Remove capa support from the MDC, LMV, and llite layers. Remove the
then unused capa code in lustre/obdclass/ and lustre_capa.h.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Id2c260cd11fe4cb753b77af8a42932cde650274b
Reviewed-on: http://review.whamcloud.com/6069
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6356 tgt: handle sec context requests properly

When dealing with security context initialization requests,
no export is associated with the request yet, because these
requests are sent before *_CONNECT requests.
So the normal *_common_slice handlers will not be called,
given there is no reference to the target.

So add a special case for handling of SEC_CTX_INIT, SEC_CTX_INIT_CONT
and SEC_CTX_FINI requests in tgt_handle_request0() and
tgt_request_handle(), in order to directly send the reply.

Also remove tgt_sec_ctx_handlers because they become useless.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Ic0947082c7acc188ebbf2e9cb9ff57e999b5ed3e
Reviewed-on: http://review.whamcloud.com/14041
Tested-by: Jenkins
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 update: change sync updates to async updates

1. change sync updates to async updates.

2. add tests(110,111,112,115) in replay-single to test
async update recovery.

3. Add replay-barrier for those old DNE replay tests,
so all of MDTs can be recoveried correctly.

4. enable DNE failover to dbench test (replay-single.sh 70b).

Change-Id: Ibe1ca42d60c7a0ec870b9cf4a70a71adc663b951
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/12450
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6695 jobstats: skip too long env variables

The PATH env variable can be very large, it may exceed the page
size sometimes. cfs_get_envrion() should skip such too long env
variables instead of returning error.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I43c103a647cde3afa499dec3a11312155c3cf924
Reviewed-on: http://review.whamcloud.com/15177
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 lov: remove LSM from struct lustre_md

In struct lustre_md replace the stuct lov_stripe_md *lsm member with
an opaque struct lu_buf layout which holds the layout metadata
returned by the server. Refactor the LOV object initialization and
layout change code to accommodate this. Simplify lov_unpackmd() and
supporting functions according to the reduced number of use cases.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iac44ba479556693932bae0b9878565155b609e07
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/13722
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6718 build: force update of lustre-osd-ldiskfs-mount

Change the .spec file to make lustre-osd-ldiskfs require a
specific version of lustre-osd-ldiskfs-mount rather than
accepting just any version.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If12f8cb6d1a9d7105b7bfb04503acf437e189d46
Reviewed-on: http://review.whamcloud.com/15227
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6455 test: disable IMA related tests

Disable tests where IMA xattrs cause test failures.
This is a short term workaround to avoid known test problems
until a good fix for the IMA issues is found.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I7622135218facb8ed9ed9c9056204bba389cf370
Reviewed-on: http://review.whamcloud.com/15164
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-6030 ldiskfs: support pdirop split for RHEL6.5

Lustre commit 953e97987426bcc43a74be5b14f3a71319b33825
which split the parallel directory patch into two parts
was not applied to the RHEL6.5 platform which now results
in a build failure. The simple fix is to add the new
patches into the RHEL6.5 ldiskfs series file.

Change-Id: I12c407ac00e5471945217656e4c7a203ef34e99c
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/15196
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>