LU-4863 lloop: avoid panic during blockdev_info

Change the LL_IOC_LLOOP_INFO ioctl in the lustre lloop
device driver to return an error instead of causing
panics with LASSERT().

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib50afa94960dee6fb580038d6e96ea648252d9ea
Reviewed-on: http://review.whamcloud.com/9888
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4659 mdd: rename forgets updating target linkea

MDD rename forgets updating target LinkEA if target is not unlinked,
which will cause LinkEA inconsistency.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I9d42960da2d07d551ce08baed925bc25218fc0a4
Reviewed-on: http://review.whamcloud.com/9745
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4853 ldiskfs: fix race in ldiskfs_ext_new_extent_cb

In ldiskfs_ext_calc_credits_for_insert(), we should use the 'depth'
stored in the 'path' instead from inode, because the extent tree
could have been changed when ldiskfs_ext_calc_credits_for_insert()
is called (by ldiskfs_ext_new_extent_cb()).

It was fixed in LU-2555, but the fix is missed in
sles11sp2/ext4-misc.patch

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9ca349ba52060f10fc980721317ba47e10572473
Reviewed-on: http://review.whamcloud.com/9868
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4826 lmv: Ensure lmv_intent_lookup cleans up reqp

Ensure there aren't invalid pointers hanging around after
ptlrpc_req_finished is called.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ieb16e70958d899a2d9320491dfd6276143accc55
Reviewed-on: http://review.whamcloud.com/9841
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4799 llite: Do not rate limit dirty page discard warning

Messages which are printed by ll_dirty_page_discard_warn() should not
be rate limited.  If they are rate limited, some files which may be
corrupted on client eviction will not be reported to the user.

This patch changes the CWARN to a CDEBUG to disable console message
rate limiting for this message.  The dirty page discard warnings are
already limited on a per-file basis by the function vvp_vmpage_error
which calls ll_dirty_page_discard_warn only if the ccc_object's
cob_discard_page_warned == 0.

Signed-off-by: Ryan Haasken <haasken@cray.com>
Change-Id: Ia8a13c7add89c890014170254e462ae861de0fc5
Reviewed-on: http://review.whamcloud.com/9752
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Ann Koehler <amk@cray.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4791 lod: subtract xattr overhead when calculate max EA size.

1. It needs to subtract the xattr overhead when calculate the max
EA size, otherwise, it might get oversize stripe_count, which cause
ENOSPC error in the following xattr_set.
2. Release stripe information, if xattr set fails.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ibe46b8cbc161f151eae4eb14878af4406fc78d41
Reviewed-on: http://review.whamcloud.com/9835
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4815 lfsck: pause LFSCK before server stack fini

To avoid the LFSCK to access the devices/objects during
the targets stopping.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifd8c1b1f878ac00a3c92ab6f2ab11e46003f584c
Reviewed-on: http://review.whamcloud.com/9774
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4813 lfsck: reuse OUT_PORTAL for orphan iteration

It is unnecessary to use independent request portal for LFSCK
orphan iteration, which will cause trouble for UT work in the
future and also may cause interoperability issues. So we will
reuse the existing OUT_PORTAL for the same purpose.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I721977adb58ef35651775bacab6b5f9e325eb26a
Reviewed-on: http://review.whamcloud.com/9772
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4699 libcfs: unify ERR_PTR definitions

Add libcfs/err.h which either includes linux/err.h or defines
equiavlent functions (including ERR_CAST()) for use by userspace or
any of Lustre's other imaginary platforms. Modify the definition of
__container_of() to remove the pre-call explicit cast. Remove two
unused function that used container_of0() on an already dereferenced
pointer.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I26b454f835b51f1fb17b94eec7cf184f0a82071e
Reviewed-on: http://review.whamcloud.com/9519
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4476 kernel: support process namespace containers

PID namespace was introduce to linux to allow the
migration of containers between hosts. This way running
processes could be migrated to a new machine without
interruption due to pid collisions. User namespace creates
a new enviroment for a exist process where in this new
namespace it will have different uid/gid. Outside that
namespace the default uid/gid. For example a unprivileged
application can create user namespace for itself which has
root privilages. Those privilages don't exist outside of
the created namespace. Both of these changed the traditonal
one to one kernel mapping for pids/gids has changed. Since
userland can now have multiple PID namespaces this means that
each namespace in userland could have the same pid assigned.
This is possible since each namespace will be isolated from
each other. In the case of User namespace the application can
report to the kernel its namespace uid/gid instead of its real
uid/gid. Since the same running kernel is used between all
namespaces on a host the pid/gid data that is pushed into a
kernel context will have to map to a unique pid/gid in kernel
space to avoid collisions. This very similar to Lustre using
FIDs to avoid inode collisions. An example of where the mapping
needs to take place are when pid/gid data is pushed via a ioctl.
Also the internal structures of the kernel store uid/gid in the
kernel internal format. Lastly even with lustre userland uid/gid
namespace being consistant to applications the transmitted
uid/gid will be represented in the kernel differently from
server to server. Lustre wire protocols transmitting uid/gid
must handle these cases.

Change-Id: I207492b9a8a762e43ac2dd8fd1fb1b7397505304
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/8817
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4257 clio: replace semaphore with mutex

According https://www.kernel.org/doc/Documentation/mutex-design.txt:
 - the mutex subsystem is slightly faster and has better scalability
   for contended workloads. In terms of 'ops per CPU cycle', the
   semaphore kernel performed 551 ops/sec per 1% of CPU time used,
   while the mutex kernel performed 3825 ops/sec per 1% of CPU time
   used - it was 6.9 times more efficient.
 - there are no fastpath tradeoffs, the mutex fastpath is just as
   tight as the semaphore fastpath. On x86, the locking fastpath is
   2 instructions.
 - 'struct mutex' semantics are well-defined and are enforced if
   CONFIG_DEBUG_MUTEXES is turned on. Semaphores on the other hand
   have virtually no debugging code or instrumentation.

One more benefit of mutex is optimistic spinning. It try to spin for
acquisition when there are no pending waiters and the lock owner is
currently running on a (different) CPU. The rationale is that if the
lock owner is running, it is likely to release the lock soon.

This significantly reduce amount of context switches when locked
region is small and we have high contention.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ic13179ab7e5ee628f550e5dfd251af31ac6d4211
Reviewed-on: http://review.whamcloud.com/9095
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-4805 lmv: lookup remote migrating object in LMV

If remote object is being found in a migrating directory,
it should continue to lookup the object in remote MDT,
instead of return.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ib3309e255e2e1fe5789854f47e470f3085d6f6e6
Reviewed-on: http://review.whamcloud.com/9806
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4063 tests: use $DIR to check file at agent host

12a used to use DIR2 which is not mounted at agent host.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I5cea35c5e20f8c1064902d6b116dbddb1096c150
Reviewed-on: http://review.whamcloud.com/7869
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 lprocfs: update nodemap lproc to newer kernel

The PDE macro no longer exist in newer kernels. This
patch moves the PDE usage in the nodemap layer to use
PDE_DATA which is support across all kernels.

Change-Id: Idcfb918e0d4e203ff0f9c6d838a68b1a204ee3bd
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/9465
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4588 code: replace semaphores with mutexes

It's just optimization. The mutex subsystem is slightly faster
and has better scalability for contended workloads.

Remove the lustre_lock and it's accessor functions l_lock(),
l_unlock(), l_lock_init(), and l_has_lock() since they have
not been used by the code since Lustre 1.6.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I1b6defe7bdea2f7b33638a98c73277290c10bea1
Reviewed-on: http://review.whamcloud.com/9294
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4199 libcfs: add wrapper of PAGE_SIZE

PAGE_SIZE should not be used from userspace. It is not a
constant on all architectures and new kernels do not export
it to userspace anymore.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ib66534c31c7134833bf4768d3cc8bbc067b1bab4
Reviewed-on: http://review.whamcloud.com/8877
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4199 libcfs: add CPU table functions for uniprocessor

Some CPU table functions for uniprocessor architecture is
missing.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I2b41af63ebb5e4a1362e026d75b87855c1e7e2fa
Reviewed-on: http://review.whamcloud.com/8873
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3963 libcfs: convert ptlrpc,quota plus others to linux atomics

This patch converts all cfs_atomic primitives in
liblustre, osp, ptlrpc and quota.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: If21fdd31441d4b2a23df8cbd9db8187bb827ab2f
Reviewed-on: http://review.whamcloud.com/7077
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>

LU-4086 tests: strengthen sanity-hsm/test_33

Instead of waiting+checking both Restore and Cancel HSM operations
sequentially, wait for both to be finished and then check each
results. This will allow to handle conditions where Restore
terminates before the Cancel is started.

Also added to wait_all_done() the capability to only work on
requests for a specific fid.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I480ce074513d6829ecb3078c273bac1fdffa6256
Reviewed-on: http://review.whamcloud.com/9600
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4759  build: Remove obsolete file. Documentation was removed from the tree FDL needs to be removed also.

Signed-off-by: Cliff White <cliff.white@intel.com>
Change-Id: I7c57491416460af6bbce4c355b4b16888427e73c
Reviewed-on: http://review.whamcloud.com/9770
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4679 liblustre: remove dead code and clean headers

In liblustre, remove statically unused functions, varaibles and
macros. Also ensure that the headers needed by the c and h files are
included at top level. Prune unneeded includes. In libsysio, include
sys/queue.h in fs.h.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I878b442b6d0ae17d8d5d33375ca7596618323c52
Reviewed-on: http://review.whamcloud.com/9429
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4585 out: memset(ta, 0) will cause memory leak

thandle_exec_args should not be memset to 0 in
out_trans_start or out_handle, otherwise ta_args
will never be released.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I3b6a8dc8c62af743015f6022afdeda14264574c8
Reviewed-on: http://review.whamcloud.com/9804
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 lustre: remove lustre/include/ioctl.h

Remove the nowhere included header lustre/include/ioctl.h.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie059084c0e4ffb946694fc0a170f9200f12dda89
Reviewed-on: http://review.whamcloud.com/9757
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4462 mdt: don't apply mdt_object_fid() to ERR_PTRs

In mdt_reconstruct_open(), mdt_reconstruct_create(), and
mdt_reconstruct_setattr() print the FID from the request rather than
applying mdt_object_fid() to a known ERR_PTR.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ifb9365dbef1492d8fb5c7ff27d00f5155b0e7525
Reviewed-on: http://review.whamcloud.com/8788
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4655 libcfs: properly (un)define CDEBUG_ENTRY_EXIT

In lustre-libcfs.m4 we defined CDEBUG_ENTRY_EXIT to 1 or 0 according
to whether libcfs-trace was enabled. But in libcfs_debug.h we tested
CDEBUG_ENTRY_EXIT using #ifdef and enabled the trace macros
accrodingly. Fix this and use CDEBUG_ENTRY_EXIT to control GOTO()
tracing as well. The ability to properly disable these macros will
make some static analyses easier.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia016b4ddd45f8c39704a84908665757aa17bd3d2
Reviewed-on: http://review.whamcloud.com/9517
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4747 tests: Call get_param from MDS

In sanity tests 120a, b, c, d, e, f and g,
"lctl get_param ldlm.services.ldlm_canceld.stats" is
called from the client, but these stats are only
on the MDS.

Adding do_facet to the MDS gets the necessary stats.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I6b8434960bf5ec5e1220460612a583f2ab763980
Reviewed-on: http://review.whamcloud.com/9581
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-4742 mdd: improve error messages in obf_lookup()

To make user able to understand that's just an invalid operation
in .lustre/fid, but not a serious error.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I928cdf61ae6a70c0ff5bbe1133338c54cd997aa2
Reviewed-on: http://review.whamcloud.com/9594
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4065 mdt: missing NL in CERROR format

Add a missing 'n' at the end of the CERROR format parameter to
avoid annoying "format at mdt_coordinator.c:977:mdt_hsm_cdt_start
doesn't end in newline" msg in syslog and the consequence of 2
merged lines/traces in Lustre debug-log.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ic8baf81886b5be3de32ee595be99e20428729f6d
Reviewed-on: http://review.whamcloud.com/9597
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4674 endianness: Fix __{LITTLE,BIG}_ENDIAN macro usage

Fixed the userspace code where __{LITTLE,BIG}_ENDIAN is tested
with #ifdef. This patch replaces those checks
with #if __BYTE_ORDER == __{LITTLE,BIG}_ENDIAN

Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Change-Id: I92d4cf0c880d8032ea5023cb47ccfe7236dd03ab
Reviewed-on: http://review.whamcloud.com/9641
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4738 lmv: access lum_stripe_offset as little endian

By the time that a struct lmv_user_md reaches lmv_placement_policy()
it has already been converted to little endian. Therefore use the
appropriate macros around accesses to this this field. This issue was
found by rewriting the definition of struct lmv_user_md to use the
__leXX typedefs and running sparse.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If641375eabd22cc3e935a2e8f1327c48081c6ed1
Reviewed-on: http://review.whamcloud.com/9671
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Swapnil Pimpale <spimpale@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4780 lnet: NI shutdown may loop forever

lnet_shutdown_lndnis() may enter endless loop if there is a busy NI,
this is injected by LNet SMP improvements. It's fixed in this patch.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I612ab9ff15aa40c05d6bd4cfee7cd107715f41b4
Reviewed-on: http://review.whamcloud.com/9706
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2982 build: make AC check for linux arch sandbox friendly

I didn't found any usage of LINUXARCH definition in sources now.
But this check is problematic in sandbox. Therefore, I removed
it from sources at all.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ibdf0a3c26950a59d7f9cc51687d725d9c8835374
Reviewed-on: http://review.whamcloud.com/9458
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4629 libcfs: fix issues found by Klocwork Insight tool

sscanf format specification '%u' expects type 'unsigned int *'
for 'u', but parameter 3 has a different type 'int*'.

Pointer 'cfs_crypto_hash_type(d->hd_hash->ha_id)' returned from
call to function 'cfs_crypto_hash_type' at line 261 may be NULL
and will be dereferenced at line 261.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I9945a7e5d44329aa6417946106212cb269395290
Reviewed-on: http://review.whamcloud.com/9400
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4629 obdclass: fix use after release

Pointer 'type->typ_name' checked for NULL at line 195 will be passed
to function and may be dereferenced there by passing argument 1 to
function 'lprocfs_try_remove_proc_entry' at line 246.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I697ffab97a5bd1ab55339722d87a3dfca5c1e173
Reviewed-on: http://review.whamcloud.com/9388
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4629 mdc: fix issue found by Klocwork Insight tool

Pointer 'mod' checked for NULL at line 160 may be dereferenced at line 208.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I130de2f5b2066b8797e2454d4a603f53b3ffcbef
Reviewed-on: http://review.whamcloud.com/9387
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4629 lnet: fix issues found by Klocwork Insight tool

Null pointer 'cp' that comes from line 2544 may be dereferenced
at line 2618.

Pointer 'ni' checked for NULL at line 1569 may be passed to
function and may be dereferenced there by passing argument 1 to
function 'lnet_ni_notify_locked' at line 1621.

Null pointer 'best_iface' that comes from line 802 may be
dereferenced at line 832.

Buffer overflow of string buffer due to non null terminated string.

sscanf format specification '%lx' expects type 'unsigned long int *'
for 'x', but parameter 4 has a different type 'long int*'.

Pointer 'tsc' returned from call to function 'sfw_find_test_case'
at line 571 may be NULL and will be dereferenced at line 572.

sscanf format specification '%i' expects type 'int *' for 'i',
but parameter 3 has a different type 'unsigned int*'.

Local variable 'hash' is never used.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I2a942735cd93906453b92dcc711a956b7adf5615
Reviewed-on: http://review.whamcloud.com/9386
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4629 ldlm: fix NULL pointer dereference

Pointer '*exp' returned from call to function 'class_conn2export'
at line 523 may be NULL and may be dereferenced at line 543.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I084ae2ae76861aef4fd93477551bc9d0dd5f4478
Reviewed-on: http://review.whamcloud.com/9323
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4496 build: enable RPMs build with --disable-modules

Enable the following RPMs build:
  ./configure --disable-modules
  make rpms

Addition to commit c92afbcd4a8234994affa444d59645b80fa19ec5
(Reviewed-on: http://review.whamcloud.com/6020)

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I7b26fac928404a75e974b414085bf1a584ad8bb9
Reviewed-on: http://review.whamcloud.com/9306
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4614 llite: remove lustre_generic_file_{read,write}

It looks like lustre_generic_file_{read,write} are a holdover from
2.6.19 where generic_file_aio_read() replaced generic_file_readv()
and cross-kernel interoperability was required for some period of
time. Lustre has since removed support for those older kernels, but
it looks like the wrappers were not deleted at that time. This patch
will delete them.

Pass &iocb->ki_pos as the last argument for these functions instead
of crw_pos, since this is the convention for other callers.  Verify
that this is the same as the current crw_pos argument.  This code can
likely be cleaned up further in a later patch.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Id19edacf93ee2538634feb95580ab885f6069ee7
Reviewed-on: http://review.whamcloud.com/9230
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>

LU-3059 lu: shrink lu_object_header by 8 bytes on x86_64

Locate the loh_flags and loh_ref fields together in lu_object_header
to avoid holes and shrink the structure by 8 bytes.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I36b6132a819b6557d6d1688adb3e1f7d282d22b5
Reviewed-on: http://review.whamcloud.com/9185
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4596 lprocfs: mdt/*/exports/*/uuid is empty after remount

need to assign exp_nid_stats even if old stats are present

Change-Id: I9b0ac4c7ccee6cbd457827b20123bf40d91aaf33
Xyratex-bug-id: MRP-1641
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-on: http://review.whamcloud.com/9171
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3254 llite: access layout version under a lock

We used to access layout version under the protection of ldlm
lock, this introduces extra overhead for dlm lock matching.

In this patch, lli_layout_lock is introduced to access the layout
version. Also, when a layout lock is losing, we should tear down
mmap of the correspoding inode to avoid stale data accessing in the
future.

This is part of technical verification of replication.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I657c6a8b27f2cdaf468dc208c43c1d92f3843705
Reviewed-on: http://review.whamcloud.com/8689
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1538 tests: unmount /sbin/mount.lustre bind mount

If the lustre/utils/mount.lustre file has changed since it was bind
mounted on /sbin/mount.lustre, unmount it and mount the new one
there, to avoid strange problems when testing from a local tree

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ie5360dae9f33553acb914c8ce549e848c1500c1e
Reviewed-on: http://review.whamcloud.com/8912
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4446 tests: block on layout lock fix up on sanityn:51c

Fixed two problems: set fail_loc on MDS and block at layout lock
by writing the test file.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8028e855aad5882981a313fb28ec5b87f593832b
Reviewed-on: http://review.whamcloud.com/8789
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3373 osd-ldiskfs: readdir replace by iterate

Use iterate instead of readdir callback in iop.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: Icb08292009c965ca693814e854ae2e77b3e7a4f0
Reviewed-on: http://review.whamcloud.com/8231
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4167 tests: update version check to conf-sanity test_32

This patch updates version check codes to skip running LFSCK on an
OST for versions >= 2.5.0 in conf-sanity.sh test_32.

Test-Parameters: testlist=conf-sanity envdefinitions=SLOW=yes, ONLY=32 ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=71 mdsbuildno=71 clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 ossarch=x86_64 mdsarch=x86_64
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ibc0c34a0c15f4882a0dd06dae75d1f1fe10b6f16
Reviewed-on: http://review.whamcloud.com/8132
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4585 out: remove OBJECT_UPDATE_PER_RPC_MAX limit.

Extend tx_args if the update inside 1 RPC is more than
the count of tx_args count in tgt_thread_info.

And also for create remote object, the remote RPC might
happen in trans stop, so we need to add return value
for mdd_trans_stop to track the error.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ibb87181d87dfb6c3ed7e6757592bccd4a836a98a
Reviewed-on: http://review.whamcloud.com/9590
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4108 tests: decrease load for zfs in performance-sanity/4

Given the poor performance of metadata on ZFS (cf LU-2600), this patch
decreases the number of files created (similar to what happens in
perforamnce-scale.sh).

Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs  mdsfilesystemtype=zfs testlist=performance-sanity
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I998c8525bf218dacad07d98ac842f4c12102cd9a
Reviewed-on: http://review.whamcloud.com/9725
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4777 lfsck: redefine LFSCK_RBTREE_BITMAP_MASK

It should be the rbtree bitmap bits size, not the bytes size.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0c090702f3f2f5037926d6384b605751a0a9c948
Reviewed-on: http://review.whamcloud.com/9703
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4775 lfsck: dump all visibile lfsck parameter correctly

We missed to handle "LPF_BROADCAST" and "LPF_ORPHAN" when dump
the lfsck parameters and caused the output via /proc interface
incomplete and bad formatted.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I462c79e80be5b384a291b30980a95698a8dd9d19
Reviewed-on: http://review.whamcloud.com/9690
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4611 osd: improve credits calculation

- llog catalog do not declare records twice (for old and new objects)
  this might be an issue on ZFS with full debug enabled..
- llog to specify append by pos=-1, so OSD can take this into account
- object create/destroy should not include OI, this is calculated yet
- EA declaration to improve few specific cases
- osd_declare_write() to recognize overwrite optimistically,
  using inode size and i_blocks
- osd_declare_write() to optimize very specific cases, like legacy
  blockmaps with small offsets and allocated indirects
- index delete modify just a single block

preliminary testing on a local setup with 7 OSTs:
1360 credits before and 436 credits after.

llog declarations and index inserts (part of llog object creation)
still consume about 70% (7 OSTs):
   create: 7/28, destroy: 1/4
   attr_set: 2/2, xattr_set: 8/21
   write: 37/179, punch: 14/56, quota 2/2
   insert: 8/135, delete: 2/5
   ref_add: 1/1, ref_del: 3/3

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Icb5c79df1f8ba248509b9d2561ac8843bb01f6af
Reviewed-on: http://review.whamcloud.com/9258
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4624 statahead: Improve statahead debug messages

The statahead debug messages include the pid of the current
process in their body.  This is both redudant (because all
lustre log messages contain the pid), and sometimes downright
misleading.  For instance the messages would say something like
"stopping statahead thread 3446".  One would probably think
that 3446 is the pid of the process that is being stopped,
but in fact it was the pid of the caller issuing the stop signal.

We remove all superfluous pids from the messages.

Next we have the ll_statahead_thread() and the ll_agl_thread() record
their respective pids in their respective ptlrpc_thread structures.
This allows to print the pid of the thread that we are trying to
stop (which is actually useful info) from other threads, such as those
calling ll_stop_statahead().

Change-Id: Id4def179e02e9097f3f22048192d45bdf15b0aa8
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/9360
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4778 llog: update catlog when plain log destroyed

When the empty plain log is destroyed while processing
catlog, we should call llog_cat_cleanup() to close the
plain log and update in catlog.

Intel-bug-id: INTL-64

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I97b1d5cc59e84fdb4888274b48f223a063293647
Reviewed-on: http://review.whamcloud.com/9702
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4200 mgs: fix counting of mgs exports

In only_mgs_is_running iterate over mgs exports and only count
non-self and non MDS-MDS exports.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I145d897fe2f222dae5e998de322ea58ac9f75375
Reviewed-on: http://review.whamcloud.com/9650
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4028 quota: fix output of 'lfs quota'

Fix the output of 'lfs quota' when '-h' option is specified:
- total allocated block limit should be converted after calculation;
- show 1~3 digit after the decimal point;

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I90a8087dc15153c1b0b895c8081fe2882d6d01e6
Reviewed-on: http://review.whamcloud.com/8407
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3489 osd-zfs: object iterator support

a trivial object iterator for ZFS using dmu_object_next().
the iterator skips all objects with no LMA EA. It does
implement simple prefetching algoright. It also enables
few tests in sanity-lfsck to be run with ZFS.

Test-Parameters: mdtfilesystemtype=zfs \
    ostfilesystemtype=zfs mdsfilesystemtype=zfs \
    testlist=sanity-scrub,sanity-lfsck
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Icf02342ac351a163e23575963d3366761c59b02b
Reviewed-on: http://review.whamcloud.com/7149
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-4743 osp: continue llog processing

when unexpected record type is met

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I3dd45f501192bc6dd3bfddb550fd455ae319b923
Reviewed-on: http://review.whamcloud.com/9574
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>

LU-4624 llite: Avoid statahead thread start/stop deadlocks

The statahead and statahead agl threads blindly set their
thread state to SVC_RUNNING without checking the state first.  If, for
instance, another thread sets the state to SVC_STOPPING that
stop signal will now have been lost.  Deadlock ensues.

We also partly improve the sai reference counting, because a race exists
where the ll_stop_statahead thread can drop the default reference, and
the statahead thread can exit and drop its reference as well.  With no
references on the sai, the final put will poison and free the buffer.  The
original do_statahead_enter() function may then continue to access
the buffer after it is freed because it did not take a reference of its
own.  We add a local reference to address that.

Change-Id: I531f17966d49ba1e6ebd99abe9cb8b128eeac4f9
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/9358
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4398 mdt: acquire an open lock for write or execute

In mdt_object_open_lock() opens for write or execute will always
acquire an open lock of the appropriate mode so that any conflicting
cached open locks on other clients will be canceled. Add a regression
test to sanityn.sh.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8092bca4c418ec99a25584abdfb635ffec19a26e
Reviewed-on: http://review.whamcloud.com/9063
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2524 tests: enable conf-sanity/58 on ZFS

Fix mount/unmount of MDS so test runs correctly on ZFS MDS.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ifb9bd12fdecdc38132dfc487f565b67f7ade4123
Reviewed-on: http://review.whamcloud.com/9675
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4767 lfsck: sub-task thread should not change lc_link

The lfsck sub-task thread (for double scan) should not change the
"lfsck_component:lc_link" which may cause the master lfsck engine
thread to access some unexpected RAM space by race.

LFSCK should not return -ENODEV to avoid to misguide ptlrpc.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifb80eaf7c1c4ba9bf7fbd5213e9eae09efed7e28
Reviewed-on: http://review.whamcloud.com/9661
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4721 obdclass: handle local storage init/fini properly

1) In local_oid_storage_fini(), take the mutex on ls_device
   before decreasing the 'los' reference to avoid others to
   obtain the mutex earlier and freed the 'los' by race.

2) When llog init the local stroage for FID_SEQ_LLOG and
   FID_SEQ_LLOG_NAME, it should record the handlers which
   can be used to fini them to avoid releasing the handler
   which is in using by others.

3) NOT forget the llog_ctxt_put() if something wrong during
   the llog_osd_setup().

4) NOT put the object in lastid_compat_check() until all the
   usages on such object have been done.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes testlist=sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ibf139cf9783d06276a6c0ed764c40dec6e3f70cb
Reviewed-on: http://review.whamcloud.com/9573
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3852 tests: disable test_251 from inside sanity-hsm

Due to test_251 frequent failures, subtest has been disabled at
the autotests config level. To be able to work on a patch to fix
the root cause (likely to be a pure consequence when running on
a too small-sized Lustre filesystem), subtest will be disabled
from inside sanity-hsm to allow autotests config changes to be
reverted and to re-enable it on a per-patch basis.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I7dfae4d278cba049062480c276f9cad10d22e0e9
Reviewed-on: http://review.whamcloud.com/9014
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4704 test: enable acl 974 and 2561 series test

Add acl/{974,974_remote,2561,2561_zfs}.test to the
lustre/tests/Makefile.am nobase_noinst_DATA list, and fix acl/run
test to fail if the test scripts are missing.
acl/2561_zfs.test is a newly created test file for zfs. Since zfs
allocates a 512 byte dnode for empty file, its "ls -s" result is
different from ldiskfs'.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I11be072eb1954fff3b7a19231b82345b3073a534
Reviewed-on: http://review.whamcloud.com/9541
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-4453 build: warn on deprecated cfs_* uses

In contrib/scripts/checkpatch.pl, use static dictionaries of
deprecated identifiers and headers rather than reading from the
kernel's Documentation/feature-removal-schedule.txt file. Populate the
deprecated identifiers dictionary with the cfs_{atomic,hlist,list}_*
macros, some out of fashion typedefs, and a few unsafe functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5af33e33693d6da59b98890ae2f27d95adcd8710
Reviewed-on: http://review.whamcloud.com/9585
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Li Xi <pkuelelixi@gmail.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4752 build: add gerrit_checkpatch.py to contrib/scripts/

Add a python script gerrit_checkpatch.py to contrib/scripts/ which
polls for recently updated changes, runs checkpatch.pl on them, and
pushes reviews back to gerrit.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I9e3c9990288b8adc8f507a6b2294f2a53991a0e8
Reviewed-on: http://review.whamcloud.com/9586
Tested-by: Jenkins
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4770 tests: tests scripts simplification

1. Xeon Phi don't support "mount -t lustre", we can use only
   "mount.lustre" utility.
2. A "head" utility don't support shortcut "-1" option.
   It should be specified as "-n 1".
3. A "sed" utility don't support expression "s/[-.]/ /3"

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib5ba22451819772ebacb38706447403ae6668cc4
Reviewed-on: http://review.whamcloud.com/9668
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4719 mdt: Don't try to print non-existent objects

mdt_dump_lmm() needs to make sure it's not trying to print
objects for a default striping from a directory, since there
are not objects in this case.

Return early if the D_INFO debug mask is disenabled, so it
doesn't iterate through the LOV/LMV layout at all.  That is
just a waste of effort.

Change-Id: I2fffdaefad2a6b05247676441451fffdca620dcd
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/9520
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Matt Ezell <ezellma@ornl.gov>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4252 osd: remove locking i_mutex in osd_punch

This piece of code intent to calm down the kernel
WARN_ON messsage show up since 3.9 upstream. So we
can just remove it for now. We'll add a ldiskfs
patch to resolve this issue while 3.9 server support
come up.

Change-Id: Ied941b98d213e3aff1c5ba7ec4c6436bdef903c7
Signed-off-by: yang sheng <yang.sheng@intel.com>
Reviewed-on: http://review.whamcloud.com/9644
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 cleanup: const correct FID/OSTID/... helpers

Add a const qualifier wherever possible to the pointer parameters of
the inline helper functions in lustre_idl.h and lustre_fid.h. Change
the return type of several predicate functions from int to bool.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ibe590d7184470cfe8a154fe0935a7d6bea261669
Reviewed-on: http://review.whamcloud.com/8641
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.6.57

Change-Id: I58633c93179d37500989b54ecd53e3b715e5f2b2

LU-4753 endianness: Fix cpu_to_leXX size mismatch

In lod_prep_md_striped_create() and lod_xattr_set_lmv we use
cpu_to_le32() to transfer __u16 values. This patch replaces
cpu_to_le32() with cpu_to_le16().

Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Change-Id: Icc33e3fac711b63cc9d67fcec82dd1b4fdc10cce
Reviewed-on: http://review.whamcloud.com/9637
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-4748 test: get one single qos_threshold_rr number

In sanity.sh test_116b, if there are multiple MDTs on the same MDS,
we should get one single qos_threshold_rr number instead of the array,
otherwise it will cause resetting qos_threshold_rr failure.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ie779769f7ee35cd118fb6412fe3352a03254b9d3
Reviewed-on: http://review.whamcloud.com/9580
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3534 out: only start transaction for modified update

Only start transaction, if the update will modify the
filesystem.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I9c516b5fe6f6bc743e1dbc69ab04fd7062b2311d
Reviewed-on: http://review.whamcloud.com/7983
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3338 llite: Limit reply buffer size

When allocating a reply buffer for the striping information don't
assume the unlikely worst case.  Instead, assume the common case
and size the buffer based on the observed default ea/cookie size.

The default size is initialized to a single stripe and allowed to
grow up to an entire page if needed.  This means that for smallish
filesystems (less than ~21 OSTs) where the worst case striping
information can fit in a single page there is effectively no
change.  Only for larger filesystem will the default be less than
the maximum.  This has a number of advantages.

* By limiting the default reply buffer size we avoid always
  vmalloc()'ing the buffer because it exceeds four pages in size
  and instead kmalloc() it.  This prevents the client from
  thrashing on the global vmalloc() spin lock.

* A reply buffer of exactly the right size (no larger) is allocated
  in the overflow case.  These larger reply buffers are still
  unlikely to exceed the 16k limit where a vmalloc() will occur.

* Saves memory in the common case.  Wide striped files exceeded
  the default are expected to be the exception.

The reason this patch works is because the ptlrpc layer is smart
enough to reallocate the reply buffer when an overflow occurs.
Therefore the client doesn't have to drop the incoming reply and
send a new request with a larger reply buffer.

It's also worth mentioning that the reply buffer always contains
a significant amount of extra padding because they are rounded up
to the nearest power of two.  This means that even files striped
wider than the default have a good chance of fitting in the
allocated reply buffer.

Also remove client eadatasize check in mdt xattr packing because
as said above client can handle -EOVERFLOW.

Change-Id: Ic6bab2aca208db1f4abcd87974a5879d06d0f2f1
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-on: http://review.whamcloud.com/6339
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-4769 tests: conf-sanity/38 fix regression

stop_mds on zfs exports the pool.  Use mount_fstype that accounts for
importing pool if needed.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id3fbfc61af032aaaca1030a5927bd509725a0f21
Reviewed-on: http://review.whamcloud.com/9669
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4728 mdt: fix NULL deference of mdt_fid_lock

When enabling hsm_control, mti_exp field of struct mdt_thread_info
could be NULL.  ldlm_cli_enqueue_local will crash the kernel when
dereference it.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I4bdb8222aec378e396e7f544834f6fcf9fdaf777
Reviewed-on: http://review.whamcloud.com/9543
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3539 osp: Fix a series of UPDATE_OBJ endianness bugs

Current UPDATE_OBJ code have the following problems:

  - Fields in "struct update_buf", except for ub_bufs, are sent in
    senders' host endianness but expected to be little-endian by
    receivers.

  - Fields in "struct update", except for u_bufs, are sent in
    little endianness but used without swabbing by receivers.

  - Most u_bufs are sent in little endianness, but some are in
    senders' host endianness.

  - Sizes for a couple of string buffers in osp_md_declare_xattr_set()
    and osp_md_xattr_get() are off-by-one.

  - Error numbers in UPDATE_OBJ replies are not translated on both
    sides.

This patch fixes them by sending all data in UPDATE_OBJ requests and
replies in host endianness, following Lustre's usual practices.

Change-Id: Id3ffb248af181b4beff2b62c9cb46fc6d9f9e673
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/7088
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4751 hsm: Fix sanity-hsm tests for non-mrsh $PDSH

The workaround for starting backgrounded copytool
monitors via pdsh -Rmrsh is not compatible with non-mrsh
values of $PDSH. This commit adds a branch such that
the workaround is only used when required by the test
cluster environment.

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: I35a57d26a3836cfd1bbb698d267a8fe9f31f86c8
Reviewed-on: http://review.whamcloud.com/9587
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4320 mdt: fixup MDS_SWAP_LAYOUTS ELC handling

In mdc_ioc_swap_layouts() cancel *any* unused locks with LAYOUT or
XATTR IBITS set on the two files. (This matches the locks acquired in
mdt_swap_layouts(). Previously only locks that conflicted with a CR
LAYOUT lock were cancelled.) Add HABEO_CLAVIS to the target handler
flags for MDS_SWAP_LAYOUTS and call ldlm_request_cancel() from
mdt_swap_layouts() to do the intended early lock cancellation before
the handler acquires its own locks.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I9d7ba9ac660b136dafa7e307177b92204a1257c3
Reviewed-on: http://review.whamcloud.com/9329
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4682 llite: a few fixes for migration.

1. Clear the client dentry cache before migrating file/directory
to the remote MDT.

2. Do not return stripe information to client, it did not get
Layout lock.

3. A few cleanups from the review of 6662.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I209079dfa07241412878b30fd02e8152046c95c7
Reviewed-on: http://review.whamcloud.com/9522
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4591 lov: cancel ungranted sub lock

When the top lock is canceled due to error, we should cancel
ungranted sub lock otherwise the sublock state is undefined.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I5c1df5220ce77ff434463d6443135c625e79ea04
Reviewed-on: http://review.whamcloud.com/9524
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4703 tests: Add test to check setxattr fails when it should

Setxattr was not correctly checking permissions in all cases. This
adds a test to sanity to ensure that trying to set xattrs on a file
fails when it should.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id95e13ec3bcbedb7cbb069c610d506d1f3d899a8
Reviewed-on: http://review.whamcloud.com/9508
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4384 ofd: remove INCOMPAT_SUPP, ROCOMPAT_SUPP defines

Removed ROCOMPAT/INCOMPAT_SUPP defines from MDT and OFD as
obsoleted. They are not used now. The tgt_server_data_init()
uses own way to set/check rocompat/incompat masks

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ib4b7b4b948e0563d1cc89faae2d423245ed4df0e
Reviewed-on: http://review.whamcloud.com/9375
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-4545 hsm: Always report errors to coordinator.

Make sure feedback on processed items gets properly delivered to the
coordinator even if errors occur between action item delivery and
item processing initialization phase.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: If1dfc88e4acb9fbd6bfb5b2d18ffdd4979944997
Reviewed-on: http://review.whamcloud.com/9310
Tested-by: Jenkins
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2785 osc: remove unused obd methods

Remove the unused methods osc_punch(), osc_sync(), osc_brw(),
osc_enqueue(), osc_cancel(), osc_llog_init(), osc_llog_finish(), and
their supporting functions. Remove the file obd_ost.h, moving all of
the structures defined there to osc_request.c. Remove several orphaned
obd methods.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I4148547c52625cb7b173f1b835c9e6e47e19e4f6
Reviewed-on: http://review.whamcloud.com/8547
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4643 hsm: Improve sanity-hsm test_60

Small fix to further reduce the chance of false failures.
Also cleans up example HSM events in comments to address style
warnings.

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Icecb69ddc417cd33f80b38c9df623ac5fc0cc466
Reviewed-on: http://review.whamcloud.com/9534
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4724 hsm: Safe copytool event logging

Protect against concurrent event log writes by multiple
threads within a copytool process. Fixes sanity-hsm test_71
failures.

Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Ie18f7368f386acf0949a038e6bdb2ba7dca5c3a3
Reviewed-on: http://review.whamcloud.com/9553
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3953 rpm: stop using unnecessary file lists

File lists are being generated for unconditonally packaged files.

RPM can handle both static and dynamic file lists, so let's use them.
It simplifies the spec file quite a bit.

There are also some files that are treated as being conditionally
included but the conditions are always true:

- lustre_types.h was removed in commit adde80ff (v1.9.50)
- libcfsutil.a was made unconditional in commit b279ac68 (v2.3.51)
- include/lustre/lustre_idl.h was added in commit 11330397 (v1.7.100)

I've removed the conditionals surrounding these and moved them into
the static file lists.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I3f08a40f3ffef468723b366848ed5645c4cb9de9
Reviewed-on: http://review.whamcloud.com/5489
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4708 ldiskfs: do not copy NUL terminator from direntry

Do not try to copy a NUL terminator from the source dirent if there
is no dir_data for that record.  This can happen with 1.x filesystems
that do not have a dirdata field in the dirent to store the FID.

The target buffer is expected to be NUL terminated, and this is
handled by using kzalloc() to allocate it and zero out the buffer,
but the source buffer may not have a trailing NUL byte if it is a
multiple of 4 bytes in length.

If the dirent is aligned at the end of the directory block/page this
can cause an oops by accessing beyond the end of the page, or will
otherwise copy a garbage byte to the target buffer.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Iab52f916d7571b3d2587255280c72a52c60399c9
Reviewed-on: http://review.whamcloud.com/9510
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4381 lov: to not hold sub locks at initialization

Otherwise, it will cause deadlock because it essentially holds
some sub locks and then to request others in an arbitrary order.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I00d51677aa94ec41856402fd8a027e09355786ee
Reviewed-on: http://review.whamcloud.com/9152
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3274 osc: allow to call brw_commit() multiple times

Sometimes the rq_commit_cb of BRW RPC can be called twice if that RPC
has already committed at reply time. This will cause inaccuracy of
unstable pages accounting and then assertion.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ic596e1d44e2c27bf13701d2c6c9691562beb7e58
Reviewed-on: http://review.whamcloud.com/8215
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4627 llite: deed taking lli_trunc_sem during file write

File write before io loop will take lli_trun_sem read semaphore to
protect osc_extent, while after generic_file_aio_write() done, it
could possible need to kill suid or sgid, which will call
ll_setattr_raw() to change the inode's attribute, and it does not
involve size.

So the ll_truc_sem write semaphore should be constrained
around ll_setattr_ost() to not come across the lli_trunc_sem read
semaphore get from the normal file write path.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic7f9cfb6d76dcf7ad08635efab154e7cdc392ce8
Reviewed-on: http://review.whamcloud.com/9267
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4702 nodemap: fix nm_rbtree_postorder_for_each_entry_safe()

It should check if the 'pos' && 'n' is NULL.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ibda34ec4cf1706f0651ed8c86a2e49e4a36cd2c0
Reviewed-on: http://review.whamcloud.com/9500
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Joshua Walgenbach <jjw@iu.edu>

LU-4422 quota: fix s-q test_6

s-q test_6 should not fail by checking whether the file is
growing, because in most time, the test is waiting for quota
slave reconnecting to quota master.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I2332febf639bdbe033c588b97a2c7a1e1e2f51e2
Reviewed-on: http://review.whamcloud.com/9339
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2524 tests: clean up conf-sanity test_38 for ZFS

Clean up conf-sanity.sh test_38 so that it mounts the filesystem
locally instead of using debugfs.  That allows it to be run on
ZFS instead of being skipped.  Other minor style fixes.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I68090d2082b298b81518e5ffde847f15b47953b4
Reviewed-on: http://review.whamcloud.com/9184
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4388 llite: issue OST_SYNC for fsync()

The last parameter @datasync of fsync() has following indication:
* if datasync=0, we'd always flush data and metadata
* if datasync=1, we'd always flush data while does not flush modifed
  metadata unless that metadata is needed in order to allow a
  subsequent data retrieval to be correctly handled. For example, a
  change to the file size would require a metadata flush.

Lustre client can not tell the difference easily, and would issue
MDS_SYNC and OST_SYNC in all cases.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I90fa8e761e91c8a04afba3fbfafe1f60f5839ad1
Reviewed-on: http://review.whamcloud.com/8684
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-4726 ptlrpc: vfs_rename api changed

In newer kernels the vfs_rename api changed to take
more arguments. This patch creates a wrapper to let us adapt
to the different versions of vfs_rename in old and new kernels.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia5dafe64570fc404cd8a037fd442ca12e687ffc1
Reviewed-on: http://review.whamcloud.com/9537
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4199 libcfs: add wrapper of dump_trace()

dump_trace() is a X86 specific function. It will cause
compilation failure if the architecture is not X86. A more
general function dump_stack() should be called under that
environment.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8a5fffa6dec15dc8c779a22e24c10b364c77f9a0
Reviewed-on: http://review.whamcloud.com/8872
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4704 acl: fix permission problem of setfacl

Setxattr does not check the permission when setting ACL xattrs. This
will cause security problem because any user can walk around
permission checking by changing ACL rules.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I1e5c44674f2e22ecbe29a589a219bed52d037b9b
Reviewed-on: http://review.whamcloud.com/9473
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>