LU-6343 lfsck: locate object only when necessary

Currently, for every item to be verified by the LFSCK assistant
thread, the LFSCK assistant thread will try to locate its parent
directory (for namespace LFSCK) or its parent MDT-object (for
layout LFSCK) firstly via the given parent FID. But under most
cases, the system is consistent, such locating parent object is
unnecessary. For example, for namespace LFSCK, the LFSCK main
engine has put the <parent_FID, child_name, child_FID> in the
pre-loaded request, then the assistant thread will locate the
child object with the child_FID and search its linkEA to find
the <parent_FID, child_name> pairs.

Be as some improvement, we will make the LFSCK to locate related
object (especially for parent object) only when really used.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If6a3677a6ad3fa1c694ee06f9688c1f38374f8de
Reviewed-on: http://review.whamcloud.com/13993
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6368 ldlm: Do not use cbpending for group locks

Currently, the CBPENDING flag is set on group locks when
the osc lock above them is released (osc_cancel_base).

This results in a situation where a new group lock request
on a resource does not match an existing group lock because
LDLM_FL_CBPENDING is set on the existing lock.

So two group locks are granted on the same resource, which
is not valid, since a given client can only have one group
lock on a particular resource.

Since group locks are manually released and not called back
like other LDLM locks, the CBPENDING flag doesn't make
sense.  Since they must be manually released, they also
cannot go in the LDLM LRU cache and must be fully released
immediately once they are no longer in use.

This was previously accomplished by setting CBPENDING when
the corresponding osc lock is released, but as noted above,
this prevents the group lock matching some future lock
requests.

This patch uses the fact that group locks have an l_writers
reference which they keep until they are manually released,
so we remove them when they have no more reader or writer
references, without checking cbpending.

Additionally, this patch adds several sendfile tests,
courtesy of Frank Zago <fzago@cray.com>.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I2845750777cbc9849b18999f1b77f791034c50b0
Reviewed-on: http://review.whamcloud.com/14093
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6421 osc: fix bug when setting max_pages_per_rpc

After setting like "lctl set_param -P osc.*.max_pages_per_rpc", it
is possible that the function osc_obd_max_pages_per_rpc_seq_write
will be called before ocd_brw_size has been set when mount.
ocd_brw_size is meaningless when it is zero. So it should not be
the limit at that time.

Signed-off-by: Wu Libin <lwu@ddn.com>
Change-Id: I1abbd1bfc089ca1fea2b7ec71a6d45ee70f1ba20
Reviewed-on: http://review.whamcloud.com/14333
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2524 test: Test suite clean up for replay-single

The patch modifying the tdir variable to a single directory
has landed; http://review.whamcloud.com/#/c/8123/. We can
now conduct miscellaneous cleanup including:

Remove the `-p` (parents) option from many calls to mkdir
Modified directory and file names to use $tdir and $tfile
Add check for and call `error` and/or add error messages for a
variety of common functions.
Replace `…` with $(...)
Removed linefeed escape after |, ||, & and && operators.
Remove test dependencies between tests 4a and 4b

Test-Parameters: testlist=replay-single alwaysuploadlogs envdefinitions=SLOW=yes mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ib09102e50f855550db801180be3f7fc42911191a
Reviewed-on: http://review.whamcloud.com/13170
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6407 lnet: set task state before scheduling

In 0b868add80281c085ce1b297d1cb078deaab802a when libcfs_sock_accept()
was made into lnet_sock_accept() a call to
set_current_state(TASK_INTERRUPTIBLE) got dropped. Restore it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I132d6a611e5677ad52358178aa566ca805536e61
Reviewed-on: http://review.whamcloud.com/14265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5579 tests: Add test for resend enqueue vs lock destroy

This test is split out from the actual patch because
some other test infrastructure issues make it always
fail right now.

Xyratex-bug-id: MRP-2094

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I6f764fd863d6046bde8c6336f003e602fc07e59f
Reviewed-on: http://review.whamcloud.com/12210
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-6486 ptlrpc: export lustre_swab_lov_mds_md

For ppc64 clients lustre would not mount due to an Unknown
symbol error. This is due to lustre_swab_lov_mds_md not
being exported. A simple export of this function resolves
this issue.

Change-Id: Ib924045988291efbc5ac129ce58354bbc72aa7b6
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/14545
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6481 kernel: kernel update RHEL 6.6 [2.6.32-504.16.2.el6]

Update RHEL6.6 kernel to 2.6.32-504.16.2.el6

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3d439ad2338b7940292fb584e09ca2c0b4fce4ef
Reviewed-on: http://review.whamcloud.com/14546
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6471 obdclass: fix llog_cat_cleanup() usage on Client

With patch/commit 3a83b4b9 for LU-5195, LLOG code has been
strengthen against catalog inconsistency by detecting a
referenced plain LLOG is missing and by clearing its
associated entry by calling llog_cat_cleanup(), which now
needs to handle the case where it is also executed on a Client
(ie, cathandle->lgh_obj == NULL) and thus must not attempt to
update on-disk catalog.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ie6fb9240ac76810358a91c7410046c626f42c2b9
Reviewed-on: http://review.whamcloud.com/14489
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 utils: split kernel comm between user and kernel

The kernel communication code used for HSM and changelog is
entangled. Move the user space bits into the liblustreapi. This will
also help for a possible relicensing. The kernel portion is also moved
from libcfs to obdclass.

The original libcfs_kernelcomm.h header is split into three parts:

  * lustre_kernelcomm.h, a new header for the kernel parts;
  * uapi_kernelcomm.h, a new header for the data structures shared
    between userspace and kernelspace;
  * lustreapi_internal.h receives the private liblustreapi prototypes.

The original code in kernel_user_comm.c is split into two parts:

  * obdclass/kernelcomm.c for the kernel part. filp_user_write() was
    moved there, and linux-fs.c deleted;
  * liblustreapi_kernelconn.c for the user part. The calls to CDEBUG
    have been removed, and calls to CERROR have been transformed to
    llapi_err_noerrno. The type lustre_kernelcomm has been removed and
    replace by struct lustre_kernelcomm.

Various names and filenames have been harmonized to *kernelcomm*.

The unused symbol KUC_FL_BLOCK has been removed.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Id15ab6ea23a160e0aa0517a1a890266b2e971982
Reviewed-on: http://review.whamcloud.com/14270
Tested-by: Jenkins
Reviewed-by: Nathan Rutman <nathan.rutman@seagate.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6300 build: Kernel string mismatch for ppc64 in RPMs

The macro in the kernel RPMs for ppc64 provided
by Redhat describing the kernel version supported,
mismatch the content of the macro of the Lustre kernel
client modules for ppc64 describing the required
kernel. This leads to unresolved dependency errors
during installation of the Lustre client modules
although the kernel versions match actually.

While Redhat uses the convention:

kernel = <major-number>-<minor-release-number>.el6

the following string is used in Lustre client RPM
for the required kernel version:

kernel = <major-number>-<minor-release-number>.el6.ppc64

The patch will remove the trailing string '.ppc64' in
macro 'krequires' defined in the lustre specfile
describing the software dependeny.

Signed-off-by: Frank Heckes <frank.heckes@intel.com>
Change-Id: I930d2fe67da819db57a6591341c5a55f0374ad43
Reviewed-on: http://review.whamcloud.com/14012
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5814 llite: remove lli_has_smd

Remove the lli_has_smd flag from struct ll_inode_info. The empty
layout case will be handled by the LOV layer. Remove the unused
function cl_local_size().

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I50b73482fdc4c415f8d800e9a848099117be8f74
Reviewed-on: http://review.whamcloud.com/13690
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6413 lod: set FLAGS all stripes of striped dir

Except permission and time, flags should be set on
each stripe of striped dir as well, otherwise
chattr can not set special flags (immutable etc)
correctly on striped dir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I8b3274b674bd1b8b41f95bb9299245aa8c2905c3
Reviewed-on: http://review.whamcloud.com/14278
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6320 lod: Use target index as the master index

Because lum_stripe_offset might be -1, if it comes
from the default stripe, and also the master stripe
always in the same MDT as the master object, so
it should use master object MDT index as the master
stripe index.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I4a700580dd7a035b243d2314231e50e12742100e
Reviewed-on: http://review.whamcloud.com/13941
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-5022 build: add rhel7 to server build selections

Add build files to allow building el7 server

Test-Parameters: clientdistro=el7 mdsdistro=el7 ossdistro=el7 \
 mdsfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs \
 ostfilesystemtype=ldiskfs testgroup=review-ldiskfs \
 envdefinitions=SANITY_EXCEPT=17m

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3f6126473aaa95874bd2e225e79147ea23046f4e
Reviewed-on: http://review.whamcloud.com/12289
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.7.52

Change-Id: I7c225701dab51f2ba32af10cec688f37af240d70

LU-5814 lov: add cl_object_layout_get()

Add cl_object_layout_get() to return the layout and generation of an
object. Replace some direct accesses to object LSM with calls to this
function.

In ll_getxattr() factor out the LOV xattr specific handling into a new
function ll_getxattr_lov() which calls cl_object_layout_get(). In
ll_listxattr() call ll_getxattr_lov() to determine if a lustre.lov
xattr should be emitted.  Add lov_lsm_pack() to generate LOV xattrs
from a LSM.

Remove the unused functions ccc_inode_lsm_{get,put}() and
lov_lsm_get().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: If0d7696473848c8eace3c5b03c016514bc28509f
Reviewed-on: http://review.whamcloud.com/13680
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5710 all: second batch of corrected typos and grammar errors

Most of them are in comments, but there was a few in user
visible help.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I32730030f84b85e0ae08b683d64bbad5e33bf0b4
Reviewed-on: http://review.whamcloud.com/12933
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6434 ofd: object reference leaks in ofd_preprw_write

In ofd_preprw_write(), the object should be released when an
error happenes.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Icd143f994d63c97472fb7d24cb9517da516280b7
Reviewed-on: http://review.whamcloud.com/14370
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6370 osc: disable to control per-OSC LRU budget

It will produce a problem where a system has lots of OSTs but
only a few of them are involved in doing read on the client side.
In that case, the per-OSC budget will cause the read ahead pages to
be dropped aggressively, which results in significantly performance
degradation.

This patch disabled per-OSC LRU budget. However, this is not a real
fix. A real fix should feed back memory pressure from LRU on the OSC
layer to the LLITE layer, so that read ahead algorithm can take that
information into consideration when it decides read ahead window size.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ifa81d8515a345b389d74af971582b2a3b1761546
Reviewed-on: http://review.whamcloud.com/14347
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3534 osp: move RPC pack from declare to execution phase

1. Since we will have full aysnc update support, i.e. do not
need order update anymore, move RPC pack from declare to
execution phase, and these remote updates will be sent during
transaction stop after local transaction is stopped.

2. Add update callback for each update, so after the update request
is done, these callback will be called correspondently for each
update.

3. Remove tu_sent_after_local_trans, because every remote
transaction will be sent after local transaction is finished, in
lod_trans_stop()->osp_trans_stop(). Note: RPC should be sent after
local transaction is stopped, to avoid sending RPC while holding
transaction.

Change-Id: I296e2eaf8b922fe58fe88df074458545f102a69a
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10794
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6106 test: skip test_16 to test_23 if MDS version older than 2.6.90

Skip sanity-sec test_16 to test_23 if MDS version older than 2.6.90.
The previous commit 0f95dae3a7a0bdef52160a3ca76fefac6765007c doesn't
prevent the tests from being run, fix the problem in this one.

Change-Id: I791f1e81c248979270524d57305e545f388a9d67
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/13806
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4772 mgs: check MGS refcounting before export barrier

Patch adds debug code to make sure that there are no extra
MGS export references taken before export barrier.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I4a7ae3130cb70a9b6847c9d4b5c2874ccf9ee72d
Reviewed-on: http://review.whamcloud.com/13920
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3536 lod: Separate thandle to different layers.

Separate thandle into different layers on MDT stack.

The current implementation use single thandle in all
layers, which might cause some issues for cross-MDT
transaction, for example during transaction stop,
it needs to stop local OSD transaction first,
then send remote RPC, because we do not want hold the
transaction, during RPC sending, but once we stop osd
transaction, which might cause this single thandle be
destroyed (see osd_trans_stop()), but all of remote
updates are still attached in this thandle.

This patch will separate the thandle to different layers:
1. MDD thandle will present itself in MDD transaction.
2. LOD thandle will distribute the thandle to all sub-thandles,
   and also attach the update blob buffer to store updates for
   cross-MDT operations.
3. OSP thandle will store the updates for the remote
   correspondent target, and also manage to send them to the
   remote target.
4. OSD thandle will still be the same as the original one,
   relying on the bottom FS to achieve the "atomic".

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I0c73cc80fa692c2e5d5a09e441c28e228d822ce0
Reviewed-on: http://review.whamcloud.com/10640
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6285 ptlrpc: Get rid of cpus_* calls as deprecated

Use cpumask_* equivalents instead.
Also we are no longer supposed to directly assign cpumasks as a method
of copy, need to use cpumask_copy instead.

Change-Id: I0425cd308a363e74de8d20f9a5f144fe07431f01
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/13925
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-6153 libcfs: rename cfs_snprintf() to scnprintf()

Rename cfs_snprintf() to scnprintf() to match the existing kernel
function. Similarly rename cfs_vsnprintf() to vscnprintf(). Remove the
unused functions cfs_[v]snprintf().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I361a28959dc8a20ab0e152eaed00fd4adf78cc34
Reviewed-on: http://review.whamcloud.com/13517
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-6245 libcfs: create userland and kernel string operations

Additonal string handling and NID string parsing code are both
used by kernel space and user land. This prevents us from
moving forward for cleaning up the userland and kernel space
headers. With the code duplicated for both environments we
can then clean up the headers independently. Since NID string
handling is only done for LNET we move it there.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I5fccdae61322d0bace7094a36d2e551d719c4982
Reviewed-on: http://review.whamcloud.com/13835
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5710 corrected some typos and grammar errors

Most of them are in comments, but there was a few in user
visible help.

Change-Id: Ib050e9042f9f2728cd3eeedeee679739b48419db
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/12201
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6285: o2iblnd: Do not use cpus_weight, it's deprecated

Replace cpus_weight and for_each_cpu mask with
cpumaskweight and for_each_cpu respectively as per latest kernel guidelines.

Change-Id: I038eb38234c0a209a68ca24c8a860d0e84522c27
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/13954
Tested-by: Jenkins
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-6405 kernel: kernel update [RHEL7.1 3.10.0-229.1.2.el7]

Update RHEL7.1 kernel to 3.10.0-229.1.2.el7

Test-Parameters: clientdistro=el7 testgroup=review-ldiskfs \
  mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I020419971dda62b2997e33a624c6e6113d465afb
Reviewed-on: http://review.whamcloud.com/14293
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5478 lustre: get rid of obd_* typedefs

We have a bunch of typedefs for common things that made no sense
and hid the actual type from plain view.
Replace them with proper uXX or sXX types.
Exception is in lustre_idl.h and lustre_ioctl.h where
they are replaced with __uXX and __sXX to be able to be included
in userspace

final patch in series.

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I2b714ec673a004561d45ad46041191bef3ec9a8e
Reviewed-on: http://review.whamcloud.com/14256
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-6394 all: fix compilation errors with FORTIFY_SOURCE

When Lustre is configured with CFLAGS="-D_FORTIFY_SOURCE=2 -O2" on
Centos 6, the compilation will fails with errors such as this one:

 cacheio.c: In function ‘qword_printhex’:
 cacheio.c:174: error: ignoring return value of ‘fwrite’, declared
   with attribute warn_unused_result

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ie06fd5a26b62daf62bfd0133a2d7ebc66ece5be6
Reviewed-on: http://review.whamcloud.com/14126
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 ptlrpc: Add a tag field to ptlrpc messages

The new tag field is used as a virtual index for multiple modifying
RPCs management. It is set by the client and allows the target to
release in-memory reply data when the tag is reused by a new RPC.

The tag field replaces the unused last_seen field of ptlrpcd_body
structure.

Additionally, the last_xid field is used to transfer the highest XID
for which a reply has been received and does not have an unreplied
lower-numbered XID.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I4a57f3710ffffe21d8d655af6ac222b65051a12d
Reviewed-on: http://review.whamcloud.com/14095
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-6285 ptlrpc: Do not recalculate siblings of CPU 0 in a loop

ptlrpc_hr_init seems to be recalculating number of siblings of CPU0
in a loop which is wasteful. Just precalculate the value before the
loop and use it in every iteration instead.

Change-Id: I807fddf29a75af4d268829e37dc91b7512cfcc50
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/13905
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-6285 libcfs: Do not unnecessarily copy cpumask

Copying a mask just to calculate a static value of siblings
is overkill, esp. if CPUMASK_OFFSTACK is set and the mask is huge,
additionally that'll workaround the incorrectness of kernel code
dealing with such a code until that is actually fixed upstream.

Change-Id: I860cc7d5b54adcde2e7622bfbc64b4e27046b083
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/13904
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-5823 clio: use CIT_SETATTR for FSFILT_IOC_SETFLAGS

Add handling of inode flags to the handlers of CIT_SETATTR in lov and
osc. In the FSFILT_IOC_SETFLAGS case of ll_iocontrol() use
cl_setattr_ost() rather than obd_setattr_rqset() to set inode flags on
OST objects. Remove the then unused OBD API methods
obd_setattr_rqset() and obd_setattr_async() along with their
supporting functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I3ccdb139f2e9aa376fb69e353c0cc6d399bf0857
Reviewed-on: http://review.whamcloud.com/13422
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5823 clio: get rid of lov_stripe_md reference

Get rid of lov_stripe_md reference in setting file's stripe info.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I303bfc98113bf1f086053225959001377879637a
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/12639
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-632 utils: fix problems of llog_reader

When the input file of llog_reader is invalid, it is easy to
crash or loop infinitely. This patch fixes these problems.

Signed-off-by: Li Xi <pkuelelixi@gmail.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ifd6bbc5e857f6910bb4103d85742ba33a843d080
Reviewed-on: http://review.whamcloud.com/10764
Tested-by: Jenkins
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@seagate.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6403 quota: fix soft lockup in qmt_adjust_qunit

If the user sets the quota limits >= ULLONG_MAX using lfs setquota,
we will set the limits to ULLONG_MAX and stuck in the infinite loop
when trying to increase qunit.

Break the loop when that happens and set qunit to
limit / (2 * slv_cnt) instead.

Signed-off-by: Li Dongyang <dongyang.li@anu.edu.au>
Change-Id: I6fb842c62ad46d8765f6c4c41187cf0dcd543c53
Reviewed-on: http://review.whamcloud.com/14187
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6142 lnet: enforce Linux kernel coding style

This patch enforces Linux kernel coding style in
lnet codes. The changes are:
- convert spaces to tabs for indentation
- align variable and struct declarations
- sprintf is deprecated, use snprintf instead
- labels should not be indented
- wrap lines at 80 columns

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I46685fec1703c39fc8e9e6ee7bb6e5cc152aac4f
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/14130
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5971 llite: reorganize variable and data structures

Rename struct ccc_grouplock to ll_grouplock and move the definition
from vvp_internal.h to llite_internal.h.

struct vvp_thread_info is used in the non-VVP parts of llite so rename
it struct ll_thread_info. Rename supporting functions accordingly.

struct ccc_thread_info is used in the VVP parts of llite so rename
it struct vvp_thread_info. Rename supporting functions accordingly.

Remove ccc_global_{init,fini}(), merging their contents into
vvp_global_{init,fini}() and {init,exit}_lustre_lite(). Rename
ccc_inode_fini_* to cl_inode_fini_*.

Move several declarations between llite_internal.h and vvp_internal.h
with the goal of reserving the latter header for functions that
pertain to vvp_{device,object,page,...}.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I79b51e6f58dee9e9488c983b4a0759fa4117d2a6
Reviewed-on: http://review.whamcloud.com/13714
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3105 osd: remove capa related stuff from servers

capability feature is broken. it's not going to be used
anytime soon. the related tests are removed as well.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I865a92b57abaae679d7ff8319e0e3fda603beff9
Reviewed-on: http://review.whamcloud.com/5572
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6416 ldlm: no canceled lock on waiting list

If a lock was not granted straight away on server, but it's granted
with LDLM_FL_AST_SENT set before ldlm_handle_enqueue0 sends out
reply, client side will know she needs to cancel this lock.

At the meanwhile, this lock can be added to a long granting list
by another server thread.

When lock cancel request arrives at server and server calls into
  ldlm_lock_cancel()->
      ldlm_cancel_callback()->
          tgt_blocking_ast(...LDLM_CB_CANCELING)->
              tgt_sync()

The other server thread eventually get a chance to send completion
AST for this lock with LDLM_FL_AST_SENT set, and add this lock to
waiting list again.

However, tgt_sync may take arbitrary time which is irrelevant
to AT of lock revoke on client, server could evict client only
because itself has slow IO.

To resolve this race, this patch does not put canceled lock on
waiting list anymore.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I86c1097d3ccbaa614b8811c1d9f37b39f019c61e
Reviewed-on: http://review.whamcloud.com/14085
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6396 kernel: kernel update [SLES11 SP3 3.0.101-0.47.52]

Update target and config files for new version

Test-Parameters: envdefinitions=SANITY_EXCEPT=170\
  mdsdistro=sles11sp3 ossdistro=sles11sp3 \
  clientdistro=sles11sp3 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
  testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4c8d24e45b0e3fbbc31a188ee416860f54371930
Reviewed-on: http://review.whamcloud.com/14164
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 lnet: remove unnecessary goto

The 'out' label and the goto's in brw_client_done_rpc()
are no longer needed, since the user space code has been
removed.

Change-Id: I0f478de2778ec25eea9f7adbd08f4caf0a29668e
Signed-off-by: Isaac Huang <he.huang@intel.com>
Reviewed-on: http://review.whamcloud.com/13698
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 ptlrpc: Add OBD_CONNECT_MULTIMODRPCS flag

The new OBD_CONNECT_MULTIMODRPCS connection flag indicates the support
of multiple modify RPCs in parallel. It can be specified by the client
within the connection request and by the server within the connection
reply.
The new ocd_maxmodrpcs connection data specifies the maximum modify
RPCs in parallel supported by the server.

To allow the MDS to send the new ocd_maxmodrpcs field, it has been
required to modify RMF_CONNECT_DATA so that its size includes the new
field. This change leads to remove the ocd_connect_data_v1 structure.
Note that the client has been allocating an extra 16*sizeof(__u64) for
the obd_connect_data reply since 2.0 (commit fd908da9, and even in
later versions of 1.8) so there is no problem for the MDS to just send
the full reply size.

This patch fixes a bug in __req_capsule_get() since it wasn't checking
RMF_F_NO_SIZE_CHECK when receiving the message. This allows legacy
clients (with version lower that this commit) to send connection
request with ocd_connect_data structure size smaller (actually size is
ocd_connect_data_v1 structure size) than new server ocd_connect_data
structure size.

This patch also fixes a bug in the routine that displays the import's
connect data.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I4b8a567241f8986d967240efff94c7f407fdd864
Reviewed-on: http://review.whamcloud.com/13960
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-5478 osc: get rid of obd_* typedefs

We have a bunch of typedefs for common things that made no sense
and hid the actual type from plain view.
Replace them with proper uXX or sXX types.
Exception is in lustre_idl.h and lustre_ioctl.h where
they are replaced with __uXX and __sXX to be able to be included
in userspace

Also fix valid flags to u64.

patch 6 in series: modify osc/osp

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib18583489ed79eaded6339a283bc48e4fda319f6
Reviewed-on: http://review.whamcloud.com/13148
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4647 tests: modify sanity-sec to test multiple MDSes

This patch modifies santiy-sec to distribute the nodemap to all the
MDSes in the system. It also uses test_mkdir to create directories on
multiple MDTs, allowing DNE and nodemapping to be tested.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
mdtcount=4 testlist=sanity-sec

Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I48f47b72e404614167c715a682d99351295d8189
Reviewed-on: http://review.whamcloud.com/13343
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6325 libcfs: shortcut to create CPT from NUMA topology

If user wants to create CPT table that can match numa topology,
she has to query cpu & numa topology, then provide a pattern
string to describe the topology, this is inconvenient.

To improve it, this patch can support shortcut expression "N" or "n"
to create CPT table from NUMA & CPU topology

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I608f47ad6856ded5bf2f5f223b77b02906ebc8cc
Reviewed-on: http://review.whamcloud.com/14049
Tested-by: Jenkins
Reviewed-by: Olaf Weber <olaf@sgi.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6376 osp: add RPC lock during RPC send

Add RPC lock in osp_remote_sync(), so only one modified
UPDATE RPC is allowed each time as other normal MDC client.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6c766625adc355d5c8827bffb78a1820efc46fd6
Reviewed-on: http://review.whamcloud.com/14098
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6373 llite: default dir stripe index only for mkdir

Default dir stripe index should only work during mkdir,
otherwise it will cause other open/create request being
sent to the wrong MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id95d7218196d52950eceea38be5612ffe4a6b080
Reviewed-on: http://review.whamcloud.com/14096
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6247 tests: fix nodemap quota test to use correct blocksize

The nodemap quota test needs to take into account indirect blocks and
other overhead when deciding if the quota used is excessive. It's
supposed to add the fs blocksize * 2, so 8k on ldiskfs and 256k on
zfs, but it was using the osc block size instead of the osd
blocksize. This patch modifies the test to use the fs_log_size helper
to determine an appropriate fuzz size.

Test-Parameters: mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs testlist=sanity-sec
Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: I3bf6380fd0a0e6b3246343da8a139c6b4ea120ae
Reviewed-on: http://review.whamcloud.com/13844
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5319 mdt: pass __u64 for storing opdata

While lcd_last_data is only __u32, the internal disposition handling
is done with __u64, so it doesn't makes sense to drop the high bits in
mdt_get_disposition(), mdt_set_disposition(), mdt_clear_disposition().

Minor whitespace and prototype cleanups.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia4880b7256564a6799f8c686fd8690ebbf3ebbe5
Reviewed-on: http://review.whamcloud.com/13297
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Grégoire Pichon <gregoire.pichon@bull.net>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6081 lfs: split setstripe and migrate help

The setstripe and migrate share all but one of their parameters
(--block). However the help is common to both, so the --block also
appears for the setstripe command but is not valid. So split the help
message for each command, while still sharing the same text to avoid
duplication.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: If8d110b04b26090d0540bb1628de75a94d92727e
Reviewed-on: http://review.whamcloud.com/13317
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6387 build: add support for power8

Expand lustre to natively support the little endian Power8
platform. The architecture is reported as powerpc64le so
add that support to lustre-build-linux.m4. For the Ubuntu
packaging the platform is reported as ppc64el so include
that to the debian build configuration.

Change-Id: I506fd7228579fe1bd6e5c9e9d39db6ca06e4768d
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/14115
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4688 mdt: remove export_put() from mdt_export_evict()

This export reference dropping is not needed here, export
referenced from ptlrpc level and will be dropped there as
well. It looks like that call to the class_export_put()
was added just similar to other places where the
class_fail_export() is called but in those cases export
reference was taken right in the same function while here
it is taken by external caller and will dropped there.

Test-Parameters: envdefinitions=SLOW=yes alwaysuploadlogs testlist=replay-dual
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I560271668c10715c7f6caa02bfbf2fccab3eeade
Reviewed-on: http://review.whamcloud.com/13706
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.7.51

Change-Id: If0b670b630c0df80474f5a77e159da1752db39db

LU-6020 kerberos: readdir bulk replies are not wrapped

target_bulk_io() does not wrap readdir replies causing readdir errors:
gss_cli_ctx_unwrap_bulk() bulk security descriptor mismatch: (0,0,2) != (0,0,0)
ll_get_dir_page() read cache page: [0x200000007:0x1:0x0] at 0: rc -71
ll_dir_read() error reading dir [0x200000007:0x1:0x0] at 0: rc -71

Change-Id: Ifa11e1f6bbc6ae8a3a7c4f296f055fd38dabd6aa
Xyratex-bug-id: SNT-15
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: http://review.whamcloud.com/14020
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6020 kerberos: bulk nob is not corrected on bulk writes

The real transferred block in the end of a file can
have a size different from the original block in
privacy mode. The proper fix should probably just
mimic the behaviour of the client.

Change-Id: I594c116c78b7746f4881e0de8b7cc63b37268381
Xyratex-bug-id: SNT-15
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: http://review.whamcloud.com/14018
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6081 lib: don't ignore the return of read()

The return of read() must be checked on some platform, possibly with a
recent version of glibc. Otherwise this gives an "ignoring return
value of 'read'" warning, which breaks the build. It happens on Ubuntu
14.04.

Use the return of read() to the random entropy to fix the issue. This
doesn't do much for the entropy, but it doesn't hurt either.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Iee7c1bce818e2f163db8f860acf8be075f5a543e
Reviewed-on: http://review.whamcloud.com/13654
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6159 hsm: add CL_CLOSE to default changelog mask

There's no point in ignoring CL_CLOSE by default in changelogs.
Robinhood needs these events else the database quickly
becomes out of sync. So let's have it by default.

Note that CL_CLOSE is not issued when the file was opened in read-only
mode.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ie5f42bc4413259e5079801a204e15125cde0c48b
Reviewed-on: http://review.whamcloud.com/13526
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5823 clio: remove IOC_LOV_GETINFO

* In cb_find_init() (lfs find) use some variant of stat() to get file
  size and times.
* Remove the then unused IOC_LOV_GETINFO ioctl.
* Remove ll_glimpse_ioctl() and ll_lsm_getattr().
* Remove the OBD API method obd_getattr_async().

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8fb6b69b1c94f0522a1405f1105ab0b7a2041601
Reviewed-on: http://review.whamcloud.com/12748
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5823 clio: add coo_obd_info_get and coo_data_version

* Add coo_obd_info_get to retrieve object attributes from servers.
* Add coo_data_version to retrieve object's data_version.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ia66a3ba1eee5c6478f3aa5c9f942ada77b2b5fe9
Reviewed-on: http://review.whamcloud.com/12638
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6047 mdt: remove Size on MDS support

Remove size on MDS support from lustre/mdt/. In struct mdt_object
change the struct mutex mot_ioepoch_mutex member to spinlock_t
mot_write_lock and rename mot_writecount to mot_write_count.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I271117618f7b88a22ddbcca4db5a4723ab48e3ea
Reviewed-on: http://review.whamcloud.com/13442
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6313 tests: more robust for scrub test_11

For the sanity-scrub test_11, except for the known created by the
test scripts, there may be other objects (such as for llog) have
been created before the first OI scrub scaning. So it is not easy
to estimate how many objects should be skipped during the first
OI scrub scanning. So we only check that the number of skipped
files is more than the number or known created.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iced9fb255559394117880514c5e716d05a81a177
Reviewed-on: http://review.whamcloud.com/13957
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6335 kernel: kernel upgrade [RHEL7.1 3.10.0-229.el7]

upgrade from RHEL7.0 to RHEL7.1 3.10.0-229.el7 kernel

Test-Parameters: clientdistro=el7 testgroup=review-ldiskfs \
  mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I6b733eb4571b57339889e927c4658c02e7ac7f34
Reviewed-on: http://review.whamcloud.com/14090
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6221 utils: hsm_root is also required for --dry-run

Not specifying hsm_root in copytool/lhsmtool_posix command line for
--dry-run mode can lead to failure/error.
This path ensures that hsm_root will be required even for --dry-run.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Icaa2af6d1365751d9e77b2be3f60aacc9c1f6a5c
Reviewed-on: http://review.whamcloud.com/13673
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6378 kernel: simplify quota-avoid-dqget-call.patch

Backport the patch from upstream kernel, which doesn't rely
on the I_NEW to skip dqget()/dqput() calls, it should check
the i_dquot directly instead.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I10e2e8284704bc7cf9ffae4ee88f06fafef14b1a
Reviewed-on: http://review.whamcloud.com/14135
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-6356 ptlrpc: ret -ECONNREFUSED if not context found in req

Return -ECONNREFUSED instead of -ENOMEM in sptlrpc_req_get_ctx()
if no context is found in req.
It it more graceful?

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: If1b142199a94d1976093a7d26a05e49a63f50469
Reviewed-on: http://review.whamcloud.com/14043
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: cleanup libcfs lock handling

Previously with libcfs being built for user land and kernel
space wrappers were created to transparently handle locking.
Now that user land support has been removed we delete all
those locking wrappers with this patch.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Icbd9b5c0918cb01202439416b220b6f327144a91
Reviewed-on: http://review.whamcloud.com/13793
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6245 lnet: remove kernel defines in userland headers

Currently the lnet headers used for user land applications
contain various kernel definations. This is due to the
fact libcfs contains kernel wrappers for user land which
will be going away. This patch sorted the header data
so all kernel containing structures are moved out of
headers that user land will use.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I3904cd692bf2debd3123cbf8ca98dfc518ce0a97
Reviewed-on: http://review.whamcloud.com/13792
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6395 mgc: one byte shorter for logname allocation

One byte shorter for logname allocation in mgc_llog_local_copy(),
which might cause buffer overflow in the following sprintf().

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie758c3650c1cf7848874d9fd3a02a5618043eb8f
Reviewed-on: http://review.whamcloud.com/14146
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5953 build: use installed OFED by default

During LNET autoconf phase, if OFED installed and its devel headers
are available, default to use it instead of in-Kernel IB driver.

Also handle wrong case where OFED installed but not devel preventing
to build against OFED. Had to add new "patches" vs "kernel_patches"
dir name use in recent OFED versions and to avoid its check to
collide with inkernel-IB builds case.

Current OFED headers detection mechanism allow for non-standard
prefix but relies on "ofed_info" command and on "%prefix/openib"
link (both are ok for 1.5.x and 3.x versions), and should work
for both source and DKMS Lustre builds.

Test-Parameters: nettypes=o2ib
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I82639f8392d5fe707a3b1a1719d53ab937e918b5
Reviewed-on: http://review.whamcloud.com/12686
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6218 osd-zfs: increase redundancy for meta data

Use DMU_OTN_UINT8_METADATA for local objects so their
data blocks would get an additional ditto copy. This
increases redundancy and hence the chance of recovery
by zpool scrub in the event of corruption.

Change-Id: I502da680521027733ea53744905c47f569a1b531
Signed-off-by: Isaac Huang <he.huang@intel.com>
Test-Parameters: mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs
Reviewed-on: http://review.whamcloud.com/13741
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5757 hsm: strengthen checks for flags and archive id

Prior to this patch undefined flags bits and out of range
archive id can be set.
Also changed the concerned error handling that has been
recently added (LU-5732) as part of sanity-hsm/test_500.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I64403de4529f0214bab55c2fc13281b0a3d30a11
Reviewed-on: http://review.whamcloud.com/13337
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: move lucache from libcfs to lustre

The lucache handling in libcfs is used only for
idmap handling in the obdclass and mdt layers.
Since this is the case we can move the lucache
handling into the lustre stack. As a bonus the
lucache will only be built when we enable server
support instead of the current state of it being
built for clients as well.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I812f2c5952ea79bd023435e5fac1955316c9c59f
Reviewed-on: http://review.whamcloud.com/13783
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: remove tcpip abstraction from libcfs

Since libcfs no longer builds for user land we can
move the tcpip abstraction that exist to the LNET
layer which is the only place that uses it. Also
the migrated code will use native linux kernel
apis directly instead of with wrappers.

Change-Id: Iaa39e4f581f18cfe586feb5bfbf4233a2f2335c7
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/13760
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5823 clio: add cl_object_fiemap()

* Add cl_object_operations::coo_fiemap().
* Add cl_object_fiemap() to get FIEMAP mappings.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ie32eb5ddb8d2daa1a66055f347cef4757d039e75
Reviewed-on: http://review.whamcloud.com/12535
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6319 tests: Clean up sanityn ALWAYS_EXCEPT list

At some point between Lustre 1.8 and 2.1, sanityn test 22 was removed.
Test number 22 is still included in the ALWAYS_EXCEPT list and in the
EXCEPT list; the list of tests that will not be run under normal
(autotest) testing. Remove test 22 from the ALWAYS_EXCEPT and EXCEPT
lists.

Also, tests 11 and 14 are skipped for SUSE10. All Lustre branches from
b2_4 to current master are not built for and are no longer tested on
SLES10. Remove the check for SUSE10 and remove tests 11 and 14 from
the ALWAYS_EXCEPT list.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I2222770874c6c2da816cfd54b371ddc9c0da370b
Reviewed-on: http://review.whamcloud.com/13953
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5929 tests: conf-sanity test 72 call tune2fs on MDSs

The call to tune2fs tuning MDTs with "-O extents" is now
run on the MDS node(s) and not the client.

Test-Parameters: alwaysuploadlogs testlist=conf-sanity

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ie4e1bb2d9447d86c9b0144e8f2564b5c2444842d
Reviewed-on: http://review.whamcloud.com/12761
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5504 utils: add const qualifier to changelog accessors.

Commit 6e1365 changed 4 functions, and commit 0f22e4 accidentally
reverted them.

This patch put them back, as well as adding new ones to
changelog_rec_size(), changelog_rec_varsize(), changelog_rec_rename()
and changelog_rec_jobid().

6e1365 also changed changelog_rec_name() and changelog_rec_sname() to
return a const, but that is not possible anymore.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I19e389c422795c2ece4d7af369099cb733d3cb1a
Reviewed-on: http://review.whamcloud.com/13787
Tested-by: Jenkins
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6322 lfsck: show start/complete time directly

It is more easy for the users to use/understand when the LFSCK
was started and/or when the LFSCK completed by showing related
time directly.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ibdacccf1abba6041eaddd6bb5456fb122e9ca994
Reviewed-on: http://review.whamcloud.com/13948
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6317 lfsck: NOT count the objects repeatedly

The namespace LFSCK uses object-table based iteration plus namespace
based directory traversing to scan the system. So one object will be
returned twice by them. Counting the objects repeatedly will confuse
the users. So the namespace LFSCK should only count the objects that
are scanned via namespace based directory traversing.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I36459743843e1db1e9372d46d3aafddef033d699
Reviewed-on: http://review.whamcloud.com/13933
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6316 lfsck: skip dot name entry

It is unnecessary for the namespace LFSCK to verify the dot
entry since it is always on the local MDT and has no linkEA.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I01289b04c8807e930c6f777007f1e1fb3295431d
Reviewed-on: http://review.whamcloud.com/13923
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6235 osd-ldiskfs: NOT skip LMAC_NOT_IN_OI in osd_check_lma

Sometimes, the ost-object may references a wrong indoe because of
the invalid OI mapping. Usually, the OSD can auto detect that via
osd_check_lma(). For old system, if the inode's LMV contains flag
LMAC_NOT_IN_OI, it would skip related checking. But such behavior
is wrong, if may cause the osd-object to reference some important
system inode, and cause system crash via subsequent modification.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ib5cdf2ee4d9893a87fde3caf81109eabdad9ecfa
Reviewed-on: http://review.whamcloud.com/13922
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5682 lfsck: optimize ldlm lock used by LFSCK

When LFSCK repairs some inconsistency, it needs to take related
ldlm lock(s) firstly to prevent concurrent modifications or purge
client side cache. Originally, to simply the implementation, the
LFSCK just simply acquires LCK_EX mode ibits lock(s) on related
object(s). But such coarse-grained lock policy may be not efficient
for some directory-based modification, such as insert name entry to
the directory.

This patch introduces lfsck PDO (Parallel Directory Operations) lock
for directory-based LFSCK modification, it only locks part of the
directory with the given <object, name> pairs, then allow others to
access or modify the different part(s) of the directory in parallel,
and also avoid to purge client-side cache unnecessarily.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I29bad81112c14e3aaecaa2b808e60ea74c10a702
Reviewed-on: http://review.whamcloud.com/12766
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6129 lnet: DLC design doc

Add a DLC design document to lustre/doc/

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I3ec283f960aba7b56afadcc1d2a7770604efb023
Reviewed-on: http://review.whamcloud.com/13419
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6384 mdt: propagate find errors in mdt_fid2path()

In mdt_fid2path() propagate the specific error from mdt_object_find()
rather than returning -EINVAL.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib09f3741f95c0f3484f9a7839e31c583ecc34761
Reviewed-on: http://review.whamcloud.com/14108
Tested-by: Jenkins
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-6134 utils: lfs should only open/stat files if needed

Since (commit 322968acf183) "lfs find" would needlessly open() and
fstat() every file if the --ost, -uid/user, -gid/group, -[amc]time,
or -size options were used, to get the MDT index for each file.
This was causing "lfs find --ost" to fail if an OST was offline, and
added needless overhead that "lfs find" was meant to avoid.

The MDT index is only needed if --mdt is used, so only get it in
that case.  It also wasn't necessary to call fstat() in this case
either because the file type was already known at this point.

Some other minor cleanups related to fetching the MDT index:
- don't use ret in cb_get_dirstripe() as it is isn't needed
- fix cb_get_mdt_index() to avoid a Coverity false positive due to
  initializing rc and having a conditional branch that is always taken
- convert spaces to tabs for related code, other minor style fixes

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ib41ced742fe5068f504f540479e6b4718d2540e5
Reviewed-on: http://review.whamcloud.com/13822
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6345 test: compare /bin/sleep in sanity-hsm.sh test_30c

In case /bin/sleep is modified during the test, we do a checksum
at the beginning and the end of the test respectively, and won't
mark the test a failure if the checksum has changed.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I1ff472ea6052e2df9ba9fd4c78a4cf53686e1ccd
Reviewed-on: http://review.whamcloud.com/14025
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6014 mdt: remove unused function

mdt_trans_stop() is not used.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I34da179288bef9a928173aa79e53cc74082abed7
Reviewed-on: http://review.whamcloud.com/12326
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-6283 ptlrpc: re-add NRS policy registration symbol exports

Export the ptlrpc_nrs_policy_(register|unregister)() functions, in
order to allow modules other than ptlrpc to load NRS policies on
demand.

These symbols were unexported as part of a subsystem-wide
symbol-unexporting effort for PTLRPC by commit 3ee0e09.

Signed-off-by: Nikitas Angelinas <nikitas.angelinas@seagate.com>
Xyratex-bug-id: MRP-2489
Change-Id: Ic294a94202fb644f997f11b931f7f9bc36d221ba
Reviewed-on: http://review.whamcloud.com/14003
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Chris Horn <hornc@cray.com>

LU-6219 utils: remove O_NONBLOCK usage for archive file

In the first implementations of Posix copytool, file archive/restore
was using a loop around select() with handling/retry on EAGAIN.
This was later found as useless for regular files and fixed/changed
in patch for LU-3971 (http://review.whamcloud.com/7583, commit
397ebc93cef378e6d77450cdd095e2737b94f2f6).
But the O_NONBLOCK flag usage, during open() of file on archive, has
been kept since and must be removed, even if ineffective, to improve
code clarity.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ie6350e6f3951545f50783c9fff6753793b7a9a33
Reviewed-on: http://review.whamcloud.com/13672
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Robert Read <robert.read@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6205 tests: fix bash expansion of fid

When calling lfs path2fid, the FID is returned between bracket. When
that fid variable is used, it may be expanded by the shell to
something else. For instance:

  $ touch x
  $ ../utils/lfs fid2path lustre [0x200000be7:0xb:0x0]
  bad FID format [x], should be [0x1:0x2:0x0]

  fid2path: error on FID x: Invalid argument

This will cause some tests, like 154A or 238, to sometimes fail.

Use quotes where the FIDs are used.

Replace "$(lfs ..." with "$($LFS ..." and made a couple variables
local.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I3c1a34585ebaa596d66063f5ada3ccfc4d202ade
Reviewed-on: http://review.whamcloud.com/13618
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-5829 ldlm: remove unnecessary EXPORT_SYMBOL

A lot of symbols don't need to be exported at all because they are
only used in the module they belong to.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ic182e844d621e6ba3c22e685c72b3702ccbb793b
Reviewed-on: http://review.whamcloud.com/13324
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5829 lnet: remove unnecessary EXPORT_SYMBOL

A lot of symbols don't need to be exported at all because they are
only used in the module they belong to.

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I8374d2da55d839e361be5721425d7270425f2286
Reviewed-on: http://review.whamcloud.com/13320
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6078 utils: fix copytool file bounds checking

Strengthen copytool file bounds checking in either full
or partial/extent mode.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I36939f9a18362ca3131e39b8d390978dfc79405a
Reviewed-on: http://review.whamcloud.com/13226
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4820 osd: drop memcpy in zfs osd

dmu_read() was called from osd_read_prep() copying from
ARC bufs into the same ARC bufs. seem to be the remainings
of pre-zerocopy age.

Change-Id: I87c10a2d484b7fe0be370349a2bfeb857ddd74e9
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: http://review.whamcloud.com/12991
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4239 tests: test FID related APIs

This adds a few stress tests to the user lustre API, related to FIDs.

Change-Id: I34144a8f4c446e55c6630d31cae6a133d61eb304
Signed-off-by: Frank Zago <fzago@cray.com>
Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=154g testlist=sanity
Reviewed-on: http://review.whamcloud.com/12545
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5657 doc: allow the use of rst2man to build man pages

The man page sources can now be written in reStructuredText (rst), and
the man pages will automatically be generated with rst2man.

Added a build dependency on rst2man and the package python-docutils.

Converted lustreapi.7 to ReST to validate the solution.

Change-Id: I69e9892a238a002eb86769ed65b758cba55543bb
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/12040
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>