LU-4086 tests: strengthen sanity-hsm/test_33

Instead of waiting+checking both Restore and Cancel HSM operations
sequentially, wait for both to be finished and then check each
results. This will allow to handle conditions where Restore
terminates before the Cancel is started.

Also added to wait_all_done() the capability to only work on
requests for a specific fid.

Lustre-change: http://review.whamcloud.com/9600
Lustre-commit: adc6a5e76348d9bb1c8355103f419405a14decca

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I480ce074513d6829ecb3078c273bac1fdffa6256
Reviewed-on: http://review.whamcloud.com/9600
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit adc6a5e76348d9bb1c8355103f419405a14decca)
Reviewed-on: http://review.whamcloud.com/10712

LU-5195: hsm: delete HSM records not found on disk

After a MDS crash, it is possible the file containing an hsm record is
not present anymore. When the MDS restarts, we get traces like this:

(mdt_hsm_cdt_actions.c:104:cdt_llog_process()) tas01-MDT0000:
    failed to process HSM_ACTIONS llog (rc=-2)
(mdt_hsm_cdt_actions.c:104:cdt_llog_process())
    Skipped 600 previous similar messages
(llog_cat.c:192:llog_cat_id2handle()) tas01-MDD0000:
    error opening log id 0x1c:1:0: rc = -2
(llog_cat.c:192:llog_cat_id2handle()) Skipped 600 previous similar messages
(llog_cat.c:556:llog_cat_process_cb()) tas01-MDD0000:
    cannot find handle for llog 0x1c:1: -2
(llog_cat.c:556:llog_cat_process_cb()) Skipped 600 previous similar messages
(mdt_hsm_cdt_actions.c:104:cdt_llog_process()) tas01-MDT0000:
    failed to process HSM_ACTIONS llog (rc=-2)
(mdt_hsm_cdt_actions.c:104:cdt_llog_process())
    Skipped 600 previous similar messages
(llog_cat.c:192:llog_cat_id2handle()) tas01-MDD0000:
    error opening log id 0x1c:1:0: rc = -2
(llog_cat.c:192:llog_cat_id2handle()) Skipped 600 previous similar messages

No HSM operation can happen, and the only way to clean is to unmount
the MDT and delete hsm_actions.

If the record can't be found, let the MDS delete it instead.

This is a backport to b2_5.

Lustre-change: http://review.whamcloud.com/11419
Lustre-commit: 3a83b4b93373db46a4ee60a7388775fa0be2eb9a

Change-Id: I61f625d4c18750c8044909ff56d53042cf0b6d86
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/11419
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ryan Haasken <haasken@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 3a83b4b93373db46a4ee60a7388775fa0be2eb9a)
Reviewed-on: http://review.whamcloud.com/11619
Reviewed-by: Patrick Farrell <paf@cray.com>

LU-5299 obdclass: avoid race during Server device start

Handle concurrent starts for same device (multiple mounts, ...).
But allows for separate nosvc and nomgs case.
Also add a specific test of concurent MDT/OST start with an
artificial delay to verify.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I442819a5b865ed3e98477f9d2602efc4d09d7860
Reviewed-on: http://review.whamcloud.com/11139
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1279 kernel: Fix concurrent module loading deadlocks

Concurrently starting multiple OSTs on a single OSS frequently
triggers 30s deadlocks on module_mutex.  This RHEL 6 kernel bug
applies to any module that results in additional request_module()
calls in its init callback.  In Lustre, at least ptlrpc and libcfs are
affected.  (RHEL 7 should have enough fixes in this area, but testing
needs to be done.)  This patch adds a fix adapted from a number of
upstream commits to the RHEL 6 kernel patch series.

Change-Id: Ibdd384fd7622a0b4fcbf4cb5fdb864de87fcc25e
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/11229
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-5428 libcfs: false alarm of libcfs watchdog

lc_watchdog_disable will not delete kernel timer, benefit of this
is no overhead of del_timer in each serivce loop, however, there
is a corner case that lc_watchdog_touch can race with lcw_cb when
service thread is actually idle:

1. service thread armed a timer before processing request, after
   it handled request, it called lc_watchdog_disable() and set
   lc_watchdog as disabled, but kernel timer is still outstanding.
2. service thread slept for many seconds because there is no
   request to handle
3. it's waken up by incoming request
4. it called lc_watchdog_touch and set watchdog status to enabled.
   now because timer is still alive, if timer is expired right
   before cfs_timer_arm(), then lcw_cb will set a disabled watchdog
   as "expired", which is wrong.
5. the next call of lc_watchdog_disable will complain for expired
   watchdog.

There are two options to fix this issue:
1. always del_timer in lc_watchdog_disable, however, it may increase
   system overhead when service threads are busy.
2. set watchdog as "enabled" after cfs_timer_arm, so there is no
   window for lcw_cb to see false status of watchdog.

This patch chooses the second approach to fix the problem

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Iac24082e2d63de8330285cf243ed585da6524ab9
Reviewed-on: http://review.whamcloud.com/11415
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5405 llog: add newly opened llog at tail of handle list

Add newly opened llog handle at the tail of handle list to increase
lookup speed, especially for cancel operation, because the canceled
log will be removed from the list, and lookup is from the beginning.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I4c5c22901ea5818a8ee50ef97d6dabe4839b9a74
Reviewed-on: http://review.whamcloud.com/11575
Tested-by: Jenkins
Reviewed-by: Ian Costello <costello.ian@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Conflicts:
lustre/obdclass/llog_cat.c

LU-4933 osc: Automatically increase the max_dirty_mb

When RPC size or the max RPCs in flight is increased, the actual
limit might be max_dirty_mb. This patch automatically increases
the max_dirty_mb value at connection time and when the related
values are tuned manually by proc file system.

This patch also changes the unit of "cl_dirty" and "cl_dirty_max"
in client_obd from byte to page.

Lustre-change: http://review.whamcloud.com/10446/
Lustre-commit: bdc5bb52c55470cf8020933f80e327c397810603

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I8480122b7370247b17de81d731c2f2b5f67892ce
Reviewed-on: http://review.whamcloud.com/10937
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-4855 fld: refer to MDT0 for fld lookup in some cases

It is possible that when fld client is trying to lookup seq
on one of MDT, but the connection between the client and the MDT
is not being initialized yet, especially during striped dir creation,
because client will only send create req to the master MDT, then
master MDT will distribute the operation to all of other MDT, instead
of client distributing these requests, which will usually trigger
the connection.

In this case, we will send the fld request to MDT0, since it has
all of location information.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I577ada2a782ecbdc863d05e8d007c0914a1e7837
Reviewed-on: http://review.whamcloud.com/9877
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/11780

LU-5144 mdt: rename vs link deadlock

it may happen that tgt child for rename is a directory, which is used
as tgt dir for link in parallel, as the result rename locks src child,
tgt child what may deadlock with tgt dir, src file locking of link.

As not dir cannot replace dir, check it before locking the tgt child.

Lustre-commit: b2c2cfc24fa461b3e54cd204a697eeee9a89d49c
Lustre-change: http://review.whamcloud.com/10570/

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=55 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanityn

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Signed-off-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Change-Id: I09e96b132d9069d4818e8dc2628c903eff38e53d
Reviewed-on: http://review.whamcloud.com/10918
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4725 mdt: child-parent lock ordering in rename

change rename so that it always has parent-child lock ordering,
otherwise it may deadlock with other operations.

Lustre-commit: 4e308ef74f271ec7e19917e3c0f88586537582c3
Lustre-change: http://review.whamcloud.com/9538/

LU-4725 obd: lu_object_find_at hung

lu_object_find_at hungs if called 2 times and object is removed
in between. It makes mdt_rename_sanity not workable for rename.
Change mdt_rename_sanity to work on existing object.

Lustre-commit: b7c72ec1ddeda2cf94ea151f974d3f94e3a7d1ed
Lustre-change: http://review.whamcloud.com/10484/

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=54 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanityn

Signed-off-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: Ic9ce52bfcd8788834347fba155cc8c6be674dcd8
Reviewed-on: http://review.whamcloud.com/11615
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5114 utils: Fix autoconf warning

Fix mount_osd_ldiskfs_LDADD autoconf warning.  This corrects a fix
left out of commit 0d77156572c2b77581f01d960c0e5738b369de6a.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2815f093951c215a8c92678348444cbd18297e25
Reviewed-on: http://review.whamcloud.com/12005
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5040 osd: fix osd declare credit for quota

osd_attr_set() always calls ll_vfs_dq_init() to initialize dquot for
the inode, while in some cases osd_declare_attr_set() does not reserve
credit for it.

This patch fixes this issue. This patch also corrects the quota credit
accounting in osd_declare_qid() by judging whether the inode's
i_dquot[quota_type] has been allocated, while old code judgement is
based on the file uid/gid existence, which is not correct.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I16555cb1097e1a3e75cdcb4852a2c5e1772ddd88
Reviewed-on: http://review.whamcloud.com/11097
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-3489 osd-zfs: varsize index support"

This seems to be causing LU-5649

This reverts commit d9e627c09ccc5c22aa10704020fb9f45bcf26ed0.

LU-5349 include: use __vmalloc() to avoid __GFP_FS default

When possible, try to Direct use of __vmalloc() which allows
for protection flag specification, and particularly to not
set __GFP_FS, which can cause some deadlock situations in our
code.

This patch is the b2_5 back-port of :
Lustre-commit: 635421641bfa8a412edebd4ffbbdf05fd0c47410
Lustre-change: http://review.whamcloud.com/11190/

Also corrects a typo in __OBD_VMALLOC_VERBOSE name.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I588ebf27d1ff7538745b732d174696c379107ea3
Reviewed-on: http://review.whamcloud.com/11739
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5610 tests: Handle quoted module options

When test-framework.sh translates module options to environment
variables for remote nodes, quotes should be escaped to the subshell.

This patch is back-ported from the following one:
Lustre-commit: 4597789df0b6b9fcc7bcbe5529adab3522f0e6ac
Lustre-change: http://review.whamcloud.com/11887

Signed-off-by: Stephen Champion <schamp@sgi.com>
Change-Id: I02daf5ca36170ea901f6b0e4e5152cf4835945dc
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11907
Tested-by: Jenkins
Reviewed-by: Stephen Champion <that.schamp@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5600 kernel: kernel update RHEL6.5 [2.6.32-431.29.2.el6]

Update RHEL6.5 kernel to 2.6.32-431.29.2.el6

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4832cf12cb49eebd1c81fc4e07a53d0a1315500d
Reviewed-on: http://review.whamcloud.com/11838
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5612 osd: typo in osd_declare_write()

Some bits are lost when porting the fix of LU-4611
3902ff4c54925b2f1fcb732a32ed7ee5428e9f77

That could result in insufficient credits in certain situation.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I828a75c674978278b5adaf210586baa177ea0dc9
Reviewed-on: http://review.whamcloud.com/11889
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5434 mdd: disregard empty POSIX ACLs

Some files may have a POSIX ACL consisting of only a struct
posix_acl_xattr_header, resulting in unexpected permission errors.
Update mdd_check_acl() to return -EAGAIN when such ACLs are
encountered so the caller falls back to standard UNIX
permissions.

These empty ACLs originate from tools like cp. In some cases (e.g.
'cp -rp') just POSIX_ACL_XATTR_VERSION is given as the value of the
system.posix_acl_default extended attribute.  Lustre accepts and
stores this value on disk, and the bad default ACL then propagates
from directories to newly created directory entries.  Separate
patches will prevent the storage of these bad ACLs.  This patch
mitigates problems caused by the ones already stored.

Other minor updates:
- Use the kernel function posix_acl_xattr_count() to count the
  number of ACL entries.
- Convert remainder of mdd_check_acl() to TAB indentation.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: Ib4f7b3874696b7645b9de3f1f10ef45dc4105646
Reviewed-on: http://review.whamcloud.com/11300
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5150 acl: 'cp -rp' set empty default ACL

'cp -rp' will set an empty default ACL on target, MDD should treat
this as removing specified ACL.

add sanity 103c to verify this.

Signed-off-by: Lai Siayo <lai.siyao@intel.com>
Change-Id: I2b6947e241675c291a43999e6324bbb295d3c88d
Reviewed-on: http://review.whamcloud.com/11158
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3660 acl: support mount option "noacl" for zfs

zfs parse mount option in zpl layer, but osd-zfs calls dmu
directly, since ldiskfs parse "noacl" at mount time, osd-zfs
should do the same in osd_mount().

Similar to ldiskfs, osd-zfs should check whether ACL is enabled
for ACL operations, if not -EOPNOTSUPP is returned.

add sanity test 103b for "noacl" mount option.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I11075a9de8759fde956e7ae8b715840a56d0428e
Reviewed-on: http://review.whamcloud.com/11157
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4334 utils: Only set a single property for nodes

For servicenode, failnode and mgsnode only set a single property but
concatenate data.  ZFS can only set a single property for a given
name so this prevents erroneously overwritting previous entries.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ia0316e138b0d68dcda1d14811e43db3bbed64457
Reviewed-on: http://review.whamcloud.com/11161
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3489 osd-zfs: varsize index support

LFSCK needs indices with keys/values of different size.
add a support for this to osd-zfs. key/value size can't
vary within a single index object so far.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6faa730ce1f778563c1fcf43d3b50c5b62c899d9
Reviewed-on: http://review.whamcloud.com/7152
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.5.3-RC1

Change-Id: I386568268b31090e3909519ff1cfcc304400a436

LU-4167 tests: correct version check to enable ff_convert

In conf-sanity.sh test_32d, filter fid convertion can be made only
when both of the following conditions are satisfied:
- client img version < 2.3.64 (struct filter_fid_old has been
introduced since 2.3.64)
- ost server version >= 2.5

Also, this patch fix the ofd_iocontrol() message to print the ioctl()
CMD argument as hex instead of a signed integer.

Test-Parameters: testlist=conf-sanity envdefinitions=SLOW=yes,ONLY=32 ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 ossdistro=el6 mdsdistro=el6
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I43b2fc9b62b84b37f6fc8245ede584547a53c878
Reviewed-on: http://review.whamcloud.com/11574
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5095 hsm: Allow receiving messages to be non-blocking

When using the HSM API for a copy tool, it's impossible to
properly exit an application because one thread might be
stuck in llapi_hsm_copytool_recv(), which in turn blocks
forever on a read.

So this patch expands the existing Lustre userspace HSM API
to be able to apply flags to the read side of the
communication pipe, so it can be opened with O_NONBLOCK,
and gives access to the pipe file descriptor used. (read side)

So now a program can do a poll/select, and then call
llapi_hsm_copytool_recv without blocking.

Notes: The flags passed in from the copy tool to
llapi_hsm_copytool_register are applied to the read side of
the pipe using fcntl, because we do not want to apply
the same flags to the read and write sides of the pipe.

No functionality is added for setting flags on the write
side of the pipe, since the kernel expects particular
behavior from that side of the pipe, and it seems unwise
to let user space modify it.

Additionally, this patch changes llapi_hsm_copytool_recv to
not return -EAGAIN when it finds a message for an archive
not serviced by that copytool.

Instead of returning -EAGAIN to the copy tool, which just
restarts the receive operation, this patch just loops inside
the recv function.

This is because this -EAGAIN conflicts with -EAGAIN
returned from the call to libcfs_ukuc_msg_get
which can occur when O_NONBLOCK is set.

This is a backport to b2_5

Lustre-change: http://review.whamcloud.com/10393
Lustre-commit: 35b3a429d1915ec147f01a42c4ec0526b887d1c7

Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I3e611d3c259de9a8c30f3939fb5e48ab88210c2e
Reviewed-on: http://review.whamcloud.com/11220
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5388 tests: add version check code into replay-dual test 24

This patch adds Lustre version check code into replay-dual test 24
to make the test interoperate with servers that do not have change
of http://review.whamcloud.com/11025 .

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=24 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=replay-dual

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Id4bf872ee4e2d478c2b6cc809711692084e990fd
Reviewed-on: http://review.whamcloud.com/11539
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5363 kernel: kernel update [SLES11 SP3 3.0.101-0.35]

Update target and config files for new version.
Back out change from LU-4351 that skips sanity test 54c.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I66713caa53d078f62a345a0cb3e32f0d426a7e36
Reviewed-on: http://review.whamcloud.com/11140
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-3929 lwp: don't connect LWP to old MDT"

This causes LU-5298

This reverts commit 220c4ec0eb7b1f2145c55b923fe1461383f81a77.

Change-Id: Ifffa567650e0da3408625a1f4cca2dac473db5a3

LU-5252 hsm: llapi_hsm_event_fd is local, so make it static.

This is a backport to b2_5.

Lustre-change: http://review.whamcloud.com/10808
Lustre-commit: 9feb80c65b077204f7091e90f314098266fb7e82

Change-Id: I9905d0251702a11a9c2546ba4bd78fd00658d28a
Signed-off-by: Frank Zago <fzago@cray.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11362
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4090 kernel: a kernel patch for jbd2 hung

patch get from linux vanilla kernel commit
0ef54180e0187117062939202b96faf04c8673bc (v3.10-rc2)

jbd2:drop checkpoint mutex when waiting in __jbd2_log_wait_for_space()

While trying to debug an an issue under extreme I/O loading
on preempt-rt kernels, the following backtrace was observed
via SysRQ output:

rm              D ffff8802203afbc0  4600  4878   4748 0x00000000
ffff8802217bfb78 0000000000000082 ffff88021fc2bb80 ffff88021fc2bb80
ffff88021fc2bb80 ffff8802217bffd8 ffff8802217bffd8 ffff8802217bffd8
ffff88021f1d4c80 ffff88021fc2bb80 ffff8802217bfb88 ffff88022437b000
Call Trace:
[<ffffffff8172dc34>] schedule+0x24/0x70
[<ffffffff81225b5d>] jbd2_log_wait_commit+0xbd/0x140
[<ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
[<ffffffff81223635>] jbd2_log_do_checkpoint+0xf5/0x520
[<ffffffff81223b09>] __jbd2_log_wait_for_space+0xa9/0x1f0
[<ffffffff8121dc40>] start_this_handle.isra.10+0x2e0/0x530
[<ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
[<ffffffff8121e0a3>] jbd2__journal_start+0xc3/0x110
[<ffffffff811de7ce>] ? ext4_rmdir+0x6e/0x230
[<ffffffff8121e0fe>] jbd2_journal_start+0xe/0x10
[<ffffffff811f308b>] ext4_journal_start_sb+0x5b/0x160
[<ffffffff811de7ce>] ext4_rmdir+0x6e/0x230
[<ffffffff811435c5>] vfs_rmdir+0xd5/0x140
[<ffffffff8114370f>] do_rmdir+0xdf/0x120
[<ffffffff8105c6b4>] ? task_work_run+0x44/0x80
[<ffffffff81002889>] ? do_notify_resume+0x89/0x100
[<ffffffff817361ae>] ? int_signal+0x12/0x17
[<ffffffff81145d85>] sys_unlinkat+0x25/0x40
[<ffffffff81735f22>] system_call_fastpath+0x16/0x1b

What is interesting here, is that we call log_wait_commit, from
within wait_for_space, but we are still holding the checkpoint_mutex
as it surrounds mostly the whole of wait_for_space.  And then, as we
are waiting, journal_commit_transaction can run, and if the
JBD2_FLUSHED bit is set, then we will also try to take the same
checkpoint_mutex.

It seems that we need to drop the checkpoint_mutex while sitting in
jbd2_log_wait_commit, if we want to guarantee that progress can be
made by jbd2_journal_commit_transaction().  There does not seem to be
anything preempt-rt specific about this, other then perhaps increasing
the odds of it happening.

This patch is back-ported from the following one:
Lustre-commit: f8e80f01d6883c3e55841528ac16811f6588e260
Lustre-change: http://review.whamcloud.com/11041

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I9adc35d316d7ea0379d0df2d38276297f682f8f7
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11492
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4579 ldlm: Properly display time on lock timeout

When lock timeout happens, need to printtime since last lock
activity, not since when the lock was granted.
As such set l_last_activity to current time when sending callbacks too.

Lustre-change: http://review.whamcloud.com/10601
Lustre-commit: f3864905b26a90ed3a9bb6f99357264fa40015c4

Change-Id: I31d0de6d9add67865cfcb16af903647f7cb932a0
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10601
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-on: http://review.whamcloud.com/10997

LU-4345 osp: store valid bits in setattr record

Backport a series of 4 patches to resolve the random ID on OST
objects problem.

1: store valid bits in setattr record (LU-4345)

We'd store LA_UID/LA_GID bit along with the UID/GID in the setattr
llog record, otherwise, osp could set a random uid/gid to the OST
object.

Lustre-commit: 80f90fcde73e2faff8b7b0ffc7c19bc52982e027
Lustre-change: http://review.whamcloud.com/10223

2: Correctly check for invalid setattr record (LU-5188)

Patch for LU-4345 (commit 80f90fcde73e2faff8b7b0ffc7c19bc52982e027 )
has a correct comment about lsr_valid member being either 0 or
having UID and GID fields set, but the check has a typo causing
it to check for lsr_valid to be both 0 and have the bits set which
is impossible.

The osp_sync_new_setattr_job() should return 0 for invalid record,
so that sync thread can continue processing on other records.

Lustre-commit: 79dd530f1352e6b57fcb870a1e0f2c2a05a0648d
Lustre-change: http://review.whamcloud.com/10706

3: don't skip attr_set for osp objects (LU-5296)

The lsr_valid handling in osp_sync_add_rec() was got a problem in
commit 80f90fcde73e because it stored all of the passed attr flags
in struct llog_setattr64_rec, even though there are only fields for
storing the UID and GID.  Since the time stamps do not need to be
propagated to the OSTs (the MDT values are good enough), this is
not a problem.  Also, osp_sync_add_rec stored LA_UID/LA_GID flags
on disk, but they are not part of lustre_idl.h or lustre_disk.h and
are not guaranteed to be constant over time.  Instead, use the flags
OBD_MD_FLUID/OBD_MD_FLGID that are in lustre_idl.h.

Lustre-commit: 2d5a5e81660adbcd05f14a10bc6c246d6f108d10
Lustre-change: http://review.whamcloud.com/10989

4: return 1 if osp_sync_xxx_job issue RPC (LU-5188)

Return 1 if osp_sync_new_xxx_job() issue RPC, so
sp_sync_process_record() can decrease opd_syn_rpc_in_flight
and opd_syn_rpc_in_progress correctly if RPC is not
being sent, otherwise the opd_sync_thread will not be
stopped, caused LBUG (see LU-5244)

Lustre-commit: 73f47cdda42305df0be57e0ea6252eb6fd36db55
Lustre-change: http://review.whamcloud.com/10828

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=34 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanity-quota

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ie231465b274627561e7eb58f30b868472751bd71
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11435
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5120 Don't really need the default-base rpm

The kernel-default-base is not actually needed to build Lustre
on SLES11 so get rid of it.
Clean up the fetching of needed packages.

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Change-Id: Ieee2b01f9a8c280de5d2cc4c43ff672539eea7ef
Reviewed-on: http://review.whamcloud.com/10485
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5252 hsm: make path argument to fifo event a const

Neither llapi_hsm_register_event_fifo() nor
llapi_hsm_unregister_event_fifo() take a const as their argument.
This leads to the following compilation warning when a caller is
using a const char *:

  ... warning: passing argument 1 of ‘llapi_hsm_register_event_fifo’
         discards ‘const’ qualifier from pointer target type
         [enabled by default]
   rc = llapi_hsm_register_event_fifo(lustre_fifo_event);
   ^
  ... note: expected ‘char *’ but argument is of type ‘const char *’
  extern int llapi_hsm_register_event_fifo(char *path);

Making path a const fixes this warning.

This is a backport to b2_5

Lustre-change://review.whamcloud.com/10807
Lustre-commit: 8264927b92b82e5ec6d223523db8680ee8df1da4

Change-Id: Ia2544cafefef8d5d1e4cf4cfc6408aac373c447b
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/10807
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 8264927b92b82e5ec6d223523db8680ee8df1da4)
Reviewed-on: http://review.whamcloud.com/11363

LU-5263 llite: update ras stride offset

When a read ahead does not reach the end of the region reserved from
ras, we'd set ras::ras_next_readahead back to where we left off; For
stride read ahead, it needs to make sure that the offset is no less
than ras_stride_offset, so that the stride read ahead can work
correctly.

This patch is back-ported from the following one:
Lustre-commit: 9e280281b8daf8ad69d04a5371121224ace28fbc
Lustre-change: http://review.whamcloud.com/11181

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I5d319a94435833153c52da6ccc1b20c4952789e9
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11455
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5415 ldlm: high load because of negative timeout

When the time of LRU resizing exceeds waiting period of
recalculation, the ldlm daemon will keep on resizing without any
interval of rest. That will cause high CPU load.

This patch fixes the problem by setting the recalculation timestamp
after LRU resizing finishes rather than before it. What is more, an
interval of one second is enforced between each recalculation.

Lustre-change: http://review.whamcloud.com/11227/
Lustre-commit: 381492e1fc3f93873bf495885859c4f63ce0f4fe

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I989255eca64d74dfcb2087060f627c2e72615248
Reviewed-on: http://review.whamcloud.com/11414
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5252 hsm: Do not unconditionally delete the HSM event fifo

The lustreapi HSM library will use a pipe if it already exists.
However the deregistration will unconditionally delete that pipe,
which is not correct.

An admin may want to create the pipe with certain rights/ownership,
which the llapi_hsm_register_event_fifo() doesn't allow. In that case,
llapi_hsm_unregister_event_fifo() should not delete it.

This is a back port to b2_5.

Lustre-change: http://review.whamcloud.com/10809
Lustre-commit: 7b7e6e2fb692eb3fa8d386addbc33763acf1a397

Signed-off-by: frank zago <fzago@cray.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I66e44f21217228c5f425e5a9312d8e381d564c7f
Reviewed-on: http://review.whamcloud.com/11421
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5128 obdclass: fix a race in recovery

in "class_export_recovery_cleanup", the check of the flag
"exp->exp_req_replay_needed" should be protected by "exp_lock".

Lustre-commit: 5a7e4480fc519c8df4ca0b11e6d5239956316bd3
Lustre-change: http://review.whamcloud.com/10849

Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Wu Libin <lwu@ddn.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib3c980101f400ce1b9cb0b3d2a4418fbec31b94b
Reviewed-on: http://review.whamcloud.com/11102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5126 libcfs: fix illegal page access of tracefiled()

After failure happens and put_pages_back() returns the pages,
tracefiled() should not go on itering on the page list.
Otherwise, some pages might be accessed illegally.

This patch is back-ported from the following one:
Lustre-commit: f111fac3f785400f892ca03bfb9078c6bd091f7d
Lustre-change: http://review.whamcloud.com/10524

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I9f2953e9e0a6473112c7d1ce16375b84c66d622c
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11454
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4824 utils: lfs find should continue after errors

'lfs find' stops as soon as it encounters an error, such as EACCES.
Change the behavior to match the regular find command: just note the
failure but continue on, returning a non-zero exit code at the end.

Lustre-change: http://review.whamcloud.com/9794
Lustre-commit: 785e7dc56613e79bdfd89fab762eeb82deb7564c

Signed-off-by: Matt Ezell <ezellma@ornl.gov>
Change-Id: I16ca35ec193e36134e7b93d91a20ff1d0778fe60
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/11093
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-3326 tests: sync after file deleted in run_*.sh

In run_*.sh, call sync to flush any dirty pages to OST
if MDT has not sent the object destroy request to it,
which will shrink the available disk space gotten before
running the tests and cause -EIO error.

This patch is back-ported from the following one:
Lustre-commit: 58c2a357202ba83d26ba9d8f1d9c093e3272e9a0
Lustre-change: http://review.whamcloud.com/8071

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,REQFAIL=100,DURATION=28800 \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 \
ossarch=x86_64 mdsarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
mdssizegb=10 ostsizegb=10 austeroptions=-R failover=true iscsi=1 \
testlist=recovery-mds-scale

Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I922774dcbf5546a9f5e3fe345bac954fd73d50a7
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11425
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5403 kernel: kernel update RHEL6.5 [2.6.32-431.23.3.el6]

Update RHEL6.5 kernel to [2.6.32-431.23.3.el6].

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I798991bbce4b30860b7b35096b430144eb2a1a9c
Reviewed-on: http://review.whamcloud.com/11246
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5334 lnet: fix build with OFED-3.5-2-MIC-beta1

The stuff for RHEL6.4 should be guarded by CONFIG_COMPAT_RHEL_6_4
also in RHEL6.5. So, we need both _COMPAT_ defines in RHEL6.5.

Lustre-Review: http://review.whamcloud.com/11104
Lustre-Commit: fa4a67d5b4871e20acd07076e7fa15613695cae5

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I322dd2ef167c7b3c272b766ee2cd3f0d464e0a13
Reviewed-on: http://review.whamcloud.com/11153
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5332 hsm: Release XATTR lock on exiting

This should be a typo where it should jump to out_unlock instead of
out_ucred.

This is a backport to b2_5.

Lustre-change: http://review.whamcloud.com/11083
Lustre-commit: 393350b37cae30efc03ffc076c77e390999c2a2d

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I03242e40f7e33a115934081375be7c86044fa254
Reviewed-on: http://review.whamcloud.com/11215
Tested-by: Jenkins
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5224 lnet: build failed with MLNX_OFED-2.2

Build lustre client can fail on the SLES11 SP3 with
MLNX_OFED-2.2 against the kernel-3.0.76-0.11, which will
cause the "'pm_qos_req' has incomplete type" error.This patch
put "#include <linux/compat-2.6.h>" in front of
"#include <net/sock.h>" to solve this problem.

This error come with the configure option
"--with-o2ib=/usr/src/ofa_kernel/default/", it affect the
search path of the compiler, which will search the header
files in this directory before the kernel header files
directory.

This patch change this way because "net/sock.h"(exactly is
"linux/netdevice.h") include header file "linux/pm_qos_params.h",
the pm_qos_params.h is exist both in ofa_kernel include
directory and kernel include directory, so we need to put the
linux/compat-2.6.h in front of net/sock.h to solve the
compatible problem.

Signed-off-by: Wu Libin <lwu@ddn.com>
Change-Id: I98df2a5faaecbd44528d3bd8a7ba796cc1d14cdc
Reviewed-on: http://review.whamcloud.com/10819
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5203 zfs: Update spl/zfs version to 0.6.3

Update SPL and ZFS version to official 0.6.3.
This also fixes the "search path" for libzfs.so.

Lustre-commit: 1f5fa56a0b824e684c32d382d0108c92d9125c5a
Lustre-change: http://review.whamcloud.com/8979

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I710d254d93ce9c3db64ab4bf250f8bf556c684aa
Reviewed-on: http://review.whamcloud.com/11298
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4606 utils: Mount uses so for backing fstype

Make mount/mkfs/tunefs functionality a per backing FSTYPE shared
object that is loadable, and seperately buildable and installable
in the appropriate lustre-osd-FSTYPE rpm.

For ZFS this allows the shared object to be directly linked to libzfs
and thus remove the hardcoding of a libzfs.so version in the lustre
source. This also has the benefit of making lustre-osd-zfs explicitly
requires libzfs (which it needs to do mount/mkfs operations).

Adjust build system to allow building of loadable module.  Including
requiering libtool as part of the build.

Lustre-change: http://review.whamcloud.com/10193
Lustre-commit: 3e8c354c09725a97b90e477a77411cc6fb306f09

LU-5091 build: Support alternate locations of libzfs.so

Due to the change for with-zfs-devel (LU-3497), also add support to
search for libzfs.so along with the alternate locations supplied with
the header files.
Also need to unpack zfs{,-devel}.  The zfs RPM is needed because it
has the actual libraries in it and the zfs-devel RPM only has un-
versioned symlinks to the libraries in the zfs RPM.
This will all change one day when we have a libzfs RPM per
https://github.com/zfsonlinux/zfs/issues/2329 and it looks like it
could be one day soon: https://github.com/zfsonlinux/zfs/pull/2341.

Lustre-change: http://review.whamcloud.com/10401
Lustre-commit: b2659030edb7d34c91430ff924bab23881b6c230

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I1d69b64eec1589f63e24f20cc624778a4be2ec60
Reviewed-on: http://review.whamcloud.com/11297
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4920 quota: always clear lqe_lockh on lock cancel

In the qsd_id_blocking_ast(), lqe_lockh should always be cleared,
otherwise, quota code will mistakenly assume the id lock is still
valid.

Lustre-change: http://review.whamcloud.com/9965/
Lustre-commit: 73ef5dcd8bcf32e4127e0011357b8bc39472cdba

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I653e263176a5415ec32324479fed86d948958ef0
Reviewed-on: http://review.whamcloud.com/11278
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4786 osc: to not pick busy pages for ELC

This is a combination of commit 154fb1f7 from LU-3321 and commit
bfae5a4e from LU-4300. This patch is only required in b2_5 as the
code is already in 2.6.

Use weigh_ast to decide if a lock covers any pages.
In recovery, weigh_ast will be used to decide if a DLM read lock
covers any locked pages, or it will be canceled instead being
recovered.

The problem with the original implementation is that it attached
each osc_page to an osc_lock also changed lock state to add every
pages for readahead.

Bugzilla-bug-Id: b=16774

Change the policy of ELC to pick locks that have no dirty pages,
no page in writeback state, and no locked pages.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie857a4dd4e235f0c3273f53cbfab9592c2793ac9
Reviewed-on: http://review.whamcloud.com/10795
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4308 llite: protect file write and page_mkwrite race

Normal file write also need to grab a read semaphore of lli_trunc_sem
so that it would not race against ll_page_mkwrite.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I6cc2980b9e67e48bfb132296cf6c36ae69d4c153
Reviewed-on: http://review.whamcloud.com/11098
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4249 quota: race in qsd_upd_thread()

qsd_upd_thread() uses list_for_each_entry_safe() to process
list items one by one, however, it has to drop lock while
processing each item, that'll race with other list processing
thread.

The proper way is to check list head each time when it acquired
lock.

Lustre-commit: f8f7c34a7bbcf22aeca7699ce76254e41e3e95b7
Lustre-change: http://review.whamcloud.com/10988

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I83e665db4209d52c1358505125abdcba75a0a6fa
Reviewed-on: http://review.whamcloud.com/11020
Tested-by: Jenkins
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5456 hsm: hold inode mutex around ll_setattr_raw()

ll_setattr_raw() expects to be called with the inode mutex held so do
when calling it from ll_hsm_import().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I91ef3d431fdf6db23bf4be6e957f2981d3e016cd
Reviewed-on: http://review.whamcloud.com/11349
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5285 mdt: reconstruct to handle non-existing objects

it's possible that reconstruction finds the object doesn't exist
because the original transaction was committed and the rep-ack
lock got released letting another clients to destroy the object.
we should handle this propely - return attributes to indicate the
object has been unlinked (nlink=0

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I3bc18ff0cdb3b7f6d28e47604cf3e142e3f5bc42
Reviewed-on: http://review.whamcloud.com/11025
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4462 mdt: don't apply mdt_object_fid() to ERR_PTRs

In mdt_reconstruct_open(), mdt_reconstruct_create(), and
mdt_reconstruct_setattr() print the FID from the request rather than
applying mdt_object_fid() to a known ERR_PTR.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ifb9365dbef1492d8fb5c7ff27d00f5155b0e7525
Reviewed-on: http://review.whamcloud.com/8788
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5270 obdclass: do not lock repetitively when geting jobid

When trying to get customized job ID, cfs_get_environ() tries to
lock/unlock mmap_sem for two times, which is unnecessary.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Iae8a45d87508e3c465c0424d5fa54e294d4fb2b1
Reviewed-on: http://review.whamcloud.com/10890
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5221 vvp: release mmap_sem in error case

The mmap_sem is downed in vvp_mmap_locks, but in case of
error from cl_io_lock_alloc_add, it is not upped.

Credit to Paul Casella at Cray for finding this.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: If172e7cb4c4e6400143d76eee8aa595ae767a20c
Reviewed-on: http://review.whamcloud.com/10741
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5220 llite: fix a flag bug of vvp_io_kernel_fault()

After vvp_io_kernel_fault() locked the page, it should set
VM_FAULT_LOCKED.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Ic63f1b830360b127df6ae3220d63bef3ebf0b89c
Reviewed-on: http://review.whamcloud.com/10740
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5179 libcfs: do not leak mm_struct

cfs_get_environ might leak an mm_struct if LU-1735 race condition
hits.
Make sure to mmput in such a case.

Change-Id: I49499c14027358961743d8928ab2edbe54bf72f7
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10759
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-5094 Fetch SLES11SP3 build artifacts automatically

If the user has an account at nu.novell.com for getting SLES packages,
they must configure that account info in the standard .netrc file for
wget/curl to use when trying to get packages from nu.novell.com.

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Change-Id: Ic3cbfc78e0a813d7ac5051bc4a4a358caac0d5a7
Reviewed-on: http://review.whamcloud.com/10396
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5053 ptlrpc: Add schedule point to ptlrpc_check_set()

Most ptlrpc sets are believed to be small and bounded in length.  However
at the very least the ptlrpcd reuses the ptlrpc sets at its primary work
queue.  This work queue can easily have work added faster than the ptlrpcd
thread can process the work.  The unbounded work can lead to the ptlrpcd
monopolizing a CPU for hundreds of seconds.  Obviously a well-behaved
kernel function should obey the scheduler and share the processor.

We address that problem by inserting a cond_resched() at the top of the
main loop of ptlrpc_check_set().

Some have suggested putting the cond_resched() lower in the loop.  However,
the only current way to bound the number of loops that we exceed our
allocated run time is to put the call at the top of the loop.  Putting it
lower would allow an unknown number (and since it is unknown, it might be
excessively large at times) of cycles through the loop before a
resched is allowed.

Change-Id: If46a6e1af1d0fdae24af623045aaf902a04fc589
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/10358
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>

LU-5039 mdd: do orphan cleanup asynchronously

Orphan cleanup shouldn't block mount process, it can be done
asynchronously.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib282dbece0cf789b33105aaeaeef04bd6f1dcf70
Reviewed-on: http://review.whamcloud.com/10584
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4937 osc: get rid of old checksum initial value

Old code residue assumes initial checksum value as ~0, and relies on
that to check whether OST server has calculated bulk data checksum.
That is not the case anymore.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I99779eaf02d22b470978448ad3f17b35f0e9a475
Reviewed-on: http://review.whamcloud.com/10354
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4861 osc: a deadlock problem in osc completion ast

In osc_ldlm_completion_ast(), it tries to hold mutex of cl_lock
and update the lock even it's known that the lock is not granted,
this can introduce a deadlock case if the process is matching a
dlm lock with the calling stack:
  osc_lock_enqueue()
    -> osc_enqueue_base()
      -> ldlm_lock_match()
        -> osc_ldlm_completion_ast()
          -> cl_lock_mutex_get()
The enqueuing lock and the lock for completion_ast() are different
lock. This is a deadlock case because we're holding a lock's mutex
and then request the other.

It's not necessary to acquire the lock in completion_ast() if it's
known that the lock has not been granted.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ia8c0a4cb1e48c9f7fd921052174ce4d8d07713b1
Reviewed-on: http://review.whamcloud.com/10581
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4800 obdclass: runtime load lustre client when needed

Manually force loading of lustre.ko during client
mounts with request_module for cases where
autoloading doesn't happen.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I807fc5cd04cf5a97d9c131208b7b13037dc10ee6
Reviewed-on: http://review.whamcloud.com/10587
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4755 ptlrpc: enlarge OST_MAXREQSIZE for 4MB RPC

This patch enlarges OST_MAXREQSIZE so as to make the
request size large enough for 4MB RPC.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I6195c05ced9e6c533c2ad88f21a941f1dddb53e6
Reviewed-on: http://review.whamcloud.com/9599
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4699 libcfs: unify ERR_PTR definitions

Add libcfs/err.h which either includes linux/err.h or defines
equiavlent functions (including ERR_CAST()) for use by userspace or
any of Lustre's other imaginary platforms. Modify the definition of
__container_of() to remove the pre-call explicit cast. Remove two
unused function that used container_of0() on an already dereferenced
pointer.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I26b454f835b51f1fb17b94eec7cf184f0a82071e
Reviewed-on: http://review.whamcloud.com/9519
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4694 hsm: Cleanup codes about return value

errno migh be zero when llapi_hsm_action_get_fd() returns a failure.
ct_archive() does not handle it correctly, which might cause HSM take
the operation as successful even if it actually fails.

The patch also cleanup other codes about return value.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I625da68836f98346b104533a556c72f06232e3ef
Reviewed-on: http://review.whamcloud.com/9455
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Robert Read <robert.read@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4557 ext4: recalc percpu counters after journal

the percpu counters is initialized before replaying the journal,
but after the journal it should be recalculate to deal with the
possibility of the percpu counters getting updated by the journal
replay.

the issue has been fixed in the new kernels (RHEL6 2.6.32-431.5.1,
SLES11SP3 3.0.101-0.15), then it is not needed for these kernels.

Change-Id: I5dfa0696b5ccd404b1add9ad6e2b0877d86a35b2
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/9908
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-4557 ldiskfs: init statfs variables after journal

in ext4_fill_super, the variables related to statfs
should be initialized after journal recovery is completed.
otherwise, if a large number of blocks were being allocated
before the filesystem crashed, then the blocks and inode
counters may become negative during use and report incorrect
values to statfs call.

Change-Id: Id7e3f61ce73f5499a6176c336c7931a47f6f76de
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/9277
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>

LU-4529 quota: call qsd_op_end() after trasn stop

qsd_op_end() shouldn't be called before the transaction stopped,
because qsd_op_end() is a quite heavy operation which could
probably allocate memory with standard allocator flag (__GFP_IO),
and allocating memory could result in dirty flush on other
filesystems, that will lead to opening transaction on different
journal and trigger the assert in jbd2_journal_start():
J_ASSERT(handle->h_transaction->t_journal == journal) at the end.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I4ea3ff011fa7e44460b9912050e90b174813e01a
Reviewed-on: http://review.whamcloud.com/8968
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-4478 ldiskfs: fix problem when ldiskfs_acct_on() fails

ldiskfs_fill_super() does not handle the failure ldiskfs_acct_on()
returns correctly. This patch fixes the problem for RHEL6.5 kernel.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Id6e9af30ac81b54c2873b8c2f2a8efd398398a8d
Reviewed-on: http://review.whamcloud.com/10938
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-4478 ldiskfs: fix problem when ldiskfs_acct_on() fails

ldiskfs_fill_super() does not handle the failure that
ldiskfs_acct_on() returns correctly. This patch fixes
the problem.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I70afca523cee1d9fe7e0b9c5ffc122fbfd6c34aa
Reviewed-on: http://review.whamcloud.com/8829
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4438 utils: handle ct_setup() errors

Problem has been encountered causing sanity-hsm/test_402 errors
because copytool threads were found still alive after MDCs
deactivation. This appears to be due to no ct_setup() error
handling with continuing+unnecessary startup actions to occur.
Also, now that copytool will exit earlier, HSMTOOL_NOERROR variable
has been added to allow errors in copytool_setup().

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I108ea679faf2731f2a64fb08d2f8fba9db01c71f
Reviewed-on: http://review.whamcloud.com/9853
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4167 tests: update version check to conf-sanity test_32

This patch updates version check codes to skip running LFSCK on an
OST for versions >= 2.5.0 in conf-sanity.sh test_32.

Test-Parameters: testlist=conf-sanity envdefinitions=SLOW=yes, ONLY=32 ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=71 mdsbuildno=71 clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 ossarch=x86_64 mdsarch=x86_64
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ibc0c34a0c15f4882a0dd06dae75d1f1fe10b6f16
Reviewed-on: http://review.whamcloud.com/8132
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4106 mdt: avoid recursive lu_object_find an object

LU-3240 (commit 762f2114d282a98ebfa4dbbeea9298a8088ad24e) set parent
dir fid the same as child fid in getattr by fid case we should not
lu_object_find() the object again, could lead to hung if there is a
concurrent unlink destroyed the object.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I75256c0fa684877cbd2e1f36f8ab2ac3faab2989
Reviewed-on: http://review.whamcloud.com/7990
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4006 lnet: abort messages whose MD has been unlinked

If LNetMDUnlink has been called, all outgoing messages
on that MD should be aborted before lnet_ni_send() is
called.

Signed-off-by: Isaac Huang <he.huang@intel.com>
Change-Id: I3ebe6d3f4ecc21cd6f00aebfb98b90109501df5a
Reviewed-on: http://review.whamcloud.com/8041
Tested-by: Jenkins
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2880 ldiskfs: Added mount option to enable dirdata.

Added the code to set mount option for enabling the dirdata in
osd_mount(). This will set the dirdata option for lustre
filesystems only. The dirdata mount option would be checked in
get_dtype() to decide whether to pass dirdata flag as well while
reading the file type.

Signed-off-by: Manisha Salve <msalve@ddn.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I07f430c5cd7ad92b81746085b05b53c1202dd725
Reviewed-on: http://review.whamcloud.com/6495
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-2234 man: Add zfs options to mkfs.lustre man page

Document zfs options in mkfs.lustre(8).  Also update index and fsname
options to match current requirements.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I06e018c41b29d8f9ca8d1f2ef595140bed16096e
Reviewed-on: http://review.whamcloud.com/10629
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>

New tag 2.5.2-RC2

Change-Id: I25517c86d132cc7201ed144652fbbb682e2d5b55

Revert "LU-4398 mdt: acquire an open lock for write or execute"

This seems to be causing LU-5197

This reverts commit 97bfe7a3c0fc74fb0e56cbc1ea9cb827fb657b48.

New tag 2.5.2-rc1

Change-Id: I1d37d5756b7f47e83d4467025ca67acc044ad09d

Revert "LU-2272 statahead: ll_intent_drop_lock() called in spinlock"

Actually I jumped the gun on this one. We'll wait until master version
is fully ready

This reverts commit eb39d82ae13ddb6010065cede13dca11f1da6aaf.

LU-5116 ptlrpc: race at req processing

Race between ptlrpc_resend_req() and ptlrpc_check_set().
1 thread do ptlrpc_check_set()->after_reply()
2 thread do ptlrpc_resend_req()
The result is request with rq_resend = 1 and MSG_REPLY flag.
When this request will came to server it will cause client eviction.
The patch skip ptlrpc_resend_req logic if rq_replied is set,
and clear rq_resend flag at reply_in_callback() when client got
reply.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Xyratex-bug-id: MRP-1888
Change-Id: If47b9a1f559ab16e4b416332ada7c73a8758d46c
Reviewed-on: http://review.whamcloud.com/10471
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10562
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-2272 statahead: ll_intent_drop_lock() called in spinlock

ll_intent_drop_lock() may sleep, which should not be called inside
spinlock.

This is back-ported from the following patch:
Lustre-change: http://review.whamcloud.com/9665

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtcount=1 testlist=lustre-rsync-test,lustre-rsync-test

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Id658c0bd3b9dc0a918995c66842b0c3fabe8a7ae
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/10674
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4569 hsm: Prevent copytool from importing existing file.

Requesting import of an already existing FID can lead to data
corruption since both inodes in lustre would refer to the same
file in the backend.

Lustre-change: http://review.whamcloud.com/10185
Lustre-commit: 8dfee22dd5a896144fb913f5f9f0e8f5c0e4ed86

Change-Id: I92d951e3e0796c9bde8bd744f39f242e2aa233eb
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10479
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4887 tests: sanity-scrub interoperability tests with master

In the master branch, the default behaviour for lfsck_start without
"-t" specified will trigger all supported LFSCK components. And the
master branch uses new "-t scrub" for control OI scrub. The changes
will cause the old (2.5.58 or older) sanity-scrub test scripts fail
at test_9/test_11.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I134f8ec7d3f9e5c60abb76d728727871a960a095
Reviewed-on: http://review.whamcloud.com/10641
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-5047 tests: correct cleanup files in sanity.sh

Don't remove all files from Lustre share. Create files that will be
cleaned up at the end tests session in check_and_cleanup_lustre().

Fix cleanup for test_54c.

Luster-Commit: c42bb35bc32f472d69276475de3cd186ca5f5247
Lustre-Review: http://review.whamcloud.com/10291

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I23624b9d14e7de3423b82b4a5b8508c4820c55d4
Reviewed-on: http://review.whamcloud.com/10291
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10633

LU-5132 tests: Add version check to sanity/160c

Account for what versions of Lustre this test should work for.
This test was added in http://review.whamcloud.com/9723 and that
was included in 2.5.58 and prior to 2.5.2 in the b2_5 branch.

Lustre-change: http://review.whamcloud.com/10557

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=160c \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I560a9900de983b06963d49946e9188649dd5e7fc
Reviewed-on: http://review.whamcloud.com/10558
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3386 lproc: improve osc/mdc "imports" connect data

Improve /proc/fs/lustre/{osc,mdc}/*/import files to print
the struct obd_connect_data data fields as "connect_data:"
in addition to the "connect_flags:" field.
Lustre-commit: 62660c799be886e557ae0426126fc3e4b61927d4
Lustre-change: http://review.whamcloud.com/6588
Also, this patch includes the fix in
http://review.whamcloud.com/#/c/10344

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iabf2231341f3d674960626fb7230e2ba5cf4f113
Reviewed-on: http://review.whamcloud.com/9264
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5133 tests: Add version check in sanity/238

Fix for LU-4659 was not included until after 2.5.57 and after 2.5.1.
Ensure MDS is running one of these valid branches before allowing the
test to run.

Lustre-change: http://review.whamcloud.com/10559

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=238 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I25787874866b38f290b21baf76a09a2aec3dc7f4
Reviewed-on: http://review.whamcloud.com/10560
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5100 llite: set dir LOV xattr length variable

the LOV xattr of directory could be either lov_user_md_v1
(size is 32) or lov_user_md_v3 (size is 48), then the actual
size of the LOV xattr should be return.

this patch also changes lod_verify_stripe to make it to accept
lov stripe from user and it's "pattern" is "LOV_PATTERN_RAID0".

Change-Id: I1ccba6da19aef0b7464c43b1b50871df44ce0eb7
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/10453
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10575

LU-4780 lnet: NI shutdown may loop forever

lnet_shutdown_lndnis() may enter endless loop if there is a busy NI,
this is injected by LNet SMP improvements. It's fixed in this patch.

Lustre-change: http://review.whamcloud.com/9706
Lustre-commit: 3efa31eacc612261ea3019fbd7243e573714042c

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: If6bf6a8362d8808cd924d6fe79dbe43151f41f30
Reviewed-on: http://review.whamcloud.com/10489
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4102 doc: recommend newer e2fsprogs version

Recommend newer e2fspogs version for 2.5.2 release so that it gets
the latest fixes.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ica1e47b4242228334f9d673ee5159471f4e1ca8c
Reviewed-on: http://review.whamcloud.com/10561
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4573 tests: check all MDTs for open files

sanityn.sh test_76 verifying mdt.*.exports.*.open_files content
did not check all of the MDTs. It was broken by commit 370de927fc5,
which created all directories striped across all MDTs by default.

There is actually no need to special-case striped or remote
directories in this test, and it should be possible to just check
for open files on all MDTs, and this also simplifies the test.

Lustre-change: http://review.whamcloud.com/10107
Lustre-commit: fed7a99441c2dd078a671cf5b3dcaf27996bf448

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I2b7b434052946c8d84bffef1fc7358cc29b00b75
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10499
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2524 test: Modify tdir to be single directory

Currently, the tdir variable is a directory with a subdirectory.
This requires the '-p' option when calling mkdir. We've made tdir
be a single directory so calls to mkdir and test_mkdir do not
require the '-p' option in most cases.

tdir was changed from d0.${TESTSUITE}/d${base} to
d${testnum}.${TESTSUITE} and tfile was changed from
f.${TESTSUITE}.${testnum} to f${testnum}.${TESTSUITE}. Now tdir and
tfile are consistent in their format and the call to remove files
and directories at the beginning of many test scripts will remove
these files and directories.

Once this patch lands, we can remove the "-p" option from many
of the calls to mkdir.

Lustre-commit: a98d44e0439ad5b20d0d7f273b29e6ea990312f0
Lustre-change: http://review.whamcloud.com/8123

LU-2524 tests: run sanity test_51ba in test_51b dir

Run the test_51ba directory cleanup in the same directory as the
test_51b subtest created its subdirectories.

Lustre-commit: 47c581b8155f865ade5c9d067f85930146361c2a
Lustre-change: http://review.whamcloud.com/9021

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ib49d7102a49ff6b5f3ec539a5b2f2f5186231a04
Reviewed-on: http://review.whamcloud.com/8123
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10349
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4830 tests: only deactivate MDTs of Lustre FSNAME

Problem has been discovered using a very specific copytool Lustre
root/archive usage. All MDCs, including archive one, were
deactivated.

Lustre-commit: 09cc4025440f7e4f9f887f4a2a9bdceef0da7689
Lustre-change: http://review.whamcloud.com/9843

Change-Id: Ifab4bd5a7d6a88ae0e0b95d5a6774dbfed8822e3
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Reviewed-on: http://review.whamcloud.com/10232
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4676 hsm: Fix return value error of ct_run()

ct_run() returns 0 after find out that fs_name is invalid.
lhsmtool_posix will exists with 0 after that error happens.
That might mislead scripts calling lhsmtool_posix.

Lustre-change: http://review.whamcloud.com/9412
Lustre-commit: d5ad6ff819e635c6ba337586cf70698b7728ff7f

Change-Id: Id182e989011e02104b5960c6692995825d939ad6
Signed-off-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10234

LU-4852 osc: osc_extent_truncate()) ASSERTION( !ext->oe_urgent ) failed

The bug was caused by race between truncate & fsync.
osc_extent_wait() doesn't takes into account oe_trunc_pending
during setting oe_urgent. The race arises after
osc_object_unlock().
osc_extent_wait() should ignore extents with oe_trunc_pending
while waiting for OES_INV. osc_cache_truncate_end() will set
oe_urgent and call osc_io_unplug_async()

Change-Id: I6f4c8420faa67bf4b14b1c5c2bc37fa282a7b072
Xyratex-bug-id: LELUS-239
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-on: http://review.whamcloud.com/10204
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4704 test: enable acl 974 and 2561 series test

Add acl/{974,974_remote,2561,2561_zfs}.test to the
lustre/tests/Makefile.am nobase_noinst_DATA list, and fix acl/run
test to fail if the test scripts are missing.
acl/2561_zfs.test is a newly created test file for zfs. Since zfs
allocates a 512 byte dnode for empty file, its "ls -s" result is
different from ldiskfs'.

Lustre-change: http://review.whamcloud.com/9541
Lustre-commit: 1116d8a6233e307abab579e2ee0a63b450087413

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ifba5b76fa7302f8b129b979aeccf3844f4fe334d
Reviewed-on: http://review.whamcloud.com/10512
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4811 mdt: remove eadatasize check in xattr packing

Remove client eadatasize check in mdt xattr packing because
client can handle -EOVERFLOW.

Originally part of Lustre-change: http://review.whamcloud.com/6339

Test-Parameters: clientjob=lustre-master clientbuildno=1963
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I47536fa0926804630315e5b78da852f6239497fd
Reviewed-on: http://review.whamcloud.com/9812
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>