LU-766 Fix sops->statfs() autoconf check

Building with RHEL6.2 beta (kernel-2.6.32-204) exposed that the
HAVE_STATFS_DENTRY_PARAM autoconf check was incorrect. The test
was setup to check the first argument of vfs_statfs() function.
However, the conditional code itself doesn't use the vfs_statfs()
function it registers a function with the sops->statfs() callback.

Historically, this always worked because both of these prototypes
were changed in a single upstream kernel commit (726c3342). However,
as of RHEL6.2 the vfs_statfs() function changed again and the
sops->statfs() callback stayed the same. The vfs_statfs() function
now takes a path structure as the first parameter.

This was fixed by simply updating the autoconf test to check the
prototype of the function were going to actually use.

Change-Id: I0cf36e758fd9c85609c47c13dbdf249321664b26
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1527
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-50 ldiskfs: pdirops patch for ldiskfs

Single directory performance is a critical for HPC workloads. In a
typical use case an application creates a separate output file for
each node and task in a job. As nodes and tasks increase, hundreds
of thousands of files may be created in a single directory within
a short window of time.
Today, both filename lookup and file system modifying operations
(such as create and unlink) are protected with a single lock for
an entire ldiskfs directory. PDO project will remove this
bottleneck by introducing a parallel locking mechanism for entire
ldiskfs directories. This work will enable multiple application
threads to simultaneously lookup, create and unlink in parallel.

This patch contains:
  - pdirops support for ldiskfs
  - N-level htree directory
  - integrate with osd-ldiskfs

Change-Id: I269c0e3112e68f3acd79e860dab052a68c7d7aaa
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/375
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-843 iokit: get_ec_devno returns trash

Some commands leak output to stdout and get_ec_devno returns
its results via stdout.  Redirect fd 8 to stdout and redirect
stdout to stderr.  Echo final results to fd 8.  Since this
only works when run as a subshell, invoke "exit" instead of
"return" when done.

Inside create_objects() function, stdin is redirected for a
shell "while" command.  Though this is done in the current
environment (process) for current BASH, it is not required
and there is no committment that BASH will always behave
this way.  Instead, set fd 3 to be reading from that file
and read from fd 3 insted of stdin.

While there, also test for "is object id" lines and extract
values using shell constructs instead of fork-ed commands.

Also while there, ensure that "ERROR" is always printed on
every error exit.

Also while there, ensure that the libecho file comes from
the same directory as the obdfilter-survey script.

Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Bruce Korb <bruce_korb@xyratex.com>
Change-Id: I50137c9ded816aac792aba4a0a79f59c28d16e01
Reviewed-on: http://review.whamcloud.com/1699
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-537 o2iblnd: support aliased interface

Current kiblnd_create_dev() doesn't allow to dev_get_by_name() to
return NULL (we added this for supporting failover of ib-bonding),
but this can happen for aliased interface, we can fix this by simply
remove the limitation and set kib_dev_t::ibd_can_failover to 0 for
aliased interface.

Change-Id: Idc3e6e4fd2d5056594f38dd43a0a560c3455ea05
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1148
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-431 o2iblnd: reduce failed alloc retry interval

o2iblnd can dynamically grow TX descriptor pool at run time,
it will take 10 seconds as retry interval if there is a failed
allocating, this interval is too long and some RPC might timeout.
So we decreased it to one second.

Change-Id: I0693e9d2cbf321e473713db6dd668afaf64e1bae
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/966
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-803 echo: fill OBD_MAGIC for echo device

Otherwise it will cause failure on obdfilter-survey.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I505da432cdf04fe3275d2a529f4115b1a3238e94
Reviewed-on: http://review.whamcloud.com/1804
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-794 llite: allow larger O_DIRECT read/write RPC

The O_DIRECT read/write RPC size limit was formerly constrained by
the maximum kmalloc() size, and would fall back to smaller kmalloc()
allocations until they succeeded.  This constrained the number of
concurrent O_DIRECT RPCs, which can significantly affect performance.

Newer code uses OBD_ALLOC_LARGE()  vmalloc() for allocations larger
than a page, so the maximum RPC limit can be increased.  Currently
the limit is arbitrarily set to KMALLOC_MAX_SIZE (which is just a
proxy for what is a reasonably large allocation is for the system).

This increases the maximum O_DIRECT limit to about 682MB, which will
allow sending 8 concurrent RPCs on an 85-stripe file with a single
write() or combination thereof.  This could be increased if needed.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic12e6536cebed94c3a3b56acbea0f0940cdafec9
Reviewed-on: http://review.whamcloud.com/1617
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-579 MRP-120 changelog CLOSE event.

add CLOSE event into changelog, but not into the default mask.

do not create close changelog record sometimes, if a dup mfd handler appeared:
- open replay (1st mfd appears)
- re-connect
- open replay (2nd mfd appears)

Reviewed-by: Nathan Rutman <Nathan_Rutman@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I8e5ff99905b84ee58416406ca7b690c86db60c62
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1197
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-543 mdd: fix rename changelog

Current rename changelog stores source fid in both CL_RENAME & CL_EXT
records, which is redundant, and the 'tfid' in CL_EXT is never been
used.

Actually, we'd store target fid in the CL_EXT record, then application
could detect the fid unlinked by rename in changelog.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I0c616f813657a2faefa60a707f4fc1d9dc971b39
Reviewed-on: http://review.whamcloud.com/1652
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-848 clio: page writeback support

Page writeback support for clio - release vmpage lock in transfer.
Sync IO still holds page lock.

Change-Id: I5137542b5b6b6eaaa4464804b45d372fd12e5c22
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1456
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-680 lfs: instance <-> mount point mapping from lfs

A new option to 'lfs' has been created to return the mapping
between Lustre filesystem instance and paths. The option
is 'getname' and it may be called with or without arguements.

'lfs getname' without arguments returns the instances of all
Lustre mount points.

'lfs getname [path...]' returns the instance of each specified
path. If the path is not a Lustre instance 'No such device' is
returned.

OBD_IOC_GETNAME has been added to file.c to provide consistent
behavior for file as well as directory paths.

A llapi_getname helper function has been added to liblustreapi
that returns a lustre instance name if a path is provided.

Documentation for 'lfs getname' is included inline an the lfs
man page has been updated.

Signed-off-by: Richard Henwood <rhenwood@whamcloud.com>
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Change-Id: Iab8ff12d604c7ce853f3c204b455e3b641f659f4
Reviewed-on: http://review.whamcloud.com/1373
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-575 MRP-133 lfs find speedup

lfs find should send getattr on mds only if needed;
lfs find should not break on matched obd but check other parameters as well;
lfs find time compare fixes;

Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Colin Faber <colin.faber@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
LU-575 MRP-260 fix quota tests

a fix for quota size units which conflicted with lfind size units

Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I42b2acee18c6c9e0d787b97a5e957431d2bb4bf1
Reviewed-on: http://review.whamcloud.com/1195
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-143 fid hash improvements

Current hash function of fid is not good enough for lu_site and
ldlm_namespace.
We have to use two totally different hash functions to hash fid into
hash table with millions of entries.

Change-Id: I6261e63a406118a93d578210c31e67fc7f9e389c
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/374
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-106 procfs: many proc entries are not accessed safely

Some in memory data may be released/uninitialized at the time
of proc entry creation/removal, this patch includes the following
fixes:
* initialize data before proc entry creation
* free data after proc entry removal
* free proc entries in obd_precleanup() because
  obd_uuid/nid/nid_stats_hash are released in class_cleanup().
* free proc entries after obd_zombie_barrier() because obd_export
  hold one refcound of nid_stat.
* check osd->od_mount before accessing osd proc entries because the
  osd proc entries are created before mount.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I03cb977e1be0747032a70f6a39fec804f81d70cc
Reviewed-on: http://review.whamcloud.com/326
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-871 build: change %L printk format to %ll

The clang-analyzer static code analysis flagged %L format specifiers
for printf/printk as deprecated, so change all of their uses to %ll.

In a couple of files (checkstat.c and truncate.c) there are existing
code style issues that need to be fixed before landing, in order to
get a clean bill of health from checkpatch.pl.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1bc75a0f43b25de112f5ae8f2273b3dc46081d09
Reviewed-on: http://review.whamcloud.com/1734
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-761: Check return of ll_prep_md_op_data() using IS_ERR().

Change-Id: I329fa923b95ae612e50f63641d2c34b121b0f6b9
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/1520
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-864 test: Hostname name doesn't equal NID and use facet_mntpt

On our test systems the hostname of the servers is on the
ethernet fabric and for lustre we use infiniband separately.
Currently several test will not run because of the assumption
that lustre will use the same fabric and we use to login and
manage the server. This patch allows one to grab the real NIDs
instead of assuming its "hostname"@NETTYPE. The second part of
the patch removes using $MOUNT% directly and uses facet_mntpt.
The reason being is on our system the mount points are different
on our servers from our clients. MOUNT is assumed to be the same
for clients and the servers. This patch allows test to work if
that is not the case.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I49f365561e341d44ace4c961fcae8b871dce7407
Reviewed-on: http://review.whamcloud.com/248
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-50 kernel: configurable BH LRU size

This patch allow per-CPU LRU size to be selected at compile time.
More complex filesystems may be modiyfing multiple blocks
within a single transaction, so keeping more buffer heads in
CPU-local cache speeds up modifications significantly.
User can set LRU size to any value between 8 and 64, linux
kernel is still using 8 and Lustre is going using 16 as
default value.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I4bb2eea30d57298b706354dd8438a10dc8ae6e98
Reviewed-on: http://review.whamcloud.com/1796
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-278 Improve regex for output of git describe

Improve the regex in version_tag-git.pl to properly parse the output
then the current commit is tagged, and when the tag names are longer
than just the digits used in the upstream version tags.

Addition from Ned:

Trim trailing newline from git describe output in the
case where the current commit is tagged.

Change-Id: I0a6a1b54f9e2dbd381a6d00f9dcf47d9b8b66616
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/509
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-824 corrupted ldiskfs after md rebuild (bz24264)

Pick up a patch from upstream to fix the md bug may
cause a corruption issue after rebuild.

Change-Id: I802ff3b3d5e86b9d9e77e57d1d98004c17e800a6
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1650
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-669 mea.c unconditionally uses ldiskfs hashs

Remove the ldiskfs hash code from mea.c as described by
comment 15 in Lustre bugzilla 22410.  The MEA_MAGIC_HASH_SEGMENT
case has been replaced with a CERROR() which will indicate if
this ever occurs.

"Yes, the two level hashing function is implemented in this patch.
And there no ldiskfs hash dependency on client anymore, so those
ldiskfs hash function can be removed from mea.c"
    -- wangdi

This was originally filed as Lustre bugzilla 22410.

With the removal of this code the ldiskfsfs_dirhash() function
no longer has any consumers in obdclass.  It is now only used
by the ldiskfs osd.  Since this functionality was originally
copied from ldiskfs it makes sense to remove it entirely and
have the osd rely on the ldiskfs implementation.

Change-Id: I017918cad57fc0641ebbd9471cb5b5717c65dea2
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1350
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-837 release semaphore for error case in gss_do_ctx_init_rpc

in gss_do_ctx_init_rpc, the r/w lock "obd->u.cli.cl_sem" is not released
if the obd->u.cli.cl_import is NULL and returns error.

Change-Id: Ib9bb4d4cb79adabe8ff661b9b4a7f6df5963d4ff
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1705
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-645 llite: Avoid unnecessary dentry rehashing

In patchless case the sequence __d_drop() then d_rehash_cond() creates
race window if dentry is already hashed, in this case rehashing should
be avoided.

Change-Id: I59e035c6d19b869955e66fb010c1b137d0aab0ec
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1434
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-878 test: pdirop fail timeout too soon

OBD_FAIL_MDS_PDO_LOCK timeout 10 seconds, but our autotest runs on
hypervisor, if sanityn test_40b can't finish in 10 seconds, it will
fail. Increase timeout to 15 should be able to fix this issue.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I85de77b949efc65f439eab64b2d6b709560df772
Reviewed-on: http://review.whamcloud.com/1751
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-335 build: Misc fixes to support SuSE platforms

Minor cleanups to the lustre base which only showed up when building
for a SuSE platform.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8708c1074b1adf87bd466c6910231e8edcbdc032
Reviewed-on: http://review.whamcloud.com/1678
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-668 mdd: Rename MDS_*_FL to LUSTRE_*_FL and update usage

Fully remove MDS_*_FL and replace them with LUSTE_*_FL.  These
values are explictly not defined using LDISKFS_*_FL as was
done in mdd_internal.h to avoid a dependency on ldiskfs.h.

"These flags are part of the wire protocol.  We already have
portable definitions of these as "MDS_APPEND_FL", et. al.
in lustre_idl.h.  Since MDS_*_FL is no longer used in the
Lustre code (in preference to LUSTRE_*_FL), it makes sense
to just rename these in lustre_idl.h, and their corresponding
uses in wire*.c."
    -- Andreas

This was originally filed as Lustre bugzilla 22408.

Change-Id: Ib606bd43a93fc7c53f1660cc63751ec4ff9a76ed
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1349
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-839 tests: test_52 fails with loop devices MRP-233

During loop mount actual device is replaced with /dev/loopX
Use mount path as umount parameter.

Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Change-Id: Ic0561ccfd52036d7ed18e05848be7b4342f4cc62
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Bruce Korb <bruce_korb@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1694
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-665 mds: Reduce MDS-OSS connection flags

Some connection flags MDSs send to OSSs are unnecessary or misleadingly
implying features not (yet) implemented by current MDSs.  This patch
removes the following ones:

  - OBD_CONNECT_64BITHASH.  MDSs don't send MDS_READPAGE or FLD_QUERY
    requests to OSSs.

  - OBD_CONNECT_BRW_SIZE.  MDSs don't do bulk I/O with OSSs.

  - OBD_CONNECT_CKSUM.  Ditto.

  - OBD_CONNECT_FID.  MDSs don't support OST FIDs.

Change-Id: If94f13cbb0a45b18f87163cfd0936566c67630e4
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1375
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-16 idl: fix obd_connect_data padding comments

Fix incorrect comments about 2.2 addition new padding fields
in obd_connect_data, since this was landed in 2.1 after all.
This doesn't really affect 2.1, since it is self-evident that
those fields are in 2.1, but may be relevant in future releases.

Add these new fields to wiretest/wirecheck as well.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0508934629dd028371211453c405b9c0a7cc9880
Reviewed-on: http://review.whamcloud.com/1447
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-846 docs: add lfs_migrate.1 to distributed RPMs

The lfs_migrate.1 map page needs to be installed so that users
more are aware of this command and how to use it.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9bb586496abc101c2de955664fd95b3b20a23818
Reviewed-on: http://review.whamcloud.com/1701
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Richard Henwood <rhenwood@whamcloud.com>

LU-527 build: update ChangeLog for recent releases

Update the lustre/ChangeLog:
- list supported kernel versions in descending order so that it is
  more clear which is the newest and oldest versions.
- change "Patchless Client" URL (which is very old news) to point
  to the "Lustre Support Matrix" page.
- update actual release date of Lustre 2.1

Update the lnet/ChangeLog:
- remove mention of old and obsolete LNDs entirely
- move some LNDs to "Available but unsupported:" for clarity
- indicate Lustre 2.1.0 was released
- add TBD Lustre 2.2.0 release block

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I623d9c0693763dce7448b5829a1ca16ee92ab048
Reviewed-on: http://review.whamcloud.com/1710
Tested-by: Hudson
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-322 tests: Pool test fixes for large number of OSTs

This patch fixes issues with large numbers of OSTs with the
ost-pools test. We need to use the hexidecimal numbers for
the pool args since the OSTs UUID are named with hex numbers.
Curently the test do pass but errors can be seen in the logs
when more than 9 OSTs exist.

Change-Id: I2bed80a1c55fc7cd405dc530876ef517a380c423
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/251
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>

LU-820 Increase LU_CDEBUG_LINE so that osc page messages fit.

LU_CDEBUG_LINE at 255 does not quite fit osc-page message out of
osc_page_print that is quite long.
As a result we get annoying "does not end in newline" console messages
when this happens.

Doubling LU_CDEBUG_LINE certainly should be enough.

Change-Id: Iec8635d6578e192d3b33643c4b1dab1dae2be6b4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1642
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-883 commit: commit-msg rewrite

 1. do global initializations in function.
    collect the read only and state monitoring initializations
    to one place where they can be easily identified.

 2. Move the signoff and changeid code to functions.
    Have them call a "ck_wrapup" function to validate
    the processing state for doing the summary/changeid/etc.
    change-id: may now appear before signed-off-by:

 3. Move the wrap up validations to the first-time-through-wrapup
    code in ck_wrapup.

 4. add a function for handling innocuous tag lines.
    Only call "ck_wrapup".

 5. Remove all $LINE echoing to the end of the case statement.
    Use "break" and/or "continue" to bypass line echo.

 6. use [ ${#foo} -gt 0 ] in preference to [ $foo ].
    This scriptlett will echo "no":
      $ f='! -z' ; [ $f ] && echo yes || echo no
    You could quote "$f", but directly checking length more
    correctly represents the intent.

 7. Removed conditions for setting HAS_LAST_BLANK=true because
    the conditions were written to be *ALWAYS* true.
      foo=false
      [ $foo ] && echo hi there
    will always echo out, "hi there".

 8. eliminate the convoluted and slightly wrong state management
    for trying to identify a "diff" block.  Instead, just read
    in the next line in that case clause and either break out
    of the read loop or echo out both of the lines.

 9. Do not fork and exec a bunch of programs when ${#LINE} will
    directly tell you how many characters are in the line.

10. do default line processing in a function, too.  Makes regex
    code selection much easier to understand.  Also allowed
    me to remove the ";&" case statement fall-through construct.
    Removed $HAS_SIGNOFF check since $IS_WRAPPING_UP is tested
    upon function entry.

11. emacs editor hints.

12. TODO: read the first two lines outside of the loop and
    verify that they are up to snuff.  Then remove a bunch of
    checks scattered about the code.  Line counting, etc., etc.

Signed-off-by: Bruce Korb <bruce_korb@xyratex.com>
Change-Id: I30a4a9b77b82af9f9b965823487001cbd5c28230
Reviewed-on: http://review.whamcloud.com/1764
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-810 changelog: Fix hsm_get_cl_xxx() helpers

Fix helpers for extracting information from HSM changelog records.

HSM records in changelog contain additionnal information from standard
records. This includes HSM event, error code and some flags.
Macros helpers exist to help user-space tools, like Policy Engine, to
extract them. Those helpers were broken, this fixes them.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I7ba1e6a3ec7635b646f7a2dfa8173bf90529fbd9
Reviewed-on: http://review.whamcloud.com/1651
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-847 quota: New quota tests to exercise accounting

three tests are added in sanity-quota.sh:
- basic disk usage tracking for user & group;
- disk usage transfer for user & group;
- disk usage across restart for user & group;

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I605370db52f07875f725f48d7b0ce1bab9af80b3
Reviewed-on: http://review.whamcloud.com/1380
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-651 osc: suppress message in can_merge_pages()

Thottle messages if adjacent brw pages are not mergeable with
different OBD_BRW_NOQUOTA flags.

Change-Id: I22ce6f8807e2541d3e6b3c9631f60faa36baa81a
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1328
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-663 kernel: Some arch do not have NUMA features anymore

Some architectures, especially x86_64, do not have cpu_to_node()
defined as a macro, and node_to_cpumask() exported by the kernel
anymore.

The cpu_to_node() routine is defined either as a macro, as an inline
routine using another exported symbol, or as an exported symbol.
Anyway, the kernel defines this service since at least version
2.6.12.

The node_to_cpumask() routine has been replaced by cpumask_of_node()
for x86 architectures since kernel version 2.6.30.

The set_cpus_allowed() routine is not defined if
CONFIG_CPUMASK_OFFSTACK=y since kernel version 2.6.32.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: If81269f403f888d4cde89c6fda5a8d7e10ea70b0
Reviewed-on: http://review.whamcloud.com/1345
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>

LU-506 kernel: FC15 - blkdev_get_by_dev() used instead of open_by_devnum().

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I9d633ba5c3004fd23de9522ebc8089792b96ed2c
Reviewed-on: http://review.whamcloud.com/1800
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-717 ldiskfs: MRP-222 Replace sysname with nodename in MMP

sysname holds "Linux" by default, i.e. what appears when doing a
"uname -s"; nodename should be used to print the machine's hostname,
i.e. what is returned when doing a "uname -n" or "hostname", and what
gethostname(2)/sethostname(2) manipulate, in order to notify the
administrator of the node which is contending to mount the
filesystem.

Andreas says this was introduced when porting the MMP patches from
RHEL5 to RHEL6, and then also pushed upstream to ext4; a patch for
upstream ext4 has already been submitted.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: I207bf145d114a9981b5a6add4bbf92ca76f71840
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1419
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-506 kernel: FC15 - tcp_sendpage() uses struct sock.

Since 2.6.36 tcp_sendpage() uses 'struct sock' as 1st argument
instead of 'struct socket'.

Change-Id: I1641541e3e52f3f47c2dfeb63a5311f5f9a0634a
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1739
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-887 mgc: prevent client IR with old server

Prevent IR enabled client start IR handling with IR unawared MGS
server.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I9773eafa437358dbf4988e0fddd490b0daf59358
Reviewed-on: http://review.whamcloud.com/1798
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-900 test: ncli.sh cause test failure

When a variable such as SRUN=$(which srun) returns empty, it
causes the test the fail
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I8e5550a6c2e65dc7fbc51c6a6ef6e7e62260d88b
Reviewed-on: http://review.whamcloud.com/1801
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-841 tests: sanity.sh 27q does not create a testing directory

sanity.sh 27q does not create a testing directory which causes
ENOENT errors from "ONLY=27q bash sanity.sh"

Signed-off-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Change-Id: I4b0b3839cc3cfd8cf643c7e4964cd0c22af39bea
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1698
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-881 utils: running bumped w/o lock

bumped_running accessed without lock

Multiple threads man increment shared_data->running until
the setting of bumped_running becomes visible.

Also, use "bool" -- First released in Issue 6 of POSIX spec:
   included for alignment with the ISO/IEC 9899:1999 standard

When inserting #include <stdbool.h>, it was noticed that stdio.h was
included twice.  Collected and ordered system headers so that
won't happen again.

Signed-off-by: Bruce Korb <bruce_korb@xyratex.com>
Change-Id: I45891e125221d29f72efee38580a56888c3a266f
Reviewed-on: http://review.whamcloud.com/1749
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>

LU-685 obdclass: lu_object reclamation is inefficient

Put only non-referenced lu_object in lru list to speed up object
reclamation.

Change-Id: I711bf14c99d63567590fa131cc0c8e41b7cc0cbe
Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1384
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-792 lbuild-rhel5 should use redhat's SRPM repo

Download EL5 kernel .src.rpm packages from redhat's repo instead
of trying to use CentOS's often-outdated repo.

Change-Id: I0e622e6481f11941b5038b36c0fd074fe470c595
Signed-off-by: Michael MacDonald <mjmac@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1590
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-542 Fix mdt xattr handler logic error

Record system ACL and user xattr change/deletion changelog.

Change-Id: I5aabf1879ec6e812361fe0d1b8255f84d0e817d6
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1158
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-506 FC15: unplug_fn removed since 2.6.39.

The old plugging interface isn't exist any more. So
disable the unplug_fn callback in this case.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I2396b845e2f2ef158f8e21e031a31ac83f3a5b02
Reviewed-on: http://review.whamcloud.com/1738
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-482 replay-dual test_0a failed

Running LVM on top of VM hypervisor has the write caching and write
reordering problem for kernels prior to 2.6.33.
This might corrupt the journal or fs metadata and lead to a mount failure
in first replay-dual test being run (typically test_0a).

Adding 10 seconds delay after the mount should be enough for the
changes to be flushed to disk.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I68bcb298f94480b26e506f92b3c018530cfe6106
Reviewed-on: http://review.whamcloud.com/1157
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-834 echo_client: fix page_is_vmlocked

->cpo_is_vmlocked() should return -EBUSY if page is locked,
or -ENODATA otherwise. Fix it for echo_client implementation.

Change-Id: I863591463fefd0c2afba32d04b62b983103a4e3d
Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1686
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-508 ldiskfs: fix race in ext4_ext_walk_space()

we should not access on-disk data (e.g. path->p_ext->*) with no
locking.

to be fixed in mainline ext4 as well.

Port from: ORI-291
Author: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic9dfcc9a90766c79c0da5ca54e9fbb2f917865a6
Reviewed-on: http://review.whamcloud.com/1618
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-553 build: improve checks for commit-msg

Improve the checks done by the commit-msg script.  It now ensures
that all the parts of the commit message are present.
- validate that the Change-Id: generated from 'git hash-object' is
  not empty, since this can happen if git is unhappy with the options
- check for only one Change-Id: line (multiple Signed-off-by: OK)
- describe the "component:" field better, with some examples

If there was an error committing the message, save a copy to a
temporary file, so that it can be edited and re-used, instead of
having to recreate it each time, or fetch it from .git/COMMIT_MSG.

Add a simple regression test with good & bad commit messages, so
it is easier to verify that any changes made to the script will
continue to both detect errors, and pass valid commit messages.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I15cb3690560400a591598997424cf79dee3a039d
Reviewed-on: http://review.whamcloud.com/1688
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>

LU-506 FC15: ctl_name & strategy removed from ctl_table.

The 'ctl_name' & 'strategy' field removed from
'struct ctl_table'.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I5eaeb46234083433109a150ea80408c9b3f9962b
Reviewed-on: http://review.whamcloud.com/1332
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>

LU-506 FC15: move some headers to include/generated

In FC15 (2.6.40-4) the generated kernel headers have moved
from "include/linux" to "include/generated". Update configure
scripts and makefiles to include this new directory. In a
number of cases, Lustre code was including generated headers
directly, but this was not really needed.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Id2094d8318681aa5ea08b416dc764bcf03bd8595
Reviewed-on: http://review.whamcloud.com/1329
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>

LU-506 FC15: update shrinker to use shrink_control callback

Linux 3.0 memory pressure shrinker now takes "struct shrink_control" as
its argument instead of "nr_to_scan" and "gfp_mask". This was backported
to Fedora 15.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Id9f6a9e10efe785d2837d1ad73098d2808a2f076
Reviewed-on: http://review.whamcloud.com/1331
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>

LU-857 security: Lustre client tolerates enforced SELinux.

Fix a bug which prevents Lustre clients to access directoriess when
SELinux is enforced, on RHEL 6.
This patch does not add a real SELinux support for Lustre but ables
to activate it for all other local filesystems, without Lustre
misbehaving.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: Ia6692c96a8439eb9239cb55ce32a1c54958241d1
Reviewed-on: http://review.whamcloud.com/1703
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-745 kernel: ost-pools test_23 hung

It could be caused by a jbd2 bug which result in forever sleep
in the do_get_write_access().

http://www.spinics.net/lists/linux-ext4/msg24689.html

In do_get_write_access() we wait on BH_Unshadow bit for buffer to get
from shadow state. The waking code in journal_commit_transaction() has
a bug because it does not issue a memory barrier after the buffer is moved
from the shadow state and before wake_up_bit() is called. Thus a waitqueue
check can happen before the buffer is actually moved from the shadow state
and waiting process may never be woken. Fix the problem by issuing proper
barrier.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I44dce352babc6699cdacc00263bfd3f24538400c
Reviewed-on: http://review.whamcloud.com/1675
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-624  Kernel update [RHEL6.1 2.6.32-131.17.1.el6]

Change-Id: I82ef82e11f846840707f9f65ca72bcda8885c4e0
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1632
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-737 utils: check device name for digit

We need to check the whole string for digit
not only the first character

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I316d931bb344d3e3fe5bb7d7a2454f200b637017
Reviewed-on: http://review.whamcloud.com/1641
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-759 mdc: Clear rq_replay on error in mdc_enqueue()

When mdc_enter_request() fails (e.g., due to signals) in mdc_enqueue(),
the request is freed without any care about its rq_replay field.  For
rq_replay requests, this results in assertion failures in
__ptlrpc_free_req().  This patch adds a call to mdc_clear_replay_flag()
to make sure __ptlrpc_free_req()'s assumption is respected.

Change-Id: I2185066a9f47b3d9563d9e1a8989754ef2e2dcb4
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1518
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-831 header: struct bit field should be unsigned type

Make sure struct bit fields be unsigned type, or else if they are read
from proc interface, they would showes as a big number equivalent to
-1.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ib252b4e89ce6b3f898e2da11a60de9aa9201119c
Reviewed-on: http://review.whamcloud.com/1685
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-835 build: skip generated files in .gitignore

Skip files automatically generated during builds on FC-13.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1cff9a351137145bb87a89460d0a1b54fae29e2c
Reviewed-on: http://review.whamcloud.com/1681
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-795 osd api: Commit callback per transaction

- ability to add commit callback per transaction in addition to
  per-device hooks. Now it is much simpler if only commit callback
  is needed.
- rewrite commit callbacks for last_commit and new_client, add commit
  callback in seq manager
- cleanup not-needed code: old commit callbacks, txn_keys
- remove osd od_env_for_commit environment and env param from commit
  callbacks
- use th_sync to mark sync operations

Change-Id: If5f8f2a6d3cd2f3e77fd13c802213a181043a2d7
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1621
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-691 Fix OST index errors in test suite - sanity 133c defect

Several test run do_facet ost which assumes ost is ost0 which does not exist
according to the way the test suite works. This patch address several areas
where the wrong ost index is used. This patch also introduces some short hand
functions to get OST properties from the index. Those functions ensure ost1,
ost10 and ost100 are seen as different which is not always the case.

Change-Id: Ic31224794563964a3415d24abeebce9dacceb686
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/1425
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Updated version to 2.1.52

Change-Id: I68a6934d997921a82670272cf30abdcc25d6a575
Signed-off-by: Oleg Drokin <green@whamcloud.com>

ORNL-3 mntopt: consider low-layer options for MDT ACL flags

Currently, MDT layer enables ACL support by default without checking
whether low-layer (ldiskfs) enables ACL support or not, then causes
unnecessary data traffic on network and through MDS stack for ACL.

So MDT should communicate with low-layer before setting ACL flags.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I804f10bf486745ddd3b23b89e959dfd585589ac0
Reviewed-on: http://review.whamcloud.com/1211
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-769 ptlrpc: Do not miss pending signals in ptlrpc_set wait

conf_sanity test 23a highlighted a problem in ptlrpc_set_wait logic,
if we enter there with a signal pending and the import is not FULL,
there is no way to interrupt such a set because we block signals
all the time. Enabling signals all the time is not an option either.
Waiting until import reconnects is questionable too since it might
never come up after all (like in the test 23a).
So for the solution we will just manually mark the set as interrupted
after the initial wait.

Change-Id: Iaa3e356e971b4f75fd7f21cc579c85f7487719a0
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1657
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>

LU-762 ldiskfs: don't drop directory nlink to 0

When landing the nlink patch for ext4, for an unknown reason the
logic in ext4_dec_count() was changed from the ext3 version of
the patch. It now drops the nlink = 0 temporarily and then
if it is a directory with nlink == 0 it increases nlink again.

Instead, only drop nlink if it is larger than 2.

Change-Id: Ieeff3e45daea56f502848f9c2b0fb04f0a9d2b6d
Author: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1644
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-780 test: improve parallel-scale to support hyperion run

We need to add support for srun/slurm, and a few tests
from hyperion-sanity script that has been used for hyperion
testing

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I7f1baa0c99980ad9001436911d23f1030aa7d0fe
Reviewed-on: http://review.whamcloud.com/1615
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Cliff White <cliffw@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-826 mdd: Fix for mdd_lov_create_finish memory free

At lov/lov_pack.c lov_packmd() use OBD_ALLOC_LARGE for memory
allocation for lov_mds_md object(lmpp), but mdd_lov_create_finish
use OBD_FREE to free memory. This bug doesn`t affect to current
version, but may be relevant in future.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: Ic37cb72022b9aac02368b11f370cbaad0c730e7c
Reviewed-on: http://review.whamcloud.com/1659
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-8 IR: minor fixes to IR patches

Summary of fixes:
1. ORNL-13: add comments to mgs_handle_fslog_hack();
2. ORNL-14: typo and accurate error/debug messages;
            minor fixes in recovery-small.sh;

Change-Id: I6317067eb6250faf5df21be719c82de17b0f4cc9
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1582
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-7 replace statahead hacking with its own cache

Original statahead hacks dcache, it inserts dentry into dcache and
associates with inode without holding parent's lock, which breaks
VFS layer synchronization mechanism.

The new statahead does not build dentry, it just pre-fetches inode's
attributes and related ldlm locks, caches them in a small statahead
cache against parent's inode. The statahead sponsor can search such
small cache similar as searching dcache to look for what it wants.
If cache hit, then builds dentry, associates with inode, and inserts
into dcache by the statahead sponsor itself.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I6948f8a438a938c51563468d775e676e4185e580
Reviewed-on: http://review.whamcloud.com/1208
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-774 test: wait till space from previous tests is released

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Idf4933f9b40c74b99bf763eb896680a4e0c942ff
Reviewed-on: http://review.whamcloud.com/1588
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-482 test-framework: Ensure dirty cache is flushed before barrier

With certain backend devices like LVM with older kernels the data
in dirty cache cannot be propagated all the way to the block device
with a single sync as there are multiple non-cooperating layers.

So convert such sync calls into triple syncs

Change-Id: If82e25223a277ec165d150b0f5f960ff845af9b0
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1656
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>

LU-773 tests: sanity:test_105b failure

The root cause of this issue is that activating osc in test sanity 104a
wasn't successful. We should wait for the recovery to finish.

Change-Id: I940419bfb1f579c4a0233b7439ac1f459ee584ad
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1542
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-601 mdd: Fix transaction credits

* mdd_create()/mdd_create_data() may need delete orphan objects on
  OSTs, so that we need preserve enough transaction credits for llog
  records.
* mdd_attr_set() may write lov llogs.
* orphan_object_destroy() also will write a llog record, we need
  reserve credit for it as well.
* add credit changelog record.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I5124d2f368e2ff794b2b2b8194bec86f63e971cf
Reviewed-on: http://review.whamcloud.com/1276
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-778 o2iblnd: Add rdma_create_id() compatibility macro

As of RHEL6.2 kernel 2.6.32-204.el6, rdma_create_id() requires a
queue-pair type as a fourth argument.  This was previously inferred
from the rdma_port_space argument.  Add an autoconf test to detect
whether the fourth argument is expected and a compatibility macro
that discards the QP type argument if the 3-argument version of
rdma_create_id() is present.

Change-Id: Idb668e1f059954ecc994ad59b366d54da8b82dc8
Signed-off-by: Ned Bass <bass6@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1556
Tested-by: Hudson
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-389 update wirecheck for master

Drop unused mds_body/mds_remote_perm definition and wirecheck.
Add missing wirecheck for mdt_xxx, ost_xxx, lu_xxx, and so on.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I858a02b0b5de8680cc8315e38aeeca271c26e9ad
Reviewed-on: http://review.whamcloud.com/985
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-26 prevent call md_set_lock_data() repeatly

md_set_lock_data() is called from different functions in llite, and
may be called more than once for the same <inode lock> pair. It is
harmless for the correctness, but will affect the performance a bit,
should be avoided.

Drop dentry flags of "DCACHE_LUSTRE_INVALID" only when we hold the
"MDS_INODELOCK_LOOKUP" lock.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I4dbe206af77ba6a619f3268c238ce98ac7aef4c0
Reviewed-on: http://review.whamcloud.com/1224
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-686 conf-sanity test 52 fails due to missing mount point

Create the mount point before mount

Change-Id: I6c813f5040e4636386eabb640526f3d1072f484b
Signed-off-by: HongChao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1379
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-620 llite: add delete_from_page_cache and remove_from_page_cache check

Later 2.6.32 kernel use memory cgroup feature but does not export
truncate_complete_page but export delete_from_page_cache or
remove_from_page_cache, we need properly use them for pachless client
code.

Change-Id: I33e3e7c32b548866ee77753ef8a8193c814d0ecb
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1399
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-000 libcfs: allow debug to be set to numeric -1

Don't warn if debug is set to "-1", which should always mean
"enable all debugging".

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1503d2f5776043e9a4ec3bc4af710ae35ca97960
Reviewed-on: http://review.whamcloud.com/1577
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-723 Enhance lustre/ldiskfs build system

Enhance the lustre/ldiskfs build system so it is more robust, flexible,
and consistent with lustre/zfs build system.  This change is being made
in the interest of standardizing the infra-structure around backend
filesystems.

This change does not effect the current behavior of the --with-ldiskfs,
--enable-ext4, or --with-ldiskfsprogs configure options.  However, it
does remove the obsolete --with-ldiskfs-inkernel configure option which
was only used by LLNL.  It also adds the --with-ldiskfs-obj configure
option which improves flexibility.  And the --enable-ldiskfs-build
configure option to support building against the lustre-ldiskfs-devel
package.  The behavior of these options is consistent with their zfs
counterparts, see commit 8c7266c for further details.

  --enable-ext4           enable ldiskfs build using ext4
  --enable-ldiskfs-build  enable ldiskfs configure/make
  --with-ldiskfs=path     set path to ldiskfs source
  --with-ldiskfs-obj=path set path to ldiskfs objects
  --with-ldiskfsprogs     use alternate names for ldiskfs-enabled e2fsprogs

Sample ./configure results when building lustre and ldiskfs using
the kernel-devel and kernel-debuginfo-common packages.

checking whether to enable ldiskfs... yes
checking ldiskfs source directory... /home/behlendo/src/git/lustre/ldiskfs
checking ldiskfs object directory... /home/behlendo/src/git/lustre/ldiskfs
checking ldiskfs module symbols... Module.symvers
checking ldiskfs source release... 3.3.0
checking whether to use ext3 or ext4 source... ext4
checking ext4 source directory...  /usr/src/debug/.../fs/ext4
checking whether to build ldiskfs... yes
checking for /home/behlendo/src/git/lustre/ldiskfs/configure... yes
checking for /usr/src/debug/.../fs/ext4/dir.c...  yes
checking for /usr/src/debug/.../fs/ext4/file.c...  yes
checking for /usr/src/debug/.../fs/ext4/inode.c...  yes
checking for /usr/src/debug/.../fs/ext4/super.c...  yes
checking if ext4_ext_walk_space() takes i_data_sem... yes
checking if LDISKFS_SINGLEDATA_TRANS_BLOCKS takes sb as argument... yes
checking if ldiskfs_discard_preallocations defined... yes
checking if ldiskfs_ext_insert_extent needs 5 arguments... yes

In the context of this change additional cleanup has been done and
all of the ldiskfs specific code relocated to lustre-build-ldiskfs.m4.

Note that this change moves us closer to supporting patchless Lustre
servers with ldiskfs.  Once the remaining kernel patches for Lustre
are dropped you will be able to build Lustre using the distribution
provided kernel-devel and kernel-debuginfo-common packages.

This change also incorperates ORI-340, commit f604951, which ensures
that the Module.symvers file will cleanly include the symbols for all
enabled Lustre backends.  While the only backend supported by master
right now is ldiskfs this brings the master and orion branchs in to
sync in this regard.

Change-Id: I6f13f266944ec6967f4d7705a30b83ab8e577b15
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1566
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-22 ptlrpc:  more comment for multi-threaded ptlrpcd

To explain share work load between ptlrpcd partners.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I72711a93af321e43e6dbbbc52b427060be47f808
Reviewed-on: http://review.whamcloud.com/1638
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-362 Fix error path in lu_kmem_init().

When echo client is failed to set up there are memory leaks were
noticed, it is related to missed error handling in some functions.

- Free all caches if lu_kmem_init() failed
- fix error handling in cl_global_init() and ccc_global_init()

Change-Id: Ide9e7ad6d40f99a7fbb4330ba63b168cc408356f
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/586
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-462 Don't alloc/free client data for self export

Self export doesn't need client data and ldlm initialization.
Patch uses uuid comparision to determine self_export.

Change-Id: Id26ef90e9857e4c1d3a0e7a3756eaf67607890d6
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1574
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-791 obdfilter: Don't clear OBD_MD_FLFLAGS mistakenly

Instead of set oa->o_valid to OBD_MD_FLID | OBD_MD_FLGROUP arbitrarily
in filter_handle_precreate(), it should be changed as "|=" to keep
the OBD_MD_FLFLAGS setted in filter_precreate().

Otherwise, client will not be aware of OST is running out of space,
and lov_create() will wait for objects forever in such case.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I84be8dde59dbb2829cd800e10b7aa6f4402b7e56
Reviewed-on: http://review.whamcloud.com/1589
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-28 recovery: rework extend_recovery_timer()

extend_recovery_timer() is used to adjust timeout value of a recovering
target. In the original implementation, there was a problem it stopped
the target from firing a timer again for version recovery case.

Change-Id: I815a15fb5d3104e52a189eed1529c58d7a8d03b9
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1620
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-611 utils: add restripe option to lfs_migrate

Add the "-R" option to have lfs_migrate restripe a migrated file
instead of keeping the original striping.  This is useful of some
directory got the wrong striping and a bunch of files were created
with the wrong striping.

Avoid possible confusion between the lfs_migrate and lfs setstripe
command-line options.  For now, deprecate the old "-c" option, since
it is redundant in any case.

Change-Id: I3a39bad93ef5c079678c65960e53d22e51431df3
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1265
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-532 mdt: improve xattr ctime warning message

Print out which xattr is not getting OBD_MD_FLCTIME set so that it
is possible to track down what code path on the client is failing.

Change-Id: I1918d2e8e0a1e03d8437846e823bca9df6f89b48
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1161
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Revert "LU-462 Don't alloc/free client data for self export"

This introduced a memory leak problem

This reverts commit 140178844e5c0e4f3cfed8199800e39bf7082cd9

Change-Id: I558da8c44e08f77e77f7d1fe79da892a579992c3
Reviewed-on: http://review.whamcloud.com/1631
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>

ORNL-22 general ptlrpcd threads pool support

Originally, there were two ptlrpcd threads on each node to serve all
async RPCs on the node, one ptlrpcd is for BRW, the other is for all
others. Such load mode cannot match more and more async RPCs process
on current large SMP node.

So we introduce ptlrpcd threads pool, any ptlrpcd threads in the pool
can be common shared by all async RPCs, like async I/O, async glimpse
lock, statahead, and ect. The async RPC sponsor can affect the system
load mode by specifying load policy when pushes the RPC into ptlrpcd
queue. On the other hand, it supports some flexible binding policies
to bind some ptlrpcd threads on CPU cores for reducing cross-CPU data
traffic, and also allow some ptlrpcd threads to be scheduled freely
on any CPU core to try to guarantee processing async RPCs in time.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: Icc0bd689df73b6863cc9adc544c3654c046cb8bd
Reviewed-on: http://review.whamcloud.com/1184
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-25 process dir page hash collision

If dir page has hash collision with others, then remove such page
from cache after using to avoid to be found unexpectedly later.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I15ff85e5233248944d77a9d93292d8690e1a715f
Reviewed-on: http://review.whamcloud.com/1234
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-770 tests: MRP-250 sanity.sh fails in CLIENTONLY mode

Skip sanity.sh tests which requires remote access to
lustre servers.

Signed-off-by: Andriy Skulysh <andriy_skulysh@xyratex.com>
Change-Id: Ia853a8cf95bf6bf638e391aab30654a06a3b6589
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1537
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-554 add gnilnd awareness to LNet

This allows servers on any network to talk to gnilnd routers.
This is 2.1 version of the Oracle 23884 attachment 31892.

Change-Id: I96777551b0caa50021ebb32755caaa01623ea97d
Signed-off-by: Wally Wang <wang@cray.com>
Reviewed-on: http://review.whamcloud.com/1285
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-715 lov: fix procfs reporting for qos values

When writing to
/proc/fs/lustre/lov/<fsname>-mdtlov/{qos_prio_free,qos_threshold_rr},
the values read back are often one less than the values written.
This happens because internally the value is stored as a number from
0-255 but accessed by the user with 0-100. Integer truncation in the
storage and retrieval stages causes this to often show lower. Adding
255 to an internal step causes the bit-shift to "round up".

Signed-off-by: Matt Ezell <ezell@nics.utk.edu>
Change-Id: I9050aadb55bfa82d14b94a78e399d315249ac48f
Reviewed-on: http://review.whamcloud.com/1532
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-700 osc: Tally BRW_{READ,WRITE}_BYTES by bytes transferred.

Call ptlrpc_lprocfs_brw() in brw_interpret() rather than
osc_send_oap_rpc() and tally by the number of bytes transferred rather
than the number requested.

Change-Id: Ia7191972d9671f01d942a46eba069191f130f516
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/1402
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-672 MRP-213 don't panic on geting version for non existent fid

lctl getobjversion can be called for a file which was removed from
MDT, but exists in cache. That producea a assert on mdd code:
LustreError: 20825:0:(mdd_object.c:2474:mdd_version_get())
ASSERTION(mdd_object_exists(mdd_obj)) failed

Author: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I7442e7aee26736741482c158ee3713df9796c953
Reviewed-on: http://review.whamcloud.com/1365
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-723 ldiskfs: remove ext3 RHEL5 kernel series

Remove the old ext3 RHEL5 kernel patch series.  This has been
deprecated since Lustre 1.8.6 in favour of the ext4 RHEL5 series.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1e09ff432f2e970446c3b43fb92f0c1a988159ae
Reviewed-on: http://review.whamcloud.com/1603
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>