LU-81 deadlock of changelog adding vs. changelog cancelling

This is a workaround for the deadlock of changelog adding vs.
changelog cancelling. Changelog adding always start transaction
before acquiring the catlog lock(lgh_lock), whereas, changelog
cancelling do start transaction after holding the catlog lock.

We start transaction earlier to avoid above deadlock.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I9647b9a559f68a27dc0d4b4885857d3cf73b5b8e
Reviewed-on: http://review.whamcloud.com/1260
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-625 test: sanityn.sh test file size should be dynamic

For large OSTCOUNT, a small write can't take extent locks on all OSTs.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ic3fb8e88444767db70545292c4b81b9fe9f1f813
Reviewed-on: http://review.whamcloud.com/1901
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-884 clio: client in memory checksum

Use page_mkwrite() method from latest kernels to correctly implement
RPC checksum functionality. Also OBD_FL_MMAP is removed because it
won't be used any more.

Change-Id: I6ec5aae14f56c95b1ac6936d21b5a273582fa4e8
Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1609
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-571 ldlm: add parallel ast flow control

Commit {hash: 8c83e7d75989ef527e43a824a0dbe46bffabd07d} removed the
parallel AST limit on the server. However, if there are too many locks
to revoke, it will have to allocate too many RPCs.

Return to having an upper limit on the number of AST RPCs in flight by
adding a flow control algorithm that allows a configurable upper limit on
the number of RPCs in flight.

Change-Id: Ifb68991acf7a33119b334447aec50f7717ed546e
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1608
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-635 tests: conf-sanity 27 28 29 30 35 43 failures

Several test in conf-sanity would fail with a error
This command must be run on the MGS. This was due
to seperating the MGS and MDS. I tracked down the fix
to doing a pdsh for several lctl commands to MGS
instead of the MDS.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I721472b15821e2dbcc636292a8e82c1a1b5e0149
Reviewed-on: http://review.whamcloud.com/1869
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-424 tests: conf-sanity test 55, 56, 58 failure fixes

The MGS service was not started in conf-sanity test 55,
56 and 58 with separate MGS and MDT configuration. This
patch fix the issue.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I0c01955b4baf81535959cdbf38bf84e7acb04ddf
Reviewed-on: http://review.whamcloud.com/1870
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New tag 2.1.53

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I0d5a2dc7af8e866f59b5d1dcf5b7ccc6e4268567

LU-417 llite: report non-zero blocks on writing client

Writing client may not report accurate allocated block numbers when
dirty pages has not been writting back to OSTs, some "cp" or "tar" may
skip the file because it thinks it is completely sparse.

This patch makes writing client consider dirty pages when reporting
allocated blocks, lest the file be treated as a completely sparse one.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I985d50a44ea1e917bf8e1cba3b5cb770eec35c3f
Reviewed-on: http://review.whamcloud.com/1647
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-935 quota: break early when b/i_unit_sz exceeded upper limit

While expanding b/i_unit_sz in dquot_create_oqaq(), we'd break the loop
early when the b/i_unit_sz exceeded upper limit, otherwise, qaq_b/iunit_sz
could be overflow and result in endless loop.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I0bf069e9259627426d7a87ec42844eaed7a733b4
Reviewed-on: http://review.whamcloud.com/1890
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-506 kernel: FC15 - fix GCC 'set-but-unused' warnings

Eliminate a number of unused variables in the code that
caused GCC-4.6.0 to warn about "set-but-unused variables".
In some cases the variables are only used in
conditionally-compiled code, so they are used in some way
just to quiet the compiler.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I8b9b744c787fff4057722420866adea86d8240f5
Reviewed-on: http://review.whamcloud.com/1330
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-506 kernel: FC15 - super_operations add evict_inode method.

super_operations add a new method evict_inode(), it hybird of
delete_inode() and clear_inode().

Change-Id: Ifa6de3bca16294386f33404b5a42ff749e1d0665
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1844
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-506 kernel: FC15 - generic_permission() need 4 arguments.

generic_permission() add a new parameter flags to determine
if in rcu-walk case. it will return -ECHILD when rcu-walk
request cannot be satisfied. check_acl callback also changed
for this intention. So we don't need check it separately.

Change-Id: I15e0660f47c5db414dbb920f68177f310a185e4d
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1843
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>

LU-335 ldiskfs: SuSE 11 SP1 support for Lustre 2.X

Lustre 2.X has been lagging for SuSE server side support. This
work brings the code up to date to support the SuSE 11 SP1 kernel
2.6.32.36-0.5. To reduce the maintenance overhead we basically
add a patch that modifies the SuSE ext4 code to match very
closely to the RHEL6.1 ext4 code base.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I554679c9b0d400eeda1d5d30157c5d86e68793f9
Reviewed-on: http://review.whamcloud.com/575
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-766 Fix sops->statfs() autoconf check

Building with RHEL6.2 beta (kernel-2.6.32-204) exposed that the
HAVE_STATFS_DENTRY_PARAM autoconf check was incorrect. The test
was setup to check the first argument of vfs_statfs() function.
However, the conditional code itself doesn't use the vfs_statfs()
function it registers a function with the sops->statfs() callback.

Historically, this always worked because both of these prototypes
were changed in a single upstream kernel commit (726c3342). However,
as of RHEL6.2 the vfs_statfs() function changed again and the
sops->statfs() callback stayed the same. The vfs_statfs() function
now takes a path structure as the first parameter.

This was fixed by simply updating the autoconf test to check the
prototype of the function were going to actually use.

Change-Id: I0cf36e758fd9c85609c47c13dbdf249321664b26
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1527
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-50 ldiskfs: pdirops patch for ldiskfs

Single directory performance is a critical for HPC workloads. In a
typical use case an application creates a separate output file for
each node and task in a job. As nodes and tasks increase, hundreds
of thousands of files may be created in a single directory within
a short window of time.
Today, both filename lookup and file system modifying operations
(such as create and unlink) are protected with a single lock for
an entire ldiskfs directory. PDO project will remove this
bottleneck by introducing a parallel locking mechanism for entire
ldiskfs directories. This work will enable multiple application
threads to simultaneously lookup, create and unlink in parallel.

This patch contains:
  - pdirops support for ldiskfs
  - N-level htree directory
  - integrate with osd-ldiskfs

Change-Id: I269c0e3112e68f3acd79e860dab052a68c7d7aaa
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/375
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-843 iokit: get_ec_devno returns trash

Some commands leak output to stdout and get_ec_devno returns
its results via stdout.  Redirect fd 8 to stdout and redirect
stdout to stderr.  Echo final results to fd 8.  Since this
only works when run as a subshell, invoke "exit" instead of
"return" when done.

Inside create_objects() function, stdin is redirected for a
shell "while" command.  Though this is done in the current
environment (process) for current BASH, it is not required
and there is no committment that BASH will always behave
this way.  Instead, set fd 3 to be reading from that file
and read from fd 3 insted of stdin.

While there, also test for "is object id" lines and extract
values using shell constructs instead of fork-ed commands.

Also while there, ensure that "ERROR" is always printed on
every error exit.

Also while there, ensure that the libecho file comes from
the same directory as the obdfilter-survey script.

Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Bruce Korb <bruce_korb@xyratex.com>
Change-Id: I50137c9ded816aac792aba4a0a79f59c28d16e01
Reviewed-on: http://review.whamcloud.com/1699
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-537 o2iblnd: support aliased interface

Current kiblnd_create_dev() doesn't allow to dev_get_by_name() to
return NULL (we added this for supporting failover of ib-bonding),
but this can happen for aliased interface, we can fix this by simply
remove the limitation and set kib_dev_t::ibd_can_failover to 0 for
aliased interface.

Change-Id: Idc3e6e4fd2d5056594f38dd43a0a560c3455ea05
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1148
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-431 o2iblnd: reduce failed alloc retry interval

o2iblnd can dynamically grow TX descriptor pool at run time,
it will take 10 seconds as retry interval if there is a failed
allocating, this interval is too long and some RPC might timeout.
So we decreased it to one second.

Change-Id: I0693e9d2cbf321e473713db6dd668afaf64e1bae
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/966
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-803 echo: fill OBD_MAGIC for echo device

Otherwise it will cause failure on obdfilter-survey.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I505da432cdf04fe3275d2a529f4115b1a3238e94
Reviewed-on: http://review.whamcloud.com/1804
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-794 llite: allow larger O_DIRECT read/write RPC

The O_DIRECT read/write RPC size limit was formerly constrained by
the maximum kmalloc() size, and would fall back to smaller kmalloc()
allocations until they succeeded.  This constrained the number of
concurrent O_DIRECT RPCs, which can significantly affect performance.

Newer code uses OBD_ALLOC_LARGE()  vmalloc() for allocations larger
than a page, so the maximum RPC limit can be increased.  Currently
the limit is arbitrarily set to KMALLOC_MAX_SIZE (which is just a
proxy for what is a reasonably large allocation is for the system).

This increases the maximum O_DIRECT limit to about 682MB, which will
allow sending 8 concurrent RPCs on an 85-stripe file with a single
write() or combination thereof.  This could be increased if needed.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic12e6536cebed94c3a3b56acbea0f0940cdafec9
Reviewed-on: http://review.whamcloud.com/1617
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-579 MRP-120 changelog CLOSE event.

add CLOSE event into changelog, but not into the default mask.

do not create close changelog record sometimes, if a dup mfd handler appeared:
- open replay (1st mfd appears)
- re-connect
- open replay (2nd mfd appears)

Reviewed-by: Nathan Rutman <Nathan_Rutman@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I8e5ff99905b84ee58416406ca7b690c86db60c62
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1197
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-543 mdd: fix rename changelog

Current rename changelog stores source fid in both CL_RENAME & CL_EXT
records, which is redundant, and the 'tfid' in CL_EXT is never been
used.

Actually, we'd store target fid in the CL_EXT record, then application
could detect the fid unlinked by rename in changelog.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I0c616f813657a2faefa60a707f4fc1d9dc971b39
Reviewed-on: http://review.whamcloud.com/1652
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-848 clio: page writeback support

Page writeback support for clio - release vmpage lock in transfer.
Sync IO still holds page lock.

Change-Id: I5137542b5b6b6eaaa4464804b45d372fd12e5c22
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1456
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-680 lfs: instance <-> mount point mapping from lfs

A new option to 'lfs' has been created to return the mapping
between Lustre filesystem instance and paths. The option
is 'getname' and it may be called with or without arguements.

'lfs getname' without arguments returns the instances of all
Lustre mount points.

'lfs getname [path...]' returns the instance of each specified
path. If the path is not a Lustre instance 'No such device' is
returned.

OBD_IOC_GETNAME has been added to file.c to provide consistent
behavior for file as well as directory paths.

A llapi_getname helper function has been added to liblustreapi
that returns a lustre instance name if a path is provided.

Documentation for 'lfs getname' is included inline an the lfs
man page has been updated.

Signed-off-by: Richard Henwood <rhenwood@whamcloud.com>
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Change-Id: Iab8ff12d604c7ce853f3c204b455e3b641f659f4
Reviewed-on: http://review.whamcloud.com/1373
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-575 MRP-133 lfs find speedup

lfs find should send getattr on mds only if needed;
lfs find should not break on matched obd but check other parameters as well;
lfs find time compare fixes;

Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Colin Faber <colin.faber@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
LU-575 MRP-260 fix quota tests

a fix for quota size units which conflicted with lfind size units

Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I42b2acee18c6c9e0d787b97a5e957431d2bb4bf1
Reviewed-on: http://review.whamcloud.com/1195
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-143 fid hash improvements

Current hash function of fid is not good enough for lu_site and
ldlm_namespace.
We have to use two totally different hash functions to hash fid into
hash table with millions of entries.

Change-Id: I6261e63a406118a93d578210c31e67fc7f9e389c
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/374
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-106 procfs: many proc entries are not accessed safely

Some in memory data may be released/uninitialized at the time
of proc entry creation/removal, this patch includes the following
fixes:
* initialize data before proc entry creation
* free data after proc entry removal
* free proc entries in obd_precleanup() because
  obd_uuid/nid/nid_stats_hash are released in class_cleanup().
* free proc entries after obd_zombie_barrier() because obd_export
  hold one refcound of nid_stat.
* check osd->od_mount before accessing osd proc entries because the
  osd proc entries are created before mount.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I03cb977e1be0747032a70f6a39fec804f81d70cc
Reviewed-on: http://review.whamcloud.com/326
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-871 build: change %L printk format to %ll

The clang-analyzer static code analysis flagged %L format specifiers
for printf/printk as deprecated, so change all of their uses to %ll.

In a couple of files (checkstat.c and truncate.c) there are existing
code style issues that need to be fixed before landing, in order to
get a clean bill of health from checkpatch.pl.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1bc75a0f43b25de112f5ae8f2273b3dc46081d09
Reviewed-on: http://review.whamcloud.com/1734
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-761: Check return of ll_prep_md_op_data() using IS_ERR().

Change-Id: I329fa923b95ae612e50f63641d2c34b121b0f6b9
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/1520
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-864 test: Hostname name doesn't equal NID and use facet_mntpt

On our test systems the hostname of the servers is on the
ethernet fabric and for lustre we use infiniband separately.
Currently several test will not run because of the assumption
that lustre will use the same fabric and we use to login and
manage the server. This patch allows one to grab the real NIDs
instead of assuming its "hostname"@NETTYPE. The second part of
the patch removes using $MOUNT% directly and uses facet_mntpt.
The reason being is on our system the mount points are different
on our servers from our clients. MOUNT is assumed to be the same
for clients and the servers. This patch allows test to work if
that is not the case.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I49f365561e341d44ace4c961fcae8b871dce7407
Reviewed-on: http://review.whamcloud.com/248
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-50 kernel: configurable BH LRU size

This patch allow per-CPU LRU size to be selected at compile time.
More complex filesystems may be modiyfing multiple blocks
within a single transaction, so keeping more buffer heads in
CPU-local cache speeds up modifications significantly.
User can set LRU size to any value between 8 and 64, linux
kernel is still using 8 and Lustre is going using 16 as
default value.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I4bb2eea30d57298b706354dd8438a10dc8ae6e98
Reviewed-on: http://review.whamcloud.com/1796
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-278 Improve regex for output of git describe

Improve the regex in version_tag-git.pl to properly parse the output
then the current commit is tagged, and when the tag names are longer
than just the digits used in the upstream version tags.

Addition from Ned:

Trim trailing newline from git describe output in the
case where the current commit is tagged.

Change-Id: I0a6a1b54f9e2dbd381a6d00f9dcf47d9b8b66616
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/509
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-824 corrupted ldiskfs after md rebuild (bz24264)

Pick up a patch from upstream to fix the md bug may
cause a corruption issue after rebuild.

Change-Id: I802ff3b3d5e86b9d9e77e57d1d98004c17e800a6
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1650
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-669 mea.c unconditionally uses ldiskfs hashs

Remove the ldiskfs hash code from mea.c as described by
comment 15 in Lustre bugzilla 22410.  The MEA_MAGIC_HASH_SEGMENT
case has been replaced with a CERROR() which will indicate if
this ever occurs.

"Yes, the two level hashing function is implemented in this patch.
And there no ldiskfs hash dependency on client anymore, so those
ldiskfs hash function can be removed from mea.c"
    -- wangdi

This was originally filed as Lustre bugzilla 22410.

With the removal of this code the ldiskfsfs_dirhash() function
no longer has any consumers in obdclass.  It is now only used
by the ldiskfs osd.  Since this functionality was originally
copied from ldiskfs it makes sense to remove it entirely and
have the osd rely on the ldiskfs implementation.

Change-Id: I017918cad57fc0641ebbd9471cb5b5717c65dea2
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1350
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-837 release semaphore for error case in gss_do_ctx_init_rpc

in gss_do_ctx_init_rpc, the r/w lock "obd->u.cli.cl_sem" is not released
if the obd->u.cli.cl_import is NULL and returns error.

Change-Id: Ib9bb4d4cb79adabe8ff661b9b4a7f6df5963d4ff
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1705
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-645 llite: Avoid unnecessary dentry rehashing

In patchless case the sequence __d_drop() then d_rehash_cond() creates
race window if dentry is already hashed, in this case rehashing should
be avoided.

Change-Id: I59e035c6d19b869955e66fb010c1b137d0aab0ec
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1434
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-878 test: pdirop fail timeout too soon

OBD_FAIL_MDS_PDO_LOCK timeout 10 seconds, but our autotest runs on
hypervisor, if sanityn test_40b can't finish in 10 seconds, it will
fail. Increase timeout to 15 should be able to fix this issue.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I85de77b949efc65f439eab64b2d6b709560df772
Reviewed-on: http://review.whamcloud.com/1751
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-335 build: Misc fixes to support SuSE platforms

Minor cleanups to the lustre base which only showed up when building
for a SuSE platform.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8708c1074b1adf87bd466c6910231e8edcbdc032
Reviewed-on: http://review.whamcloud.com/1678
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-668 mdd: Rename MDS_*_FL to LUSTRE_*_FL and update usage

Fully remove MDS_*_FL and replace them with LUSTE_*_FL.  These
values are explictly not defined using LDISKFS_*_FL as was
done in mdd_internal.h to avoid a dependency on ldiskfs.h.

"These flags are part of the wire protocol.  We already have
portable definitions of these as "MDS_APPEND_FL", et. al.
in lustre_idl.h.  Since MDS_*_FL is no longer used in the
Lustre code (in preference to LUSTRE_*_FL), it makes sense
to just rename these in lustre_idl.h, and their corresponding
uses in wire*.c."
    -- Andreas

This was originally filed as Lustre bugzilla 22408.

Change-Id: Ib606bd43a93fc7c53f1660cc63751ec4ff9a76ed
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1349
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-839 tests: test_52 fails with loop devices MRP-233

During loop mount actual device is replaced with /dev/loopX
Use mount path as umount parameter.

Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Change-Id: Ic0561ccfd52036d7ed18e05848be7b4342f4cc62
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Bruce Korb <bruce_korb@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1694
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-665 mds: Reduce MDS-OSS connection flags

Some connection flags MDSs send to OSSs are unnecessary or misleadingly
implying features not (yet) implemented by current MDSs.  This patch
removes the following ones:

  - OBD_CONNECT_64BITHASH.  MDSs don't send MDS_READPAGE or FLD_QUERY
    requests to OSSs.

  - OBD_CONNECT_BRW_SIZE.  MDSs don't do bulk I/O with OSSs.

  - OBD_CONNECT_CKSUM.  Ditto.

  - OBD_CONNECT_FID.  MDSs don't support OST FIDs.

Change-Id: If94f13cbb0a45b18f87163cfd0936566c67630e4
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1375
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-16 idl: fix obd_connect_data padding comments

Fix incorrect comments about 2.2 addition new padding fields
in obd_connect_data, since this was landed in 2.1 after all.
This doesn't really affect 2.1, since it is self-evident that
those fields are in 2.1, but may be relevant in future releases.

Add these new fields to wiretest/wirecheck as well.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0508934629dd028371211453c405b9c0a7cc9880
Reviewed-on: http://review.whamcloud.com/1447
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-846 docs: add lfs_migrate.1 to distributed RPMs

The lfs_migrate.1 map page needs to be installed so that users
more are aware of this command and how to use it.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9bb586496abc101c2de955664fd95b3b20a23818
Reviewed-on: http://review.whamcloud.com/1701
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Richard Henwood <rhenwood@whamcloud.com>

LU-527 build: update ChangeLog for recent releases

Update the lustre/ChangeLog:
- list supported kernel versions in descending order so that it is
  more clear which is the newest and oldest versions.
- change "Patchless Client" URL (which is very old news) to point
  to the "Lustre Support Matrix" page.
- update actual release date of Lustre 2.1

Update the lnet/ChangeLog:
- remove mention of old and obsolete LNDs entirely
- move some LNDs to "Available but unsupported:" for clarity
- indicate Lustre 2.1.0 was released
- add TBD Lustre 2.2.0 release block

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I623d9c0693763dce7448b5829a1ca16ee92ab048
Reviewed-on: http://review.whamcloud.com/1710
Tested-by: Hudson
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-322 tests: Pool test fixes for large number of OSTs

This patch fixes issues with large numbers of OSTs with the
ost-pools test. We need to use the hexidecimal numbers for
the pool args since the OSTs UUID are named with hex numbers.
Curently the test do pass but errors can be seen in the logs
when more than 9 OSTs exist.

Change-Id: I2bed80a1c55fc7cd405dc530876ef517a380c423
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/251
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>

LU-820 Increase LU_CDEBUG_LINE so that osc page messages fit.

LU_CDEBUG_LINE at 255 does not quite fit osc-page message out of
osc_page_print that is quite long.
As a result we get annoying "does not end in newline" console messages
when this happens.

Doubling LU_CDEBUG_LINE certainly should be enough.

Change-Id: Iec8635d6578e192d3b33643c4b1dab1dae2be6b4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1642
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-883 commit: commit-msg rewrite

 1. do global initializations in function.
    collect the read only and state monitoring initializations
    to one place where they can be easily identified.

 2. Move the signoff and changeid code to functions.
    Have them call a "ck_wrapup" function to validate
    the processing state for doing the summary/changeid/etc.
    change-id: may now appear before signed-off-by:

 3. Move the wrap up validations to the first-time-through-wrapup
    code in ck_wrapup.

 4. add a function for handling innocuous tag lines.
    Only call "ck_wrapup".

 5. Remove all $LINE echoing to the end of the case statement.
    Use "break" and/or "continue" to bypass line echo.

 6. use [ ${#foo} -gt 0 ] in preference to [ $foo ].
    This scriptlett will echo "no":
      $ f='! -z' ; [ $f ] && echo yes || echo no
    You could quote "$f", but directly checking length more
    correctly represents the intent.

 7. Removed conditions for setting HAS_LAST_BLANK=true because
    the conditions were written to be *ALWAYS* true.
      foo=false
      [ $foo ] && echo hi there
    will always echo out, "hi there".

 8. eliminate the convoluted and slightly wrong state management
    for trying to identify a "diff" block.  Instead, just read
    in the next line in that case clause and either break out
    of the read loop or echo out both of the lines.

 9. Do not fork and exec a bunch of programs when ${#LINE} will
    directly tell you how many characters are in the line.

10. do default line processing in a function, too.  Makes regex
    code selection much easier to understand.  Also allowed
    me to remove the ";&" case statement fall-through construct.
    Removed $HAS_SIGNOFF check since $IS_WRAPPING_UP is tested
    upon function entry.

11. emacs editor hints.

12. TODO: read the first two lines outside of the loop and
    verify that they are up to snuff.  Then remove a bunch of
    checks scattered about the code.  Line counting, etc., etc.

Signed-off-by: Bruce Korb <bruce_korb@xyratex.com>
Change-Id: I30a4a9b77b82af9f9b965823487001cbd5c28230
Reviewed-on: http://review.whamcloud.com/1764
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-810 changelog: Fix hsm_get_cl_xxx() helpers

Fix helpers for extracting information from HSM changelog records.

HSM records in changelog contain additionnal information from standard
records. This includes HSM event, error code and some flags.
Macros helpers exist to help user-space tools, like Policy Engine, to
extract them. Those helpers were broken, this fixes them.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I7ba1e6a3ec7635b646f7a2dfa8173bf90529fbd9
Reviewed-on: http://review.whamcloud.com/1651
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-847 quota: New quota tests to exercise accounting

three tests are added in sanity-quota.sh:
- basic disk usage tracking for user & group;
- disk usage transfer for user & group;
- disk usage across restart for user & group;

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I605370db52f07875f725f48d7b0ce1bab9af80b3
Reviewed-on: http://review.whamcloud.com/1380
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-651 osc: suppress message in can_merge_pages()

Thottle messages if adjacent brw pages are not mergeable with
different OBD_BRW_NOQUOTA flags.

Change-Id: I22ce6f8807e2541d3e6b3c9631f60faa36baa81a
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1328
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-663 kernel: Some arch do not have NUMA features anymore

Some architectures, especially x86_64, do not have cpu_to_node()
defined as a macro, and node_to_cpumask() exported by the kernel
anymore.

The cpu_to_node() routine is defined either as a macro, as an inline
routine using another exported symbol, or as an exported symbol.
Anyway, the kernel defines this service since at least version
2.6.12.

The node_to_cpumask() routine has been replaced by cpumask_of_node()
for x86 architectures since kernel version 2.6.30.

The set_cpus_allowed() routine is not defined if
CONFIG_CPUMASK_OFFSTACK=y since kernel version 2.6.32.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: If81269f403f888d4cde89c6fda5a8d7e10ea70b0
Reviewed-on: http://review.whamcloud.com/1345
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>

LU-506 kernel: FC15 - blkdev_get_by_dev() used instead of open_by_devnum().

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I9d633ba5c3004fd23de9522ebc8089792b96ed2c
Reviewed-on: http://review.whamcloud.com/1800
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-717 ldiskfs: MRP-222 Replace sysname with nodename in MMP

sysname holds "Linux" by default, i.e. what appears when doing a
"uname -s"; nodename should be used to print the machine's hostname,
i.e. what is returned when doing a "uname -n" or "hostname", and what
gethostname(2)/sethostname(2) manipulate, in order to notify the
administrator of the node which is contending to mount the
filesystem.

Andreas says this was introduced when porting the MMP patches from
RHEL5 to RHEL6, and then also pushed upstream to ext4; a patch for
upstream ext4 has already been submitted.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: I207bf145d114a9981b5a6add4bbf92ca76f71840
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1419
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-506 kernel: FC15 - tcp_sendpage() uses struct sock.

Since 2.6.36 tcp_sendpage() uses 'struct sock' as 1st argument
instead of 'struct socket'.

Change-Id: I1641541e3e52f3f47c2dfeb63a5311f5f9a0634a
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1739
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-887 mgc: prevent client IR with old server

Prevent IR enabled client start IR handling with IR unawared MGS
server.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I9773eafa437358dbf4988e0fddd490b0daf59358
Reviewed-on: http://review.whamcloud.com/1798
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-900 test: ncli.sh cause test failure

When a variable such as SRUN=$(which srun) returns empty, it
causes the test the fail
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I8e5550a6c2e65dc7fbc51c6a6ef6e7e62260d88b
Reviewed-on: http://review.whamcloud.com/1801
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-841 tests: sanity.sh 27q does not create a testing directory

sanity.sh 27q does not create a testing directory which causes
ENOENT errors from "ONLY=27q bash sanity.sh"

Signed-off-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Change-Id: I4b0b3839cc3cfd8cf643c7e4964cd0c22af39bea
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1698
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-881 utils: running bumped w/o lock

bumped_running accessed without lock

Multiple threads man increment shared_data->running until
the setting of bumped_running becomes visible.

Also, use "bool" -- First released in Issue 6 of POSIX spec:
   included for alignment with the ISO/IEC 9899:1999 standard

When inserting #include <stdbool.h>, it was noticed that stdio.h was
included twice.  Collected and ordered system headers so that
won't happen again.

Signed-off-by: Bruce Korb <bruce_korb@xyratex.com>
Change-Id: I45891e125221d29f72efee38580a56888c3a266f
Reviewed-on: http://review.whamcloud.com/1749
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>

LU-685 obdclass: lu_object reclamation is inefficient

Put only non-referenced lu_object in lru list to speed up object
reclamation.

Change-Id: I711bf14c99d63567590fa131cc0c8e41b7cc0cbe
Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1384
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-792 lbuild-rhel5 should use redhat's SRPM repo

Download EL5 kernel .src.rpm packages from redhat's repo instead
of trying to use CentOS's often-outdated repo.

Change-Id: I0e622e6481f11941b5038b36c0fd074fe470c595
Signed-off-by: Michael MacDonald <mjmac@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1590
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-542 Fix mdt xattr handler logic error

Record system ACL and user xattr change/deletion changelog.

Change-Id: I5aabf1879ec6e812361fe0d1b8255f84d0e817d6
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1158
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-506 FC15: unplug_fn removed since 2.6.39.

The old plugging interface isn't exist any more. So
disable the unplug_fn callback in this case.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I2396b845e2f2ef158f8e21e031a31ac83f3a5b02
Reviewed-on: http://review.whamcloud.com/1738
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-482 replay-dual test_0a failed

Running LVM on top of VM hypervisor has the write caching and write
reordering problem for kernels prior to 2.6.33.
This might corrupt the journal or fs metadata and lead to a mount failure
in first replay-dual test being run (typically test_0a).

Adding 10 seconds delay after the mount should be enough for the
changes to be flushed to disk.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I68bcb298f94480b26e506f92b3c018530cfe6106
Reviewed-on: http://review.whamcloud.com/1157
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-834 echo_client: fix page_is_vmlocked

->cpo_is_vmlocked() should return -EBUSY if page is locked,
or -ENODATA otherwise. Fix it for echo_client implementation.

Change-Id: I863591463fefd0c2afba32d04b62b983103a4e3d
Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1686
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-508 ldiskfs: fix race in ext4_ext_walk_space()

we should not access on-disk data (e.g. path->p_ext->*) with no
locking.

to be fixed in mainline ext4 as well.

Port from: ORI-291
Author: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ic9dfcc9a90766c79c0da5ca54e9fbb2f917865a6
Reviewed-on: http://review.whamcloud.com/1618
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-553 build: improve checks for commit-msg

Improve the checks done by the commit-msg script.  It now ensures
that all the parts of the commit message are present.
- validate that the Change-Id: generated from 'git hash-object' is
  not empty, since this can happen if git is unhappy with the options
- check for only one Change-Id: line (multiple Signed-off-by: OK)
- describe the "component:" field better, with some examples

If there was an error committing the message, save a copy to a
temporary file, so that it can be edited and re-used, instead of
having to recreate it each time, or fetch it from .git/COMMIT_MSG.

Add a simple regression test with good & bad commit messages, so
it is easier to verify that any changes made to the script will
continue to both detect errors, and pass valid commit messages.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I15cb3690560400a591598997424cf79dee3a039d
Reviewed-on: http://review.whamcloud.com/1688
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>

LU-506 FC15: ctl_name & strategy removed from ctl_table.

The 'ctl_name' & 'strategy' field removed from
'struct ctl_table'.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I5eaeb46234083433109a150ea80408c9b3f9962b
Reviewed-on: http://review.whamcloud.com/1332
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>

LU-506 FC15: move some headers to include/generated

In FC15 (2.6.40-4) the generated kernel headers have moved
from "include/linux" to "include/generated". Update configure
scripts and makefiles to include this new directory. In a
number of cases, Lustre code was including generated headers
directly, but this was not really needed.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Id2094d8318681aa5ea08b416dc764bcf03bd8595
Reviewed-on: http://review.whamcloud.com/1329
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>

LU-506 FC15: update shrinker to use shrink_control callback

Linux 3.0 memory pressure shrinker now takes "struct shrink_control" as
its argument instead of "nr_to_scan" and "gfp_mask". This was backported
to Fedora 15.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: Id9f6a9e10efe785d2837d1ad73098d2808a2f076
Reviewed-on: http://review.whamcloud.com/1331
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>

LU-857 security: Lustre client tolerates enforced SELinux.

Fix a bug which prevents Lustre clients to access directoriess when
SELinux is enforced, on RHEL 6.
This patch does not add a real SELinux support for Lustre but ables
to activate it for all other local filesystems, without Lustre
misbehaving.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: Ia6692c96a8439eb9239cb55ce32a1c54958241d1
Reviewed-on: http://review.whamcloud.com/1703
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-745 kernel: ost-pools test_23 hung

It could be caused by a jbd2 bug which result in forever sleep
in the do_get_write_access().

http://www.spinics.net/lists/linux-ext4/msg24689.html

In do_get_write_access() we wait on BH_Unshadow bit for buffer to get
from shadow state. The waking code in journal_commit_transaction() has
a bug because it does not issue a memory barrier after the buffer is moved
from the shadow state and before wake_up_bit() is called. Thus a waitqueue
check can happen before the buffer is actually moved from the shadow state
and waiting process may never be woken. Fix the problem by issuing proper
barrier.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I44dce352babc6699cdacc00263bfd3f24538400c
Reviewed-on: http://review.whamcloud.com/1675
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-624  Kernel update [RHEL6.1 2.6.32-131.17.1.el6]

Change-Id: I82ef82e11f846840707f9f65ca72bcda8885c4e0
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1632
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-737 utils: check device name for digit

We need to check the whole string for digit
not only the first character

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I316d931bb344d3e3fe5bb7d7a2454f200b637017
Reviewed-on: http://review.whamcloud.com/1641
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-759 mdc: Clear rq_replay on error in mdc_enqueue()

When mdc_enter_request() fails (e.g., due to signals) in mdc_enqueue(),
the request is freed without any care about its rq_replay field.  For
rq_replay requests, this results in assertion failures in
__ptlrpc_free_req().  This patch adds a call to mdc_clear_replay_flag()
to make sure __ptlrpc_free_req()'s assumption is respected.

Change-Id: I2185066a9f47b3d9563d9e1a8989754ef2e2dcb4
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1518
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-831 header: struct bit field should be unsigned type

Make sure struct bit fields be unsigned type, or else if they are read
from proc interface, they would showes as a big number equivalent to
-1.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ib252b4e89ce6b3f898e2da11a60de9aa9201119c
Reviewed-on: http://review.whamcloud.com/1685
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-835 build: skip generated files in .gitignore

Skip files automatically generated during builds on FC-13.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1cff9a351137145bb87a89460d0a1b54fae29e2c
Reviewed-on: http://review.whamcloud.com/1681
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-795 osd api: Commit callback per transaction

- ability to add commit callback per transaction in addition to
  per-device hooks. Now it is much simpler if only commit callback
  is needed.
- rewrite commit callbacks for last_commit and new_client, add commit
  callback in seq manager
- cleanup not-needed code: old commit callbacks, txn_keys
- remove osd od_env_for_commit environment and env param from commit
  callbacks
- use th_sync to mark sync operations

Change-Id: If5f8f2a6d3cd2f3e77fd13c802213a181043a2d7
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1621
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-691 Fix OST index errors in test suite - sanity 133c defect

Several test run do_facet ost which assumes ost is ost0 which does not exist
according to the way the test suite works. This patch address several areas
where the wrong ost index is used. This patch also introduces some short hand
functions to get OST properties from the index. Those functions ensure ost1,
ost10 and ost100 are seen as different which is not always the case.

Change-Id: Ic31224794563964a3415d24abeebce9dacceb686
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/1425
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Updated version to 2.1.52

Change-Id: I68a6934d997921a82670272cf30abdcc25d6a575
Signed-off-by: Oleg Drokin <green@whamcloud.com>

ORNL-3 mntopt: consider low-layer options for MDT ACL flags

Currently, MDT layer enables ACL support by default without checking
whether low-layer (ldiskfs) enables ACL support or not, then causes
unnecessary data traffic on network and through MDS stack for ACL.

So MDT should communicate with low-layer before setting ACL flags.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I804f10bf486745ddd3b23b89e959dfd585589ac0
Reviewed-on: http://review.whamcloud.com/1211
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-769 ptlrpc: Do not miss pending signals in ptlrpc_set wait

conf_sanity test 23a highlighted a problem in ptlrpc_set_wait logic,
if we enter there with a signal pending and the import is not FULL,
there is no way to interrupt such a set because we block signals
all the time. Enabling signals all the time is not an option either.
Waiting until import reconnects is questionable too since it might
never come up after all (like in the test 23a).
So for the solution we will just manually mark the set as interrupted
after the initial wait.

Change-Id: Iaa3e356e971b4f75fd7f21cc579c85f7487719a0
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1657
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>

LU-762 ldiskfs: don't drop directory nlink to 0

When landing the nlink patch for ext4, for an unknown reason the
logic in ext4_dec_count() was changed from the ext3 version of
the patch. It now drops the nlink = 0 temporarily and then
if it is a directory with nlink == 0 it increases nlink again.

Instead, only drop nlink if it is larger than 2.

Change-Id: Ieeff3e45daea56f502848f9c2b0fb04f0a9d2b6d
Author: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1644
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-780 test: improve parallel-scale to support hyperion run

We need to add support for srun/slurm, and a few tests
from hyperion-sanity script that has been used for hyperion
testing

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I7f1baa0c99980ad9001436911d23f1030aa7d0fe
Reviewed-on: http://review.whamcloud.com/1615
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Cliff White <cliffw@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-826 mdd: Fix for mdd_lov_create_finish memory free

At lov/lov_pack.c lov_packmd() use OBD_ALLOC_LARGE for memory
allocation for lov_mds_md object(lmpp), but mdd_lov_create_finish
use OBD_FREE to free memory. This bug doesn`t affect to current
version, but may be relevant in future.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: Ic37cb72022b9aac02368b11f370cbaad0c730e7c
Reviewed-on: http://review.whamcloud.com/1659
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-8 IR: minor fixes to IR patches

Summary of fixes:
1. ORNL-13: add comments to mgs_handle_fslog_hack();
2. ORNL-14: typo and accurate error/debug messages;
            minor fixes in recovery-small.sh;

Change-Id: I6317067eb6250faf5df21be719c82de17b0f4cc9
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1582
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-7 replace statahead hacking with its own cache

Original statahead hacks dcache, it inserts dentry into dcache and
associates with inode without holding parent's lock, which breaks
VFS layer synchronization mechanism.

The new statahead does not build dentry, it just pre-fetches inode's
attributes and related ldlm locks, caches them in a small statahead
cache against parent's inode. The statahead sponsor can search such
small cache similar as searching dcache to look for what it wants.
If cache hit, then builds dentry, associates with inode, and inserts
into dcache by the statahead sponsor itself.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I6948f8a438a938c51563468d775e676e4185e580
Reviewed-on: http://review.whamcloud.com/1208
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-774 test: wait till space from previous tests is released

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Idf4933f9b40c74b99bf763eb896680a4e0c942ff
Reviewed-on: http://review.whamcloud.com/1588
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-482 test-framework: Ensure dirty cache is flushed before barrier

With certain backend devices like LVM with older kernels the data
in dirty cache cannot be propagated all the way to the block device
with a single sync as there are multiple non-cooperating layers.

So convert such sync calls into triple syncs

Change-Id: If82e25223a277ec165d150b0f5f960ff845af9b0
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1656
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>

LU-773 tests: sanity:test_105b failure

The root cause of this issue is that activating osc in test sanity 104a
wasn't successful. We should wait for the recovery to finish.

Change-Id: I940419bfb1f579c4a0233b7439ac1f459ee584ad
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1542
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-601 mdd: Fix transaction credits

* mdd_create()/mdd_create_data() may need delete orphan objects on
  OSTs, so that we need preserve enough transaction credits for llog
  records.
* mdd_attr_set() may write lov llogs.
* orphan_object_destroy() also will write a llog record, we need
  reserve credit for it as well.
* add credit changelog record.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I5124d2f368e2ff794b2b2b8194bec86f63e971cf
Reviewed-on: http://review.whamcloud.com/1276
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-778 o2iblnd: Add rdma_create_id() compatibility macro

As of RHEL6.2 kernel 2.6.32-204.el6, rdma_create_id() requires a
queue-pair type as a fourth argument.  This was previously inferred
from the rdma_port_space argument.  Add an autoconf test to detect
whether the fourth argument is expected and a compatibility macro
that discards the QP type argument if the 3-argument version of
rdma_create_id() is present.

Change-Id: Idb668e1f059954ecc994ad59b366d54da8b82dc8
Signed-off-by: Ned Bass <bass6@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1556
Tested-by: Hudson
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-389 update wirecheck for master

Drop unused mds_body/mds_remote_perm definition and wirecheck.
Add missing wirecheck for mdt_xxx, ost_xxx, lu_xxx, and so on.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I858a02b0b5de8680cc8315e38aeeca271c26e9ad
Reviewed-on: http://review.whamcloud.com/985
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-26 prevent call md_set_lock_data() repeatly

md_set_lock_data() is called from different functions in llite, and
may be called more than once for the same <inode lock> pair. It is
harmless for the correctness, but will affect the performance a bit,
should be avoided.

Drop dentry flags of "DCACHE_LUSTRE_INVALID" only when we hold the
"MDS_INODELOCK_LOOKUP" lock.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I4dbe206af77ba6a619f3268c238ce98ac7aef4c0
Reviewed-on: http://review.whamcloud.com/1224
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-686 conf-sanity test 52 fails due to missing mount point

Create the mount point before mount

Change-Id: I6c813f5040e4636386eabb640526f3d1072f484b
Signed-off-by: HongChao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1379
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-620 llite: add delete_from_page_cache and remove_from_page_cache check

Later 2.6.32 kernel use memory cgroup feature but does not export
truncate_complete_page but export delete_from_page_cache or
remove_from_page_cache, we need properly use them for pachless client
code.

Change-Id: I33e3e7c32b548866ee77753ef8a8193c814d0ecb
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1399
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-000 libcfs: allow debug to be set to numeric -1

Don't warn if debug is set to "-1", which should always mean
"enable all debugging".

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1503d2f5776043e9a4ec3bc4af710ae35ca97960
Reviewed-on: http://review.whamcloud.com/1577
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-723 Enhance lustre/ldiskfs build system

Enhance the lustre/ldiskfs build system so it is more robust, flexible,
and consistent with lustre/zfs build system.  This change is being made
in the interest of standardizing the infra-structure around backend
filesystems.

This change does not effect the current behavior of the --with-ldiskfs,
--enable-ext4, or --with-ldiskfsprogs configure options.  However, it
does remove the obsolete --with-ldiskfs-inkernel configure option which
was only used by LLNL.  It also adds the --with-ldiskfs-obj configure
option which improves flexibility.  And the --enable-ldiskfs-build
configure option to support building against the lustre-ldiskfs-devel
package.  The behavior of these options is consistent with their zfs
counterparts, see commit 8c7266c for further details.

  --enable-ext4           enable ldiskfs build using ext4
  --enable-ldiskfs-build  enable ldiskfs configure/make
  --with-ldiskfs=path     set path to ldiskfs source
  --with-ldiskfs-obj=path set path to ldiskfs objects
  --with-ldiskfsprogs     use alternate names for ldiskfs-enabled e2fsprogs

Sample ./configure results when building lustre and ldiskfs using
the kernel-devel and kernel-debuginfo-common packages.

checking whether to enable ldiskfs... yes
checking ldiskfs source directory... /home/behlendo/src/git/lustre/ldiskfs
checking ldiskfs object directory... /home/behlendo/src/git/lustre/ldiskfs
checking ldiskfs module symbols... Module.symvers
checking ldiskfs source release... 3.3.0
checking whether to use ext3 or ext4 source... ext4
checking ext4 source directory...  /usr/src/debug/.../fs/ext4
checking whether to build ldiskfs... yes
checking for /home/behlendo/src/git/lustre/ldiskfs/configure... yes
checking for /usr/src/debug/.../fs/ext4/dir.c...  yes
checking for /usr/src/debug/.../fs/ext4/file.c...  yes
checking for /usr/src/debug/.../fs/ext4/inode.c...  yes
checking for /usr/src/debug/.../fs/ext4/super.c...  yes
checking if ext4_ext_walk_space() takes i_data_sem... yes
checking if LDISKFS_SINGLEDATA_TRANS_BLOCKS takes sb as argument... yes
checking if ldiskfs_discard_preallocations defined... yes
checking if ldiskfs_ext_insert_extent needs 5 arguments... yes

In the context of this change additional cleanup has been done and
all of the ldiskfs specific code relocated to lustre-build-ldiskfs.m4.

Note that this change moves us closer to supporting patchless Lustre
servers with ldiskfs.  Once the remaining kernel patches for Lustre
are dropped you will be able to build Lustre using the distribution
provided kernel-devel and kernel-debuginfo-common packages.

This change also incorperates ORI-340, commit f604951, which ensures
that the Module.symvers file will cleanly include the symbols for all
enabled Lustre backends.  While the only backend supported by master
right now is ldiskfs this brings the master and orion branchs in to
sync in this regard.

Change-Id: I6f13f266944ec6967f4d7705a30b83ab8e577b15
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1566
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

ORNL-22 ptlrpc:  more comment for multi-threaded ptlrpcd

To explain share work load between ptlrpcd partners.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I72711a93af321e43e6dbbbc52b427060be47f808
Reviewed-on: http://review.whamcloud.com/1638
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-362 Fix error path in lu_kmem_init().

When echo client is failed to set up there are memory leaks were
noticed, it is related to missed error handling in some functions.

- Free all caches if lu_kmem_init() failed
- fix error handling in cl_global_init() and ccc_global_init()

Change-Id: Ide9e7ad6d40f99a7fbb4330ba63b168cc408356f
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/586
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-462 Don't alloc/free client data for self export

Self export doesn't need client data and ldlm initialization.
Patch uses uuid comparision to determine self_export.

Change-Id: Id26ef90e9857e4c1d3a0e7a3756eaf67607890d6
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1574
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>