LU-4219 mdd: limit os_namelen to the max of NAME_MAX

While running pathconf(".", _PC_NAME_MAX) on ldiskfs-based
Lustre filesystem, it returns 255 (NAME_MAX). However, on
zfs-based Lustre filesystem, it returns 256 (MAXNAMELEN).
Since Lustre can not handle the name length large than
NAME_MAX, this patch limits the value of os_namelen in
mdd_statfs() to the max of NAME_MAX.

Test-Parameters: envdefinitions=SLOW=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
ostcount=2 testlist=posix

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I13b5398eff5b15c74b0e08e2a3394b79a3ff6a44
Reviewed-on: http://review.whamcloud.com/8217
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4093 tests: prevent zombie requests when stopping CT

Fix to prevent any current requests to become zombie when stopping
copytool and thus not to sometimes to induce failures during next
sanity-hsm sub-tests.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I3274e5864469251e6f2040377b84429e4f218f47
Reviewed-on: http://review.whamcloud.com/8157
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4181 tests: disable lnet_selftest.sh for DNE.

Disable lnet_selftest for DNE to make review pass with DNE.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie4abd794d0184955af84752f0a9c64181b85f1ec
Reviewed-on: http://review.whamcloud.com/8130
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

New tag 2.5.51

Change-Id: Ib4389ce656c878704b329472ab3a848e1aff6b53

LU-4149 tests: disable sanity_scrub(7, 15) for dne test.

Disable sanity_scrub(7, 15) for dne test, so make dne pass
on maloon right now.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie6553f44655c8f38451ed2cb1086e5848ca033a4
Reviewed-on: http://review.whamcloud.com/8101
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-20 kernel: remove obsolete kernel patches

The jbd2-jcberr and lustre_version patches have been obsoleted since
Lustre 2.2.  They were kept around to allow transitioning between
Lustre 1.8/2.1 and 2.4/2.5 but can be removed from Lustre 2.6 since
this does not allow a direct upgrade path anyway.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I5406272fe22580e48679c18dea562001d23ebbe5
Reviewed-on: http://review.whamcloud.com/7881
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3920 tests: check MDS version before testing HSM feature

Sanity tests 65y and 229 fail when MDS does not have HSM
support. Check MDS version and skip the tests in that case.

Xyratex-bug-id: MRP-1417
Signed-off-by: Vladimir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I80496ac67ec1b6b8032f465a0b5914bb12839d3b
Reviewed-on: http://review.whamcloud.com/8121
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>

LU-3335 osd: self repair LMA by the RPC service thread

Current OI scrub uses the FID in LMA to locate the OI mapping and
repair the inconsistent OI mapping. Such inode table based system
scanning does not guarantee all OI inconsistency can be repaired,
such as inode without LMA or with crashed LMA, or cannot find out
dummy OI mappings (the object for the OI mapping does not exist).

In fact, after the inode table based scanning, the OI scrub can
make another OI table based scanning to resolve above mentioned
issues. But such scanning will be nonlinear, so performance may
be bad.

Currently, we have some other relative simple solution although
they are not perfect. We can make the RPC service thread to fix
the found inconsistency by itself: when verify the LMA with the
given FID, and found no LMA, then generate LMA according to the
given FID and store the new LMA as the inode xattr. Usually, it
is rare case that the inode has no LMA. So such additional work
for the RPC service thread will not affect performance much.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic6b8b533e56b6a0e388d5c38de7b113cb9bd6dad
Reviewed-on: http://review.whamcloud.com/7144
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-3973 tests: use -P option of df to avoid line breaks

Sometimes df command output requires to break lines to
allow for columns alignement. This causes fields parsing
to fail. Use -P option to select compatibility output
and ensure one single line format per file-system is used.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ibc9d4b8f50857e680a5590dbbfe312fce137bcb6
Reviewed-on: http://review.whamcloud.com/7915
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-2479 ldiskfs: do not check dir max size for regular files

ldiskfs_append() is used not only to extend directory but also to
maintain iam container. In later case ldiskfs_append() should not
check for max directory size. The iam container is distunguished as
regular file.

Signed-off-by: Vladimir Saveliev <vladimir_saveliev@xyratex.com>
Change-Id: I205b0b1149d7bd477b3965ce68a546dfd9dc6a6f
Reviewed-on: http://review.whamcloud.com/8137
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2528 misc: quiet warning for 2.1/2.5 connections

The Lustre 2.5.4 client will print a warning about connections with
2.1.3 servers, yet they are still supposed to be interoperable.
Increase the window of warning to be up to Lustre 2.5.50, since we
do not intend to allow interoperability between 2.1 and 2.6 systems.
This was from the time when major releases like Lustre 1.4 and 1.8
were many years apart would have no chance for interoperation.

Only print this message once per client, to avoid flooding the console
for connections to many servers or frequent network reconnections.
Server versions should all be nearly the same in any case.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I503523f80144c00c5ea1d312cc9204d9123ebbe5
Reviewed-on: http://review.whamcloud.com/7916
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3858 test: fix sanity 27A failure caused by stripe delay

When clearing the stripe of root directory, the updating of stripe
might be delayed because of the processing delay of config log.
This will cause the failure of sanity 27A test.

This makes the test suit wait for the update of root directory
stripe.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Ib4f96d9d4d0ae8a0b54e3e306a7abed758dbdf39
Reviewed-on: http://review.whamcloud.com/7495
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2613 recovery: free open/close request promptly

- For the non-create open or committed open, the open request
  should be freed along with the close request as soon as the
  close done, despite that the transno of open/close is
  greater than the last committed transno known by client or not.

- Move the committed open request into another dedicated list,
  that will avoid scanning a huge replay list on receiving each
  reply (when there are many open files).

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I1a25a35fe7a16681368d92d16964680b6209b3ee
Reviewed-on: http://review.whamcloud.com/6665
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3968 lbuild: Extend script with build for Xeon Phi card

Automatically download, compile and produce Lustre client RPMs
for Xeon Phi(TM) card if "--mpss-version" option is specified
for contrib/lbuild/lbuild script.

Also try to compile with Xeon Phi(TM) OFED if it's available.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ida07d764dc824c13f22ffb53d24e2c6f79ce3573
Reviewed-on: http://review.whamcloud.com/7066
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4016 tests: use user and group names in metadata-updates

pdsh eats %u passed to stat executed on remote nodes.  Avoid this
problem by using the user and group names instead of numerical
identifiers in metadata-updates ownership checks.

Test-Parameters: testlist=metadata-updates
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Stephen Champion <schamp@sgi.com>
Change-Id: Id0a283d58873bb64772f28cbc28c5a6abad9a48e
Reviewed-on: http://review.whamcloud.com/8052
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2887 tests: disable SLOW for ZFS in parallel-scale

Because all zfs pools are on the same physical device in autotest
ZFS backed OSTs run VERY slowly so disable SLOW.  Decrease load on zfs
backed MDS.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs testlist=parallel-scale
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I99d980adcf4259f8431c351ce6fecb1088419130
Reviewed-on: http://review.whamcloud.com/7778
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4060 test: skip sanityn.sh test_74 on older than 2.4.93 MDS

2.5 client 2.4 server interop testing shows that sanityn.sh test_74
needs to be skipped for Lustre MDS versions older than 2.4.93 .

Test-Parameters: testlist=sanityn clientdistro=el6 clientarch=x86_64 serverdistro=el6 serverarch=x86_64 serverjob=lustre-b2_4 serverbuildno=46
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I92d29a8db3ed5d145bd7db7bd18f5593ba80576c
Reviewed-on: http://review.whamcloud.com/7911
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3321 osc: Adjustment on osc LRU for performance

Add and discard pages from LRU in batch.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I90287b73c05066d420a31bff21866dfa1ffec665
Reviewed-on: http://review.whamcloud.com/7890
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3321 clio: revert LU-2622 for removing global env list

Revert "LU-2622 obdclass: Remove the global cl_env list"

This reverts commit d5390bbde9b525b13ab91f5610abe1c4bb98eff9.

The performance dropped a lot when memory reclaim process kicked
in as ll_releasepage() was called to destroy lustre pages. It turned
out that big overhead to allocate cl_env and keys on the fly so we
have to revert this patch.

The problem of LU-2622 will be solved in another patch later on.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: If5f9faeabbe78d3a88b25bf986a2c9a2ecc9ba12
Reviewed-on: http://review.whamcloud.com/7888
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4156 wireshark: Fix build against wireshark 1.10.x

Some of the API changes that were depricated in 1.6 and 1.8 have been
removed in 1.10.  This patch uses the new API calls (these work in 1.6
and later).

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I83a469b973007a58d81b2460eec9da418c172b4a
Reviewed-on: http://review.whamcloud.com/8079
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Frank Heckes <Frank.Heckes@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3971 hsm: Copytool code cleanup

Minor refactoring of the bandwidth controling code.
Deletion of a superfluous select() call on regular files.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: Iae550bb69c1524865b38a92d9b7674fce2f58258
Reviewed-on: http://review.whamcloud.com/7583
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4118 kernel: kernel update [RHEL6.4 2.6.32-358.23.2.el6]

Update RHEL6.4 kernel to 2.6.32-358.23.2.el6.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: Icfe1416916f98c165ec6517d7f7a7a28b0167e4a
Reviewed-on: http://review.whamcloud.com/8053
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3850 obdecho: create remote dir from echo client

Because echo client can only talk to local MDD, it will
send create remote dir req to the MDT where the dir is
located, which is different as the normal remote create
request. This patch fixes a few problem of this process

1. Not set parent during ah_init, which will make local OSD
to access the remote parent.

2. set correct attribute for remote parent in the declare
phase.

3. set root_fid for other MDT, which is needed by echo_client
to resolve the path on non-MDT0.

4. set child_base_id in oi_id directly to avoid the sanity
check in ostid_set_id,  and also child_base_id is actually
being used as the base of name, instead of FID.

5. Disable rmdir on remote MDT for echo client, i.e. echo
client can only unlink the directory on the MDT which echo
client is attached to, so lctl test_rmdir will be consistent
with normal rmdir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I16df8f0211afae670bfeea1ba1087a77a6b775da
Reviewed-on: http://review.whamcloud.com/7502
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2139 osc: Use SOFT_SYNC to urge server commit

This change adds a BRW page flag, OBD_BRW_SOFT_SYNC. This flag is
intended to urge a server to commit a client's unstable pages to
stable storage. A client will add this flag to any BRW requests while
it is in a state where it has "many" unstable pages pinned in its cache.

The server, upon receiving a page with this flag set, *should* begin
an async commit. The idea being that, with the proactive async commit,
the client's pinned unstable pages will transition into a stable state
faster than they would have otherwise. Although, the server side portion
of this agreement is still unimplemented, so the OBD_BRW_SOFT_SYNC flag
will currently fall on deaf ears.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I08946a5ec6afab390655643f6d3c8320bf010aea
Reviewed-on: http://review.whamcloud.com/4375
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2139 osc: Track number of "unstable" pages per osc

This change adds simple accounting hooks for "unstable" pages on a per
OSC basis. Now, in addition to the per filesystem tracking, each OSC
will maintain a running total of its unstable pages. These counters are
exported through the proc interface, and can be read using the lctl
command.

For example:

    # Read number of unstable pages contained by each OSC
    lctl get_param osc.*.unstable_stats

The motivation for this change is in anticipation of implementing a
"soft sync" functionality, urging servers to commit these unstable
pages to stable storage. The per OSC accounting allows a client to
limit the soft sync request to only the OSCs which have outstanding
unstable pages.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Ife1f90515cb60ba9f85daba5d50483562f2697d1
Reviewed-on: http://review.whamcloud.com/4374
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2139 osc: Track and limit "unstable" pages

This change adds a global counter to track the number of "unstable"
pages held by a given client, along with per file system counters. An
"unstable" page is defined as a page which has been sent to the server
as part of a bulk request, but is uncommitted to stable storage.

In addition to simply tracking the unstable pages, they now also count
towards the maximum number of "pinned" pages on the system at any given
time. Thus, a client will now be bound on the number of dirty and
unstable pages it can pin in memory. Previously only dirty pages were
accounted for in this limit.

In addition to tracking the number of unstable pages in Lustre, the
NR_UNSTABLE_NFS memory zone is also incremented and decremented for
easy monitoring using the "NFS_Unstable:" field in /proc/meminfo.
This field is also used internally by the kernel to limit the total
amount of unstable pages on the system.

The motivation for this change is twofold. First, the client must not
allow itself to disconnect from an OST while still holding unstable
pages. Otherwise, these unstable pages can get lost due to an OST
failure, and replay is not possible due to the disconnect via unmount.

Secondly, the client needs a mechanism to prevent it from allocating too
much of its available RAM to unreclaimable pages pinned by the ptlrpc
layer. If this case occurs, out of memory events can trigger as a side
effect, which we need to avoid.

The current number of unstable pages accounted for on a per file system
granularity is exported by the unstable_stats proc file, contained under
each file system's llite namespace. An example of retrieving this
information is below:

$ lctl get_param llite.*.unstable_stats

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I67ab9ee244e6d6081069e3c3d5b2e493ddb97874
Reviewed-on: http://review.whamcloud.com/6284
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4107 build: fix lustre_user.h to C++ compatible

Fix anonymous union.
Fix warnings
Workaround offsetof due to a bug in g++
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=14932

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I4c1b8165ab012f4226aabb60009b48063669e271
Reviewed-on: http://review.whamcloud.com/8033
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3985 tests: Archive access must be done from an agent

In sanity-hsm, all access to the archive backend must be
done from an agent. Tests 111a and 111b call the
copytool_remove_backend routine that calls find on the
archive backend. With this patch, the call to find is
done on the agent with a call to "do_facet $SINGLEAGT".

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I537fe5e019861908612140b3c96ea2018a738e46
Reviewed-on: http://review.whamcloud.com/8076
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4055 build: fix 'error handling' issues

Fix 'error handling' issues found by Coverity version 6.6.1:
Unchecked return value (CHECKED_RETURN)
Calling function without checking return value.
Argument cannot be negative (NEGATIVE_RETURNS)
Negative value used as argument to a function expecting a
positive value.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I1ea3428febee741be15215fcc89266a814e5fe1b
Reviewed-on: http://review.whamcloud.com/7842
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4012 gss: upcall fails due to removed generic cache calls

With Linux kernel version >= 3.4.X the generic cache registering
routines cache_register(), cache_unregister() are removed.
This effects the GSS upcall routines for the Kerberos
support in Lustre. Before (Linux kernel <= 3.2.X) the
generic routines were coded as:
int cache_register(struct cache_detail *cd)
{
        return cache_register_net(cd, &init_net);
}
void cache_unregister(struct cache_detail *cd)
{
        cache_unregister_net(cd, &init_net);
}
To keep compatibility with former kernel versions,
cache_register_net() and cache_unregister_net() can
be applied instead, rather than the old and
removed cache_register() and cache_unregister().

Signed-off-by: Thomas Stibor <thomas@stibor.net>
Change-Id: I3eb012a1883c463ecafe3646e7e706ab96d91c5c
Reviewed-on: http://review.whamcloud.com/7770
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Korty <ajk@iu.edu>

LU-3843 clio: add debug message in osc_completion()

Replace LASSERT with LASSERTF in osc_completion, thus we can get
more info when the LASSERT is triggered.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I1b365050d5c2348b4cdc83f2ea71f36cce7f9456
Reviewed-on: http://review.whamcloud.com/7494
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3491 llite: Replace printing of i_ino with ll_inode2fid()

The printing of i_ino/i_generation in llite messages is not nearly so
useful as printing the full inode FID, since i_ino is a "compressed"
version of the FID and there may be duplicate values for i_ino in some
cases (especially if running on a 32-bit client).

All instances of printing i_ino/i_generation are replaced with
the FID using ll_inode2fid(). All instances, except for one, of
printing just i_ino was replaced by printing the FID. In all
CERROR lines touched by the i_ino replacements, the device name
or fsname was added at the beginning of the message if it did not
already exist.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ia32af1357729bf1951e0b713e75986c8177dd946
Reviewed-on: http://review.whamcloud.com/6848
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-1199 rpm: use %defattr instead of %attr for every file

Every file in the spec file is installed with %attr(-,root,root)
explicitly. Use %defattr once per %files section instead.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia37757db39a6ee81fb0acd910d7d3204f3c194b1
Reviewed-on: http://review.whamcloud.com/5486
Tested-by: Jenkins
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3570 libcfs: accelerate crc32c with pclmulqdq

Using hardware provided PCLMULQDQ instruction to accelerate
CRC32C check sum. This instruction is present starting with
Intel Westmere and AMD Bulldozer CPUs.

Also reorganize assembler code of other PCLMUL inmplementation
with using common assembler macros from inst.h.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I2099ff8af3591b3e2267cb30486d7a4f0a3e90c0
Reviewed-on: http://review.whamcloud.com/6927
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4007 libcfs: fix for removing lcw at lcw_dispatch_main

lcw_dispatch_main() creates zombies list and moves lcw into it.
But removes entry from another list in the zombies loop.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Xyratex-bug-id: MRP-1179
Change-Id: I9daec3d9b3dbbf244a28b829cd724c8ec60c4b6c
Reviewed-on: http://review.whamcloud.com/7755
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>

LU-4178 tests: disable HSM sanity subtests

Turn off high failure rate subtests in sanity-hsm.
If fixes for these failures land tests may be turned on again later.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iaeedd2946f33075242caad851c7f473d86cbcb04
Reviewed-on: http://review.whamcloud.com/8122
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4135 mdt: restore mdt_save_lock() functionality

Missed updates for mti_has_trans cause mdt_save_lock() to do
just decref for lock and never try to save it as intended.

Patch restores updating mti_has_trans variable and fix issue
with NULL req in case of mdt_export_cleanup() codepath

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I2e4ccae018635842df1e02d72a639b0ca1fbdb04
Reviewed-on: http://review.whamcloud.com/8048
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1458 tests: wait for dbench to start

Use wait_for_function instead of just guessing that 20s will be long
enough for dbench to start.  Wait for up to 6min in 5 second
intervals.  This also ensures that dbench will run for a little while
after it starts.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ic9c61b487c4013cccd620491d617c10e2206fd62
Reviewed-on: http://review.whamcloud.com/7914
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2842 mdt: Do not lbug on invalid fid entry in name

Instead of crashing MDS when the name exists but has invalid fid,
we should just return error and let lfsck deal with it later.

Change-Id: Iea19dac75e9dc75c43a18efa757fc1c5e4d1743e
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/7617
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-2906 ldlm: use accessor macros for l_flags

Convert most all of the ldlm lock's l_flags references from direct
bit twiddling to using bit specific macros.  A few multi-bit
operations are left as an exercise for the reader.

The changes are mostly in ldlm, but also in llite, osc and quota, but
also fix a typo in the LDLM_{SET,CLEAR}_FLAG macros and:

Add a multi-bit (mask) test.
Remove the now obsolete LDLM_AST_FLAGS and LDLM_INHERIT_FLAGS defines.
Remove the obsolete LDLM_FL_HIDE_LOCK_MASK define.
Rename "local_only" mask to "off_wire" since it is confusingly similar
to a flag that (I think) means, "do not copy this lock over the wire."
The "local_only/off_wire" mask is also never used.  It's mostly
informational.

Wireshark output moved to "lustre_dlm_flags_wshark.c" and only
bits that can actually appear "on the wire" are emitted.
The "packet-lustre.c" code that references these bits now gets
emitted into that file.  e.g. the "local_only" bit is never put
on the wire, so references to it in wireshark are gone.

Signed-off-by: Bruce Korb <bruce.korb@gmail.com>
Reviewed-by: Keith Mannthey <Keith.Mannthey@intel.com>
Change-Id: I2527c46835e434f1009cf83919a203a358b04737
Reviewed-on: http://review.whamcloud.com/7963
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3679 lnet: reflect down routes in /proc/sys/lnet/routes

We consider routes "down" if the router is down or the router
NI for the target network is down. This should be reflected
in the output of /proc/sys/lnet/routes

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I82ee769d88aec92f1690ad9c095e32c9a9f9e282
Reviewed-on: http://review.whamcloud.com/7857
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3485 tests: add resize2fs testing for MDT and OST filesystems

This patch adds a test case into conf-sanity.sh to test running
resize2fs on MDT and OST filesystems. Currently, only offline
resizing is possible until LU-137 is finished. Before and after
resize2fs is used, e2fsck is run to verify that no corruption
is introduced during the process.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I0957cba7fdd76276331c40b746cd87efac10ecc4
Reviewed-on: http://review.whamcloud.com/7796
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 obdclass: build and install llog_test.ko properly

There's a hack in lustre.spec.in to copy the llog_test.ko module into
/lib/modules.

It's only needed because that module is built unconditionally and
is expected to only be installed if --enable-tests is configured.

The right way to do this is to only build it at all if --enable-tests
is configured and installation comes automatically,

The conditional in the spec file for --enable-tests already exists,
so it will be packaged correctly without the hack.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I2705848e11a96ab6769ebba4481d2d0621f53d3e
Reviewed-on: http://review.whamcloud.com/5493
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Stephen Champion <schamp@sgi.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2800 autoconf: remove LIBCFS_SOCK_MAP_FD_2ARG

v2.6.27 added a second argument to sock_map_fd. Earlier versions are
out of scope so we can remove the test for the number of arguments.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: I4e12da8c102b5d682db9658187330ae601b00a21
Reviewed-on: http://review.whamcloud.com/5406
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4100 utils: Remove extra space of lfs getstripe

'lfs getstripe $dir' command prints an extra space at the end of
the output line, which is particularly annoying for parsing in
scripts.

This patch fixes this problem.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Iee50939f2b93bbd8d1c506bb90bc761a9f4dbeaf
Reviewed-on: http://review.whamcloud.com/7992
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4113 gss: uncatched error in gss_svc_upcall causes drops

Since kernel version 2.6.20 the function cache_check() in
net/sunrpc/cache.c can return the error -ETIMEDOUT. This
error should cause to recheck the cache. In current
implementation gss_svc_upcall_handle_init() is not
handling the error which causes to fall through the
switch statement returning GOTO(out, rc = SECSVC_DROP).
As a consequence this error is propagating to
ERROR:do_nego_rpc(): status: -110 (Unknown error 18446744073709551506)
ERROR:lgssc_negotiation(): negotiation rpc error: -85
ERROR:lgssc_kr_negotiate(): key 08786b66: failed to negotiation
and causes client hanging after switching to flavor krb5i/krb5p.

Signed-off-by: Thomas Stibor <thomas@stibor.net>
Change-Id: I7301bea70d96193e0f4a332c6e4b34ba326134ff
Reviewed-on: http://review.whamcloud.com/7960
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andrew Korty <ajk@iu.edu>
Reviewed-by: Ken Hornstein <kenh@cmf.nrl.navy.mil>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3373 tests: small fixes for sanity tests

--test_24d fix typo
--test_24A cleanup work directory in advance
--test_27k make up message more clear
--test-framework enforce LANG to en_US

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I61c227028b44a198482b4e5c9a9f715af08c3e5f
Reviewed-on: http://review.whamcloud.com/7948
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4098 lmv: kernel crash due to misconfigured MDT

There are few places with access to lmv->tgts[] without check for NULL.
Usually it may happens when MDT configured starting from index 1
instead of 0. For example:
  mkfs.lustre --reformat --mgs --mdt --index=1 /dev/sdd1

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I7d9bc8876bb0b2c2669050904d4629069b61e639
Reviewed-on: http://review.whamcloud.com/7941
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 procfs: remove unsed proc entry for target handler

With 3.10 and above kernels we end up using a new data
structure for proc handling. In the target handler code
we have an unsed lprocfs_vars so we remove it to avoid
compile issues.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I22d3d3b1aed852df93e448d3b793884344d1719a
Reviewed-on: http://review.whamcloud.com/7929
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4067 osd: apply PTR_ERR() to right pointer

In ofd_precreate_objects(), osd_index_ea_insert(), and
osd_object_create() ensure that PTR_ERR() is applied to the right
pointer. Also convert some explicit casts of ERR_PTRs to ERR_CAST()
calls.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I846b10543ae426652ad72c277d2c0f0bc259fd6e
Reviewed-on: http://review.whamcloud.com/7887
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4078 mdc: fix bad ERR_PTR usage in mdc_locks.c

In mdc_intent_open_pack() return an ERR_PTR() rather than NULL when
ldlm_prep_enqueue_req() fails. In mdc_intent_getattr_async() check the
return value of mdc_intent_getattr_pack() using IS_ERR(). Clean up the
includes in mdc_locks.c.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I89348389ccd102d1226caf16b315ac9b2a96b307
Reviewed-on: http://review.whamcloud.com/7886
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-4064 hsm: create files to be imported on agent node

In sanity-hsm test 13 create the files to be imported on the agent
node rather than the local node.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3f50cc5269512d3b99644119ea18272d3a80f8a7
Reviewed-on: http://review.whamcloud.com/7885
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4071 build: Ignore semicolon for Intel compiler

The build fails when if gcc is installed, but Intel compiler
is defined for mpicc. It does ignore semicolon for this case.

Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Change-Id: Ia3306b32c8d886f7a3123ff3c6c967343f0c26b0
Reviewed-on: http://review.whamcloud.com/7864
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-4070 build: RPM build errors

Move %{_bindir}/mcreate and %{_bindir}/munlink files under
condition %{with lustre_tests}.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I888086f672f690d9e3518f16edbf090edc8d0f14
Reviewed-on: http://review.whamcloud.com/7860
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-4066 test: use $LUSTRE to locate server modules

In test-framework.sh's init_test_env look for $LUSTRE/mdt/mdt.ko
rather than `$dirname $0`/../mdt/mdt.ko to determine if the server
modules are present when they cannot be found in /lib/modules.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib16eee4d7811afcb310c7b0fd4a401ea5af2e7ad
Reviewed-on: http://review.whamcloud.com/7854
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4052 build: simplify autogen.sh

The concept of conditional subdirectories dates back to the days
when we had Lustre spread across multiple CVS repositories.  For
years now we have used git, and the tree is entirely available
at all times.

Therefore we can simplify autogen.sh a bit more.

Change-Id: I3d7c8e5e93b57c492bb61554a546ceb6d90b2ffe
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/7840
Tested-by: Hudson
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4046 build: fix 'uninitialized variables' errors

Fix 'uninitialized variables' defects found by Coverity version 6.5.1:
Uninitialized scalar variable (UNINIT)
Using uninitialized value.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I4b5a3293e7d01436a82b6f7cea50b84c60660b3a
Reviewed-on: http://review.whamcloud.com/7821
Tested-by: Hudson
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4036 autoconf: don't check ext4 src when disable server

We should not check for ext4 src to determine if we want
to build server codes when user explicitly use --disable-server

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I04b45cf0c0837a7b8760b7ff46645cabfc2c6ebd
Reviewed-on: http://review.whamcloud.com/7818
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3880 tests: Allow error_ignore to accept any bug number

error_ignore is called when a test and an error is encountered
that is supposed to be ignored. The first argument to
error_ignore is a bug number, justification as to why the error
should be ignored, that the routine assumes is a bugzilla bug
number and automatically attaches a “bz” prefix to the number.
This patch removes the assumption that all bug numbers are bugzilla
numbers and moves adding the “bz” to the bug number to the
individual calls to error_ignore. Now, we can use “LU” bug
numbers with error ignore.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I40c47ccf6af76e76bff99c67816e0d2aebd37e71
Reviewed-on: http://review.whamcloud.com/7791
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-3764 tests: Make calls to error_ignore not trigger FAIL

Currently, error_ignore calls error_noexit with a status of IGNORE
and a message/comment. error_noexit writes the comment to
the $LOGDIR/err file. If there is an err file, the pass
routine assumes the test failed and reports the test status as FAIL.
So, for errors that should be ignored, we don't want to be
writing the comment out to the err file.

Now, any message with an "IGNORE" status is written to
$LOGDIR/ignore file and, thus, the test will no longer be
classified as FAIL and the error will really be ignored.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I30f5387105f6cbce54050682b16387f73ab541d6
Reviewed-on: http://review.whamcloud.com/7782
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3996 mgs: Don't close llog that we failed to open

Attempting to do so leads to a crash:
LustreError: 8136:0:(llog_osd.c:241:llog_osd_read_header()) MGS-osd: error reading log header from [0xa:0xa:0x0]: rc = -14
LustreError: 8136:0:(mgs_llog.c:1386:record_start_log()) MGS: can't start log lustre-params: rc = -14
BUG: unable to handle kernel NULL pointer dereference at 00000000000000b8
IP: [<ffffffffa07f9e99>] llog_handle_put+0x9/0x70 [obdclass]
PGD 8120c8067 PUD 7b8c65067 PMD 0
Oops: 0000 [#1] SMP
...
Call Trace:
 [<ffffffffa07f9f5d>] llog_close+0x5d/0x190 [obdclass]
 [<ffffffffa0e98e8d>] mgs_write_log_direct_all+0x28d/0x7a0 [mgs]
 [<ffffffffa06c8951>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
 [<ffffffffa0e99d3d>] mgs_write_log_param+0x99d/0x2980 [mgs]
 [<ffffffff81167610>] ? cache_alloc_refill+0x1c0/0x240
 [<ffffffffa0ea00b0>] mgs_setparam+0xde0/0x11a0 [mgs]
 [<ffffffffa0e8c7e2>] mgs_iocontrol+0x10c2/0x16e0 [mgs]
 [<ffffffffa07f70a7>] ? obd_ioctl_getdata+0x477/0x1120 [obdclass]
 [<ffffffffa080e344>] class_handle_ioctl+0x12e4/0x1e70 [obdclass]
 [<ffffffffa06c8951>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
 [<ffffffffa07f62ab>] obd_class_ioctl+0x4b/0x190 [obdclass]
 [<ffffffff81195352>] vfs_ioctl+0x22/0xa0
 [<ffffffff811954f4>] do_vfs_ioctl+0x84/0x580
 [<ffffffff81195a71>] sys_ioctl+0x81/0xa0
 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b

Change-Id: I2bd72c130a0530cc9d218556b5b6cc9c14efc2ea
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/7742
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2800 autoconf: remove LIBCFS_CPUMASK_SIZE

The function cpumask_size has been in the linux kernel
since 2.6.28. Also HAVE_CPUMASK_SIZE is not even used
anywhere in the source tree. We can remove this obsolete
test.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ie6090055105dcaa55fc031bf52e5a25cd7bcf321
Reviewed-on: http://review.whamcloud.com/7693
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3055 libcfs: remove page_collection::pc_lock

page_collection::pc_lock is supposed to protect race between
functions called by smp_call_function(), however we don't have
this use-case for ages and page_collection only lives in stack
of thread, so it is safe to remove it.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I89ca46e6593838d85fc0b91d1ff7c35cc28cf4ce
Reviewed-on: http://review.whamcloud.com/7660
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3616 mdt: deny write on executed file during restore

After commit c42b426c87c3d3b1dc9eda612cc831293dc80d68, from Gerrit
patch/Change-Id Ic8f82ddc9a56206307c2e5be2523fb7ce42b8638
(at http://review.whamcloud.com/3035) for LU-1338 (now HSM-5), it
is allowed to write in a file being executed during it is restored.
This patch revert to expected behavior (ETXTBSY).
Also added test_30c in sanity-hsm to verify this behavior.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Id1e2f624dd02c2bec18ffe6cb1ae9198c1cfd23a
Reviewed-on: http://review.whamcloud.com/7636
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3936 ldlm: ldlm_cancel_stale_locks()) ASSERTION( count > 0 ) failed

number of granted locks can be really huge.
Use 64bit math to calculate amount of locks to cancel

Change-Id: Iac0b52dd578baf9955b50597d0832ac8715895f1
Xyratex-bug-id: MRP-1296
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-by: Alexey Lyashkov <Alexey_Lyashkov@xyratex.com>
Reviewed-by: Alexander Boyko <Alexander_Boyko@xyratex.com>
Reviewed-by: Vitaly Fertman <Vitaly_Fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/7626
Tested-by: Hudson
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3806 obdclass: add LCT_SERVER_SESSION for server session

Add LCT_SERVER_SESSION for server session, and separate the
server session flag from LCT_SESSION, so to avoid allocating
session info for client stack for each server request, if
client and server are on the same node.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I808c3f58cd7a03ebc166e51fe1e32ea34ae0e3e8
Reviewed-on: http://review.whamcloud.com/7412
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3692 api: More flexible logging API.

Allow applications to register their own logging callback instead of
writing every log message to stderr/stdout. Registered callbacks are
global to the process, just like the log level.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I5ccfd1e24f1786408ce67b2cd87d53443bcd028a
Reviewed-on: http://review.whamcloud.com/7572
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3624 iokit: fix mds-survey for multiple MDT

The mds-survey script has several issues when called with multiple
MDT targets. This patch fixes the issues.

- make echo_client devices attached to different MDTs work in
  different directories
- correctly compute global statistics from each MDT stats
- fix mds-survey output so that number of field displayed is
  constant. This allows parsing the output for future plotting.
- fix lctl test_xxx commands to correctly handle specified
  parent directory

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I3ef0b82b59124f2246db535b074be94c7235936a
Reviewed-on: http://review.whamcloud.com/7558
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3790 tests: test_pios_ssf failed with 24

Wrong param to get free space in space_check () function

Signed-off-by: Konstantin Sandalov <konstantin_sandalov@xyratex.com>
Change-Id: I34abf6d63f081a515aaf95248bf5c4fc0896e024
Reviewed-on: http://review.whamcloud.com/7397
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3815 tests: HSM sanity test suites

Reevaluate the exception list.
Enable all sanity-hsm tests except #34, #35, #36 and #40.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I66dbf6f5119fc75b379752a33a8d5e44562ff985
Reviewed-on: http://review.whamcloud.com/7374
Tested-by: Hudson
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3963 lnet: convert cfs_atomic primitives

This patch convers all cfs_atomic primitives in lnet/
directory.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I81426bd7a8a1a97ea5f198d99193649fbbf5389a
Reviewed-on: http://review.whamcloud.com/7070
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1538 tests: clean up runtests code style

Use $(...) instead of `...` for sub-shells.
Improve some error messages and code style.
Use $DIR/$tdir for tests instead of $MOUNT/$$.

Test-Parameters: envdefinitions=SLOW=yes  testlist=runtests
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I4f3eb4a3e288c6fc45ae6ef1fb2008e8e6babe84
Reviewed-on: http://review.whamcloud.com/5739
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2158 lvfs: remove the fsfilt and lvfs layer

The fsfilt/lvfs layer has been obsoleted by the osd
layer. This patch moves the remaining fsfilt code
related to ldiskfs to osd-ldiskfs where it belongs.
The remaining bits of the lvfs code it moved to the
ptlrpc layer where it is used. The last of the procfs
code is integrated into the obdclass layer as well.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ie72103639fbcbbcedd0f810ac68044b38a0f0b65
Reviewed-on: http://review.whamcloud.com/5512
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 rpm: remove SLES10 support

SLES10 is out of scope for Lustre 2.4 so we can drop
the spec file hacks.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I94ea2164a4b339ab26825058678b54859d2617a8
Reviewed-on: http://review.whamcloud.com/5490
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2800 autoconf: remove LN_5ARGS_SYSCTL_PROC_HANDLER test

ctl_table.proc_handler has taken 5 arguments since v2.6.32.

We remove the test and dead code. Also eliminate the ll_* prefixed
compatibility shims and empty macros. I've left the LL_PROC_PROTO
macro because it definitely simplifies the appearance of the handler
definitions.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8c5fc509e1c8bc52705090f5eedf8fb10d549cf5
Reviewed-on: http://review.whamcloud.com/5411
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2800 autoconf: remove LIBCFS_FUNC_DUMP_TRACE test

dump_trace has been upstream since v2.6.23. The reliable argument was
added in v2.6.25. The other bits were added in v2.6.19.
We can remove the test and dead code.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I3b998f62ce4908adf377acd395725493dd717c3a
Reviewed-on: http://review.whamcloud.com/5403
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4072 tests: Decrease load on MDT for ZFS in sanity/24v

Due to performance of ZFS, reduce the number of file creates until
LU-2887/LU-4072 are resolved.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I15bc5b69d8ff114bf3cd7fa874c238ecbe29f6f9
Reviewed-on: http://review.whamcloud.com/7870
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2887 tests: Reduce performance expectatiomn for zfs

Drop ZFS Performance expectation to below observed values.  This is a
temporary measure until ZFS performance issues are dealt with.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I86c025e15ba67e86269344faa2ea56be0966a873
Reviewed-on: http://review.whamcloud.com/7848
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3626 tests: More time to allow mds-ost sync being finished

Because mds-ost unlink log synchronization is not enforced by any
timeout, so we will add more time to allow mds-ost sync being
finished in replay-single test_20b. And also add more information
in the tests to help us understand which MDT slow down the
synchronziation.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I5f63540e602ad29cd221de6dac0c22546629a9cd
Reviewed-on: http://review.whamcloud.com/7676
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-3467 mdt: call MDT handlers via unified request handler

Move MDT to the unified request handler. All MDT-specific handlers
are swtched to use tgt_session_info instead of mdt_thread_info.
To minimize amount of changes the mdt_thread_info is initialized
just from tgt_session_info in most cases for now.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I72e342cc900dc3dd34ca709542ad67de5015dd1f
Reviewed-on: http://review.whamcloud.com/6973
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3161 lod: Added proper error handling instead of LASSERT

LASSERT are replaced with proper error handling in
lod_initialize_objects.

Signed-off-by: Gaurav Mahajan <gmahajan@ddn.com>
Change-Id: Ia8fbbb4956765835bf46b46b3a7addc99cb191fa
Reviewed-on: http://review.whamcloud.com/7234
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4023 build: wrong type used

Fix PPC build, because it needs to use LPX64 instead of %#llx.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I0f4756a1e529b114d535604e122500230fb8f3ef
Reviewed-on: http://review.whamcloud.com/7856
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-4050 llite: NFS reexport issue

Suppress erroneous/confusing messages when NFS
is out of sync and requests old data.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Idf382769fe3b5b502ca6f545fab6b72fcc62527c
Reviewed-on: http://review.whamcloud.com/7850
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

Split 2.5 and 2.6 development branches apart

New tag 2.5.50 for what will eventually become 2.6.0

Change-Id: I6ba45bb595f43fb0dfc842b1ff21c7dc73513e14

LU-3240 llite: Do not send parent dir fid in getattr by fid

Sending getattr by fid in this case is pointless, as the parent
might havelong changed and we have no control over it, but it's
irrelevant anyway, since we already have the child fid.

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I8ea05e931cd85c126274b024a440137c309e5eb7
Reviewed-on: http://review.whamcloud.com/7910
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3240 mdc: Check for all attributes validity in revalidate

GETATTR needs to return attributes protected by different bits, so
we need to ensure all we have locks with all of those bits, not
just UPDATE bit.

Xyratex-bug-id: MRP-1052
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I0e27a33341c424a8666bdbf3cdb83b1b4fe0058f
Reviewed-on: http://review.whamcloud.com/6460
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3027 lov: to not modify lov lock when sublock is canceled

Otherwise it will cause wrong lock to be enqueued.

lls_ever_canceled is introduced to solve potential performance
problem by this patch.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I1ea62902c9d1a468f5952f30a8dc5dec22b8bb1c
Reviewed-on: http://review.whamcloud.com/7841
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3999 hsm: Rename several HSM files in /proc for MDT.

Some files in /proc, available for MDT are misnamed. This could be
incoherent or troublesome for sysadmins. This patch renames them.

`actions' is used when refering to HSM requests stored in MDT llog.
`active_requests' is used when refering to HSM requests currently
being handled by a copytool, on an agent node.

Also rename some HSM variable names and functions to have them
matching proc file names more consistently.

sanity-hsm tests have been updated accordingly.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: If2b5c24e06d480554040d8f69b09236e0ddd98e9
Reviewed-on: http://review.whamcloud.com/7740
Tested-by: Hudson
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4059 tests: Archive access must be done from agent

In sanity-hsm test_11, the size of an archived file is
requested from a client and not necessarily from an agent.
So, this patch now makes the call to stat from an agent.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I2f7ef294a0e9e28301312ad8529c6178279f5c19
Reviewed-on: http://review.whamcloud.com/7851
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3587 tests: add sanity test for crash in changelog

This patch adds a sanity test to verify that the previous fixes committed
for this bug prevent a crash during a changelog fetch after renaming
one very long filename to another.

Test-Parameters: testlist=sanity envdefinitions=ONLY=160b

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I559bbd1ba9210f21ac58246366c61ee33f34ae1a
Reviewed-on: http://review.whamcloud.com/7751
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3866 hsm: permission checks on HSM operations

In the LL_IOC_HSM_CT_START case of ll_dir_ioctl() require
CAP_SYS_ADMIN, since the local handler for this ioctl may modify the
global KUC table.

In the MDC HSM handlers that do not pack a real suppgid, use -1 rather
than 0 for the suppgid in mdt_body.

In mdt_hsm_release() and the the MDT HSM RPC handlers require a
read-write client mount for all operations except restore, get HSM
state, and get HSM actions.  Require CAP_SYS_ADMIN for
MDS_HSM_PROGRESS, MDS_HSM_CT_REGISTER, and
MDS_HSM_CT_UNREGISTER. Require CAP_SYS_ADMIN in mdt_hsm_state_set()
for setting flags not in HSM_USER_MASK.

Add per-coordinator bit masks (cdt_{user,group,other}_request_mask)
indexed by the HSMA constants to govern permissions on the various
requests types. By default each mask is set to allow restore only.
Add files /proc/fs/lustre/mdt/*/hsm/{user,group,other}_request_mask to
get and set these masks.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ifcb3c0950ebb11187cce62f15abbe8746f1ff7c2
Reviewed-on: http://review.whamcloud.com/7565
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4024 tests: cleanup file system before DNE tests.

Cleanup filesystem before DNE tests in conf-sanity.

Test-Parameters: mdtcount=2 mdscount=2
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I02c025f1ba1af3983f70694ccf99bfb82645cf14
Reviewed-on: http://review.whamcloud.com/7801
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-4056 hsm: reorder MDT proc cleanup

In mdt_procfs_init() remove the call to ptlrpc_lprocfs_register_obd().
The single file that this provides (../mdt/*/stats) is backed by the
MDT OBD's obd_svc_stats which are not tallied. In mdt_procfs_fini()
remove the corresponding call to ptlrpc_lprocfs_unregister_obd() and
move the call to hsm_cdt_procfs_fini() to before
lprocfs_obd_cleanup(). This prevents a use after free of the entries
in the MDT's hsm subdirectory.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iad6e7738047377660f0077c9368f79d96f903d02
Reviewed-on: http://review.whamcloud.com/7846
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4054 llog: don't leak llog handle in llog_cat_process_cb()

An early return from llog_cat_process_cb() was leaking the llog
handle. Fix this by not doing that.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I01e8b5f389a097f17bb1370aaec7382bc9d840c9
Reviewed-on: http://review.whamcloud.com/7847
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-4023 build: wrong type used

Fixed implicit conversion from 'unsigned long long' to 'int'.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I331a8ebe1fb9ef53ebd4fc92603b4100f006ee2b
Reviewed-on: http://review.whamcloud.com/7799
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4002 hsm: avoid layout lock on unlink and rename onto

In mdt_reint_unlink() and mdt_reint_rename() do not take a FULL inode
lock on the file to be unlinked or the rename target. This avoids a
potential deadlock when unlinking a file under HSM restore.

Signed-off-by: John L. Hammond <johnlockwoodhammond@gmail.com>
Change-Id: Id6b11b9f5c1433221670d154d65d9cec30e260af
Reviewed-on: http://review.whamcloud.com/7792
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4010 lov: Don't wait for active target with OBD_STATFS_NODELAY

commit 6c3c5c7b breaks OBD_STATFS_NODELAY behaviour.
It adds unnecessary delay while running df command with inactive OSTs.
We shouldn't try to recover connection to OST in this case.

Change-Id: Ia04cfb76b393df4f0c4c8d841b48d20306025c92
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-on: http://review.whamcloud.com/7762
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3864 lov: return minimal FIEMAP for released files

Since st_blocks = NULL is returned for released files, FIEMAP
should at least return a minimal mapping to make users aware
that file contains data but it is not immediately available.
This will make coreutils and tools such tar happy and have
them presume file is sparse.

Also, add a new test_228 in sanity-hsm to verify it works
for "[cp,tar] --sparse" commands.

Also fix a LBUG ("lov_fiemap()) ASSERTION( fm_local ) failed")
likely to occur when no-object/ENOMEM conditions and also now
when released.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Idc59f755d407fd34e7095898c68b4aceb5bf9c28
Reviewed-on: http://review.whamcloud.com/7584
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>

LU-3882 hsm: Prevent duplicate CT registrations

Associate copytool registration to a given MDC import so that
multiple mounts of the same filesystem do not lead to having the
copytool registered multiple time.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I5767df50331675a1650dda1ab5fc8440aad2a52e
Reviewed-on: http://review.whamcloud.com/7612
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3750 mdt: fix typo in mdt_txn_stop_cb()

Wrong check was added to mdt_txn_stop_cb() to identify object
for version update. Version is not updated for all local objects
due to that typo and leads to replay-vbr.sh failures

Test-Parameters: testgroup=review,failover
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I9c41f7208d7aa8cf307858b9c6f51ae73c1081e4
Reviewed-on: http://review.whamcloud.com/7787
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>