LU-3200 mdc: layout lock rpc must not take rpc_lock

When a client issue an RPC to get a layout lock, it
must not hold rpc_lock because in case of a restore
the rpc can be blocking for a long time

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Id9ea9be47669fd52c8d6e681090b4c911cfe2dc1
Reviewed-on: http://review.whamcloud.com/6115
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-2743 ptlrpc: Translate between host and network errnos

Lustre puts system errors (e.g., ENOTCONN) on wire as numbers
essentially specific to senders' architectures.  While this is fine
for x86-only sites, where receivers share the same error number
definition with senders, problems will arise, however, for sites
involving multiple architectures with different error number
definitions.  For instance, an ENOTCONN reply from a sparc server will
be put on wire as -57, which, for an x86 client, means EBADSLT
instead.

To solve the problem, this patch defines a set of network errors for
on-wire or on-disk uses.  These errors correspond to a subset of the
x86 system errors and share the same number definition, maintaining
compatibility with existing x86 clients and servers.

Then, either error numbers could be translated at run time, or all
host errors going on wire could be replaced with network errors in the
code.  This patch does the former by introducing both generic and
field-specific translation routines and calling them at proper places,
so that translations for existing fields are transparent.
(Personally, I tend to think the latter way might be worthwhile, as it
is more straightforward conceptually.  Do we really need so many
different errors?  Should errors returned by kernel routines really be
passed up and eventually put on wire?  There could even be security
implications in that.)

Thank Fujitsu for the original idea and their contributions that make
this available upstream.

Change-Id: I98a1421e90f0ddd1d1b558d2c4afbebbd6dc42bd
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5577
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3411 osd: fsfilt_get_ops returns an error, not NULL

osd_mount() should be able to recognize this properly.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ife1f52cf264fd6041dfc01606c6e48e47f2f8ff0
Reviewed-on: http://review.whamcloud.com/6496
Tested-by: Hudson
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3032 ptlrpc: race in pinger (use-after-free situation)

The race is result of use-after-free situation:

~ ptlrpc_stop_pinger()          ~ ptlrpc_pinger_main()
---------------------------------------------------------------
thread_set_flags(SVC_STOPPING)
cfs_waitq_signal(pinger_thread) ...
...                             thread_set_flags(SVC_STOPPED)
l_wait_event(thread_is_stopped)
OBD_FREE_PTR(pinger_thread)
...                             cfs_waitq_signal(pinger_thread)
---------------------------------------------------------------

The memory used by pinger_thread might have been freed and
reallocated to something else, when ptlrpc_pinger_main()
used it in cvs_waitq_signal().

Signed-off-by: Li Wei <wei.g.li@intel.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I3516c7ece2b2e1e4e0b2e47acb0583c174a3e631
Reviewed-on: http://review.whamcloud.com/6040
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2930 test: add tests for data version

This patch adds support of dataversion access in
multiop and use it to test dataversion on volatile
file.
It also adds a test in sanity.sh to validate dataversion works as
expected (changes between writes)

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I62c2b2124d225331873185d92f5c2404a74b1196
Reviewed-on: http://review.whamcloud.com/5660
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-3089 tests: Give ZFS more time to complete dd's

Due to LU-2887 and ilk, give zfs more time to do disk transfers.  Also
cleanup test formatting.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ia22f2244ccec552046d2efeaeaa6e2194547c96c
Reviewed-on: http://review.whamcloud.com/6217
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2193 ldlm: print FID in lvbo_init(), lvbo_update

Print the namespace and OBD device name, as well as the first two
lock resource fields (typically the FID) if there is an error with
loading the object from disk.  This will be more important with
FID-on-OST and also the MDS.  Using fid_extract_from_res_name() isn't
possible in the LDLM code, since the lock resource may not be a FID.

Make fid_extract_quota_resid() argument order and name consistent
with other fid_*_res() functions, with FID first and resource second.

Fix a bug in ofd_lvbo_init() where NULL lvb is accessed on error.

Print FID in ofd_lvbo_update() CDEBUG() and CERROR() messages.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7f323f15a5bf4a12865a64b2cd2bb0ebe68c6e78
Reviewed-on: http://review.whamcloud.com/4501
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3243 utils: print value in lctl set_param error

Print out the invalid value that was trying to be set via
"lctl set_param parameter=value", so that it is more clear
what is going wrong when this is being called from a script.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Iebd0ae70dc9dbd378775339ccba2ab90a97d1093
Reviewed-on: http://review.whamcloud.com/6205
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3268 lod: support BE servers in lod_verify_striping()

To support big-endian Lustre servers, in lod_verify_striping() use
leNN_to_cpu() to access the members of struct lov_user_md.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I146e35ab75bc290e92aa9d68211fa1ffedada29b
Reviewed-on: http://review.whamcloud.com/6273
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3180 tests: lfsck.sh improved messages

Use skip_env() for the SHARED_DIRECTORY check, so that lfsck is not
skipped for the normal test environment (VMs sharing a single host)
or single-node tests.

Add some more information about lfsck.sh corruption, for improved
debugging in case of problems.

Test-Parameters: testlist=lfsck
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I56d7a52b0f880587d988c842fb6ee0b14b699ca2
Reviewed-on: http://review.whamcloud.com/6206
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2559 tests: enable "slow" lustre-rsync tests

The "SLOW=yes" tests that are disabled in lustre-rsync-test.sh
appear to only be taking about 1-2 minutes each, and are always
passing in the "full" test runs on master.

It won't impact testing throughput significantly to enable them
even for review builds.  It increases overall test time by about
10 minutes in total, but improves our test coverage noticeably.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Iccec870c8b0936aa9e4532da6e8bde2d991ccab0
Reviewed-on: http://review.whamcloud.com/5214
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1538 utils: remove obsolete scripts

A number of obsolete test, build, and helper scripts are removed,
because they are no longer useful.

  lustre/doc/chbar.sh
  lustre/doc/postbar
  lustre/doc/tex2pdf
  lustre/scripts/bdev-io-survey.sh
  lustre/scripts/dodiff.sh
  lustre/scripts/maketags.sh
  lustre/tests/sanity-nano.sh
  lustre/utils/loadmod_all.sh

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I109133aadfc75f9812259e560710dc70036e3dd6
Reviewed-on: http://review.whamcloud.com/6227
Tested-by: Hudson
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2498 mount: Set 'deadline' scheduler on ldiskfs

The recommended IO scheduler to use with Lustre on top of ldiskfs is
either the 'deadline' scheduler, or the 'noop' scheduler. This patch
attempts to automatically configure a device to use the 'deadline'
scheduler for the user at mount time. This makes it unnecessary to use
any out of band mechanism to ensure this value is set properly (i.e.
start up scripts, kernel configuration, etc.).

Also, if the scheduler is already set to 'noop', this patch will leave
that setting in place. This makes it easy for a user to manually set the
scheduler to 'noop', if they would prefer to use that scheduler instead
of 'deadline'.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: If869468ba5370afddb29233dd893d74b9c45f367
Reviewed-on: http://review.whamcloud.com/4853
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1617 build: ignore automatically generated files

Do not list automatically generated files in "git status" output,
so that it does not hide checkpatch.pl output in the commit message.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Iab5c5be25de7dc54225fb2aa6540a4d75e500c1e
Reviewed-on: http://review.whamcloud.com/6229
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3110 tests: OSD_TRACK_DECLARES_LBUG default & handling

This patch sets OSD_TRACK_DECLARES_LBUG as yes/on by
default in Tests and also implements its correct
handling to only apply it on servers running with
ldiskfs back-end.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ie1764a3a576809f9f913f6effc54965a3bd153dc
Reviewed-on: http://review.whamcloud.com/6280
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-3262 tests: add slow ost-pools tests to SLOW list

Add test 5b, 18, 22, and 25 to the SLOW list, which cuts about
30 minutes off the ost-pools test time if run with SLOW=no.

Test-Parameters: testlist=ost-pools,ost-pools,ost-pools
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I430bdf5132516351e345df3ccf5461cb62500c1e
Reviewed-on: http://review.whamcloud.com/6235
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3364 tests: Skip sanity-lfsck if < 2.2.90

Skip sanity-lfsck if server version is older
than 2.2.90

Change-Id: I65119e7896a433edf50c1eee835ebafd8bca76da
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/6424
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3283 llite: check ll_prep_md_op_data() using IS_ERR()

In ll_file_ioctl() and ll_swap_layouts() check the result of
ll_prep_md_op_data() using IS_ERR().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib916635559863cd7a8aee3c88bb1d2bb2e7921b8
Reviewed-on: http://review.whamcloud.com/6275
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3157 llite: A not locked mutex can be unlocked.

In case of memory pressure a not locked mutex can be unlocked
in function ll_file_open(). This is not allowed and subsequent
behavior is not defined.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I21368247cac23038f9b995e9de328f9286909731
Reviewed-on: http://review.whamcloud.com/6028
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Tested-by: Hudson
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3154: Fix plot-sgpdd to match new sgpdd-survey output format

The fields in sgpdd-survey output have changed. This updates plot-sgpdd to
work correctly. One field 'rsz' has changed from numeric to string, new
variable added to deal with that.
Fix for trailing tabs on two lines. Cleaned up tabs per Andreas.
Removed one more trailing tab.

Change-Id: I754d5fec6f6aebbc0882386d2734aa6805eb8de5
Signed-off-by: Cliff White <cliffwhi@intel.com>
Signed-off-by: Cliff White <cliffw@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/6023
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>

LU-3174 build: missing noiokit option in lbuild

We can not use --noiokit since it's missing in options

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I155e5da94b67d1e4baa6613a787146174945246f
Reviewed-on: http://review.whamcloud.com/6073
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2008 utils: quote TUNE2FS options in ldiskfs_label_lustre

This patch improves ldiskfs_label_lustre() to quote the options
for TUNE2FS in case the label contains space(s).

Test-Parameters: envdefinitions=SLOW=yes \
clientcount=4 osscount=2 mdscount=2 \
austeroptions=-R failover=true useiscsi=true \
testlist=mmp

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I3eaeb604517dba29e07f9750aeb3afed9443e94f
Reviewed-on: http://review.whamcloud.com/6077
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3088 build: fix 'resource leak' errors

Fix 'resource leak' defects found by Coverity version 6.5.1:
Resource leak (RESOURCE_LEAK)
Variable going out of scope leaks the storage it points to.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I12312b081ea735118d5375c0dc9c20ead5ed9286
Reviewed-on: http://review.whamcloud.com/5918
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2753 llite: check alloc in ll_file_data_get, ll_dir_ioctl

In ll_file_data_get() and ll_dir_ioctl() return error on failed
allocations.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I122db2cede770261d75af209e92aa60d86ea85f2
Reviewed-on: http://review.whamcloud.com/5845
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>

LU-3054 build: fix 'program hangs' errors

Fix 'program hangs' defects found by Coverity version 6.5.1:
Missing unlock (LOCK)
Returning without unlocking.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I96c2bb05a5875ed493ebe99587e9901baea80fc8
Reviewed-on: http://review.whamcloud.com/5870
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <DELETED>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3385 llite: missing last bit in ll_have_md_lock

Missing the last bit during INODELOCK check in ll_have_md_lock.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I0801554904cab106513b76ace4f1467d14cfdf1a
Reviewed-on: http://review.whamcloud.com/6438
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3384 llite: use READ, WRITE around ll_rw_stats_tally()

In vvp_io_write_start() the stats function ll_rw_stats_tally() was
incorrectly called with a rw argument of 0. Correct this and use the
macros READ and WRITE in and around ll_rw_stats_tally() for clarity.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Icca6cfcc102b2944761ff211f1b2c2734aae8ecf
Reviewed-on: http://review.whamcloud.com/6447
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3409 llite: silence lockdep warning in ll_md_blocking_ast

Got bellow lockdep warning during tests. It is false alarm though.

[ 1184.479097] =============================================
[ 1184.479187] [ INFO: possible recursive locking detected ]
[ 1184.479277] 3.10.0-rc3+ #13 Tainted: G         C
[ 1184.479355] ---------------------------------------------
[ 1184.479444] mkdir/2215 is trying to acquire lock:
[ 1184.479521]  (&(&dentry->d_lock)->rlock){+.+...}, at:
[<ffffffffa06cc27c>] ll_md_blocking_ast+0x55c/0x655 [lustre]
[ 1184.479801]
but task is already holding lock:
[ 1184.479895]  (&(&dentry->d_lock)->rlock){+.+...}, at:
[<ffffffffa06cc1b1>] ll_md_blocking_ast+0x491/0x655 [lustre]
[ 1184.480101]
other info that might help us debug this:
[ 1184.480206]  Possible unsafe locking scenario:

[ 1184.480300]        CPU0
[ 1184.480340]        ----
[ 1184.480380]   lock(&(&dentry->d_lock)->rlock);
[ 1184.480458]   lock(&(&dentry->d_lock)->rlock);
[ 1184.480536]
 *** DEADLOCK ***

[ 1184.480761]  May be due to missing lock nesting notation

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I34c54ddc7a36dbef77564b1e4f14a16d848b2eaa
Reviewed-on: http://review.whamcloud.com/6469
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>

LU-3232 build: Move fsfilt_ldiskfs.ko into lustre-osd-ldiskfs

The fsfilt_ldiskfs.ko module depends on symbols in osd_ldiskfs.ko and
ldiskfs.  However, osd_ldiskfs.ko is only part of the optional package
lustre-osd-ldiskfs, while fsfilt-ldiskfs.ko is in the non-optional
package lustre-modules.

This means that a person using only the lustre-osd-zfs osd will see a
screen full of warnings about missing ldiskfs-related symbols.

We remedy this problem by moving the fsfilt-ldiskfs.ko into the
lustre-osd-ldiskfs rpm.

Change-Id: I631a98dd56c7d4df69266e4f922f949132774d84
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/6260
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3015 osd-zfs: Fix a typo in udmu_objs_count_estimate

The typo is relatively straightforward given the diff below.
It leads to larger-than-actual average dnode size estimations,
which further causes overly conservative os_ffree (number of
available objects) values. For example, without this patch,
os_ffree values for "empty" 1 GB ZFS-based MDT/OSTs look like:

  [root@linux tests]# df -h
  Filesystem            Size  Used Avail Use% Mounted on
  [...]
  lustre-mdt1/mdt1      929M  3.0M  924M   1% /mnt/mds1
  lustre-ost1/ost1      929M  1.3M  926M   1% /mnt/ost1
  lustre-ost2/ost2      929M  1.3M  926M   1% /mnt/ost2
  linux@tcp:/lustre     1.9G  2.5M  1.9G   1% /mnt/lustre
  [root@linux tests]# df -i
  Filesystem            Inodes   IUsed   IFree IUse% Mounted on
  [...]
  lustre-mdt1/mdt1        7583     178    7405    3% /mnt/mds1
  lustre-ost1/ost1        7631     212    7419    3% /mnt/ost1
  lustre-ost2/ost2        7631     212    7419    3% /mnt/ost2
  linux@tcp:/lustre       7583     178    7405    3% /mnt/lustre

With this patch, they look like:

  [root@linux tests]# df -h
  Filesystem            Size  Used Avail Use% Mounted on
  [...]
  lustre-mdt1/mdt1      929M  2.9M  924M   1% /mnt/mds1
  lustre-ost1/ost1      929M  1.3M  926M   1% /mnt/ost1
  lustre-ost2/ost2      929M  1.3M  926M   1% /mnt/ost2
  linux@tcp:/lustre     1.9G  2.5M  1.9G   1% /mnt/lustre
  [root@linux tests]# df -i
  Filesystem            Inodes   IUsed   IFree IUse% Mounted on
  [...]
  lustre-mdt1/mdt1      161966     178  161788    1% /mnt/mds1
  lustre-ost1/ost1      223825     212  223613    1% /mnt/ost1
  lustre-ost2/ost2      223825     212  223613    1% /mnt/ost2
  linux@tcp:/lustre     161966     178  161788    1% /mnt/lustre

Only after fixing this typo could we evaluate how the intented
estimation algorithm works.

Test-Parameters: mdsfilesystemtype=zfs ostfilesystemtype=zfs mdtfilesystemtype=zfs
Signed-off-by: Li Wei <wei.g.li@intel.com>
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic3cbe569261d2c0e4338001c47f3dd64c621621a
Reviewed-on: http://review.whamcloud.com/5806
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-2784 build: Provide RPMs build for Xeon Phi(TM) card

Enhance Lustre build to provide RPMs which can be installed on
Xeon Phi(TM) card automatically. This means the binaries
should be compiled by cross compiler and deployed into Xeon
Phi(TM) infrastructure on host to be automatically installed
on card after boot.

To produce Lustre client RPMs for Xeon Phi(TM) card just
execute the following commands:

export PATH=/usr/linux-k1om-4.7/bin:$PATH

sh ./autogen.sh

./configure --with-linux=/opt/intel/mic/src/card/kernel \
    --disable-server --without-o2ib \
    --host=x86_64-k1om-linux --build=x86_64-pc-linux

make rpms

NOTE: You should have "intel-mic-gpl-<version>.x86_64" package
installed and MIC GPL sources unpacked in /opt/intel/mic/src.

and the following RPMs will be produced:

lustre-client-mic-<version>.x86_64.rpm
lustre-client-mic-debuginfo-<version>.x86_64.rpm
lustre-client-mic-modules-<version>.x86_64.rpm
lustre-client-mic-source-<version>.x86_64.rpm
lustre-client-mic-tests-<version>.x86_64.rpm
lustre-client-mic-<version>.src.rpm

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I8d61133614443e2a6a33f5c1b1b097250b11d749
Reviewed-on: http://review.whamcloud.com/5324
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3360 osc: check ostid if no OBD_CONNECT_FID

in lustre_set_wire_obdo, if "ocd->ocd_connect_flags" doesn't contain
OBD_CONNECT_FID, use "ost_id" to check the type of the object
instead of using ost_id.oi_fid only.

Change-Id: I4d9ef133c3d26c3f8f36d5b5b9430e941e445068
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/6426
Tested-by: Hudson
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2598 tests: check nr_local in ofd_preprw()

This patch checks the number of bulk I/O RPC pages with
PTLRPC_MAX_BRW_PAGES in ofd_preprw() to avoid
LASSERT(iobuf->dr_npages < iobuf->dr_max_pages) occurring
while larger I/O size is specified.

The patch also fixes echo_client_prep_commit() to reuse
the env context so as to avoid LASSERT(info->fti_exp == NULL)
occurring while the bulk I/O size is larger than
PTLRPC_MAX_BRW_SIZE.

The patch also improves obdfilter-survey to handle the
case while interoprating with old server.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
testlist=obdfilter-survey

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ifa7b720da1e3fc752f85bcfa6968eb16f93af2fa
Reviewed-on: http://review.whamcloud.com/6394
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3359 osc: fixed a typo in osc_cache_truncate_start

Dump osc_extent waiting instead of ext in case error occurred.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I7df560a0aa83ddc6010cad1c51fa2013e941ae60
Reviewed-on: http://review.whamcloud.com/6402
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3369 scrub: skip test_1a for old 2.x (x <= 3) release

Lustre-2.3 or older does not support initial OI scrub,
so skip related sanity-scrub test_1a.

Test-Parameters: testlist=sanity-scrub
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I17752be675bdda76588c55c38c20c37a009eff00
Reviewed-on: http://review.whamcloud.com/6417
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-3350 tests: allow test-error for lfsck/scrub speed

Affected by the test environment and CPU schedule randomness,
especailly test in virtal machine, it is not easy to control
the lfsck/scrub speed accurtely. So we have to consider more
test-error.

Test-Parameters: testlist=sanity-scrub,sanity-lfsck

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I794d26d2bfa73d7000e77790cc7a2c8203f7db06
Reviewed-on: http://review.whamcloud.com/6374
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

Revert "LU-3264 tests: support failover testing with ZFS"

Unfortunately zfs testing took a nosedive after this commit was included.

This reverts commit fb8bf83fabd45730345b3eb8c5289334174462be.

Revert "LU-3264 tests: import ZFS pool which has file-based vdev"

Unfortunately zfs testing took a nosedive after this commit was included.

This reverts commit 3dffd5b5c4c95f29e718cab63e8ef1c7da0c1f8b.

LU-3164 utils: Fix Wireshark decode of Lustre under Infiniband

Due to a difference in the IB LND vs TCP LND header sizes, some
hard-coded offsets in the LNet dissector will not believe that the
payload is lustre.  As such, the lustre protocol may not be decoded
under Infiniband.

This patch addresses this by adding a variable to applicable offsets
which is set to 0 for TCP, and 24 (extra bytes in IB LND header)
for Infiniband.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I3783bbb563df109e160752d246db9bc5148cf132
Reviewed-on: http://review.whamcloud.com/6045
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3300 lprocfs: interpret result of dt_statfs() correctly

In a7369bcd860af61a073cb2424139e3acccdcb203, I accidentally reversed
the sense of the error check after the call to dt_statfs() in
lprocfs_dt_rd_{blksize,{files,kbytes}{free,avail}. Unreverse the error
checking.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I91873f23cd69c2217474c26b9751aec259fae155
Reviewed-on: http://review.whamcloud.com/6385
Reviewed-by: Robert Read <robert.read@intel.com>
Tested-by: Hudson
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3354 kernel: Kernel update [RHEL6.4 2.6.32-358.6.2.el6]

Update RHEL6.4 kernel to 2.6.32-358.6.2.el6.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5c6b48d82382974b3cdc8a0c84c5ef54194d5217
Reviewed-on: http://review.whamcloud.com/6379
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3306 mdt: Do not LBUG on double set of err_serious

It seems the twisted code pathes in mdt could take strange routes
and it's quite hard to guarantee they won't pass through setting
err_serious twice, esp. since error handlers are traditionally
poorly tested.

Change-Id: I88fe880c22670263273bd79073efaecbe53d9215
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/6383
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-3318 llite: Always build 64bit ino internally

Always build 64bit ino internally except the client is mounted
with "32bitapi" option, so client will always use 64bit ino
internally. It will build 32bit ino, only if application requires
32 bit ino.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I717e0551a7df4f8a7f02168702d8c9c58ecafd3d
Reviewed-on: http://review.whamcloud.com/6371
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-3264 tests: import ZFS pool which has file-based vdev

For "zpool import" command, if the -d option is not specified,
the command will only search for devices in "/dev". However,
for the pool which was created with file-based virtual device,
we need explicitly specify the search directory otherwise the
import command will not find the device.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I63ac141fdefcace99a1cf7c366e15f0067fa2a56
Reviewed-on: http://review.whamcloud.com/6358
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.4.50

After branching off b2_4, master becomes 2.5.0 development branch.

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I1c8207394016d793a64f57d410d33c43ac96b34c

LU-3292 build: kernel update for 3.0.74-0.6.6 sles11sp2

Delete base and ldiskfs patches that are now obsolete
in the latest kernel update version for SLES11 SP2.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I9a5ef1347afb228453db79b213e3881c6a112938
Reviewed-on: http://review.whamcloud.com/6293
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3036 mdt: set ATTR_xTIME_SET to make atime update properly

To make atime update properly between 1.8 client and 2.x server,
this patch includes the following fixes:
- if MDS_ATTR_xTIME is set without MDS_ATTR_xTIME_SET and the client
  does not have OBD_CONNECT_FULL20, convert it to LA_xTIME in
  mdt_setattr_unpack().
- set both MDS_ATTR_xTIME | MDS_ATTR_xTIME_SET for timestamps in
  ll_prepare_close(). This allows us to fix the server-side timestamp
  setting in the future.
- remove attr_unpack() and convert the flags from MDS_ATTR_ to LA_*
  directly in mdt_attr_valid_xlate() instead.
- improve sanityn.sh test_23().

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I9454d7689243a268ebd4f40566f77e4f53996d5f
Reviewed-on: http://review.whamcloud.com/6327
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3279 changelog: fix CL_LAYOUT comment, string

Fix the CL_LAYOUT comment, since it is possible that layout swap
could be used in cases where the file content is changed.  I don't
think there are any real world use cases for this, but that depends
on what tools are being used on the filesystem.

Also change the "LAYOUT" string for this ChangeLog record type to
be "LYOUT" to match the convention of other strings to only be
5 characters long.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ibf9a57216e0c45a1e15cef9cfe5ee7f509500c1e
Reviewed-on: http://review.whamcloud.com/6338
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3328 osp: Only signal thread completion when we are done

IT seems that by signalling the completion of a thread before
we are done inspecting it's memory is racy and could cause access
to freed memory.

Change-Id: Ia992e9c7d0d78fca1afbfc98f51c5df3f6dc3a28
Signed-off-by: Oleg Drokin <green@intel.com>
Reviewed-on: http://review.whamcloud.com/6329
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2915 lfsck: NO scrub for agent inode and remote parent

Under DNE case, for the object on remote MDT, there will be an agent
inode on the local MDT where the name entry resides. We do NOT build
OI mapping for such agent inode, so OI scrub should skip it.The same
for the "/REMOTE_PARENT_DIR".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I164473bcba80043800d491639c027e7c13f30bd9
Reviewed-on: http://review.whamcloud.com/6332
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3219 ost: Ensure dirty flushed on fiemap ioctl

Data corruption is possible if cp(coreutils) uses FIEMAP to obtain
data holes in shared file, since there could be dirty cache on
other clients which hasn't been flushed back.

To ensure all the dirty on remote clients being flushed back on
fiemap ioctl, we'd acquire ldlm lock on server side for fiemap,
unless the local client (which invoke fiemap) has cached lock.

Xyratex-bug-id: MRP-1001
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I67fe05bac1ce75f98c3d3806a4d08f6cb02048e6
Reviewed-on: http://review.whamcloud.com/6127
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3279 changelog: fix CL_LAYOUT, accept all types

In order to avoid compatibility issues with older ChangeLog consumers,
change the new CL_LAYOUT record to use the value previously assigned
to CL_IOCTL.  The CL_IOCTL type was never used anywhere, and it didn't
really make any sense as a ChangeLog record, since it could really
mean anything at all.

The changelog_show_cb() function is renamed to changelog_kkuc_cb(),
since it is really about consuming the ChangeLog and passing it up to
the kernel-user-coms interface.  At some point we should consider to
implement a DBUS-based ChangeLog interface as well.

The changelog_kkuc_cb() "sanity check" on cr_type was removed, since
there is no reason the client kernel needs to know every record type
that is being passed to userspace.  It is up to the client tool to
determine what records that it can process.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id8e6471827a89d8c62808e6b3cfd9693d6026431
Reviewed-on: http://review.whamcloud.com/6308
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2979 lprocfs: use stats counter index for *pos

In lprocfs_stats_seq_{start,next,show,stop}() encode the counter index
(rather than the counter address) into *pos. Doing so simplifies these
functions and fixes a bug in the case of per-CPU stats where no stats
would be displayed at all if no events had yet occurred on CPU 0.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Id50f8ae480e151f8e16382db27eef8025ef69f15
Reviewed-on: http://review.whamcloud.com/6328
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3269 build: Remove circular rpm dependency

The rpm package dependencies look a bit like this currently:

 lustre -> lustre-modules <-> lustre-osd-[ldiskfs|zfs] (via lustre-osd)

Note that the dependency between lustre-modules and lustre-osd-* goes both
ways.

We did that in the one direction because we need _something_ to
depend on the generic "Provides: lustre-osd" when we have server support
compiled in, so users will be clued in to the fact that they need to
select at least one of the lustre-osd-* packages.

In the other direction, the lustre-osd-* packages need to depend on
lustre-modules because the kernel modules contained in the lustre-osd-*
rpms really do use symbols from modules in the lustre-modules rpm.

RPM allows this circular dependency, but because of it the modules are not
necessarily installed in the order that we prefere.  Namely:

1) lustre
2) lustre-modules
3) lustre-osd-*

It can happen that the lustre-osd-* packages are installed before the
lustre-modules package, and then the sysadmin will see a screen full of
scary messages about missing symbols.

To fix that problem, we move the dependency on lustre-osd up one package
into the "lustre" package.  That makes the dependency graph look a bit
like:

   lustre -> lustre-modules
        |--> lustre-osd-[ldiskfs|zfs] (via lustre-osd)
   lustre-osd-[ldiskfs|zfs] -> lustre-modules

I by breaking the dependency loop, we get the desired package installation
order described above.

Change-Id: I7bab1d536e75ce4cf3eb70fa4569c435ec8b2d66
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/6259
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3264 tests: support failover testing with ZFS

This patch improves test-framework.sh to support failover testing with
ZFS under shared storage configuration.

After formatting a ZFS target, "cachefile=none" property will be set
on the ZFS storage pool so that the pool is not automatically imported
on system startup. And then the pool will be exported so as to leave
the import and export operations handled by mount_facet() and stop().

For SOFT failure mode (unmount and mount), "zpool export …" will be
run on the active server node to export the ZFS storage pool, and
"zpool import -f …" will be run on the failover partner node to import
the pool.

For HARD failure mode (power off and on), since the "cachefile=none"
property has been set on the ZFS storage pool on the active server
node, after the node is power-cycled, the pool will not be imported.
Then on the failover partner node, "zpool import -f …" can be run to
import the pool.

Test-Parameters: envdefinitions=PTLDEBUG=-1,DEBUG_SIZE=128 \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
clientdistro=el6 serverdistro=el6 clientarch=x86_64 \
serverarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
austeroptions=-R failover=true useiscsi=true \
testlist=recovery-double-scale

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I6b9646cb36f7fa4924af03a055ccb137187843a5
Reviewed-on: http://review.whamcloud.com/6258
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3314 scrub: use special fixed FID for .lustre

For lustre-2.x (x <= 3), the ".lustre" has NO FID-in-LMA,
so the client will get IGIF for the ".lustre" object when
the MDT restart.

From the OI scrub view, when the MDT upgrade to Lustre-2.4,
it does not know whether there are some old clients cached
the ".lustre" IGIF during the upgrading. Two choices:

1) Generate IGIF-in-LMA and IGIF-in-OI for the ".lustre".
   It will allow the old connected clients to access the
   ".lustre" with cached IGIF. But it will cause others
   on the MDT failed to check "fid_is_dot_lustre()".

2) Use fixed FID {FID_SEQ_DOT_LUSTRE, FID_OID_DOT_LUSTRE, 0}
   for ".lustre" in spite of whether there are some clients
   cached the ".lustre" IGIF or not. It enables the check
   "fid_is_dot_lustre()" on the MDT, although it will cause
   that the old connected clients cannot access the ".lustre"
   with the cached IGIF.

Usually, it is rare case for the old connected clients
to access the ".lustre" with cached IGIF. So we prefer
to the solution 2).

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifa491850ddab0de0b67aab124dc206ad4f714428
Reviewed-on: http://review.whamcloud.com/6309
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3187 ost: check pre 2.4 echo client in obdo validation

Because old echo client still uses o_id/o_seq for objid,
but new echo client will uses FID for the objid. Add
OBD_CONNECT_FID for 2.4 echo client, so 2.4 OST will
convert o_id/o_seq to FID if the request from old echo
client.

Add local flag OBD_FL_OSTID for o_flags to indicate
OST does not support FID yet, then echo client will
still send o_id/o_seq to OST.

cleanup ost_validate_obdo

Test-Parameters: clientjob=lustre-b2_1 clientbuildno=197 testlist=sanity,obdfilter-survey
Test-Parameters: serverjob=lustre-b2_1 serverbuildno=197 testlist=sanity,obdfilter-survey
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6001c813b668cf53a66d0d9d74f322bad63765ed
Reviewed-on: http://review.whamcloud.com/6287
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-3303 test: sanity-quota test_18 incomplete error message

test_18 is sending the error message to quota_error with
two string literals with no line continuation. quota_error
essentially ignores the second string literal.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I54e5cccf307349fab072d6fac2832f6bd98a3bfd
Reviewed-on: http://review.whamcloud.com/6302
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3100 tests: Skip recovery-small test_111

Recovery-small test_111 now checks the metadata server version and
will run the test for versions 2.3.62 and above. The test will be
skipped for all other server versions.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I58aa58dd7cd8a27209fb1c4ee258de9d3a84b144
Reviewed-on: http://review.whamcloud.com/5975
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

Revert "LU-3188 osc: shorten IO calling path"

This commit seems to have caused more problems than it fixed.

This reverts commit 83ae17df2bdce837e62473aec27c03d67312c8ea.

LU-3302 llog: Do not use ostid swab for llogid

Since logid still use id/seq format in the request,
it will be swabbed by its own swab func, instead of
using ostid swab, which might see logid as FID incorrectly.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I35f554e99a765384218ee3a357c931d6be1de6b3
Reviewed-on: http://review.whamcloud.com/6305
Tested-by: Hudson
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2886 scrub: non-fixed FIDs for some local files

For old lustre-2.x (x < 4), the local files PENDING/lfsck_bookmark
used fixed/reserved FIDs when created. The cases have been changed
to use variable local FIDs since lustre-2.4. So adjust OI scrub to
match the changes.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6d1f2f19afa3d777b16838ae03f46a39a58b537a
Reviewed-on: http://review.whamcloud.com/6299
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3301 utils: Replace %llu with LPU64

Replace incorrect use of %llu with LPU64 to allow compilation
on ppc64.

Change-Id: I85794527a43ed1577cf43ddb3470a9c8d070f11b
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/6296
Reviewed-by: Bobi Jam <bobijam.xu@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2886 obdclass: use common way to store lastid

Local files last id are stored in root in files named seq-xxx-lastid
while lastid for OST objects is stored in O/seq/LAST_ID special
object with zero OID and handled by OSD.
Patch reworks local files lastid to be stored in O/seq/LAST_ID too
and using the same format.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I72710e84cf0f7a0903b0b88ac3f9432eb59ea716
Reviewed-on: http://review.whamcloud.com/6199
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-3271 lov: only dump header in lsm_lmm_verify

If lsm_lmm_verify_*() find an error in the lov_mds_md header
structure, don't dump the full stripe information, since this
can be totally bogus (e.g. if stripe_count == -1 or similar).
Instead, just dump the header information for debugging.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Idf8c8bc35b156181aff9f0c5f0ea1f73c89e33d1
Reviewed-on: http://review.whamcloud.com/6261
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2886 obdclass: remove obsoleted md_local_file.c

This library is not used anymore and is replaced by
local_storage.c. Patch removed last remnants of it.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I4d1371908db898efc8c5cd650357c449f369f51b
Reviewed-on: http://review.whamcloud.com/6107
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-3281 osc: some cleanup to reduce stack overflow chance

ptlrpcd_add_req() will wake_up other process, do not hold a spinlock
before calling ptlrpcd_queue_work()->ptlrpcd_add_req().

If current process is allocating memory, memory shrinker could get to
osc_lru_del(), don't call osc_lru_shrink() further since it could
lead a long calling chain.

Use static string OES_STRINGS in OSC_EXTENT_DUMP() to reduce stack
footprint.

Alloc crattr on heap for osc_build_rpc() to reduce stack footprint.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I1a1ce0b46850773a2ae45ce16be6b708adc40ab8
Reviewed-on: http://review.whamcloud.com/6270
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3280 ldlm: suppress useless lock RPC for layout

In ldlm_lock_decref_internal() when l_lvb_data is freed to
reduce memory consumption, LDLM_FL_LVB_READY is not
cleared, so later when the lock is reused lvb is not
updated. But clearing LDLM_FL_LVB_READY forces layout refetch
at each file access, so the better is to remove the optimization.
The use case is after a restore in HSM.

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I3aa56cf39fe34941d227400410b27db32479b1b1
Reviewed-on: http://review.whamcloud.com/6268
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3190 mdd: not return linkEA for dead obj

1. Not return linkEA for dead object.
2. Check lma_self_fid to match object FID after get
real LMA from the object.
3. clear oi cache during oi delete.
4. correct error value if ldlm_handle_enqueue return
some value other than ldlm_err_t(for example -ESTALE).

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1d2345eb01ff58584ffba31f86bb408396780aeb
Reviewed-on: http://review.whamcloud.com/6252
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

New tag 2.3.65

Change-Id: I65f1eca5083842566443fd577e5a88f1dc57e88e

LU-3275 osd-ldiskfs: re-order osd device init/fini

There was race condition between the background OI scrub thread and
osd_device_init0: OI scrub thread may try to access non-initialized
osd_device::od_ost_map. So the osd initization/start process should
NOT trigger OI scrub until all the OI (OI files, /O, and ect.) have
been initialized. Reverse ordre for osd divice fini.

Test-Parameters: testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I99a2a72ace947a0f79ace5c6f445cf896d884e63
Reviewed-on: http://review.whamcloud.com/6267
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3117 build: zfs-0.6.1 kmod+dkms compatibility

With the release of zfs-0.6.1 the default install location of
the zfs kmod headers and objects has changed.  The kmod headers
which are common for a zfs version are now installed under
/usr/src/zfs-<version>/ path.  The objects, which are kernel
specific, are installed under /usr/src/zfs-<version>/<kernel>/.

This was done just prior to the official 0.6.1 release because
this scheme satisfies the packaging requirements of the major
distributions.  Making the change now means we shouldn't need
to change it again.

To accomidate this change the lustre-build-zfs.m4 has been
updated in the following ways:

* The new zfs header and object paths were added to the list
  of default search paths.  The DKMS build paths were also added
  to allow compilation against zfs-kmod or zfs-dkms packages.

* Support for building the spl and zfs code recursively as
  part of the Lustre build process was removed.

* The lustre-osd-zfs packages 'Requires' line was changed to
  require zfs-kmod.  Either the zfs-kmod or zfs-dkms packages
  can be used to satisfy this requirement.

* Fix incorrect usage of @ZFS_OBJ@ in osd-zfs/Makefile.in,
  the include directory us under @ZFS@ with the headers.
  These happens to be the same location before so it never
  caused issues.

* EXTRA_LIBZFS_INCLUDE renamed ZFS_LIBZFS_INCLUDE, this was
  done for consistency.

* Failing to build ldiskfs should not automatically disable
  all server support.  The zfs osd may still be buildable.

* General m4 cleanup and simplification of lustre-build-zfs.m4.

* Ensure new zfs/spl build correctly with lbuild.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: Ib686211c4f9ace39a41053ce8a20112d1121def9
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-on: http://review.whamcloud.com/5960
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2752 build: Enhance build for cross compilation for MIC

Enhance lustre build for cross compilation for the Intel(R) Xeon
Phi(TM) card. In addition to standard build the GNU cross
toolchain for the Intel(R) Xeon Phi(TM) can be used to produce
client binaries for the Intel(R) Xeon Phi(TM) card. To enable
this just specify appropriate --host and --build option for
./configure.

For example, to produce Lustre client binaries for Intel(R) Xeon
Phi(TM) card just execute the following commands:

NOTE: You should have "intel-mic-gpl-<version>.x86_64" package
installed and MIC GPL sources unpacked in /opt/intel/mic/src.

export PATH=/usr/linux-k1om-4.7/bin:$PATH

sh ./autogen.sh

./configure --with-linux=/opt/intel/mic/src/card/kernel \
    --disable-server --without-o2ib \
    --host=x86_64-k1om-linux --build=x86_64-pc-linux

make

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I4347c65f67bd836116532989c2132457f5eee934
Reviewed-on: http://review.whamcloud.com/5273
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2915 doc: update Lustre doc for LFSCK

Add section to describe the new LFSCK.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I11f1763a1f1527f8a7c0893df0fbce2788679775
Reviewed-on: http://review.whamcloud.com/5913
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-2139 osc: Track and limit "unstable" pages"

This seems to be causing multiple issues: LU-3274, LU-3277

This reverts commit 5661651b2cc6414686e7da581589c2ea0e1f1969.

LU-1095 debug: quiet noisy console error messages

Quiet a number of overly noisy and unhelpful console error
messages.  Improve the format of other nearby errors.

In the case of {lod,lov}_fix_desc_stripe_size(), this doesn't
even need a console message unless it is actually changing
some stripe size that is below the minimum.  Typically it is
only zero and is being bumped up to the default value.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0830e514c426080b8c3446c0a1a359c313b530d9
Reviewed-on: http://review.whamcloud.com/6264
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2193 ofd: check object exists before orphan destroy

MDS replay object destroys after recovery could meet non-existing
objects, skip it before following futile actions.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ibf233a07fa73d3226fdde5e2c020e73f51428f74
Reviewed-on: http://review.whamcloud.com/6266
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3267 utils: missing setting echo seq for getattr/setattr

It should set echo seq before do echo getattr/setattr, otherwise
echo_client will regard it as the object with MDT0 sequence.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1ea6cf0c9ef1edc5a81ba6f50345916deea4f95c
Reviewed-on: http://review.whamcloud.com/6263
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>

LU-2742 oi: handle IGIF lookup

For the MDT upgraded from 1.8 device, OI scrub will immobilize the
IGIF and insert the mapping for "IGIF <=> ino#" into the OI file.
There are three cases to be processed:

1) After the upgrading, all IGIFs have been processed and inserted
   into OI files. Then any IGIF object lookup should be processed
   as normal FID does, means via OI. If no entry in the OI file,
   then the IGIF object does not exist.

2) During the upgrading, some IGIFs may be already in OI files,
   others may be not yet. Under such case, lookup an IGIF in OI
   files may return -ENOENT, but it does not means the IGIF obj
   does not exist. Since the new immobilize IGIF is the same as
   the original one before backup/restore, OSD will generate
   local identifier from IGIF directly as old 2.x did.

3) The upgrading is paused, and backup/restore. Means upgrading
   from 1.8 may be partly processed, but some clients may hold
   some immobilized IGIFs, and use them to access related objects.
   Under such case, OSD does not know whether an given IGIF has
   been processed or to be processed, and it also cannot generate
   local ino#/gen# directly from the immobilized IGIF because of
   the backup/restore. Then force OSD to lookup the given IGIF
   in OI files, and if no entry, then ask the client to retry
   after upgrading completed. No better choice.

Test-Parameters: testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I7fff7898e2fcdf6b075bb04817f3b825c9a911b0
Reviewed-on: http://review.whamcloud.com/6254
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3188 osc: shorten IO calling path

By using osc_io_unplug_aync() for osc_queue_sync_pages() to shorten
the IO calling path, to reduce the chance of stack overflow.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I06112d9fb7b069d68c7945641e65d6430d0bb380
Reviewed-on: http://review.whamcloud.com/6191
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Keith Mannthey <kemannthey@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3245 mdd: Fixed OBF of the FS root

Added a check to prevent fid access to the filesystem root directory
from returning EINVAL.

Added a sanity check accordingly.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iffb8bdb71377b878d165e9e48049c6b2ebd14859
Reviewed-on: http://review.whamcloud.com/6209
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2677 utils: update e2fsprogs to 1.42.7.wc1

Update the required e2fsprogs version to 1.42.7.wc1 to handle
smaller on-disk LMA and filter_fid structure sizes.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id5bcbe1cd1bc2c170e253dbcb0bd0beb1a46c36f
Reviewed-on: http://review.whamcloud.com/6233
Tested-by: Hudson
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3179 fids: fix compilation error with gcc 4.7.2

initialize oi.oi.oi_id which gcc 4.7.2 is afraid of being
used later.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Iaeb6eac01340d80786463efe67ff74479017f074
Reviewed-on: http://review.whamcloud.com/6064
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2442 kernel: SLES11 performance fixes and updates

This patch completes the SLES11 SP2 kernel support. The first problem
solved is the metadata performance slow down due to the heavy dqptr_sem.
So we remove dqptr_sem (but kept in struct quota_info to keep kernel ABI
unchanged), and the functionality of this lock is implemented
by other locks:

  * i_dquot is protected by i_lock, however only this pointer, the
    content of this struct is by dq_data_lock.

  * Q_GETFMT is now protected with dqonoff_mutex instead of dqptr_sem.

The second are small optimizations block level tunable optimizations.
Added in support to simulate fail over for testing purposes. The bulk
of the changes are removal of obsolete SLES10 and syncing SLES11 SP1
kernel side support to 2.6.32.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I474bdd8cf8293d2918273afa074e442aafa28d4c
Reviewed-on: http://review.whamcloud.com/6168
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3208 tests: Fix typo in replay-single/70b

Missing space in if statement, caused syntax error:

replay-single.sh: line 1912 [: missing `]`

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2543f9b8fe41153d6109a64d2619a8443e700d9d
Reviewed-on: http://review.whamcloud.com/6131
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3172 ost: call lustre_msg_get_opc() once inside ost_handle()

lustre_msg_get_opc() is called a few times inside ost_handle();
there is no real benefit from this, and we can just make one call
when entering the function.

There may be other occurrences in the source where this applies,
apart from ost_handle(), but we can at least make this change here
for now, as this is a frequently-called function.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: I7a8badc30ca31cb6826463ae5390cef96dec345f
Xyratex-bug-id: MRP-698
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/6055
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3023 build: fix fuzzy logic in get_root_path()

Thanks to the call to llapi_is_lustre_mnt(), we are sure that
mnt.mnt_fsname contains ":/".
And, if no other mountpoint is found (len ==0), there is no reason
to abort scanning, so the check and -EINVAL can just be removed.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Id77c41fbf53782d657034f152f724a3bdc83408c
Reviewed-on: http://review.whamcloud.com/5832
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-2826 tests: add useful text to error() calls

Many tests just call "error" without any arguments, but this does not
provide any information to Maloo about why the test fails.  This
in turn causes autovet to match the empty failure message to many
different and irrelevant bugs, making Maloo statistics inaccurate.

Change a number of error() calls to have some valid error text, and
print something more useful if error() is called without arguments.
Not all of the error() calls in sanity.sh and sanityn.sh have been
converted, but it is at least a good start at cleaning this up, and
catches the most frequent failure cases currently.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: Ie148e3e26e1ecb888ac021a8df7cd995183ebbe5
Reviewed-on: http://review.whamcloud.com/5750
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-2950 lnet: Add a mechanism to configure routes from a file

Created a bash script lustre_rotues_config which takes in a file
with routes configured in the following format:
<network>: { gateway: <gateway>, [hop:<hop>], [priority: <prio>] }
The script shall parse the file and generate:
lctl --net <network> add_route <gateway> [hop [priority]]
for each route.
The script can be used to unconfigure routes as well by running it
as follows:

   lustre_routes_config --cleanup <file>

In this case it will remove all routes configures via lctl del_route.

Also added another script: lustre_routes_conversion, which will be
used to convert from legacy syntax for configuring routes to the
new syntax described above. The script can be run on a file which
contains the legacy syntax and will generate a new file with the
passed in name:

   lustre_routes_conversion <legacy file> <new file>

Added two man pages to describe the usages of these scripts

Added a test case in conf-sanity.sh to test the new scripts
lustre_routes_conversion and lustre_routes_config. The test case
takes in a sample routes file which has the old synatx, run the
lustre_routes_conversion script to convert to the new syntax, then
runs the lustre_routes_config script in --dry-run to ensure that
the script configures the routes

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I57f81176c2d926fedefa8ea3be34586aa1ac9d76
Reviewed-on: http://review.whamcloud.com/5757
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1199 build: Remove unused LB_LDISKFS_RELEASE macros

The products of the LB_LDISKFS_RELEASE macros are unused.  They
also make some bad assumtions that complicate building against
a future lustre-devel package.  So we remove them.

If Lustre needs to know ldiskfs's version in the future, we
should add explicit version defines to the new ldiskfs_config.h.in
header that is in the works (or possibly landed by now).

Change-Id: I40b96dc72f5076ba9a9dda3f41b31088fdfd4341
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5881
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-2677 utils: fix swabbing of f_oid

In ll_recover_lost_found_objs it is using le64_to_cpu() to swab
the f_oid field, which is a 32-bit value.  Swab it correctly using
le32_to_cpu().

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2679b49dc16351140dbd1170b17056489c3ebbe5
Reviewed-on: http://review.whamcloud.com/6159
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3239 ofd: refill env in ofd_get_info

Because ofd_get_info(KEY_FIEMAP) might be called from
ptlrpc_server_handle_req_in(see the stack below),
where env might not be initialized correctly(see LBUG below),
so it refill refill in ofd_get_info.

LutreError: 19182:0:(ofd_internal.h:518:ofd_info_init()) LBUG
Pid: 19182, comm: ll_ost_io00_001
Call Trace:
[<ffffffffa044e895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
[<ffffffffa044ee97>] lbug_with_loc+0x47/0xb0 [libcfs]
[<ffffffffa0e03e62>] ofd_info_init+0x92/0x130 [ofd]
[<ffffffffa0e05835>] ofd_get_info+0x2e5/0xa90 [ofd]
[<ffffffff812805cd>] ? pointer+0x8d/0x830
[<ffffffffa029f7e5>] ? lprocfs_counter_add+0x125/0x182 [lvfs]
[<ffffffffa078528a>] nrs_orr_range_fill_physical+0x18a/0x540
[ptlrpc]
[<ffffffffa0762dd6>] ? __req_capsule_get+0x166/0x700 [ptlrpc]
[<ffffffffa073e630>] ? lustre_swab_ost_body+0x0/0x10 [ptlrpc]
[<ffffffffa07871d7>] nrs_orr_res_get+0x817/0xb80 [ptlrpc]
[<ffffffffa077d306>] nrs_resource_get+0x56/0x110 [ptlrpc]
[<ffffffffa077dccb>] nrs_resource_get_safe+0x8b/0x100 [ptlrpc]
[<ffffffffa0780248>] ptlrpc_nrs_req_initialize+0x38/0x90 [ptlrpc]
[<ffffffffa074cff0>] ptlrpc_main+0x1170/0x16f0 [ptlrpc]
[<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc]
[<ffffffff8100c0ca>] child_rip+0xa/0x20
[<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc]
[<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc]
[<ffffffff8100c0c0>] ? child_rip+0x0/0x20

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Iee4b68fe331a895c61e6ccd0a14d6c60f5f9215c
Reviewed-on: http://review.whamcloud.com/6204
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3030 build: Update Master Copyrights pre 2.4 split

This is the output of the tool. Please note the tool
changed to using commit dates over author dates and
there are a few spots with erroneous 2011 copyrights
that have been removed by the tool.

Please see script update for further details.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I6600369df53c01c425f33f62d5d6c4f8f1b48498
Reviewed-on: http://review.whamcloud.com/5841
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3030 build: Update Copyright Script

Add pevious mass copyright update and cfs wrapper changes
to the list of excluded patches.

Many files have incorrect author date set, revert to
using the git committer date for the patch date.

--follow and --author were not working properly.
Screen for Author after the git log call.  Git log
call code came via Andreas Dilger.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I46cf74b48095b54ac58dc17b6b0f94c9cf23cb8a
Reviewed-on: http://review.whamcloud.com/6183
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-2154 osp: osd-zfs: remove 2.3.x debugging code

When LUSTRE_VERSION_CODE is increased beyond 2.3.90 the version
based debug code generates a compile warning and fails to build.
Originally added in commit 04e1d0cb95e1ad12 and 2acb75c36511aca9.

Since we have been running with these checks for months without
problem, and it will soon be inactivated anyway, we may as well
remove the code entirely instead of just fixing it.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1c104ef388f112c619d4ff4b7f00d17383500c1e
Reviewed-on: http://review.whamcloud.com/6219
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3160 clio: don't ignore layout on writeback

In some cases such as kernel writeback, we shouldn't ignore the
layout, otherwise, it could race with layout change undergoing.

Test-Parameters: envdefinitions=DURATION=7200  clientdistro=el6 serverdistro=el6 clientcount=4  osscount=2 mdscount=2 austeroptions=-R failover=true  useiscsi=true testlist=recovery-random-scale
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ib9d0aa581de90711c92db4c631c52f1950ad5b67
Reviewed-on: http://review.whamcloud.com/6154
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3244 utils: tunefs.lustre should preserve virgin label

so that the filesystem registers can register properly on MGS,
if tunefs.lustre was used right after mkfs.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6245e5e4d10cd0a13a4e9068a9b758da8580b537
Reviewed-on: http://review.whamcloud.com/6216
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1606 api: compile and build lustreapi test

To avoid regression of the client api conf-sanity includes a new test
73 to verify the lustreapi can be compiled and linked against. All the
files in the directory $LUSTRE_TESTS_API_DIR with the extension .c are
compiled and linked. The directory is located in the build tree at
./lustre/tests/clientapi.

Signed-off-by: Richard Henwood <richard.henwood@intel.com>
Change-Id: I0ad6a1671bf7033ec2ad5bc7a82d82e468cd31c2
Reviewed-on: http://review.whamcloud.com/3440
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 build: Increase ldiskfs version to 4.1.0

The ldiskfs version number has not changed in quite some time.

This brings the versioning in line with what LLNL has been using
externally, and allows us to realign on the same code.

Add an explicit requirement for "lustre-ldiskfs >= 4.1.0"
to the lustre spec file at the same time.

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I72c9f52093942d881ee02dad8f65c2f04dbef35e
Reviewed-on: http://review.whamcloud.com/5938
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-2038 osd: Rename do_*punch to dbo_*punch

Minor fix where dt_body_operations::do_punch and
dt_body_operations::do_declare_punch were renamed to
dt_body_operations::dbo_punch and
dt_body_operations::dbo_declare_punch, respectively, to keep the
field names consistent.

Signed-off-by: Girish Shilamkar <gshilamkar@ddn.com>
Change-Id: Id807805bd69bb20460898552830a658d8a25c238
Reviewed-on: http://review.whamcloud.com/5895
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>