LU-3164 utils: Fix Wireshark decode of Lustre under Infiniband

Due to a difference in the IB LND vs TCP LND header sizes, some
hard-coded offsets in the LNet dissector will not believe that the
payload is lustre.  As such, the lustre protocol may not be decoded
under Infiniband.

This patch addresses this by adding a variable to applicable offsets
which is set to 0 for TCP, and 24 (extra bytes in IB LND header)
for Infiniband.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I3783bbb563df109e160752d246db9bc5148cf132
Reviewed-on: http://review.whamcloud.com/6045
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3300 lprocfs: interpret result of dt_statfs() correctly

In a7369bcd860af61a073cb2424139e3acccdcb203, I accidentally reversed
the sense of the error check after the call to dt_statfs() in
lprocfs_dt_rd_{blksize,{files,kbytes}{free,avail}. Unreverse the error
checking.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I91873f23cd69c2217474c26b9751aec259fae155
Reviewed-on: http://review.whamcloud.com/6385
Reviewed-by: Robert Read <robert.read@intel.com>
Tested-by: Hudson
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3354 kernel: Kernel update [RHEL6.4 2.6.32-358.6.2.el6]

Update RHEL6.4 kernel to 2.6.32-358.6.2.el6.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5c6b48d82382974b3cdc8a0c84c5ef54194d5217
Reviewed-on: http://review.whamcloud.com/6379
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3306 mdt: Do not LBUG on double set of err_serious

It seems the twisted code pathes in mdt could take strange routes
and it's quite hard to guarantee they won't pass through setting
err_serious twice, esp. since error handlers are traditionally
poorly tested.

Change-Id: I88fe880c22670263273bd79073efaecbe53d9215
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/6383
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-3318 llite: Always build 64bit ino internally

Always build 64bit ino internally except the client is mounted
with "32bitapi" option, so client will always use 64bit ino
internally. It will build 32bit ino, only if application requires
32 bit ino.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I717e0551a7df4f8a7f02168702d8c9c58ecafd3d
Reviewed-on: http://review.whamcloud.com/6371
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-3264 tests: import ZFS pool which has file-based vdev

For "zpool import" command, if the -d option is not specified,
the command will only search for devices in "/dev". However,
for the pool which was created with file-based virtual device,
we need explicitly specify the search directory otherwise the
import command will not find the device.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I63ac141fdefcace99a1cf7c366e15f0067fa2a56
Reviewed-on: http://review.whamcloud.com/6358
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.4.50

After branching off b2_4, master becomes 2.5.0 development branch.

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I1c8207394016d793a64f57d410d33c43ac96b34c

LU-3292 build: kernel update for 3.0.74-0.6.6 sles11sp2

Delete base and ldiskfs patches that are now obsolete
in the latest kernel update version for SLES11 SP2.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I9a5ef1347afb228453db79b213e3881c6a112938
Reviewed-on: http://review.whamcloud.com/6293
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3036 mdt: set ATTR_xTIME_SET to make atime update properly

To make atime update properly between 1.8 client and 2.x server,
this patch includes the following fixes:
- if MDS_ATTR_xTIME is set without MDS_ATTR_xTIME_SET and the client
  does not have OBD_CONNECT_FULL20, convert it to LA_xTIME in
  mdt_setattr_unpack().
- set both MDS_ATTR_xTIME | MDS_ATTR_xTIME_SET for timestamps in
  ll_prepare_close(). This allows us to fix the server-side timestamp
  setting in the future.
- remove attr_unpack() and convert the flags from MDS_ATTR_ to LA_*
  directly in mdt_attr_valid_xlate() instead.
- improve sanityn.sh test_23().

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I9454d7689243a268ebd4f40566f77e4f53996d5f
Reviewed-on: http://review.whamcloud.com/6327
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3279 changelog: fix CL_LAYOUT comment, string

Fix the CL_LAYOUT comment, since it is possible that layout swap
could be used in cases where the file content is changed.  I don't
think there are any real world use cases for this, but that depends
on what tools are being used on the filesystem.

Also change the "LAYOUT" string for this ChangeLog record type to
be "LYOUT" to match the convention of other strings to only be
5 characters long.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ibf9a57216e0c45a1e15cef9cfe5ee7f509500c1e
Reviewed-on: http://review.whamcloud.com/6338
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3328 osp: Only signal thread completion when we are done

IT seems that by signalling the completion of a thread before
we are done inspecting it's memory is racy and could cause access
to freed memory.

Change-Id: Ia992e9c7d0d78fca1afbfc98f51c5df3f6dc3a28
Signed-off-by: Oleg Drokin <green@intel.com>
Reviewed-on: http://review.whamcloud.com/6329
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2915 lfsck: NO scrub for agent inode and remote parent

Under DNE case, for the object on remote MDT, there will be an agent
inode on the local MDT where the name entry resides. We do NOT build
OI mapping for such agent inode, so OI scrub should skip it.The same
for the "/REMOTE_PARENT_DIR".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I164473bcba80043800d491639c027e7c13f30bd9
Reviewed-on: http://review.whamcloud.com/6332
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3219 ost: Ensure dirty flushed on fiemap ioctl

Data corruption is possible if cp(coreutils) uses FIEMAP to obtain
data holes in shared file, since there could be dirty cache on
other clients which hasn't been flushed back.

To ensure all the dirty on remote clients being flushed back on
fiemap ioctl, we'd acquire ldlm lock on server side for fiemap,
unless the local client (which invoke fiemap) has cached lock.

Xyratex-bug-id: MRP-1001
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I67fe05bac1ce75f98c3d3806a4d08f6cb02048e6
Reviewed-on: http://review.whamcloud.com/6127
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3279 changelog: fix CL_LAYOUT, accept all types

In order to avoid compatibility issues with older ChangeLog consumers,
change the new CL_LAYOUT record to use the value previously assigned
to CL_IOCTL.  The CL_IOCTL type was never used anywhere, and it didn't
really make any sense as a ChangeLog record, since it could really
mean anything at all.

The changelog_show_cb() function is renamed to changelog_kkuc_cb(),
since it is really about consuming the ChangeLog and passing it up to
the kernel-user-coms interface.  At some point we should consider to
implement a DBUS-based ChangeLog interface as well.

The changelog_kkuc_cb() "sanity check" on cr_type was removed, since
there is no reason the client kernel needs to know every record type
that is being passed to userspace.  It is up to the client tool to
determine what records that it can process.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id8e6471827a89d8c62808e6b3cfd9693d6026431
Reviewed-on: http://review.whamcloud.com/6308
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2979 lprocfs: use stats counter index for *pos

In lprocfs_stats_seq_{start,next,show,stop}() encode the counter index
(rather than the counter address) into *pos. Doing so simplifies these
functions and fixes a bug in the case of per-CPU stats where no stats
would be displayed at all if no events had yet occurred on CPU 0.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Id50f8ae480e151f8e16382db27eef8025ef69f15
Reviewed-on: http://review.whamcloud.com/6328
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3269 build: Remove circular rpm dependency

The rpm package dependencies look a bit like this currently:

 lustre -> lustre-modules <-> lustre-osd-[ldiskfs|zfs] (via lustre-osd)

Note that the dependency between lustre-modules and lustre-osd-* goes both
ways.

We did that in the one direction because we need _something_ to
depend on the generic "Provides: lustre-osd" when we have server support
compiled in, so users will be clued in to the fact that they need to
select at least one of the lustre-osd-* packages.

In the other direction, the lustre-osd-* packages need to depend on
lustre-modules because the kernel modules contained in the lustre-osd-*
rpms really do use symbols from modules in the lustre-modules rpm.

RPM allows this circular dependency, but because of it the modules are not
necessarily installed in the order that we prefere.  Namely:

1) lustre
2) lustre-modules
3) lustre-osd-*

It can happen that the lustre-osd-* packages are installed before the
lustre-modules package, and then the sysadmin will see a screen full of
scary messages about missing symbols.

To fix that problem, we move the dependency on lustre-osd up one package
into the "lustre" package.  That makes the dependency graph look a bit
like:

   lustre -> lustre-modules
        |--> lustre-osd-[ldiskfs|zfs] (via lustre-osd)
   lustre-osd-[ldiskfs|zfs] -> lustre-modules

I by breaking the dependency loop, we get the desired package installation
order described above.

Change-Id: I7bab1d536e75ce4cf3eb70fa4569c435ec8b2d66
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/6259
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3264 tests: support failover testing with ZFS

This patch improves test-framework.sh to support failover testing with
ZFS under shared storage configuration.

After formatting a ZFS target, "cachefile=none" property will be set
on the ZFS storage pool so that the pool is not automatically imported
on system startup. And then the pool will be exported so as to leave
the import and export operations handled by mount_facet() and stop().

For SOFT failure mode (unmount and mount), "zpool export …" will be
run on the active server node to export the ZFS storage pool, and
"zpool import -f …" will be run on the failover partner node to import
the pool.

For HARD failure mode (power off and on), since the "cachefile=none"
property has been set on the ZFS storage pool on the active server
node, after the node is power-cycled, the pool will not be imported.
Then on the failover partner node, "zpool import -f …" can be run to
import the pool.

Test-Parameters: envdefinitions=PTLDEBUG=-1,DEBUG_SIZE=128 \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
clientdistro=el6 serverdistro=el6 clientarch=x86_64 \
serverarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
austeroptions=-R failover=true useiscsi=true \
testlist=recovery-double-scale

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I6b9646cb36f7fa4924af03a055ccb137187843a5
Reviewed-on: http://review.whamcloud.com/6258
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3314 scrub: use special fixed FID for .lustre

For lustre-2.x (x <= 3), the ".lustre" has NO FID-in-LMA,
so the client will get IGIF for the ".lustre" object when
the MDT restart.

From the OI scrub view, when the MDT upgrade to Lustre-2.4,
it does not know whether there are some old clients cached
the ".lustre" IGIF during the upgrading. Two choices:

1) Generate IGIF-in-LMA and IGIF-in-OI for the ".lustre".
   It will allow the old connected clients to access the
   ".lustre" with cached IGIF. But it will cause others
   on the MDT failed to check "fid_is_dot_lustre()".

2) Use fixed FID {FID_SEQ_DOT_LUSTRE, FID_OID_DOT_LUSTRE, 0}
   for ".lustre" in spite of whether there are some clients
   cached the ".lustre" IGIF or not. It enables the check
   "fid_is_dot_lustre()" on the MDT, although it will cause
   that the old connected clients cannot access the ".lustre"
   with the cached IGIF.

Usually, it is rare case for the old connected clients
to access the ".lustre" with cached IGIF. So we prefer
to the solution 2).

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifa491850ddab0de0b67aab124dc206ad4f714428
Reviewed-on: http://review.whamcloud.com/6309
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3187 ost: check pre 2.4 echo client in obdo validation

Because old echo client still uses o_id/o_seq for objid,
but new echo client will uses FID for the objid. Add
OBD_CONNECT_FID for 2.4 echo client, so 2.4 OST will
convert o_id/o_seq to FID if the request from old echo
client.

Add local flag OBD_FL_OSTID for o_flags to indicate
OST does not support FID yet, then echo client will
still send o_id/o_seq to OST.

cleanup ost_validate_obdo

Test-Parameters: clientjob=lustre-b2_1 clientbuildno=197 testlist=sanity,obdfilter-survey
Test-Parameters: serverjob=lustre-b2_1 serverbuildno=197 testlist=sanity,obdfilter-survey
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6001c813b668cf53a66d0d9d74f322bad63765ed
Reviewed-on: http://review.whamcloud.com/6287
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-3303 test: sanity-quota test_18 incomplete error message

test_18 is sending the error message to quota_error with
two string literals with no line continuation. quota_error
essentially ignores the second string literal.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I54e5cccf307349fab072d6fac2832f6bd98a3bfd
Reviewed-on: http://review.whamcloud.com/6302
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3100 tests: Skip recovery-small test_111

Recovery-small test_111 now checks the metadata server version and
will run the test for versions 2.3.62 and above. The test will be
skipped for all other server versions.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I58aa58dd7cd8a27209fb1c4ee258de9d3a84b144
Reviewed-on: http://review.whamcloud.com/5975
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

Revert "LU-3188 osc: shorten IO calling path"

This commit seems to have caused more problems than it fixed.

This reverts commit 83ae17df2bdce837e62473aec27c03d67312c8ea.

LU-3302 llog: Do not use ostid swab for llogid

Since logid still use id/seq format in the request,
it will be swabbed by its own swab func, instead of
using ostid swab, which might see logid as FID incorrectly.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I35f554e99a765384218ee3a357c931d6be1de6b3
Reviewed-on: http://review.whamcloud.com/6305
Tested-by: Hudson
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2886 scrub: non-fixed FIDs for some local files

For old lustre-2.x (x < 4), the local files PENDING/lfsck_bookmark
used fixed/reserved FIDs when created. The cases have been changed
to use variable local FIDs since lustre-2.4. So adjust OI scrub to
match the changes.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6d1f2f19afa3d777b16838ae03f46a39a58b537a
Reviewed-on: http://review.whamcloud.com/6299
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3301 utils: Replace %llu with LPU64

Replace incorrect use of %llu with LPU64 to allow compilation
on ppc64.

Change-Id: I85794527a43ed1577cf43ddb3470a9c8d070f11b
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/6296
Reviewed-by: Bobi Jam <bobijam.xu@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2886 obdclass: use common way to store lastid

Local files last id are stored in root in files named seq-xxx-lastid
while lastid for OST objects is stored in O/seq/LAST_ID special
object with zero OID and handled by OSD.
Patch reworks local files lastid to be stored in O/seq/LAST_ID too
and using the same format.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I72710e84cf0f7a0903b0b88ac3f9432eb59ea716
Reviewed-on: http://review.whamcloud.com/6199
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-3271 lov: only dump header in lsm_lmm_verify

If lsm_lmm_verify_*() find an error in the lov_mds_md header
structure, don't dump the full stripe information, since this
can be totally bogus (e.g. if stripe_count == -1 or similar).
Instead, just dump the header information for debugging.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Idf8c8bc35b156181aff9f0c5f0ea1f73c89e33d1
Reviewed-on: http://review.whamcloud.com/6261
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2886 obdclass: remove obsoleted md_local_file.c

This library is not used anymore and is replaced by
local_storage.c. Patch removed last remnants of it.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I4d1371908db898efc8c5cd650357c449f369f51b
Reviewed-on: http://review.whamcloud.com/6107
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-3281 osc: some cleanup to reduce stack overflow chance

ptlrpcd_add_req() will wake_up other process, do not hold a spinlock
before calling ptlrpcd_queue_work()->ptlrpcd_add_req().

If current process is allocating memory, memory shrinker could get to
osc_lru_del(), don't call osc_lru_shrink() further since it could
lead a long calling chain.

Use static string OES_STRINGS in OSC_EXTENT_DUMP() to reduce stack
footprint.

Alloc crattr on heap for osc_build_rpc() to reduce stack footprint.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I1a1ce0b46850773a2ae45ce16be6b708adc40ab8
Reviewed-on: http://review.whamcloud.com/6270
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3280 ldlm: suppress useless lock RPC for layout

In ldlm_lock_decref_internal() when l_lvb_data is freed to
reduce memory consumption, LDLM_FL_LVB_READY is not
cleared, so later when the lock is reused lvb is not
updated. But clearing LDLM_FL_LVB_READY forces layout refetch
at each file access, so the better is to remove the optimization.
The use case is after a restore in HSM.

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I3aa56cf39fe34941d227400410b27db32479b1b1
Reviewed-on: http://review.whamcloud.com/6268
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3190 mdd: not return linkEA for dead obj

1. Not return linkEA for dead object.
2. Check lma_self_fid to match object FID after get
real LMA from the object.
3. clear oi cache during oi delete.
4. correct error value if ldlm_handle_enqueue return
some value other than ldlm_err_t(for example -ESTALE).

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1d2345eb01ff58584ffba31f86bb408396780aeb
Reviewed-on: http://review.whamcloud.com/6252
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

New tag 2.3.65

Change-Id: I65f1eca5083842566443fd577e5a88f1dc57e88e

LU-3275 osd-ldiskfs: re-order osd device init/fini

There was race condition between the background OI scrub thread and
osd_device_init0: OI scrub thread may try to access non-initialized
osd_device::od_ost_map. So the osd initization/start process should
NOT trigger OI scrub until all the OI (OI files, /O, and ect.) have
been initialized. Reverse ordre for osd divice fini.

Test-Parameters: testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I99a2a72ace947a0f79ace5c6f445cf896d884e63
Reviewed-on: http://review.whamcloud.com/6267
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3117 build: zfs-0.6.1 kmod+dkms compatibility

With the release of zfs-0.6.1 the default install location of
the zfs kmod headers and objects has changed.  The kmod headers
which are common for a zfs version are now installed under
/usr/src/zfs-<version>/ path.  The objects, which are kernel
specific, are installed under /usr/src/zfs-<version>/<kernel>/.

This was done just prior to the official 0.6.1 release because
this scheme satisfies the packaging requirements of the major
distributions.  Making the change now means we shouldn't need
to change it again.

To accomidate this change the lustre-build-zfs.m4 has been
updated in the following ways:

* The new zfs header and object paths were added to the list
  of default search paths.  The DKMS build paths were also added
  to allow compilation against zfs-kmod or zfs-dkms packages.

* Support for building the spl and zfs code recursively as
  part of the Lustre build process was removed.

* The lustre-osd-zfs packages 'Requires' line was changed to
  require zfs-kmod.  Either the zfs-kmod or zfs-dkms packages
  can be used to satisfy this requirement.

* Fix incorrect usage of @ZFS_OBJ@ in osd-zfs/Makefile.in,
  the include directory us under @ZFS@ with the headers.
  These happens to be the same location before so it never
  caused issues.

* EXTRA_LIBZFS_INCLUDE renamed ZFS_LIBZFS_INCLUDE, this was
  done for consistency.

* Failing to build ldiskfs should not automatically disable
  all server support.  The zfs osd may still be buildable.

* General m4 cleanup and simplification of lustre-build-zfs.m4.

* Ensure new zfs/spl build correctly with lbuild.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: Ib686211c4f9ace39a41053ce8a20112d1121def9
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-on: http://review.whamcloud.com/5960
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2752 build: Enhance build for cross compilation for MIC

Enhance lustre build for cross compilation for the Intel(R) Xeon
Phi(TM) card. In addition to standard build the GNU cross
toolchain for the Intel(R) Xeon Phi(TM) can be used to produce
client binaries for the Intel(R) Xeon Phi(TM) card. To enable
this just specify appropriate --host and --build option for
./configure.

For example, to produce Lustre client binaries for Intel(R) Xeon
Phi(TM) card just execute the following commands:

NOTE: You should have "intel-mic-gpl-<version>.x86_64" package
installed and MIC GPL sources unpacked in /opt/intel/mic/src.

export PATH=/usr/linux-k1om-4.7/bin:$PATH

sh ./autogen.sh

./configure --with-linux=/opt/intel/mic/src/card/kernel \
    --disable-server --without-o2ib \
    --host=x86_64-k1om-linux --build=x86_64-pc-linux

make

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I4347c65f67bd836116532989c2132457f5eee934
Reviewed-on: http://review.whamcloud.com/5273
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2915 doc: update Lustre doc for LFSCK

Add section to describe the new LFSCK.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I11f1763a1f1527f8a7c0893df0fbce2788679775
Reviewed-on: http://review.whamcloud.com/5913
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-2139 osc: Track and limit "unstable" pages"

This seems to be causing multiple issues: LU-3274, LU-3277

This reverts commit 5661651b2cc6414686e7da581589c2ea0e1f1969.

LU-1095 debug: quiet noisy console error messages

Quiet a number of overly noisy and unhelpful console error
messages.  Improve the format of other nearby errors.

In the case of {lod,lov}_fix_desc_stripe_size(), this doesn't
even need a console message unless it is actually changing
some stripe size that is below the minimum.  Typically it is
only zero and is being bumped up to the default value.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0830e514c426080b8c3446c0a1a359c313b530d9
Reviewed-on: http://review.whamcloud.com/6264
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2193 ofd: check object exists before orphan destroy

MDS replay object destroys after recovery could meet non-existing
objects, skip it before following futile actions.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ibf233a07fa73d3226fdde5e2c020e73f51428f74
Reviewed-on: http://review.whamcloud.com/6266
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3267 utils: missing setting echo seq for getattr/setattr

It should set echo seq before do echo getattr/setattr, otherwise
echo_client will regard it as the object with MDT0 sequence.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1ea6cf0c9ef1edc5a81ba6f50345916deea4f95c
Reviewed-on: http://review.whamcloud.com/6263
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>

LU-2742 oi: handle IGIF lookup

For the MDT upgraded from 1.8 device, OI scrub will immobilize the
IGIF and insert the mapping for "IGIF <=> ino#" into the OI file.
There are three cases to be processed:

1) After the upgrading, all IGIFs have been processed and inserted
   into OI files. Then any IGIF object lookup should be processed
   as normal FID does, means via OI. If no entry in the OI file,
   then the IGIF object does not exist.

2) During the upgrading, some IGIFs may be already in OI files,
   others may be not yet. Under such case, lookup an IGIF in OI
   files may return -ENOENT, but it does not means the IGIF obj
   does not exist. Since the new immobilize IGIF is the same as
   the original one before backup/restore, OSD will generate
   local identifier from IGIF directly as old 2.x did.

3) The upgrading is paused, and backup/restore. Means upgrading
   from 1.8 may be partly processed, but some clients may hold
   some immobilized IGIFs, and use them to access related objects.
   Under such case, OSD does not know whether an given IGIF has
   been processed or to be processed, and it also cannot generate
   local ino#/gen# directly from the immobilized IGIF because of
   the backup/restore. Then force OSD to lookup the given IGIF
   in OI files, and if no entry, then ask the client to retry
   after upgrading completed. No better choice.

Test-Parameters: testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I7fff7898e2fcdf6b075bb04817f3b825c9a911b0
Reviewed-on: http://review.whamcloud.com/6254
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3188 osc: shorten IO calling path

By using osc_io_unplug_aync() for osc_queue_sync_pages() to shorten
the IO calling path, to reduce the chance of stack overflow.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I06112d9fb7b069d68c7945641e65d6430d0bb380
Reviewed-on: http://review.whamcloud.com/6191
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Keith Mannthey <kemannthey@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3245 mdd: Fixed OBF of the FS root

Added a check to prevent fid access to the filesystem root directory
from returning EINVAL.

Added a sanity check accordingly.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iffb8bdb71377b878d165e9e48049c6b2ebd14859
Reviewed-on: http://review.whamcloud.com/6209
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2677 utils: update e2fsprogs to 1.42.7.wc1

Update the required e2fsprogs version to 1.42.7.wc1 to handle
smaller on-disk LMA and filter_fid structure sizes.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id5bcbe1cd1bc2c170e253dbcb0bd0beb1a46c36f
Reviewed-on: http://review.whamcloud.com/6233
Tested-by: Hudson
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3179 fids: fix compilation error with gcc 4.7.2

initialize oi.oi.oi_id which gcc 4.7.2 is afraid of being
used later.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Iaeb6eac01340d80786463efe67ff74479017f074
Reviewed-on: http://review.whamcloud.com/6064
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2442 kernel: SLES11 performance fixes and updates

This patch completes the SLES11 SP2 kernel support. The first problem
solved is the metadata performance slow down due to the heavy dqptr_sem.
So we remove dqptr_sem (but kept in struct quota_info to keep kernel ABI
unchanged), and the functionality of this lock is implemented
by other locks:

  * i_dquot is protected by i_lock, however only this pointer, the
    content of this struct is by dq_data_lock.

  * Q_GETFMT is now protected with dqonoff_mutex instead of dqptr_sem.

The second are small optimizations block level tunable optimizations.
Added in support to simulate fail over for testing purposes. The bulk
of the changes are removal of obsolete SLES10 and syncing SLES11 SP1
kernel side support to 2.6.32.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I474bdd8cf8293d2918273afa074e442aafa28d4c
Reviewed-on: http://review.whamcloud.com/6168
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3208 tests: Fix typo in replay-single/70b

Missing space in if statement, caused syntax error:

replay-single.sh: line 1912 [: missing `]`

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2543f9b8fe41153d6109a64d2619a8443e700d9d
Reviewed-on: http://review.whamcloud.com/6131
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3172 ost: call lustre_msg_get_opc() once inside ost_handle()

lustre_msg_get_opc() is called a few times inside ost_handle();
there is no real benefit from this, and we can just make one call
when entering the function.

There may be other occurrences in the source where this applies,
apart from ost_handle(), but we can at least make this change here
for now, as this is a frequently-called function.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: I7a8badc30ca31cb6826463ae5390cef96dec345f
Xyratex-bug-id: MRP-698
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/6055
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3023 build: fix fuzzy logic in get_root_path()

Thanks to the call to llapi_is_lustre_mnt(), we are sure that
mnt.mnt_fsname contains ":/".
And, if no other mountpoint is found (len ==0), there is no reason
to abort scanning, so the check and -EINVAL can just be removed.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Id77c41fbf53782d657034f152f724a3bdc83408c
Reviewed-on: http://review.whamcloud.com/5832
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-2826 tests: add useful text to error() calls

Many tests just call "error" without any arguments, but this does not
provide any information to Maloo about why the test fails.  This
in turn causes autovet to match the empty failure message to many
different and irrelevant bugs, making Maloo statistics inaccurate.

Change a number of error() calls to have some valid error text, and
print something more useful if error() is called without arguments.
Not all of the error() calls in sanity.sh and sanityn.sh have been
converted, but it is at least a good start at cleaning this up, and
catches the most frequent failure cases currently.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: Ie148e3e26e1ecb888ac021a8df7cd995183ebbe5
Reviewed-on: http://review.whamcloud.com/5750
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-2950 lnet: Add a mechanism to configure routes from a file

Created a bash script lustre_rotues_config which takes in a file
with routes configured in the following format:
<network>: { gateway: <gateway>, [hop:<hop>], [priority: <prio>] }
The script shall parse the file and generate:
lctl --net <network> add_route <gateway> [hop [priority]]
for each route.
The script can be used to unconfigure routes as well by running it
as follows:

   lustre_routes_config --cleanup <file>

In this case it will remove all routes configures via lctl del_route.

Also added another script: lustre_routes_conversion, which will be
used to convert from legacy syntax for configuring routes to the
new syntax described above. The script can be run on a file which
contains the legacy syntax and will generate a new file with the
passed in name:

   lustre_routes_conversion <legacy file> <new file>

Added two man pages to describe the usages of these scripts

Added a test case in conf-sanity.sh to test the new scripts
lustre_routes_conversion and lustre_routes_config. The test case
takes in a sample routes file which has the old synatx, run the
lustre_routes_conversion script to convert to the new syntax, then
runs the lustre_routes_config script in --dry-run to ensure that
the script configures the routes

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I57f81176c2d926fedefa8ea3be34586aa1ac9d76
Reviewed-on: http://review.whamcloud.com/5757
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1199 build: Remove unused LB_LDISKFS_RELEASE macros

The products of the LB_LDISKFS_RELEASE macros are unused.  They
also make some bad assumtions that complicate building against
a future lustre-devel package.  So we remove them.

If Lustre needs to know ldiskfs's version in the future, we
should add explicit version defines to the new ldiskfs_config.h.in
header that is in the works (or possibly landed by now).

Change-Id: I40b96dc72f5076ba9a9dda3f41b31088fdfd4341
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5881
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-2677 utils: fix swabbing of f_oid

In ll_recover_lost_found_objs it is using le64_to_cpu() to swab
the f_oid field, which is a 32-bit value.  Swab it correctly using
le32_to_cpu().

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2679b49dc16351140dbd1170b17056489c3ebbe5
Reviewed-on: http://review.whamcloud.com/6159
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3239 ofd: refill env in ofd_get_info

Because ofd_get_info(KEY_FIEMAP) might be called from
ptlrpc_server_handle_req_in(see the stack below),
where env might not be initialized correctly(see LBUG below),
so it refill refill in ofd_get_info.

LutreError: 19182:0:(ofd_internal.h:518:ofd_info_init()) LBUG
Pid: 19182, comm: ll_ost_io00_001
Call Trace:
[<ffffffffa044e895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
[<ffffffffa044ee97>] lbug_with_loc+0x47/0xb0 [libcfs]
[<ffffffffa0e03e62>] ofd_info_init+0x92/0x130 [ofd]
[<ffffffffa0e05835>] ofd_get_info+0x2e5/0xa90 [ofd]
[<ffffffff812805cd>] ? pointer+0x8d/0x830
[<ffffffffa029f7e5>] ? lprocfs_counter_add+0x125/0x182 [lvfs]
[<ffffffffa078528a>] nrs_orr_range_fill_physical+0x18a/0x540
[ptlrpc]
[<ffffffffa0762dd6>] ? __req_capsule_get+0x166/0x700 [ptlrpc]
[<ffffffffa073e630>] ? lustre_swab_ost_body+0x0/0x10 [ptlrpc]
[<ffffffffa07871d7>] nrs_orr_res_get+0x817/0xb80 [ptlrpc]
[<ffffffffa077d306>] nrs_resource_get+0x56/0x110 [ptlrpc]
[<ffffffffa077dccb>] nrs_resource_get_safe+0x8b/0x100 [ptlrpc]
[<ffffffffa0780248>] ptlrpc_nrs_req_initialize+0x38/0x90 [ptlrpc]
[<ffffffffa074cff0>] ptlrpc_main+0x1170/0x16f0 [ptlrpc]
[<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc]
[<ffffffff8100c0ca>] child_rip+0xa/0x20
[<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc]
[<ffffffffa074be80>] ? ptlrpc_main+0x0/0x16f0 [ptlrpc]
[<ffffffff8100c0c0>] ? child_rip+0x0/0x20

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Iee4b68fe331a895c61e6ccd0a14d6c60f5f9215c
Reviewed-on: http://review.whamcloud.com/6204
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3030 build: Update Master Copyrights pre 2.4 split

This is the output of the tool. Please note the tool
changed to using commit dates over author dates and
there are a few spots with erroneous 2011 copyrights
that have been removed by the tool.

Please see script update for further details.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I6600369df53c01c425f33f62d5d6c4f8f1b48498
Reviewed-on: http://review.whamcloud.com/5841
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3030 build: Update Copyright Script

Add pevious mass copyright update and cfs wrapper changes
to the list of excluded patches.

Many files have incorrect author date set, revert to
using the git committer date for the patch date.

--follow and --author were not working properly.
Screen for Author after the git log call.  Git log
call code came via Andreas Dilger.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I46cf74b48095b54ac58dc17b6b0f94c9cf23cb8a
Reviewed-on: http://review.whamcloud.com/6183
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-2154 osp: osd-zfs: remove 2.3.x debugging code

When LUSTRE_VERSION_CODE is increased beyond 2.3.90 the version
based debug code generates a compile warning and fails to build.
Originally added in commit 04e1d0cb95e1ad12 and 2acb75c36511aca9.

Since we have been running with these checks for months without
problem, and it will soon be inactivated anyway, we may as well
remove the code entirely instead of just fixing it.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1c104ef388f112c619d4ff4b7f00d17383500c1e
Reviewed-on: http://review.whamcloud.com/6219
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3160 clio: don't ignore layout on writeback

In some cases such as kernel writeback, we shouldn't ignore the
layout, otherwise, it could race with layout change undergoing.

Test-Parameters: envdefinitions=DURATION=7200  clientdistro=el6 serverdistro=el6 clientcount=4  osscount=2 mdscount=2 austeroptions=-R failover=true  useiscsi=true testlist=recovery-random-scale
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ib9d0aa581de90711c92db4c631c52f1950ad5b67
Reviewed-on: http://review.whamcloud.com/6154
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3244 utils: tunefs.lustre should preserve virgin label

so that the filesystem registers can register properly on MGS,
if tunefs.lustre was used right after mkfs.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6245e5e4d10cd0a13a4e9068a9b758da8580b537
Reviewed-on: http://review.whamcloud.com/6216
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1606 api: compile and build lustreapi test

To avoid regression of the client api conf-sanity includes a new test
73 to verify the lustreapi can be compiled and linked against. All the
files in the directory $LUSTRE_TESTS_API_DIR with the extension .c are
compiled and linked. The directory is located in the build tree at
./lustre/tests/clientapi.

Signed-off-by: Richard Henwood <richard.henwood@intel.com>
Change-Id: I0ad6a1671bf7033ec2ad5bc7a82d82e468cd31c2
Reviewed-on: http://review.whamcloud.com/3440
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 build: Increase ldiskfs version to 4.1.0

The ldiskfs version number has not changed in quite some time.

This brings the versioning in line with what LLNL has been using
externally, and allows us to realign on the same code.

Add an explicit requirement for "lustre-ldiskfs >= 4.1.0"
to the lustre spec file at the same time.

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I72c9f52093942d881ee02dad8f65c2f04dbef35e
Reviewed-on: http://review.whamcloud.com/5938
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-2038 osd: Rename do_*punch to dbo_*punch

Minor fix where dt_body_operations::do_punch and
dt_body_operations::do_declare_punch were renamed to
dt_body_operations::dbo_punch and
dt_body_operations::dbo_declare_punch, respectively, to keep the
field names consistent.

Signed-off-by: Girish Shilamkar <gshilamkar@ddn.com>
Change-Id: Id807805bd69bb20460898552830a658d8a25c238
Reviewed-on: http://review.whamcloud.com/5895
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3203 mdd: check result of dt_trans_create() using IS_ERR().

In mdd_convert_linkea() and orphan_object_destroy() check the result
of dt_trans_create() using IS_ERR().  In mdd_close() avoid passing an
error pointer to mdd_trans_stop().  In local_oid_storage_init() avoid
a spurious mutex unlock after failure in dt_trans_create().  Trivially
simplify cleanup in mdd_convert_remove_dots() and mdd_convert_lma().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I4a5c53d3efa9c69c1428a8e5a875a531c9206d12
Reviewed-on: http://review.whamcloud.com/6117
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3106 ofd: create additional symlinks to osd

to preserve compatibility with obdfilter stats:
read_cache_enable, readcache_max_filesize and
writethrough_cache_enable are accessible via
/proc/.../obdfilter/.. again.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Iee5315a58c6434f4633692a80f57e68890f5c415
Reviewed-on: http://review.whamcloud.com/5873
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3086 build: fix 'uninitialized variables' errors

Fix 'uninitialized variables' defects found by Coverity version 6.5.1:
Uninitialized scalar variable (UNINIT)
Using uninitialized value.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I8df7c60823e392b37880ca5ddb1ea107554784e6
Reviewed-on: http://review.whamcloud.com/5916
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1330 proc: remove osd proc handlers from lprocfs_status.c

Move lprocfs_osd_rd_{blksize,{files,kbytes}{free,avail} to dt_object.c
and rename them to lprocfs_dt_rd_blksize....  Remove the unused
function lprocfs_obd_rd_mntdev().  Do not include
lustre_{fsfilt,log,disk}.h or dt_object.h in lprocfs_status.c.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic54973f8bc6ec1d440fa76aadf0dd13629ce345c
Reviewed-on: http://review.whamcloud.com/5721
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1199 build: Improve git hook link installation

Check for .git/hooks directory before trying anything else.  Removes
one barrier to compiling lustre without .git being present.

Change-Id: I5fdf8fae9a4958099a40772d927fa59e42446a5b
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5879
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2989 build: some make targets are broken

Add make targets "distclean" and "maintainer-clean" in
ldiskfs/ldiskfs/Makefile. The global "make distclean" now
executes whithout an error.

Also lustre/target/.gitignore file is created to hide
message from GIT.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ic8841d11960d5aacd8293ecfcaf8c628dd743c45
Reviewed-on: http://review.whamcloud.com/5766
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1199 lbuild: Fix error handling

Improper grouping of expressions in the error() function meant
that even a simple "lbuild -h" would trigger a backtrace, and email to
someone at whamcloud.com.  That is fixed here.

Also stop sending email to a static email address on errors.

Also fix a minor typo.

Change-Id: I5a3f6131baaadc9dba23414e9e959f72fbda2679
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5961
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2877 Add a roundtrip delay in sanity test 34h

It seems we are still getting false failures in test 34h due to
overloaded server that takes longer than 2 seconds to process
a request.
To even things out - do another sync RPC to OST first before
starting the timeout.

Change-Id: I070345233398d7a15a105162523ef6dc81c1a929
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/5882
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-3242 obdclass: skip zero cookie in class_handle_hash()

In class_handle_hash() really skip the zero cookie when it comes
around.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I042d7a43b70feb9710a7b3a27e732fc314ec1cf5
Reviewed-on: http://review.whamcloud.com/6198
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3182 lmv: fix duplicate directory entries

lmv_readpage() fails to overwrite the ldp_hash_end and ldp_flags of
the first lu_dirpage in a CFS_PAGE with the values from the last
lu_dirpage. This causes duplicate directory entries to be returned
from readdir() for sufficiently large directories.  This is only
an issue on platforms where CFS_PAGE_SIZE > LU_PAGE_SIZE, i.e. PPC.

* Fix the regression introduced in commit 5e91e5b, which was the
  apparently accidental removal of these lines from lmv_readpage():

-                        hash_end = dp->ldp_hash_end;
-                        flags = dp->ldp_flags;

* Refactor the lmv_readpage() function and add some comments.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I04e9a98b90216a7da7ce9d9325080d6b6c4010c7
Reviewed-on: http://review.whamcloud.com/6071
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-718 mds: fix mds-max-threads configure parameter

Commit 648b69c2c9 renamed MDT_MAX_THREADS to MDS_MAX_THREADS, but
didn't update the configuration parameter for setting this value
at build time.  Fortunately, the configure parameter itself has
the correct name to begin with, so no build-visible changes needed.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I21dc22e470e7e3788ebef8009643b5c7a98bc220
Reviewed-on: http://review.whamcloud.com/5723
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2838 build: Syntax issues in sources

Just fix syntax issues in source code.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I822b943f31785e18ca09125466a9b21d7f7ce558
Reviewed-on: http://review.whamcloud.com/5476
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>

LU-2856 mdt: Workaround for filenames w/zero hash

Zero is a valid return value of full_name_hash(), but
consumers of mlh_pdo_hash don't handle this case correctly.
Work around problems by consistently mapping zero hashes to
a different value.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: I8b2622cb9b04d9f005f8e414ad35102155820c7d
Reviewed-on: http://review.whamcloud.com/6166
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-2139 osc: Track and limit "unstable" pages

This change adds a global counter to track the number of "unstable"
pages held by a given client, along with per file system counters. An
"unstable" page is defined as a page which has been sent to the server
as part of a bulk request, but is uncommitted to stable storage.

In addition to simply tracking the unstable pages, they now also count
towards the maximum number of "pinned" pages on the system at any given
time. Thus, a client will now be bound on the number of dirty and
unstable pages it can pin in memory. Previously only dirty pages were
accounted for in this limit.

In addition to tracking the number of unstable pages in Lustre, the
NR_UNSTABLE_NFS memory zone is also incremented and decremented for
easy monitoring using the "NFS_Unstable:" field in /proc/meminfo.
This field is also used internally by the kernel to limit the total
amount of unstable pages on the system.

The motivation for this change is twofold. First, the client must not
allow itself to disconnect from an OST while still holding unstable
pages. Otherwise, these unstable pages can get lost due to an OST
failure, and replay is not possible due to the disconnect via unmount.

Secondly, the client needs a mechanism to prevent it from allocating too
much of its available RAM to unreclaimable pages pinned by the ptlrpc
layer. If this case occurs, out of memory events can trigger as a side
effect, which we need to avoid.

The current number of unstable pages accounted for on a per file system
granularity is exported by the unstable_stats proc file, contained under
each file system's llite namespace. An example of retrieving this
information is below:

$ lctl get_param llite.*.unstable_stats

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Ic43d34bfa39da9ac4fe159000c7db5908467fd7b
Reviewed-on: http://review.whamcloud.com/4245
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3196 tests: a few fixes for > 10 OSTs.

1. Because the stripe index not necessarily follow OST index order,
even it specifies the index, so we should not check the layout by
OST inde order in sanity 27C.

2. In test 33c, it should print the OSTXXXX by %.4x instead of
%.4d.

3. In replay 44, timeout is under mds, instead of mdt.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Id60ffece99f76cd7dd2e88a655e78e9abcbdfa9b
Reviewed-on: http://review.whamcloud.com/6110
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3133 lfsck: remove objects from OST

In lfsck test, remove_objects() should remove objects from OST,
just like remove_files() removes files from MDT.

Test-Parameters: testlist=lfsck
Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I826bea7c780de275c1890e920deb3c8e8e942c53
Reviewed-on: http://review.whamcloud.com/5981
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3014 utils: Fix an offset overflow in file_create()

On an x86_64 machine, creating a 3 GB ZFS-based target using a file
VDev failed like this:

  mkfs.lustre FATAL: mkfs.lustre: Unable to truncate backing store:
  Invalid argument

The error, returned by the ftruncate() call in file_create(), was due
to the "int"-type calculation for the "off_t" argument.  The byte
number of 3 GB overflowed the "int" type and became a negative
"off_t".  This patch changes file_create() to take an "__u64" size
instead of an "int" one and adds "_FILE_OFFSET_BITS=64" to the
AM_CPPFLAGS of lustre/utils, so that file VDevs larger than 2 GB can
be created on both 32-bit and 64-bit x86 architectures.

Change-Id: Id7e6bfc963b0ccba8266795ba2bf9832e9c641ba
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5805
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2311 osd-ldiskfs: fix link refcounting

Fix potential problem with osd_object_ref_{add,del}() where the link
count is temporarily set to an invalid value before being corrected
later on.  Since the link count is accessed in some places without a
lock, there is a chance an invalid value is sampled.  This was fixed
in ext4 via kernel commit 909a4cf1ffe4b875c87abf38239a9bfd25167e0c.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I396bff24f8a23d6b7622cebfb39b8f847e500c1e
Reviewed-on: http://review.whamcloud.com/4675
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-3110 osd-ldiskfs: Dynamic LBUG in osd declares tracking

This patch implements a dynamic way to enable/disable osd
declaration tracking LBUGs.

OSD_TRACK_DECLARES define usage has been removed, and
tracking of declares is no longer a compile time option.

Enable/disable of declares tracking LBUGs is done via new
global lprocfs "track_declares_assert" boolean, also a
module-parameter.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I5164c51c3c3362a77d1a0c7cb7b9f63383b00403
Reviewed-on: http://review.whamcloud.com/6032
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2831 tests: Sync MDS data also when doing sync_all_data

The quota data was differing in sanity-quota/35 before and after
restart because the MDS data had not synced prior to checking the
values.  This patch forces MDS data to sync whenever sync_all_data is
called.  This should prevent this error from recurring here and
elsewhere.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ic2e1dafefbc2d83fc281d76845504c4547dd0694
Reviewed-on: http://review.whamcloud.com/6142
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2547 tests: EXCEPT recovery-small/24a for ZFS

EXCEPT this test for ZFS as it is failing over 50% of the time
currently.  Re-enable when root cause of failure is located.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I6c02a4db5a0d9f876cb4e8cff4e614eb3ac86dfd
Reviewed-on: http://review.whamcloud.com/6119
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3238 ptlrpc: expression expansion bug in macro

slab flag passed into __OBD_SLAB_ALLOC_VERBOSE is not bracketed,
so __OBD_SLAB_ALLOC_VERBOSE will get wrong value from unexpected
expression expansion and hit assertion.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I4f09ccf743d6bda765bab64c60801a87d9a6b9b5
Reviewed-on: http://review.whamcloud.com/6190
Tested-by: Hudson
Reviewed-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3231 fld: fix previous range copy in fld_cache_lookup()

In fld_cache_lookup() when returning the previous seq range, copy the
appropriate part of the previous fld_cache_entry.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iabc1146e5c5ac188c8d37c953ffbb645450e070d
Reviewed-on: http://review.whamcloud.com/6171
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-3135 wireshark: Add version check to Makefile

This will check for a minimum wireshark version, and warn if not
present.  While the info is in the README, this will make it more
clear why things fail to build.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id4d5a5734befaa86e8e4fd77d4c35c25d1bbfb85
Reviewed-on: http://review.whamcloud.com/6011
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3205 llite: Set layout_gen before compatibility check

We should set IO's layout generation to current generation of inode,
no matter what it is. Otherwise, it will go into an infinite loop
on layout change check.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I9f1f0d9ae0dd53dea130db46bbcbdbd29526b6eb
Reviewed-on: http://review.whamcloud.com/6137
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3006 utils: mount to pass/clear UPDATE flag

this flag is maintained in mountdata (or fs attribute in case
of ZFS) and used to signal MGS parameters have been changed
and this need to be reflected in configuration profiles.

the flag is cleared before actual mount as the device can
be exclusively open by the filesystem preventing any updates.

the scope of the patch is limited to UPDATE flag, but later
we can try to use the approach to deal with WRITECONF/VIRGIN
flags as well.

test 73 added to conf-sanity to verify the case.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ic0b4eb6b86798450e4dd4201611e4f5a1c54ef40
Reviewed-on: http://review.whamcloud.com/5982
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>

LU-2459 osd: add LMA incompat flag check

* Add LMA incompatibility flags checking after object initialization.
* Add a sanity test case (test_17o).

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Iec83f3c87bd5ff769c3544c5897011333aa2d656
Reviewed-on: http://review.whamcloud.com/6121
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-3142 tests: gather logs on passive server nodes

This patch adds all_mdts_nodes(), all_osts_nodes(), all_server_nodes()
and all_nodes() into the test-framework.sh to get the active and
passive server nodes. By using these functions, after recovery-*-scale
tests failed in failover configuration, the logs on passive server
nodes can also be gathered.

This patch also fixes yml_entities() to add MGS entity and entities
for passive server nodes.

Test-Parameters: clientdistro=el6 serverdistro=el6 clientcount=4 \
osscount=2 mdscount=2 austeroptions=-R failover=true \
useiscsi=true testlist=recovery-mds-scale

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Idbba6f02f4f50b7d5097a78ea6b6954d5347919a
Reviewed-on: http://review.whamcloud.com/6112
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3142 tests: use echoerr() to redirect debug logs

This patch fixes run_dd.sh to use echoerr() for redirecting
debug logs to the log file instead of messing up the output.

Test-Parameters: clientdistro=el6 serverdistro=el6 \
clientarch=x86_64 serverarch=x86_64 clientcount=4 \
osscount=2 mdscount=2 austeroptions=-R failover=true \
useiscsi=true testlist=recovery-double-scale

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I72c02a38c22fca4e7e806af6771e135efd506478
Reviewed-on: http://review.whamcloud.com/6027
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3216 kernel: Kernel update [RHEL6.4 2.6.32-358.6.1.el6]

Update RHEL6.4 kernel to 2.6.32-358.6.1.el6.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I8c02c826a4e40eb58bc0df017eb0654e3c0465de
Reviewed-on: http://review.whamcloud.com/6153
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3142 mdt: update mdt_max_mdsize in mdt_lvbo_size

during calling mdt_lvbo_size to get the size of lsm, mdt_max_mdsize
could be less than the lsm size of this object in failover mode.
then it is needed to check and update this value.

Test-Parameters: envdefinitions=RECOVERY_MDS_SCALE_EXCEPT=failover_ost \
clientdistro=el6 serverdistro=el6 clientarch=x86_64 serverarch=x86_64 \
clientcount=4 osscount=2 mdscount=2 austeroptions=-R failover=true \
useiscsi=true testlist=recovery-mds-scale

Change-Id: Id490c346ca9f34edcc963334e396678f1b41562c
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/6102
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1189 tests: run save_lustre_params() on facets

This patch fixes save_lustre_params() and restore_lustre_params()
to get and set Lustre parameters through do_facet() instead of
do_node(). This will fix the issue that restore_lustre_params() still
tries to restore the saved params on the original node which will
likely become inactive under failover test environment.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
clientcount=4 osscount=2 mdscount=2 austeroptions=-R \
failover=true useiscsi=true \
testlist=replay-vbr,replay-dual

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I5bfc4797727aaaa15fac1e34b6a7182ae8f26da8
Reviewed-on: http://review.whamcloud.com/5628
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>

LU-992 build: remove obsolete rhel5 server configurations

For 2.4 RHEL5 server side support has been removed but
some technical debt remained before. This patch removes
the no longer needed rhel5 kernel configurations and
the CONFIG_LDISKFSDEV_* options that were left over
from the rhel5 ext3 days.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I345b1b5d182602b82be018b1e18a7bdbe939f578
Reviewed-on: http://review.whamcloud.com/5930
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3202 osd: no inode::i_mutex inside osd_object_destroy

Originally, to control the race between the OI scrub inserting
OI mapping and the osd_object_destroy() removing OI mapping on
the same target, the inode::i_mutex was used.

But the unlink thread which called osd_object_destroy() already
started transaction handle. Such order is different from others
as to may cause some deadlock between transaction start and the
obtain inode::i_mutex.

So now, the osd_object_destroy() will not obtain inode::i_mutex,
instead, the OI scrub will check whether someone unlinked the
inode or not during the OI scrub rebuilding the OI mapping, and
remove the new-inserted OI mapping if the race happened.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic5c2e2b1d967e52a2b980d4f6bcaed4bdcf8368b
Reviewed-on: http://review.whamcloud.com/6124
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Cliff White <cliff.white@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3029 osd-ldiskfs: clear old FMODE_32BITHASH for readdir

On MDS, the RPC service thread's "osd_thread_info" will be reused
without completely reset when switches from one RPC processing to
another RPC processing.

Some old client may not claim "OBD_CONNECT_64BITHASH" when connect
to the MDS, then it will be regarded as 32-bit client. For readdir
RPC from such old client, the MDS will use 32-bit dirhash, the RPC
service thread's "osd_thread_info::oti_it_ea::oie_file::f_mode" is
set as "FMODE_32BITHASH", which will not be dropped until restart.

If some RPC service threads (for readdir) are "FMODE_32BITHASH",
but some NOT, then for new client, which support 64-bit dirhash,
it may get trouble when traverses large directroy as following:

1) The first readdir RPC is served by the RPC service thread1,
   which is marked as "FMODE_32BITHASH" because it ever served
   readdir RPC from old client. So the thread1 still generates
   32-bit dirhash (major hash only) for the new client readdir.

2) The new client triggers another readdir RPC for the same dir
   with the 32-bit dirhash as cookie, which was returned by the
   thread1.

3) This time, the readdir RPC is served by another RPC service
   thread2, which is NOT marked as "FMODE_32BITHASH". Then the
   thread2 explains the readdir cookie as 64-bit dirhash, It's
   wrong. So the thread2 can NOT locate the position correctly.

So we need to clear some fields in "osd_thread_info" to avoid
to be reused when switch from one RPC processing to another.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I3b9aaede0bccab900f1d198c7093b98f0fc48945
Reviewed-on: http://review.whamcloud.com/6138
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-631 osc: wait OSC to complete initial connection with OST

in functions lov_prep_*_set, if one OSC isn't active and it is trying
its first connection with OST, then spend some time (obd_timeout)
to wait it to complete.

Change-Id: I424dbf81b6ceebf2cfd1cf48b0f89be40c4c3df4
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/2469
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3143 tests: pass correct facet to wait_osc_import_state()

This patch fixes recovery-small test 29a and some other tests
to pass correct facet to wait_osc_import_state(), which will
make the tests pass under failover configuration.

Test-Parameters: envdefinitions=SLOW=yes \
clientdistro=el6 serverdistro=el6 clientarch=x86_64 \
serverarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
austeroptions=-R failover=true useiscsi=true \
testlist=recovery-small

Change-Id: I68b8d2f2528223a5140eed5724c2a4ffdd1988fa
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/6100
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3201 lmv: check result of lmv_find_target()

In lmv_locate_mds() and lmv_iocontrol() check the result of
lmv_find_target() using IS_ERR() before dereferencing it.  In
lmv_get_target() return ERR_PTR(-ENODEV) rather than NULL when the
indicated MDS cannot be found among the members of lmv->tgts.  In
__lmv_fid_alloc() check the result of lmv_get_target() using IS_ERR().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8cfce869e4be329fd432a7b6a88f48fbc81d69dd
Reviewed-on: http://review.whamcloud.com/6116
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>