LU-2444 build: fix 'error handling' issues

Fix 'error handling issues' defects found by Coverity version
6.0.3:
Unchecked return value (CHECKED_RETURN)
Value returned from a function is not checked for errors before
being used.
Argument cannot be negative (NEGATIVE_RETURNS)
Negative value used as argument to a function expecting a
positive value.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I44022a7d7227ee46345b77701b4c5199a74c386d
Reviewed-on: http://review.whamcloud.com/4772
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2302 scripts: make lfs_migrate compatible with 1.8

Earlier versions of lfs setstripe/getstripe use option -s instead
of -S. Make sure lfs_migrate can be used with any version.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: Ic3caaeaf3d6c764afbd75cd7a8a7279462d21d47
Reviewed-on: http://review.whamcloud.com/4702
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2590 obdclass: correct swab lov_mds_md

For caller's convenience, the 2nd parameter of obd_free_diskmd()
could be host endianness, it needs swab to LE if necessary, while just
lov_mds_md header needs it for figuring out how much memory needs to
be freed.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I537e83b56b4be6e08a7431bfda1de71efefa33ee
Reviewed-on: http://review.whamcloud.com/5145
Tested-by: Hudson
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2158 fsfilt: Remove unused opcodes

This change removes the unused fsfilt opcode definitions and the
accompanying code in fsfilt_ext3_start. This is strictly cleanup.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Ifccb6e7c9c728bc98e4ddfa52093e660dd117b8b
Reviewed-on: http://review.whamcloud.com/5192
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-2158 fsfilt: Remove lvfs_common.c

This change removes the lvfs_common.c file, and moves it's only
remaining functionality directly into the obd_lfs_fid2dentry function.
This should have no functional impact, and is merely a code cleanup
exercise.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I5f8588a17f6e7659a7d33c2a5f79792826976de4
Reviewed-on: http://review.whamcloud.com/5193
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2360 ptlrpc: don't clobber rq_history_seq with CPT_ANY

In ptlrpc_req_add_history() don't allow service_parts with cpt CPT_ANY
(-1) to clobber the timestamp and sequence bits of rq_history_seq.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I52161d6f36dcd70dd569b0be294e6d23cc5ac4a9
Reviewed-on: http://review.whamcloud.com/5207
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2764 tests: Remove obsolete comment in local.sh

The comment above LOAD_MODULES_REMOTE in local.sh was written for the
special case of the 2.3 OSS stack and is no longer applicable to
current master.

Signed-off-by: Li Wei <wei.g.li@intel.com>
Change-Id: I75518e9cebfa717ccfb625e2786c8d191c6bddbc
Reviewed-on: http://review.whamcloud.com/5277
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2749 ofd: check return value of ofd_seq_load

Return value ofd_seq_load was not always checked for both NULL and
errno.  This caused a NULL pointer dereference.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ib2e1658f16ba2012c9410966c08201cab7a5aa5c
Reviewed-on: http://review.whamcloud.com/5263
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>

LU-2414 tests: cleanup replay-single/0c and 44c

Switch from return codes to error messages in test_0c and test_44c.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I61895442fa52e2f7bff421e0b8309f0f11c97de8
Reviewed-on: http://review.whamcloud.com/4856
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1434 utils: extend wireshark parsing

Add support for MGS_CONFIG_READ
Add capabilities where they're defined in layout.c
Fix processing of MGS_BULK_PORTAL (not a Lustre packet)
Make use of LUSTRE_BUFFER_LEN everywhere.
Add dissecting of capabilities field
Add dissecting of fld field
Add dissecting of MGS_CONFIG_BODY
Fixed all compiler warnings.
Fixed parsing of get/set (x)attrs

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ic75bb4814570b30a1f1dca19d2f49311294ff2fe
Reviewed-on: http://review.whamcloud.com/5155
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2689 interop: use stripe md in ll_update_inode()

When connected to servers that do not support LL_SBI_LAYOUT_LOCK, if
stripe md is available in ll_inode_update() then use it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I949a0129c3fc92c5434288df0549a54ccd440f97
Reviewed-on: http://review.whamcloud.com/5289
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2562 test: cleanup previous test result

in sanity.sh, test_51ba only depends on test_51b, but test_51a
also create files under $DIR/$tdir, which causes test_51ba fail
for it also takes those files into account.

Change-Id: I42f491f64c8d2d65bee656c167c9523274cd7200
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/5183
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2449 osd: osd-zfs to handle ./.. properly

ZFS does not store directory records . and .. on a disk.
Instead they are generated upon request (like in zfs_readdir()).
with the patch osd-zfs does the same for namespace directories.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I910f9bad807c0e5a2bee1fd481470576ceaf4444
Reviewed-on: http://review.whamcloud.com/4990
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>

LU-2240 mds: Assign special fid sequence to root.

In current implementation, we still use IGIF as the ROOT FID,
but ZFS does not have IGIF at all, so it will move root ROOT fid
to a new sequence.

But for compatible with clients, which are being mounted
during upgrade, it will still keep the old IGIF root FID.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I2e6b81cd359436d2e5b96c8a315a56c415f8e631
Reviewed-on: http://review.whamcloud.com/5257
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-1866 lfsck: LFSCK for namespace consistency (3)

It is the third part of LFSCK 1.5 for namespace consistency
check and repair. It mainly implements the LFSCK namespace
component functionality lfsck_operations::lfsck_double_scan:
process multiple-linked objects.

1) Find out and repair crashed linkEA entry.

2) Find out and remove invalid linkEA entry

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes testlist=sanity-lfsck

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia38fa343ed4da10635c1af4d90b9039c5e01688c
Reviewed-on: http://review.whamcloud.com/4914
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2726 procfs: Invalid assertion for counter_{add,sub}

lprocfs_stats::lc_sum_irq is only used for obd_memory for accurate
memory usage counting, other procfs stats counters do not need it,
even though they can be used in RCU callbacks.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I4c2dec50eecc0c63631f1915ba30a54a5da2fec1
Reviewed-on: http://review.whamcloud.com/5237
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2017 mdc: add layout swap between 2 objects

This patch adds the client and MDT code to swap layouts
between 2 files:
- a lfs command
- a llapi call, based on a new ioctl
- the mdc/mdt codes for the ioctl (with layout locking)

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I7e710a1ab3ca38e8b26582e49d08e9943aa445cd
Reviewed-on: http://review.whamcloud.com/4507
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1866 lfsck: LFSCK for namespace consistency (2)

It is the second part of LFSCK 1.5 for namespace consistency
check and repair. It mainly implements the LFSCK namespace
component functionality lfsck_operations::lfsck_exec_dir:
for linkEA verification and repair.

The basic functionality is to find out missed linkEA entries
and add them to the object. It is also part of handling IGIF
objects upgrading.

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes testlist=sanity-lfsck

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I9df2a1fddfa4b15838ecbdfb6279c240d01d0f95
Reviewed-on: http://review.whamcloud.com/4913
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1866 lfsck: FID-in-{dirent,LMA} check and repair

As part of LFSCK 1.5, it implementes the following functionalities:

1) Verify and rebuild FID-in-dirent, mainly for MDT file-level
   backup/restore.

2) (Partly) verify and rebuild FID-in-LMA: if the FID-in-dirent
   is valid but FID-in-LMA is missed, then the later one will be
   repaired; but if both are missed, then it is OI scrub's duty
   to repair the inconsistency.

3) (Partly) handle IGIF objects upgrading: build IGIF-in-dirent
   for IGIF objects. The other two parts work for IGIF objects
   upgrading are:

3.1) Generate IGIF-in-LMA, which has been done by OI scrub in
     former patch(es).

3.2) Generate linkEA, which will be done by up layer LFSCK in
     other patch(es).

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes testlist=sanity-lfsck

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ibb317f0f3f3e9cf6db56bfa25f55d2017b0e1937
Reviewed-on: http://review.whamcloud.com/4912
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1866 lfsck: LFSCK 1.5 user space control

Include the user space command to start/stop the LFSCK for
namespace consistency check and repair. And proc interface
to query the LFSCK status, statistical informtion, and ect.

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes testlist=sanity-lfsck

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Icf4046b313dc1653b5b8b1a3add7fa0329ca7ad0
Reviewed-on: http://review.whamcloud.com/4911
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1866 lfsck: LFSCK for namespace consistency (1)

It is the first part of LFSCK 1.5 for namespace consistency
check and repair. It mainly implements the LFSCK namespace
component internal framework and external APIs.

The functionality parts for the LFSCK namespace component,
such as lfsck_operations::lfsck_{exec_dir,dump,double_scan},
are not included. They will be implemented in other patches.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I011e06c991577e632eadf364b87be74f1f5535c6
Reviewed-on: http://review.whamcloud.com/4910
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1866 lfsck: LFSCK main engine

There are two engines for LFSCK 1.5:

1) One is low layer object table based iteration, which can
   guarantee that all the objects can be scanned at least once.

2) The other is namespace based directory traversal. All the
   namespace based components, such as FID-in-dirent and linkEA,
   can be handled during that.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic5a404cde35fdb20c24f6522faa8b21117cf4254
Reviewed-on: http://review.whamcloud.com/4909
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1866 lfsck: general framework for LFSCK 1.5

LFSCK 1.5 main data structure definations, LFSCK
component APIs, setup/cleanup/start/stop interfaces.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4507b9432289ca8d54cbf686893b706052d0eeb3
Reviewed-on: http://review.whamcloud.com/4908
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1866 lfsck: ancillary work for namespace LFSCK

1) Share related linkEA functions with LFSCK.

2) Multiple linked objects are rare, and the objects with
   a lot of linkea entries are more rare. It is unnecessary
   to worry about performance under such cases. So drop the
   linkea entries count limitation: LINKEA_MAX_COUNT.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I75f90c9619c56f2f62063435aea26ed01337d6dd
Reviewed-on: http://review.whamcloud.com/4907
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2733 osd: osd-zfs to initialize existing sequence

osd_find_or_add_seq() to use osd_oi_find_or_create() so that
existing sequence is loaded if exists

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I9f173615ef047dd4696d8ed51687c7c268b32dbf
Reviewed-on: http://review.whamcloud.com/5247
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2728 fld: check for NULL param in fldb_seq_stop()

Check for NULL param in fldb_seq_stop().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I44ed10f9ce8bceabb203e6ef0b843b23340ced2b
Reviewed-on: http://review.whamcloud.com/5238
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2737 osc: double unlock in osc_lru_shrink()

It could unlock twice in osc_lru_shrink() if the cl_io_init() failed.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Idf3ebafced7f7d3a68c13ea20d12cf221e9d74ce
Reviewed-on: http://review.whamcloud.com/5258
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1187 mdt: Add MDS_INODELOCK_PERM lock for remote dir

Add MDS_INODELOCK_PERM for remote directory on remote MDT to
protect permission changes on remote MDT. So, Master MDT, where
the parent is, will grant LOOKUP lock to the client. Remote MDT,
where the child is, will grant UPDATE|PERM lock to the client.

And client will clear the dcache during, when either PERM or LOOKUP
lock is being revoked.

Change-Id: Ie11d14bbe7c706a76f7c4bd8f75b07807e2dac02
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/4346
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2722 clio: directIO thread races with completion thread

ll_direct_IO_26()) ASSERTION( obj->cob_transient_pages == 0 ) failed

The directio thread puts pages to the transfer in osc_io_submit(), and
waits for its completion. Upon completion, osc_completion() is called,
which notifies waiters IO is completed and only after that put the
page. the original directIO thread wakes up and may succeed to assert
on the last page which is not yet put by osc_completion().

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: Ib912e913885a9ff2cb1e9c865ae2003a676c2b5b
Xyratex-bug-id: MRP-838
Reviewed-on: http://review.whamcloud.com/5223
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2467 ptlrpc: Allow OBD_PINGs to be suppressed

This patch introduces a new ptlrpc module parameter, "suppress_pings",
to provide an option for reducing excessive OBD_PINGs in large
clusters.  The parameter affects all MDTs and OSTs on a node.  It is
off (zero) by default, giving a behavior identical to current
implementation.  If it is on (non-zero), all clients of the affected
targets who understand OBD_CONNECT_PINGLESS will know, at connect
time, that pings are not required.  When suppressing pings, there must
be an external mechanism to notify the targets of client deaths, via
the targets' "evict_client" procfs entries.  In addition, a highly
available standalone MGS is also recommended when suppressing pings,
so that clients are notified (through Imperative Recovery) of target
recoveries.

The changes do two basically independent things.  One is initializing
import and export states (i.e., imp_connect_data and
exp_obd_chain_timed) according to "suppress_pings", since whether to
ping or not is a property of each import-export pair.  MGC pings can
not be suppressed, because maintaining MGS connections is dictated by
the reliance on Imperative Recovery.

The other thing is changing pinger and import routines to respect the
import property set earlier.  (The export side does not need any
change at all.)  Pings are still needed to query last committed
transactions if there are uncommitted requests on an import, so that
resources pinned for replays can be released even when applications
become idle.  An early version of this patch removes imports that do
not need to be pinged from pinger_imports and add them back when last
committed transactions are needed or recoveries must be initiated.
This version does not do that because a) the overheads of iterating
through 10,000 imports are not prohibitively large---around 10 ms, b)
adding imports back to pinger_imports requires the global pinger_mutex
to be held, and c) the imp_lock contention added on each import is
small.

Change-Id: Iabc84d395c978c3f156c52aebfad83621facb4fe
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5009
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Hudson
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2467 protocol: Add OBD_CONNECT_PINGLESS

Reserve a bit for OBD_CONNECT_PINGLESS, which indicates a client is
capable of suppressing keep-alive OBD_PINGs.  If granted by a server,
it means the server does not require (but still allows) pings.

Signed-off-by: Li Wei <wei.g.li@intel.com>
Change-Id: I7575082244a84f2e633a1936fcf894da3c6ee7dd
Reviewed-on: http://review.whamcloud.com/5229
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2723 clio: a client hangs on osc_extent_wait in dio

If dio overlaps with mapped region, first overlapped pages are sent
by generic_file_direct_write->filemap_write_and_wait_range->writepages
After that the dio pages are sent by write_begin/write_end, writepages
However, write_end->osc_page_cache_add does not release extent because
the IO is not considered as synchronous. as the result, the following
writepages->osc_cache_writeback_range() does not send pages as the
extent is still ACTIVE and osc_cache_wait_range() hangs forever.

The patch make directIO to be considered as sync by lustre so that
extent was released at write_end.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Xyratex-bug-id: MRP-839
Change-Id: I0c0e1599ee180f467826e4b6715c3cbfd0c6eb5c
Reviewed-on: http://review.whamcloud.com/5224
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2194 tests: add debugging to recovery-small/19

Add error messages to 19a and 19b

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ief35c063734654c5a6f3dab0d283c4b7a32706e8
Reviewed-on: http://review.whamcloud.com/5141
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1187 mdt: directory remote open fix

Directory remote open continue as normal open, i.e. return the
remote fid to client, and client will send the open request to
the right MDT.

Set correct it disposition in mdt_open_by_fid.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I32d5dee9cd1eac47787e2d85a5db1e164d2846d3
Reviewed-on: http://review.whamcloud.com/4934
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-1187 mdd: a few missing stuff in MD stack for DNE.

1. Assign the index operation for the directory create
in declare phase, which is needed for creating the
object in OUT.

2. Declare dotdot insertion for remote directory creation,
so the insert update can be packed into RPC and send to
the remote MDT (out) to be executed.

3. Add hint to the object declare phase, by which OSP can
get some info during update RPC packing, like parent fid.

4. Separate lu_object_exists into lu_object_remote and
lu_object_exists, so it can check whether remote object
exists as well.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Icb359443d9982ee8567af933c5def42dc51a3a7a
Reviewed-on: http://review.whamcloud.com/4930
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-2654 ldlm: ldlm_resource_dump should be called with resource locked

Fix ldlm_resource_complain to lock resource before calling
ldlm_resource_dump

Change-Id: I9dfbe84bd63de3093c99e7379daeca30d02fc875
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/5254
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1187 mdt: add sanity check for rename and link

Add sanity check for rename/link, so remote rename/link
will return EXDEV.

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: I841c8ffaa74b162fcca5cd1661c383215350cccc
Reviewed-on: http://review.whamcloud.com/4348
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1187 mdt: unlink remote directory

Send unlink req to the slave MDT, so both open/close
and unlink will send request to the slave MDT, where
the remote object resides. Then it would be able to
check orphan (unlink open file) locally.

Add lock_object api for enqueue remote object.

Change-Id: I7483b0f023e4e3de6597da58d3d9f3e96c0d53b7
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/4339
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1187 utils: add lfs setdirstripe/getdirstripe

1.Add lfs setdirstripe/mkdir/getdirstripe to create
remote directory.

2.Add lfs rm_entry to just remove the entry of remote
directory.

Change-Id: Idaecfbb9dd97e3ae9fbc70e5b902a91a7b2ed18e
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/4341
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1866 lfsck: enhance otable-based iteration

1) Use ::load() method to unplug OSD layer iteration.

2) Use ::put() method to wakeup the LFSCK main engine
   if it is blocked at low layer for ::next() call.

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes testlist=sanity-scrub
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I28e5689ad3a4ee96a3f17a64e8d6dca553a86d77
Reviewed-on: http://review.whamcloud.com/4906
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1866 osd: FID-in-LMA and OI files

Generate FID-in-LMA not only for normal objects, but also for
IGIF objects, server-side local files, and other non-OST/LLOG
objects.

LAST_ID has FID-in-LMA but no related mappings in the OI file.

At the same time, OI mapping usage is also extended to above
objects. Only the objects marked as "LDISKFS_STATE_LUSTRE_NO_OI"
or OST objects have no OI mappings the OI files.

It is part of 1.8 IGIF objects upgrage handling. OI scrub will
automatically generate IGIF-in-LMA and add related OI mappings
in the OI files.

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes testlist=sanity-scrub

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If1e1434c238f2ab737e35433bfad1abbb00fcb00
Reviewed-on: http://review.whamcloud.com/4904
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2733 osd: not return "-ENOENT" for zfs osd_object_init

For zfs backend, the osd_object_init() should not return
"-ENOENT" if osd_fid_lookup() cannot find the object,
which is normal for new created cases.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I76e5d802d0caf8ac3a9992278ec2f7c6c6e2cf07
Reviewed-on: http://review.whamcloud.com/5242
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1187 osd: add agent/local inode for remote directory

A local inode will be created for remote directory entry, so
to make e2fsck check pass.

The remote directory on the remote MDT will be put under the
agent directory.

Remove LA_TYPE checking during osd_inode_set, because when
initialize the remote object, it needs to get TYPE information
from remote attr get, where we need type information anyway,
(osp_md_attr_get ->out_attr_get->osd_attr_get | LA_TYPE).
So osd_attr_get will always return LA_TYPE valid information
here, then if it do attr_get/modify/attr_set might hit this
ASSERT, So during osd_attr_set, we can just ignore type.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I7f177fa8ae2ab450cde78a8c2da9a2770ec6b0cb
Reviewed-on: http://review.whamcloud.com/4931
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1866 scrub: initial OI scrub

We will extend OI mapping usage (in OI directories/files) to
all the objects, except for the objects which are marked as
"LDISKFS_STATE_LUSTRE_NO_OI" or OST object.

The OI mappings for the server-side local files may become
invalid because of server-side file-level back/restore, or
server crash. Those files are necessary for server mount,
so related OI mappings must be checked and rebuilt before
they are really used. That is the initial OI scrub work.

Initial OI scrub is used for quickly checking and rebuilding
OI mappings for server-side local files during server mount.

Another change is about MDT_LAST_RECV_OID/OFD_LAST_RECV_OID.
Currently, we can just use the same OID for the "last_rcvd"
files on different targets. So merge the two OIDs as single
OID: LAST_RECV_OID, which also simplifies OI scrub work.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iddca4812d3c82ca43f1e852fd74475b11b6e6dea
Reviewed-on: http://review.whamcloud.com/4903
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1866 osd: ancillary work for initial OI scrub

1) Abstract some general interfaces that can be shared by common
   OSD operations, by normal OI scrub, and by initial OI scrub.

2) Mark the object as "LDISKFS_STATE_LUSTRE_NO_OI" if it is not
   unnecessary to add OI mapping for the object, such as backend
   local root, OI directories/files, and others created before OI
   files created.

3) Do not check fld in OI scurb, which may be not ready yet.

4) Reorganize OI scrub code to be more readable.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ife8168df2454437ad18550b73d26f7372c21776f
Reviewed-on: http://review.whamcloud.com/4902
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2725 ost: ost_rw_hpreq_check should return 0 or 1

ost_rw_hpreq_check was returning number of matched locks which is wrong.

Change-Id: Ic4d3dd4a186b616c13ffd9f897254da3c19f84f8
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/5233
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1431 ptlrpc: PTLRPC_BRW_MAX_SIZE usage cleanup.

Clean-up the layering in current code by eliminating direct
usage of PTLRPC_BRW_MAX_SIZE macro outside of the ptlrpc
module. This should help us acheive "floating" max brw size
value across the cluster, which in turn should help with
4MB IO task.

Signed-off-by: Sergii Glushchenko <sergii_glushchenko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Alexander Zarochentsev <Alexander_Zarochentsev@xyratex.com>
Change-Id: Ib9f6265bd28f25c34b27eb6ad1a56a752b71a4ac
Xyratex-bug-id: MRP-687
Reviewed-on: http://review.whamcloud.com/4876
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1866 fid: cleanup object visibility definition and check

1) The normal fid, IGIF fid, "ROOT" fid, ".lustre" fid are
   used for client-mdt visible objects.

2) Remove redundan osd_igif.{c,h}

3) Hide server-side local root for up layer dir readpage.

4) It is unnecessary to append FID-in-dient for dot name entry.
   Because we can get it from the directory object directly.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If98de0c15c0cd3a0e93b666a8d48366b0d8a2cb0
Reviewed-on: http://review.whamcloud.com/4901
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1866 misc: fix some issues found during LFSCK

1) Test scripts issues for sanity.sh and sanity-scrub.sh

2) Incorrectly use open flag 'MDS_OPEN_OWNEROVERRIDE" for
   permission check for set_attr.

3) Other code clean, such as mdt_body, md_op_spec, wiretest.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I52fbfe8481e35edba75e8fe7e0ab2fec094eabae
Reviewed-on: http://review.whamcloud.com/5046
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>

LU-1187 mdt: Only create remote dir on MDT0

1.Add checking for only creating remote dir on MDT0.
2.Add enable_remote_dir parameter to enalbe remote dir creation
on other MDTs.

Change-Id: I272d4763f07115c5aa0432eb4bbdda5349711d25
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4336
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1187 lod: prepare default stripe EA for remote directory

During remote directory create, it should prepare the default
stripe EA in declare, so the default EA can be set correctly
on the remote directory.

Change-Id: Ief93e72cc033f3553865f2abba53e81df0f3fa89
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4335
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1187 mdt: Add pre_cleanup phase in MDT stack cleanup

Add pre_cleanup phase in MDT stack cleanup, so MD OSP can be
disconnected before cleanup MDT stack.

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: Id1a56c393c22b4fc526c81c05b11bfb656a80559
Reviewed-on: http://review.whamcloud.com/4929
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1187 mdt: add out handler for object update

Add object update handler on remote MDT. Because the RPC
only includes object updates, (no metadata operation), the
out_handler will skip MDD layer and call osd layer directly.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I9d553145cedfffcbb26d839ed8c7d9cc0887ed86
Reviewed-on: http://review.whamcloud.com/4928
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1187 osp: add osp_md_object for remote directory.

In declare phase, the update for remote object will be added
in the remote list of the transaction, which will then being
sent to the remote MDT during trans_start.

Add osp_md_object for remote directory, any updates for
remote operation will be packed into RPC by api in
the osp_md_object

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I5fac24e21e6663dca0e2d2fa5d90d4a07a002136
Reviewed-on: http://review.whamcloud.com/4927
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2466 lnet: Use Hash Table for Remote Route List

This change updates the remote route list (the_lnet.ln_remote_nets)
to be a hash table (the_lnet.ln_remote_nets_hash) to speed up
access when dealing with a large number of routes (over 1000).

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I2dac8075ff38d29bd38905b5b1b002603c5a56b6
Reviewed-on: http://review.whamcloud.com/5023
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Hudson
Reviewed-by: John Hammond <john.hammond@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2700 mdt: serializing the rename

BFL is needed in the rename process even though the source and
target are under the same directory, because we still need
serialize the locking of these two objects.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I61593e3902f1f81ac282c020d912fdf28ce2cc2c
Reviewed-on: http://review.whamcloud.com/5203
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1876 hsm: Avoid deadlock on i_mutex and layout change

The deadlock is as follows:
Proc 1 (write): start an IO -> try to acquire i_mutex in
            generic_file_aio_write()
Proc 2 (setattr): Acquire i_mutex in ll_setattr_raw -> initialise IO
            -> ll_layout_refresh() -> wait for IO above.

It's actually not necessary to acquire i_mutex in ll_setattr_raw()
until setattr really starts.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I1988d67eab8bcac2ee63b9ec0ecab06d0b7cfc66
Reviewed-on: http://review.whamcloud.com/5159
Tested-by: Hudson
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-1876 hsm: bugfix for deadlock in glimpse and layout change

This dead lock is as follows: glimpse calls ccc_inode_lsm_get()
inside glimpse IO, it will be blocked at grabbing lo_type_guard
sem, because another thread is changing layout which holds
lo_type_guard but waiting for all active IO to complete.

The basic rule: any attempt to grab lo_type_guard inside an IO is
generically risky.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ib41c03f0a73e2ef7cd37e7689c2d55f7fd0484bf
Reviewed-on: http://review.whamcloud.com/5158
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-1282 lprocfs: reduce lprocfs stats memory use

* Move percpu counter common data out, do not need to store them
  redundantly in percpu counter.
* LPROCFS_STATS_FLAG_IRQ_SAFE flag implies three things:
  1. lprocfs_counter needs lc_sum_irq.
  2. when a stats is a percpu counter stat, all its percpu counter
     gets allocated along with the stats itself.
  3. when a stats is a non-percpu stat, lprocfs_stats_lock() needs
     disable irq.
* change lprocfs_counter to make non-irq-safe stats counter do not
  use lc_sum_irq counter, which can save memory.
* Right now, only obd_memory stats use LPROCFS_STATS_FLAG_IRQ_SAFE
  flag.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I2eb037f9dcda983844857fc068c428c4fa387e7a
Reviewed-on: http://review.whamcloud.com/3246
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2221 ptlrpc: kerberos support for kernel>=2.6.24

Since kernel 2.6.24 the scatterlist struct has no field "page".
Then related Lustre kerberos code cannot work anymore.

So do not access scatterlist::page directly, instead, use the
scatterlist functions sg_set_page and sg_assign_page.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Thomas Stibor <thomas@stibor.net>
Change-Id: I446925bb42c1e018a55a69948383c8f71976f1fa
Reviewed-on: http://review.whamcloud.com/4394
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-2592 tests: do not remove $TMP/*active in rpc.sh

The $TMP/*active files record the current active facets info
under failover test configuration. They are removed from
init_test_env() initially. If failover tests use do_rpc_nodes()
which performs rpc.sh and in which init_test_env() is used,
then the active facets info will be lost during the testing.

This patch fixes the above issue by introducing an RPC_MODE
variable which controls that the $TMP/*active files will not
be removed from init_test_env() in rpc.sh.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes clientcount=4 osscount=2 mdscount=2 austeroptions=-R failover=true useiscsi=true testlist=replay-vbr
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Iab464449e7c41ffd76256e9ec60b48b326f3ea9e
Reviewed-on: http://review.whamcloud.com/5127
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2693 tests: check Lustre version in conf-sanity test 66

Lustre starts supporting the "replace nids" feature
(commit 6aa30ff) since version 2.3.59. We need add
a Lustre version check into conf-sanity test 66 to
make sure the test does not run on unsupported
Lustre version.

Test-Parameters: envdefinitions=SLOW=yes \
serverjob=lustre-b2_1 serverbuildno=164 \
testlist=conf-sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I7eee38664e66059bbc028c3eb7d9741b1351b883
Reviewed-on: http://review.whamcloud.com/5198
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2697 osp: remove trailing newline on thread names

The trailing newline on the thread name causes confusion with some
utilities that examine this string.

Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Change-Id: I031aafd2541b1829bc721ebcb9f52cb5e73101b3
Reviewed-on: http://review.whamcloud.com/5189
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2709 build: fix 'memory corruption' errors

Fix 'memory corruption' defects found by Coverity version 6.5.0:
Out-of-bounds access (OVERRUN_STATIC)
Overrunning static array by passing it to a
function which indexes it at a higher byte position.
Unbounded source buffer (STRING_SIZE)
Passing string of unknown size to a function that expects
a string of a particular size.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Iabc9bd7609d285a3709737748f2100b90769d795
Reviewed-on: http://review.whamcloud.com/5210
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1897 test: replay-single test_70b dbench not found

dbench is exiting before test_70b completes is work.

dbench is being used as a background load in this test.
In one observed case test was taking a little under 90s but
dbench was exiting at 62s. Duration was set at 90.

Presently dbench starts for 60s then the system sleeps
for 12s, then we start a new 60s timer in which we expect
dbench to be running.  There is a large window in which
this test will fail.

The main issue is the use of Duration for use as both the
timer of the real test and the timer of the background load.

I raised the real Duration and started the timer for the real
test before starting dbench such that the real test should
finish before dbench does or about the same time.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: Iba243fe3132a5fa677e5a7bcd09e491727ba092a
Reviewed-on: http://review.whamcloud.com/4973
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2101 clio: cl_sync_io_wait() need a barrier

cl_sync_io_note() has a small window between cfs_atomic_dec_and_test
and cfs_waitq_broadcast, another thread might have destroyed anchor
between this window, then cl_sync_io_note() is calling wakeup on a
poisoned waitq and crash.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I04bfc738c8c5bec3a1e8b51131fae4199ef5ec9a
Reviewed-on: http://review.whamcloud.com/5199
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2010 tests: replay-single/44a,b,45 find only tested mdc

If multiple filesystems are mounted, then test_45 would have found
mdcdevs for each filesystem.  This corrects that to only look for the
filesystem that is being tested against.

Also correctly delete old test files.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I0297cf9b21e9c97ee468cd81d5d4ac409c59d1e0
Reviewed-on: http://review.whamcloud.com/4724
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2576 osc: various fix in osc_enter_cache()

- It should not go to sleep if there isn't any inflight write
  RPC, otherwise, it could not be waked up for a long time,
  until dirty flush triggered on other OSCs or other objects.

- If the page can't be granted due to too many dirty pages
  (> obd_max_dirty_pages), osc_enter_cache() should return
  -EDQUOT.

- It should pass NULL osc object to  osc_io_unplug(), otherwise
  dirty flush can't be triggered if the passed object is
  clean.

- It should not wait in a loop of "while (cli->cl_dirty > 0)",
  trigger the dirty flush once is enough. If there are too
  many threads that consumed grants immediately, then we
  should turn to sync write, but not trigger flush and wait
  for grant again (that'll block io process on grant).

- Refuse setting max_dirty_mb as 0 via proc file.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I449cba07bd427749ab023c249d9e200aba1b406a
Reviewed-on: http://review.whamcloud.com/4963
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2663 tests: don't remove objects by debugfs

In lfsck.sh, we shouldn't use debugfs to remove objects, that'll
cause quota usage inconsistence warning in e2fsck.

Fix improper LBUG() in mdd_create_data(), MDS_OPEN_HAS_OBJS is
still used by lfsck.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I93c6f5c40e359d0d8402868be548088d6e2ac5da
Reviewed-on: http://review.whamcloud.com/5186
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2640 osd: enhance transaction debug to trace rollback

Some modifications may need to rollback because of failures during
the transactions. The original transaction trace mechanism did not
consider rollback cases, so cannot correctly record sub-operations
and the rollback cases.

In this patch, not only the specified sub-operation will be traced,
but also its rollback operation (if may rollback) will be recorded
also. For example:

When declare for ref_add, it will record the ref_add sub-operation
by "osd_thandle::ot_declare_ops[OSD_OT_REF_ADD]"; when consume the
ref_add credit, the possible rollback will be recorded through the
"osd_thandle::ot_declare_ops_rb[OSD_OT_REF_ADD]"; and when need to
rollback the ref_add (by ref_del) in subsequent processing, it can
be traced by the "osd_thandle::ot_declare_ops_rb[OSD_OT_REF_ADD]";
and once rollback starts, only "osd_thandle::ot_declare_ops_rb[X]"
can be consumed.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iade214eff468c2298f84920b997040e06f1a204f
Reviewed-on: http://review.whamcloud.com/5138
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2061 hsm: send HSM resquest to CDT

This patch implements the user interface to ask for hsm requests

It implements:
- lfs hsm_archive
- lfs hsm_restore
- lfs hsm_cancel
- lfs hsm_remove
- lfs hsm_release

The matching llapi call:
- llapi_hsm_user_request_alloc()
- llapi_hsm_request()

related MDS functions:
- mdt_hsm_request()

and the MDS RPC:
- MDS_HSM_REQUEST

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I635f6e0518ec12c359dbb528c3a61216650d952d
Reviewed-on: http://review.whamcloud.com/5030
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2061 hsm: get list of HSM actions

This patch implements a MDS RPC to get
the list of registered and running hsm actions on files

It implements:
- lfs hsm_action

The matching llapi call:
- llapi_hsm_current_action()

The matching MDS function:
- mdt_hsm_action()

and MDS RPC:
- MDS_HSM_ACTION

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: If4b00949ca8c94c8572cc81a7f7b8db32914542c
Reviewed-on: http://review.whamcloud.com/5029
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1338 hsm: HSM flags feature

This extends lustre_hsm_attrs struct in MDD to store HSM per-file
information. It adds infrastructure (function, RPCs, ...) to read and
modify it, from MDT code or from user-space on clients through llapi
and lfs command.

This patch implements:
- lfs hsm_state
- lfs hsm_set
- lfs hsm_clear

matching llapi functions:
- llapi_hsm_state_get()
- llapi_hsm_state_set()

matching mdt functions:
- mdt_hsm_state_get()
- mdt_hsm_state_set()

and MDS RPC:
- MDS_HSM_STATE_GET
- MDS_HSM_STATE_SET

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Ic8f82ddc9a56206307c2e5be2523fb7ce42b8638
Reviewed-on: http://review.whamcloud.com/3035
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2638 ldiskfs: update dotdot entry for rename

On master, when rename a directory, its old dotdot entry will
be removed firstly, then insert the new dotdot entry, and try
to append FID-in-dirent. But the space for dotdot entry maybe
not enough to hold the new dotdot with FID-in-dirent, such as
the MDT device restored from file-level backup, or the device
is upgraded from 1.8. If we do not move data in the directory
block, the FID-in-dirent will overwrite the dx_root and cause
the directory crashed. Currently, we do not want to introduce
complex logic to handle directory data moving, instead, under
such case, ignore the FID-in-dirent for the new dotdot entry,
and just insert the new dotdot entry.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Id38f5136fb2bc607344c37ca10a32e63ab427e3d
Reviewed-on: http://review.whamcloud.com/5179
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1187 lod: move seq client init from MDT to LOD

Move seq client init from MDT to LOD, so to unify
config log for adding OSP and MD OSP.

Change-Id: I93f91c0dd6f3568196a99eecf422e0b409abda31
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4337
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1187 lod: Add remote object for DNE

LOD will do FLD lookup to check whether the object
is for OST or remote MDT object.

Because there are remote object invlove, it might
return some error other than ENOMEM for object allocation.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie226b3c475524fc7e705665c6200e4c66d50acf0
Reviewed-on: http://review.whamcloud.com/4924
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2062 hsm: llapi hsm calls for CopyTool

llapi hsm calls for CopyTool
- llapi_hsm_copytool_start()
- llapi_hsm_copytool_fini()
- llapi_hsm_copytool_recv()
- llapi_hsm_copytool_free()
- llapi_hsm_copy_start()
- llapi_hsm_copy_end()
- llapi_hsm_progress()
- llapi_hsm_import()

and related MDT functions
- mdt_hsm_ct_register()
- mdt_hsm_ct_unregister()
- mdt_hsm_progress()

The new RPC are:
- MDS_HSM_CT_REGISTER
- MDS_HSM_CT_UNREGISTER
- MDS_HSM_PROGRESS

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I93617ac6e66a3324344b57f1a22d7c851d715262
Reviewed-on: http://review.whamcloud.com/4736
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

New tag 2.3.60

Also another can kicking for LU-2449, hopefully last this time.

Change-Id: I1455e48f1b557bc8787ef234e1894b0f9d7f4843
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-2183 osp: cleanup osp-on-ost

With new quota design and FIDonOST introduced, a new connection
from OST/MDTn to MDT0 is required for sending quota and sequence
RPCs. This connection was implemented by adding a new special
osp device named osp-on-ost, which introduced complexity to osp
code, and many hacks of checking the osp device name in both
client and server side code.

In this patch, we removed the osp-on-ost, and introduced a new
device type lwp (Light Weight Proxy) to manage the light weight
connection established from OSTs/MDTs to MDT0.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: Ic1ba25f941a4f971a542dc4be8a81e28ee0f477b
Reviewed-on: http://review.whamcloud.com/4923
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-398 ptlrpc: Add the NRS framework and FIFO policy

The Network Request Scheduler (NRS) allows a user to control the way
in which RPCs are dispatched from PTLRPC services. This can be used to
various effects, most importantly improved fairness across the
cluster, better I/O performance under certain workloads, and allowing
for QoS semantics to be used at the filesystem level. Central to the
operation of NRS are NRS policies, each of which implements handling
of RPCs in a different way, aiming to achieve a particular goal.

This patch adds the core NRS framework and the NRS FIFO policy, which
is a logical wrapper around previous, non-NRS functionality, and is
used as the default policy for handling all types of RPCs.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Co-authored-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I7fcd7885cc89b653d9bc482da533aae2f129bdf9
Oracle-bug-id: b=13634
Xyratex-bug-id: MRP-73
Reviewed-on: http://review.whamcloud.com/4411
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2441 mdd: Implement volatile file

Add a way to create files which are created unlinked, so
they are automatically removed at last close

User interface is:
- llapi_create_volatile_idx()
- llapi_create_volatile()

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Ia33a6832d31b83191e4d364da4350cedbe6eef39
Reviewed-on: http://review.whamcloud.com/4836
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-1187 tests: chmod +x for dir_remote.sh

chmod +x for dir_remote.sh

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: I77d670594c128d4f18903bd247a49df7f8ead970
Reviewed-on: http://review.whamcloud.com/4997
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1876 hsm: account for active IOs correctly

We used to use refcount of lsm to account for active IOs but this
turns out wrong because empty layout files don't have a lsm.

In this patch, lov_object::lo_active_ios is invented to account for
active IOs for both raid0 and empty layout;

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I1b7aa16367af1d0a54f6c70bb4155b20071af225
Reviewed-on: http://review.whamcloud.com/5157
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1187 tests: add cleanup for sanity 24q.

Add trap cleanup for sanity 24q.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie8b096cd9906d07812d743cdccdf619b977aaed6
Reviewed-on: http://review.whamcloud.com/4521
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1187 mdt: return lock to client for remote dir.

In mdt_reint_open, MDT should return LOOKUP inodebit lock for
remote directory.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Iaa7c6a13b71b627c5792f4b28bb3c5520b2132e5
Reviewed-on: http://review.whamcloud.com/5026
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1445 ost: enable fid on OST support on OST

Enable fid on OST support on OST.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I51c242e7630d52c578f4ddfe1149b43eb9d5f7c5
Reviewed-on: http://review.whamcloud.com/4791
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1506 ldlm: correct lock res build for FID on OST

Because 1.8/2.1/2.2 client always put oid to res[0] and
oseq to res[1], OFD will do the same to avoid incompatible
problems.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ic4cf66b1bc23631af42530787796510653f82a61
Reviewed-on: http://review.whamcloud.com/3081
Tested-by: Hudson
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1445 mdt: allow lightweight connection even if no OST

MDT should allow lightweight connection from OST even if there
is no OST being activated, because OST might need do FLD lookup
during recovery phase.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ica44aafdbcbf29f10d314bc65de8c1f5ff5d2940
Reviewed-on: http://review.whamcloud.com/4920
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-1445 fld: allow fld lookup during recovery

Allow fld lookup during recovery process, so OST can send fld
lookup req to MDT0 during recovery phase, instead of waiting
the recovery is done.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I69990eb4225d98f20ce2dc197cd3c16b5c6f9617
Reviewed-on: http://review.whamcloud.com/4919
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1445 osd: Add multiple sequence support for osd-zfs

Add osd_seq_list for osd-zfs, then it can allocate new sequence
after FID on OST is landed.

Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I66bae3987a1ae167aa48a3a1d4b6e50f300f1dd8
Reviewed-on: http://review.whamcloud.com/4837
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-1445 osd: add fld look up in osd.

Lookup FLDB during oi_insert, so we know it is MDT or
OST objects. And it also needs to init FLD service
on OFD as well, so object on OST can lookup FLDB
as well.

Change fci_lock to read/write lock, since fld_cache_lookup
will become the hot path after FLD lookup is added in OSD
layer.

Change-Id: I79ef723de603214b7136a51c36f0cb63513b5640
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/4330
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1445 osp: rollover to the new seq synchronously

For simplicity, OSP will track only one sequence for precreation,
so it will need

1. Stop precreation when the current sequence is about to used up,
until all of the precreation objects has been used up.

2. ofd will precreate objects synchronously when the current sequence
is about to used up.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I4c0de8f2869c9a8e7a4b0d2abe06041e84885b2e
Reviewed-on: http://review.whamcloud.com/4790
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-1445 osp: Use FID to track precreate cache.

Change last_used/pre_used/pre_created_id to real fid, so
to track precreate cache with real fid.

For simplicity, precreate/orphan_cleanup between osp and
OFD must be within the same sequence.

Precreate process will be:

1. OSP will request one sequence from OST initially.

2. During precreation, osp will allocate a group of fids
for the precreation request within the current sequence,
then send the request to OST(OFD) with these fids.

3. When getting the precreate request, OST will precreate
the objects by those fids in the request.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ib7f1016321c064656158c54f4475922a199e19b3
Reviewed-on: http://review.whamcloud.com/4789
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2674 osp: set proper connection flag

Set proper connection flag for the osp-on-ost.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Idf80df593f22309565c92630f5c04fc3781e3d34
Reviewed-on: http://review.whamcloud.com/5167
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2561 mdd: create decares attr set with parent attr

mdd_declare_create() mistakenly declares attr set for child with
parent attr, as will cause truncate operation.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I79dbbb2b52e1d41aaa70c36ce6c197ab2cd684d8
Reviewed-on: http://review.whamcloud.com/4956
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2555 ldiskfs: fix ext4_mb_add_n_trim()

In ext4_mb_add_n_trim(), lg_prealloc_lock should be taken
when changing the lg_prealloc_list.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Icc62d87a682b0ebf70ad12e148a52b864bf1fdb8
Reviewed-on: http://review.whamcloud.com/4961
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-2550 osc: set resend count properly

The resend count of new io request should be set properly
in osc_brw_redo_request().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ifce58cf45ae3bfa3b73427f732e281b468cf982b
Reviewed-on: http://review.whamcloud.com/5012
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2517 kernel: Kernel update [RHEL6.3 2.6.32-279.19.1.el6]

Update RHEL6.3 kernel to 2.6.32-279.19.1.el6.

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: I1e89c76d4ec0cc736a0e479a465e59e2262522c0
Reviewed-on: http://review.whamcloud.com/4980
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-1486 tests: fix sanityn test_45g() to add error check

This patch fixes sanityn test_45g() to add error check for
"stat must fail".

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: Iff65f2684244ab58e0d0cab68db0795f5604187e
Reviewed-on: http://review.whamcloud.com/3826
Tested-by: Hudson
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-661 tests: don't check file-2 in replay-dual 0b

It is uncertain was it replayed on not yet, so it might exist
and test will fail. Remove wrong check from test

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I724f389f0504f3c8c388b2b271b3a64cb61339ab
Reviewed-on: http://review.whamcloud.com/4999
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>