LU-2194 test: avoid wrong eviction in recovery_small

in subtest 19a and 19b of recovery_small, the locks from OST/MDT
should be cancelled before "drop_ldlm_cancel" to avoid wrong eviction
from OST/MDT for it will also drop its lock cancellation request.

Change-Id: I49d1d189f6791224a3f132dc8cf2dd8b3d51d43e
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/5679
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2640 osd: conditional enable transaction debug

Current transaction debug mechanism will be disabled
when Lustre-2.4 released.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I516a9026b2d887935c5714dc0b777d65d487dac7
Reviewed-on: http://review.whamcloud.com/5698
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2882 build: build broken with no zfs libraries installed

Reverting default values (originaly set in LU-2391 patch) for
ldiskfs/zfs OSD RPMs builds, to comply for more ways
("rpmbuild -tb <lustre_source_tarball>") used to build Lustre.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I07eca22d03f597942284bbbc0bfd1b680ecb199b
Reviewed-on: http://review.whamcloud.com/5661
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2803 osd: osd-zfs to handle echo sequence (2) properly

use visible OI (/O directory) to map object from this sequence
into dnodes. this let obdecho to work with zfs.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I63e50789df502d11863c69658c9524fbb3cd9f22
Reviewed-on: http://review.whamcloud.com/5414
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2910 clio: skip iov update when tot_nrsegs is zero

When tot_nrsegs is zero, we should skip the iov update too.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I52d4d52a65802f3967dd6a96ad46ec40fd4ef355
Reviewed-on: http://review.whamcloud.com/5747
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1032 build: Support --disable-maintainer mode

We add "AM_MAINTAINER_MODE([enable])" to all configure
scripts to allow us to use --disable-maintainer-mode.

By default, without the AM_MAINTAINER_MODE macro, autotools
"maintainer mode" is enabled.  By specifying "enable" we
maintain our previous default behaviour.

Change-Id: I88366ad658795145af80ed96c6e708c385799ffa
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5423
Tested-by: Hudson
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2468 libcfs: quiet spurious debug message

When cfs_trace_get_tage_try()->cfs_tage_alloc() is allocating a debug
buffer, since e2a2fab993d01597010cb2b44df44a522af0eec8 (b=21776) this
allocation is denied when the allocation is happening in a memory
freeing path.  This caused a spurious "cannot allocate a tage" message
on the console each time.  Quiet that message, since it is expected.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ic800c474dc33f62843b74e06d9ca642cad3ebbe5
Reviewed-on: http://review.whamcloud.com/5470
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2424 ptlrpc: reduce initial buffers count

Separate buffers count for server and client services.
Reduce inititial allocation for client ("ldlm_cbd") because
its not required a lot. This reduced unreclaimable memory
usage after just mount from 233036kB to 97386kB.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I56fcbe6c45c61ba4876bce5482169ea06a03638c
Reviewed-on: http://review.whamcloud.com/5719
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2836 quota: improve test_3 & test_6 of s-q

When approaching quota limit, client turns to sync write, then the
tests which assume io can be finished quickly could fail, we need
to enlarge the time margin to make sure those tests can pass on slow
system.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib14a905344eb78cdbd0cd79e3bfd8e50ab21a4d8
Reviewed-on: http://review.whamcloud.com/5539
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2922 changelog: recommend e2fsprogs-1.42.6.wc2

This patch updates lustre/ChangeLog to recommend
a newer e2fsprogs-1.42.6.wc2 release instead of the
old 1.41.90.wc4 version.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib56092500545f0373a307d7645d41d91079ae086
Reviewed-on: http://review.whamcloud.com/5718
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2883 hsm: Mark file DIRTY as soon as pages are written

Since the dirty flag has to be packed in close, it should be set when
pages are written and not when building BRWs like SOM which relies on
MDS_DONE_WRITING.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I9cf0a71cf3228a7aadb8205cff2735a7abff5ef0
Reviewed-on: http://review.whamcloud.com/5543
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-2622 obdclass: Remove the global cl_env list

New cl_env structures are allocated using a SLAB cache specifically
created for allocating and freeing these structure. Without this patch,
when a thread is finished with a cl_env structure it places it on a
global list instead of freeing it back to the SLAB cache. With this
patch, this global list is completely removed, and cl_env structures are
released immediately back to the SLAB cache.

The motivation for this change essentially boils down to this secondary
global list cache being completely unnecessary, and only serving to
serialize any calls to cl_env_get and cl_env_put. This has proven to
cause a severe performance impact on large core count systems,
specifically during memory reclamation (i.e. ll_releasepage).

For example, on BG/Q Sequoia IO nodes, we've experienced nearly all
68 cores of a machine spinning on the lock protecting this global
list. Some example stack traces showcasing this problem were gathered
using sysrq-l, and are displayed below:

    CPU56:
    Call Trace:
    [c00000000fe3bb30] [c000000000008d1c] .show_stack+0x7c/0x184 (unreliable)
    [c00000000fe3bbe0] [c00000000027604c] .showacpu+0x64/0x94
    [c00000000fe3bc70] [c000000000068b30] .generic_smp_call_function_interrupt+0x10c/0x230
    [c00000000fe3bd40] [c00000000001d11c] .smp_message_recv+0x34/0x78
    [c00000000fe3bdc0] [c00000000002526c] .bgq_ipi_dispatch+0x118/0x18c
    [c00000000fe3be50] [c00000000007b20c] .handle_IRQ_event+0x88/0x18c
    [c00000000fe3bf00] [c00000000007dc90] .handle_percpu_irq+0x8c/0x100
    [c00000000fe3bf90] [c00000000001b808] .call_handle_irq+0x1c/0x2c
    [c0000003e1c4a4c0] [c0000000000059f0] .do_IRQ+0x154/0x1e0
    [c0000003e1c4a570] [c0000000000144dc] exc_external_input_book3e+0x110/0x114
    --- Exception: 501 at ._raw_spin_lock+0xd8/0x1a8
        LR = ._raw_spin_lock+0x104/0x1a8
    [c0000003e1c4a860] [8000000000b04f38] libcfs_nidstrings+0x2acc/0xfffffffffffe5824 [libcfs] (unreliable)
    [c0000003e1c4a910] [c00000000042d4cc] ._spin_lock+0x10/0x24
    [c0000003e1c4a980] [80000000024c2f4c] .cl_env_get+0xec/0x480 [obdclass]
    [c0000003e1c4aa60] [80000000024c336c] .cl_env_nested_get+0x8c/0xf0 [obdclass]
    [c0000003e1c4aaf0] [800000000692070c] .ll_releasepage+0xbc/0x200 [lustre]
    [c0000003e1c4aba0] [c000000000094110] .try_to_release_page+0x68/0x8c
    [c0000003e1c4ac10] [c0000000000a4190] .shrink_page_list.clone.0+0x3d8/0x63c
    [c0000003e1c4adc0] [c0000000000a47d8] .shrink_inactive_list+0x3e4/0x690
    [c0000003e1c4af90] [c0000000000a4f54] .shrink_zone+0x4d0/0x4d4
    [c0000003e1c4b0c0] [c0000000000a5a68] .try_to_free_pages+0x204/0x3d0
    [c0000003e1c4b220] [c00000000009d044] .__alloc_pages_nodemask+0x460/0x738
    [c0000003e1c4b3a0] [c000000000095af4] .grab_cache_page_write_begin+0x7c/0xec
    [c0000003e1c4b450] [8000000006920964] .ll_write_begin+0x94/0x270 [lustre]
    [c0000003e1c4b520] [c0000000000968c8] .generic_file_buffered_write+0x148/0x374
    [c0000003e1c4b660] [c000000000097050] .__generic_file_aio_write+0x374/0x3d8
    [c0000003e1c4b760] [c00000000009712c] .generic_file_aio_write+0x78/0xe8
    [c0000003e1c4b810] [800000000693ed4c] .vvp_io_write_start+0xfc/0x3e0 [lustre]
    [c0000003e1c4b8e0] [80000000024d9c6c] .cl_io_start+0xcc/0x220 [obdclass]
    [c0000003e1c4b980] [80000000024e1a84] .cl_io_loop+0x194/0x2c0 [obdclass]
    [c0000003e1c4ba30] [80000000068ba1d8] .ll_file_io_generic+0x498/0x670 [lustre]
    [c0000003e1c4bb30] [80000000068ba834] .ll_file_aio_write+0x1d4/0x3a0 [lustre]
    [c0000003e1c4bc00] [80000000068bab50] .ll_file_write+0x150/0x320 [lustre]
    [c0000003e1c4bce0] [c0000000000d1ba8] .vfs_write+0xd0/0x1c4
    [c0000003e1c4bd80] [c0000000000d1d98] .SyS_write+0x54/0x98
    [c0000003e1c4be30] [c000000000000580] syscall_exit+0x0/0x2c

    CPU63:
    Call Trace:
    [c00000000fe03b30] [c000000000008d1c] .show_stack+0x7c/0x184 (unreliable)
    [c00000000fe03be0] [c00000000027604c] .showacpu+0x64/0x94
    [c00000000fe03c70] [c000000000068b30] .generic_smp_call_function_interrupt+0x10c/0x230
    [c00000000fe03d40] [c00000000001d11c] .smp_message_recv+0x34/0x78
    [c00000000fe03dc0] [c00000000002526c] .bgq_ipi_dispatch+0x118/0x18c
    [c00000000fe03e50] [c00000000007b20c] .handle_IRQ_event+0x88/0x18c
    [c00000000fe03f00] [c00000000007dc90] .handle_percpu_irq+0x8c/0x100
    [c00000000fe03f90] [c00000000001b808] .call_handle_irq+0x1c/0x2c
    [c0000003c4f0a510] [c0000000000059f0] .do_IRQ+0x154/0x1e0
    [c0000003c4f0a5c0] [c0000000000144dc] exc_external_input_book3e+0x110/0x114
    --- Exception: 501 at ._raw_spin_lock+0xdc/0x1a8
        LR = ._raw_spin_lock+0x104/0x1a8
    [c0000003c4f0a8b0] [800000000697a578] msgdata.87439+0x20/0xfffffffffffccf88 [lustre] (unreliable)
    [c0000003c4f0a960] [c00000000042d4cc] ._spin_lock+0x10/0x24
    [c0000003c4f0a9d0] [80000000024c17e8] .cl_env_put+0x178/0x420 [obdclass]
    [c0000003c4f0aa70] [80000000024c1ab0] .cl_env_nested_put+0x20/0x40 [obdclass]
    [c0000003c4f0aaf0] [8000000006920794] .ll_releasepage+0x144/0x200 [lustre]
    [c0000003c4f0aba0] [c000000000094110] .try_to_release_page+0x68/0x8c
    [c0000003c4f0ac10] [c0000000000a4190] .shrink_page_list.clone.0+0x3d8/0x63c
    [c0000003c4f0adc0] [c0000000000a47d8] .shrink_inactive_list+0x3e4/0x690
    [c0000003c4f0af90] [c0000000000a4f54] .shrink_zone+0x4d0/0x4d4
    [c0000003c4f0b0c0] [c0000000000a5a68] .try_to_free_pages+0x204/0x3d0
    [c0000003c4f0b220] [c00000000009d044] .__alloc_pages_nodemask+0x460/0x738
    [c0000003c4f0b3a0] [c000000000095af4] .grab_cache_page_write_begin+0x7c/0xec
    [c0000003c4f0b450] [8000000006920964] .ll_write_begin+0x94/0x270 [lustre]
    [c0000003c4f0b520] [c0000000000968c8] .generic_file_buffered_write+0x148/0x374
    [c0000003c4f0b660] [c000000000097050] .__generic_file_aio_write+0x374/0x3d8
    [c0000003c4f0b760] [c00000000009712c] .generic_file_aio_write+0x78/0xe8
    [c0000003c4f0b810] [800000000693ed4c] .vvp_io_write_start+0xfc/0x3e0 [lustre]
    [c0000003c4f0b8e0] [80000000024d9c6c] .cl_io_start+0xcc/0x220 [obdclass]
    [c0000003c4f0b980] [80000000024e1a84] .cl_io_loop+0x194/0x2c0 [obdclass]
    [c0000003c4f0ba30] [80000000068ba1d8] .ll_file_io_generic+0x498/0x670 [lustre]
    [c0000003c4f0bb30] [80000000068ba834] .ll_file_aio_write+0x1d4/0x3a0 [lustre]
    [c0000003c4f0bc00] [80000000068bab50] .ll_file_write+0x150/0x320 [lustre]
    [c0000003c4f0bce0] [c0000000000d1ba8] .vfs_write+0xd0/0x1c4
    [c0000003c4f0bd80] [c0000000000d1d98] .SyS_write+0x54/0x98
    [c0000003c4f0be30] [c000000000000580] syscall_exit+0x0/0x2c

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Ief4b524784e07d7677ecb8a9ce97a7b54ccc6f75
Reviewed-on: http://review.whamcloud.com/5204
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2240 osd: change FID of /ROOT on zfs

Pre-production 2.4 code used FID_SEQ_LOCAL_FILE for /ROOT. With
ldiskfs that FID turns into IGIF which is mapped to MDT0 permanently.
With ZFS original local sequence, was used which makes existing setup
incompatibile with DNE. The intention of the patch is to fix this on
existing ZFS setups and replace FID with one from special FID_SEQ_ROOT
sequence which is mapped to MDT0 as well. For simplicity this is done
in few steps:

 - osd-zfs replaces direntry for /ROOT with the new FID and fixes OI
   so that the new FID is mapped to the same dnode
 - MDD finds all objects listed in /ROOT and updates linkEA properly
 - MDD removes ./.. which may be on disk for pre-production setups
 - finally MDD resets LMA on /ROOT with the new FID, which is later
   used to recognize already converted filesystems (or ones created
   with correct FID from the beginning) and skip this conversion code

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I03062c6909146f9a3aed72f41c0708f9ef92bb82
Reviewed-on: http://review.whamcloud.com/5249
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>

LU-951 test: waiting import state in fail().

Anyway, There still has a rare chance that the request meet
a invalid import after fail() return. So we should waiting
import restore to a certain state and doing next operation.
Add wait_import_state_mount() to check import state while
client has a mount point.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I55ccb0fd30d69eae651978804ef1e303d9939a71
Reviewed-on: http://review.whamcloud.com/5531
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-2712 tests: enable sanity/sanityn SLOW tests

Enable a number of subtests in sanity.sh and sanityn.sh that are
currently skipped for normal "review" builds because of "SLOW=no".
Checking the maloo test history for these tests, they are verified
as always passing for "full" builds in the SLOW=yes case.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I340ccdf1323d14a34dc87a5217b3256c8f62cab0
Reviewed-on: http://review.whamcloud.com/5218
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1346 libcfs: remove cfs_ file wrappers

Replace file relevant wrappers with kernel API.

Affected primitives:
file, dentry, dirent, kstatfs, filp_size, filp_poff,
filp_open, do_fsync, filp_close, filp_read, filp_write,
filp_fsync, get_file, fget, fput, file_count, flock_t,
flock_type, flock_set_type, flock_pid, flock_set_pid,
flock_start, flock_set_start, flock_end, flock_set_end.

completion, init_completion, fini_completion,
wait_for_completion, complete

Change some API implementations of darwin/winnt to make it be
consistent with linux kernel API such as filp_open/filp_close etc.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ibe00df71c658aeb5dda854481f6ab5c181b3de7b
Reviewed-on: http://review.whamcloud.com/2830
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2910 clio: restore iov when restart io

The iovector needs be restored on restarted io. This patch also
added some LASSERT and debug message to make it easier to debug
this kind of problem later.

Test-Parameters: testlist=racer
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I5d737cfa083ae3b3d9f040a2dc36d6b8693b548b
Reviewed-on: http://review.whamcloud.com/5652
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2871 lod: stripe data across the OSTs correctly

Since the ost-in-use array is initialized with 0 and isn't set with
OST index number correctly, OST0 is always skipped when allocating
objects on OSTs with specific stripe offset (offset > 0).

For example, when running command "lfs setstripe -c -1 -i 2 testfile"
on 4 OSTs, we will get a wrong layout, like
        obdidx           objid           objid           group
             2               3            0x3                0
             3               3            0x3                0
             1               4            0x4                0
             2               4            0x4                0

To fix the problem, we initialize the array with -1 instead, and store
the correct OST index number in it.

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I5eeced6b66ae40771f8896204c5a6ed8e6663e57
Reviewed-on: http://review.whamcloud.com/5554
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2701 osp: wake up sync thread

osp_sync_process_committed() to wake up sync thread when it
is requested to stop (e.g. umount) and there is no pending
work left. the patch adds a sanity check to ensure this
process is not taking too long.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5251013afc2aee55627c806a11eb826a9d3dbec9
Reviewed-on: http://review.whamcloud.com/5463
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2925 out: increase reqbuf size for OUT

OUT service for DNE can have request size up to 9K, however, it's
using default definition of MDS request buffer size which is 5K.
This patch added individual definitions for OUT reqsize and
req_bufsize.
This patch also made some changes to MDS_BUFSIZE and MDS_LOV_BUFSIZE
to unify style of macros.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I102d6b2aed1e0ed495f055fa3de0c7de7de8c28d
Reviewed-on: http://review.whamcloud.com/5653
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2735 test: disable LRU to avoid page deletion

In sanity.sh, subtest 151 checks whether the page written previously
is in cache or not, this patch disable the LRU of objects in OST to
avoid the object and its content(pages) to be dropped.

Change-Id: Ie481d13215dac599e0b7e122fcc7e9819a053af5
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/5475
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2852 script: sanity 27u should cleanup test dir

sanity 27u should cleanup test dir before test in case there
are remaining files from previous tests.

remove 27u from ALWAYS_EXCEPT since it can pass now.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Id21589c1e1ecf0f4143fad3574cd779c73adb7aa
Reviewed-on: http://review.whamcloud.com/5670
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2449 osd: lookup(..) to fetch fid from parent's LMA

in case LinkEA is not accessible for a reason, osd-zfs
will be trying to get fid for ".." using parent's dnode
which is stored in regular ZFS attributes. parent's
LMA can be used to get fid for ".."

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I34e9f884eb60f036c2f941013bf22e154efc2ff4
Reviewed-on: http://review.whamcloud.com/5629
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-2829 tests: sanityn/33a cleanup messages for zfs

Remove ldiskfs specific checks from running on zfs.
Account for lvm disks for ldiskfs.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I15cdb33d095c465b117761d40d61579eb3fbf52a
Reviewed-on: http://review.whamcloud.com/5693
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>

LU-2926 ldiskfs: crash in is_bad_inode()

Fix error handling in ldiskfs_xattr_inode_iget

Xyratex-bug-id: MRP-883
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Change-Id: I9840b7bd32f2c96763cae402a7bcb51d5798ea6c
Reviewed-on: http://review.whamcloud.com/5631
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2809 llite: Do not return layout_gen for getxattr

The problem is that layout_gen and stripe_offset are sharing the
same field in lov_user_md{}. Layout gen would be wrongly interpreted
as stripe_offset when a backup is restored.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: If2c120bf861eeffd2db5f92e1d23cb1b9a2f5c63
Reviewed-on: http://review.whamcloud.com/5664
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>

LU-2816 llite: Set RAS_INCREASE_STEP correctly

RAS_INCREASE_STEP is by pages instead of by bytes.

However, I found it caused performance loss by setting it to be 4MB so
I set it back to 1MB. After 4MB RPC is enabled by default, more work
should be done to pick up a right value of max_read_ahead_mb to
maximize performance gain of 4MB RPC.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ic000f266cfefc827e112e03f76cf467c73ba88ad
Reviewed-on: http://review.whamcloud.com/5691
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2775 fid: allow FID-on-OST in fid_seq_is_mdt()

The LASSERT_SEQ_IS_MDT() macro used fid_seq_is_mdt() in several
places to verify that a FID was "sane" for where it was being used.
However, there should never be LASSERTs for data from the network
or disk.

The use of LASSERT_SEQ_IS_MDT() is removed from clients, since this
is "validating" data from the network.  This macro is no longer used,
so remove it.  The old CMD objseq_to_mdsno() and mdt_to_obd_objseq()
helpers are also no longer needed for DNE, and can be removed.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I74bc9198799045b8bd91510cb45e8f876012cab0
Reviewed-on: http://review.whamcloud.com/5456
Tested-by: Hudson
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2918 tests: sanity 184c to start copy before layout swap

The test script has to make sure dd already starts to copy file
before swapping layout, otherwise the attempt to open file1 would
fail.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I8217c9a38f1d09830c0ab259f65c9716c06736d1
Reviewed-on: http://review.whamcloud.com/5617
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2791 ldlm: release reference against failed lock

On client-side, when ldlm_cli_enqueue_fini() gets reply from
server, which contains unexpected LVB size, it will mark the
lock as failure, but it does not release one reference, then
the failed lock prevents the lock from being freed. And then
umount client will be blocked.

The ldlm_lvbo_fill() caller should handle failure cases.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I197759a0b964e028627ecb6025820db9517fad7e
Reviewed-on: http://review.whamcloud.com/5634
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2942 ptlrpc: Fix an unswabbed status check in after_reply()

The -EINPROGRESS handling in after_reply() checks reply status while
pb_status still contains unswabbed data.  This patch moves the block
below the unpack_reply() call.

Change-Id: I619b26d213708b8f5c250cdf085f359bba31ffae
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5667
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2713 hsm: limit HSM RPC count from client

Put HSM RPCs under the control of the max_rpcs_in_flight param which
limits the number of concurrent RPCs in flight between a single client
and an MDT.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I760333c7391ffd5aafca396b5fd97ef139799076
Reviewed-on: http://review.whamcloud.com/5616
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2936 ptlrpc: Do not try to fetch hp request blindly

ptlrpc_svcpt_health_check tries to blindly fetch a hp requet from
any service it happens to be called on, but some services don't have
any hp policies registered resulting in an underlyign assertion
in nrs_svcpt2nrs.
Make sure there are in fact pending hp requests on a service before
attempting to fetch them.

Change-Id: Ia38dcce758db948a1e4c187d009da4a8d5f2cbc6
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/5665
Tested-by: Hudson
Reviewed-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>

LU-2190 utils: Use LDD_PREFIX in LDD_*_PROP macros

This patch replaces the hard coded "lustre:" string used in the
LDD_*_PROP macro definitions with the LDD_PREFIX macro. This is
functionally equivalent, but eases the burden if the prefix string
is ever changed (and subjectively makes the code easier to read).

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Ief0e710b90cf922eb4d4cc8a162ee1b0d21317b4
Reviewed-on: http://review.whamcloud.com/5671
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2523 mdt: handle -ENOENT && !MDS_OPEN_CREAT in reint open

If mdt_open_by_fid_lock() returns -ENOENT and MDS_OPEN_CREAT is not
set in create flags then bail out and return -ENOENT from
mdt_reint_open().  In mdt_open_by_fid_lock() if -ENOENT is returned
then ensure that DISP_IT_EXECD is set in the reply disposition.  Add
sanity test_27B to call the LL_IOC_LOV_SETSTRIPE ioctl() on an open
unlinked file, a situation which triggers the first case.  In sanityn
test 30, remote the requirement that opening an unlinked file via
/proc/PID/exe return -ESTALE.  In racer's file_create.sh, reinstate
the call to 'lfs setstripe' before dd is started, effectively
reverting commit 0be1c87fe2d4ffddaca9e568cc137518b7368b2d.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I74e2297c4210ff18acbf60efdf51049d9a88cbea
Reviewed-on: http://review.whamcloud.com/5417
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1468 o2iblnd: Support OFED-3.5 for o2ib

OFED is having new structure based on linux kernel code + backports
and packaging. Here is detailed information.
http://lists.openfabrics.org/pipermail/ewg/2011-December/017156.html

This patches are the lustre build improvements to support OFED 3.5,
3.x whatever future OFED release.

Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Change-Id: Id4ffc39bc7fc24cc591bf6fb47e9b0e662993bda
Reviewed-on: http://review.whamcloud.com/3011
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2449 osd: osd-zfs to initialize parent attribute

to follow ZFS on-disk format osd_object_create() should
initialize regular attribute storing parent dnode.
parent dnode is taken from object passed to create as an
allocation hint.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I1b4b4b5d7c6989c39f6bdecd52af48f270ad5beb
Reviewed-on: http://review.whamcloud.com/5642
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2748 osd: allocate buffers on demand

instead of putting a lot of buffers statically within osd_thread_info,
we can allocate them on the first demand within this thread.
we also can allocate not the maximum, but some optimal amount and
reallocate if really needed. dr_created is not used, so removed.
the number of blocks is calculated using actual blocksize, not the
smallest one, so no need to multiply by 8 in 99.9% cases.

with PTLRPC_MAX_BRW_PAGES=1024 (as default in master branch) and
regular 1MB IO,
before: sizeof(struct osd_thread_info) = 82104
after:  sizeof(struct osd_thread_info) = 4328 + 4K (if IO thread)

should improve threads not doing IO: all MDS threads, LDLM threads,
MGS threads.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ie07780537a4598c6a888ed9be4ef0bbb0d9b3d54
Reviewed-on: http://review.whamcloud.com/5444
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2899 lod: get ldo_stripenr correctly

Current code relies on lod_statfs_and_check() to count the number of
activated LOD targets, while lod::ldo_stripenr derivation happens
before calling lod_statfs_and_check(), and that makes
lod::ldo_stripenr not accurate.

This patch make sure lod_statfs_and_check() called before updating
::ldo_stripenr. And if there is [de]activation happens on OST target,
client needs wait 2*lod_qos_maxage seconds to get accurate
ld_active_tgt_count number.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I37bebc69f876dd68da609fb5180bc6db36f01e84
Reviewed-on: http://review.whamcloud.com/5573
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2804 mdd: move ACL mode handling to MDD

Move ACL mode handling from OSD to MDD, so both ldiskfs and zfs
can be set mode correctly, and also by this, it can avoid to
transfer the local mask to the remote MDT for fixing the mode.

Move sanity 103 out of sanity SLOW list, so it can be run in the
normal review/landing check, so to avoid regression about ACL.
And also it only take 15 seconds in my local VM run, which probably
not be put to the SLOW list at all.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I73146425baa7d8b712ce46e18955ecaa2a3fd9a4
Reviewed-on: http://review.whamcloud.com/5421
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2694 test: improve lfsck.sh

The test directory of lfsck.sh contains some files referencing
same object, which could cause error when removing the directory
on test cleanup.

Test-Parameters: testlist=lfsck
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I334ff2b7b5f77498eed940f009e4bc18728bb5da
Reviewed-on: http://review.whamcloud.com/5606
Tested-by: Hudson
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2694 test: fix is_empty_fs in t-f

The original is_empty_fs is incorrect, which can cause unexpected
error when running lfsck.sh, because lfsck sets the max error
level based on whether filesystem is empty.

Test-Parameters: testlist=lfsck
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I2269cc41744e3c9fe228898323a1508a03616efe
Reviewed-on: http://review.whamcloud.com/5576
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2730 mdt: fix erroneous LASSERT in mdt_reint_opcode

Only set return code err_serious(-EFAULT) when neccessary and do not
run unknown opcode through err_serious.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Iea616d8afd676ee5ccda52cf09b398198f38f992
Reviewed-on: http://review.whamcloud.com/5416
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2859 osc: unplug IO queue async to avoid stack overflow

Otherwise, there is a stack overflow problem with the following
stacktrace:

18 cfs_trace_unlock_tcd at ffffffffa034336c [libcfs]
19 libcfs_debug_vmsg2 at ffffffffa0354038 [libcfs]
20 libcfs_debug_msg at ffffffffa03545f1 [libcfs]
21 osc_key_init at ffffffffa0a90e17 [osc]
22 keys_fill at ffffffffa06d8c3f [obdclass]
23 lu_context_init at ffffffffa06dcc6b [obdclass]
24 lu_env_init at ffffffffa06dce3e [obdclass]
25 cl_env_new at ffffffffa06e40bd [obdclass]
26 cl_env_get at ffffffffa06e4acb [obdclass]
27 lov_sub_get at ffffffffa0b3777d [lov]
28 lov_page_subio at ffffffffa0b37c5d [lov]
29 lov_page_own at ffffffffa0b311af [lov]
30 cl_page_own0 at ffffffffa06e8b5b [obdclass]
31 cl_page_own_try at ffffffffa06e8db3 [obdclass]
32 discard_pagevec at ffffffffa0a926a9 [osc]
33 osc_lru_shrink at ffffffffa0a935d9 [osc]
34 osc_lru_del at ffffffffa0a94aa6 [osc]
35 osc_page_delete at ffffffffa0a951b4 [osc]
36 cl_page_delete0 at ffffffffa06e99e5 [obdclass]
37 cl_page_delete at ffffffffa06e9e62 [obdclass]
38 ll_releasepage at ffffffffa0bfe41b [lustre]
39 try_to_release_page at ffffffff81110070
40 shrink_page_list.clone.0 at ffffffff8112a501
41 shrink_inactive_list at ffffffff8112a8cb
42 shrink_zone at ffffffff8112b5df
43 zone_reclaim at ffffffff8112c384
44 get_page_from_freelist at ffffffff81122834
45 __alloc_pages_nodemask at ffffffff81123ab1
46 kmem_getpages at ffffffff8115e2f2
47 cache_grow at ffffffff8115e95f
48 cache_alloc_refill at ffffffff8115ebb2
49 __kmalloc at ffffffff8115f8d9
50 cfs_alloc at ffffffffa0344c40 [libcfs]
51 ptlrpc_request_alloc_internal at ffffffffa085d407 [ptlrpc]
52 ptlrpc_request_alloc_pool at ffffffffa085d66e [ptlrpc]
53 osc_brw_prep_request at ffffffffa0a8451b [osc]
54 osc_build_rpc at ffffffffa0a8a513 [osc]
55 osc_io_unplug0 at ffffffffa0aa642d [osc]
56 osc_io_unplug at ffffffffa0aa7ce1 [osc]
57 osc_enter_cache at ffffffffa0aa8473 [osc]
58 osc_queue_async_io at ffffffffa0aae916 [osc]
59 osc_page_cache_add at ffffffffa0a94fc9 [osc]
60 cl_page_cache_add at ffffffffa06e61d7 [obdclass]
61 lov_page_cache_add at ffffffffa0b31325 [lov]
62 cl_page_cache_add at ffffffffa06e61d7 [obdclass]
63 vvp_io_commit_write at ffffffffa0c1161d [lustre]
64 cl_io_commit_write at ffffffffa06f5b1d [obdclass]
65 ll_commit_write at ffffffffa0be68be [lustre]
66 ll_write_end at ffffffffa0bfe4e0 [lustre]
67 generic_file_buffered_write at ffffffff81111684
68 __generic_file_aio_write at ffffffff81112f70
69 generic_file_aio_write at ffffffff8111320f
70 vvp_io_write_start at ffffffffa0c11f3c [lustre]
71 cl_io_start at ffffffffa06f244a [obdclass]
72 cl_io_loop at ffffffffa06f6d54 [obdclass]
73 ll_file_io_generic at ffffffffa0bbda7b [lustre]
74 ll_file_aio_write at ffffffffa0bbdce2 [lustre]
75 ll_file_write at ffffffffa0bbeaac [lustre]
76 vfs_write at ffffffff81177b98
77 sys_write at ffffffff811785a1

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Id61b481fe2036a4d7adb7140c39e50fe61c264ba
Reviewed-on: http://review.whamcloud.com/5526
Tested-by: Hudson
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2190 utils: Map ldd_params to individual props

At mkfs/tunefs/mount time the configuration utilities will access
a .ldd_params configuraton string.  This string consists of space
delimited key values pairs which must be saved somewhere.

For ldiskfs it makes sense to store this string on disk as part
of the binary lustre_disk_data structure.  But the logical place
for ZFS is to use the user dataset properties where all the other
lustre_disk_data fields are stored.

This patch updates the ZFS portions of the utilities to take the
.ldd_params string and break it apart in to individual key value
pairs which can be stored as user dataset properties.  This is
done by simply appending 'lustre:' to the provided param key.

The original .ldd_params field can then be reconstructed by
fetching all the dataset properties prefixed by 'lustre:' and
appending them with their keys.

The advantage in this approach is new parameters can be added
without the need to update utilties.

Change-Id: Ida1b0e845b27154b209cc807726ae6ed3d22d189
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5220
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2738 mdt: add proc entry to enable lfs mkdir for non-admin

Add enable_remote_dir_gid to enable lfs mkdir for non-admin user,
1. enable_remote_dir_gid = group, only users, whose gid == group
   can create remote dir.
2. enable_remote_dir_gid = -1, all users can create remote dir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6605016aa89ef3e2763ca14f94a763685fc689b6
Reviewed-on: http://review.whamcloud.com/5442
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2840 tests: Clean the environment for test.

Use unique directory for sanityn test_21. Avoid it
impacted by previous test.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I36a06605732bb0048c25c622d4c8f1192d761a88
Reviewed-on: http://review.whamcloud.com/5556
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2148 kernel: Kernel update for latest FC18 kernel

Add fc18 support for build system

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: Iaeb24b5e44f969eb23a55d115b866c926b25bd55
Reviewed-on: http://review.whamcloud.com/5194
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2922 ldiskfs: depend on e2fsprogs-1.42.6.wc2

This patch updates lustre-ldiskfs.spec.in to depend on
a newer e2fsprogs-1.42.6.wc2 release instead of the
old 1.41.12.2.ora1 version.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I58828fa5427e5f50cfb3b285f980e946e7eb62e7
Reviewed-on: http://review.whamcloud.com/5623
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2858 mdt: check .lustre before reint operation.

1. Check .lustre in MDT layer to make sure any attempt to change
.lustre will return EPERMT.

2. send parent fid for remote open, so mdt can check whether it
will create lov object for the fid under OBF.

3. add a few test cases in sanity 154(OBF checking) to check .lustre
and remote fid

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia708856e026a959ca30b06fdd3acd907e5d1b913
Reviewed-on: http://review.whamcloud.com/5555
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>

LU-2724 ptlrpc: skip NULL obd_svc_stats in lprocfs_rd_import()

In lprocfs_rd_import() don't print trivial obd_svc_stats if they were
not allocated for this import. Add a general proc file read check to
sanity.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic94056eb4f02d71491cdaf948cbe27b82de2153d
Reviewed-on: http://review.whamcloud.com/5234
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2628 tests: disable test_40 of replay-single

This test case assumes that IO to OSTs could go on even when the
connect to MDT is lost, this is not true any more because clients
have to verify the layout is correct before operating OST objects.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I08eaad7b97da7ee152c066426f24bc1d15db5738
Reviewed-on: http://review.whamcloud.com/5056
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2714 hsm: limit MDT-side allocations for HSM RPCs

Limit the amount of memory an MDT will allocate for a single HSM RPC
to 1 MB and add some sanity checking to the HSM handlers that use
variable length buffers.  In hur_len() compute the size of an HSM
request in a portable way.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie03b85a8524cb377bf43446be429cc60c2fe39a7
Reviewed-on: http://review.whamcloud.com/5507
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-2523 tests: disable racer setstripe until fixed

Disable the use of "lfs setstripe" in racer file_create.sh until
the LU-2523 and LU-2789 are fixed, so that racer can be added to
the "review" test workload instead of only in the "full" workload.

Test-Parameters: testlist=racer,racer,racer
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I8b0814ef7864efd817ff585f715183fbf73ebbe5
Reviewed-on: http://review.whamcloud.com/5424
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-903 ldlm: inode references moved to resource

There is a race condition while get_attr after cancel_lru_locks
and sysctl drop_caches. ll_clear_inode clears l_ast_data for
ldlm lock and this lock can't be canceled because "inode == NULL".
ll_mdc_blocking_ast finds such lock. As result DCACHE_LUSTRE_INVALID
is not set and lookup returns wrong inode.

This patch moves inode structure reference to
"ldlm_lock::l_resource::lr_lvb_inode". This prevents from different
inode references for same resource's lock.

Xyratex-bug-id: MRP-338
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Change-Id: I4105b2aec38c90d3f5a20d1498a563192a74de55
Reviewed-on: http://review.whamcloud.com/2627
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2896 mgs: fix handling non IR targets.

non IR targets don't know about extra command as have single
"register target" so we need to treat - none command
as register new target

Xyratex-bug-id: MRP-880
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I22a8fd034772c9355d2d56a166fdf3766edec719
Reviewed-on: http://review.whamcloud.com/5574
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2855 kuc: error management in KUC broadcast delivery

In KUC broadcast groups, the message delivery fails if
a userland process terminates without closing its kuc
channel properly.
This patch improves the behavior of KUC broadcast groups,
to make it more adapted for broadcast usage: the message
delivery is successful if at least 1 userland process is
present in this group and receives
the message successfully.

Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I2a121c7cfcaa6c2ee5ac48b721668bf2f254d848
Reviewed-on: http://review.whamcloud.com/5521
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.3.62

Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I5671b6f298d9f0850d937a443c163758df866341

LU-2885 tests: don't e2fsck zfs partition

Skip check of ext4 extent info on zfs.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I14d98d1d5dd0400c4d6c166a0b4a12c58992f62b
Reviewed-on: http://review.whamcloud.com/5548
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2863 tests: fix lfsck/OI_scrub test scripts issues

1) sanity-scrub.sh checks Lustre version after Lustre initialization.

2) Re-calculate the expactation for lfsck/OI_scrub speed test:
   2.1) 1.1 * the theoretical value for speed upper limit.
   2.2) 0.9 * the theoretical value for speed lower limit.

3) Reformat the device before running sanity-scrub test_11.

4) Other cleanup.

Test-Parameters: testlist=sanity-scrub,sanity-lfsck

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ib811fa24ac7581c341d596afbed064a4fc3e7357
Reviewed-on: http://review.whamcloud.com/5551
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2688 tests: verify quota in conf-sanity 32a

Verify quota in the upgrade test of conf-sanity 32a

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I4b22b16ea31f124f5e39f1b2554df9d4b7dd4b43
Reviewed-on: http://review.whamcloud.com/5293
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-2424 ptlrpc: buffer utilization of rqbd

This patch covered a few things:
- dfferent request buffer size for different MDS service
  Size of MDS request/reply without LOV EA can be way smaller than
  request/reply size with LOV EQ
  This patch defined four different buffer size for different
  MDS services: MDS_MAXREQSIZE, MDS_MAXREPSIZE, MDS_LOV_MAXREQSIZE
  and MDS_LOV_MAXREPSIZE

- add extra 128K to MDS_LOV_BUFSIZE
  MDS_LOV_BUFSIZE should be at least (max_reqsize + max sptlrpc
  payload size which is (MDS_LOV_MAXREQSIZE + 1024)), but if
  MDS_LOV_BUFSIZE is only a little larger than MDS_LOV_MAXREQSIZE,
  then it can only fit in one request even there are 48K  bytes
  left in a rqbd, memory utilization is very low.
  In the meanwhile, size of rqbd can't be too large, because rqbd
  can't be reused until all requests fit in it have been processed
  and released, which means one long blocked request can prevent
  the rqbd bereused.
  Now we give extra 128K to buffer size, so even each rqbd is unlinked
  from LNet with unused 48K, buffer utilization will be above 70%.

Xyratex-bug-id: MRP-689
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I19107918e62f9de59dd88652f3513234c30e56ce
Reviewed-on: http://review.whamcloud.com/4940
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-2083 build: install git commit hooks automatically

Fix for previous commit http://review.whamcloud.com/4175

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I014455aa4e73b7a384356c8d6e65f3feb28e7e1c
Reviewed-on: http://review.whamcloud.com/5494
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2874 tests: mark slow sync zfs tests as EXCEPT_SLOW

For tests that take a very long time on ZFS but not on ldiskfs, place
them in EXCEPT_SLOW when running on zfs.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I52193d9a1c51e1276a5ac86c46f32e5ba5dd6299
Reviewed-on: http://review.whamcloud.com/5553
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2877 tests: sanity test_34h needs to flush cache after write

We need to ensure that the cache is clean after dd in the test,
otherwise subsequent multiop might block trying to flush the
dirty pages for more than 2 seconds and trigger a false failure.

Change-Id: Ifb5a0aa0f9c627b353abe0d402c42a4e14d67609
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/5541
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-2645 ldlm: use correct lvb size to reply 1.8 lock enqueue

For 1.8 client, it does not support variable-sized LVB.
The 2.4 server should correctly distinguish whether the
client support it or not, and fill the reply buffer with
suitable LVB size when processing lock enqueue.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
clientjob=lustre-b1_8 clientbuildno=256 testlist=runtests

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I9241efe25dc64b26e86c4e75da72ab74bb1bc750
Reviewed-on: http://review.whamcloud.com/5459
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2832 ptlrpc: cleanup bulk for resend case

when the request with bulk(ptlrpc_bulk_desc) is resent or replayed,
the stats of the bulk should be cleaned up for it will be reused.

Change-Id: Ie340c8aa43e1a19595c50bed05134537d8d07d74
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/5532
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2776 tests: sleep longer time to yield CPU

It used to be 0.1 seconds and turned out too less.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I2776d8387aee8a55f325459999c8c9454dd2b4fa
Reviewed-on: http://review.whamcloud.com/5321
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2793 fld: send special seq lookup request to MDT0

Since almost all special sequence locates on MDT0, we should
send all seq lookup req to MDT0, especially in an evironment
where other MDTs might not be started at all.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ifda9cb434a217d1e54dc2ef4fcb7628fca049d9d
Reviewed-on: http://review.whamcloud.com/5319
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-2683 lov: release all locks in closure to release sublock

We used to only release current parent lock, this may cause deadlock
if the sublock is shared. See stacktrace of LU-2683 and LU-874 for
details.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ibe5fc364ef22a279f23bb24ad1311a1ad09be369
Reviewed-on: http://review.whamcloud.com/5208
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2805 tests: compare the file correctly for layout swapping

The size of file ref2 may be less than copied bytes so the result of
command cmp will be false due to EOF. Check differ string instead.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I344f90bf4b75f535961f1acdc5574aeca8e85126
Reviewed-on: http://review.whamcloud.com/5420
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2449 mdd: set linkea on /ROOT

osd-zfs/ needs linkea set to handle lookup(..) and readdir.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I35d601dbbc0598da4aa73abb69a59480fa108e0a
Reviewed-on: http://review.whamcloud.com/5413
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1812 osd-ldiskfs: handle 4-arg fsync properly

Fix locking around fsync in osd_object_sync.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: Idc35c1649af7ef7d65fba316ffbcd86574fa847a
Reviewed-on: http://review.whamcloud.com/5115
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1546 mdd: return nlink as zero for dead objects

In mdd_attr_get_internal() check for dead objects and set nlink to
zero accordingly.  Add a sanity check to verify that this works as
expected.

Change-Id: I9dd574cce63bcbf47c9f8f3c7ae6d151c704a45a
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Reviewed-on: http://review.whamcloud.com/3154
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2467 utils: Update wirecheck for new connect flags

This patch adds OBD_CONNECT_PINGLESS to wirecheck and updates
wiretests accordingly.  Also, the connect flag readme comment is
updated to include this as a guidance for adding connect flags in the
future.

Signed-off-by: Li Wei <wei.g.li@intel.com>
Change-Id: I131291dd2dcfda07045a09cbfde4833128464966
Reviewed-on: http://review.whamcloud.com/5269
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1812 ldiskfs: Add kernel patch series for SLES11SP2

Add a kernel patch series for SLES11SP2.

Signed-off-by: James Simmons - <uja.ornl@gmail.com>
Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: I5e3902eb90e69c9652cdfbfc0d8ea1d99d15b453
Reviewed-on: http://review.whamcloud.com/4972
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2548 quota: proper iam index truncation

In quota migration, the iam container needs be reinitialized
after truncation.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I622cc8e4e095cc0c9dcd2d2ce0e245a6de376b3d
Reviewed-on: http://review.whamcloud.com/5292
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2844 mdd: mdt prepare failure should not oops

If mdd prepare fails, lfsck data may not be initialized, mdd
lfsck stop should check this flag.

Add one test to fail mdt prepare, and system should not oops
upon this failure.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ie3e81a2a18aedaa47965800881292c122937506a
Reviewed-on: http://review.whamcloud.com/5519
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1812 ext4: ec_type has been dropped from ext4_ext_cache

In short, EXT4_EXT_CACHE_EXTENT is indicated when both
ee_len != 0 and ee_start != 0.

see kernel commit b05e6ae58a13b56e3e11882c1fc71948c9b29760

Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Change-Id: I88bb07c0e09ea13ac17399edb47de8e16f105543
Reviewed-on: http://review.whamcloud.com/4970
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2473 ldiskfs: Reorganize ldiskfs kernel patches

This commit makes no changes to the patches, only moving
their location.

For some time we have supported only a single kernel per major
OS release, e.g. RHEL 6.3, or RHEL 6.2, but not both.  While
it is understandable to only support a single kernel, it is
quite another to intentionally destroy the patchset for the previous
kernel every time we upgrade.

This makes the logistics of upgrading an OS release (even a minor
one like RHEL 6.2 to 6.3, or 6.3 to 6.4) quite complicated.

We should really start a new patch series when we update support
for a new kernel.  That way we can leave the patch sets for
previous kernels in place.  We do not need to "support" them, but
at least leaving them unmolested until the users of the previous
patch set have time to upgrade would be greatly appreciated.

This commit attempts to organize the patches into subdirectories
according to the kernel/os in which they were first created.  A
later commit will then add support for RHEL6.4.

Also, fix build/confirmpatches.sh to allow checking
ldiskfs/kernel_patches (a bug prevent checking anything but the
default).

Extend both confirmpatches.sh and clearpatches.sh to work with
series files that point to patches organized into subdirectories.

Change-Id: I6552fc271fff1f00657ba1430c4a1215dea5b530
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4803
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Hudson
Reviewed-by: Jeff Mahoney <jeffm@suse.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2658 llite: too many arguments in generic_file_llseek_size

A generic_file_llseek_size starting kernel 3.5 has 5 arguments.
We need to detect and use with the correct number of arguments.

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: I966de37a880a94cde54be84062dec2b91e8334f5
Reviewed-on: http://review.whamcloud.com/5244
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2828 osp: correct osp device finialize order

* Should stop osp precreate thread before releasing its last used
  oid/seq files.

* git commit 71bdcf99 changes the return value of
  writeconf_or_reformat, it's a cleanup function and when writeconf
  does not work it reformat device and should return 0 (success)
  in the end.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I82307dea94bd9ec62088bc931c6b4dd7b90776a0
Reviewed-on: http://review.whamcloud.com/5528
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2781 mdt: initialize mti_spec.sp_rm_entry

Prevent previous assignments to mti_spec.sp_rm_entry from affecting
the current operation by setting it to 0 in mdt_thread_info_init().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iee41cd0e48718338d83b62e0610bd5d23d80dc47
Reviewed-on: http://review.whamcloud.com/5327
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2469 test: Skip failing ZFS tests

Skip tests that are currently failing for ZFS.  Associate with bugs so
that they can be unskipped when they are fixed.

Test-Parameters: mdsfilesystemtype=zfs ostfilesystemtype=zfs   mdtfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I7fe3a2ad51659c4e31240594702a4cdc87a67336
Reviewed-on: http://review.whamcloud.com/5472
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2280 tests: wait osc import to be FULL after sanityn 34.

1. It should wait osc import to be FULL before proceeding to the
following test.

2. fixes in osc_wait_import_event for waiting multiple osc imports.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: Ia054264954a6e0aacadc7c8062c2f353cd87f834
Reviewed-on: http://review.whamcloud.com/4468
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2624 ptlrpc: improve stop of ptlrpcd threads

This fix improves the stop of ptlrpcd threads so that
it occurs in parallel. On machines with a large number
of cores this significantly reduce the time of unload of
ptlrpc module.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I744de5d9887bfaa4c7a742d34923680e3a536ee7
Reviewed-on: http://review.whamcloud.com/5039
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2720 clio: remove CILR_PEEK optimization

some issues with that optimization in mkwrite are found:
1) ENODATA handling wrote the page and invalidated it, but the page
could have been re-added to the cache in between these 2 steps;
2) lock could be busy, so we get ENODATA on trylock, whereas a racy
ll_fault waits for lock mutex and re-adds the page to the cache;

The patch also adds a loop over trying to take a cached lock while
it is used.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I7799b45ff3a0a60f307e05ffd1618d23e0e77b72
Xyratex-bug-id: MRP-752
Reviewed-on: http://review.whamcloud.com/5222
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2591 lov: race between ptlrpc_rcv and umount/umount thread

The race which is refered here happens in the following scenario

1) mount runs but fails to communicate with some OSTs. Then the
   import objects which represents the OSTs are registered to
   a pinger list.
2) pinger succeeds to communicate with, at least, ONE OST. Then
   ptlrpc_rcv calls lov_set_osc_active to activate the OST and
   holds lov_refcount.
4) For some reason ... possibly mount finally fails or umount runs,
   ll_put_super is called
5) ll_put_super tries to disconnect all OSTs with lov_disconnect
   and this func calls lov_del_target to set all OSC's
   target->ltd_reap flags in order for lov_putref to handle all
   of them.
6) ptlrpc_rcv thread puts lov_refcount and if lov_refcount becomes
   0 here, the thread has to disconnect all the OSCs whose
   ltd->reap has been set by __lov_del_obd.
7) Some OSCs' imports have still been LUSTRE_IMP_CONNECTING state
   because of (2), so ptlrpc_rcv thread has to wait for these
   import states to be changed to non-recovery states, such as FULL,
   CLOSED or DISCON at ptlrpc_disconnect_import.

Now that ptlrpc_rcv thread is waiting for the import states to be
changed to non-recovery states but ptlrpc_rcv is the one who is
supposed to change a recovery state to a non-recovery state, So
ptlrpc_rcv mush hung, And mount/umount thread which has called
ll_put_super has to also wait for changing state at
ptlrpc_disconnect_import, so umount must hung too.

Signed-off-by: Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com>
Change-Id: Idcf6831d1ee6b72332c943dfde5316fddba6c13f
Reviewed-on: http://review.whamcloud.com/4979
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2792 fld: add dot_seq into in_fldb check

Add FID_SEQ_DOT_LUSTRE into fid_seq_in_fldb, because the
dot_seq is inserted into fldb cache list in fld_server_init.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6417bab77d48c9d9c991119b29f3d2ab04e88ef2
Reviewed-on: http://review.whamcloud.com/5318
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2801 mdt: checking remote object in mdt_swap_layouts

It should use mdt_remote_object to check remote object in
mdt_swap_layouts, instead of mdt_object_exists < 0.

Send swap layout req to the right MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia24635e16cb67e5683abc9797ec6c9e61f648c62
Reviewed-on: http://review.whamcloud.com/5332
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2739 mdt: Deny non-DNE client to access remote directory

Return EIO to old clients, if they try to access the remote
directory.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1cc4e8fa6e0f3a4b31b5610642e204bd100c1743
Reviewed-on: http://review.whamcloud.com/5425
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2729 mgs: fix cleanup of nonexistant MGSSELF_NAME proc file

The mgs teardown code gets invoked in the failure case but assumes
that the MGSSELF_NAME proc file has been created. If it hasn't been
created, the proc code issues a WARN_ON.

The setup code clears the FSDB_MGS_SELF from &fsdb->fsdb_flags when it
creates the proc file, so we can key off of that to decide whether to
remove it.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: Idf58d1fa572eeced5dbaa72f03e38ee46a708166
Reviewed-on: http://review.whamcloud.com/5122
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2642 osd-ldiskfs: Use __REQ_WRITE instead of BIO_RW.

Commit 7b6d91da removed BIO_RW and replaced it with __REQ_WRITE.
(v2.6.35)

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: I33f87d1e7d8ffe90efcd8b173ad56d9ecf723d7a
Reviewed-on: http://review.whamcloud.com/5121
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2158 lvfs: move obdclass related functions to obclass

Currently a lot of functions exist in the lvfs layer
that really belongs to different parts of the stack.
This patch moves obdclass specific code that is located
in the lvfs layer into the proper location.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I5be4dede1afdbbc4af0157cbcd13a5133ce7a1db
Reviewed-on: http://review.whamcloud.com/5250
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2158 lvfs: move ldiskfs only functions to osd-ldiskfs

Currently a lot of functions exist in the lvfs layer
that really belongs to different parts of the stack.
This patch moves osd-ldiskfs specific code that
is located in the lvfs layer into the proper
location.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia197391599cc7fffad396d487873edd9d6c0b1c4
Reviewed-on: http://review.whamcloud.com/5246
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2789 lod: initialize objects in a detached stripe array

Initializing the ldo_stripe member of lod_device in place exposes some
races with attribute setting so instead we use a detached array and
assign that to ldo_stripe once it's complete.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I63d57a21ed7402829d4b3889c1f32d4ddec1f096
Reviewed-on: http://review.whamcloud.com/5422
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2770 llite: specify ea_size for md_getattr_name

For nfs re-exported case, miss to set ea_size for md_getattr_name(),
then causes the ll_get_parent failure.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I99c8301f9b00cafdf79a6c15541f868f18e4a109
Reviewed-on: http://review.whamcloud.com/5455
Tested-by: Hudson
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2620 tests: Flush changes from test 5 in replay-ost-single

Use wait_delete_completed_mds to flush out and wait for all the
effects on kbytesavail from test 5 to finish before test 6 starts.

Test-Parameters: testlist=replay-ost-single envdefinitions=SLOW=yes

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Id37cf23402dd609e6743c36a1aff8e71dd246776
Reviewed-on: http://review.whamcloud.com/5042
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 cleanup: make bitfield (un)signedness explicit

Fix this common sparse warning (dubious one-bit signed bitfield) in
case ones-complement machines come back or something.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I24ea468efedd8a1269ea92022d13e802f4838ba1
Reviewed-on: http://review.whamcloud.com/5166
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2786 llite: allow setting pool name via fsetxattr()

Setting the pool name in the striping data of a file fails when the
fsetxattr() interface is used to store the lustre extended attributes,
while using ioctl() works as expected.  This is because ll_setxattr()
assumes the size of the lustre user metadata structure to be that of
the older v1 format which doesn't include a pool name field.  Fix this
by using the magic number to detect the format version and setting
the size accordingly when passing the user data down the stack.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I6f6969dfaf2149f268d52272f538fee05459697b
Reviewed-on: http://review.whamcloud.com/5313
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>