LU-1812 fsfilt: ext_pblock renamed to ext4_ext_pblock

For kernels 2.6.35 and above ext_pblock was renamed to
ext4_ext_pblock. With no more RHEL5 kernel support for
ldiskfs we also clean up more macros.

see kernel commit bf89d16f6ef5389f1b9d8046e838ec87b9b3f8b9
for pblock change.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Change-Id: I3ce7f27f6fd6826380e6f2f54b2d50d09d36f78a
Reviewed-on: http://review.whamcloud.com/5001
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1812 fsfilt: ext4_free_blocks() has changed slightly

ext4_free_blocks() now takes a buffer_head and explicit flags instead
of just metadata.  Test the presence of the buffer_head argument to
determine which ext4_free_blocks() is available.

see kernel commit e6362609b6c71c5b802026be9cf263bbdd67a50e

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Change-Id: I925df73a054613469866ec025ae412ead0ce9e56
Reviewed-on: http://review.whamcloud.com/4991
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 build: Cleanup ldiskfs kernel config defines

Make it more clear what the purpose of the "CONFIG_LDISKFS_*"
defines are with a comment, and separate them from options
that do not necessarily originate in the kernel source. All of
these options are needed for ldiskfs and some are needed by
osd-ldiskfs in the lustre file system. To handle these shared
options a extra autoconf header file is create containing only
the values of interest to osd-ldiskfs.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8e71b6cf471c4317b20fdce14d66f8b2883a226e
Reviewed-on: http://review.whamcloud.com/5675
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3043 build: init local variable that breaks sles11sp2 build

Fix a recently introduced build breakage by initing a local variable
in new code.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3c9c1a864e2b529b6a6e2578d0bfbfe2c920688c
Reviewed-on: http://review.whamcloud.com/5854
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-3012 ubuntu: fix client module build

Add --disable-servers to confgiure command when
building the client modules.

Signed-off-by: Robert Read <robert.read@intel.com>
Change-Id: Ie29bf9336c476dd0b8b04b4601d409c4bc7f90f1
Reviewed-on: http://review.whamcloud.com/5804
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2988 mgs: Fix two "lctl replace_nids" resource leaks

When conf-sanity 66 was run on a single machine, it failed to remove
some Lustre kernel modules in the cleanup phase:

  Modules still loaded:
  ldiskfs/ldiskfs/ldiskfs.o lustre/mdd/mdd.o lustre/mgs/mgs.o
  lustre/quota/lquota.o lustre/mgc/mgc.o lustre/fid/fid.o
  lustre/fld/fld.o lustre/ptlrpc/ptlrpc.o lustre/obdclass/obdclass.o
  lustre/lvfs/lvfs.o lnet/klnds/socklnd/ksocklnd.o lnet/lnet/lnet.o
  libcfs/libcfs/libcfs.o

Some simple experiments quickly narrowed down the bad guy to the first
"lctl replace_nids" command.  In mgs_iocontrol(), the
OBD_IOC_REPLACE_NIDS case does not destroy the lu_env, which
references several Lustre kernel modules via the keys.  This patch
fixes the leak by replacing "RETURN(rc)" with "break".

Local testing revealed another issue, however, after the first issue
was fixed.  unload_modules() complained about a memory leak after
removing all modules:

  LustreError: 14530:0:(class_obd.c:701:cleanup_obdclass()) obd_memory
  max: 28770011, leaked: 18446744073709551608

The leaked number is clearly suspicious.  It turned out that
mgs_replace_nids_log() frees more memory, with regard to accounting,
than it allocates.  This patch also fixes the fake leak.

To prevent regressions, this patch adds error checking to the
cleanup() call in conf-sanity 66.

Change-Id: Ia3b1531b558a2a12947ff9a783b383962ae5da78
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5765
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2467 ptlrpc: Fix an unideal symbol export

The "suppress_pings" symbol is a little bit too short to export.  This
patch exports a function, ptlrpc_pinger_suppress_pings(), instead.

Change-Id: Ifdcf5d2459baa7ae2709572a2fd4d02b72e440df
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5270
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3011 ubuntu: Fix build failures on Ubuntu 12.04

Fix "set-but-unused" warning by using the variable,
and move -lreadline to end of link command line.

Signed-off-by: Robert Read <robert.read.@intel.com>
Change-Id: I676e319ed81dbb6ba41d039e7b075b02d5122b48
Reviewed-on: http://review.whamcloud.com/5803
Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1330 obdclass: split client-server mount routines

Move server side mount routines to obd_mount_server.c.  Const correct
several server_name2xxx type functions.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8abdb6fdd0411f2e75f6fb6ee4ff8502e50ef213
Reviewed-on: http://review.whamcloud.com/2672
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-398 ptlrpc: NRS framework follow-up patch

This patch addresses some outstanding issues that had been raised
by reviewers of the "Add the NRS framework and FIFO policy" patch,
and include some other improvements, e.g. it reworks the API
slightly in order to optimize some frequently-used operations,
does not uanncessarily policies in liblustre and client-only
kernel builds, and makes sure we hold module references when
required, for policies registering from other modules.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: I9306d43e2aef20aa64d6870a56ae99859ce40cd5
Oracle-bug-id: b=13634
Xyratex-bug-id: MRP-73
Reviewed-on: http://review.whamcloud.com/5274
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2442 kernel: protect i_dquot with i_lock

Remove dqptr_sem (but kept in struct quota_info to keep kernel ABI
unchanged), and the functionality of this lock is implemented by
other locks:
* i_dquot is protected by i_lock, however only this pointer, the
  content of this struct is by dq_data_lock.
* Q_GETFMT is now protected with dqonoff_mutex instead of dqptr_sem.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I6be343fb7e431bb3b0ce68066a36f621ebdd9df5
Reviewed-on: http://review.whamcloud.com/5010
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2189 tests: ensure sync stability before sanityn/36

Encourage data to be fully sync'd before getting new measurements.

Test-Parameters: testlist=sanityn  mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I42be5714b59cd4c22c65d7524f2b71ec0a07dfa4
Reviewed-on: http://review.whamcloud.com/5722
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2790 ldlm: handle lvbo_init failure in ldlm_resource_get

Under some special cases, such as RAM pressure, lvbo_init()
may be failed, then the caller - ldlm_resource_get() should
handle the failure to prevent subsequent operations to use
non-exist resource.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0eabbf5daaaba9aa163a45f24b6b621477ec4d32
Reviewed-on: http://review.whamcloud.com/5699
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3068 build: fix 'incorrect expression' errors

Fix 'program hangs' defects found by Coverity version 6.5.1:
Array compared against 0 (NO_EFFECT)
Comparing an array to null is not useful.
Copy-paste error (COPY_PASTE_ERROR)
This line looks like a copy-paste error.
Self assignment (NO_EFFECT)
Assignment operation has no effect.
Side effect in assertion (ASSERT_SIDE_EFFECT)
Assignment has a side effect. This code will work differently in a
non-debug build. You might have intended to use a comparison instead.
Wrong sizeof argument (SIZEOF_MISMATCH)
Passing argument is suspicious.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Icf6ea9632da6159beca0fd9fd3ff9bb57effc305
Reviewed-on: http://review.whamcloud.com/5887
Tested-by: Hudson
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3066 llite: fix crash in fdatasync(directory)

kernel NULL pointer dereference for fdatasync(directory)

fd = open("/mnt/lustre", O_RDONLY|O_NONBLOCK|O_DIRECTORY);
fdatasync(fd);
close(fd);

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib14b25d1694131e1a65373654008b7f337ce959e
Reviewed-on: http://review.whamcloud.com/5886
Tested-by: Hudson
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3035 clio: improper LASSERT in ll_file_io_generic()

This LASSERT was introduced from the fix of LU-2910, and
which is incorrect, since the crw_count could be changed
in lov_io_rw_iter_init() even if no any read/write done.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib2cee88f0a7f75f8fb63330912aa053fb5b9393e
Reviewed-on: http://review.whamcloud.com/5864
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2909 osc: flush until no dirty in osc_enter_cache()

In osc_enter_cache(), when there is high contention on grant, the
returned grant could be consumed by other process immediately, then
we should repeat flush until get enough grant or no dirty to be
flushed. Otherwise, mmap writer could easily get -EDQUOT on the
osc_enter_cache() and result in SIGBUS at the end.

Because we now changed to async flush in osc_enter_cache(), the
wakeup condition is changed accordingly.

This patch also temporarily disabled osc lru shrinker in
osc_io_unplug() to avoid the potential stack overrun problem.
See LU-2859.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I0c7c90ffe27dab6ded7ad07ed78017acb8665d59
Reviewed-on: http://review.whamcloud.com/5732
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2388 statahead: race in do_sa_entry_fini()

Two fixes:
* When iterating sa_entry list in do_sa_entry_fini(), there is
  no lock, as may cause one entry put twice. To fix this, all
  entries are put in one list, and only 'scanner' will drop
  entry from this list.
* sa_entry may be linked to sai_sent_entries, but not hashed
  yet, if ll_sa_entry_fini() is called at this moment, this
  sa_entry may be unhashed.

Also include minor cleanup:
* rename do_sai_entry_fini() to do_sa_entry_fini().
* rename do_sai_entry_to_stated() to do_sa_entry_to_stated().
* rename do_statahead_interpret() to ll_post_statahead() to
  distinguish from ll_statahead_interpret().
* ll_post_statahead() always post handle statahead from received
  list to simplify logic.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I3d0911d0bd3b940c9650473099604646408200c4
Reviewed-on: http://review.whamcloud.com/5842
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 build: Relocate missed lbuild-fc18

lbuild-fc18 was missed in the move of the other lbuild files
to contrib/lbuild.  This relocates that file as well.

Change-Id: I2775f1e0aa4c7d17d2e1d8a114f2bea3702fec68
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5872
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3024 test: use facet name in init_param_vars

When configuring jobstats init_parm_vars should call
set_conf_param_and_check with the facet name (client) rather then the
hostname.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I00b99ec3172d3e108fb6dd6d94825badf81e84df
Reviewed-on: http://review.whamcloud.com/5835
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2826 tests: Re-enable sanity/18

Due to ZFS work in LU-2449, sanity/18 should now work correctly.

Test-Parameters: mdtfilesystemtype=zfs mdsfilesystemtype=zfs  ostfilesystemtype=zfs testlist=sanity
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I5d9d986b00a36a2b5f30ef9b4ff3d52779070498
Reviewed-on: http://review.whamcloud.com/5720
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2900 llite: Null deref in ll_fsync:mkdir on NFSmounted Lus

When a Lustre file system is mounted via NFS and a mkdir
operation is attempted, a null pointer dereference occurs
in ll_fsync.

With 2.x, Lustre added support for different VFS fsync APIs
that do not include a dentry parameter.

To make the logic the same in all cases, the old ll_fsync
interface was changed to pull the inode from f_dentry
in the *file parameter.

In some cases when using the old ll_fsync interface, the
caller does not set the f_dentry part of the *file parameter
resulting in a NULL dereference. The fix to this is to
restore the old logic in those cases: when a dentry
parameter is provided, get the inode from that parameter
rather than the file parameter.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I93ecf04e23121c76e571383d74e2fc902565614e
Reviewed-on: http://review.whamcloud.com/5585
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3038 mgs: clear mti in the end of marker

The target information should be cleared when it met
the end marker of each mdc/osc, otherwise, the information
left over in mti will be brought into the config log of
next target, even worse some target might be identfied
wrongly as failover pair.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I67838914825caf0a9b4da8bc3cdfbb779a6eadd4
Reviewed-on: http://review.whamcloud.com/5851
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2995 osd: Delete the ref of remote parent in del ..

Try to delete remote dir from REMOTE PARENT during delete ..,
because the object might not be destroyed if it is being
opened by someone else, instead it will be inserted to the
ORPHAN dir, so if it does not delete the {.., REMOTE_PARENT},
it will be two parents for this remote dir.

Add remote object FID into op_data->op_fid3, so the stale check
will not fail during remote open.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6bbac9174869ae02f385ac52740bb2046f0f1bab
Reviewed-on: http://review.whamcloud.com/5791
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2926 ldiskfs: crash in is_bad_inode() on SLES11 SP2

Fix error handling in ldiskfs_xattr_inode_iget
for SLES11 SP2 platform

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I05568cac5f4cc8a12cfd95034a99b06541289673
Reviewed-on: http://review.whamcloud.com/5709
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1199 build: Clean out the build directory

Clean up the build directory.  Move in the direction of reserving
"build/" for the special-purpose Makefile that lives there.
Eventually we could rewrite the autoconf tests to eliminate that
Makfile, and the build directory could disappear altogether (after
finding homes for anything else that is left).

The autoconf m4 file move into a top level "config" directory.

Most other things that have moved are put in the new "contrib"
top-level directory.  For instance, "contrib/lbuild" contains
all of the lbuild-related files, and "contrib/git-hooks" contains
the git hooks for lustre developers.

Most of the moved files were unchanged, however the lbuild scripts
needed some tweaking to deal with the new location.

Because of the way that Intel's build farm and git hooks expect to
find certain files in fixed locations I have had to leave a few
symlinks in place that point to the new locations.

Change-Id: I04dc529d4f4060b892e1e4eaa8613bbc3337c414
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5035
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2682 fid: cleanup direct _id and _seq access

Access/copying the individual _id and _seq fields should be
replaced with the _oi field, and compatibility macros for
accessing the individual fields should be removed.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: Iedce5cfb308c1b6de6aa24872b8c71e7b616b67c
Reviewed-on: http://review.whamcloud.com/5601
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2970 lov: wait for active IOs before deleting the object

lov_layout_operations::llo_delete will be called when the file's
layout is going to change or file is being taken out of cache so we
must make sure there is no active IOs on this object.

For layout change case, it's lov_layout_change()'s responsibility to
make sure there is no active IOs before entering into llo_delete.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Id56a66548f9f05555e6d112355d2a71da5ab4705
Reviewed-on: http://review.whamcloud.com/5812
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1187 osd: add remote entry insert for ZFS DNE.

Add remote entry insert in osd_index_insert for DNE on ZFS.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I4c2337e69e4c4c4754b8ff7995dd69318f3d674d
Reviewed-on: http://review.whamcloud.com/4933
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2783 ofd: Implement OBD_IOC_GET_OBJ_VERSION

This patch implements OBD_IOC_GET_OBJ_VERSION for ofd, so that tests
can use "lctl getobjversion" on OST objects.  jt_get_obj_version() now
takes ID and group pairs, as well as FIDs, for OST objects.  Although
the code are for testing purposes only, dereferencing an ioc_inlbuf
without checking its size is still unacceptable.  Hence,
mdt_ioc_version_get() is changed to check sizes first.

Signed-off-by: Li Wei <wei.g.li@intel.com>
Change-Id: If259aeca281dcb3f4ee4dc719719f722cf72cad4
Reviewed-on: http://review.whamcloud.com/5467
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3025 hsm-llapi: bad type cast in llapi_hsm_copytool_fini()

Correct a wrong type cast in llapi_hsm_copytool_fini()

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Iebe4a36178afdfad8b99140efd6b3afc8b9eb942
Reviewed-on: http://review.whamcloud.com/5837
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2646 osd: add special flag for agent inode.

1. Add special flag into the LMA of agent inode of remote entry,
so lfsck check can skip this inode.

2. Assign a local FID for Agent dir, so lfsck can skip this agent
dir as well.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ifc50fbc531a4306a76d5263dd6135c613d7a2a30
Reviewed-on: http://review.whamcloud.com/5625
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3009 tests: failure on conf-sanity test_17

Test 17 is failing due to the failure of the debugfs command
in this subtest.  The command fails because MGSDEV is undefined.
Using an alternative that is available in the current test framework.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I93407dd3c349d91a2ea4690b70dd67c68d65582c
Reviewed-on: http://review.whamcloud.com/5816
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3002 tests: Fix an OST index issue in replay-single

When run with 16 OSTs, replay-single 90 happened to pick ost14 as
ostfail and complained:

  lfs find does not report the affected lustre-OST000d_UUID for fd

The "d" in "fd" is a hexadecimal OST index, taken from
"lustre-OST000d_UUID".  Because the actual file names use decimal OST
indices, the test failed to match the file "f13" with "fd".  Moreover,
the test assumes the OST index is a single-digit number, which may not
be case with more than 15 OSTs.  This patch changes the test to simply
calculate the index from the facet name.

Test-Parameters: ostcount=32 testlist=replay-single envdefinitions=ONLY=90
Change-Id: I0c95d28f83c41cabf4ed8e5ed618897d6000da3b
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5796
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3003 tests: Do "lctl conf_param" on mds1 in t32_test()

With DNE, conf-sanity 32c inovkes "lctl conf_param" on mds2's host,
which may not be where the MGS is running, causing errors like:

  CMD: c05 /usr/sbin/lctl conf_param
  t32fs-MDT0001.mdc.max_rpcs_in_flight=9
  c05: No device found for name MGS: Invalid argument
  c05: This command must be run on the MGS.
  c05: error: conf_param: No such device
   conf-sanity test_32c: @@@@@@ FAIL: Setting MDT1
  "max_rpcs_in_flight"

This patch changes these "lctl conf_param" commands to set the same
parameters but on mds1's host, which always hosts the MGS.

Change-Id: Ibdca2802eaf33db504ed2b41a356d7a18be33f10
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5829
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1897 test: replay-single test_70b dbench not found

dbench is taking too long to start. It was observed
that a 12 seconds sleep was not long enough to start
dbench on all clients.

I added a loop to rally all clients before starting
the testing and improved the error messages.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I7b0fce4c39c003c4db23686b295fb5e45a6f8a92
Reviewed-on: http://review.whamcloud.com/5761
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2556 osp: count requests in osp_sync_interpret() only

as it's called by ptlrpc always, so any counting in
osp_sync_request_commit_cb() is extra and wrong.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ib67ba11d2c382b1ff9cd19522ce9e15b57ebca94
Reviewed-on: http://review.whamcloud.com/5453
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1330 obdclass: splits server-side object stack from client

1. removes client-side dependencies of server object stack:
md_object.h/md_local_object.c, and dt_object.h/dt_object.c.
2. removes client-side dependencies of lustre_idmap.h/idmap.c
3. splits some server-side acl handlings

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie3a3cd998ef7b2d7d1090673a7c2d8f1152ca976
Reviewed-on: http://review.whamcloud.com/2676
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-2951 mdt: Increase bc_req_max_size for MDS_REQUEST_PORTAL

Large EA tests triggered error messages like this on MDSs:

  10:38:21:LNetError: 3022:0:(lib-ptl.c:190:lnet_try_match_md())
  Matching packet from 12345-10.10.17.9@tcp, match 1429230968490588
  length 65928 too big: 117674 left, 49386 allowed

These were the REINT_SETXATTR requests carrying large EA values.  They
were dropped because the MDSs did not expect request buffers larger
than 49386 bytes.  This patch increases bc_req_max_size (and
bc_rep_max_size) for MDS_REQUEST_PORTAL (and MDS_REPLY_PORTAL) to
accommodate REINT_SETXATTR requests (and MDS_GETXATTR replies) with
255-byte names and 65536-byte values (or 65536-byte lists of EA
names).

Change-Id: Ifdcda6d3e91aa6115e5cafd5abd6c89b15485020
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5703
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2445 utils: lfs migrate support

Add a new command migrate to lfs utility.
migrate takes sames args as setstripe and a --block option
It:
- create a volatile file based on new stripe information
- copy file data to this volatile file
- swap the 2 layouts
- if --block is set, take the grouplock to block other access
  during copy

So after migrate the file is restriped (mtime, atime are not
changed).
lfs_migrate is also updated to use the new call and
keeps the rsync mode as a fallback.

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I96bafb0be9bc273295c4c900c65b4028864fcbaa
Reviewed-on: http://review.whamcloud.com/5620
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2895 osp: return proper error in osp_precreate_reserve

Make sure the osp_precreate_reserve() always return proper error
when reserving object failed.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9d16135091ebe5ef1243416a15722c4859026e46
Reviewed-on: http://review.whamcloud.com/5683
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2960 lmv: typo fix for output of "lfs mdts"

Using ltd_idx(in lmv_tgt_descs) directly for the output of
lctl get_params lustre.lmv.*-clilmv-*.target_obd.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6a4ff4b82e4d413baf1f36f85764d3c11cd82435
Reviewed-on: http://review.whamcloud.com/5782
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2756 ost: buffer utilization rate of OST rqbd

This patch covered a couple of things:
- different request buffer size for different OST services
  only requests on OST_IO_PORTAL need large buffers

- add extra 46K to OST_IO_BUFSIZE (buffer size is increased to 64K)
  size of OST_IO_MAXREQSIZE is about 18K, if OST_IO_BUFSIZE is only
  a little bigger than OST_IO_MAXREQSIZE, then each rqbd can only fit
  in 2-3 request even there are 18K bytes left in this rqbd, memory
  utilization rate is low. Extra bytes can make buffer utilization
  rate increase to 72%. Also, it can avoid version compatibility
  issue in the future if we want to increase request size again.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Iaf8b45520b000dd5219a4af2ec3f5b2ea7f53134
Reviewed-on: http://review.whamcloud.com/5626
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2927 mdt: Not return -EREMOTE for open lock.

Instead of returning -EREMOTE for open_lock, it will check
the open lock by the DISP_OPEN_LOCK, otherwise it will cause
transno being wrongly set to 0.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I4f3bccea87872ca191b0b6c2142e8897be3d731a
Reviewed-on: http://review.whamcloud.com/5694
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2920 autoconf: delete unused LB_LINUX_MODPOST

LB_LINUX_MODPOST does not seem to be used anywhere.

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: Ibec2002ae8b74b9c6f1f6df82a35d44bd00916dc
Reviewed-on: http://review.whamcloud.com/5615
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.3.63

Change-Id: I02e9d699d3a7c5dc4253ad24dee0b10651cb4ba7
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2398 osc: ignore layout for osc_lru_shrink()

osc_lru_shrink() doesn't need to check verify layout change
because layout can't be destroyed if a page already exists.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I034c15cbecfaf1491d7d7fcb992b74e3d94d9fa5
Reviewed-on: http://review.whamcloud.com/4701
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2955 tests: make replay-ost-single/8b SLOW for ZFS

This test runs dd with osync, which is currently a VERY slow
operation, so mark it slow until LU-2887 is addressed.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I3f55278490bd3c2e3bce8a2de58828a3a716b2fe
Reviewed-on: http://review.whamcloud.com/5767
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2405 tests: account for inter-op testing sanity/17g

Account for where patch hasn't been applied and skip known broken
servers.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I1dd354c36c38e2569ce0cfaf57d4cd35a65117ff
Reviewed-on: http://review.whamcloud.com/5754
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2813 hsm: Add user credential information to copy tool RPC

Add MDT_BODY data to the RPC used for coytool, so the
coordinator can make some check based on the user

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Ia45aacfdc78411f7c202bafb6bd6411f686606a6
Reviewed-on: http://review.whamcloud.com/5436
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2783 include: Fix a conditional indentation

Just an indentation fix to make the conditional expression in
fid_is_sane() easier to read.

Signed-off-by: Li Wei <wei.g.li@intel.com>
Change-Id: I1112d7d43c29f032c79277c6434d4c02e96e49bd
Reviewed-on: http://review.whamcloud.com/5466
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2743 mgs: Add a missing rc reset

Current mgs_write_log_target() does not reset rc when EALREADY is
found.  This may cause the positive errno to leak to upper layers and
eventually to wire.  This patch adds the missing rc reset.

Change-Id: I0ba8a0decdb9583224a5de02f630742ae94fff39
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5730
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2743 llite: Fix a negative hpk_errval assignment

The hpk_errval field should store positve errnos.  This patch fixes a
place where errnos are assigned to hpk_errval as negative numbers.

Change-Id: I77b1f9e6a19a64aedcdcc5fe38b08babb810a472
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/5729
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2929 build: fix unused/uninitilized virables error

As reported by gcc (GCC) 4.6.3 20120306 (Red Hat 4.6.3-2).

Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I05aeb88dbd00b6b005ddde543a527339a9d66ce0
Reviewed-on: http://review.whamcloud.com/5656
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org>

LU-2335 lnet: remove unnecessary libcfs include

libcfs_pack/unpack.h are only needed for WINNT.
Only include them when necessary.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: Ibe6230d37534fd087c2afe6166d10259fd59c076
Reviewed-on: http://review.whamcloud.com/4647
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>

LU-2675 cleanup: remove unused recov_thread.c and llog code

Remove recov_thread.c and the following exported functions:
  llog_cat_process_thread
  llog_obd_origin_add
  llog_catalog_list
  llog_receptor_accept
  llog_origin_connect
  llog_handle_connect
  llog_obd_repl_cancel
  llog_obd_repl_sync
  llog_obd_repl_connect
  llog_get_cat_list
  llog_put_cat_list
  llog_recov_thread_start
  llog_recov_thread_stop

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I620c641646294fcf697f1e2d4d1f389d71bb3ddd
Reviewed-on: http://review.whamcloud.com/5190
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liu Xuezhao <xuezhao.liu@emc.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-2396 ofd: Remove unmaintained brw_stats

Remove unused nid_brw_stats member from struct nid_stat and the files
/proc/fs/lustre/obdfilter/*/{brw_stats,exports/*/brw_stats}.  If the
corresponding osd has a brw_stats proc entry then add a symlink from
the ofd proc dir.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I79a10816c612f67b80172d679b875ca94157b5f4
Reviewed-on: http://review.whamcloud.com/4689
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-2961 build: Fix commit-msg to love new files

It is annoying to receive errors like this when the changes being
committed include new files:

  $ git commit -asv --amend
  line 65: diff --git a/lustre/include/lustre_errno.h
  b/lustre/include/lustre_errno.h
  new file mode 100644
  error: commit message invalid signoff section line
  line 66: index 0000000..89b8d1b
  error: commit message invalid signoff section line
  line 67: --- /dev/null
  error: commit message invalid signoff section line
  ...

This patch adds the "new file mode" pattern to commit-msg.  Hopefully,
this is enough.

Signed-off-by: Li Wei <wei.g.li@intel.com>
Change-Id: I39c985d51afb04184322a8fc725bcef3938bf419
Reviewed-on: http://review.whamcloud.com/5712
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Bruce Korb <bruce_korb@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1434 llog: remove unused llog_create_rec struct

Remove the unused llog_create_rec and llog_orphan_rec definitions
from wireshark.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I21c1215b627fb972d8da89a7ff1965848027cab0
Reviewed-on: http://review.whamcloud.com/2877
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-812 ldiskfs: super_operations->dirty_inode now takes a flag

Currently this flag is unused by ext4, so just pass in 0.  This change
happened in kernel commit aa38572954ade525817fe88c54faebf85e5a61c0.
Apparently the flag is used to tell the difference between timestamp
updates and anything else.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil>
Change-Id: I24536546256f5f043c1f53e15220b0c956be343f
Reviewed-on: http://review.whamcloud.com/4966
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1994 kernel: fix reference counting with l_dentry_open

Commit 78b1d1bd (LU-1994 kernel: 3.6 dentry_open uses struct path
as first arg) added support for the new dentry_open call that
accepts struct path instead of a dentry/vfsmount pair, but missed
the new reference counting rules that go along with it.

Upstream commit 765927b2 also makes dentry_open grab references itself
so it no longer frees references that weren't passed to it.

On failure, we'll end up with an extra reference to the dentry
that was passed in.

Since new dentry_open is the one that will be around for the
foreseeable future, let's just map to that directly for the path case.

For the other two cases, we'll take the references ourselves in
ll_dentry_open, free them on failure, and adjust callers to expect
that it won't free any references passed to it.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: I05a95cf735a5b2d70273a485335d571fcda7a6b0
Reviewed-on: http://review.whamcloud.com/5330
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-1798 tests: don't set jobid if not changing

Don't set jobid_var conf_param if the value would not change.  This
avoids setting the same parameter in the config log multiple times,
and avoids the extra delay on each startup waiting for the conf_param
to propagate from the MGS to the client each time.

Fix the console error printing if the jobid_env does not exist.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I36ed60451a875bcd46fd0d6f3d7068d1b1398df5
Reviewed-on: http://review.whamcloud.com/3867
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2484 mgs: remove unused md_stats

Do not allocate md_stats or create /proc/.../MGS/md_stats for mgs
devices since they do not implement the md_ops interface.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8b417bc2433484a0e008ff9bf7fef69ba1e62416
Reviewed-on: http://review.whamcloud.com/4813
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2513 osc: compute grant targets in bytes

In osc_shrink_grant() and osc_shrink_grant_to_target() convert page
unit target values to bytes before comparing to cl_avail_grant.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie8e6e8b4b3245efa3b14777608f3a48bbab7e4e2
Reviewed-on: http://review.whamcloud.com/5495
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2675 cleanup: remove obsolete llog-test crud

Do not create the llog-test.c symlink as this Linux 2.4-ism is no
longer needed. Also remove the now defunct llog-test.sh.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I72ba710659e168ead206a2b9b28633724d0e44c5
Reviewed-on: http://review.whamcloud.com/5239
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2916 llite: call ll_permission rather than inode_permission

So that we can build on kernels older than 2.6.27.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I288030c3ee37ccb909d45121d457adb4dccafe0a
Reviewed-on: http://review.whamcloud.com/5607
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2642 osd-ldiskfs: fix typo in REQ_WRITE check

Commit eecb3086 defines __REQ_WRITE as BIO_RW if __REQ_WRITE is
not defined. __REQ_WRITE is an enum. REQ_WRITE is a define.

Let's use that instead.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I500c771e6f3e69e1be3fab872150019103515d30
Reviewed-on: http://review.whamcloud.com/5503
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2873 tests: t-f generates numerous dots garbage

Remove 'echo -n "."' from run_test t-f function when skipping test

Signed-off-by: Kyrylo Shatskyy <kyrylo_shatskyy@xyratex.com>
Reviewed-by: Roman Grigoryev <Roman_Grigoryev@xyratex.com>
Xyratex-bug-id: MRP-863
Change-Id: I2c6de9fff3676aee7a506a2c2f6b6f9fe7a3b472
Reviewed-on: http://review.whamcloud.com/5540
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2731 scripts: Speed up /etc/init.d/lustre stop

This patch parallelizes the shutdown of multiple services running on the
same node. This has been empirically shown to drastically reduce the
runtime of the script for an OSS with many OSTs.

This patch was tested on a Lustre 2.1 ldiskfs OSS node with 32 OSTs
attached, by recording startup and shutdown times for the OSS. The
number of OSTs used in varied, ranging from a single one, up to all 32,
incrementing by powers of two (i.e. timed startup/shutdown of 1 OST,
then of 2 OSTs, then 4, etc.).

Results of startup and shutdown times *without* this patch applied:

    +------------------------------------------------+
    | $ time /etc/init.d/lustre start # (w/o patch)  |
    +-----------+------------+-----------+-----------+
    | # of OSTs |    real    |    user   |    sys    |
    +-----------+------------+-----------+-----------+
    |      1    | 0m  2.184s | 0m 0.162s | 0m 0.077s |
    |      2    | 0m  4.285s | 0m 0.281s | 0m 0.148s |
    |      4    | 0m  8.508s | 0m 0.500s | 0m 0.302s |
    |      8    | 0m 16.961s | 0m 1.017s | 0m 0.568s |
    |     16    | 0m 33.884s | 0m 1.964s | 0m 1.176s |
    |     32    | 1m  7.744s | 0m 3.986s | 0m 2.280s |
    +-----------+------------+-----------+-----------+

    +------------------------------------------------+
    | $ time /etc/init.d/lustre stop # (w/o patch)   |
    +-----------+------------+-----------+-----------+
    | # of OSTs |    real    |    user   |    sys    |
    +-----------+------------+-----------+-----------+
    |     1     | 0m  4.758s | 0m 0.072s | 0m 0.030s |
    |     2     | 0m  9.018s | 0m 0.118s | 0m 0.049s |
    |     4     | 0m 18.813s | 0m 0.185s | 0m 0.083s |
    |     8     | 0m 37.586s | 0m 0.337s | 0m 0.141s |
    |    16     | 1m 16.092s | 0m 0.597s | 0m 0.263s |
    |    32     | 2m 37.550s | 0m 1.181s | 0m 0.403s |
    +-----------+------------+-----------+-----------+

Results of startup and shutdown time *with* this patch:

    +------------------------------------------------+
    | $ time /etc/init.d/lustre start # (w/ patch)   |
    +-----------+------------+-----------+-----------+
    | # of OSTs |    real    |    user   |    sys    |
    +-----------+------------+-----------+-----------+
    |      1    | 0m  2.183s | 0m 0.158s | 0m 0.083s |
    |      2    | 0m  4.282s | 0m 0.274s | 0m 0.153s |
    |      4    | 0m  8.519s | 0m 0.510s | 0m 0.303s |
    |      8    | 0m 16.966s | 0m 1.019s | 0m 0.583s |
    |     16    | 0m 33.878s | 0m 1.984s | 0m 1.154s |
    |     32    | 1m  7.745s | 0m 3.944s | 0m 2.322s |
    +-----------+------------+-----------+-----------+

    +------------------------------------------------+
    | $ time /etc/init.d/lustre stop # (w/ patch)    |
    +-----------+------------+-----------+-----------+
    | # of OSTs |    real    |    user   |    sys    |
    +-----------+------------+-----------+-----------+
    |      1    | 0m  4.566s | 0m 0.075s | 0m 0.023s |
    |      2    | 0m  4.857s | 0m 0.105s | 0m 0.070s |
    |      4    | 0m  4.777s | 0m 0.175s | 0m 0.064s |
    |      8    | 0m  5.449s | 0m 0.323s | 0m 0.153s |
    |     16    | 0m  5.862s | 0m 0.606s | 0m 0.208s |
    |     32    | 0m  6.307s | 0m 1.183s | 0m 0.811s |
    +-----------+------------+-----------+-----------+

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I90c1f6a265a8d86bbc8ddfb88aa635e5b96fd975
Reviewed-on: http://review.whamcloud.com/5235
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2760 ldiskfs: update rhel6.3 series to handle mkdir errors

ext4_mkdir can fail in several paths to dirtying an inode but
the errors aren't caught. This patch adds the upstream commit
that handles the errors and adjusts the dependent patches
accordingly.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: If86538e88c7386a06016ffae6893bacc8ba131e4
Reviewed-on: http://review.whamcloud.com/5279
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-913  test:  Framework needs to record the test filesystem.

Add a section

file_system: XXX

to the node info in yaml.sh

Because this runs on the node being recorded it can use
node_fstypes $HOSTNAME
to fetch the filesystemtype of the local machine

Signed-off-by: Chris Gearing <chris.gearing@intel.com>
Change-Id: I721e4084096c75b69290959190526ca27b573e1b
Reviewed-on: http://review.whamcloud.com/4591
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 cleanup: remove unused mkdirdeep.c and lltrace.h

That's all.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I547623a98541d545057776a01424fa6a362f06ee
Reviewed-on: http://review.whamcloud.com/5177
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2342 tests: account for log size in replay-single/20b

Account for larger ondisk log size in ZFS (256) vs. ldiskfs (50).

Test-Parameters: mdsfilesystemtype=zfs ostfilesystemtype=zfs   mdtfilesystemtype=zfs testlist=replay-single
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I4150d58556240b128c02ba667c4c390c79b8a463
Reviewed-on: http://review.whamcloud.com/5666
Tested-by: Hudson
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2903 tests: calculation of available space

Better debug logging for statfs path.
Account for log size in space calculation.
Sleep slightly longer to wait for good statfs because zfs doesn't
count uncommitted changes in available space.

Test-Parameters: mdsfilesystemtype=zfs ostfilesystemtype=zfs   mdtfilesystemtype=zfs testlist=replay-ost-single
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ib4b5a7c645d6b2a630dcc729483422c8b3a095db
Reviewed-on: http://review.whamcloud.com/5662
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2911 llite: add obd_fid_init/fini() back to llite

If without obd_fid_init() in llite, when filesystem is upgraded from
branch 1.8 to 2.4, obd_fid_init() in lmv won't be triggered because
of no lmv in branch 1.8 based config log. This will cause LBUG during
running mkdir after upgrade, like

seq_client_alloc_fid()) ASSERTION( seq != ((void *)0) ) failed.
seq_client_alloc_fid()) LBUG
Call Trace:
[<ffffffffa0371895>] libcfs_debug_dumpstack+0x55/0x80 [libcfs]
[<ffffffffa0371e97>] lbug_with_loc+0x47/0xb0 [libcfs]
[<ffffffffa080bea9>] seq_client_alloc_fid+0x379/0x440 [fid]
[<ffffffffa03822e1>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
[<ffffffffa082470b>] mdc_fid_alloc+0xbb/0xf0 [mdc]
[<ffffffffa0832b1c>] mdc_create+0xcc/0x780 [mdc]
[<ffffffffa09c487b>] ll_new_node+0x19b/0x6a0 [lustre]
[<ffffffffa09c50a7>] ll_mkdir+0x97/0x1f0 [lustre]

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I0eab1298b8d02ca08ecd4ac8bb422a2de12b7f83
Reviewed-on: http://review.whamcloud.com/5733
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2948 osc: keep trying to shrink more LRU slots

It used to try once. It can enter into a livelock state as we can
see it in this ticket.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: If603686d1d9fa7cf6513143fcc6ef962cfea9863
Reviewed-on: http://review.whamcloud.com/5760
Tested-by: Hudson
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2849 hsm: don't run sanity-hsm for ver < 2.3.61

Add version check in sanity-hsm.
Now that sanity-hsm has been added, we should consider running it by
default.

Test-Parameters: testlist=sanity-hsm

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I8cec304bed46ac24354a27716ad12f5233c75a3f
Reviewed-on: http://review.whamcloud.com/5502
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-2845 osp: fix osp precreate thread init error handling

If osp device hasn't connected OST, osp_precreate_thread() should
heed to that and bypass the normal quitting path.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I27d18dfd4d7d55c97eeb169b5d7dc7042a42fd33
Reviewed-on: http://review.whamcloud.com/5508
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2884 libcfs: SMP optimizations cleanups

Miscelaneous cleanups for the SMP optimizations code:
- Fixed typos.
- Fixed resource leak in lnet_create_locks().
- Fixed incorrect symbols in EXPORT_SYMBOL().

Signed-off-by: Isaac Huang <he.huang@intel.com>
Change-Id: I3b617367e5ed6b11ae327e477fc2c201c453e347
Reviewed-on: http://review.whamcloud.com/5547
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1876 hsm: revise ll_setattr_raw to not check stripe data

It used to check if the file has stripe data, or it won't send file
size to the MDT. After layout lock is introduced, it's unreliable to
check stripe data at llite layer.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ib947c568232b4025210d1e2e4e05bcf3514fd36a
Reviewed-on: http://review.whamcloud.com/5291
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2868 lfsck: finish otable iteration if no more objects

If there are no objects to be scanned just at the LFSCK beginning,
then the LFSCK should mark the otable-based iteration as finished
to avoid to access non-initialized data via dt_it_ops::rec().

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I67e880ccfa13e867c4ca7f1858d67909ba0415b3
Reviewed-on: http://review.whamcloud.com/5622
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2912 mdd: move fid2path from MDD to MDT

1. Move some linkEA API to obdclass, so both MDD(set linkea)
and MDT(fid2path) can access these LINKEA operation.

2. Move fid2path from MDD to MDT, so it can detect remote
object, and return -EREMOTE to the client. Then the client
will try fid2path on another MDT, and finally the client
will assemble the path fragments from different MDT.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I464cd1fdd44ebbe02c94821f910294829f3d9b94
Reviewed-on: http://review.whamcloud.com/5676
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2748 fsfilt: ext4_map_inode_page in osd and ldisk out of sync

The functon ext4_map_inode_page is mismatched in its use between
osd-ldiskfs and ldiskfs. The integer array is no longer used so
we remove its handling from the ldiskfs layer.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I9b5522ce187f06983f328408cbcd0ce077e72ea1
Reviewed-on: http://review.whamcloud.com/5708
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-2902 test: Debug for roc_hit v1

This will output all the stats and help sort out what
is happening on the systems.  I would like to know
what the proc values are the roc_hit is blank.

More debug may be needed but I want to start here.

This patch should be dropped when LU-2902 is closed.

I have used this patch with auster without issue it
just added noise to the test results.

Signed-off-by: Keith Mannthey <keith.mannthey@intel.com>
Change-Id: I3efe9aa87e8c51909667f9fbc3b9c2d6779d8d0d
Reviewed-on: http://review.whamcloud.com/5648
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1565 ldlm: make blocking threads async wherenever possible

There is no need to wait for the cancel lru lock completion in the
client side pool recalculation, make it asynchronous.

make all the ldlm_cli_cancel() calls from blocking callbacks async

Change-Id: Ie510c7361f1025a78c693a11b457baf1652f8c90
Xyratex-bug-id: MRP-690
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Bruce Korb <bruce_korb@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/4181
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2802 lmv: get tgt by checking ltd_idx in lmv_find_target.

Currently, lmv_find_target return lmv->tgts[mds]
according to mdt_index, which is not correct. LMV
index is created by mount sequence, while mdt_index
by indicated by --index. So we should check ltd_idx
in lmv_find_target.

Signed-off-by: Wang Di <di.wang@intel.com>
Change-Id: I67a941cca00eb80ba91af6eb3f3441982d4fcab3
Reviewed-on: http://review.whamcloud.com/5412
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-1187 out: add resend check for update.

1. Add update resend check between MDTs.

2. Even during creating the new object, osp_init_object
   still needs to get remote object attrs, because the
   object might be created already by some partial failed
   operation.

3. During resend handling, OSP needs to delete the existing
   orphan objects first, then do remote create.

4. MDT will check whether the name has been deleted during
   the resend of remote unlink, and only delete the local
   directory if the name on the remote MDT has been delete.

5. Fix the fail_id assignment location to only fail real
   update RPC.

6. Some fixes for replay DNE test sctipts.

Change-Id: Ia9ba8091b6622b0e2fd1f1b4fd355b5ff3eb9758
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/4343
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2571 lfsck: run lfsck on server node

Usually we run e2fsck on OST and MDS nodes, and run lfsck on clients,
but it's not necessary to install e2fsprogs rpm on clients.
So, if lfsck is not found on client, we will try server node instead.

Test-Parameters: testlist=lfsck
Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I3979dc0236e81163f3283eac3148c36c8ddccf63
Reviewed-on: http://review.whamcloud.com/5139
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2194 test: avoid wrong eviction in recovery_small

in subtest 19a and 19b of recovery_small, the locks from OST/MDT
should be cancelled before "drop_ldlm_cancel" to avoid wrong eviction
from OST/MDT for it will also drop its lock cancellation request.

Change-Id: I49d1d189f6791224a3f132dc8cf2dd8b3d51d43e
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/5679
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2640 osd: conditional enable transaction debug

Current transaction debug mechanism will be disabled
when Lustre-2.4 released.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I516a9026b2d887935c5714dc0b777d65d487dac7
Reviewed-on: http://review.whamcloud.com/5698
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>

LU-2882 build: build broken with no zfs libraries installed

Reverting default values (originaly set in LU-2391 patch) for
ldiskfs/zfs OSD RPMs builds, to comply for more ways
("rpmbuild -tb <lustre_source_tarball>") used to build Lustre.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I07eca22d03f597942284bbbc0bfd1b680ecb199b
Reviewed-on: http://review.whamcloud.com/5661
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2803 osd: osd-zfs to handle echo sequence (2) properly

use visible OI (/O directory) to map object from this sequence
into dnodes. this let obdecho to work with zfs.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I63e50789df502d11863c69658c9524fbb3cd9f22
Reviewed-on: http://review.whamcloud.com/5414
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2910 clio: skip iov update when tot_nrsegs is zero

When tot_nrsegs is zero, we should skip the iov update too.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I52d4d52a65802f3967dd6a96ad46ec40fd4ef355
Reviewed-on: http://review.whamcloud.com/5747
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1032 build: Support --disable-maintainer mode

We add "AM_MAINTAINER_MODE([enable])" to all configure
scripts to allow us to use --disable-maintainer-mode.

By default, without the AM_MAINTAINER_MODE macro, autotools
"maintainer mode" is enabled.  By specifying "enable" we
maintain our previous default behaviour.

Change-Id: I88366ad658795145af80ed96c6e708c385799ffa
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5423
Tested-by: Hudson
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2468 libcfs: quiet spurious debug message

When cfs_trace_get_tage_try()->cfs_tage_alloc() is allocating a debug
buffer, since e2a2fab993d01597010cb2b44df44a522af0eec8 (b=21776) this
allocation is denied when the allocation is happening in a memory
freeing path.  This caused a spurious "cannot allocate a tage" message
on the console each time.  Quiet that message, since it is expected.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ic800c474dc33f62843b74e06d9ca642cad3ebbe5
Reviewed-on: http://review.whamcloud.com/5470
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2424 ptlrpc: reduce initial buffers count

Separate buffers count for server and client services.
Reduce inititial allocation for client ("ldlm_cbd") because
its not required a lot. This reduced unreclaimable memory
usage after just mount from 233036kB to 97386kB.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I56fcbe6c45c61ba4876bce5482169ea06a03638c
Reviewed-on: http://review.whamcloud.com/5719
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2836 quota: improve test_3 & test_6 of s-q

When approaching quota limit, client turns to sync write, then the
tests which assume io can be finished quickly could fail, we need
to enlarge the time margin to make sure those tests can pass on slow
system.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib14a905344eb78cdbd0cd79e3bfd8e50ab21a4d8
Reviewed-on: http://review.whamcloud.com/5539
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2922 changelog: recommend e2fsprogs-1.42.6.wc2

This patch updates lustre/ChangeLog to recommend
a newer e2fsprogs-1.42.6.wc2 release instead of the
old 1.41.90.wc4 version.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib56092500545f0373a307d7645d41d91079ae086
Reviewed-on: http://review.whamcloud.com/5718
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-2883 hsm: Mark file DIRTY as soon as pages are written

Since the dirty flag has to be packed in close, it should be set when
pages are written and not when building BRWs like SOM which relies on
MDS_DONE_WRITING.

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I9cf0a71cf3228a7aadb8205cff2735a7abff5ef0
Reviewed-on: http://review.whamcloud.com/5543
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>

LU-2622 obdclass: Remove the global cl_env list

New cl_env structures are allocated using a SLAB cache specifically
created for allocating and freeing these structure. Without this patch,
when a thread is finished with a cl_env structure it places it on a
global list instead of freeing it back to the SLAB cache. With this
patch, this global list is completely removed, and cl_env structures are
released immediately back to the SLAB cache.

The motivation for this change essentially boils down to this secondary
global list cache being completely unnecessary, and only serving to
serialize any calls to cl_env_get and cl_env_put. This has proven to
cause a severe performance impact on large core count systems,
specifically during memory reclamation (i.e. ll_releasepage).

For example, on BG/Q Sequoia IO nodes, we've experienced nearly all
68 cores of a machine spinning on the lock protecting this global
list. Some example stack traces showcasing this problem were gathered
using sysrq-l, and are displayed below:

    CPU56:
    Call Trace:
    [c00000000fe3bb30] [c000000000008d1c] .show_stack+0x7c/0x184 (unreliable)
    [c00000000fe3bbe0] [c00000000027604c] .showacpu+0x64/0x94
    [c00000000fe3bc70] [c000000000068b30] .generic_smp_call_function_interrupt+0x10c/0x230
    [c00000000fe3bd40] [c00000000001d11c] .smp_message_recv+0x34/0x78
    [c00000000fe3bdc0] [c00000000002526c] .bgq_ipi_dispatch+0x118/0x18c
    [c00000000fe3be50] [c00000000007b20c] .handle_IRQ_event+0x88/0x18c
    [c00000000fe3bf00] [c00000000007dc90] .handle_percpu_irq+0x8c/0x100
    [c00000000fe3bf90] [c00000000001b808] .call_handle_irq+0x1c/0x2c
    [c0000003e1c4a4c0] [c0000000000059f0] .do_IRQ+0x154/0x1e0
    [c0000003e1c4a570] [c0000000000144dc] exc_external_input_book3e+0x110/0x114
    --- Exception: 501 at ._raw_spin_lock+0xd8/0x1a8
        LR = ._raw_spin_lock+0x104/0x1a8
    [c0000003e1c4a860] [8000000000b04f38] libcfs_nidstrings+0x2acc/0xfffffffffffe5824 [libcfs] (unreliable)
    [c0000003e1c4a910] [c00000000042d4cc] ._spin_lock+0x10/0x24
    [c0000003e1c4a980] [80000000024c2f4c] .cl_env_get+0xec/0x480 [obdclass]
    [c0000003e1c4aa60] [80000000024c336c] .cl_env_nested_get+0x8c/0xf0 [obdclass]
    [c0000003e1c4aaf0] [800000000692070c] .ll_releasepage+0xbc/0x200 [lustre]
    [c0000003e1c4aba0] [c000000000094110] .try_to_release_page+0x68/0x8c
    [c0000003e1c4ac10] [c0000000000a4190] .shrink_page_list.clone.0+0x3d8/0x63c
    [c0000003e1c4adc0] [c0000000000a47d8] .shrink_inactive_list+0x3e4/0x690
    [c0000003e1c4af90] [c0000000000a4f54] .shrink_zone+0x4d0/0x4d4
    [c0000003e1c4b0c0] [c0000000000a5a68] .try_to_free_pages+0x204/0x3d0
    [c0000003e1c4b220] [c00000000009d044] .__alloc_pages_nodemask+0x460/0x738
    [c0000003e1c4b3a0] [c000000000095af4] .grab_cache_page_write_begin+0x7c/0xec
    [c0000003e1c4b450] [8000000006920964] .ll_write_begin+0x94/0x270 [lustre]
    [c0000003e1c4b520] [c0000000000968c8] .generic_file_buffered_write+0x148/0x374
    [c0000003e1c4b660] [c000000000097050] .__generic_file_aio_write+0x374/0x3d8
    [c0000003e1c4b760] [c00000000009712c] .generic_file_aio_write+0x78/0xe8
    [c0000003e1c4b810] [800000000693ed4c] .vvp_io_write_start+0xfc/0x3e0 [lustre]
    [c0000003e1c4b8e0] [80000000024d9c6c] .cl_io_start+0xcc/0x220 [obdclass]
    [c0000003e1c4b980] [80000000024e1a84] .cl_io_loop+0x194/0x2c0 [obdclass]
    [c0000003e1c4ba30] [80000000068ba1d8] .ll_file_io_generic+0x498/0x670 [lustre]
    [c0000003e1c4bb30] [80000000068ba834] .ll_file_aio_write+0x1d4/0x3a0 [lustre]
    [c0000003e1c4bc00] [80000000068bab50] .ll_file_write+0x150/0x320 [lustre]
    [c0000003e1c4bce0] [c0000000000d1ba8] .vfs_write+0xd0/0x1c4
    [c0000003e1c4bd80] [c0000000000d1d98] .SyS_write+0x54/0x98
    [c0000003e1c4be30] [c000000000000580] syscall_exit+0x0/0x2c

    CPU63:
    Call Trace:
    [c00000000fe03b30] [c000000000008d1c] .show_stack+0x7c/0x184 (unreliable)
    [c00000000fe03be0] [c00000000027604c] .showacpu+0x64/0x94
    [c00000000fe03c70] [c000000000068b30] .generic_smp_call_function_interrupt+0x10c/0x230
    [c00000000fe03d40] [c00000000001d11c] .smp_message_recv+0x34/0x78
    [c00000000fe03dc0] [c00000000002526c] .bgq_ipi_dispatch+0x118/0x18c
    [c00000000fe03e50] [c00000000007b20c] .handle_IRQ_event+0x88/0x18c
    [c00000000fe03f00] [c00000000007dc90] .handle_percpu_irq+0x8c/0x100
    [c00000000fe03f90] [c00000000001b808] .call_handle_irq+0x1c/0x2c
    [c0000003c4f0a510] [c0000000000059f0] .do_IRQ+0x154/0x1e0
    [c0000003c4f0a5c0] [c0000000000144dc] exc_external_input_book3e+0x110/0x114
    --- Exception: 501 at ._raw_spin_lock+0xdc/0x1a8
        LR = ._raw_spin_lock+0x104/0x1a8
    [c0000003c4f0a8b0] [800000000697a578] msgdata.87439+0x20/0xfffffffffffccf88 [lustre] (unreliable)
    [c0000003c4f0a960] [c00000000042d4cc] ._spin_lock+0x10/0x24
    [c0000003c4f0a9d0] [80000000024c17e8] .cl_env_put+0x178/0x420 [obdclass]
    [c0000003c4f0aa70] [80000000024c1ab0] .cl_env_nested_put+0x20/0x40 [obdclass]
    [c0000003c4f0aaf0] [8000000006920794] .ll_releasepage+0x144/0x200 [lustre]
    [c0000003c4f0aba0] [c000000000094110] .try_to_release_page+0x68/0x8c
    [c0000003c4f0ac10] [c0000000000a4190] .shrink_page_list.clone.0+0x3d8/0x63c
    [c0000003c4f0adc0] [c0000000000a47d8] .shrink_inactive_list+0x3e4/0x690
    [c0000003c4f0af90] [c0000000000a4f54] .shrink_zone+0x4d0/0x4d4
    [c0000003c4f0b0c0] [c0000000000a5a68] .try_to_free_pages+0x204/0x3d0
    [c0000003c4f0b220] [c00000000009d044] .__alloc_pages_nodemask+0x460/0x738
    [c0000003c4f0b3a0] [c000000000095af4] .grab_cache_page_write_begin+0x7c/0xec
    [c0000003c4f0b450] [8000000006920964] .ll_write_begin+0x94/0x270 [lustre]
    [c0000003c4f0b520] [c0000000000968c8] .generic_file_buffered_write+0x148/0x374
    [c0000003c4f0b660] [c000000000097050] .__generic_file_aio_write+0x374/0x3d8
    [c0000003c4f0b760] [c00000000009712c] .generic_file_aio_write+0x78/0xe8
    [c0000003c4f0b810] [800000000693ed4c] .vvp_io_write_start+0xfc/0x3e0 [lustre]
    [c0000003c4f0b8e0] [80000000024d9c6c] .cl_io_start+0xcc/0x220 [obdclass]
    [c0000003c4f0b980] [80000000024e1a84] .cl_io_loop+0x194/0x2c0 [obdclass]
    [c0000003c4f0ba30] [80000000068ba1d8] .ll_file_io_generic+0x498/0x670 [lustre]
    [c0000003c4f0bb30] [80000000068ba834] .ll_file_aio_write+0x1d4/0x3a0 [lustre]
    [c0000003c4f0bc00] [80000000068bab50] .ll_file_write+0x150/0x320 [lustre]
    [c0000003c4f0bce0] [c0000000000d1ba8] .vfs_write+0xd0/0x1c4
    [c0000003c4f0bd80] [c0000000000d1d98] .SyS_write+0x54/0x98
    [c0000003c4f0be30] [c000000000000580] syscall_exit+0x0/0x2c

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Ief4b524784e07d7677ecb8a9ce97a7b54ccc6f75
Reviewed-on: http://review.whamcloud.com/5204
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>