Whamcloud - gitweb
fs/lustre-release.git
7 years agoLU-1666 obdclass: reduce lock contention on coh_page_guard
Jinshan Xiong [Mon, 13 Aug 2012 23:57:49 +0000 (16:57 -0700)]
LU-1666 obdclass: reduce lock contention on coh_page_guard

Define a per-page spinlock to get and put a cl_page instead of
grabbing per-object lock coh_page_guard.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iecf5e6840ab1a28edcf2c4bcde6a72c2f9b5bdae
Reviewed-on: http://review.whamcloud.com/3627
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2119 osd: missing osd_shutdown()
Niu Yawei [Wed, 10 Oct 2012 02:40:03 +0000 (22:40 -0400)]
LU-2119 osd: missing osd_shutdown()

For zfs osd, osd_shutdown() should be called in osd_device_fini()
just like ldiskfs osd does, to can make sure that everything is
cleared even if the osd_process_config(CLEANUP) has no chance to
be called. (when OFD/MDT wasn't started)

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I54f08ac657e01ffb7a367278810016b585b3c0da
Reviewed-on: http://review.whamcloud.com/4239
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-107 scripts: rename heartbeat resource agent
Ned Bass [Tue, 9 Oct 2012 20:10:18 +0000 (13:10 -0700)]
LU-107 scripts: rename heartbeat resource agent

Add a .ha_v2 extension to the Lustre heartbeat resource agent script
to resolve conflicts with the lustre init script on case-insensitive
filesystems.  The extension also provides a clue as to what the
script's purpose is.  Add a compatibility symlink with the plain name
during package installation.   If we later add v3-style scripts we can
control which one gets linked to via a configure option.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I0e37a390fbfb3f00c1c1e666a7cdbe5d37fa885b
Reviewed-on: http://review.whamcloud.com/4233
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: remove leftovers from old code
Johann Lombardi [Mon, 8 Oct 2012 12:34:47 +0000 (14:34 +0200)]
LU-1842 quota: remove leftovers from old code

- remove all references to qunit_data which is the old format for
  slave-master RPCs
- gather all quota-related data structure definitions in one single
  place in lustre_idl.h
- rename lquota.h to lustre_quota.h since this is the standard naming
  scheme for all lustre components.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Ifbadb639517c52741c210668f28452ea55bf6a43
Reviewed-on: http://review.whamcloud.com/4221
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2131 tests: sanity using ost0 instead of ost1
James Simmons [Tue, 9 Oct 2012 17:20:25 +0000 (13:20 -0400)]
LU-2131 tests: sanity using ost0 instead of ost1

In sanity test 81a,81b and the jobstat function are testing
against ost0 which technically don't exist. The test
framework starts the OSTs with ost1. This patch has the test
use the proper ost1.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ibb7e647b8d6a736ca930e9294aacf584c9c49880
Reviewed-on: http://review.whamcloud.com/4232
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 oi: handle failure cases for osd_oi_fini
Fan Yong [Mon, 8 Oct 2012 05:44:19 +0000 (13:44 +0800)]
LU-1842 oi: handle failure cases for osd_oi_fini

Sometimes the osd_oi_fini() may be called without osd_oi_init()
called firstly, or with osd_oi_init() failure. Under such cases,
the osd_device::od_oi_table may be invalid, so just skip related
OI cleanup.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: Iffa008b29bb8763fcdc4d12c1e4cae93026a25b3
Reviewed-on: http://review.whamcloud.com/4219
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2099 osd: set dr_elapsed before dr_numreqs
Alex Zhuravlev [Mon, 8 Oct 2012 06:36:39 +0000 (10:36 +0400)]
LU-2099 osd: set dr_elapsed before dr_numreqs

so that service thread always get dr_elapsed and
dr_elapsed_valid initialized when the bio is done.
also, the patch adds a bit of debug to help with
the original bug if this patch doesn't help.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I7c41af9a2be5be2f37190fb053d7449653fc7a99
Reviewed-on: http://review.whamcloud.com/4216
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2102 llog: wrong handle was used for changelog processing
Mikhail Pershin [Tue, 9 Oct 2012 06:40:43 +0000 (10:40 +0400)]
LU-2102 llog: wrong handle was used for changelog processing

- Wrong llog handle was passed tp changelog_user_init_cb() for
  processing

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I3656829e283232ff8c17829e18df68b320e26f2f
Reviewed-on: http://review.whamcloud.com/4229
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1943 tests: Refresh conf-sanity 32[ab]
Alex Zhuravlev [Sat, 6 Oct 2012 17:01:46 +0000 (21:01 +0400)]
LU-1943 tests: Refresh conf-sanity 32[ab]

Existing conf-sanity 32[ab] does not run on multi-node clusters or
network types other than TCP.  This patch rewrites the tests to
start the targets on the MDS and mount the upgraded file system on
the primary client.  This scheme works both on a single-node
development environment and a typical Autotest cluster.

The disk image tarball format has been updated to include more
"metadata".  For example, the 1.8.7-wc1 and 2.1.1 tarballs added
by this patch include the kernel versions, architectures, Lustre
versions, and commit SHAs (currently only Lustre version) used to
create the disk images.  In addition, the script used to create
the tarballs is recorded as a test in test_32newtarball().

A couple of new data verifications are added to make sure the name
space, the file data, and most of the file attributes are consistent
across upgrades.

Signed-off-by: Li Wei <liwei@whamcloud.com>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I3b0d38d67e86e0e2ac24d9aae4406b055b63ce61
Reviewed-on: http://review.whamcloud.com/4213
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1606 api: Rename liblustreapi.h -> lustreapi.h
Christopher J. Morrone [Wed, 18 Jul 2012 00:22:18 +0000 (17:22 -0700)]
LU-1606 api: Rename liblustreapi.h -> lustreapi.h

The header file "liblustreapi.h" is unfortunately named,
implying that it is only part of the lustre-in-user-space-client
rather than the more general lustre api for users that it is.

This patch renames it to simply "lustreapi.h" to avoid that
confusion.  This also helps to move towards making it
easier to use the lustre API for users.  They can simply

  #include <lustre/lustreapi.h>

and then compile with "-llustreapi".

For backwards compatibility, liblustreapi.h becomes a stub
that includes lustreapi.h, and can be removed at some date
in the future.

Since LU-113, LASSERT should be removed from userspace.
In this change, lustre_user.h has had LASSERTs removed.

All code examples in the man pages have been tested.

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Signed-off-by: Richard Henwood <richard.henwood@intel.com>
Change-Id: Id1e709d36a855ad2a8eff206fa9f6bbe87182a29
Reviewed-on: http://review.whamcloud.com/3427
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
7 years agoLU-2110 mount: do not start osp twice
Alex Zhuravlev [Mon, 8 Oct 2012 19:31:20 +0000 (23:31 +0400)]
LU-2110 mount: do not start osp twice

failover nid can be added under the same marker as the
main device setup block. so do not call lustre_osp_setup()
on a subsequent LCFG_ADD_UUID command in the config.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ic4858812e942f73dd264dca5e33b27d51509f670
Reviewed-on: http://review.whamcloud.com/4227
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
7 years agoLU-1842 test: remove code from s-q to use OFD
Johann Lombardi [Mon, 8 Oct 2012 13:19:59 +0000 (15:19 +0200)]
LU-1842 test: remove code from s-q to use OFD

Now that obdfilter is gone and OFD is the default, we don't need to
restart lustre with USE_OFD any more.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Ifa3647bd815866aa29bd88a9b3c6be16c6c66fa5
Reviewed-on: http://review.whamcloud.com/4222
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2097 quota: missing ll_vfs_dq_init()
Niu Yawei [Mon, 8 Oct 2012 09:55:36 +0000 (05:55 -0400)]
LU-2097 quota: missing ll_vfs_dq_init()

ll_vfs_dq_init() should be called before operating on the system
objects (like llog, oi, last_rcvd, etc.), otherwise, block accounting
will be missed if object is already existing.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I922c9ca8caebf52ae07fd130c41286dad68c1b8f
Reviewed-on: http://review.whamcloud.com/4220
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1691 kernel: kernel update [SLES11 SP2 3.0.34-0.7.9]
James Simmons [Mon, 8 Oct 2012 13:34:22 +0000 (09:34 -0400)]
LU-1691 kernel: kernel update [SLES11 SP2 3.0.34-0.7.9]

Add SLES11 SP2 client support to 3.0.34-0.7.9.

Allow 2.6 (SP1) and 3.0 (SP2) clients to be built for SLES11
Standard lbuild will build for the kernel version that builder
is using, this can be overridden by specifying the target
directly as an lbuild parameter

This change explictly does not make changes to support SLES11SP2
server

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Chris Gearing <chris.gearing@intel.com>
Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: Iece4c0286a3f0dcd28fe96e03a8aec9bda065ed5
Reviewed-on: http://review.whamcloud.com/3734
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1199 build: Change various %defines to %globals
Christopher J. Morrone [Thu, 8 Mar 2012 02:43:38 +0000 (18:43 -0800)]
LU-1199 build: Change various %defines to %globals

As a general rule, "%define" should not be used inside
of %{ } blocks.  %define is locally scoped.  While it
appears to work with constructs like this:

   %{!?foo: %define foo bar}

that is only because of an rpm quirk that fails to
free non-global scope variables immediately.  Later use
of parameterized macros in the file can trigger cleanup
of the local variables, and "foo" will be once again
undefined.

The solution is to use %global like so:

   %{!?foo: %global foo bar}

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: Ie18b0b86324334330b726bf69249d97e47e9350e
Reviewed-on: http://review.whamcloud.com/3420
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1898 test: new tests 8[a,b,c,d,e] need 2.3 or later
Bob Glossman [Thu, 13 Sep 2012 21:34:02 +0000 (14:34 -0700)]
LU-1898 test: new tests 8[a,b,c,d,e] need 2.3 or later

check lustre version to make sure tests do not run
on unsupported version

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5b284af44a6132070b8e89659b7981844a74555e
Reviewed-on: http://review.whamcloud.com/3986
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoNew tag 2.3.53 2.3.53 v2_3_53 v2_3_53_0
Oleg Drokin [Mon, 8 Oct 2012 07:35:23 +0000 (03:35 -0400)]
New tag 2.3.53

Change-Id: Id9d4ed992ee7995f121f37dce05c8ae40c807779
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-753 obd: remove obsolete commit callback code
Andreas Dilger [Tue, 25 Oct 2011 21:56:45 +0000 (15:56 -0600)]
LU-753 obd: remove obsolete commit callback code

Remove old obd_transno_commit_cb() function, which was replaced
by lut_cb_last_committed() in the new LU stack implementation.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I789687c63761c2532c02f5a1b827d8625b770c1c
Reviewed-on: http://review.whamcloud.com/4211
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1943 filter: remove obdfilter from tree
Mikhail Pershin [Sat, 6 Oct 2012 16:02:17 +0000 (20:02 +0400)]
LU-1943 filter: remove obdfilter from tree

Remove obdfilter and keep only OFD

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: If72fe59238325cc63f804f84804b180156c94ea9
Reviewed-on: http://review.whamcloud.com/4209
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1175 tests: use ofd by default instead of obdfilter
Alex Zhuravlev [Wed, 3 Oct 2012 19:30:58 +0000 (23:30 +0400)]
LU-1175 tests: use ofd by default instead of obdfilter

Set USE_OFD=yes by default in order to start testing the ofd module in
preference to the obdfilter module.  Force LOAD_MODULES_REMOTE to be
set to avoid obdfilter being loaded by modprobe remotely on mount.

There was a bug in comma_list() appending a space to the list of
nodes, which caused do_rpc_nodes() to be unhappy when a nearly-empty
list of nodes was being passed by callers.  Rework this function, and
clean up other callers to just pass the nodes and let do_rpc_nodes()
detect this case and skip any remote execution.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I896c9c9b7216c418b635e893202810333d8ef871
Reviewed-on: http://review.whamcloud.com/4171
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2089 ofd: do not pin pages provided by osd
Alex Zhuravlev [Fri, 5 Oct 2012 10:25:44 +0000 (14:25 +0400)]
LU-2089 ofd: do not pin pages provided by osd

depending on implementation, some pages can be allocated
in order > 0 and kernel does increase refcounter on the
first page only. in this case ptlrpc_free_bulk() calling
cfs_unpin() will try to release such pages leading to
warning and other bad things in the kernel.

instead let ofd/ost to rely on dbo_bufs_get/dbo_bufs_put
as they know details of the pages provided.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I3591e21eef9557d6004d29e63986c7bd5987802b
Reviewed-on: http://review.whamcloud.com/4198
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: tunefs.lustre --quota
Niu Yawei [Sat, 6 Oct 2012 09:31:30 +0000 (05:31 -0400)]
LU-1842 quota: tunefs.lustre --quota

Add "--quota" for tunefs.lustre to convert old quota disk format
into new format.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ie1951c9183b940c41956e863f76f5e357a1b2bd8
Reviewed-on: http://review.whamcloud.com/4205
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 tests: async IO is enough for sanity/80
Alex Zhuravlev [Tue, 10 Apr 2012 13:06:22 +0000 (17:06 +0400)]
LU-1943 tests: async IO is enough for sanity/80

Port of ORI-620 to the master

it can be hard to write 1MB in a second for some
backends (like ZFS), so relax current synchronous
semantics as this doesn't affect the logic.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Ie87815de4a2ae51fc166287d74d43278101fe76b
Reviewed-on: http://review.whamcloud.com/4212
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1718 client: Restore NFS export for Lustre on 3.X kernels
James Simmons [Mon, 27 Aug 2012 18:55:07 +0000 (14:55 -0400)]
LU-1718 client: Restore NFS export for Lustre on 3.X kernels

In Linux 3.0+ kernels struct file_system_type changed the
get_sb function to a new function called mount which was
different in that the vfsmount data was no longer passed in.
The vfsmount data was used by the llite layer for nfs export
function called get_name to search for filp that was then used
with the ll_readdir method. The approach to solve this change
was to go the route of btrfs and gfs2 to refactor some of the
llite methods to implement a directory scan independent of
filp which could be shared with nfs export funtionality.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I72730476b120cec1ede6e03c774c9e470a1a5a70
Reviewed-on: http://review.whamcloud.com/3624
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1962 util: Add --lazy flag to lfs df
Andriy Skulysh [Wed, 19 Sep 2012 13:37:22 +0000 (16:37 +0300)]
LU-1962 util: Add --lazy flag to lfs df

add --lazy flag to lfs df
it allows to skip unavailable OSTs and report
"Resource temporarily unavailable" for them

Xyratex-bug-id: MRP-583
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: Idb6b728b52c5fa1590d07201700a3fad0ef7cc78
Reviewed-on: http://review.whamcloud.com/4007
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1963 ptlrpc: Incorrect accounting for srv_hpreq_ratio
Andriy Skulysh [Mon, 27 Aug 2012 09:23:53 +0000 (12:23 +0300)]
LU-1963 ptlrpc: Incorrect accounting for srv_hpreq_ratio

Allow one thread to always handle HP requests.
Without this patch 1 thread dedicated to serve HP requests
will do nothing if all normal requests are in progress for
a long time but there are HP requests in input queue.

Xyratex-bug-id: MRP-661
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Nathan Rutman <nathan.rutman@xyratex.com>
Change-Id: I35f5329ca019a5d2e2dcee9d7a13eaa74e85233e
Reviewed-on: http://review.whamcloud.com/4008
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1833 util: don't update mtab if linked to /proc
Peng Tao [Wed, 5 Sep 2012 10:36:05 +0000 (18:36 +0800)]
LU-1833 util: don't update mtab if linked to /proc

Some distros may link /etc/mtab to /proc/mounts, or
/proc/self/mounts.  In such case, we don't need to
update mtab. Otherwise we false alart user with
"
mount.lustre: addmntent: Invalid argument:
"

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I2e3e213e4ee3bc177865c4ca7435a7eecd274b46
Reviewed-on: http://review.whamcloud.com/3881
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2093 osd: maintain O/<group>/LAST_ID properly
Alex Zhuravlev [Sat, 6 Oct 2012 16:14:25 +0000 (20:14 +0400)]
LU-2093 osd: maintain O/<group>/LAST_ID properly

unfortunately this is not easy to do from ofd,
we have to do this in osd.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I389f278ac53da2a65ba8ed4a6e93cb9622eedae2
Reviewed-on: http://review.whamcloud.com/4210
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1607 tests: allow MODOPTS_LIBCFS module params
Andreas Dilger [Sat, 15 Sep 2012 02:53:18 +0000 (20:53 -0600)]
LU-1607 tests: allow MODOPTS_LIBCFS module params

The changes made for forcing non-default CPT partition counts in
commit 95d67b28076f7938a6c962a5256e9b581a439f71 now prevents
overriding libcfs module options using MODOPTS_LIBCFS, because
there is always the cpu_npartitions parameter specified at module
loading time that overrides the environment variable.

Instead add the cpu_npartitions=2 argument to the environment
variable instead of making it a parameter.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I94bf500164dc23e81080bd2675646b5fa7500c1e
Reviewed-on: http://review.whamcloud.com/3999
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 test: Support --with-ldiskfsprogs in test framework
Brian Behlendorf [Thu, 29 Sep 2011 19:27:24 +0000 (12:27 -0700)]
LU-1943 test: Support --with-ldiskfsprogs in test framework

ORI-347 port to the master

When Lustre is built with the --with-ldiskfsprogs option the
Lustre utilities will expect to find the e2fsprogs utilties
repackaged as ldiskfsprogs on the system.  This is done to
avoid having to replace the distribution provided e2fsprogs
package.

This change extends the test framework to check for the
ldiskfsprogs utilties to avoid having to explicitly set
the shell variables when --with-ldiskfsprogs is used.
If the tools are not available it will fallback to using
the e2fsprogs utilities.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Ia094d9b005f7e455beca1c5d6d6ad03bdc03671f
Reviewed-on: http://review.whamcloud.com/4194
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 tests: Allocate sequences on MOUNT2 in replay_barrier()
Li Wei [Tue, 10 Jan 2012 08:03:21 +0000 (16:03 +0800)]
LU-1943 tests: Allocate sequences on MOUNT2 in replay_barrier()

Port of ORI-448 to the master

In order to avoid sequence file updates after a target is made
read-only, replay_barrier() creates a file in "MOUNT" on every
client node. This is not enough because the "MOUNT2" clients may
also trigger sequence file updates. This patch makes sure the trick
is applied to "MOUNT2" clients as well.

Change-Id: I832689a2d2ca205d927bc5d0a15ab14fceb3bf80
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4196
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1943 tests: unlink all files when mdsrate times out
Johann Lombardi [Mon, 9 Jan 2012 17:18:26 +0000 (18:18 +0100)]
LU-1943 tests: unlink all files when mdsrate times out

Port of ORI-453 to the master

When mdsrate cannot create the requested number of files in the
allocated timeframe (i.e. the --time option), filenames are not
necessarily consecutive and can have an index higher than the
number of files that was actually created. Here is an example:

0: iters 0 nfiles 100000 time 1 begin 0 end 99998 dirthreads 2.
1: iters 0 nfiles 100000 time 1 begin 1 end 99999 dirthreads 2.
0: client-4.lab.whamcloud.com starting at Mon Jan  9 08:26:07 2012
1: client-12.lab.whamcloud.com starting at Mon Jan  9 08:26:07 2012
0: created 1884 files, last file 'f3766'.
Rate: 2470.42 eff 2470.59 aggr 1235.29 avg client creates/sec
     (total: 2 threads 2440 creates 1 dirs 2 threads/dir 0.99 secs)
0: client-4.lab.whamcloud.com finished at Mon Jan  9 08:26:08 2012
1: created 556 files, last file 'f1111'.
1: client-12.lab.whamcloud.com finished at Mon Jan  9 08:26:08 2012

2440 files have been created and the last file name was f1111 for
rank 1 and f3766 for rank 0. Running mdsrate --unlink --nfiles 2440
in this case would discard all files with an index >= 2440.

With this patch, each mdsrate task will complete properly and report
the right number of executed operation as soon as one file is missing
in the per-task sequence. This behavior can be changed with the
--ignore and --recreate option which are already available.
This way, mdsrate --unlink --nfiles 100000 will work fine in the
example above since it will unlink all the existing files, without
doing tons of useless lookups.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I96f690ad0f36c65353bbc5a401455479b4c3783b
Reviewed-on: http://review.whamcloud.com/4195
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: don't setup qsd for standalone MGS
Johann Lombardi [Fri, 5 Oct 2012 22:14:06 +0000 (00:14 +0200)]
LU-1842 quota: don't setup qsd for standalone MGS

Only set up the qsd instance if the associated target is either a MDT
or a OST. We typically have no interest in setting up a QSD instance
on a standalone MGS.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I0facb08feef0b36f141b749d7be538cbde5f1114
Reviewed-on: http://review.whamcloud.com/4203
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 test: re-enable all sanity-quota tests
Johann Lombardi [Fri, 5 Oct 2012 13:26:39 +0000 (15:26 +0200)]
LU-1842 test: re-enable all sanity-quota tests

This patch enables all sanity-quota tests that were disabled in order
to land the new quota architecture (based on qsd/qmt). All those
tests should now pass.

This patch also reduces the time it takes to run test 6 (it used to
sleep for at_max = 600s).

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Idac4812663a3c90a213cc59d03b5f28f561ac39d
Reviewed-on: http://review.whamcloud.com/4200
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1842 quota: add core QSD code
Johann Lombardi [Thu, 4 Oct 2012 14:18:26 +0000 (16:18 +0200)]
LU-1842 quota: add core QSD code

Add acquire/release logic on the qsd side as well as reintegration
which is handled in 3 steps:
- enqueue global index lock
- fetch global index and update local enforce list
- fetch slave index and update local copy
- adjust/release quota space as appropriate

This patch also fixes the ofd code to call ->ldo_recovery_complete
across the stack.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Ib4a8910fe34de878159827c76b40a8f899aaad10
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-on: http://review.whamcloud.com/4182
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2033 scrub: lookup server local files by name
Fan Yong [Wed, 26 Sep 2012 16:53:18 +0000 (00:53 +0800)]
LU-2033 scrub: lookup server local files by name

after backup/restore ldiskfs's OI becomes invalid (inodes
change). OI scrubber scans a filesystem to rebuild OI. this
can take quite long. to be able to start MDS earlier (right
after restore) MDS should lookup by name all internal files
required for startup as name->inode mapping is preserved by
backup/restore. this patch make local object library to
lookup a file storing last used fid within a given sequence.

the patch also fixes sanity-scrub/11 to expect skipping of
files created + directory + per-ost llog objects.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I56179c03279062464960af73de031f13d31bac14
Reviewed-on: http://review.whamcloud.com/4106
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: add acq/rel logic on QMT
Johann Lombardi [Thu, 4 Oct 2012 08:34:09 +0000 (10:34 +0200)]
LU-1842 quota: add acq/rel logic on QMT

This patch adds the quota acquire/release logic on the QMT.
The QMT is in charge of distributing quota space across quota
slaves. It grants a significant amount of quota space (75%) initially
and then tries to balance spare quota space among slaves.
To do so, it sends glimpse on per-ID lock to notify slaves of qunit
shrink. qunit is now the maximal amount of available quota space a
slave can own. As for qtune, this value (typically 1/4 of qunit) is
used on the quota slave to decide when to release/pre-acquire quota
space.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I8064ce614955421681ca1da7e2bb774001ca3bea
Reviewed-on: http://review.whamcloud.com/4178
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: add quota locks support on QMT
Johann Lombardi [Wed, 3 Oct 2012 13:02:42 +0000 (15:02 +0200)]
LU-1842 quota: add quota locks support on QMT

This patch adds methods for the quota lock lvb management and
implements an intent policy for quota locks.
It also introduces the rebalance thread which is in charge of
sending glimpse callbacks to slave to notify them of new quota
settings.

This patch also enables sanity-quota.sh test 36 which should now
pass.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I0cf7f86d480296f7e8fe774ed8937aae27736f88
Reviewed-on: http://review.whamcloud.com/4166
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1943 ofd: save server last transaction number
Mikhail Pershin [Thu, 4 Oct 2012 05:17:53 +0000 (09:17 +0400)]
LU-1943 ofd: save server last transaction number

After reading server data from the last_rcvd file we must store the
last transaction number in lu_target.  Otherwise we write a zero last
transaction number back to last_rcvd, causing transaction numbers to
roll back to 2^32 every time the server is restarted.  This can lead
to 'server went back in time' client console errors.

Whamcloud-bug-id: ORI-670
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Iaff4af3e4bd6fce0da1d1b0886147538c3300c1e
Reviewed-on: http://review.whamcloud.com/4190
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Ian Colle <Ian.Colle@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2071 lprocfs: Memory leak on lustre proc access
Oleg Drokin [Tue, 2 Oct 2012 02:55:27 +0000 (22:55 -0400)]
LU-2071 lprocfs: Memory leak on lustre proc access

Make __LPROC_SEQ_FOPS to call lprocfs_single_release in release so that
the memory allocated in single_open to be freed.
Rename some of the auto-named methods to better reflect on the nature
of the underlying calls.
The previous code used to leak 32 bytes per every lustre proc file open
that uses lprocfs_..._seq_open primitive.

Change-Id: I08cc1bbb28af444f3fa5e903e38fd4a16dc4c79b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4150
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
7 years agoLU-1798 tests: add set_and_check to test-framework
Andreas Dilger [Mon, 1 Oct 2012 04:53:55 +0000 (22:53 -0600)]
LU-1798 tests: add set_and_check to test-framework

Move the set_and_check() helper function from conf-sanity.sh into
test-framework.sh as set_conf_param_and_check() so that it can be
used in other test scripts.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4ad3179decff55a0ec17a4c983e30941cf49cab0
Reviewed-on: http://review.whamcloud.com/4140
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2068 build: Disable obdfilter with osd-zfs
James Simmons [Thu, 4 Oct 2012 14:41:26 +0000 (10:41 -0400)]
LU-2068 build: Disable obdfilter with osd-zfs

This change introduces a few different cleanup and build system
changes. Most notably, the obdfilter modules is now disabled and
the mkfs utilities does not call into ldiskfs helpers when ldiskfs
is disabled. Apart from that, the other changes are various minor
cleanup and dead code removal.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Idcfda525e4b76269c2cf17c1b3232033b92fc8a8
Reviewed-on: http://review.whamcloud.com/4159
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2087 ofd: refresh blocks in lvb from rpc
Alex Zhuravlev [Thu, 4 Oct 2012 10:51:24 +0000 (14:51 +0400)]
LU-2087 ofd: refresh blocks in lvb from rpc

like obdfilter does. let blocks only increase unless
specified with the flag. this should fix sanityn/4.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I85bf45f091637faf39c65e0acf331f37dab7af7e
Reviewed-on: http://review.whamcloud.com/4180
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
7 years agoLU-1943 class: FID_SEQ_LOCAL_NAME set to the Orion value
Mikhail Pershin [Thu, 4 Oct 2012 10:24:43 +0000 (14:24 +0400)]
LU-1943 class: FID_SEQ_LOCAL_NAME set to the Orion value

Keep the same numbers for Orion and master for compatibility

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I318eba9860be7849ee4a8d828cf27e5fb91164e9
Reviewed-on: http://review.whamcloud.com/4179
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2092 Free quota pools before removing quota master proc entry
Oleg Drokin [Thu, 4 Oct 2012 23:13:14 +0000 (19:13 -0400)]
LU-2092 Free quota pools before removing quota master proc entry

When quota master proc entry is removed, it reqursively removes
all children entries including quota pool ones, then when quota pools
are freed they try to unregister already not present proc entries and
crash/lockup.
So free pools before unregistering any proc entires.

Change-Id: Ice0be373bacbd54f4863aec92fb2427c7fe27464
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4191
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2088 utils: lctl to generate gdb symbols for lod/osp
Alex Zhuravlev [Thu, 4 Oct 2012 14:50:57 +0000 (18:50 +0400)]
LU-2088 utils: lctl to generate gdb symbols for lod/osp

useful for people using gdb

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5dbd3c1757b3462f6c98c5ad03fc3f83a7457e40
Reviewed-on: http://review.whamcloud.com/4183
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1574 osd: don't use obd_name but od_svname in error messages
Mikhail Pershin [Wed, 3 Oct 2012 07:46:59 +0000 (11:46 +0400)]
LU-1574 osd: don't use obd_name but od_svname in error messages

obd_name is not initialized in OSD, but there is od_svname in
osd_device to be used instead.

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I87a83badc358ad508ecb8a4e593a7e744277e57f
Reviewed-on: http://review.whamcloud.com/4165
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: add quotactl support on qmt
Johann Lombardi [Tue, 2 Oct 2012 16:11:21 +0000 (18:11 +0200)]
LU-1842 quota: add quotactl support on qmt

This patch implements quotactl ({get,set}{quota,info}) support on the
QMT. This involves defining the lqe operations for the qmt as well as
pool operations in order to support per-pool quota in the future.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I122e7cb85eb6d1d3460e86217e2a6723763c8cbf
Reviewed-on: http://review.whamcloud.com/4160
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1028 tests: regrab some grant after oos test
Jinshan Xiong [Fri, 17 Aug 2012 17:15:33 +0000 (10:15 -0700)]
LU-1028 tests: regrab some grant after oos test

After out-of-space test, the client can see a state that avail_grant,
dirty_bytes and lost_grant are zero. In this case, page_mkwrite()
won't work because it can't allocate any grant and this affects fsx
test.

This problem is unlikely to be seen in real world. Therefore, we
wrote a small piece of data to each OSTs through each client mount
to reallocate grant.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I1d71af0a80491855cf6328a407dd417954c4e60e
Reviewed-on: http://review.whamcloud.com/3711
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
7 years agoLU-1952 tests: fix sanity test_51b on small MDTs
Andreas Dilger [Sun, 16 Sep 2012 23:09:26 +0000 (17:09 -0600)]
LU-1952 tests: fix sanity test_51b on small MDTs

Running sanity.sh test_51b() fails when the MDT filesystem is 256MB,
because the filesystem can create fewer than 64k directories consume
at 4kB each, even if there are enough inodes.  The following
test_51bb() fails if NUMTEST is reduced in test_51b(), because this is
done inside the subshell, and does not get propagated to the next
test.

Add an additional check that there are enough free blocks in the
filesystem for creating subdirectories, or reduce the subdirectory
count appropriately.   Also, write the actual number of subdirectories
created into a temporary file in test_51b() and then read it back in
test_51bb() so that the test completes correctly.

Do some minor cleanup of test style to match current guidelines.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I36119be4006d8a1d2b5034045b075c81de500c1e
Reviewed-on: http://review.whamcloud.com/4005
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 tests: restore CONF_SANITY_EXCEPT handling
Andreas Dilger [Mon, 1 Oct 2012 04:08:24 +0000 (22:08 -0600)]
LU-1581 tests: restore CONF_SANITY_EXCEPT handling

Commit 6a6561972406043efe41ae43b64fd278f360a4b9 removed the check in
conf-sanity.sh for a global test exception list.  This prevents
skipping tests from conf-sanity.sh when it is being run as part of a
larger set of tests, such as acceptance-small.sh.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I822cf5e9f873710c35321a88c94e576f15500c1e
Reviewed-on: http://review.whamcloud.com/4138
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 fld: Simplify transaction handling in FID/FLD
Mikhail Pershin [Wed, 3 Oct 2012 20:36:23 +0000 (00:36 +0400)]
LU-1943 fld: Simplify transaction handling in FID/FLD

This is port of ORI-107 to the master
- cleanup FID/FLD transaction handling, fix IGIF_FLD_RANGE start value
- use th_local in all places where transno is not needed
- remove mti_no_need_transno flag

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I4647dd7f8cd33b836cc959a9f74610c9fbc0e29c
Reviewed-on: http://review.whamcloud.com/4172
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: add per-filesystem information
Johann Lombardi [Mon, 1 Oct 2012 19:17:32 +0000 (21:17 +0200)]
LU-1842 quota: add per-filesystem information

Add a global list to track per-filesystem quota information which
will be used by slaves to determine whether quota enforcement is
enabled.
This patch also enables test 22 in sanity-quota.sh which should now
pass and avoids spurious test failures due to sync_all_data.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I336e7704d3e16dc3f5a92350c652cd6f2f25f25a
Reviewed-on: http://review.whamcloud.com/4156
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1730 obdclass: fix cfs_lockdep_set_class() typos
Ned Bass [Thu, 9 Aug 2012 21:42:17 +0000 (14:42 -0700)]
LU-1730 obdclass: fix cfs_lockdep_set_class() typos

During cl_object_header initialization, coh_attr_guard was mistakenly
passed to cfs_lockdep_set_class() for all three spinlocks in
cl_object_header.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I725b4f463fc3cbf8a6a5254f910f0e35588a7969
Reviewed-on: http://review.whamcloud.com/3590
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1935 scripts: don't generate lustre init script
Ned Bass [Sat, 15 Sep 2012 00:00:29 +0000 (17:00 -0700)]
LU-1935 scripts: don't generate lustre init script

An "overriding commands for target `lustre'" build warning resulted
from the addition of lustre to the genscripts Makefile variable.  In
fact the genscripts processing is superfluous for the lustre script,
so we could just fix the warning by removing it.  However, generating
the script from a .in file seems unnecessarily complicated, so this
change goes a step further by making the script static.  It now
detects at run-time whether to use the ldiskfsprogs or the e2fsprogs
utilities, instead of using autoconf macros to encode it at build
time.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: Ie000a87adabbc2a001ed9ff2f5750041a5abea5b
Reviewed-on: http://review.whamcloud.com/4015
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1943 osd: ldiskfs osd to set param.ddp_inodespace
Alex Zhuravlev [Wed, 3 Oct 2012 14:10:54 +0000 (18:10 +0400)]
LU-1943 osd: ldiskfs osd to set param.ddp_inodespace

so that grants code do proper reservation for metadata overhead.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ia63e60215d98ab4d950a86137a594a87d0569299
Reviewed-on: http://review.whamcloud.com/4167
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
7 years agoNew tag 2.3.52 2.3.52 v2_3_52 v2_3_52_0
Oleg Drokin [Thu, 4 Oct 2012 03:38:15 +0000 (23:38 -0400)]
New tag 2.3.52

Change-Id: Ie4048d4b0cee204ea5680e82a8cb2a31152e6d08
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1826 ldiskfs: Kernel update [SLES11 SP1 2.6.32.59-0.3.1]
James Simmons [Tue, 4 Sep 2012 17:30:11 +0000 (13:30 -0400)]
LU-1826 ldiskfs: Kernel update [SLES11 SP1 2.6.32.59-0.3.1]

Update SLES11 SP1 kernel to 2.6.32.59-0.3.1. Only difference
is EXT_MAX_BLOCK was renamed to match upstream EXT_MAX_BLOCKS.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I308c943a796090cd691a15ee32f0094fe8ec6da7
Reviewed-on: http://review.whamcloud.com/3861
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
7 years agoLU-2070 osp: debug for the issue
Alex Zhuravlev [Tue, 2 Oct 2012 05:04:51 +0000 (09:04 +0400)]
LU-2070 osp: debug for the issue

OSP to dump objects in the site if the reference > 0

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Id3d5e265a81c94a90f02cf7d4f4895d905ef9fc5
Reviewed-on: http://review.whamcloud.com/4151
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1855 build: fix 'out-of-bounds access' errors
Sebastien Buisson [Fri, 7 Sep 2012 09:04:36 +0000 (11:04 +0200)]
LU-1855 build: fix 'out-of-bounds access' errors

Fix 'out-of-bounds access' defects found by Coverity version 6.0.3:
Out-of-bounds access (OVERRUN_STATIC)
Overrunning static array by passing it to a
function which indexes it at a higher byte position.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Ic0fd90787ef969c786e9b0149df83491cdddc631
Reviewed-on: http://review.whamcloud.com/3902
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2076 osp: fix typo in osp2fsname
wangdi [Tue, 2 Oct 2012 10:49:47 +0000 (03:49 -0700)]
LU-2076 osp: fix typo in osp2fsname

Fix name converstion problem for ospname-->fsname.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I402639e0b5bce49af844e7979b290267e689c9dc
Reviewed-on: http://review.whamcloud.com/4157
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1943 echo: echo_client to get attributes like MDT does
Mikhail Pershin [Wed, 3 Oct 2012 07:18:41 +0000 (11:18 +0400)]
LU-1943 echo: echo_client to get attributes like MDT does

with changes in MDT/MDD the echo_client start working not-correctly,
it uses always 0-stiped files because gets mdsize == 0 from MDD.

- All attributes should be taken explicitely from MDD. Introduce
echo_attr_get_complex() and use it.
- don't use md_getsize_max(), it is not supported anymore.
- fix error handling issues
- remove obsoleted code

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I6f1277018d4b8716aa2e799e99aea6402c04b605
Reviewed-on: http://review.whamcloud.com/4164
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1749 llog: MDT should work if cancel llog failed with -ESTALE
Lai Siyao [Wed, 3 Oct 2012 04:11:05 +0000 (12:11 +0800)]
LU-1749 llog: MDT should work if cancel llog failed with -ESTALE

Cancel catlog may be stale, in this case MDT should continue to work.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Ib8cad1e2c935e73da0805875e3163f1a59edaeb5
Reviewed-on: http://review.whamcloud.com/3708
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1537 libcfs: 64-bit values are l64 on powerpc
Prakash Surya [Mon, 18 Jun 2012 18:31:20 +0000 (11:31 -0700)]
LU-1537 libcfs: 64-bit values are l64 on powerpc

The Linux kernel defines 64-bit values as 'long' rather than 'long long'
for user space applications on powerpc. Previously Lustre would assume
64-bit values were 'long long', which would cause compilation errors for
user space application when compiled on powerpc machines. This change
removes this assumption, correctly defining 64-bit types on powerpc.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I99f210e9402d3b2aea1a1ee5458b670387f65323
Reviewed-on: http://review.whamcloud.com/3127
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2078 ofd: drop locking in ofd_sync
Prakash Surya [Thu, 27 Sep 2012 22:37:10 +0000 (15:37 -0700)]
LU-2078 ofd: drop locking in ofd_sync

Improve concurrency by dropping the write lock around
dt_object_sync(). This is safe because all consistency
related to a single object is supposed to be maintained
by the osd internally.  In this case:

  1) osd_object_sync() doesn't rely on the object at all:
     global sync is called
  2) even if object is being destroyed at the same time -
     we're holding a reference, so we're safe
  3) dt_version_get() -> osd_xattr_get() has own internal
     locking, protecting xattr consistency to improve
     concurrency

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I60c5b6a1d10afc717a24e7471071f1e2fdb89c29
Reviewed-on: http://review.whamcloud.com/4117
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1337 build: fix build on powerpc64
Christopher J. Morrone [Tue, 2 Oct 2012 01:23:01 +0000 (18:23 -0700)]
LU-1337 build: fix build on powerpc64

Previous LU-1337 commit ca512093cd29e79ff8a43d4cd921058163305784
broke the build on powerpc64 by changing the way system type is
looked up, but failing to adjust the sed pattern to match the
$target_cpu name.

Change-Id: Ie0c8166f80c5361a8e89ab447c2025a31824914d
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4149
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2068 build: Respect "--without-ldiskfs" option
Prakash Surya [Tue, 2 Oct 2012 00:36:23 +0000 (17:36 -0700)]
LU-2068 build: Respect "--without-ldiskfs" option

The "--without-ldiskfs" configure option was not properly
being respected by a few of the Makefiles. This change is
an attempt to remedy this issue. Now, one should be able
to disable building ldiskfs and the related lvfs and osd
code with "--without-ldiskfs".

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I502e125639a60f62a37e3f95ecc943dedc37f5ea
Reviewed-on: http://review.whamcloud.com/4148
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2075 fld: use predefined FIDs
Alex Zhuravlev [Tue, 2 Oct 2012 19:52:42 +0000 (23:52 +0400)]
LU-2075 fld: use predefined FIDs

and let OSD do mapping to the names internally.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ibfaf9ae210d91dd32c3bb0a21c1078042596197a
Reviewed-on: http://review.whamcloud.com/4158
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 zfs-osd: different fixes for zfs
Alex Zhuravlev [Sun, 30 Sep 2012 17:38:02 +0000 (21:38 +0400)]
LU-1943 zfs-osd: different fixes for zfs

- echo_create_md_object() to mark la_mode is valid

- ofd_statfs() to set avail/free blocks to 2 when the counters
  are converted to 4K units. otherwise with zfs 2 blocks of size
  128K can be > 0.01% leading to failures in sanity/27*

- zfs-osd to initialize LMA

- zfs-osd's osd_attr_get() to calculate size of directory from
  blocks and do not use cached size (as we don't control it as
  with regular files)

- zfs-osd to track enabled/disabled status for SA and follow it

- take from orion additional locking in osd_object_sa_dirty_add()
  and osd_object_sa_dirty_rele()

- lu_cache_shrink() do not purge objects if __GFP_FS is not set
  this should prevent deadlock when used with zfs backend

- ost_checksum_bulk() to allocate new page to simulate corruption,
  otherwise it corrupts zfs's internal buffers which do not rely
  on PageUptoDate flag

- t-f to modprobe zfs module if osd-zfs is in use

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I027483162735cd0fb4be9fa8bd12c2f2dc43b44a
Reviewed-on: http://review.whamcloud.com/4141
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1934 echo: echo client to work over OFD
Mikhail Pershin [Mon, 1 Oct 2012 16:44:41 +0000 (20:44 +0400)]
LU-1934 echo: echo client to work over OFD

- echo client doesn't call obd_alloc/free_memmd() to underlaying
  device but implement own methods
- Use real lu_env in the prep/commitrw calls
- fill OBD_BRW_ASYNC flag for async writes

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Ic02d4c2ad08693cd47d8dff5e8a86812a1534763
Reviewed-on: http://review.whamcloud.com/4146
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1095 debug: conditionally print recovery status
Minh Diep [Wed, 19 Sep 2012 17:50:11 +0000 (10:50 -0700)]
LU-1095 debug: conditionally print recovery status

Avoid printing non-sense messages like this when the recovery
code hasn't run. For example, this can occur when your starting
a new target for the first time.

  Lustre: lustre-OST0001: Recovery over after 22277843:04,
  of 0 clients 0 recovered and 0 were evicted.

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: Ib4bd8783c0c17aa15052fc9b9cf78e18274b2783
Reviewed-on: http://review.whamcloud.com/4043
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
7 years agoLU-1934 ofd: implement precreate batching
Mikhail Pershin [Mon, 1 Oct 2012 17:17:48 +0000 (21:17 +0400)]
LU-1934 ofd: implement precreate batching

Bulk precreate objects in a single tx handle.  For zfs this minimizes
the chance that the creates will be spread over multiple txgs.  This
can occur when the osts are under a concurrent write workload which
can consume bulk of the available space in a txg.

For ldiskfs this change shouldn't be harmful and the default value
can be set to 1 for ldiskfs if needed to get the previous behavior.

Change-Id: I81e310a8b630a1a29f3e017f6980901d6b6436fa
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4147
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-294 Fix typo in tests/test-groups/regression
Li Wei [Sun, 8 May 2011 05:48:47 +0000 (13:48 +0800)]
LU-294 Fix typo in tests/test-groups/regression

Signed-off-by: Li Wei <liwei@whamcloud.com>
Change-Id: I91015ba4385853db08d9a79fa574e556e1fce48c
Reviewed-on: http://review.whamcloud.com/513
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: setup/shutdown qmt device
Johann Lombardi [Fri, 28 Sep 2012 11:40:49 +0000 (13:40 +0200)]
LU-1842 quota: setup/shutdown qmt device

The quota master target (QMT) is in charge of the cluster wide quota
settings. The QMT device is setup/shutdown by the MDT which then
forwards all quota requests to the QMT via a list of request/lock
handlers exported by the QMT.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I0bd800d2996f5c3b9d861d4c6fbd70dabc3b2860
Reviewed-on: http://review.whamcloud.com/4126
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: qsd lock
Niu Yawei [Sat, 29 Sep 2012 06:12:43 +0000 (02:12 -0400)]
LU-1842 quota: qsd lock

qsd lock glimpse/blocking ast, qsd lock cancel/match functions.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ief424ca77dd9538d56689410fadfd71b01a03974
Reviewed-on: http://review.whamcloud.com/4129
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: qsd entry
Niu Yawei [Fri, 28 Sep 2012 09:34:48 +0000 (05:34 -0400)]
LU-1842 quota: qsd entry

qsd entry operations.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I88c8b0658798d0b2bea62ef80c79b28badc289d2
Reviewed-on: http://review.whamcloud.com/4119
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: qsd request
Niu Yawei [Thu, 27 Sep 2012 12:44:55 +0000 (08:44 -0400)]
LU-1842 quota: qsd request

qsd request handling APIs.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Id88e0a49482e7d56af39f9aa1d3105316b6cb3dd
Reviewed-on: http://review.whamcloud.com/4109
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 osp: osp-on-ost device
Niu Yawei [Fri, 21 Sep 2012 08:54:54 +0000 (04:54 -0400)]
LU-1842 osp: osp-on-ost device

Setup osp-on-ost device for each OST to maintain the connection
to MDT0, quota slave & ofd will send quota or fid sequence request
over this connection.

We also setup one such device on MDT0 for now, it'll be used by
the quota slave on MDT0, once the DNE code landed, this device
(on MDT0) could be replaced by normal osp device.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I2229cec1fa206dcce65a4cceae032566f4c20901
Reviewed-on: http://review.whamcloud.com/4068
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2040 tests: interop test failure on mds-survey
Bob Glossman [Fri, 28 Sep 2012 15:32:18 +0000 (08:32 -0700)]
LU-2040 tests: interop test failure on mds-survey

Add a version check to skip mds-survey in interop testing.
This is known to fail and should not block testing.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4d1ef336cecf525d7df54816cb9731ec6c6edd91
Reviewed-on: http://review.whamcloud.com/4124
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
7 years agoLU-1870 utils: correctly handle mount of multipath devices
Gregoire Pichon [Mon, 10 Sep 2012 13:27:26 +0000 (15:27 +0200)]
LU-1870 utils: correctly handle mount of multipath devices

The mount.lustre command had lost the correct handling of multipath
devices. When one is specified, an entry is added into /etc/mtab
using the device name: /dev/mapper/mpathXX. This ensures unmount
works fine with such devices.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I9060be882af944967a8c4b6b1c4403ea9250ad55
Reviewed-on: http://review.whamcloud.com/3922
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1540 tests: only run test for ldiskfs MDT
Andreas Dilger [Thu, 13 Sep 2012 22:43:10 +0000 (16:43 -0600)]
LU-1540 tests: only run test for ldiskfs MDT

Skip the test for LU-1540 (NUL symlink termination) on ZFS MDT
filesystems, since e2fsck won't work there.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibba41352119f77b296e95520603b8d5527ab91b7
Reviewed-on: http://review.whamcloud.com/3987
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2057 ofd: implement jobstats and few fixes
Mikhail Pershin [Mon, 1 Oct 2012 10:42:30 +0000 (14:42 +0400)]
LU-2057 ofd: implement jobstats and few fixes

- implement jobstats for OFD like in obdfilter
- fix ofd_health_check to ignore external lu_env and use local one
  always, because obd_proc_read_health uses it without env
- do object sync only if its version is not yet committed

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Idf74ef280f9d71e3db6b45cfacb2f5303b302a77
Reviewed-on: http://review.whamcloud.com/4143
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2050 lod: set l_ost_gen to 0
Alex Zhuravlev [Mon, 1 Oct 2012 07:21:59 +0000 (11:21 +0400)]
LU-2050 lod: set l_ost_gen to 0

like LOV does.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I68a2fcda210ab34f98d64b1d1075593b0310066a
Reviewed-on: http://review.whamcloud.com/4142
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2053 crypto: Fix cfs_crypto_hash memleak
Oleg Drokin [Sat, 29 Sep 2012 20:15:40 +0000 (16:15 -0400)]
LU-2053 crypto: Fix cfs_crypto_hash memleak

Commit 353e3c2f8d9f195c0f87a16259f22b2f84de11d4 that introduced cryptoapi
support to Lustre is allocating a hdesc in cfs_crypto_hash_init and never
freeing it.
This function is called every time a checksum needs to be calculated,
so on every bulk RPC send or receive, so even though allocations are small,
they tend to still quickly consume all available memory with any significant
amount of IO.

Change-Id: Ic654c6102ea4306a81c1f7592c81cfb6e597f1f2
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4135
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
7 years agoLU-1867 tests: replay-single test_89: @@ FAIL: 4 blocks leaked
yangsheng [Sat, 29 Sep 2012 09:18:53 +0000 (17:18 +0800)]
LU-1867 tests: replay-single test_89: @@ FAIL: 4 blocks leaked

The config llog data will changed while OST remount. It may
impact 'df' output. So we should allow extra one block size
different(4k).

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I663685d55a5a31af48d541e08e9e97fed2b8e25c
Reviewed-on: http://review.whamcloud.com/4130
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1943 osd: fix lost functionality in OFD and OSD
Mikhail Pershin [Fri, 14 Sep 2012 21:39:00 +0000 (01:39 +0400)]
LU-1943 osd: fix lost functionality in OFD and OSD

- restore ofd_fmd_expire() in ofd_ping()
- set OS_STATE_READONLY in osd_statfs()
- fix readcache initialization and add lprocfs entries for it
- fix test_19 to work with OSD if OFD is used

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I357b28d8d5de1e48c55b12faa4d31ac75dbd5f66
Reviewed-on: http://review.whamcloud.com/3991
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2022 utils: Mount MGS lacking index
James Simmons [Tue, 25 Sep 2012 16:02:42 +0000 (12:02 -0400)]
LU-2022 utils: Mount MGS lacking index

The MGT is the only device that can't be formatted with a
index but mount.lustre expects all devices to be formated
with a index. This patch allows the MGT to be the exception.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Icea4e18173e9c670302a795bd53b2769b55fc38f
Reviewed-on: http://review.whamcloud.com/4084
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Keith Mannthey <keith@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1873 test: Must set USE_OFD=yes when backend FS is zfs
Minh Diep [Tue, 11 Sep 2012 07:07:44 +0000 (00:07 -0700)]
LU-1873 test: Must set USE_OFD=yes when backend FS is zfs

We need to make sure that USE_OFD=yes when backend FS is
zfs by checking the *FSTYPE=zfs.

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: Ia8c646ea82a0ba19c0cc1b997e1c9322a4a54f01
Reviewed-on: http://review.whamcloud.com/3930
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: remove quota code from lvfs
Johann Lombardi [Thu, 27 Sep 2012 13:55:53 +0000 (15:55 +0200)]
LU-1842 quota: remove quota code from lvfs

This patch removes the quota code from the lvfs directory.
Most of this code has already been moved to ldiskfs OSD.
This patch also removes the dt_init_quota_ctxt which isn't
used any more.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I6ab6a26ae877d5c0bebab674204f7d36f9d83078
Reviewed-on: http://review.whamcloud.com/4110
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2044 libcfs: fix to build libcfs without pthreads
James Simmons [Fri, 28 Sep 2012 12:14:20 +0000 (08:14 -0400)]
LU-2044 libcfs: fix to build libcfs without pthreads

Currently libcfs fails to build on systems without pthreads
in the libcfs_lock code with its pseudo lock implementation.
Instead of creating a phony address pointer we create a real
static lock and test the pointer to it. The compiler is much
more happy with this approach.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia66d0972951afbfe9bc69a6375d57e6a13ae4e43
Reviewed-on: http://review.whamcloud.com/4121
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoNew tag 2.3.51 2.3.51 v2_3_51 v2_3_51_0
Oleg Drokin [Sun, 30 Sep 2012 15:36:29 +0000 (11:36 -0400)]
New tag 2.3.51

Change-Id: I9bb65dfdc670beef6a82ec6d71ef4c9bc5d0a081

7 years agoLU-1303 osd: zfs-osd should initialize local objects on MDS
Alex Zhuravlev [Sat, 29 Sep 2012 18:47:07 +0000 (22:47 +0400)]
LU-1303 osd: zfs-osd should initialize local objects on MDS

zfs-osd to call llo_local_objects_setup() to initialize local
MDS objects (like ldiskfs-osd does).

zfs-osd should not try to control naming for objects maintained
by MDT/MDT (initialized with llo_local_objects_setup()).

disable LASSERT() in MGS: with zfs backend it's not correct.

MDT does not need to access configs, so disable setup of llogs
in MDT.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I464a78372daf8baf1900f2dbb33e74ba9ffb33b1
Reviewed-on: http://review.whamcloud.com/4134
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 fld: verify support for range lookups
Alex Zhuravlev [Sat, 29 Sep 2012 20:38:18 +0000 (00:38 +0400)]
LU-1303 fld: verify support for range lookups

if the underlying storage does not support range lookups,
then FLD disables any mappings with server index different
from 0, thus disabling DNE and FID-on-OST features.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Id93b01385acfafde024dff33557656afdcd1e4b3
Reviewed-on: http://review.whamcloud.com/4133
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2034 changelog: redo changelog using OSD llog
Mikhail Pershin [Thu, 27 Sep 2012 12:16:26 +0000 (16:16 +0400)]
LU-2034 changelog: redo changelog using OSD llog

changelog to use llog over OSD

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I4b9d1ee5146ef882db6031e996a6d81b6cae870b
Reviewed-on: http://review.whamcloud.com/4120
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2047 obdfilter: init glimpse desc to NULL for ext lock
Johann Lombardi [Fri, 28 Sep 2012 20:46:12 +0000 (22:46 +0200)]
LU-2047 obdfilter: init glimpse desc to NULL for ext lock

There is no glimpse descriptor when glimpsing extent locks, we should
thus set ldlm_glimpse_work::gl_desc to NULL to allocate glimpse
request with the correct format.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I5cdedfb79631328114ef7b0bc8e035dde985b4fa
Reviewed-on: http://review.whamcloud.com/4127
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1303 mds: integration lod/osp into the stack
Alex Zhuravlev [Fri, 21 Sep 2012 19:45:32 +0000 (23:45 +0400)]
LU-1303 mds: integration lod/osp into the stack

 - config handler recognizes MDS config and replace LOV/OSC devices
   with LOD/OSP

 - the devices in MDS stack are setup from bottom to top using
   obd_connect() namely MDT->MDD->LOD->OSD, when new OSPs arrive,
   they connect to the OSD and LOD connects to every OSP). all the
   devices are referenced by exports, preventing them from early
   release.

 - once the whole configuration is processed, prepare() method of the
   top device is called, that in turn call ->prepare() down through
   the stack, etc. at this point the stack is considered ready to
   access storage, start recovery and do regular processing.

 - at umount ->ldo_process_config(LCFG_CLEANUP) is called down through
   the stack asking devices to release the resources (like objects
   referenced for internal purposes), then the series of calls to
   obd_disconnect()'s starting from the top MDT causes all the devices
   to become unreferenced and subject to release by zombie thread.

 - obd_notify() mechanism is not used by MDT/MDD/LOD/OSD, it's still
   used by OSP to be aware of current state:

      - to learn when the first connection to OST happens and let
        clients to connect to MDS, MDS uses obd_health_check() and
        caches positive result.

      - during object creation LOD uses dt_statfs() and data returned
        by OSP tells LOD whether specific OST active/inactive, space
        available, number of precreated objects, etc.

 - LOD takes care of striping: it maintains the list of available OST
   represented locally by OSP devices. on create, LOD finds suitable
   OSPs, create objects by means of OSD API and stores striping info
   into a local object. similarly LOD handles object destroy and
   UID/GID changes: load striping and execute on them.

 - llog is used by OSP as a local library, cookies are not visible out
   side of OSP. thus neither MDT nor MDD needs to care about cookies,
   prepare buffers. MDD doesn't need to send RPCs to destroy objects
   on behalf of evicted clients. all is hidden behind OSP's object
   destroy method.

 - OSP generates llog records for OST object being destroyed and, once
   corresponded local transaction is committed, send OST_DESTROY RPC.
   once OST_DESTROY is reported to be committed, corresponded llog
   record is cancelled.

 - the same logic is used for UID/GID changes.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ifb282ace94f583ffa86020b763f22922e5d0b032
Reviewed-on: http://review.whamcloud.com/4087
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1301 mgs: Symlink mntdev procfs entry from osd
Alex Zhuravlev [Thu, 9 Aug 2012 18:52:17 +0000 (22:52 +0400)]
LU-1301 mgs: Symlink mntdev procfs entry from osd

MGS won't be able to get device names directly on top of OSD.

Change-Id: Ief808f058a785cc4bacf1743ff534a81be234544
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/3672
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1981 doc: initial revision of OSD API doc
Alex Zhuravlev [Fri, 28 Sep 2012 21:43:03 +0000 (01:43 +0400)]
LU-1981 doc: initial revision of OSD API doc

this is the initial revision of the document describing
OSD API, its terms and methods.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I26459c5bd1d876a4b9a8ca778c835d3faf6a9087
Reviewed-on: http://review.whamcloud.com/4128
Reviewed-by: Ian Colle <Ian.Colle@intel.com>
Tested-by: Ian Colle <Ian.Colle@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: propagate real object size on a new striping
Alex Zhuravlev [Thu, 20 Sep 2012 12:00:04 +0000 (16:00 +0400)]
LU-1303 lod: propagate real object size on a new striping

it's possible to truncate file with no striping, when create
striping. in such a case lod's object create propagate actual
size down to the stack.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Iaf4a91516892db23f11052bd1a656ee2108a7113
Reviewed-on: http://review.whamcloud.com/4059
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: QoS allocation policy
Alex Zhuravlev [Thu, 20 Sep 2012 11:56:31 +0000 (15:56 +0400)]
LU-1303 lod: QoS allocation policy

basically a copy of lov_qos.c implemented with OSD API.

the patch introduces a special lu-helpers allowing objects
with zero fid - used by LOD/OSP as fid is being created late.
this is a requirement of current orphan cleanup protocol.
probably will be replace with natural fids generated outside
of OSP/transaction at some point.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I9f41d8a770eb7b8ca04b729bbf0866e5d27fff8a
Reviewed-on: http://review.whamcloud.com/4058
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>