Whamcloud - gitweb
fs/lustre-release.git
7 years agoLU-2033 scrub: lookup server local files by name
Fan Yong [Wed, 26 Sep 2012 16:53:18 +0000 (00:53 +0800)]
LU-2033 scrub: lookup server local files by name

after backup/restore ldiskfs's OI becomes invalid (inodes
change). OI scrubber scans a filesystem to rebuild OI. this
can take quite long. to be able to start MDS earlier (right
after restore) MDS should lookup by name all internal files
required for startup as name->inode mapping is preserved by
backup/restore. this patch make local object library to
lookup a file storing last used fid within a given sequence.

the patch also fixes sanity-scrub/11 to expect skipping of
files created + directory + per-ost llog objects.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I56179c03279062464960af73de031f13d31bac14
Reviewed-on: http://review.whamcloud.com/4106
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: add acq/rel logic on QMT
Johann Lombardi [Thu, 4 Oct 2012 08:34:09 +0000 (10:34 +0200)]
LU-1842 quota: add acq/rel logic on QMT

This patch adds the quota acquire/release logic on the QMT.
The QMT is in charge of distributing quota space across quota
slaves. It grants a significant amount of quota space (75%) initially
and then tries to balance spare quota space among slaves.
To do so, it sends glimpse on per-ID lock to notify slaves of qunit
shrink. qunit is now the maximal amount of available quota space a
slave can own. As for qtune, this value (typically 1/4 of qunit) is
used on the quota slave to decide when to release/pre-acquire quota
space.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I8064ce614955421681ca1da7e2bb774001ca3bea
Reviewed-on: http://review.whamcloud.com/4178
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: add quota locks support on QMT
Johann Lombardi [Wed, 3 Oct 2012 13:02:42 +0000 (15:02 +0200)]
LU-1842 quota: add quota locks support on QMT

This patch adds methods for the quota lock lvb management and
implements an intent policy for quota locks.
It also introduces the rebalance thread which is in charge of
sending glimpse callbacks to slave to notify them of new quota
settings.

This patch also enables sanity-quota.sh test 36 which should now
pass.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I0cf7f86d480296f7e8fe774ed8937aae27736f88
Reviewed-on: http://review.whamcloud.com/4166
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1943 ofd: save server last transaction number
Mikhail Pershin [Thu, 4 Oct 2012 05:17:53 +0000 (09:17 +0400)]
LU-1943 ofd: save server last transaction number

After reading server data from the last_rcvd file we must store the
last transaction number in lu_target.  Otherwise we write a zero last
transaction number back to last_rcvd, causing transaction numbers to
roll back to 2^32 every time the server is restarted.  This can lead
to 'server went back in time' client console errors.

Whamcloud-bug-id: ORI-670
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Iaff4af3e4bd6fce0da1d1b0886147538c3300c1e
Reviewed-on: http://review.whamcloud.com/4190
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Ian Colle <Ian.Colle@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2071 lprocfs: Memory leak on lustre proc access
Oleg Drokin [Tue, 2 Oct 2012 02:55:27 +0000 (22:55 -0400)]
LU-2071 lprocfs: Memory leak on lustre proc access

Make __LPROC_SEQ_FOPS to call lprocfs_single_release in release so that
the memory allocated in single_open to be freed.
Rename some of the auto-named methods to better reflect on the nature
of the underlying calls.
The previous code used to leak 32 bytes per every lustre proc file open
that uses lprocfs_..._seq_open primitive.

Change-Id: I08cc1bbb28af444f3fa5e903e38fd4a16dc4c79b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4150
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
7 years agoLU-1798 tests: add set_and_check to test-framework
Andreas Dilger [Mon, 1 Oct 2012 04:53:55 +0000 (22:53 -0600)]
LU-1798 tests: add set_and_check to test-framework

Move the set_and_check() helper function from conf-sanity.sh into
test-framework.sh as set_conf_param_and_check() so that it can be
used in other test scripts.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4ad3179decff55a0ec17a4c983e30941cf49cab0
Reviewed-on: http://review.whamcloud.com/4140
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2068 build: Disable obdfilter with osd-zfs
James Simmons [Thu, 4 Oct 2012 14:41:26 +0000 (10:41 -0400)]
LU-2068 build: Disable obdfilter with osd-zfs

This change introduces a few different cleanup and build system
changes. Most notably, the obdfilter modules is now disabled and
the mkfs utilities does not call into ldiskfs helpers when ldiskfs
is disabled. Apart from that, the other changes are various minor
cleanup and dead code removal.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Idcfda525e4b76269c2cf17c1b3232033b92fc8a8
Reviewed-on: http://review.whamcloud.com/4159
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2087 ofd: refresh blocks in lvb from rpc
Alex Zhuravlev [Thu, 4 Oct 2012 10:51:24 +0000 (14:51 +0400)]
LU-2087 ofd: refresh blocks in lvb from rpc

like obdfilter does. let blocks only increase unless
specified with the flag. this should fix sanityn/4.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I85bf45f091637faf39c65e0acf331f37dab7af7e
Reviewed-on: http://review.whamcloud.com/4180
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
7 years agoLU-1943 class: FID_SEQ_LOCAL_NAME set to the Orion value
Mikhail Pershin [Thu, 4 Oct 2012 10:24:43 +0000 (14:24 +0400)]
LU-1943 class: FID_SEQ_LOCAL_NAME set to the Orion value

Keep the same numbers for Orion and master for compatibility

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I318eba9860be7849ee4a8d828cf27e5fb91164e9
Reviewed-on: http://review.whamcloud.com/4179
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2092 Free quota pools before removing quota master proc entry
Oleg Drokin [Thu, 4 Oct 2012 23:13:14 +0000 (19:13 -0400)]
LU-2092 Free quota pools before removing quota master proc entry

When quota master proc entry is removed, it reqursively removes
all children entries including quota pool ones, then when quota pools
are freed they try to unregister already not present proc entries and
crash/lockup.
So free pools before unregistering any proc entires.

Change-Id: Ice0be373bacbd54f4863aec92fb2427c7fe27464
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4191
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2088 utils: lctl to generate gdb symbols for lod/osp
Alex Zhuravlev [Thu, 4 Oct 2012 14:50:57 +0000 (18:50 +0400)]
LU-2088 utils: lctl to generate gdb symbols for lod/osp

useful for people using gdb

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5dbd3c1757b3462f6c98c5ad03fc3f83a7457e40
Reviewed-on: http://review.whamcloud.com/4183
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1574 osd: don't use obd_name but od_svname in error messages
Mikhail Pershin [Wed, 3 Oct 2012 07:46:59 +0000 (11:46 +0400)]
LU-1574 osd: don't use obd_name but od_svname in error messages

obd_name is not initialized in OSD, but there is od_svname in
osd_device to be used instead.

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I87a83badc358ad508ecb8a4e593a7e744277e57f
Reviewed-on: http://review.whamcloud.com/4165
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: add quotactl support on qmt
Johann Lombardi [Tue, 2 Oct 2012 16:11:21 +0000 (18:11 +0200)]
LU-1842 quota: add quotactl support on qmt

This patch implements quotactl ({get,set}{quota,info}) support on the
QMT. This involves defining the lqe operations for the qmt as well as
pool operations in order to support per-pool quota in the future.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I122e7cb85eb6d1d3460e86217e2a6723763c8cbf
Reviewed-on: http://review.whamcloud.com/4160
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1028 tests: regrab some grant after oos test
Jinshan Xiong [Fri, 17 Aug 2012 17:15:33 +0000 (10:15 -0700)]
LU-1028 tests: regrab some grant after oos test

After out-of-space test, the client can see a state that avail_grant,
dirty_bytes and lost_grant are zero. In this case, page_mkwrite()
won't work because it can't allocate any grant and this affects fsx
test.

This problem is unlikely to be seen in real world. Therefore, we
wrote a small piece of data to each OSTs through each client mount
to reallocate grant.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I1d71af0a80491855cf6328a407dd417954c4e60e
Reviewed-on: http://review.whamcloud.com/3711
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
7 years agoLU-1952 tests: fix sanity test_51b on small MDTs
Andreas Dilger [Sun, 16 Sep 2012 23:09:26 +0000 (17:09 -0600)]
LU-1952 tests: fix sanity test_51b on small MDTs

Running sanity.sh test_51b() fails when the MDT filesystem is 256MB,
because the filesystem can create fewer than 64k directories consume
at 4kB each, even if there are enough inodes.  The following
test_51bb() fails if NUMTEST is reduced in test_51b(), because this is
done inside the subshell, and does not get propagated to the next
test.

Add an additional check that there are enough free blocks in the
filesystem for creating subdirectories, or reduce the subdirectory
count appropriately.   Also, write the actual number of subdirectories
created into a temporary file in test_51b() and then read it back in
test_51bb() so that the test completes correctly.

Do some minor cleanup of test style to match current guidelines.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I36119be4006d8a1d2b5034045b075c81de500c1e
Reviewed-on: http://review.whamcloud.com/4005
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 tests: restore CONF_SANITY_EXCEPT handling
Andreas Dilger [Mon, 1 Oct 2012 04:08:24 +0000 (22:08 -0600)]
LU-1581 tests: restore CONF_SANITY_EXCEPT handling

Commit 6a6561972406043efe41ae43b64fd278f360a4b9 removed the check in
conf-sanity.sh for a global test exception list.  This prevents
skipping tests from conf-sanity.sh when it is being run as part of a
larger set of tests, such as acceptance-small.sh.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I822cf5e9f873710c35321a88c94e576f15500c1e
Reviewed-on: http://review.whamcloud.com/4138
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 fld: Simplify transaction handling in FID/FLD
Mikhail Pershin [Wed, 3 Oct 2012 20:36:23 +0000 (00:36 +0400)]
LU-1943 fld: Simplify transaction handling in FID/FLD

This is port of ORI-107 to the master
- cleanup FID/FLD transaction handling, fix IGIF_FLD_RANGE start value
- use th_local in all places where transno is not needed
- remove mti_no_need_transno flag

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I4647dd7f8cd33b836cc959a9f74610c9fbc0e29c
Reviewed-on: http://review.whamcloud.com/4172
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: add per-filesystem information
Johann Lombardi [Mon, 1 Oct 2012 19:17:32 +0000 (21:17 +0200)]
LU-1842 quota: add per-filesystem information

Add a global list to track per-filesystem quota information which
will be used by slaves to determine whether quota enforcement is
enabled.
This patch also enables test 22 in sanity-quota.sh which should now
pass and avoids spurious test failures due to sync_all_data.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I336e7704d3e16dc3f5a92350c652cd6f2f25f25a
Reviewed-on: http://review.whamcloud.com/4156
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1730 obdclass: fix cfs_lockdep_set_class() typos
Ned Bass [Thu, 9 Aug 2012 21:42:17 +0000 (14:42 -0700)]
LU-1730 obdclass: fix cfs_lockdep_set_class() typos

During cl_object_header initialization, coh_attr_guard was mistakenly
passed to cfs_lockdep_set_class() for all three spinlocks in
cl_object_header.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I725b4f463fc3cbf8a6a5254f910f0e35588a7969
Reviewed-on: http://review.whamcloud.com/3590
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1935 scripts: don't generate lustre init script
Ned Bass [Sat, 15 Sep 2012 00:00:29 +0000 (17:00 -0700)]
LU-1935 scripts: don't generate lustre init script

An "overriding commands for target `lustre'" build warning resulted
from the addition of lustre to the genscripts Makefile variable.  In
fact the genscripts processing is superfluous for the lustre script,
so we could just fix the warning by removing it.  However, generating
the script from a .in file seems unnecessarily complicated, so this
change goes a step further by making the script static.  It now
detects at run-time whether to use the ldiskfsprogs or the e2fsprogs
utilities, instead of using autoconf macros to encode it at build
time.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: Ie000a87adabbc2a001ed9ff2f5750041a5abea5b
Reviewed-on: http://review.whamcloud.com/4015
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1943 osd: ldiskfs osd to set param.ddp_inodespace
Alex Zhuravlev [Wed, 3 Oct 2012 14:10:54 +0000 (18:10 +0400)]
LU-1943 osd: ldiskfs osd to set param.ddp_inodespace

so that grants code do proper reservation for metadata overhead.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ia63e60215d98ab4d950a86137a594a87d0569299
Reviewed-on: http://review.whamcloud.com/4167
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
7 years agoNew tag 2.3.52 2.3.52 v2_3_52 v2_3_52_0
Oleg Drokin [Thu, 4 Oct 2012 03:38:15 +0000 (23:38 -0400)]
New tag 2.3.52

Change-Id: Ie4048d4b0cee204ea5680e82a8cb2a31152e6d08
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1826 ldiskfs: Kernel update [SLES11 SP1 2.6.32.59-0.3.1]
James Simmons [Tue, 4 Sep 2012 17:30:11 +0000 (13:30 -0400)]
LU-1826 ldiskfs: Kernel update [SLES11 SP1 2.6.32.59-0.3.1]

Update SLES11 SP1 kernel to 2.6.32.59-0.3.1. Only difference
is EXT_MAX_BLOCK was renamed to match upstream EXT_MAX_BLOCKS.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I308c943a796090cd691a15ee32f0094fe8ec6da7
Reviewed-on: http://review.whamcloud.com/3861
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
7 years agoLU-2070 osp: debug for the issue
Alex Zhuravlev [Tue, 2 Oct 2012 05:04:51 +0000 (09:04 +0400)]
LU-2070 osp: debug for the issue

OSP to dump objects in the site if the reference > 0

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Id3d5e265a81c94a90f02cf7d4f4895d905ef9fc5
Reviewed-on: http://review.whamcloud.com/4151
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1855 build: fix 'out-of-bounds access' errors
Sebastien Buisson [Fri, 7 Sep 2012 09:04:36 +0000 (11:04 +0200)]
LU-1855 build: fix 'out-of-bounds access' errors

Fix 'out-of-bounds access' defects found by Coverity version 6.0.3:
Out-of-bounds access (OVERRUN_STATIC)
Overrunning static array by passing it to a
function which indexes it at a higher byte position.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Ic0fd90787ef969c786e9b0149df83491cdddc631
Reviewed-on: http://review.whamcloud.com/3902
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2076 osp: fix typo in osp2fsname
wangdi [Tue, 2 Oct 2012 10:49:47 +0000 (03:49 -0700)]
LU-2076 osp: fix typo in osp2fsname

Fix name converstion problem for ospname-->fsname.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I402639e0b5bce49af844e7979b290267e689c9dc
Reviewed-on: http://review.whamcloud.com/4157
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1943 echo: echo_client to get attributes like MDT does
Mikhail Pershin [Wed, 3 Oct 2012 07:18:41 +0000 (11:18 +0400)]
LU-1943 echo: echo_client to get attributes like MDT does

with changes in MDT/MDD the echo_client start working not-correctly,
it uses always 0-stiped files because gets mdsize == 0 from MDD.

- All attributes should be taken explicitely from MDD. Introduce
echo_attr_get_complex() and use it.
- don't use md_getsize_max(), it is not supported anymore.
- fix error handling issues
- remove obsoleted code

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I6f1277018d4b8716aa2e799e99aea6402c04b605
Reviewed-on: http://review.whamcloud.com/4164
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1749 llog: MDT should work if cancel llog failed with -ESTALE
Lai Siyao [Wed, 3 Oct 2012 04:11:05 +0000 (12:11 +0800)]
LU-1749 llog: MDT should work if cancel llog failed with -ESTALE

Cancel catlog may be stale, in this case MDT should continue to work.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Ib8cad1e2c935e73da0805875e3163f1a59edaeb5
Reviewed-on: http://review.whamcloud.com/3708
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1537 libcfs: 64-bit values are l64 on powerpc
Prakash Surya [Mon, 18 Jun 2012 18:31:20 +0000 (11:31 -0700)]
LU-1537 libcfs: 64-bit values are l64 on powerpc

The Linux kernel defines 64-bit values as 'long' rather than 'long long'
for user space applications on powerpc. Previously Lustre would assume
64-bit values were 'long long', which would cause compilation errors for
user space application when compiled on powerpc machines. This change
removes this assumption, correctly defining 64-bit types on powerpc.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I99f210e9402d3b2aea1a1ee5458b670387f65323
Reviewed-on: http://review.whamcloud.com/3127
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2078 ofd: drop locking in ofd_sync
Prakash Surya [Thu, 27 Sep 2012 22:37:10 +0000 (15:37 -0700)]
LU-2078 ofd: drop locking in ofd_sync

Improve concurrency by dropping the write lock around
dt_object_sync(). This is safe because all consistency
related to a single object is supposed to be maintained
by the osd internally.  In this case:

  1) osd_object_sync() doesn't rely on the object at all:
     global sync is called
  2) even if object is being destroyed at the same time -
     we're holding a reference, so we're safe
  3) dt_version_get() -> osd_xattr_get() has own internal
     locking, protecting xattr consistency to improve
     concurrency

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I60c5b6a1d10afc717a24e7471071f1e2fdb89c29
Reviewed-on: http://review.whamcloud.com/4117
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1337 build: fix build on powerpc64
Christopher J. Morrone [Tue, 2 Oct 2012 01:23:01 +0000 (18:23 -0700)]
LU-1337 build: fix build on powerpc64

Previous LU-1337 commit ca512093cd29e79ff8a43d4cd921058163305784
broke the build on powerpc64 by changing the way system type is
looked up, but failing to adjust the sed pattern to match the
$target_cpu name.

Change-Id: Ie0c8166f80c5361a8e89ab447c2025a31824914d
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4149
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2068 build: Respect "--without-ldiskfs" option
Prakash Surya [Tue, 2 Oct 2012 00:36:23 +0000 (17:36 -0700)]
LU-2068 build: Respect "--without-ldiskfs" option

The "--without-ldiskfs" configure option was not properly
being respected by a few of the Makefiles. This change is
an attempt to remedy this issue. Now, one should be able
to disable building ldiskfs and the related lvfs and osd
code with "--without-ldiskfs".

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I502e125639a60f62a37e3f95ecc943dedc37f5ea
Reviewed-on: http://review.whamcloud.com/4148
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2075 fld: use predefined FIDs
Alex Zhuravlev [Tue, 2 Oct 2012 19:52:42 +0000 (23:52 +0400)]
LU-2075 fld: use predefined FIDs

and let OSD do mapping to the names internally.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ibfaf9ae210d91dd32c3bb0a21c1078042596197a
Reviewed-on: http://review.whamcloud.com/4158
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1943 zfs-osd: different fixes for zfs
Alex Zhuravlev [Sun, 30 Sep 2012 17:38:02 +0000 (21:38 +0400)]
LU-1943 zfs-osd: different fixes for zfs

- echo_create_md_object() to mark la_mode is valid

- ofd_statfs() to set avail/free blocks to 2 when the counters
  are converted to 4K units. otherwise with zfs 2 blocks of size
  128K can be > 0.01% leading to failures in sanity/27*

- zfs-osd to initialize LMA

- zfs-osd's osd_attr_get() to calculate size of directory from
  blocks and do not use cached size (as we don't control it as
  with regular files)

- zfs-osd to track enabled/disabled status for SA and follow it

- take from orion additional locking in osd_object_sa_dirty_add()
  and osd_object_sa_dirty_rele()

- lu_cache_shrink() do not purge objects if __GFP_FS is not set
  this should prevent deadlock when used with zfs backend

- ost_checksum_bulk() to allocate new page to simulate corruption,
  otherwise it corrupts zfs's internal buffers which do not rely
  on PageUptoDate flag

- t-f to modprobe zfs module if osd-zfs is in use

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I027483162735cd0fb4be9fa8bd12c2f2dc43b44a
Reviewed-on: http://review.whamcloud.com/4141
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1934 echo: echo client to work over OFD
Mikhail Pershin [Mon, 1 Oct 2012 16:44:41 +0000 (20:44 +0400)]
LU-1934 echo: echo client to work over OFD

- echo client doesn't call obd_alloc/free_memmd() to underlaying
  device but implement own methods
- Use real lu_env in the prep/commitrw calls
- fill OBD_BRW_ASYNC flag for async writes

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Ic02d4c2ad08693cd47d8dff5e8a86812a1534763
Reviewed-on: http://review.whamcloud.com/4146
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1095 debug: conditionally print recovery status
Minh Diep [Wed, 19 Sep 2012 17:50:11 +0000 (10:50 -0700)]
LU-1095 debug: conditionally print recovery status

Avoid printing non-sense messages like this when the recovery
code hasn't run. For example, this can occur when your starting
a new target for the first time.

  Lustre: lustre-OST0001: Recovery over after 22277843:04,
  of 0 clients 0 recovered and 0 were evicted.

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: Ib4bd8783c0c17aa15052fc9b9cf78e18274b2783
Reviewed-on: http://review.whamcloud.com/4043
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
7 years agoLU-1934 ofd: implement precreate batching
Mikhail Pershin [Mon, 1 Oct 2012 17:17:48 +0000 (21:17 +0400)]
LU-1934 ofd: implement precreate batching

Bulk precreate objects in a single tx handle.  For zfs this minimizes
the chance that the creates will be spread over multiple txgs.  This
can occur when the osts are under a concurrent write workload which
can consume bulk of the available space in a txg.

For ldiskfs this change shouldn't be harmful and the default value
can be set to 1 for ldiskfs if needed to get the previous behavior.

Change-Id: I81e310a8b630a1a29f3e017f6980901d6b6436fa
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4147
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-294 Fix typo in tests/test-groups/regression
Li Wei [Sun, 8 May 2011 05:48:47 +0000 (13:48 +0800)]
LU-294 Fix typo in tests/test-groups/regression

Signed-off-by: Li Wei <liwei@whamcloud.com>
Change-Id: I91015ba4385853db08d9a79fa574e556e1fce48c
Reviewed-on: http://review.whamcloud.com/513
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: setup/shutdown qmt device
Johann Lombardi [Fri, 28 Sep 2012 11:40:49 +0000 (13:40 +0200)]
LU-1842 quota: setup/shutdown qmt device

The quota master target (QMT) is in charge of the cluster wide quota
settings. The QMT device is setup/shutdown by the MDT which then
forwards all quota requests to the QMT via a list of request/lock
handlers exported by the QMT.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I0bd800d2996f5c3b9d861d4c6fbd70dabc3b2860
Reviewed-on: http://review.whamcloud.com/4126
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: qsd lock
Niu Yawei [Sat, 29 Sep 2012 06:12:43 +0000 (02:12 -0400)]
LU-1842 quota: qsd lock

qsd lock glimpse/blocking ast, qsd lock cancel/match functions.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ief424ca77dd9538d56689410fadfd71b01a03974
Reviewed-on: http://review.whamcloud.com/4129
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: qsd entry
Niu Yawei [Fri, 28 Sep 2012 09:34:48 +0000 (05:34 -0400)]
LU-1842 quota: qsd entry

qsd entry operations.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I88c8b0658798d0b2bea62ef80c79b28badc289d2
Reviewed-on: http://review.whamcloud.com/4119
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: qsd request
Niu Yawei [Thu, 27 Sep 2012 12:44:55 +0000 (08:44 -0400)]
LU-1842 quota: qsd request

qsd request handling APIs.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Id88e0a49482e7d56af39f9aa1d3105316b6cb3dd
Reviewed-on: http://review.whamcloud.com/4109
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 osp: osp-on-ost device
Niu Yawei [Fri, 21 Sep 2012 08:54:54 +0000 (04:54 -0400)]
LU-1842 osp: osp-on-ost device

Setup osp-on-ost device for each OST to maintain the connection
to MDT0, quota slave & ofd will send quota or fid sequence request
over this connection.

We also setup one such device on MDT0 for now, it'll be used by
the quota slave on MDT0, once the DNE code landed, this device
(on MDT0) could be replaced by normal osp device.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I2229cec1fa206dcce65a4cceae032566f4c20901
Reviewed-on: http://review.whamcloud.com/4068
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2040 tests: interop test failure on mds-survey
Bob Glossman [Fri, 28 Sep 2012 15:32:18 +0000 (08:32 -0700)]
LU-2040 tests: interop test failure on mds-survey

Add a version check to skip mds-survey in interop testing.
This is known to fail and should not block testing.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4d1ef336cecf525d7df54816cb9731ec6c6edd91
Reviewed-on: http://review.whamcloud.com/4124
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
7 years agoLU-1870 utils: correctly handle mount of multipath devices
Gregoire Pichon [Mon, 10 Sep 2012 13:27:26 +0000 (15:27 +0200)]
LU-1870 utils: correctly handle mount of multipath devices

The mount.lustre command had lost the correct handling of multipath
devices. When one is specified, an entry is added into /etc/mtab
using the device name: /dev/mapper/mpathXX. This ensures unmount
works fine with such devices.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I9060be882af944967a8c4b6b1c4403ea9250ad55
Reviewed-on: http://review.whamcloud.com/3922
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1540 tests: only run test for ldiskfs MDT
Andreas Dilger [Thu, 13 Sep 2012 22:43:10 +0000 (16:43 -0600)]
LU-1540 tests: only run test for ldiskfs MDT

Skip the test for LU-1540 (NUL symlink termination) on ZFS MDT
filesystems, since e2fsck won't work there.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibba41352119f77b296e95520603b8d5527ab91b7
Reviewed-on: http://review.whamcloud.com/3987
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2057 ofd: implement jobstats and few fixes
Mikhail Pershin [Mon, 1 Oct 2012 10:42:30 +0000 (14:42 +0400)]
LU-2057 ofd: implement jobstats and few fixes

- implement jobstats for OFD like in obdfilter
- fix ofd_health_check to ignore external lu_env and use local one
  always, because obd_proc_read_health uses it without env
- do object sync only if its version is not yet committed

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Idf74ef280f9d71e3db6b45cfacb2f5303b302a77
Reviewed-on: http://review.whamcloud.com/4143
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2050 lod: set l_ost_gen to 0
Alex Zhuravlev [Mon, 1 Oct 2012 07:21:59 +0000 (11:21 +0400)]
LU-2050 lod: set l_ost_gen to 0

like LOV does.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I68a2fcda210ab34f98d64b1d1075593b0310066a
Reviewed-on: http://review.whamcloud.com/4142
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2053 crypto: Fix cfs_crypto_hash memleak
Oleg Drokin [Sat, 29 Sep 2012 20:15:40 +0000 (16:15 -0400)]
LU-2053 crypto: Fix cfs_crypto_hash memleak

Commit 353e3c2f8d9f195c0f87a16259f22b2f84de11d4 that introduced cryptoapi
support to Lustre is allocating a hdesc in cfs_crypto_hash_init and never
freeing it.
This function is called every time a checksum needs to be calculated,
so on every bulk RPC send or receive, so even though allocations are small,
they tend to still quickly consume all available memory with any significant
amount of IO.

Change-Id: Ic654c6102ea4306a81c1f7592c81cfb6e597f1f2
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4135
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
7 years agoLU-1867 tests: replay-single test_89: @@ FAIL: 4 blocks leaked
yangsheng [Sat, 29 Sep 2012 09:18:53 +0000 (17:18 +0800)]
LU-1867 tests: replay-single test_89: @@ FAIL: 4 blocks leaked

The config llog data will changed while OST remount. It may
impact 'df' output. So we should allow extra one block size
different(4k).

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I663685d55a5a31af48d541e08e9e97fed2b8e25c
Reviewed-on: http://review.whamcloud.com/4130
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1943 osd: fix lost functionality in OFD and OSD
Mikhail Pershin [Fri, 14 Sep 2012 21:39:00 +0000 (01:39 +0400)]
LU-1943 osd: fix lost functionality in OFD and OSD

- restore ofd_fmd_expire() in ofd_ping()
- set OS_STATE_READONLY in osd_statfs()
- fix readcache initialization and add lprocfs entries for it
- fix test_19 to work with OSD if OFD is used

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I357b28d8d5de1e48c55b12faa4d31ac75dbd5f66
Reviewed-on: http://review.whamcloud.com/3991
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-2022 utils: Mount MGS lacking index
James Simmons [Tue, 25 Sep 2012 16:02:42 +0000 (12:02 -0400)]
LU-2022 utils: Mount MGS lacking index

The MGT is the only device that can't be formatted with a
index but mount.lustre expects all devices to be formated
with a index. This patch allows the MGT to be the exception.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Icea4e18173e9c670302a795bd53b2769b55fc38f
Reviewed-on: http://review.whamcloud.com/4084
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Keith Mannthey <keith@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1873 test: Must set USE_OFD=yes when backend FS is zfs
Minh Diep [Tue, 11 Sep 2012 07:07:44 +0000 (00:07 -0700)]
LU-1873 test: Must set USE_OFD=yes when backend FS is zfs

We need to make sure that USE_OFD=yes when backend FS is
zfs by checking the *FSTYPE=zfs.

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: Ia8c646ea82a0ba19c0cc1b997e1c9322a4a54f01
Reviewed-on: http://review.whamcloud.com/3930
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1842 quota: remove quota code from lvfs
Johann Lombardi [Thu, 27 Sep 2012 13:55:53 +0000 (15:55 +0200)]
LU-1842 quota: remove quota code from lvfs

This patch removes the quota code from the lvfs directory.
Most of this code has already been moved to ldiskfs OSD.
This patch also removes the dt_init_quota_ctxt which isn't
used any more.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I6ab6a26ae877d5c0bebab674204f7d36f9d83078
Reviewed-on: http://review.whamcloud.com/4110
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-2044 libcfs: fix to build libcfs without pthreads
James Simmons [Fri, 28 Sep 2012 12:14:20 +0000 (08:14 -0400)]
LU-2044 libcfs: fix to build libcfs without pthreads

Currently libcfs fails to build on systems without pthreads
in the libcfs_lock code with its pseudo lock implementation.
Instead of creating a phony address pointer we create a real
static lock and test the pointer to it. The compiler is much
more happy with this approach.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia66d0972951afbfe9bc69a6375d57e6a13ae4e43
Reviewed-on: http://review.whamcloud.com/4121
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoNew tag 2.3.51 2.3.51 v2_3_51 v2_3_51_0
Oleg Drokin [Sun, 30 Sep 2012 15:36:29 +0000 (11:36 -0400)]
New tag 2.3.51

Change-Id: I9bb65dfdc670beef6a82ec6d71ef4c9bc5d0a081

7 years agoLU-1303 osd: zfs-osd should initialize local objects on MDS
Alex Zhuravlev [Sat, 29 Sep 2012 18:47:07 +0000 (22:47 +0400)]
LU-1303 osd: zfs-osd should initialize local objects on MDS

zfs-osd to call llo_local_objects_setup() to initialize local
MDS objects (like ldiskfs-osd does).

zfs-osd should not try to control naming for objects maintained
by MDT/MDT (initialized with llo_local_objects_setup()).

disable LASSERT() in MGS: with zfs backend it's not correct.

MDT does not need to access configs, so disable setup of llogs
in MDT.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I464a78372daf8baf1900f2dbb33e74ba9ffb33b1
Reviewed-on: http://review.whamcloud.com/4134
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 fld: verify support for range lookups
Alex Zhuravlev [Sat, 29 Sep 2012 20:38:18 +0000 (00:38 +0400)]
LU-1303 fld: verify support for range lookups

if the underlying storage does not support range lookups,
then FLD disables any mappings with server index different
from 0, thus disabling DNE and FID-on-OST features.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Id93b01385acfafde024dff33557656afdcd1e4b3
Reviewed-on: http://review.whamcloud.com/4133
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2034 changelog: redo changelog using OSD llog
Mikhail Pershin [Thu, 27 Sep 2012 12:16:26 +0000 (16:16 +0400)]
LU-2034 changelog: redo changelog using OSD llog

changelog to use llog over OSD

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I4b9d1ee5146ef882db6031e996a6d81b6cae870b
Reviewed-on: http://review.whamcloud.com/4120
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2047 obdfilter: init glimpse desc to NULL for ext lock
Johann Lombardi [Fri, 28 Sep 2012 20:46:12 +0000 (22:46 +0200)]
LU-2047 obdfilter: init glimpse desc to NULL for ext lock

There is no glimpse descriptor when glimpsing extent locks, we should
thus set ldlm_glimpse_work::gl_desc to NULL to allocate glimpse
request with the correct format.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I5cdedfb79631328114ef7b0bc8e035dde985b4fa
Reviewed-on: http://review.whamcloud.com/4127
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1303 mds: integration lod/osp into the stack
Alex Zhuravlev [Fri, 21 Sep 2012 19:45:32 +0000 (23:45 +0400)]
LU-1303 mds: integration lod/osp into the stack

 - config handler recognizes MDS config and replace LOV/OSC devices
   with LOD/OSP

 - the devices in MDS stack are setup from bottom to top using
   obd_connect() namely MDT->MDD->LOD->OSD, when new OSPs arrive,
   they connect to the OSD and LOD connects to every OSP). all the
   devices are referenced by exports, preventing them from early
   release.

 - once the whole configuration is processed, prepare() method of the
   top device is called, that in turn call ->prepare() down through
   the stack, etc. at this point the stack is considered ready to
   access storage, start recovery and do regular processing.

 - at umount ->ldo_process_config(LCFG_CLEANUP) is called down through
   the stack asking devices to release the resources (like objects
   referenced for internal purposes), then the series of calls to
   obd_disconnect()'s starting from the top MDT causes all the devices
   to become unreferenced and subject to release by zombie thread.

 - obd_notify() mechanism is not used by MDT/MDD/LOD/OSD, it's still
   used by OSP to be aware of current state:

      - to learn when the first connection to OST happens and let
        clients to connect to MDS, MDS uses obd_health_check() and
        caches positive result.

      - during object creation LOD uses dt_statfs() and data returned
        by OSP tells LOD whether specific OST active/inactive, space
        available, number of precreated objects, etc.

 - LOD takes care of striping: it maintains the list of available OST
   represented locally by OSP devices. on create, LOD finds suitable
   OSPs, create objects by means of OSD API and stores striping info
   into a local object. similarly LOD handles object destroy and
   UID/GID changes: load striping and execute on them.

 - llog is used by OSP as a local library, cookies are not visible out
   side of OSP. thus neither MDT nor MDD needs to care about cookies,
   prepare buffers. MDD doesn't need to send RPCs to destroy objects
   on behalf of evicted clients. all is hidden behind OSP's object
   destroy method.

 - OSP generates llog records for OST object being destroyed and, once
   corresponded local transaction is committed, send OST_DESTROY RPC.
   once OST_DESTROY is reported to be committed, corresponded llog
   record is cancelled.

 - the same logic is used for UID/GID changes.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ifb282ace94f583ffa86020b763f22922e5d0b032
Reviewed-on: http://review.whamcloud.com/4087
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1301 mgs: Symlink mntdev procfs entry from osd
Alex Zhuravlev [Thu, 9 Aug 2012 18:52:17 +0000 (22:52 +0400)]
LU-1301 mgs: Symlink mntdev procfs entry from osd

MGS won't be able to get device names directly on top of OSD.

Change-Id: Ief808f058a785cc4bacf1743ff534a81be234544
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/3672
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1981 doc: initial revision of OSD API doc
Alex Zhuravlev [Fri, 28 Sep 2012 21:43:03 +0000 (01:43 +0400)]
LU-1981 doc: initial revision of OSD API doc

this is the initial revision of the document describing
OSD API, its terms and methods.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I26459c5bd1d876a4b9a8ca778c835d3faf6a9087
Reviewed-on: http://review.whamcloud.com/4128
Reviewed-by: Ian Colle <Ian.Colle@intel.com>
Tested-by: Ian Colle <Ian.Colle@intel.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: propagate real object size on a new striping
Alex Zhuravlev [Thu, 20 Sep 2012 12:00:04 +0000 (16:00 +0400)]
LU-1303 lod: propagate real object size on a new striping

it's possible to truncate file with no striping, when create
striping. in such a case lod's object create propagate actual
size down to the stack.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Iaf4a91516892db23f11052bd1a656ee2108a7113
Reviewed-on: http://review.whamcloud.com/4059
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: QoS allocation policy
Alex Zhuravlev [Thu, 20 Sep 2012 11:56:31 +0000 (15:56 +0400)]
LU-1303 lod: QoS allocation policy

basically a copy of lov_qos.c implemented with OSD API.

the patch introduces a special lu-helpers allowing objects
with zero fid - used by LOD/OSP as fid is being created late.
this is a requirement of current orphan cleanup protocol.
probably will be replace with natural fids generated outside
of OSP/transaction at some point.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I9f41d8a770eb7b8ca04b729bbf0866e5d27fff8a
Reviewed-on: http://review.whamcloud.com/4058
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: object create to create underlying objects
Alex Zhuravlev [Thu, 20 Sep 2012 11:53:25 +0000 (15:53 +0400)]
LU-1303 lod: object create to create underlying objects

and generate LOV EA for just created striping. the policies
choosing specific OSPs to use in the striping is in the
subsequent patch.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I2675dd3e2ba4455b2b0daaf4aa7af30b910e9a75
Reviewed-on: http://review.whamcloud.com/4057
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1303 lod: directories to inherit default striping
Alex Zhuravlev [Thu, 20 Sep 2012 09:04:52 +0000 (13:04 +0400)]
LU-1303 lod: directories to inherit default striping

on creation. the patch adds sanity check on default striping
being set by an user.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I4142a2189b00158454c7525939a9c6019a5e874a
Reviewed-on: http://review.whamcloud.com/4056
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: introcude QoS structures
Alex Zhuravlev [Thu, 20 Sep 2012 07:23:17 +0000 (11:23 +0400)]
LU-1303 lod: introcude QoS structures

like LOV before LOD will be maintaining OST->OSS mapping to be
able to distribute load among OSSs.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I132382cb09b50aec1edcd667cde1851c9ee043dc
Reviewed-on: http://review.whamcloud.com/4055
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: transfer default striping from parent/fs
Alex Zhuravlev [Thu, 20 Sep 2012 06:34:28 +0000 (10:34 +0400)]
LU-1303 lod: transfer default striping from parent/fs

lod_ah_init() transfers default striping from the provided
parent if present or from filesystem defaults.
loaded/parsed default striping is cached in the parent.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I575f2c2bddbf464c1eb93953b845ed563cfd081d
Reviewed-on: http://review.whamcloud.com/4054
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1303 lod: destroy and setattr to use striping
Alex Zhuravlev [Wed, 19 Sep 2012 20:13:14 +0000 (00:13 +0400)]
LU-1303 lod: destroy and setattr to use striping

destroy and setattr methods fetch striping, instantiate
involved objects and then call corresponding methods.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I33eea2717d10eb5c34b5a324ef35a83f1430242b
Reviewed-on: http://review.whamcloud.com/4050
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 lod: object methods
Alex Zhuravlev [Wed, 19 Sep 2012 18:57:13 +0000 (22:57 +0400)]
LU-1303 lod: object methods

in this patch - just passing control down to the stack.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I0f4f0cbc2e08cf5d3db9771d88593beb53df927c
Reviewed-on: http://review.whamcloud.com/4049
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1303 lod: functions to manipulate sub-devices
Alex Zhuravlev [Wed, 19 Sep 2012 18:02:05 +0000 (22:02 +0400)]
LU-1303 lod: functions to manipulate sub-devices

lod needs to be connected directly to all OST. configuration
subsystem brings OSP for every OST in the cluster, then
LOD connects to them and use.

the patch fixes cfs_foreach_bit() so that it continues scan
from the next bit.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6a8b98ed8964fba1410552c94c408b004eae00a4
Reviewed-on: http://review.whamcloud.com/4048
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1303 tests: Support for MDS-initiated OST_DESTROYs
Li Wei [Mon, 11 Jun 2012 03:39:10 +0000 (11:39 +0800)]
LU-1303 tests: Support for MDS-initiated OST_DESTROYs

This patch makes sure the tests work with MDSs that destroy OST
objects asynchronously on behalf of clients.

Change-Id: I3d365766f1af8a305cf723546e868ff3d6b2501d
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2982
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1303 osp: add OSP lprocfs functions
Mikhail Pershin [Wed, 26 Sep 2012 06:49:26 +0000 (10:49 +0400)]
LU-1303 osp: add OSP lprocfs functions

OSP lprocfs functions

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Id85447311847c7e75ed6b0505734f5e8eb909ed5
Reviewed-on: http://review.whamcloud.com/4100
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1303 osp: OSP logging functionality
Mikhail Pershin [Mon, 24 Sep 2012 09:54:54 +0000 (13:54 +0400)]
LU-1303 osp: OSP logging functionality

OSP writes unlink and setattr llogs and tracks corresponding commits
at OST side, then cancel llog records locally

Add missed obd_device methods and import event handling

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I63ecc975e483a7f9caac9554cc15c3a4fb4a6e6d
Reviewed-on: http://review.whamcloud.com/4095
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1303 lod: methods to manage pools
Alex Zhuravlev [Wed, 19 Sep 2012 17:08:02 +0000 (21:08 +0400)]
LU-1303 lod: methods to manage pools

external components (like configuration) call these
methods to initialize and manage in-core list of
pools in lod. so that lod itself doesn't need to
implement store/load functionality.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6b8bbc55b9acf116cc8c06e756ad26b327b5d4e8
Reviewed-on: http://review.whamcloud.com/4047
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1303 lod: add lprocfs support to lod
Alex Zhuravlev [Wed, 19 Sep 2012 11:16:24 +0000 (15:16 +0400)]
LU-1303 lod: add lprocfs support to lod

and initialize it at device startup time

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Id8f5e40f18685f3ffbf50405932754f594d7e60f
Reviewed-on: http://review.whamcloud.com/4046
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1867 tests: disable replay-single/89
Alex Zhuravlev [Wed, 26 Sep 2012 16:35:50 +0000 (20:35 +0400)]
LU-1867 tests: disable replay-single/89

as it's been coloring tests red quite often and seem to be very
localized issued I suggest to disable it till the root cause or
the test is fixed properly.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I85feb214d6d23077cfdde47a6a439b806c5d0b6e
Reviewed-on: http://review.whamcloud.com/4105
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 ldlm: add support for quota LVB
Johann Lombardi [Wed, 26 Sep 2012 21:28:21 +0000 (23:28 +0200)]
LU-1842 ldlm: add support for quota LVB

This patch defines new LVB types for quota.
A new request buffer is also added to glimpse AST to allow the server
to pass additional information about the callback to the client.
This additional buffer will only be used in conjunction with quota
locks for the time being.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: If5842bfff79d24e7d11ef33c47edb96fe6f11e76
Reviewed-on: http://review.whamcloud.com/4107
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: new sanity-quota tests
Niu Yawei [Thu, 20 Sep 2012 08:41:37 +0000 (04:41 -0400)]
LU-1842 quota: new sanity-quota tests

Added new sanity-quota.sh for the new quota architecture, meanwhile,
the old sanity-quota.sh is reserved for the interoprability with
old server.

Removed the quotacheck tests in conf-sanity, since quotacheck is
deprecated in the new quota architecture.

Test-Parameters: envdefinitions=SLOW=yes testlist=sanity-quota

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ia3e3248a167e9f312a782aefea7af0aeced52220
Reviewed-on: http://review.whamcloud.com/4040
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: lquota entry
Niu Yawei [Wed, 26 Sep 2012 07:34:36 +0000 (03:34 -0400)]
LU-1842 quota: lquota entry

quota entry management.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I57e6a736680b25c737859be2cd9434997b2ccdbf
Reviewed-on: http://review.whamcloud.com/4101
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1302 mgs: mgs uses llog over OSD
Mikhail Pershin [Tue, 25 Sep 2012 08:50:41 +0000 (12:50 +0400)]
LU-1302 mgs: mgs uses llog over OSD

- llog over OSD in MGS. Switch llog_test to work over OSD too.
- local_oid_store_init() don't create names for llog because all
  llogs have names in O/d/n
- Enable all llog tests functionality

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I60cc5ec7094a2640d7fc0a0d87519c56b1383557
Reviewed-on: http://review.whamcloud.com/4104
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1302 llog: llog ioctl to use env
Mikhail Pershin [Mon, 24 Sep 2012 21:08:57 +0000 (01:08 +0400)]
LU-1302 llog: llog ioctl to use env

Update llog_ioctl.c to use lu_env

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I8f5476cccacae2d4922ac9af87d334929c0eba6c
Reviewed-on: http://review.whamcloud.com/4094
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-1302 llog: llog over OSD primitives
Mikhail Pershin [Sun, 23 Sep 2012 19:22:26 +0000 (23:22 +0400)]
LU-1302 llog: llog over OSD primitives

set of low-level functions implementing llog functionality over
OSD device

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I6b623a821d39ecca90c27a7a84b969aada11c0cd
Reviewed-on: http://review.whamcloud.com/4093
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
7 years agoLU-1981 build: External OSD support in configure
Alex Zhuravlev [Thu, 17 Feb 2011 08:23:18 +0000 (11:23 +0300)]
LU-1981 build: External OSD support in configure

With configure option "--with-osd=<dir>" one can get additional
(external to main tree) directory to be built with main tree.

Change-Id: Ibe1c1d136d9c8bf8245344791f0f51933266018a
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4099
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1667 ui: fix inaccurate warning message.
Richard Henwood [Wed, 25 Jul 2012 20:07:53 +0000 (16:07 -0400)]
LU-1667 ui: fix inaccurate warning message.

Trivial fix to make stripe size message accurate. Message is
generated if a user issues a 'lfs setstripe' command with a
size 4G or larger. The corrected message now reads:
"stripe size 4G or larger is not currently supported"

Signed-off-by: Richard Henwood <rhenwood@whamcloud.com>
Change-Id: I6978e84aee547f40dc20f9b0bfa243d4960e084d
Reviewed-on: http://review.whamcloud.com/3464
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Cliff White <cliffw@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1842 quota: remove quota code
Johann Lombardi [Mon, 24 Sep 2012 13:03:21 +0000 (15:03 +0200)]
LU-1842 quota: remove quota code

This patch removes the quota code (from quota directory, but also
from mdd & obdfilter) which is going to be replaced by the new
implementation.
This patch also modifies the mdt to use the new space accounting
scheme as already used by the ofd layer.
sanity-quota.sh now also uses ofd by default and only runs the space
accounting tests until quota enforcement support is landed.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I85cbbf6058859e5d19f80aa128f45b6d41c37d92
Reviewed-on: http://review.whamcloud.com/4098
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2007 ldlm: reconnect can be racy
Alex Zhuravlev [Sat, 4 Aug 2012 08:14:31 +0000 (12:14 +0400)]
LU-2007 ldlm: reconnect can be racy

exp_connecting should be checked under spinlock

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I271bc349ae1ad63a4acd1042f30ac7aa7176f105
Reviewed-on: http://review.whamcloud.com/4075
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-2024 tests: skip OI scrub tests until the tag 2.2.90
Fan Yong [Tue, 25 Sep 2012 08:46:19 +0000 (16:46 +0800)]
LU-2024 tests: skip OI scrub tests until the tag 2.2.90

Do not trigger OI scrub related tests on old releases.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I9a4289b03ee8e222329b69891bb901fff9fbc86d
Reviewed-on: http://review.whamcloud.com/4089
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1303 lod: introduce lod device
Alex Zhuravlev [Wed, 19 Sep 2012 11:16:24 +0000 (15:16 +0400)]
LU-1303 lod: introduce lod device

this is the first patch in the series landing lod.
LOD stands for LOgical Device, it's purpose is to
hide striping for other layers like MDD.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib33b3e8b2de6e816397a17af65a0bffab912960b
Reviewed-on: http://review.whamcloud.com/4045
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
7 years agoLU-1302 llog: simplify llog_setup functionality
Mikhail Pershin [Fri, 31 Aug 2012 05:18:35 +0000 (09:18 +0400)]
LU-1302 llog: simplify llog_setup functionality

llog catalog used llog_obd_origin_setup as llog_setup() function
It does actually setup, open and llog processing inside that
makes it different from plain llog and breaks new llog API

Patch does the following:

- llog operations are needed for both ctxt and handler, in ctxt
  there are disk and network operations possible, per-handle there
  are plain and catalog llogs. in-handle operations are taken from
  ctxt at first then updated for catalogs by the caller
- rework llog_setup, keep only ctxt allocation in it, remove unused
  parameters, remove llog_setup_named as name was never used

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I8641e697898a1a57bbc500b649f065ba3f2d41b0
Reviewed-on: http://review.whamcloud.com/3949
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: migrate old quota admin files
Niu Yawei [Mon, 24 Sep 2012 08:34:02 +0000 (04:34 -0400)]
LU-1842 quota: migrate old quota admin files

Migrate old admin quota files into new global indexes when mount
with an old device.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I9e834eabf83f0da273b4ba3309adf71f55eae758
Reviewed-on: http://review.whamcloud.com/4077
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1876 ldlm: extend ldlm_valblock_ops{}
Jinshan Xiong [Mon, 17 Sep 2012 16:41:20 +0000 (09:41 -0700)]
LU-1876 ldlm: extend ldlm_valblock_ops{}

To extend the usage of RQF_DLM_LVB for quota, layout lock and nano
seconds, two extra callbacks will be added:
 - lvbo_size: determine the size of lvb data
 - lvbo_fill: fill in RPC buffer by lvb data

Also framework of mdt_lvb is added for future extension.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I2b3022525f8fbef72549a25766aa73d94b8b997c
Reviewed-on: http://review.whamcloud.com/3960
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1303 osd: support for forced sync in procfs
Alex Zhuravlev [Mon, 24 Sep 2012 15:00:48 +0000 (19:00 +0400)]
LU-1303 osd: support for forced sync in procfs

will be used by t-f to speedup file/object removals, etc.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ic9d933b627eed0a47438b66890933ddc39cc8c8b
Reviewed-on: http://review.whamcloud.com/4083
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
7 years agoLU-1842 quota: t-f changes for new quota
Niu Yawei [Tue, 18 Sep 2012 15:00:12 +0000 (11:00 -0400)]
LU-1842 quota: t-f changes for new quota

Added new quota functions in the t-f according to new quota
architecutre, at the same time, old functions are kept for
the interoprability with old server.

two minor defects fixed:
- in facet_up(), add '-x' option to grep for exact matching,
  otherwise this function will not work in single node test;
- missing 'shift' in wait_update_facet();

Test-Parameters: envdefinitions=ENABLE_QUOTA=yes

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I86915976f53af8620679ab267e04beea18dd5cf3
Reviewed-on: http://review.whamcloud.com/4031
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1842 quota: add quota disk operations
Johann Lombardi [Tue, 18 Sep 2012 20:18:14 +0000 (22:18 +0200)]
LU-1842 quota: add quota disk operations

Add low level quota disk operations that will be used by both the
QMT (quota master) and QSD (quota slave) to manipulate on-disk
structures.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I783bf9519cf37fdfa35a04cff8207825a7ab1b49
Reviewed-on: http://review.whamcloud.com/4042
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
7 years agoLU-1753 test: use large resend count for 118i in sanity
Hongchao Zhang [Fri, 21 Sep 2012 23:16:33 +0000 (07:16 +0800)]
LU-1753 test: use large resend count for 118i in sanity

in sanity.sh, the subtest 118i will fail if the resend count
of OSC is small for 'SLOW=no' case (it's 4, default is 10).

Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Change-Id: I3c49628ed259e7adbff4a100d482e693502bf99d
Reviewed-on: http://review.whamcloud.com/4071
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1304 mdt: get attributes with explicit calls
Alex Zhuravlev [Fri, 17 Aug 2012 04:47:00 +0000 (08:47 +0400)]
LU-1304 mdt: get attributes with explicit calls

to mdd, so that mdd doesn't need to care of attribute packs

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0c2c636e4751f599c5a580526c87ff068a5409a5
Reviewed-on: http://review.whamcloud.com/3782
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
7 years agoLU-1999 tests: check LW client eviction in logs
Johann Lombardi [Fri, 21 Sep 2012 07:54:17 +0000 (09:54 +0200)]
LU-1999 tests: check LW client eviction in logs

A lightweight client can reconnect rather quickly after a server
restart (they are allowed to reconnect during recovery), which
confuses wait_client_evicted().
A more reliable check is to search for eviction message in the log.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: Idb5fda48c7d73c465c8d69c18f2040d5fa08e3ba
Reviewed-on: http://review.whamcloud.com/4069
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
7 years agoLU-1303 osp: osp object operations
Mikhail Pershin [Tue, 18 Sep 2012 13:07:46 +0000 (17:07 +0400)]
LU-1303 osp: osp object operations

add object operations to OSP

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I9e460efe2cbfed984cb57e5e77992127b9084556
Reviewed-on: http://review.whamcloud.com/4030
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>