Whamcloud - gitweb
fs/lustre-release.git
11 years agoLU-1994 kernel: kernel 3.6 changes i_dentry/d_alias to hlist
yang sheng [Wed, 5 Dec 2012 19:57:22 +0000 (03:57 +0800)]
LU-1994 kernel: kernel 3.6 changes i_dentry/d_alias to hlist

kernel 3.6 changes i_dentry/d_alias from list to hlist.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: If05f01fe6fe34225ec24b90bf85eb9e9e80f44f3
Reviewed-on: http://review.whamcloud.com/4385
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-1994 kernel: 3.5 kernel encode_fh passes in parent inode
Peng Tao [Thu, 23 Aug 2012 05:55:46 +0000 (13:55 +0800)]
LU-1994 kernel: 3.5 kernel encode_fh passes in parent inode

Changed by upsteam commit b0b0382b.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I6d7223b934bfe4151371744ad72bdab6b170c700
Reviewed-on: http://review.whamcloud.com/3757
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
11 years agoLU-1994 llite: kernel 3.5 renames end_writeback to clear_inode
yang sheng [Wed, 5 Dec 2012 19:38:15 +0000 (03:38 +0800)]
LU-1994 llite: kernel 3.5 renames end_writeback to clear_inode

kernel commit dbd5768f8 renames end_writeback to clear_inode,
which used to exist for ~2.6.36 kernels.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: I2435e820092085c0fb55539818f4bad9e48ff386
Reviewed-on: http://review.whamcloud.com/3756
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-2329 quota: wait longer in test_7c
Niu Yawei [Tue, 4 Dec 2012 09:25:17 +0000 (04:25 -0500)]
LU-2329 quota: wait longer in test_7c

In test_7c of s-q, we'd wait longer than usual to make sure
the reintegration triggered by quota wb thread is done.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I977b3a03a0e57424a531df13fed5550a753f41c2
Reviewed-on: http://review.whamcloud.com/4739
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
11 years agoLU-1741 test: fix conf_sanity test_18 test case
Bobi Jam [Tue, 14 Aug 2012 11:35:35 +0000 (19:35 +0800)]
LU-1741 test: fix conf_sanity test_18 test case

* We need always check block device's size bigger than its minimum
  requirement.

* Correct code indentation for formatall().

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I884287f43a2d1499a5be986e7d57d8d1a3827a24
Reviewed-on: http://review.whamcloud.com/3632
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2302 scripts: prevent lfs_migrate data disclosure
Daniel Kobras [Wed, 28 Nov 2012 17:57:34 +0000 (09:57 -0800)]
LU-2302 scripts: prevent lfs_migrate data disclosure

Tighten umask to make sure temporary rsync files are created
with strict permissions that don't grant access to unprivileged
users.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: Ie662edfd615a24ce08cbddb9347f197de41d4d27
Reviewed-on: http://review.whamcloud.com/4699
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
11 years agoLU-2302 scripts: null-terminated file lists in lfs_migrate
Daniel Kobras [Wed, 28 Nov 2012 18:12:24 +0000 (10:12 -0800)]
LU-2302 scripts: null-terminated file lists in lfs_migrate

New option -0 allows to use a null-terminated list of file names
on stdin. Changed internal lfs find to use null-terminates lists.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: Iccd928a766968a7513833035684d41a1060c4a4d
Reviewed-on: http://review.whamcloud.com/4700
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
11 years agoLU-2302 scripts: fix lfs_migrate with non-English locale
Daniel Kobras [Wed, 28 Nov 2012 17:47:18 +0000 (09:47 -0800)]
LU-2302 scripts: fix lfs_migrate with non-English locale

Parsing of stat output assumes an English locale. Make sure
this is always the case.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: I66acd6eb49cac749db701b97a5a375dc3b141a01
Reviewed-on: http://review.whamcloud.com/4697
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
11 years agoLU-2371 ptlrpc: get new xid for resend on EINPROGRESS
Johann Lombardi [Fri, 30 Nov 2012 14:16:41 +0000 (15:16 +0100)]
LU-2371 ptlrpc: get new xid for resend on EINPROGRESS

Allocate new xid for ptlrpc resend on EINPROGRESS to avoid MDT's reply
recontruction code which will cause EINPROGRESS to always been replied
on subsequent resent.
Also fix non-intent creation path to set rq_no_retry_einprogress.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Id423142e4440952777b74a834e2b4fa38c338280
Reviewed-on: http://review.whamcloud.com/4719
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
11 years agoLU-2388 statahead: don't statahead if it's stopped
Lai Siyao [Fri, 30 Nov 2012 07:45:55 +0000 (15:45 +0800)]
LU-2388 statahead: don't statahead if it's stopped

Statahead may be stopped but lli->lli_sai is not NULL(inflight
statahead RPCs), so check lli->lli_opendir_key before accessing
statahead data.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I4bcd4da0dc2913697473bf85ff6e502df43ce5a2
Reviewed-on: http://review.whamcloud.com/4718
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2361 quota: keep slave's glb idx consistent with master's
Johann Lombardi [Tue, 20 Nov 2012 15:55:28 +0000 (16:55 +0100)]
LU-2361 quota: keep slave's glb idx consistent with master's

Issue glimpse callback and bump global version each time quota
settings are modified in the global index. This way, the global
recovered if required.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I91c11b8bce68c8f96289588e612295ece7664c3f
Reviewed-on: http://review.whamcloud.com/4632
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2346 quota: set default grace time
Niu Yawei [Fri, 16 Nov 2012 13:45:48 +0000 (08:45 -0500)]
LU-2346 quota: set default grace time

Set default grace time as MAX_IQ_TIME or MAX_DQ_TIME.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I8f57e5483cdf0948532d3185fdf43c6efec73233
Reviewed-on: http://review.whamcloud.com/4598
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
11 years agoLU-2358 procfs: Implement /proc/fs/lustre/mgs/MGS/fstype as symlink
John L. Hammond [Tue, 20 Nov 2012 17:33:04 +0000 (11:33 -0600)]
LU-2358 procfs: Implement /proc/fs/lustre/mgs/MGS/fstype as symlink

Add symlinks:
  /proc/fs/lustre/mgs/MGS/osd    -> ../../osd-ldiskfs/lustre-MDT0000
  /proc/fs/lustre/mgs/MGS/fstype -> osd/fstype
  /proc/fs/lustre/mgs/MGS/mntdev -> osd/mntdev
Remove unused lprocfs_rd_fstype().

Change-Id: Ic5cce7cf31962c7d6ea97c5d3117fba81539cb68
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/4635
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-1287 mountconf: write failover nid config correctly
Lai Siyao [Thu, 14 Jun 2012 10:21:00 +0000 (18:21 +0800)]
LU-1287 mountconf: write failover nid config correctly

Once OST starts before MDT, MDT needs steal OSC config from client
config, but both target nids and failover nids are stored as the
same type of config LCFG_ADD_UUID. To handle this correctly, all
LCFG_ADD_UUID before LCFG_SETUP is treated as target nids, while
after as failover nids. NB this implies that records should be
written into config file in fixed order.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I42d014911edad1bc53bb1a66def160ee0da3e670
Reviewed-on: http://review.whamcloud.com/3107
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1632 fid: remove fid_delete in delete_inode procedure
wangdi [Wed, 18 Jul 2012 09:17:27 +0000 (02:17 -0700)]
LU-1632 fid: remove fid_delete in delete_inode procedure

Remove fid_delete in delete_inode procedure, which is
completely obsolete in 2.x infrastructure , especially
when lmv layer is not being created during upgrade,
because it will cause the client fid space to be zero
out whenever an inode is being deleted.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I4002945603790f8dda3e33847160c13afb7b41a4
Reviewed-on: http://review.whamcloud.com/3434
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2363 lod: Fix statfs entries in lod procfs
John L. Hammond [Mon, 26 Nov 2012 21:46:07 +0000 (15:46 -0600)]
LU-2363 lod: Fix statfs entries in lod procfs

Factor toplevel lod procfs out of lod_pools_{init,fini}.  Support
lod statfs entries using osd methods.

Change-Id: I27f8c1441079ad455a03f16abb05491fd655d2e3
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/4637
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <kemannthey@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-1756 kernel: cleanup lustre_compat25.h
Peng Tao [Thu, 16 Aug 2012 07:59:21 +0000 (15:59 +0800)]
LU-1756 kernel: cleanup lustre_compat25.h

1. unused functions:
   KIOBUF_GET_BLOCKS()
2. container_of() always defined
3. place linux/mpage.h at where it is needed

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I3369572fbd8aef2a5ad4881ec3f0f67c11772881
Reviewed-on: http://review.whamcloud.com/3687
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liu Xuezhao <xuezhao.liu@emc.com>
11 years agoLU-1484 kernel: fix build error with 2.6.18 kernel
Peng Tao [Thu, 15 Nov 2012 11:17:20 +0000 (19:17 +0800)]
LU-1484 kernel: fix build error with 2.6.18 kernel

When crossing building 2.6.18 kernel, I always got following
errors:
lustre-release/libcfs/include/libcfs/params_tree.h:107:2:
   error: #error proc_dir_entry->deleted is conflicted
   with proc_dir_entry->pde_users

It is because after commit 76bf16d1, HAVE_PROCFS_USERS and
HAVE_PROCFS_DELETED can both be true on old kernels with
backported code.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I2b420f401f221c619b9643acee2ff54057af0fc9
Reviewed-on: http://review.whamcloud.com/4589
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
11 years agoLU-1887 ptlrpc: grant shrink rpc format is special
Lai Siyao [Mon, 15 Oct 2012 07:02:19 +0000 (00:02 -0700)]
LU-1887 ptlrpc: grant shrink rpc format is special

Grant shrink rpc is packed in OST_SET_INFO request, but its format is
different from others.

Change-Id: I8b3398d534fbae683722f942403e031c68023b16
Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4277
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1924 build: configure can not find libgssapi_krb5.so
Thomas Stibor [Wed, 24 Oct 2012 10:20:31 +0000 (12:20 +0200)]
LU-1924 build: configure can not find libgssapi_krb5.so

./configure --enable-gss cannot find on Debian amd64/x86_64
wheezy the file libgssapi_krb5.so although it
exists in the directory /usr/lib/x86_x64-linux-gnu.
Probably some other amd64/x86_64 Linux distributions
are effected as well.

Signed-off-by: Thomas Stibor <thomas@stibor.net>
Change-Id: Ife12e11224db4ef64adbcedb31cc1c07cf1c9b2e
Reviewed-on: http://review.whamcloud.com/4378
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-398 libcfs: Add libcfs heap, a binary heap implementation
Nikitas Angelinas [Tue, 30 Oct 2012 15:46:39 +0000 (15:46 +0000)]
LU-398 libcfs: Add libcfs heap, a binary heap implementation

The heap can be used to build and maintain sorted arrays and lists,
and prioritized queues of large numbers of elements, with minimal
insertion and removal time. The first user for the data structure are
NRS policies, which use it to maintain prioritized queues of RPCs at
PTLRPC services.

There is no 'search' operation, but the data type aims to be useful
in cases where performing searches on the data set is not required,
and instead the lowest priority element is usually removed from the
data set for consumption.

Original-author: Eric Barton <eeb@whamcloud.com>
Original-author: Liang Zhen <liang@whamcloud.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: Ifd293d0a8201e45bd6030d7a16ce0be691f79cb9
Oracle-bug-id: b=13634
Xyratex-bug-id: MRP-73
Reviewed-on: http://review.whamcloud.com/4412
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2016 mdd: add layout swap between 2 objects
jcl [Thu, 4 Oct 2012 19:51:49 +0000 (21:51 +0200)]
LU-2016 mdd: add layout swap between 2 objects

This patch add a new method in mdd to swap the layouts between
2 lustre objects.
The 2 objects have to be of the same type.

Change-Id: I26dfef2745eac67168aeceac196453c5126148fd
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-on: http://review.whamcloud.com/4189
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-744 obdclass: revise cl_page refcount
Jinshan Xiong [Fri, 7 Dec 2012 18:06:49 +0000 (10:06 -0800)]
LU-744 obdclass: revise cl_page refcount

By holding a refcount for radix tree, we can remove a couple of
spin lock and atomic operation for cl_page.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I9440d6b86a63d00ce716cb11676101acca07d535
Reviewed-on: http://review.whamcloud.com/4617
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2194 tests: Wait for reconnect in recovery-small/19
Nathaniel Clark [Wed, 21 Nov 2012 20:09:54 +0000 (15:09 -0500)]
LU-2194 tests: Wait for reconnect in recovery-small/19

Fully wait for clients to reconnect before checking if evicted.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I332d9b43344eb2c7c883fc28adfbbfb4eeb8a0d0
Reviewed-on: http://review.whamcloud.com/4652
Tested-by: Hudson
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2378 lma: move HSM & SOM attributes to dedicated xattrs
Johann Lombardi [Fri, 23 Nov 2012 14:23:29 +0000 (15:23 +0100)]
LU-2378 lma: move HSM & SOM attributes to dedicated xattrs

Reduce LMA to its minimum and move HSM & SOM attributes to dedicated
on-disk extented attributes. This is more consistent with the MDS
layering (HSM, LMA & SOM are managed in different layers) and avoids
locking + read/modify/write in many cases.

As far as interoperability is concerned, the old code works just fine
if it reads a smaller LMA size and the new code retries with a
larger buffer on ERANGE error.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Id4ee465ea69820229f7214d800821864330a8aec
Reviewed-on: http://review.whamcloud.com/4746
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-2156 debug: quiet overly verbose console errors
Andreas Dilger [Wed, 17 Oct 2012 18:42:17 +0000 (12:42 -0600)]
LU-2156 debug: quiet overly verbose console errors

A number of error messages are printed on the console that do not
represent error conditions, or print the message in an overly
confusing manner.  One in an ongoing series of patches to clean
up the console errors.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie830cfb49e390ca4da1f4b9ed7187d421e74f12f
Reviewed-on: http://review.whamcloud.com/4286
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Hudson
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1403 ucred: ucred cleanup
Vitaly Fertman [Sun, 9 Dec 2012 21:26:51 +0000 (01:26 +0400)]
LU-1403 ucred: ucred cleanup

Add ucred_check() and ucred_assert() methods to ensure ucred is
installed and/or initialised everywhere.

As echo client may be sitting on mdd it needs to initialize md_ucred
too. As echo client may be sitting on osc without mdd module loaded,
make md_ucred a common key, move it to lu_object.

Reviewed-by: Alexander Zarochentsev <Alexander_Zarochentsev@xyratex.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Xyratex-bug-id: MRP-484
Change-Id: I9c4aaa31148ff10e4a609393a0cf018cdf731d9c
Reviewed-on: http://review.whamcloud.com/2733
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-744 obdclass: revise percpu stats of lprocfs_stats
Jinshan Xiong [Mon, 5 Nov 2012 21:18:52 +0000 (13:18 -0800)]
LU-744 obdclass: revise percpu stats of lprocfs_stats

Atomic doesn't scale well and there is an atomic counter to remember
if updating stats is on going. This will introduce performance
problem if the percpu stats is used in a highly contended code.

I'll remove that atomic counter in this patch, this is okay as percpu
counter is not accurate essentially.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I5c7a094b78ab53798266d48d56f3a16e7c7d436f
Reviewed-on: http://review.whamcloud.com/4472
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
11 years agoLU-744 obdclass: revise stats for cl_object cache
Jinshan Xiong [Fri, 7 Dec 2012 18:01:53 +0000 (10:01 -0800)]
LU-744 obdclass: revise stats for cl_object cache

It turns out that atomic doesn't scale well for highly contented
cl_page stats. Enable it only when CONFIG_DEBUG_PAGESTATE_TRACKING
is turned on.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I0163f65557135759f100fddde70c7e3ee2e45aed
Reviewed-on: http://review.whamcloud.com/4519
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
11 years agoLU-797 tests: improve test_23b of ost-pools.sh
Niu Yawei [Tue, 6 Nov 2012 04:02:23 +0000 (23:02 -0500)]
LU-797 tests: improve test_23b of ost-pools.sh

- fail the test once the written space is greater than the
  available space.
- use it's own maxfree limit, and increase the limit a lot.
- terminate the test when the write speed is too slow.

Test-Parameters: envdefinitions=SLOW=yes testlist=ost-pools
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ib7deca89ebd7267fa7be0a51ceac01fe3cb2df65
Reviewed-on: http://review.whamcloud.com/4474
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
11 years agoLU-1229 tests: fix set module parameters from modprobe.conf
James Simmons [Thu, 6 Dec 2012 16:08:29 +0000 (11:08 -0500)]
LU-1229 tests: fix set module parameters from modprobe.conf

Currently the test framework can't process multiple options
on separate lines for the same module from the module config
file. Once those parameters are collected they are supposed
to be stored in the MODOPT_X shell variable to be used later
to reload the modules but this was not being done. This patch
address both of these issues.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I08a6a562009c44e2a00ee5d489815164f638954d
Reviewed-on: http://review.whamcloud.com/2342
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1538 tests: allow SLOW from environment for auster
Andreas Dilger [Fri, 19 Oct 2012 23:26:56 +0000 (17:26 -0600)]
LU-1538 tests: allow SLOW from environment for auster

Allow "SLOW=yes" to be passed from the environment when running
auster via acceptance-small.sh.  Otherwise, it overrides the value
passed from the environment.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I646adad77ade050c80e0da174dedb25d23500c1e
Reviewed-on: http://review.whamcloud.com/4707
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2406 tests: remove USE_OFD from test-framework
Andreas Dilger [Fri, 30 Nov 2012 23:45:48 +0000 (16:45 -0700)]
LU-2406 tests: remove USE_OFD from test-framework

Remove the old USE_OFD from test-framework.sh, since this is no
longer needed for master, but it confuses interop testing when
run with b2_3 (where it hits LU-1883 and causes random failures).

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I417ac3eb5b2cbb8b4cebfc5d7f3cd2cab1b0974d
Reviewed-on: http://review.whamcloud.com/4727
Tested-by: Hudson
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1861 osd: set proper type for compatibility files
Mikhail Pershin [Sun, 25 Nov 2012 19:50:02 +0000 (23:50 +0400)]
LU-1861 osd: set proper type for compatibility files

The compatibility code in osd-ldiskfs uses osd_compat_add_entry()
to add dir entry for system files, but doesn't type in directory
entry properly so e2fsch complains about that.

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I8b1529b3303fba311432f4aa79d71afb6dc992d7
Reviewed-on: http://review.whamcloud.com/4669
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-2333 utils: mount.lustre to ignore fsname
Alex Zhuravlev [Thu, 15 Nov 2012 17:39:23 +0000 (21:39 +0400)]
LU-2333 utils: mount.lustre to ignore fsname

if MGS is the only service on a given device, then it's OK
to have empty fsname: fsname does not apply to MGS generally.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ie2269c46280b1ef2504fc73a61cc1d28ff2261fc
Reviewed-on: http://review.whamcloud.com/4592
Tested-by: Hudson
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-1279 revert: Silence modprobe ptlrpc output in mount.lustre
Andreas Dilger [Wed, 5 Dec 2012 22:08:21 +0000 (15:08 -0700)]
LU-1279 revert: Silence modprobe ptlrpc output in mount.lustre

This reverts commit 20e5b84f086a5e45e2692a239e2f74a37d580774.
This reverts commit d8d9b78a5c08eb1d938ab9e3bdaf7f756bfbb5ec.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2ffdf18227c3a423096f765608592ccf8391cab0
Reviewed-on: http://review.whamcloud.com/4750
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1187 tests: Add DNE tests to conf sanity.
wangdi [Sun, 18 Nov 2012 21:24:20 +0000 (13:24 -0800)]
LU-1187 tests: Add DNE tests to conf sanity.

1. 32c: upgrade 2.1/1.8 to DNE.
2. 70/71: start targets in mixed order for DNE.
3. Add start_mds to start all of MDTs during the test.
4. Fix writeconf to erase the config for all targets.

Change-Id: I62e9173d53b96082059b1faf7e0455d0c07eb967
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4367
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1187 tests: add DNE test cases in replay-single
wangdi [Sun, 18 Nov 2012 21:21:05 +0000 (13:21 -0800)]
LU-1187 tests: add DNE test cases in replay-single

Add DNE test cases 80(remote create) and 81(remote unlink)
in replay-single.

Change-Id: I7609139357de4bfc7a08ca30e24faa8bc11b096d
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4362
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
11 years agoLU-279 tests: Ensure lgetxattr is consistent
Prakash Surya [Fri, 13 May 2011 15:48:36 +0000 (08:48 -0700)]
LU-279 tests: Ensure lgetxattr is consistent

* Added the lgetxattr_size_check.c program to compare the size returned
  by lgetxattr() when passed zero size buffers and non zero size buffers
* Sanity test 17l was added to make use of this new test program and
  ensure that the values returned by lgetxattr remain consistent

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: Id46dbe888d42b058e02ec32b6d1612705d7f6349
Reviewed-on: http://review.whamcloud.com/547
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-1710 lvb: variable sized LVB support
Jinshan Xiong [Thu, 6 Dec 2012 17:03:27 +0000 (09:03 -0800)]
LU-1710 lvb: variable sized LVB support

Originally, the LVB on wire is fixed sized only for transferring
object {m/a/c}time and block/size information.

But with introducing layout lock, new quota and nanosecond features
in the near futures, such fixed sized on wire LVB structure cannot
match the requirements from these new features.

So change on wire protocol to allow variable sized LVB transerred
between nodes.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ifcd2d303495d2b3d1c09c38af823d246fff196fa
Reviewed-on: http://review.whamcloud.com/3965
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1187 tests: Add DNE tests cases in replay-dual
wangdi [Sun, 18 Nov 2012 21:18:04 +0000 (13:18 -0800)]
LU-1187 tests: Add DNE tests cases in replay-dual

Add 22(remote creation) and 23(remote unlink) in
replay-dual for DNE.

Change-Id: Ife1e6e67f0ffbf7fefab7c605c28381dc1c3ed88
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4361
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
11 years agoLU-1187 tests: define MGSDEV in right way.
wangdi [Sat, 8 Dec 2012 22:23:23 +0000 (14:23 -0800)]
LU-1187 tests: define MGSDEV in right way.

For Combined mgs/mds, MGSDEV definition should be same for
LDISKFS and ZFS.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I107990a97e7daa0d72be874125ced65f3ed6c337
Reviewed-on: http://review.whamcloud.com/4774
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1994 kernel: 3.4 kmap_atomic has one argument
yangsheng [Wed, 5 Dec 2012 19:50:39 +0000 (03:50 +0800)]
LU-1994 kernel: 3.4 kmap_atomic has one argument

kernel 3.4 kmap_atomic removes second argument by commit 1ec9c5ddc17.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: I6471291fd881c9e76833654602fa4b18fd1f1028
Reviewed-on: http://review.whamcloud.com/4384
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-2449 Postpone fixing of . and .. hadling in ZFS again
Oleg Drokin [Sun, 9 Dec 2012 03:24:16 +0000 (22:24 -0500)]
LU-2449 Postpone fixing of . and .. hadling in ZFS again

 I totally forgot about it and so it broke again with the new tag.

Change-Id: I1e116ab950a615e10e6db3d23ac5fe1b3484f24e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
11 years agoNew tag 2.3.57 2.3.57 v2_3_57 v2_3_57_0
Oleg Drokin [Sun, 9 Dec 2012 03:02:05 +0000 (22:02 -0500)]
New tag 2.3.57

Change-Id: I79f6a73e4e711646f7b6b1551fdc1f460aeaae1c

11 years agoLU-2284 tests: clear limits before testing
Niu Yawei [Fri, 7 Dec 2012 08:08:50 +0000 (03:08 -0500)]
LU-2284 tests: clear limits before testing

The quota limits left over from other tests which has quota enabled
could confuse the sanity-quota, so we'd clear all the before testing.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ie24e6891d920a0c7fa6ffc29ee8ff9400b8636af
Reviewed-on: http://review.whamcloud.com/4766
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-2304 lov: check enq flags when matching top lock
Jinshan Xiong [Wed, 21 Nov 2012 18:10:42 +0000 (10:10 -0800)]
LU-2304 lov: check enq flags when matching top lock

cld_enq_flags must be checked when matching top lock, otherwise
if a sublock is missing then it will be reenqueued with wrong enq
flags.

For LU-2304, a previous truncate to zero lock(with CEF_DISCARD_DATA)
flags was matched by not-to-zero truncate and then wrongly reenqueued
with discard flag. This caused dirty pages to be discarded on the
other clients.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I290df2fdb9453c2fd7c8c3d3e7289e60bae9b095
Reviewed-on: http://review.whamcloud.com/4651
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
11 years agoLU-797 tests: process lfs df output properly
Niu Yawei [Thu, 29 Nov 2012 09:59:40 +0000 (04:59 -0500)]
LU-797 tests: process lfs df output properly

There is a space in the "filesystem summary" of lfs df output,
which could cause trouble for the parsing script, test_23b of
ost-pool.sh is an example.

We introduced lfs_df() function to remove this space internally,
this will allow fixing the "lfs df" output line in the future.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I524d42a5e792a1b29db52166e92624bf42312c3f
Reviewed-on: http://review.whamcloud.com/4705
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1337 llite: kernel 3.4 converts d_alloc_root to d_make_root
Liu Xuezhao [Tue, 30 Oct 2012 09:14:02 +0000 (17:14 +0800)]
LU-1337 llite: kernel 3.4 converts d_alloc_root to d_make_root

kernel 3.4 converts d_alloc_root to d_make_root.
(kernel commit 32991ab305ace7017c62f8eecbe5eb36dc32e13b)

Add LC_HAVE_D_MAKE_ROOT/HAVE_D_MAKE_ROOT to ckeck it.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: I7ff1f0eff25495b655e650997016377ca0a200aa
Reviewed-on: http://review.whamcloud.com/3580
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1337 llite: kernel 3.4 touch_atime switchs to 1 argument
Liu Xuezhao [Mon, 8 Oct 2012 08:54:30 +0000 (16:54 +0800)]
LU-1337 llite: kernel 3.4 touch_atime switchs to 1 argument

touch_atime switchs to use 1 argument of  struct path since
kernel 3.4 (commit 68ac1234fb949b66941d94dce4157742799fc581).

Add LC_TOUCH_ATIME_1ARG/HAVE_TOUCH_ATIME_1ARG to check it.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: I1a0478459538376761b96e1dc65328eeeef60d0f
Reviewed-on: http://review.whamcloud.com/3578
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1337 llite: 3.3 changes super_operations/inode_operations
Liu Xuezhao [Wed, 31 Oct 2012 07:43:54 +0000 (15:43 +0800)]
LU-1337 llite: 3.3 changes super_operations/inode_operations

1. Kernel 3.3 switchs super_operations to use dentry as parameter
   (kernel commit 34c80b1d93e6e20ca9dea0baf583a5b5510d92d4).
   Add LC_SUPEROPS_USE_DENTRY/HAVE_SUPEROPS_USE_DENTRY to check it.
2. Kernel 3.3 switchs inode_operations to use umode_t as parameter
   (kernel commit 1a67aafb5f72a436ca044293309fa7e6351d6a35).
   Add LC_INODEOPS_USE_UMODE_T/HAVE_INODEOPS_USE_UMODE_T to check it.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: I7564506cf4365c8da113a81058f68b9ef8b092a4
Reviewed-on: http://review.whamcloud.com/3581
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1346 libcfs: replace libcfs wrappers with kernel API
Andreas Dilger [Tue, 4 Dec 2012 20:44:31 +0000 (13:44 -0700)]
LU-1346 libcfs: replace libcfs wrappers with kernel API

The libcfs kernel portability library had wrappers for many low-level
kernel functions (locking, bit operations, etc) that were simple
wrappers around Linux kernel functions.  This provides no value for
Linux clients and clients for other kernels are not under development.

Remove the cfs_ prefix from these simple wrapper functions.  For other
kernels, they will need to use the Linux kernel API for portability.

Affected primitives:
spinlock_t, spin_lock_init, spin_lock, spin_unlock, spin_lock_bh,
spin_lock_bh_init, spin_unlock_bh, spin_trylock, spin_is_locked,
spin_lock_irq, spin_unlock_irq, read_lock_irqsave, write_lock_irqsave,
read_lock_irqrestore, write_lock_irqrestore, spin_lock_irqsave,
spin_unlock_irqrestore, SPIN_LOCK_UNLOCKED

rw_semaphore, init_rwsem, down_read, down_read_trylock, up_read,
down_write, down_write_trylock, up_write, fini_rwsem, DECLARE_RWSEM

semaphore, rw_semaphore, init_completion_module, call_wait_handler,
wait_handler_t, mt_completion_t, mt_init_completion,
mt_wait_for_completion, mt_complete, mt_fini_completion, mt_atomic_t,
mt_atomic_read, mt_atomic_set, mt_atomic_dec_and_test, mt_atomic_inc,
mt_atomic_dec, mt_atomic_add, mt_atomic_sub

rw_lock_t, rwlock_init, read_lock, read_unlock,
read_unlock_irqrestore, write_lock, write_unlock, write_lock_bh,
write_unlock_bh, RW_LOCK_UNLOCKED

completion_t, DECLARE_COMPLETION, INIT_COMPLETION, complete,
COMPLETION_INITIALIZER, init_completion, wait_for_completion,
wait_for_completion_interruptible, complete_and_exit, fini_completion

semaphore_t, DEFINE_SEMAPHORE, sema_init, up, down,
down_interruptible, down_trylock

mutex_t, DEFINE_MUTEX, mutex_init, mutex_lock, mutex_unlock,
mutex_lock_interruptible, mutex_trylock, mutex_is_locked,
mutex_destroy

lock_kernel, unlock_kernel

lock_class_key, lock_class_key_t, lockdep_set_class, lockdep_off,
lockdep_on, mutext_lock_nexted, spin_lock_nexted, down_read_nested,
down_write_nested

test_bit, set_bit, clear_bit, test_and_set_bit, test_and_clear_bit,
find_first_bit, find_first_zero_bit, find_next_bit,
find_next_zero_bit, ffz, ffs, fls

Change-Id: I36db204c703ed414504eaa9ba22e97ad7eb6cc2c
Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2829
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-2393 tests: conf-sanity/32b ensure uniform sort ordering
Nathaniel Clark [Tue, 27 Nov 2012 17:58:53 +0000 (12:58 -0500)]
LU-2393 tests: conf-sanity/32b ensure uniform sort ordering

Different hosts can have different locales (sles vs. rhel), so do all
sorts on a single node to ensure uniform ordering.

Test-Parameters: clientdistro=sles11sp2 testlist=conf-sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I9b10b8d482add2ac18f0fa51f8ec995c06e52cec
Reviewed-on: http://review.whamcloud.com/4685
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1783 tests: test failure on sanity, subtest 39l
Bob Glossman [Fri, 16 Nov 2012 16:40:08 +0000 (08:40 -0800)]
LU-1783 tests: test failure on sanity, subtest 39l

Failure of directory atime updates is due to bad build of mount.lustre in SLES.
Missing MS_* #defines led to mount.lustre being built in SLES
without conditional options and code related to atime mount options.
This caused all lustre mounts to be mounted with the option relatime.
Build fix puts back in all the supported options that were missing.
This mod also tears out the test skip put in previously to skip failing test,
since the test no longer fails.

Test-Parameters: clientdistro=sles11sp2 testlist=sanity

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I65876761065fc11c74a56bc4b5a6b7910c9d241e
Reviewed-on: http://review.whamcloud.com/4605
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
11 years agoLU-2297 test: fix replay-single test_74
Oleg Drokin [Mon, 3 Dec 2012 17:30:38 +0000 (12:30 -0500)]
LU-2297 test: fix replay-single test_74

unmount client before stopping OST, otherwise if client has dirty
data belongig to that OST, client umount will try to wait for
the data flushing back to the unavailable OST and waits forever
for the OST to come back.

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I8af14134b5f417dfd96baef14e02d9f4c8912f6f
Reviewed-on: http://review.whamcloud.com/4717
Tested-by: Hudson
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
11 years agoLU-2373 kernel: Kernel update [RHEL6.3 2.6.32-279.14.1.el6]
James Simmons [Wed, 21 Nov 2012 19:21:46 +0000 (14:21 -0500)]
LU-2373 kernel: Kernel update [RHEL6.3 2.6.32-279.14.1.el6]

Update RHEL6.3 kernel to 2.6.32-279.14.1.el6.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8dbb8c87d7e86ff0ccff2ccfb30f28e9f2a48827
Reviewed-on: http://review.whamcloud.com/4131
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1650 obdclass: find the lock by index of subpage
Jinshan Xiong [Wed, 28 Nov 2012 18:04:42 +0000 (10:04 -0800)]
LU-1650 obdclass: find the lock by index of subpage

In check_and_discard_cb(), subpage's index should be used to find if
there exists another lock to cover the same page.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Id8e1da89ef81fad4c7917360d591fc7bd1309a00
Reviewed-on: http://review.whamcloud.com/4673
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
11 years agoLU-1681 llite: remove unnecessary OBD_MD_MDTIDX
wangdi [Thu, 26 Jul 2012 12:47:25 +0000 (05:47 -0700)]
LU-1681 llite: remove unnecessary OBD_MD_MDTIDX

Remove unnecessary OBD_MD_MDTIDX to save bits
for OBD_MD_XXX.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I80e4abdf861605e09879e6bc1d7c3da0ef06ad66
Reviewed-on: http://review.whamcloud.com/3481
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
11 years agoLU-2282 obdclass: Slightly increase the size of lu_keys[]
Li Wei [Tue, 6 Nov 2012 13:39:57 +0000 (21:39 +0800)]
LU-2282 obdclass: Slightly increase the size of lu_keys[]

After an "FSTYPE=ldiskfs sh llmount.sh" on a single-node setup, all
the thirty-two slots in lu_keys[] will be consumed.  For example,

  Slot         Key Address       Owner
  ------------------------------------------
  lu_keys[0]   ffffffffa0a4bd00  obdclass
  lu_keys[1]   ffffffffa0a4fac0  obdclass
  lu_keys[2]   ffffffffa0a4d4a0  obdclass
  lu_keys[3]   ffffffffa0a58900  obdclass
  lu_keys[4]   ffffffffa0a4fd80  obdclass
  lu_keys[5]   ffffffffa0a2d940  obdclass
  lu_keys[6]   ffffffffa0c34260  ptlrpc
  lu_keys[7]   ffffffffa05577a0  fld
  lu_keys[8]   ffffffffa0131240  fid
  lu_keys[9]   ffffffffa0692140  osc
  lu_keys[10]  ffffffffa0692180  osc
  lu_keys[11]  ffffffffa074a320  lov
  lu_keys[12]  ffffffffa074a360  lov
  lu_keys[13]  ffffffffa07d3860  obdecho
  lu_keys[14]  ffffffffa07d38a0  obdecho
  lu_keys[15]  ffffffffa0819140  lquota
  lu_keys[16]  ffffffffa0822460  lquota
  lu_keys[17]  ffffffffa081c280  lquota
  lu_keys[18]  ffffffffa0afbd60  mgs
  lu_keys[19]  ffffffffa0d00fa0  mdd
  lu_keys[20]  ffffffffa0d01c20  mdd
  lu_keys[21]  ffffffffa0d00f60  mdd
  lu_keys[22]  ffffffffa0dd56a0  osd_ldiskfs
  lu_keys[23]  ffffffffa0e6bf40  mdt
  lu_keys[24]  ffffffffa0ecb6c0  lod
  lu_keys[25]  ffffffffa08576c0  osp
  lu_keys[26]  ffffffffa0857700  osp
  lu_keys[27]  ffffffffa0f17200  ofd
  lu_keys[28]  ffffffffa0fd48e0  lustre
  lu_keys[29]  ffffffffa0fd4920  lustre
  lu_keys[30]  ffffffffa0fd64e0  lustre
  lu_keys[31]  ffffffffa0fd6520  lustre

Loading osd_zfs.ko at this point, which is necessary under certain
debugging/testing circumstances, will get ENFILE from
lu_context_key_register().  To make developing on single-node setups
easier while minimizing new array iterating overhead, this patch
increases the size of lu_keys[] by eight words.

A couple of additional things were considered too.  First, the number
of keys may be reducible.  For instance, the md_ucred and md_capainfo
keys in mdd might be merged into the mdd_thread_info key, at a quick
glance.  However, the amount of change is not small.  The other thing
is that lu_keys[] could be dynamically allocated and grown on demand.
But additional locking is required when synchronizing
lu_context::lc_value[]s with lu_keys[].

Change-Id: I211bdb8af524a79f99b11f73e7260572459ccb17
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/4476
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
11 years agoLU-2258 mount: solve races between evict and umount
Hiroya Nozaki [Fri, 2 Nov 2012 09:11:23 +0000 (18:11 +0900)]
LU-2258 mount: solve races between evict and umount

There are two race conditions between evict and umount.

1) The one is that obd_export_evict_by_xxx() can touch a lustre hash
which has already been finalized but not unregistered yet.

2) The other is that the class_decref() in lprocfs_wr_evict_client()
can call lprocfs_remove() via osc_cleanup() while having _lprocfs_lock
already. That's why this case ends up in a deadlock.

This patch solve these problems with the below solutions.
- see if obd_nid_hash and obd_uuid_hash is still available when
  evicting.
- move class_incref() to below LPROCFS_EXIT() and class_decref()
  before LPROCFS_ENTRY() in order to avoid dead-locking

Signed-off-by: Hiroya Nozaki <nozaki.hiroya@jp.fujitsu.com>
Change-Id: I2cc19a88ffd6a230ab115bc1e4b9d31fbbbb4615
Reviewed-on: http://review.whamcloud.com/4444
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1187 tests: Fixes in test-framework for DNE
wangdi [Mon, 12 Nov 2012 22:17:00 +0000 (14:17 -0800)]
LU-1187 tests: Fixes in test-framework for DNE

1. Add dev parameter to mkfs_opts, so it can check if it needs
--mgs for dev. combined_mgs_mdt is only useful for mdt0.
2. Add multi-node facet failover support, so it can fail multiple
targets at the same time.
3. Remove mdsn_HOST/failover_HOSTs in local.sh and add them to
facet_host.
4. replace MDSDEV1 and MGSDEV with mdsdevname and mgsdevname, because
these vars are not always defined.

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: I979b72375a731cf163c6811be3e705ce2298b52c
Reviewed-on: http://review.whamcloud.com/4520
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2357 test: fix spelling in sanityn/36 test name
Nathaniel Clark [Mon, 19 Nov 2012 22:30:08 +0000 (14:30 -0800)]
LU-2357 test: fix spelling in sanityn/36 test name

"Correctly" was ironcally spelled.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I403401fdf411bb3c6230fe2e9212f3f0ddc000df
Reviewed-on: http://review.whamcloud.com/4616
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-2273 tests: test failure on lustre-rsync-test
Bob Glossman [Wed, 14 Nov 2012 20:43:59 +0000 (12:43 -0800)]
LU-2273 tests: test failure on lustre-rsync-test

One off allocation errors in lustre_rsync test program cause
corruption of malloc arena and program faults.

Test-Parameters: clientdistro=sles11sp2 \
    envdefinitions=SANITY_EXCEPT=39l,CONF_SANITY_EXCEPT=32b

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If347bedcac7d843560b5e215cf09002316ae7b47
Reviewed-on: http://review.whamcloud.com/4583
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
11 years agoLU-2390 tests: read quota_type from MDS node
Niu Yawei [Tue, 27 Nov 2012 09:14:23 +0000 (04:14 -0500)]
LU-2390 tests: read quota_type from MDS node

In the quota_type() of t-f, we should read quota_type from MDS node
instead of MGS.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Idad791d0a39e36305845c0623343d0f58b706ee0
Reviewed-on: http://review.whamcloud.com/4681
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-2370 ldiskfs: fix procfs access to mb groups.
Alexey Lyashkov [Mon, 19 Nov 2012 10:53:00 +0000 (14:53 +0400)]
LU-2370 ldiskfs: fix procfs access to mb groups.

backporting a fix 95599968d19db175829fb580baa6b68939b320fb
from kernel.org
with additional memory allocation error handling.

Xyratex-bug-id: MRP-757
Change-Id: I33e7daa18c8b74e34bbd1e12120e27ab7be23cf5
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-on: http://review.whamcloud.com/4654
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2284 tests: enable D_QUOTA after mds/ost restart
Niu Yawei [Wed, 28 Nov 2012 10:33:46 +0000 (05:33 -0500)]
LU-2284 tests: enable D_QUOTA after mds/ost restart

Always enable D_QUOTA after mds/ost restart.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ifc069b1bb6444c41545dd7498c7c17e3a03bbcee
Reviewed-on: http://review.whamcloud.com/4695
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
11 years agoLU-2275 mdc: Don't leak requests with "strange" dispositions
Oleg Drokin [Mon, 5 Nov 2012 05:17:16 +0000 (00:17 -0500)]
LU-2275 mdc: Don't leak requests with "strange" dispositions

Make sure that a request with no transno will get its rq_replay reset,
also ensure that on error path from ll_file_open we drop reference to
request occupied by open if needed.

Change-Id: If4d59274d2574698567970821b94ef1c5c35385b
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4458
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
11 years agoLU-1199 build: Remove check for obsolete compilers
Christopher J. Morrone [Sat, 24 Nov 2012 06:27:10 +0000 (22:27 -0800)]
LU-1199 build: Remove check for obsolete compilers

The bad compilers that we were checking for are no longer in
use.

Change-Id: I754c8d23ef6547448ecc4b502ce6cf058d98f11c
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4680
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1199 build: Remove obsolete check for 2.4 kernels
Christopher J. Morrone [Sat, 24 Nov 2012 06:04:32 +0000 (22:04 -0800)]
LU-1199 build: Remove obsolete check for 2.4 kernels

Remove check for 2.4 kernels, which are no longer supported.

Change-Id: I4ed5b7c2686fa9b0c104444853e45900f90e1eb8
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4679
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1199 build: Remove obsolete ext3 check
Christopher J. Morrone [Sat, 24 Nov 2012 05:13:09 +0000 (21:13 -0800)]
LU-1199 build: Remove obsolete ext3 check

Remove the check for journal_callback_set() which was only needed
for ext3.

Also correct a couple of comments that still mentioned "ext3".

Change-Id: I0b4362eb1171051589f3772ddc253fa249019e6b
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4678
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
11 years agoLU-1199 build: Fix style and indention in build system files
Christopher J. Morrone [Sat, 24 Nov 2012 03:12:06 +0000 (19:12 -0800)]
LU-1199 build: Fix style and indention in build system files

Fix indentation by replacing spaces with tabs.

Remove stray whitespace at end of lines.

Command substitution changed from `...` notation to $(...) notation.
The latter is part of the POSIX 1003.2, and is supported by Debian's
dash shell, so it is relativly portable these days.

build/Makefile is checked in to git, not auto-generated by the
build system.  Therefore a !Makefile exception is added to
build/.gitignore.

Also, in build/autogen.sh, fix default setting of the IFS variable
to be a space and a tab, rather than 9 spaces.  I believe the 9
spaces were a result of a bad cut-and-paste.

Change-Id: I46270d808d7fe036f748da1cb45826bb42ab01e2
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4677
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
11 years agoLU-1876 hsm: bugfix about layout lock on the client
Jinshan Xiong [Wed, 7 Nov 2012 00:56:16 +0000 (16:56 -0800)]
LU-1876 hsm: bugfix about layout lock on the client

The following issues are fixed in this patch:
* deadlock at add_lsmref
* lov_delete_raid0 should wait for refcount of lsm to be zero
* handle empty layout at lov layer since layout can be changed
  anytime so original assumption of skipping lov layer when lsm is
  NULL is no longer correct
* other fixes

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie7cd744b188f4d28fdbebda8870259b931328d00
Reviewed-on: http://review.whamcloud.com/4416
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1565 ptlrpc: resend CANCEL rpc
Vitaly Fertman [Tue, 6 Nov 2012 19:09:47 +0000 (23:09 +0400)]
LU-1565 ptlrpc: resend CANCEL rpc

it is better to deliver CANCEL rpc to server reliably in the case of:
    RPC timeout, re-connect, CANCEL resend
because server may have sent BL AST and is waiting for this CANCEL.
this avoids possible idle time on server and later client evictions.

CANCEL is always has up-todate lock handle or both enqueue and cancel
are not replayed on recovery, with the exception of the case of:
    BL AST is sent; recovery starts, lock is re-enqueued, BL AST comes
    to client, cancel is created, recovery ends (lock handle has
    changed), CANCEL is sent, its reply gets estale as lock handled is
    not updated in the RPC.
this case is left unfixed and still may result in lock callback
timeout and client eviction, but this race window is much much shorter
than the target case being fixed by this fix.

Also remove lock cancelling from client_common_put_super() as it is
done later in client_disconnect_export().

Change-Id: I1bfe70444299d93c3fb348b737cb9721ea63eda3
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Xyratex-bug-id: MRP-477
Reviewed-on: http://review.whamcloud.com/3189
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2394 llog: Fix a NULL pointer dereference
Prakash Surya [Mon, 26 Nov 2012 23:54:15 +0000 (15:54 -0800)]
LU-2394 llog: Fix a NULL pointer dereference

If llog_cat_id2handle fails and returns a positive value, it may free
its 'res' parameter and set it to NULL. Thus, a caller must be cautious
not to cause a NULL pointer dereference on error.

This patch fixes the cat_cancel_cb function, keeping it from dereferencing
this potentially NULL pointer.

In addition, any console message regarding a failed call to
llog_cat_id2handle in llog_cat.c now outputs the same message.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I949b558d0ea4569685e8ed85b7a920b2dca548c3
Reviewed-on: http://review.whamcloud.com/4676
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
11 years agoLU-1565 disabling ELC through procfs
Vitaly Fertman [Tue, 26 Jun 2012 12:03:37 +0000 (16:03 +0400)]
LU-1565 disabling ELC through procfs

adding a possibility to disable ELC through procfs, for test purposes only.

Change-Id: I45153d67f325bd464943c8f618d5c86dd663166e
Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Reviewed-by: Alexander Zarochentsev <Alexander_Zarochentsev@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Xyratex-bug-id: MRP-477
Reviewed-on: http://review.whamcloud.com/3188
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-974 security: ignore umask if acl enabled
Lai Siyao [Fri, 13 Jan 2012 10:34:33 +0000 (18:34 +0800)]
LU-974 security: ignore umask if acl enabled

* add OBD_CONNECT_UMASK to show whether MDS supports umask.
* client packs umask in create/open request, MDS will use it in
  object creation.
* client ignores umask if acl is enabled.
* client enforces umask if MDS doesn't support OBD_CONNECT_UMASK.
* don't update inode->i_mode after creation because la_mode may
  not be correct which doesn't consider umask.
* add an acl test for this.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Ibbb45dd79378d116eb428b76070a417fe08a8142
Reviewed-on: http://review.whamcloud.com/1972
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2285 osp: Fix last_created updates after orphan cleanups
Li Wei [Mon, 12 Nov 2012 09:50:01 +0000 (17:50 +0800)]
LU-2285 osp: Fix last_created updates after orphan cleanups

After an OST orphan cleanup, the OST's last created ID plus one is
stored into opd_pre_last_created.  This is weird and contradicts with
osp_precreate_send() and a few other places, like the following except
from osp_precreate_reserve():

  ...
  precreated = d->opd_pre_last_created - d->opd_pre_used_id;
  if (precreated > d->opd_pre_reserved) {
          d->opd_pre_reserved++;
  ...

The issue is not only about coding style, as suggested by this piece
of debug log dumped by a run of replay-ost-single 2:

  00000004:00080000:0.0:1353838616.328831:0:14474:0:(osp_precreate.c:4
  51:osp_precreate_cleanup_orphans()) lustre-OST0000-osc-MDT0000: Got
  last_id 130 from OST, last_used is 36, pre_used 130
  ...
  00000004:00080000:1.0:1353838616.328935:0:14230:0:(osp_precreate.c:7
  42:osp_precreate_reserve()) lustre-OST0000-osc-MDT0000: precreated 1
  : opd_pre_last_created 131 opd_pre_used_id 130 opd_pre_reserved 0
  00000004:00080000:1.0:1353838616.329173:0:14230:0:(osp_precreate.c:8
  01:osp_precreate_get_id()) lustre-OST0000-osc-MDT0000: Incremented o
  pd_pre_used_id: 131

The first line says that opd_pre_last_created was 131, while on the OST
side the last created ID was 130.  Then, according to the second line, a
request tried to reserve an object and found one.  The last line tells
us that the request consumed 131.  Actually, the OSP at that time hadn't
officially created 131 on the OST yet.

This patch makes sure opd_pre_last_created stores what its name
implies.

Change-Id: I8b94df2c4c702aaac9d68859f59c2d5ec1e8b796
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/4625
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-2356 tests: ensure recovery-small/10-12 recover fully
Nathaniel Clark [Mon, 19 Nov 2012 22:12:57 +0000 (14:12 -0800)]
LU-2356 tests: ensure recovery-small/10-12 recover fully

Ensure recovery compeletes in tests 10, 11, and 12 before proceeding
to next test, otherwise it can cause the next test to fail.

This also ensure that partial test runs can be restarted cleanly by
fixing initial rm's of old test run files.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ifa0e5c7344a23c112a69550ff308e3081e5a21d5
Reviewed-on: http://review.whamcloud.com/4615
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2171 ost: remove old checksum verification
Alex Zhuravlev [Tue, 20 Nov 2012 19:41:45 +0000 (23:41 +0400)]
LU-2171 ost: remove old checksum verification

it was used long ago to catch bugs. now it conflicts with
LU-2150 where pages are not pinned by the bulk descriptor
and can not be accessed after a call to obd_commitrw().

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5cc05e0836e8e42871aa2eca1d7ec0f402b42ee7
Reviewed-on: http://review.whamcloud.com/4400
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-20 kernel: add descriptions to kernel patches
Andreas Dilger [Mon, 19 Nov 2012 19:30:53 +0000 (12:30 -0700)]
LU-20 kernel: add descriptions to kernel patches

Add some descriptive text to kernel patches so that it is more clear
what the patches are needed for, and whether they can simply be
dropped, or if they should be pushed upstream.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie19de1feedc8bedeb7ae890886aab73815500c1e
Reviewed-on: http://review.whamcloud.com/4613
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Keith Mannthey <kemannthey@gmail.com>
11 years agoLU-2251 osp: remove invalid assertion
James Simmons [Mon, 26 Nov 2012 17:56:51 +0000 (12:56 -0500)]
LU-2251 osp: remove invalid assertion

Given we initiate precreation before current pool is exhausted,
we can easly get final number of precreated objects more than
OST_MAX_PRECREATE but it should always be less then the
theortical max of OST_MAX_PRECREATE * 2. If its more than that
then the LASSERT is valid.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ie3ec88c95d94529e457981942e8196b5a6f7d7ae
Reviewed-on: http://review.whamcloud.com/4495
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-2285 osp: Enhance pre-creation-related debug messages
Li Wei [Sun, 25 Nov 2012 07:48:00 +0000 (15:48 +0800)]
LU-2285 osp: Enhance pre-creation-related debug messages

This patch adds a couple of debug messages to help diagnose osp's
pre-creation code.  It also fixes OBD names and print formats in a few
related messages.

Change-Id: Ide153834511360855f390e7f35e85a389d0bd4db
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/4668
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-2181 tests: test failure on conf-sanity, subtest 23a
Bob Glossman [Tue, 20 Nov 2012 20:39:40 +0000 (12:39 -0800)]
LU-2181 tests: test failure on conf-sanity, subtest 23a

Disable subtests that do umount with mds down in SLES11 SP2.
Problems with upstream umount command cause hangs.

Test-Parameters: fortestonly, clientdistro=sles11sp2, \
 envdefinitions=SANITY_EXCEPT=39l

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3d5f0c02dcc0deef1d38e2d1a09c8ed6cf2b295f
Reviewed-on: http://review.whamcloud.com/4639
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-2268 o2iblnd: initialize tx-pool after fmr-pool
Liang Zhen [Mon, 5 Nov 2012 12:29:18 +0000 (20:29 +0800)]
LU-2268 o2iblnd: initialize tx-pool after fmr-pool

TX pool should be created after FMR pool, otherwise
kiblnd_create_tx_pool() will not allocate tx_pages for
kib_tx_t even if FMR is enabled.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I30d3c53b4e29ac249500cf1f780619dcfaa59b10
Reviewed-on: http://review.whamcloud.com/4670
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2200 tests: conf-sanity/32 does not work with IB
Nathaniel Clark [Fri, 16 Nov 2012 21:22:01 +0000 (16:22 -0500)]
LU-2200 tests: conf-sanity/32 does not work with IB

Check if running over infiniband and skip test if that is the case.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I45ca12c15935df268c2699feb7a135b80782664c
Reviewed-on: http://review.whamcloud.com/4607
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-2118 mdt: fetch parent fid if requested
Alex Zhuravlev [Fri, 9 Nov 2012 11:15:46 +0000 (15:15 +0400)]
LU-2118 mdt: fetch parent fid if requested

open-by-fid replay needs this functionality to use correct parent
and appropriate default striping.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I8aa618f98f7a7dbbcf32f3407fa89c79a8b98d83
Reviewed-on: http://review.whamcloud.com/4503
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-921 test: drop subtest 24 from ALWAYS_EXCEPT
Hongchao Zhang [Tue, 30 Oct 2012 19:02:11 +0000 (03:02 +0800)]
LU-921 test: drop subtest 24 from ALWAYS_EXCEPT

a new subtest 24a is introduced, drop it from ALWAYS_EXCEPT
to allow it to be tested, and verify whether the newer linux
kernel handle fsync(by filemap_fdatawait) correctly.

Pls see BZ5494 to more detail.

Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Change-Id: I88d5c6870827cafc5534aba33661b0c8180a8d5c
Reviewed-on: http://review.whamcloud.com/4443
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1940 ost: wait a while for OBD_FAIL_OST_EROFS
Hongchao Zhang [Tue, 13 Nov 2012 22:36:43 +0000 (06:36 +0800)]
LU-1940 ost: wait a while for OBD_FAIL_OST_EROFS

for OSC, -EROFS is a recoverable error, it will resend the failed
request at most cl_resends times (obd->u.cli.cl_resends), and
its default value is 10, then for test case, the failed request
will be resent repeatly without delay, and the cl_resends could be
not enough to handle, replaces OBD_FAIL_CHECK with OBD_FAIL_TIMEOUT
will introduce some delay for -EROFS.

Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Change-Id: Ic2a8fe6f2d75e74f30d007940a8fd631997135e4
Reviewed-on: http://review.whamcloud.com/4622
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-930 utils: update lfs_migrate example usage
Andreas Dilger [Tue, 6 Nov 2012 23:12:40 +0000 (16:12 -0700)]
LU-930 utils: update lfs_migrate example usage

Update the lfs_migrate example usage to exclude files created within
the past two days.  This makes it more clear that the current script
needs to take care to avoid migrating files that may still be modified
by applications.  While two day old files are not guaranteed to be
unused, it definitely would avoid the vast majority of cases of files
that were just created and are being written to by running apps.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If7c06ef3b0688db492930a56d7ba5ad4599b04e2
Reviewed-on: http://review.whamcloud.com/4481
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
11 years agoLU-1978 llite: Fix i_size race with truncate and mkwrite
michael.mckay [Wed, 19 Sep 2012 14:54:56 +0000 (10:54 -0400)]
LU-1978 llite: Fix i_size race with truncate and mkwrite

These changes address a race between the truncate
process and the mkwrite process. We add an additional
check in the case of a mkwrite to make sure we are not
past the end of the file. While holding the lli_trunc_sem
lock we check the size and make sure we are not past
the end of the file. If we are we return -ENODATA which
will trigger a SIGBUS. This is similar to what occurs in the
ext4 filesystem in these cases. The return status to -ENODATA
which will eventually get mapped to
VM_FAULT_NOPAGE (ll_page_mkwrite()).
The kernel will then return a SIGBUS. These changes
return -ENODATA to ensure we correctly clean up. In the
case of the mapping being null nothing will change.

Xyratex-bug-id: MRP-651
Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Reviewed-by: Alexander Zarochentsev <Alexander_Zarochentsev@xyratex.com>
Signed-off-by: Michael McKay <michael_mckay@xyratex.com>
Change-Id: Ice41b20ac16d81be44fb0c6a1bf0da8b2fcf9b7c
Reviewed-on: http://review.whamcloud.com/4044
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1517 ptlrpc: throw net error to ptlrpc for bulk
Alexander.Boyko [Mon, 17 Sep 2012 13:48:59 +0000 (17:48 +0400)]
LU-1517 ptlrpc: throw net error to ptlrpc for bulk

Start reconnect and resend if network error occures
for the bulk transfer.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: I0cf2ee1230a039336f081fbb520c1ce768882088
Xyratex-bug-id: MRP-523
Reviewed-on: http://review.whamcloud.com/4299
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1944 ldlm: Add another net_latency to recoverry timer.
wangdi [Fri, 14 Sep 2012 14:39:21 +0000 (07:39 -0700)]
LU-1944 ldlm: Add another net_latency to recoverry timer.

Add another net_latency to this request deadline, i.e.
we need add 2 net_latency to extend the recovery timer,
one for balance rq_deadline (see ptl_send_rpc), one for
resend the req to server.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I6cb6e57358f4fe32ae2ac8d982af6ce03b68014b
Reviewed-on: http://review.whamcloud.com/4351
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoRevert "LU-1756 kernel: cleanup lustre_compat25.h"
Oleg Drokin [Wed, 21 Nov 2012 16:42:00 +0000 (11:42 -0500)]
Revert "LU-1756 kernel: cleanup lustre_compat25.h"

This is causing build failures on SLES11SP2

This reverts commit 472fe13982952c9ccad969a996c054f9cf643847.

11 years agoLU-2139 osc: Move cl_client_lru to cl_client_cache
Prakash Surya [Tue, 6 Nov 2012 21:24:27 +0000 (13:24 -0800)]
LU-2139 osc: Move cl_client_lru to cl_client_cache

The cl_client_lru structure was renamed to cl_client_cache. This is
intended as a purely cosmetic change motivated by the future inclusion
of other, non LRU, information in that structure. For example, the plan
is to include "unstable" page information in this structure; which is
unrelated to the LRU list of clean pages, but still part of a client's
cache.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I286a1bbb2fb0d7dc52f007407bb823f960ae7c62
Reviewed-on: http://review.whamcloud.com/4483
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1756 kernel: cleanup lustre_compat25.h
Peng Tao [Thu, 16 Aug 2012 07:59:21 +0000 (15:59 +0800)]
LU-1756 kernel: cleanup lustre_compat25.h

1. unused functions:
   ll_bdi_wb_cnt()
2. call bdi_init/bdi_destroy directly
3. call add_to_page_cache_lru directly
4. fix a typo of ll_pagevec_init
5. remove QUOTA_OK/NO_QUOTA. They are defined in libcfs

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I9953dabbc758e6fb7a3d0649bdc64ba1455efb99
Reviewed-on: http://review.whamcloud.com/3689
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
11 years agoLU-2031 obdclass: check if client is connected to the target
Jinshan Xiong [Tue, 13 Nov 2012 19:56:17 +0000 (11:56 -0800)]
LU-2031 obdclass: check if client is connected to the target

When the client is notified by IR, it may not connect to the OST
yet, verify it before writing the new NID to the import.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ia144b695443a9850ecccd0c8946de029492b4446
Reviewed-on: http://review.whamcloud.com/4525
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2244 lov: remove unused bits from lov, osc
Alex Zhuravlev [Sun, 28 Oct 2012 18:38:11 +0000 (21:38 +0300)]
LU-2244 lov: remove unused bits from lov, osc

- precreation logic in OSC
- QoS code in LOV
- fake requests in ptlrpc

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I7a3e3ec6cf254a9fcd53ae7eab6c2d23b7520234
Reviewed-on: http://review.whamcloud.com/4399
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
11 years agoLU-591 mdd: link ea code cleanup
Vitaly Fertman [Mon, 22 Oct 2012 19:55:12 +0000 (23:55 +0400)]
LU-591 mdd: link ea code cleanup

Change-Id: I7504a2a3ae540471ef37d75e46ca0d7cf9d8a9f5
Xyratex-bug-id: MRP-139
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Nathan Rutman <Nathan_Rutman@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1230
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-2345 tests: remove $FSTYPE from tests
Niu Yawei [Fri, 16 Nov 2012 10:50:13 +0000 (05:50 -0500)]
LU-2345 tests: remove $FSTYPE from tests

$FSTYPE is deprecated, it should be replaced by facet_fstype.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I6671249c88a53b01badebd7172fb3307c8f4f48b
Reviewed-on: http://review.whamcloud.com/4596
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
11 years agoLU-1606 api: Do not include lustre_idl.h from lustreapi.h
Christopher J. Morrone [Sat, 10 Nov 2012 02:06:29 +0000 (18:06 -0800)]
LU-1606 api: Do not include lustre_idl.h from lustreapi.h

lustre_idl.h has become increasingly difficult to use from
user-space.  Normal users of the lustre api should not
be looking into lustre wire protocol anyway, so this change
eliminates the include of lustre_idl.h.

After removing lustre_idl.h, it became obvious that a number
of programs have been picking up normal user-space headers
through a very windy path of includes.  With the include
of lustre_idl.h gone, they no longer compiled, so we also
add the missing explicit includes.

It became clear that copytool.c explicity requires
libcfs/libcfs.h, and lustre_rsync.c require lustre_idl.h.
But I believe that it is far better to have those includes
explicitly stated, so it is obvious that those programs
are peeking into lustre's internals.  In the future we
can work on creating new lustre API calls that provide the
information that they need without side-stepping abstraction
layers.

Change-Id: I8ce55ffe0e1766515061f3dbc63c2372d5713f58
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4505
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>