Whamcloud - gitweb
fs/lustre-release.git
8 years agoLU-831 header: struct bit field should be unsigned type
Bobi Jam [Thu, 10 Nov 2011 07:43:44 +0000 (15:43 +0800)]
LU-831 header: struct bit field should be unsigned type

Make sure struct bit fields be unsigned type, or else if they are read
from proc interface, they would showes as a big number equivalent to
-1.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ib252b4e89ce6b3f898e2da11a60de9aa9201119c
Reviewed-on: http://review.whamcloud.com/1685
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-835 build: skip generated files in .gitignore
Andreas Dilger [Wed, 9 Nov 2011 22:53:12 +0000 (15:53 -0700)]
LU-835 build: skip generated files in .gitignore

Skip files automatically generated during builds on FC-13.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1cff9a351137145bb87a89460d0a1b54fae29e2c
Reviewed-on: http://review.whamcloud.com/1681
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-795 osd api: Commit callback per transaction
Mikhail Pershin [Thu, 27 Oct 2011 04:03:33 +0000 (08:03 +0400)]
LU-795 osd api: Commit callback per transaction

- ability to add commit callback per transaction in addition to
  per-device hooks. Now it is much simpler if only commit callback
  is needed.
- rewrite commit callbacks for last_commit and new_client, add commit
  callback in seq manager
- cleanup not-needed code: old commit callbacks, txn_keys
- remove osd od_env_for_commit environment and env param from commit
  callbacks
- use th_sync to mark sync operations

Change-Id: If5f8f2a6d3cd2f3e77fd13c802213a181043a2d7
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1621
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-691 Fix OST index errors in test suite - sanity 133c defect
James Simmons [Thu, 3 Nov 2011 14:39:36 +0000 (10:39 -0400)]
LU-691 Fix OST index errors in test suite - sanity 133c defect

Several test run do_facet ost which assumes ost is ost0 which does not exist
according to the way the test suite works. This patch address several areas
where the wrong ost index is used. This patch also introduces some short hand
functions to get OST properties from the index. Those functions ensure ost1,
ost10 and ost100 are seen as different which is not always the case.

Change-Id: Ic31224794563964a3415d24abeebce9dacceb686
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/1425
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoUpdated version to 2.1.52 2.1.52 2.1.52.0 v2_1_52_0
Oleg Drokin [Sat, 12 Nov 2011 05:23:50 +0000 (00:23 -0500)]
Updated version to 2.1.52

Change-Id: I68a6934d997921a82670272cf30abdcc25d6a575
Signed-off-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-3 mntopt: consider low-layer options for MDT ACL flags
Fan Yong [Fri, 11 Nov 2011 02:18:39 +0000 (10:18 +0800)]
ORNL-3 mntopt: consider low-layer options for MDT ACL flags

Currently, MDT layer enables ACL support by default without checking
whether low-layer (ldiskfs) enables ACL support or not, then causes
unnecessary data traffic on network and through MDS stack for ACL.

So MDT should communicate with low-layer before setting ACL flags.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I804f10bf486745ddd3b23b89e959dfd585589ac0
Reviewed-on: http://review.whamcloud.com/1211
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-769 ptlrpc: Do not miss pending signals in ptlrpc_set wait
Oleg Drokin [Mon, 7 Nov 2011 03:34:41 +0000 (22:34 -0500)]
LU-769 ptlrpc: Do not miss pending signals in ptlrpc_set wait

conf_sanity test 23a highlighted a problem in ptlrpc_set_wait logic,
if we enter there with a signal pending and the import is not FULL,
there is no way to interrupt such a set because we block signals
all the time. Enabling signals all the time is not an option either.
Waiting until import reconnects is questionable too since it might
never come up after all (like in the test 23a).
So for the solution we will just manually mark the set as interrupted
after the initial wait.

Change-Id: Iaa3e356e971b4f75fd7f21cc579c85f7487719a0
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1657
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
8 years agoLU-762 ldiskfs: don't drop directory nlink to 0
yangsheng [Thu, 3 Nov 2011 18:25:42 +0000 (02:25 +0800)]
LU-762 ldiskfs: don't drop directory nlink to 0

When landing the nlink patch for ext4, for an unknown reason the
logic in ext4_dec_count() was changed from the ext3 version of
the patch. It now drops the nlink = 0 temporarily and then
if it is a directory with nlink == 0 it increases nlink again.

Instead, only drop nlink if it is larger than 2.

Change-Id: Ieeff3e45daea56f502848f9c2b0fb04f0a9d2b6d
Author: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1644
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-780 test: improve parallel-scale to support hyperion run
Minh Diep [Fri, 4 Nov 2011 23:12:57 +0000 (16:12 -0700)]
LU-780 test: improve parallel-scale to support hyperion run

We need to add support for srun/slurm, and a few tests
from hyperion-sanity script that has been used for hyperion
testing

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I7f1baa0c99980ad9001436911d23f1030aa7d0fe
Reviewed-on: http://review.whamcloud.com/1615
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Cliff White <cliffw@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-826 mdd: Fix for mdd_lov_create_finish memory free
Alexander.Boyko [Mon, 7 Nov 2011 15:22:26 +0000 (18:22 +0300)]
LU-826 mdd: Fix for mdd_lov_create_finish memory free

At lov/lov_pack.c lov_packmd() use OBD_ALLOC_LARGE for memory
allocation for lov_mds_md object(lmpp), but mdd_lov_create_finish
use OBD_FREE to free memory. This bug doesn`t affect to current
version, but may be relevant in future.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: Ic37cb72022b9aac02368b11f370cbaad0c730e7c
Reviewed-on: http://review.whamcloud.com/1659
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-8 IR: minor fixes to IR patches
Jinshan Xiong [Thu, 20 Oct 2011 18:34:23 +0000 (11:34 -0700)]
ORNL-8 IR: minor fixes to IR patches

Summary of fixes:
1. ORNL-13: add comments to mgs_handle_fslog_hack();
2. ORNL-14: typo and accurate error/debug messages;
            minor fixes in recovery-small.sh;

Change-Id: I6317067eb6250faf5df21be719c82de17b0f4cc9
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1582
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-7 replace statahead hacking with its own cache
Fan Yong [Tue, 8 Nov 2011 02:28:54 +0000 (10:28 +0800)]
ORNL-7 replace statahead hacking with its own cache

Original statahead hacks dcache, it inserts dentry into dcache and
associates with inode without holding parent's lock, which breaks
VFS layer synchronization mechanism.

The new statahead does not build dentry, it just pre-fetches inode's
attributes and related ldlm locks, caches them in a small statahead
cache against parent's inode. The statahead sponsor can search such
small cache similar as searching dcache to look for what it wants.
If cache hit, then builds dentry, associates with inode, and inserts
into dcache by the statahead sponsor itself.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I6948f8a438a938c51563468d775e676e4185e580
Reviewed-on: http://review.whamcloud.com/1208
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-774 test: wait till space from previous tests is released
Bobi Jam [Tue, 25 Oct 2011 10:12:34 +0000 (18:12 +0800)]
LU-774 test: wait till space from previous tests is released

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Idf4933f9b40c74b99bf763eb896680a4e0c942ff
Reviewed-on: http://review.whamcloud.com/1588
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-482 test-framework: Ensure dirty cache is flushed before barrier
Oleg Drokin [Sun, 6 Nov 2011 19:06:40 +0000 (14:06 -0500)]
LU-482 test-framework: Ensure dirty cache is flushed before barrier

With certain backend devices like LVM with older kernels the data
in dirty cache cannot be propagated all the way to the block device
with a single sync as there are multiple non-cooperating layers.

So convert such sync calls into triple syncs

Change-Id: If82e25223a277ec165d150b0f5f960ff845af9b0
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1656
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
8 years agoLU-773 tests: sanity:test_105b failure
Jinshan Xiong [Fri, 21 Oct 2011 18:07:30 +0000 (11:07 -0700)]
LU-773 tests: sanity:test_105b failure

The root cause of this issue is that activating osc in test sanity 104a
wasn't successful. We should wait for the recovery to finish.

Change-Id: I940419bfb1f579c4a0233b7439ac1f459ee584ad
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1542
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-601 mdd: Fix transaction credits
Bobi Jam [Tue, 23 Aug 2011 03:34:06 +0000 (11:34 +0800)]
LU-601 mdd: Fix transaction credits

* mdd_create()/mdd_create_data() may need delete orphan objects on
  OSTs, so that we need preserve enough transaction credits for llog
  records.
* mdd_attr_set() may write lov llogs.
* orphan_object_destroy() also will write a llog record, we need
  reserve credit for it as well.
* add credit changelog record.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I5124d2f368e2ff794b2b2b8194bec86f63e971cf
Reviewed-on: http://review.whamcloud.com/1276
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-778 o2iblnd: Add rdma_create_id() compatibility macro
Ned Bass [Wed, 19 Oct 2011 21:47:50 +0000 (14:47 -0700)]
LU-778 o2iblnd: Add rdma_create_id() compatibility macro

As of RHEL6.2 kernel 2.6.32-204.el6, rdma_create_id() requires a
queue-pair type as a fourth argument.  This was previously inferred
from the rdma_port_space argument.  Add an autoconf test to detect
whether the fourth argument is expected and a compatibility macro
that discards the QP type argument if the 3-argument version of
rdma_create_id() is present.

Change-Id: Idb668e1f059954ecc994ad59b366d54da8b82dc8
Signed-off-by: Ned Bass <bass6@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1556
Tested-by: Hudson
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-389 update wirecheck for master
Fan Yong [Thu, 27 Oct 2011 17:46:32 +0000 (01:46 +0800)]
LU-389 update wirecheck for master

Drop unused mds_body/mds_remote_perm definition and wirecheck.
Add missing wirecheck for mdt_xxx, ost_xxx, lu_xxx, and so on.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I858a02b0b5de8680cc8315e38aeeca271c26e9ad
Reviewed-on: http://review.whamcloud.com/985
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-26 prevent call md_set_lock_data() repeatly
Fan Yong [Thu, 3 Nov 2011 16:40:32 +0000 (00:40 +0800)]
ORNL-26 prevent call md_set_lock_data() repeatly

md_set_lock_data() is called from different functions in llite, and
may be called more than once for the same <inode lock> pair. It is
harmless for the correctness, but will affect the performance a bit,
should be avoided.

Drop dentry flags of "DCACHE_LUSTRE_INVALID" only when we hold the
"MDS_INODELOCK_LOOKUP" lock.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I4dbe206af77ba6a619f3268c238ce98ac7aef4c0
Reviewed-on: http://review.whamcloud.com/1224
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-686 conf-sanity test 52 fails due to missing mount point
Hongchao Zhang [Tue, 1 Nov 2011 05:44:15 +0000 (13:44 +0800)]
LU-686 conf-sanity test 52 fails due to missing mount point

Create the mount point before mount

Change-Id: I6c813f5040e4636386eabb640526f3d1072f484b
Signed-off-by: HongChao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1379
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-620 llite: add delete_from_page_cache and remove_from_page_cache check
Bobi Jam [Wed, 21 Sep 2011 10:17:13 +0000 (18:17 +0800)]
LU-620 llite: add delete_from_page_cache and remove_from_page_cache check

Later 2.6.32 kernel use memory cgroup feature but does not export
truncate_complete_page but export delete_from_page_cache or
remove_from_page_cache, we need properly use them for pachless client
code.

Change-Id: I33e3e7c32b548866ee77753ef8a8193c814d0ecb
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1399
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-000 libcfs: allow debug to be set to numeric -1
Andreas Dilger [Fri, 21 Oct 2011 21:45:17 +0000 (15:45 -0600)]
LU-000 libcfs: allow debug to be set to numeric -1

Don't warn if debug is set to "-1", which should always mean
"enable all debugging".

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1503d2f5776043e9a4ec3bc4af710ae35ca97960
Reviewed-on: http://review.whamcloud.com/1577
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-723 Enhance lustre/ldiskfs build system
Brian Behlendorf [Wed, 20 Apr 2011 23:46:54 +0000 (16:46 -0700)]
LU-723 Enhance lustre/ldiskfs build system

Enhance the lustre/ldiskfs build system so it is more robust, flexible,
and consistent with lustre/zfs build system.  This change is being made
in the interest of standardizing the infra-structure around backend
filesystems.

This change does not effect the current behavior of the --with-ldiskfs,
--enable-ext4, or --with-ldiskfsprogs configure options.  However, it
does remove the obsolete --with-ldiskfs-inkernel configure option which
was only used by LLNL.  It also adds the --with-ldiskfs-obj configure
option which improves flexibility.  And the --enable-ldiskfs-build
configure option to support building against the lustre-ldiskfs-devel
package.  The behavior of these options is consistent with their zfs
counterparts, see commit 8c7266c for further details.

  --enable-ext4           enable ldiskfs build using ext4
  --enable-ldiskfs-build  enable ldiskfs configure/make
  --with-ldiskfs=path     set path to ldiskfs source
  --with-ldiskfs-obj=path set path to ldiskfs objects
  --with-ldiskfsprogs     use alternate names for ldiskfs-enabled e2fsprogs

Sample ./configure results when building lustre and ldiskfs using
the kernel-devel and kernel-debuginfo-common packages.

checking whether to enable ldiskfs... yes
checking ldiskfs source directory... /home/behlendo/src/git/lustre/ldiskfs
checking ldiskfs object directory... /home/behlendo/src/git/lustre/ldiskfs
checking ldiskfs module symbols... Module.symvers
checking ldiskfs source release... 3.3.0
checking whether to use ext3 or ext4 source... ext4
checking ext4 source directory...  /usr/src/debug/.../fs/ext4
checking whether to build ldiskfs... yes
checking for /home/behlendo/src/git/lustre/ldiskfs/configure... yes
checking for /usr/src/debug/.../fs/ext4/dir.c...  yes
checking for /usr/src/debug/.../fs/ext4/file.c...  yes
checking for /usr/src/debug/.../fs/ext4/inode.c...  yes
checking for /usr/src/debug/.../fs/ext4/super.c...  yes
checking if ext4_ext_walk_space() takes i_data_sem... yes
checking if LDISKFS_SINGLEDATA_TRANS_BLOCKS takes sb as argument... yes
checking if ldiskfs_discard_preallocations defined... yes
checking if ldiskfs_ext_insert_extent needs 5 arguments... yes

In the context of this change additional cleanup has been done and
all of the ldiskfs specific code relocated to lustre-build-ldiskfs.m4.

Note that this change moves us closer to supporting patchless Lustre
servers with ldiskfs.  Once the remaining kernel patches for Lustre
are dropped you will be able to build Lustre using the distribution
provided kernel-devel and kernel-debuginfo-common packages.

This change also incorperates ORI-340, commit f604951, which ensures
that the Module.symvers file will cleanly include the symbols for all
enabled Lustre backends.  While the only backend supported by master
right now is ldiskfs this brings the master and orion branchs in to
sync in this regard.

Change-Id: I6f13f266944ec6967f4d7705a30b83ab8e577b15
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1566
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-22 ptlrpc: more comment for multi-threaded ptlrpcd
Fan Yong [Wed, 2 Nov 2011 04:41:09 +0000 (12:41 +0800)]
ORNL-22 ptlrpc:  more comment for multi-threaded ptlrpcd

To explain share work load between ptlrpcd partners.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I72711a93af321e43e6dbbbc52b427060be47f808
Reviewed-on: http://review.whamcloud.com/1638
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-362 Fix error path in lu_kmem_init().
Mikhail Pershin [Wed, 19 Oct 2011 19:55:04 +0000 (23:55 +0400)]
LU-362 Fix error path in lu_kmem_init().

When echo client is failed to set up there are memory leaks were
noticed, it is related to missed error handling in some functions.

- Free all caches if lu_kmem_init() failed
- fix error handling in cl_global_init() and ccc_global_init()

Change-Id: Ide9e7ad6d40f99a7fbb4330ba63b168cc408356f
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/586
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-462 Don't alloc/free client data for self export
Mikhail Pershin [Fri, 21 Oct 2011 12:38:29 +0000 (16:38 +0400)]
LU-462 Don't alloc/free client data for self export

Self export doesn't need client data and ldlm initialization.
Patch uses uuid comparision to determine self_export.

Change-Id: Id26ef90e9857e4c1d3a0e7a3756eaf67607890d6
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1574
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-791 obdfilter: Don't clear OBD_MD_FLFLAGS mistakenly
Niu Yawei [Tue, 25 Oct 2011 11:31:11 +0000 (04:31 -0700)]
LU-791 obdfilter: Don't clear OBD_MD_FLFLAGS mistakenly

Instead of set oa->o_valid to OBD_MD_FLID | OBD_MD_FLGROUP arbitrarily
in filter_handle_precreate(), it should be changed as "|=" to keep
the OBD_MD_FLFLAGS setted in filter_precreate().

Otherwise, client will not be aware of OST is running out of space,
and lov_create() will wait for objects forever in such case.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I84be8dde59dbb2829cd800e10b7aa6f4402b7e56
Reviewed-on: http://review.whamcloud.com/1589
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-28 recovery: rework extend_recovery_timer()
Jinshan Xiong [Thu, 27 Oct 2011 05:52:43 +0000 (23:52 -0600)]
ORNL-28 recovery: rework extend_recovery_timer()

extend_recovery_timer() is used to adjust timeout value of a recovering
target. In the original implementation, there was a problem it stopped
the target from firing a timer again for version recovery case.

Change-Id: I815a15fb5d3104e52a189eed1529c58d7a8d03b9
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1620
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-611 utils: add restripe option to lfs_migrate
Andreas Dilger [Fri, 19 Aug 2011 11:21:24 +0000 (05:21 -0600)]
LU-611 utils: add restripe option to lfs_migrate

Add the "-R" option to have lfs_migrate restripe a migrated file
instead of keeping the original striping.  This is useful of some
directory got the wrong striping and a bunch of files were created
with the wrong striping.

Avoid possible confusion between the lfs_migrate and lfs setstripe
command-line options.  For now, deprecate the old "-c" option, since
it is redundant in any case.

Change-Id: I3a39bad93ef5c079678c65960e53d22e51431df3
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1265
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-532 mdt: improve xattr ctime warning message
Andreas Dilger [Thu, 28 Jul 2011 22:13:53 +0000 (16:13 -0600)]
LU-532 mdt: improve xattr ctime warning message

Print out which xattr is not getting OBD_MD_FLCTIME set so that it
is possible to track down what code path on the client is failing.

Change-Id: I1918d2e8e0a1e03d8437846e823bca9df6f89b48
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1161
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoRevert "LU-462 Don't alloc/free client data for self export"
Oleg Drokin [Mon, 31 Oct 2011 06:45:50 +0000 (02:45 -0400)]
Revert "LU-462 Don't alloc/free client data for self export"

This introduced a memory leak problem

This reverts commit 140178844e5c0e4f3cfed8199800e39bf7082cd9

Change-Id: I558da8c44e08f77e77f7d1fe79da892a579992c3
Reviewed-on: http://review.whamcloud.com/1631
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-22 general ptlrpcd threads pool support
Fan Yong [Sat, 15 Oct 2011 16:09:53 +0000 (00:09 +0800)]
ORNL-22 general ptlrpcd threads pool support

Originally, there were two ptlrpcd threads on each node to serve all
async RPCs on the node, one ptlrpcd is for BRW, the other is for all
others. Such load mode cannot match more and more async RPCs process
on current large SMP node.

So we introduce ptlrpcd threads pool, any ptlrpcd threads in the pool
can be common shared by all async RPCs, like async I/O, async glimpse
lock, statahead, and ect. The async RPC sponsor can affect the system
load mode by specifying load policy when pushes the RPC into ptlrpcd
queue. On the other hand, it supports some flexible binding policies
to bind some ptlrpcd threads on CPU cores for reducing cross-CPU data
traffic, and also allow some ptlrpcd threads to be scheduled freely
on any CPU core to try to guarantee processing async RPCs in time.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: Icc0bd689df73b6863cc9adc544c3654c046cb8bd
Reviewed-on: http://review.whamcloud.com/1184
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-25 process dir page hash collision
Fan Yong [Fri, 28 Oct 2011 07:03:26 +0000 (15:03 +0800)]
ORNL-25 process dir page hash collision

If dir page has hash collision with others, then remove such page
from cache after using to avoid to be found unexpectedly later.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I15ff85e5233248944d77a9d93292d8690e1a715f
Reviewed-on: http://review.whamcloud.com/1234
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-770 tests: MRP-250 sanity.sh fails in CLIENTONLY mode
Andriy Skulysh [Tue, 18 Oct 2011 12:08:37 +0000 (15:08 +0300)]
LU-770 tests: MRP-250 sanity.sh fails in CLIENTONLY mode

Skip sanity.sh tests which requires remote access to
lustre servers.

Signed-off-by: Andriy Skulysh <andriy_skulysh@xyratex.com>
Change-Id: Ia853a8cf95bf6bf638e391aab30654a06a3b6589
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1537
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-554 add gnilnd awareness to LNet
Wally Wang [Wed, 24 Aug 2011 20:22:47 +0000 (13:22 -0700)]
LU-554 add gnilnd awareness to LNet

This allows servers on any network to talk to gnilnd routers.
This is 2.1 version of the Oracle 23884 attachment 31892.

Change-Id: I96777551b0caa50021ebb32755caaa01623ea97d
Signed-off-by: Wally Wang <wang@cray.com>
Reviewed-on: http://review.whamcloud.com/1285
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-715 lov: fix procfs reporting for qos values
Matt Ezell [Mon, 17 Oct 2011 14:43:22 +0000 (10:43 -0400)]
LU-715 lov: fix procfs reporting for qos values

When writing to
/proc/fs/lustre/lov/<fsname>-mdtlov/{qos_prio_free,qos_threshold_rr},
the values read back are often one less than the values written.
This happens because internally the value is stored as a number from
0-255 but accessed by the user with 0-100. Integer truncation in the
storage and retrieval stages causes this to often show lower. Adding
255 to an internal step causes the bit-shift to "round up".

Signed-off-by: Matt Ezell <ezell@nics.utk.edu>
Change-Id: I9050aadb55bfa82d14b94a78e399d315249ac48f
Reviewed-on: http://review.whamcloud.com/1532
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-700 osc: Tally BRW_{READ,WRITE}_BYTES by bytes transferred.
John Hammond [Wed, 21 Sep 2011 19:00:39 +0000 (14:00 -0500)]
LU-700 osc: Tally BRW_{READ,WRITE}_BYTES by bytes transferred.

Call ptlrpc_lprocfs_brw() in brw_interpret() rather than
osc_send_oap_rpc() and tally by the number of bytes transferred rather
than the number requested.

Change-Id: Ia7191972d9671f01d942a46eba069191f130f516
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/1402
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-672 MRP-213 don't panic on geting version for non existent fid
Mikhail Pershin [Thu, 22 Sep 2011 13:53:38 +0000 (17:53 +0400)]
LU-672 MRP-213 don't panic on geting version for non existent fid

lctl getobjversion can be called for a file which was removed from
MDT, but exists in cache. That producea a assert on mdd code:
LustreError: 20825:0:(mdd_object.c:2474:mdd_version_get())
ASSERTION(mdd_object_exists(mdd_obj)) failed

Author: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I7442e7aee26736741482c158ee3713df9796c953
Reviewed-on: http://review.whamcloud.com/1365
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-723 ldiskfs: remove ext3 RHEL5 kernel series
Andreas Dilger [Wed, 26 Oct 2011 07:34:34 +0000 (01:34 -0600)]
LU-723 ldiskfs: remove ext3 RHEL5 kernel series

Remove the old ext3 RHEL5 kernel patch series.  This has been
deprecated since Lustre 1.8.6 in favour of the ext4 RHEL5 series.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1e09ff432f2e970446c3b43fb92f0c1a988159ae
Reviewed-on: http://review.whamcloud.com/1603
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-20 ldiskfs: remove spurious warning message
Andreas Dilger [Fri, 21 Oct 2011 10:06:16 +0000 (04:06 -0600)]
LU-20 ldiskfs: remove spurious warning message

Remove the spurious warning message that we added for the missing
extents option since the MDS never gets the extents option enabled.

ldiskfs_fill_super: extents feature not enabled on this
filesystem, use tune2fs

Remove the obsolete patch that removed this from the patch we added.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9d896258b89ec3db528db59094572daac8dc207b
Reviewed-on: http://review.whamcloud.com/1572
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-334 llite: Add LPROC_LL_OSC_{READ,WRITE}.
John Hammond [Wed, 21 Sep 2011 22:17:25 +0000 (17:17 -0500)]
LU-334 llite: Add LPROC_LL_OSC_{READ,WRITE}.

This patch adds quick and easy access to aggregate OSC BRW (bytes on
the wire) statistics through llite.  To accomplish this, we pass the
number of bytes transferred by a successful request to
cl_req_completion(), and on to ll_stats_ops_tally() by way of the
ccc_req_completion().

Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Change-Id: I4b464c27f6dc87fcc19d35b2bc45dc7cb9bf7741
Reviewed-on: http://review.whamcloud.com/1341
Reviewed-by: Richard Henwood <rhenwood@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-646 port bz23485 (clarification of lustre fsync behavior)
Lai Siyao [Tue, 30 Aug 2011 03:44:31 +0000 (20:44 -0700)]
LU-646 port bz23485 (clarification of lustre fsync behavior)

Add directory fsync operation.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I9915bab70291c503ff1462328d5f2fbcff2b700e
Reviewed-on: http://review.whamcloud.com/1309
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-553 build: fix commit-msg line width check
Andreas Dilger [Wed, 19 Oct 2011 20:05:33 +0000 (14:05 -0600)]
LU-553 build: fix commit-msg line width check

Fix the calculation of the commit message line width to skip the
trailing linefeed character.

Don't use the "git hash_object -t commit" option, since this makes
some versions of Git unhappy and generate an empty Commit-Id string.

Skip diffstat output from "commit -v" when validating commit comment.

Validate the Change-Id: line has the proper ID format.

Reported-by: Bobi Jam <bobijam@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0bcfdedbe6d33fd5f81381ca33862a10d6b41b38
Reviewed-on: http://review.whamcloud.com/1553
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-629 ptlrpc: fix _debug_req to print opc/status
Andreas Dilger [Wed, 24 Aug 2011 21:17:53 +0000 (15:17 -0600)]
LU-629 ptlrpc: fix _debug_req to print opc/status

The 2.x _debug_req() function was changed in bug 16359/commit 5467a86021
to avoid problems with accessing unswabbed message buffers. Unfortunately,
this broke the printing of many/most _debug_req() messages, because it
didn't check whether swabbing was actually needed in the first place.

Also, in ptlrpc_expire_one_request() some extra debugging information was
added in bug 21636/commit 368689640 but never removed, making this common
message overly verbose.

Fix _debug_req() so that it prints opcode/flags/status, unless the
ptlrpc_body _needs_ to be swabbed, but isn't.  Also print out more
useful idenfifiers for the nodes (the obd_name and NID instead of
the connection UUID).  This removes some of the added verbosity from
ptlrpc_expire_one_request(), and most of the rest was already being
printed out (deadline, current, etc).

Change-Id: I88a78486becd19f5b38f5578e5cc30e649564908
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1286
Tested-by: Hudson
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-753 obdfilter: improper LASSERT in filter_commitrw_write()
Niu Yawei [Sat, 22 Oct 2011 08:16:55 +0000 (16:16 +0800)]
LU-753 obdfilter: improper LASSERT in filter_commitrw_write()

In rare cases fsfilt_commit_wait() will wake up and return after the
transaction has finished its work and updated j_commit_sequence but
the commit callbacks have not been run yet. Which will trigger the
LASSERT(oti->oti_transno <= obd->obd_last_committed) improperly.

We should just wait for the commit callback finished instead of put
an improper LASSERT here.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ibd5add8d352d2e7598be49b0bf8fa37d40ce6e1f
Reviewed-on: http://review.whamcloud.com/1583
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-24 skip unnecessary client obj hash lookup
Fan Yong [Sun, 9 Oct 2011 14:27:51 +0000 (22:27 +0800)]
ORNL-24 skip unnecessary client obj hash lookup

The client-side object will be initialized when inode is established
in memory. Such object will be inserted into global hash table for
further using. The caller follows "lookup-alloc-lookup-insert". It is
standard, but maybe unnecessary for some cases. If it is sure that
there is only one caller will insert the new established object, then
we can skip the two lookup, and "alloc-insert" directly.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: Ie173c57627b2e5b4ed9b8a93f368d88ba8e54c31
Reviewed-on: http://review.whamcloud.com/1225
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-15 slow IO with read-intense application
yangsheng [Wed, 17 Aug 2011 18:43:49 +0000 (02:43 +0800)]
LU-15 slow IO with read-intense application

Align the readahead extent by 1M after when it is trimed by ra_max_pages.

Change-Id: I4102d2fe956fd01457949f0eb7c63654b5c2d095
signed-off-by: Wang Di <di.wang@whamcloud.com>
signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1255
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-555 ll_have_md_lock() optimization to accelerate multiple bits locks
jcl [Sat, 30 Jul 2011 12:54:01 +0000 (14:54 +0200)]
LU-555 ll_have_md_lock() optimization to accelerate multiple bits locks

Change-Id: Ie300ad2abf285a7413553da2a86cc74216cbae7d
Signed-off-by: jcl <jacques-charles.lafoucriere@cea.fr>
Reviewed-on: http://review.whamcloud.com/1170
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-488 ptlrpc_connection_put() LASSERT(!cfs_hlist_unhashed(&conn->c_hash))
Lai Siyao [Fri, 8 Jul 2011 16:35:47 +0000 (09:35 -0700)]
LU-488 ptlrpc_connection_put() LASSERT(!cfs_hlist_unhashed(&conn->c_hash))

Connection hash may be rehashed while ptlrpc_connection_put() is
called, ASSERT &conn->c_refcount > 1 instead of this.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Iec6d35419e0c4d8497bd0b84c6210abc8eb23882
Reviewed-on: http://review.whamcloud.com/1074
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: zhen liang <liang.zhen@live.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-513 Make cfs_wait_event_interruptible_exclusive really exclusive
Christopher J. Morrone [Wed, 20 Jul 2011 00:43:15 +0000 (17:43 -0700)]
LU-513 Make cfs_wait_event_interruptible_exclusive really exclusive

Change-Id: Iea0556a006f8826f8597824131fb5110a848c434
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1118
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoIncrease lustre version to 2.1.51 2.1.51 2.1.51.0 v2_1_51_0
Oleg Drokin [Tue, 25 Oct 2011 21:41:45 +0000 (17:41 -0400)]
Increase lustre version to 2.1.51

Change-Id: I91b5cc909aae9434349ec1ddbd16fd1eedadc412
Signed-off-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-28: remove lock contention in reconnecting
Jinshan Xiong [Mon, 19 Sep 2011 02:58:21 +0000 (19:58 -0700)]
ORNL-28: remove lock contention in reconnecting

target_handle_connect() used to grab obd::obd_recovery_task_lock to update
obd_next_recovery_transno and obd_connected_clients. In this patch, I revised
this piece of code by:
- modify obd_connected_clients to cfs_atomic_t
- grab obd_recovery_task_lock only if ocd_transno is less than
  obd_next_recovery_transno

In this way, target_handle_connect() don't grab any global lock.

Change-Id: I971897de82566e239fe750cf95f2c3f1646325bc
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1293
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-735 Remove fixme() macro
Andreas Dilger [Mon, 3 Oct 2011 21:16:28 +0000 (14:16 -0700)]
LU-735 Remove fixme() macro

This NULL export warning is the only consumer of the fixme()
macro.  Change it to a CWARN and remove the fixme() macro.

Change-Id: I194a9d92369a6a4dce35701fe261631420f9894d
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1472
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-707 Fix FID string quoting in replay-vbr
Li Wei [Fri, 9 Sep 2011 06:05:56 +0000 (14:05 +0800)]
LU-707 Fix FID string quoting in replay-vbr

The FID string printed by the "path2fid" lfs command was passed to
do_facet() without any quoting.  Depending on the names of the files in
the remote shell's working directory, the FID string could be replaced,
by the shell's filename expansion, to a list of matching file names.
This patch adds the necessary quoting prevent the remote shell from
doing filename expansion on the FID string.

Change-Id: I0bc28eb7f29df717a661a4c1bce8c918117b997c
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1409
Tested-by: Hudson
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-443 LNet: Only squawk when md->start is NULL on non-zero length
Wally Wang [Wed, 24 Aug 2011 18:08:58 +0000 (11:08 -0700)]
LU-443 LNet: Only squawk when md->start is NULL on non-zero length

Only squawk when md->start is NULL on non-zero length.
The md->start == NULL check prevents anyone from creating a ME/MD
with no buffer. These are used as backstop buffers to generate events
when traffic has exceeded the local buffer space.

Change-Id: I1389b0a45d3f8ff548f6400c66b30a69bafb4f39
Signed-off-by: Wally Wang <wang@cray.com>
Reviewed-on: http://review.whamcloud.com/989
Tested-by: Hudson
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-448 add lst stat --count
Wally Wang [Thu, 22 Sep 2011 23:19:28 +0000 (16:19 -0700)]
LU-448 add lst stat --count

This adds lst stat --count, making it easier to script up data
collection for performance tests.

The patch is from Oracle bug 22638 attachment 33001

Change-Id: I46aa1de12ea8e6c2d5eafff8ce5c31d7d618c06f
Signed-off-by: Wally Wang <wang@cray.com>
Reviewed-on: http://review.whamcloud.com/1004
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-357 racer test cleanup
hongchao.zhang [Thu, 23 Jun 2011 03:24:31 +0000 (11:24 +0800)]
LU-357 racer test cleanup
 1, increase the test time to 300s(900s for SLOW)
 2, fixing the problem of recursively calling racer.sh

Change-Id: I91ac7e5c42ed5bc98b3a647c30d7e37af0573f09
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/905
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-743 conf-sanity: test_46a failure
Jinshan Xiong [Fri, 7 Oct 2011 18:27:19 +0000 (11:27 -0700)]
LU-743 conf-sanity: test_46a failure

This failure is because client still didn't see the adding OSTs so it
met a problem when decoding lsm because the # of OSTs was over tgt count
at the client side.

Change-Id: I49ee22379734375a2a92f7495f6d849e47db0909
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1494
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-28: Set recovery timeout correctly
Jinshan Xiong [Fri, 26 Aug 2011 00:30:50 +0000 (17:30 -0700)]
ORNL-28: Set recovery timeout correctly

make sure recovery window uses timeout value from lustre config;
in current implementation this piece of code is totally wrong since
it just disregards timeout configuration.

Change-Id: I0cb0d777569cccd96f30da11834c6e333a673816
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1292
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-28: Solve reconnecting race between IR and SR
Jinshan Xiong [Wed, 24 Aug 2011 23:03:53 +0000 (16:03 -0700)]
ORNL-28: Solve reconnecting race between IR and SR

if there is a connecting request on the fly when client import is
notified by IR, it will set the corresponding conn uuid to a higher
prio and set imp_force_verify so that it will do reconnection
immediately in case RPC timeout happens.

Change-Id: I77e799e1f12b49f3c0271585c1ce812d16dc1ef6
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1291
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-27: Cancel on completion lock on the MGS
Jinshan Xiong [Thu, 6 Oct 2011 20:35:02 +0000 (13:35 -0700)]
ORNL-27: Cancel on completion lock on the MGS

We should cancel the recover/config LCK_EX lock immediately
when they are granted to accelerate enqueue process.

Also, it doesn't make sense to add mgc recover/config lock into
LRU list because these kinds of lock would never be canceled
voluntarily. Restore LDLM_FL_NO_LRU flag and apply it for mgc
lock.

Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Change-Id: I369b57ca4780b0bfa07d33b4423b468481263ade
Reviewed-on: http://review.whamcloud.com/1261
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-14: Configuration and Unit test cases
Jinshan Xiong [Tue, 4 Oct 2011 04:35:24 +0000 (21:35 -0700)]
ORNL-14: Configuration and Unit test cases

In this patch, some procfs and test cases are added for imperative recovery.

Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Change-Id: Ied49b622208157a74e8547e4609d61ee5041f624
Reviewed-on: http://review.whamcloud.com/1219
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-687 clio: retry if fault page was truncated
Jinshan Xiong [Wed, 5 Oct 2011 05:54:48 +0000 (22:54 -0700)]
LU-687 clio: retry if fault page was truncated

In vvp_io_fault_start, if a page was truncated we should retry it in
ll_fault() instead of return -EFAULT because it will cause fake OOM.

Change-Id: Ia2ba40ca4ecd67170e6c2eb81ddc0ee34d9379a8
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1453
Tested-by: Hudson
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-699 tests: replay-dual test_1 failure
Jinshan Xiong [Wed, 19 Oct 2011 23:54:26 +0000 (16:54 -0700)]
LU-699 tests: replay-dual test_1 failure

The root cause of this problem is due to data corruption on lustre_disk_data.
In ldd_write(), it just waits for log to be committed but NOT data to actually
write to disk. So in this test case, if the data is not written into disk when
we mark the device as readonly, the data will be lost and cause remount mdt
failed.

Change-Id: I8c46a925ce2df3d2db69b1e7fd8813eb0668401d
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1557
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
8 years agoORNL-13: NonIR client support
Jinshan Xiong [Thu, 11 Aug 2011 01:19:15 +0000 (18:19 -0700)]
ORNL-13: NonIR client support

In this task, NonIR support is added. NONIR clients means those clients who
doesn't know the protocol of imperative recovery. This means they won't be
notified for the restarting of target.

To support NonIR clients, the MGS has to record how many NonIR clients per
file system, and track those clients. In this way, if there are NonIR clients
for a specific file system, the MGS should tell the restarting target to disable
imperative recovery; otherwise, these `old' clients are to be evicted easily.

Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Change-Id: I3725c66b74d702aa213644ee9a6f89d59b8a8083
Reviewed-on: http://review.whamcloud.com/1218
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-462 Don't alloc/free client data for self export
Mikhail Pershin [Mon, 27 Jun 2011 17:02:14 +0000 (21:02 +0400)]
LU-462 Don't alloc/free client data for self export

Self export doesn't need client data and ldlm initialization.

Change-Id: I31307d2212e3d11c79f1ab215edbb840c3cfb8c6
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1023
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-23 avoid unnecessary CMD lock contention in LMV
nasf [Mon, 5 Sep 2011 04:48:00 +0000 (12:48 +0800)]
ORNL-23 avoid unnecessary CMD lock contention in LMV

There are some redundant lock operations in LMV layer which is only
useful under old CMD mode. We should avoid those lock operations to
reduce unnecessary lock contention.

Change-Id: I2ad1ba728da39ce69b0e521ab3a992dd2b253f2e
Signed-off-by: nasf <yong.fan@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1226
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-515 canonicalize the devices names
Wally Wang [Wed, 20 Jul 2011 03:50:18 +0000 (20:50 -0700)]
LU-515 canonicalize the devices names

Perform a readlink on the device name if path is /dev/disk/by-id...
See Oracle bug 24487.

Change-Id: I964b224d764677d60064901f4238ae77b9cfb5ea
Signed-off-by: Wally Wang <wang@cray.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1120
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-760 obdecho: initialization and awk problem
Jinshan Xiong [Thu, 13 Oct 2011 23:27:08 +0000 (16:27 -0700)]
LU-760 obdecho: initialization and awk problem

echo_client registers obd type and then initializes kmem cache, this is
problematic because echo_key will be immidiately accessed after obd type
is registered. This will cause kernel fault.

Also, an awk problem is fixed. If the length of output buffer is longer
than 1024 bytes, awk will run into problem.

Change-Id: Ief287e8f4eeb6a39bc336e7a9f5c21e921b79a58
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1521
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-389 update packet-lustre.c for master
nasf [Thu, 14 Jul 2011 09:51:30 +0000 (17:51 +0800)]
LU-389 update packet-lustre.c for master

Drop unused mds_body/mds_rec_xxx/mds_status_req. Add missing mdt_body/mdt_rec_xxx.

Change-Id: Ic530541f58d12c721fa6efd0bc9a1096a15d7e33
Signed-off-by: nasf <yong.fan@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/995
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
8 years agoORNL-10: Basic IR implementation
Jinshan Xiong [Fri, 19 Aug 2011 23:44:50 +0000 (16:44 -0700)]
ORNL-10: Basic IR implementation

To support imperative recovery, there is a target status table defined for
each file system defined on the MGS. When a target registers itself to the
MGS, the MGS will change this table correspondingly.

In the status table, one important field is target NID. This NID information
is used by clients locating server where target lives. By transferring this NID
to clients, clients can know the restarting of targets earlier. This is
so-called imperative recovery - the MGS notifies clients to do recovery
imperatively instead of timeout based standard recovery.

To implement imperative recovery, clients are asked to cache a NID table, which
contains the location information of all servers. Clients need to hold a read
mode ldlm plain lock - recover lock - to cache this table. Whenever the MGS
wants to change this table, it will enqueue an EXCL recover lock so that all
clients will be notified for this change. Clients will request for a new read
recover lock and then query for the MGS for NID table updates.

Change-Id: I3b38ba142b810df507805b71972feeb1bade1ac2
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1217
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-666 mdd depends on ldiskfs-only dynlocks feature
Brian Behlendorf [Wed, 24 Aug 2011 17:22:27 +0000 (10:22 -0700)]
LU-666 mdd depends on ldiskfs-only dynlocks feature

As discussed in Lustre bugzilla 22435 dynlocks are provided by
the ldiskfs and are not a generic facility.  They should be
removed from the mdd-level to allow building --without-ldiskfs
per Andreas's comment in 22435:

  https://bugzilla.lustre.org/show_bug.cgi?id=22435#c3

This change converts the instances of "struct dynlock_handle *"
to a "void *" pointer which is safe because they are never
dereferenced at this level.

It also wraps the "struct dynlock" entry in the mdd_object with
MDD_DISABLE_PDO_LOCK.  This effectively limits all direct dynlock
usage to the !MDD_DISABLE_PDO_LOCK case which is unconditionally
disabled in the code.  Presumable this code will simply be removed
at some point which will fully resolve the issue.

Change-Id: I7c8d2fb04b31736782e5f6ea2ff6b631b475fb91
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1348
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-499 grant/cancel_rate are static when OST is idle
Lai Siyao [Fri, 5 Aug 2011 03:43:20 +0000 (20:43 -0700)]
LU-499 grant/cancel_rate are static when OST is idle

ldlm_pool_recalc() shouldn't be skipped if namespace->ns_bref eqauls
zero, instead a flag ns_stopping is added to mark ldlm namespace is
being freed.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Ic6485c34ec3e9868ae531a4dc25aee969c374eb5
Reviewed-on: http://review.whamcloud.com/1185
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-690 test: wait_osc_import_state() fixes
Yu Jian [Sun, 9 Oct 2011 06:36:22 +0000 (14:36 +0800)]
LU-690 test: wait_osc_import_state() fixes

b=24498 wait_osc_import_state() fixes

i=Vladimir.Saveliev

-- increase maxtime to wait the timeout of 1st request;
   take into account at_min value;
-- cleanup wait_osc_import_state() to use _wait_import_state();
-- ost-pools test_1 fix: use local var instead of global NAME

Signed-off-by: Elena Gryaznova <elena.gryaznova@oracle.com>
Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I36b163a291d783b1d772f05898ccee783d815298
Reviewed-on: http://review.whamcloud.com/1499
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-731 ldlm: revise and export ldlm_lock_set_data
Jinshan Xiong [Mon, 3 Oct 2011 22:48:36 +0000 (15:48 -0700)]
LU-731 ldlm: revise and export ldlm_lock_set_data

We can only reset ->l_ast_data if it's NULL; otherwise, returning an
error if ->l_ast_data is trying to be changed to a different value.

Change-Id: I8f999a98101daa2c465256792285582f581909c5
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1466
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-580: update mgc llog process code
Jinshan Xiong [Tue, 9 Aug 2011 02:39:22 +0000 (19:39 -0700)]
LU-580: update mgc llog process code

This patch does the following work:
1. fix a bug about setting mgc lock callback data;
2. define log instance types;
3. simplify and cleanup mgc_requeue_thread code

Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Change-Id: Ic8d25eda9f0a7884b4a8cbeccc89c1e66c19f0ff
Reviewed-on: http://review.whamcloud.com/1198
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-573: conf-sanity test_22 failed with 41
Jinshan Xiong [Sat, 6 Aug 2011 02:32:22 +0000 (19:32 -0700)]
LU-573: conf-sanity test_22 failed with 41

Make sure recovery on OST is finished before trying to create file

Change-Id: I4a36685a5cd9c55de729906bff50c29b1108c931
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1192
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-433 remove jbd2-jcberr patch from kernel
Bobi Jam [Tue, 21 Jun 2011 01:07:57 +0000 (09:07 +0800)]
LU-433 remove jbd2-jcberr patch from kernel

In the upstream ext4 code there is a new functionality to allow a
per-commit callback to be set (j_commit_callback), which is used by
the mballoc code to manage free space bitmaps after deleted blocks
have been released. We expand it to contain multiple different
callbacks.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I6397ccabd8d729658cf2ee13c9c3731a9eb31219
Reviewed-on: http://review.whamcloud.com/983
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-553 build: Lustre style commit hooks for Git
Andreas Dilger [Thu, 25 Aug 2011 21:08:04 +0000 (15:08 -0600)]
LU-553 build: Lustre style commit hooks for Git

Adds recommended commit hooks for Lustre commit messages, that
can/should be installed into lustre/.git/hooks for all Lustre trees.

prepare-commit-msg: adds the output of checkpatch.pl as a
    comment to the commit message, to "soft" validate the new
    patch is matching the Lustre coding style. It doesn't check
    100% of the Lustre coding style, but at least finds most of
    the obvious errors so that they can be fixed locally before
    any manual inspection time is spent on the patch.

commit-msg: verifies the format of the commit message (somewhat).
    Checks that there is a valid-looking Signed-off-by: line.
    Adds the Gerrit Change-Id: line for change tracking.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I35fdfb8206d1f0efb3bb401522e8412c62b7c02a
Reviewed-on: http://review.whamcloud.com/1296
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoStart of 2.2 development, updated version to 2.1.50 2.1.50 v2_1_50_0
Oleg Drokin [Wed, 5 Oct 2011 08:40:41 +0000 (04:40 -0400)]
Start of 2.2 development, updated version to 2.1.50

Change-Id: Ibaca591c4db2b839ece2419b77404cdcd869d843
Signed-off-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-459 quiet too noisy console messages at mount
Andreas Dilger [Mon, 22 Aug 2011 22:56:45 +0000 (16:56 -0600)]
LU-459 quiet too noisy console messages at mount

Quiet a number of extra debug messages printed to the console after a
remount or recovery.  They provide no value and just add to the general
confusion of reading Lustre debug messages.

Change-Id: Ia0b43bbd337501169dec234ff66dd56f67f75131
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1275
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-14: debug: print recovery completion message
Andreas Dilger [Tue, 23 Aug 2011 21:31:58 +0000 (15:31 -0600)]
ORNL-14: debug: print recovery completion message

Print a proper "end of recovery" message to match the one printed
by 1.x so that it is clear when recovery has finished, how long it
took, and how many clients recovered.

This should have been part of the original patch committed for
eda8eab5abeef1117ae1082533b5888a91875e12.

Change-Id: I3a8793fdcb3c665e8df0d39ae82b4c56fb1bb527
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1278
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-258 tests: delete obsolete filter_survey script
Andreas Dilger [Wed, 21 Sep 2011 23:34:12 +0000 (17:34 -0600)]
LU-258 tests: delete obsolete filter_survey script

The filter_survey.sh was a very early version of obdfilter-survey.sh
script, but is long obsolete (last updated 2003) and can be removed.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2fa46d393355ab35352ac693272071424d1a57e1
Reviewed-on: http://review.whamcloud.com/1406
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-610 fix the wrong data type in find_value_cmp
hongchao.zhang [Fri, 2 Sep 2011 08:57:18 +0000 (16:57 +0800)]
LU-610 fix the wrong data type in find_value_cmp

in function find_value_cmp (lustre/utils/liblustreapi.c), the data type for
paramters "file" and "limit" is converted from "unsigned long long" type into
"unsigned int" type by mistake, and the file size should be defined as
"unsigned long long" type too.

Change-Id: Icacebccdf55d8ad761390bb03aa9aab0fdc67131
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1289
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-241 Support crc32c with hardware accelerated instruction as one of lustre checksums
Shuichi Ihara [Thu, 23 Jun 2011 13:08:39 +0000 (22:08 +0900)]
LU-241 Support crc32c with hardware accelerated instruction as one of lustre checksums

Adding CRC32C as new lustre checksum algorithm. Because of crc32c,
the Intel Nehalem based CPU supports h/w accelerated crc32c
instruction as one of SSE4.2 instruction set. The new crc32c codes
in the lustre also use this h/w instruction. The lustre also
automatically detects whether crc32c instruction is available, if
not, use adler (fastest checksum other than crc32c).

Change-Id: I764851a46a94a879239cd127eac411e98342e67f
Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-on: http://review.whamcloud.com/1009
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-110 Recognize old mdt.quota_type param
Niu Yawei [Wed, 23 Mar 2011 06:30:33 +0000 (23:30 -0700)]
LU-110 Recognize old mdt.quota_type param

Minor changes in mdt_process_config() to make it capable of recognizing old mdt.quota_type.

Issue:
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ie13851eee573eb633a68bf5ee3cd86dbbf1bd1a9
Reviewed-on: http://review.whamcloud.com/354
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-570: Add function to find connect uuid by nid
Jinshan Xiong [Wed, 10 Aug 2011 19:30:20 +0000 (12:30 -0700)]
LU-570: Add function to find connect uuid by nid

In this patch, two functions are added:
- class_find_uuid(), find conn uuid by peer nid
- client_import_find_conn(), find a conn uuid in import connection list

Also, a code cleanup is performed.

Change-Id: I50e8e9392a39ef78719504cf083c0c22f5d39dcb
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1189
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-641 obdclass: use rwlock for obd_dev_lock
Jinshan Xiong [Thu, 25 Aug 2011 20:32:02 +0000 (13:32 -0700)]
LU-641 obdclass: use rwlock for obd_dev_lock

In target_handle_connect() it grabs obd_dev_lock to find obd_device
by target uuid. This causes lock contention if there are two many connection
requests on the same time. Based on obd_dev_lock is a read-mostly lock, it
is much reasonable to revise it as a rwlock.

Change-Id: I7f936d6182df9e7ed46b8a1b6fb259226188acbc
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1294
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-574: conf-sanity test_41b failed with 1
Jinshan Xiong [Sat, 6 Aug 2011 03:18:45 +0000 (20:18 -0700)]
LU-574: conf-sanity test_41b failed with 1

Do not run this case if the MGS and MDS are on different devices.

Change-Id: I3b3839a0860dbf55ba1e066e02358abc131f643a
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1193
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-572 ptlrpc: make ptlrpc_recover_import() async
Jinshan Xiong [Fri, 5 Aug 2011 23:48:29 +0000 (16:48 -0700)]
LU-572 ptlrpc: make ptlrpc_recover_import() async

Add an async argument for ptlrpc_recover_import() and with async nonzero,
ptlrpc_recover_import() will return immediately after sending connecting
RPC.

This is needed for imperative recovery since it may recover multiple
targets meanwhile.

Change-Id: Iccc40fca4d3d5d3fb0b2965e62beda23e2fb3b4c
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1191
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-571 ldlm: Remove parallel AST limitation
Jinshan Xiong [Thu, 11 Aug 2011 23:56:06 +0000 (16:56 -0700)]
LU-571 ldlm: Remove parallel AST limitation

Imperative Recovery needs to notify clients as soon as possible.
However, limiting the AST count to 200 will definitely block it.
This may result in the recovery window has passed before the MGS
notifies all clients.

In this patch, I revised the above limitation to be configurable,
and it is no limit by default.

Change-Id: I141da9dbd8b282a76252da15b93c432ac5253bfd
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1190
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-569: Make lu_object cache size adjustable
Jinshan Xiong [Fri, 5 Aug 2011 19:24:33 +0000 (12:24 -0700)]
LU-569: Make lu_object cache size adjustable

lu_object cache is specified to consume 20% of total memory. This
limits 200 clients at most can be mounted on one node. We should make
it adjustable so that customers have a chance to configure it by
their needs.

Change-Id: I8dcb993c88a3abd9fc8ba11ff1578aa3897c3933
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1188
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-721 llite: Parallel writes to same file results in zero file
Bobi Jam [Mon, 3 Oct 2011 04:44:37 +0000 (12:44 +0800)]
LU-721 llite: Parallel writes to same file results in zero file

Canceling lock may contain data being sent to OSTs. Change
find_cbdata iterator to take that into account.

Author: Vladimir Saveliev <vladimir.saveliev@oracle.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1b30fe13ee5303fe3b98e45ad969f5273c41d676
Reviewed-on: http://review.whamcloud.com/1460
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoUpdate release date for 2.1 in Changelog to reflect RC2 2.1.0 2.1.0-RC2 v2_1_0_0 v2_1_0_0_RC2
Oleg Drokin [Fri, 16 Sep 2011 17:20:21 +0000 (13:20 -0400)]
Update release date for 2.1 in Changelog to reflect RC2

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: Ifda31666cac43cb964184903a3d3de20f671cb7d

8 years agoLU-683: handle short write in cl_io_commit_write
Jinshan Xiong [Thu, 15 Sep 2011 05:49:21 +0000 (22:49 -0700)]
LU-683: handle short write in cl_io_commit_write

CLIO has to handle short write case where no bytes were
actually copied from user space. In this case, we should
avoid adding that page into cache.

Change-Id: I78c870e741d7bdb2a9c4c0bfba1adc50a6e814e7
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1383
Tested-by: Hudson
Tested-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
8 years agoLU-653 i_version shouldn't be used for VBR
Mikhail Pershin [Wed, 7 Sep 2011 06:26:31 +0000 (10:26 +0400)]
LU-653 i_version shouldn't be used for VBR

RHEL6 plus EXT4 combination cause i_version is to be used to store
Lustre versions instead of i_fs_version. Meanwhile this field is used
by ext4 internally and may be changed breaking Lustre recovery.

Patch makes only i_fs_version to be used for versions.

Change-Id: Idf7cd257aa2d288abce2a7c18c4d8b139bc9c5c6
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1342
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-653 Ignore last transno from clients with no outstanding transactions
Oleg Drokin [Thu, 1 Sep 2011 20:19:11 +0000 (16:19 -0400)]
LU-653 Ignore last transno from clients with no outstanding transactions

Clients that did not have any unreplied transactions should not
artificially reduce next_recovery_transno since those transactions are
never coming anyway and would just elay all replay activity
until all clients are reconnected.

Change-Id: Iafa32b315211ada93838f4b16fe0f2c800d619c4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1318
Tested-by: Hudson
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-662 fix conflict between seq_client_flush and seq_client_alloc_fid
Hongchao Zhang [Fri, 9 Sep 2011 11:00:39 +0000 (19:00 +0800)]
LU-662 fix conflict between seq_client_flush and seq_client_alloc_fid

seq_client_flush modifies the lu_client_seq.lcs_space by holding the
lu_client_seq.lcs_sem, but seq_client_alloc_fid uses lcs_update to
exclude other concurrent modification.

Change-Id: Ibc3335841b9a045360283e15aab9b6c36b7a39af
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1364
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-650 Max possible file size is mutable
Bobi Jam [Sun, 2 Oct 2011 05:18:59 +0000 (13:18 +0800)]
LU-650 Max possible file size is mutable

Move maxbytes fields out of wire data part of lov_stripe_md,
it can change when OSTs change their status, e.g. disconnected
or inactivated.

Change-Id: I9fb1f2597df1e98d993792706ac94f469d23a78c
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1327
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-614 tests: speed up needlessly long tests
Andreas Dilger [Fri, 26 Aug 2011 08:26:51 +0000 (02:26 -0600)]
LU-614 tests: speed up needlessly long tests

Fix unlinkmany to use the original $next_id instead of the one
seen after createmany was run.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I816081ab1ee04fe66d6c14b76eb076262215bc32
Reviewed-on: http://review.whamcloud.com/1298
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>