Whamcloud - gitweb
fs/lustre-release.git
11 years agoLU-1493 quota: extra release caused by race
Niu Yawei [Mon, 11 Jun 2012 10:55:32 +0000 (03:55 -0700)]
LU-1493 quota: extra release caused by race

There is a race between the check_cur_qunit() and the
dqacq_completion(): check_cur_qunit() read hardlimit
and calculate how much quota need be acquired/released
based on the hardlimit, however, the hardlimit can be
changed by the dqacq_completion() at anytime. So that
could result in extra quota acquire/release when there
is inflight dqacq.

In general, such extra dqacq dosen't bring fatal error,
unless an extra release is going to release more than
'hardlimit' quota.

To minimize the code changes (anyway, it'll be totally
rewritten in the new quota design), we just do one more
check here to avoid the extra release which could bring
fatal error. A better solution could be calculating the
qd_count here and removing the lqs_blk/ino_rec stuff.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I0ad5ff0f32e39f32872c201ad1d545fbd9d1a57d
Reviewed-on: http://review.whamcloud.com/3074
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
11 years agoLU-1438 quota: quota active checking is missed on slave
Niu Yawei [Wed, 13 Jun 2012 02:42:01 +0000 (19:42 -0700)]
LU-1438 quota: quota active checking is missed on slave

On quota slave, we missed checking if quota is enabled in the
quota_check_common() and several other places. Which could cause
slave retry acquire quota in quota_chk_acq_common() infinitely
when the quota is already turned off on master.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Iaa48c7cca05daf595b6d3b7e4025c7650e460918
Reviewed-on: http://review.whamcloud.com/3097
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
11 years agoLU-957 scrub: osd-ldiskfs itable based iteration
Fan Yong [Thu, 14 Jun 2012 07:40:39 +0000 (15:40 +0800)]
LU-957 scrub: osd-ldiskfs itable based iteration

Implement inode table based object iteration in osd-ldiskfs.
It is implemented as DT iteration APIs, which can be used by
up layer LFSCK to scan the whole device sequentially.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I3d80bb4a174d47429764e5cca35e4f07be52d50b
Reviewed-on: http://review.whamcloud.com/2553
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: code cleanup for lib-move.c
Liang Zhen [Wed, 6 Jun 2012 14:15:59 +0000 (22:15 +0800)]
LU-56 lnet: code cleanup for lib-move.c

Most changes in this patch are just code cleanup:
- remove one unnecessary lock dance for message forwarding
- move some code blocks to make functions cleaner
- rename lnet_ni_peer_alive to lnet_ni_query_locked

It's an intermediate patch LNet SMP improvements.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ia7cfe37a5a4f896be4fcc7f6c5cf9c27268de9ba
Reviewed-on: http://review.whamcloud.com/3048
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: match-table for Portals
Liang Zhen [Tue, 5 Jun 2012 10:02:55 +0000 (18:02 +0800)]
LU-56 lnet: match-table for Portals

Create sub-object named as "match-table" for each Portal, MEs will
be attached match-table instead of Portal.
Although we only have one match-table for each Portal in this patch,
but in upcoming changes, we will create multiple match-tables
for each Portal:
- unique-match Portal
  MEs will be scattered to different match-tables by match info
- wildcard Portal
  LND threads just grab ME/MD from match-table corresponding to
  current CPT (CPU partition).

We also did some code cleanup for delayed message in this patch.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I2b24723c3bd2a6664f2b241840de19d5f43be11f
Reviewed-on: http://review.whamcloud.com/3043
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: code cleanup for lib-md.c
Liang Zhen [Sat, 2 Jun 2012 05:45:32 +0000 (13:45 +0800)]
LU-56 lnet: code cleanup for lib-md.c

This patch just did some code cleanup for lib-md.c, it's an
intermediate patch for LNet SMP improvements.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5d162852008284ae08f3b9e7935c9251b245fb5c
Reviewed-on: http://review.whamcloud.com/3010
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: split lnet_commit_md and cleanup
Liang Zhen [Fri, 1 Jun 2012 05:53:01 +0000 (13:53 +0800)]
LU-56 lnet: split lnet_commit_md and cleanup

Briefly, LNet has two types of operations: 1) resource operations
on EQ, MD and ME; 2) network operations on credits, peer, NI etc.
Most of them are in different logic blocks, but there are still
a few functions have mixed operations, lnet_commit_md() is one of
these functions.

This patch is a key step for LNet SMP improvements, lnet_commit_md()
of current LNet did two things: 1) consume MD threshold and commit
it to network; 2) attach the message on active message list to show
it might have taken LNet message credits. This patch split it into
two sets of functions, lnet_msg_attach/detach_md and
lnet_msg_commit/decommit,

We need to call lnet_msg_attach_md and lnet_msg_comit before deliver
a message to network, call lnet_msg_detach_md and lnet_msg_decommit
while finalizing a message. These two sets of functions will be
protected by different locks in upcoming patches.

Another change in this patch is, we moved most counters operations
into lnet_msg_decommit, this will make code cleaner and easier for
the next step work. The reason that we put counters operations in
lnet_msg_decommit instead of lnet_msg_commit is because we can't
guarantee lnet_msg_commit is always called after lnet_msg_attach_md,
and counters need information filled by lnet_msg_attach_md, which
means lnet_msg_commit might not have enough information for counters.
The thing we know for sure is lnet_msg_decommit is always called
after lnet_msg_attach_md, so it's safe to put counter operations
inside lnet_msg_decommit.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Iee961fbff6e6e39ed76c14c0bd5b6560a7730af9
Reviewed-on: http://review.whamcloud.com/2997
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1095 debug: fix missing CDEBUG() newline
Andreas Dilger [Tue, 19 Jun 2012 04:31:24 +0000 (22:31 -0600)]
LU-1095 debug: fix missing CDEBUG() newline

The console message cleanup 389fde827be2ee6fb4ee08e955d773a2a16e70c6
lost a newline in one of the messages, which itself generates a
warning on the console.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie5eed2977e496e082d5b8e62bfc39e0df93fcab0
Reviewed-on: http://review.whamcloud.com/3134
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1030 test: delete test file at the end of test script
Jinshan Xiong [Thu, 21 Jun 2012 05:59:49 +0000 (13:59 +0800)]
LU-1030 test: delete test file at the end of test script

At sanity.sh:test_49, we need to delete the test file after test is
finished otherwise the following test cases may fail due to lack of
storage capacity.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: Iec9e93f1a4844156345608bf9f7a9bf679b36253
Reviewed-on: http://review.whamcloud.com/3158
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1305 osd: procfs support for osd-zfs
Alex Zhuravlev [Tue, 29 May 2012 10:47:56 +0000 (14:47 +0400)]
LU-1305 osd: procfs support for osd-zfs

procfs support

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I599d1317f762b50965f760a590f2f99ffd86bef5
Reviewed-on: http://review.whamcloud.com/2965
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1305 obdclass: lprocfs helpers
Alex Zhuravlev [Tue, 12 Jun 2012 06:51:47 +0000 (10:51 +0400)]
LU-1305 obdclass: lprocfs helpers

few lprocfs helpers to dump statfs data via OSD API

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If90c410c9155f480baa8d344a670cc1a48077f07
Reviewed-on: http://review.whamcloud.com/3089
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: LNet message event cleanup
Liang Zhen [Wed, 30 May 2012 07:38:43 +0000 (15:38 +0800)]
LU-56 lnet: LNet message event cleanup

In current version, LNet message events are assigned at many places,
it's not easy to maintain, also, assignments of some fields of event
need to be protected by lock and some don't need, this will be a
problem for upcoming LNet SMP improvements.
This patch created a new function lnet_build_msg_event(), all events
are assigned inside this function and it can be called w/o lock
protection.
This is an intermediate patch for LNet SMP improvements.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ief8722a15d17909e3844149c55d24fb22ed0c04c
Reviewed-on: http://review.whamcloud.com/2983
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: eliminate a few locking dance in LNet
Liang Zhen [Tue, 29 May 2012 07:34:13 +0000 (15:34 +0800)]
LU-56 lnet: eliminate a few locking dance in LNet

This patch gets rid of a few unnecessary locking dance from
LNet, also changed type of lnet_ni_t::ni_last_alive from jiffy to
second because per-second checking should be good enough.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I0e3269f0a5ae0cfecd9611584d5e213a670e38f6
Reviewed-on: http://review.whamcloud.com/2933
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: parse RC ping in event callback
Liang Zhen [Tue, 29 May 2012 06:15:39 +0000 (14:15 +0800)]
LU-56 lnet: parse RC ping in event callback

In current LNet, we don't parse router-checker ping in EQ callback,
NI information from remote side is attached on peer and it's parsed
for each sending. This is OK for now but with upcoming SMP
improvements, the global LNET_LOCK is removed and EQ callback and
peer/router are proteced by different locks, which means sink buffer
of RC ping could be in progressing of change when other thread is
reading it in lnet_send(), and get wrong information.

This patch will parse ping information in EQ callback, and store it
in a new member lnet_route_t::lr_downis which is just a integer,
although reader in lnet_send() still don't have lock while reading
it, but it's OK because it's just an integer and reader can
atomically read reliable result even w/o any lock.

It should also be a little faster than original implememtation
because we don't have to parse RC ping, which should be updated
once for many seconds, for each sending.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5b89fabfd22f22ba171ef63d30da2d32e2788b16
Reviewed-on: http://review.whamcloud.com/2932
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: router-checker (RC) cleanup
Liang Zhen [Tue, 29 May 2012 03:05:55 +0000 (11:05 +0800)]
LU-56 lnet: router-checker (RC) cleanup

This patched covered a couple of things:
- remove global router-checker data
  In current LNet we can have both global RC data and per-peer
  RC data, the later is only for asymmetrical router failure so far,
  however we can use per-peer RC data even asym_router_failure is
  disabled, it's just increase a little memory overhead but will
  simplify router checker.
  Also, I'm thinking that asym_router_failure is a nice feature and
  it should be enabled by default in the future.

- Code cleanup for router checker, again, it's still an intermediate
  patch for LNet SMP improvements, we need many code cleanup before
  we change locking mechanism of LNet.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I830e706ce5099f15f122b521f74b6dfcde443592
Reviewed-on: http://review.whamcloud.com/2930
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1534 osd: Fix LBUGs when destroying IGIF objects
Li Wei [Sun, 17 Jun 2012 05:16:47 +0000 (13:16 +0800)]
LU-1534 osd: Fix LBUGs when destroying IGIF objects

When destroying MDT objects with IGIF FIDs, the following assertion
may occur:

  LustreError: 28147:0:(osd_internal.h:665:osd_fid2oi()) ASSERTION(
  !fid_is_igif(fid) ) failed:
  LustreError: 28147:0:(osd_internal.h:665:osd_fid2oi()) LBUG
  Pid: 28147, comm: mdt_00

With call traces like this:

  osd_oi_delete()
  osd_object_destroy()
  mdd_object_kill()
  mdd_finish_unlink()
  mdd_unlink()

This patch fixes osd_oi_delete() to avoid trying to delete IGIF FIDs
from OI.  Also, the assertion is enhanced to print more debugging
information.

Signed-off-by: Li Wei <liwei@whamcloud.com>
Change-Id: I95fe9db8aec53a496a8dc84c87594ed81f3fe106
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/3120
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
11 years ago2.2.57 tag 2.2.57 v2_2_57 v2_2_57_0
Oleg Drokin [Tue, 19 Jun 2012 20:47:19 +0000 (16:47 -0400)]
2.2.57 tag

Change-Id: I75ef5cf27c4cd7c0c4dd4b1885f4838b5da39002

11 years agoLU-1428 ldlm: fix a race in ldlm_lock_destroy_internal
Liang Zhen [Tue, 5 Jun 2012 08:34:34 +0000 (16:34 +0800)]
LU-1428 ldlm: fix a race in ldlm_lock_destroy_internal

ldlm_lock::l_exp_hash should be protected by internal lock of
cfs_hash, but we called cfs_hlist_unhashed(lock::l_exp_hash)
w/o holding cfs_hash lock in ldlm_lock_destroy_internal,
which means if someone called ldlm_lock_cancel on a lock while
export::exp_lock_hash is in progress of rehashing (thread context of
cfs_workitem), there could be tiny window between deleting this lock
from bucket[A] and re-adding it to bucket[B] of l_exp_hash, and
cfs_hlist_unhashed(lock::l_exp_hash) will return 1 in this window,
then we destroyed a lock but left it on l_exp_hash forever because
lock::l_destroyed has been set to 1 and ldlm_lock_destroy_internal()
wouldn't be able to remove the lock from l_exp_hash even it's called
infinite times in ldlm_cancel_locks_for_export_cb().

This patch also added some debug information to
ldlm_cancel_locks_for_export_cb in case this patch can't fix this
problem.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ia0932658b3f085a55535e36bee4fb833e74fa242
Reviewed-on: http://review.whamcloud.com/3028
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1518 mdd: Revert "Fixup mdd_{obf,dot_lustre}_obj_ops."
Johann Lombardi [Tue, 19 Jun 2012 08:44:29 +0000 (04:44 -0400)]
LU-1518 mdd: Revert "Fixup mdd_{obf,dot_lustre}_obj_ops."

This reverts commit cf0d73004a20923c690591c1f3f61a8c872ad86e

Reverting this patch due to build failure (moo_version_get not defined).

Change-Id: I7cac5c6d6de2924d46c3a74ed4aa138f50501a16
Reviewed-on: http://review.whamcloud.com/3137
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Johann Lombardi <johann@whamcloud.com>
11 years agoLU-1371 ldiskfs: fix osd_it_iam_rec() to support binary keys
Johann Lombardi [Thu, 12 Apr 2012 23:29:49 +0000 (01:29 +0200)]
LU-1371 ldiskfs: fix osd_it_iam_rec() to support binary keys

osd_it_iam_rec() assumes a string key and does not support binary key.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I1a09c9b9ad53e593cad642b6ae8a7d417ea83c9d
Reviewed-on: http://review.whamcloud.com/2647
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1518 mdd: Fixup mdd_{obf,dot_lustre}_obj_ops.
John L. Hammond [Wed, 13 Jun 2012 16:20:12 +0000 (11:20 -0500)]
LU-1518 mdd: Fixup mdd_{obf,dot_lustre}_obj_ops.

Define several missing md_object ops for .lustre/fid.  Unify
attribute handling for .lustre with that of normal md_objects.

Change-Id: I892904af4ef01b3687e60a41250b018d67c86734
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/3103
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-988 ost: add debugging for inode timestamps
Bobi Jam [Sat, 9 Jun 2012 10:12:51 +0000 (18:12 +0800)]
LU-988 ost: add debugging for inode timestamps

Intermittent test failures for sanity.sh test_39j are showing the
inode timestamps going backward. Add internal debugging to track the
LVB timestamps to see where the old timestamps are coming from, to
determine if this is a problem with the VM environment being used for
testing, or if it is a genuine problem in the Lustre code.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I590d8b28b7a37010af6a43d329917022b840d074
Reviewed-on: http://review.whamcloud.com/3069
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1305 libcfs: S_OSD subsystem
Alex Zhuravlev [Tue, 12 Jun 2012 06:45:57 +0000 (10:45 +0400)]
LU-1305 libcfs: S_OSD subsystem

new definition for osd components

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia224dbd09caecb5cc31488627bb6a958b680377a
Reviewed-on: http://review.whamcloud.com/3088
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1030 osc: new IO engine implementation
Jinshan Xiong [Tue, 12 Jun 2012 00:03:56 +0000 (17:03 -0700)]
LU-1030 osc: new IO engine implementation

New IO engine to manage dirty pages with osc_extent.

Osc_extent is a data structure to manage a series of contiguous
blocks; however, the pages in an extent is not required to be
contiguous. An extent must be written out in one RPC.

The purpose of introducing extents are:
1. make grants work for extent-based OSD;
2. form better IO by picking up contiguous pages to compose RPC;
3. reimplement ll_writepages() with CIT_FSYNC.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I3ef619c1f07eefd201236ab55e5fd858791d41e0
Reviewed-on: http://review.whamcloud.com/2270
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1014 mountconf: Revert "MGS should process parameter config"
Lai Siyao [Wed, 23 May 2012 05:48:49 +0000 (13:48 +0800)]
LU-1014 mountconf: Revert "MGS should process parameter config"

This reverts commit 6cde8692384a1d3a6d0e8114629fa943c951fe40.

Previous commit is wrong, will fix it in another patch.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I5868cb1908f9b123db5b8bd7be9e364ae9a51d6a
Reviewed-on: http://review.whamcloud.com/2843
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1305 osd: quota support for osd-zfs
Alex Zhuravlev [Tue, 29 May 2012 10:54:43 +0000 (14:54 +0400)]
LU-1305 osd: quota support for osd-zfs

export of accouting information and enforcement

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iac5100960a5107277ccd628ba3a682cb9b03c0ed
Reviewed-on: http://review.whamcloud.com/2970
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1305 defs: zfs definitions
Alex Zhuravlev [Fri, 25 May 2012 11:56:42 +0000 (15:56 +0400)]
LU-1305 defs: zfs definitions

couple definitions required for osd-zfs

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I994ab7353edef7f4ccc39707534e7c262e635af3
Reviewed-on: http://review.whamcloud.com/2962
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1406 ofd: add OBD methods to handle OST requests
Mikhail Pershin [Thu, 24 May 2012 08:52:22 +0000 (12:52 +0400)]
LU-1406 ofd: add OBD methods to handle OST requests

Finally add OBD handlers to process incoming requests.
Add stub for quota functions, quota is non-functional in OFD.
Fix couple minor issues:
- slot checking in ofd_last_rcvd_update() must check lr_idx
- remove osd_write_locked() assertion from __osd_xattr_set(),
  it is too excessive there

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Iab270cdf5f2ad41df86845190c5c6db3436279c8
Reviewed-on: http://review.whamcloud.com/2894
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-1406 ofd: IO operations
Mikhail Pershin [Wed, 23 May 2012 19:00:33 +0000 (23:00 +0400)]
LU-1406 ofd: IO operations

add IO functions to OFD

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Ie78e92f5f770ff07c94bf87b2d5ee8300673d271
Reviewed-on: http://review.whamcloud.com/2893
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-1406 ofd: add DLM and LVB code
Mikhail Pershin [Wed, 23 May 2012 19:49:51 +0000 (23:49 +0400)]
LU-1406 ofd: add DLM and LVB code

OFD dlm and lvb functions

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Ieddd2b16129f62c6e380d007aaa1f2de70e8a9ca
Reviewed-on: http://review.whamcloud.com/2892
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-1305 osd: lnb_file_offset in struct niobuf_local
Alex Zhuravlev [Wed, 23 May 2012 07:57:13 +0000 (11:57 +0400)]
LU-1305 osd: lnb_file_offset in struct niobuf_local

introduced in orion so that every buffer can be marked
with a specific offset.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ic8c69fd401300107d589a093023ffe41c0f0eddd
Reviewed-on: http://review.whamcloud.com/2884
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1305 osd: new index flag DT_IND_RANGE
Alex Zhuravlev [Wed, 23 May 2012 07:55:01 +0000 (11:55 +0400)]
LU-1305 osd: new index flag DT_IND_RANGE

introduced in orion to recognize whether the index support
range lookups (used by fldb)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I40efa613697ebe8394557d5f687132ca9f542859
Reviewed-on: http://review.whamcloud.com/2883
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years ago2.3.56 tag 2.2.56 v2_2_56 v2_2_56_0
Oleg Drokin [Mon, 18 Jun 2012 23:14:15 +0000 (19:14 -0400)]
2.3.56 tag

Change-Id: Ia76ec8d6266348c1135292d6de4ded530e6a3889

11 years agoLU-1031: add grouplock test to sanity.sh
jcl [Sun, 5 Feb 2012 21:37:06 +0000 (22:37 +0100)]
LU-1031: add grouplock test to sanity.sh

add a simple grouplock test to sanity.sh (test_82)
replace hardcoded use of multiop by a $MULTIOP in
test scripts.

Change-Id: I94ba2248b7efafe258873c468e6d0cabdfcb7d09
Signed-off-by: jcl <jacques-charles.lafoucriere@cea.fr>
Reviewed-on: http://review.whamcloud.com/2098
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1513 libcfs: call cfs_alloc with proper masks
Fan Yong [Sat, 16 Jun 2012 07:16:55 +0000 (15:16 +0800)]
LU-1513 libcfs: call cfs_alloc with proper masks

Do not use kernel GFP masks directly when call cfs_alloc,
instead, Lustre special flags can be used to control the
allocator's behavior.

cfs_alloc will convert these Lustre special flags to some
kernle GFP masks, then call kmalloc with these GFP masks.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I3189e143ff0cf65a08a1bdf2b8476ab151dc308e
Reviewed-on: http://review.whamcloud.com/3118
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 ptlrpc: common code to validate nthreads
Liang Zhen [Wed, 23 May 2012 04:03:08 +0000 (12:03 +0800)]
LU-56 ptlrpc: common code to validate nthreads

Add a common function to validate ptlrpc service threads number,
instead of checking it before registration of each service.
This function (ptlrpc_server_nthreads_check) will be enhanced
soon for CPT affinity ptlrpc service.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5db09396f4a95e36d8f5d38777c51eafb7dea5b1
Reviewed-on: http://review.whamcloud.com/2879
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-482 tests: cleanup clients/OSTs after MDS error
Andreas Dilger [Mon, 4 Jun 2012 03:59:36 +0000 (21:59 -0600)]
LU-482 tests: cleanup clients/OSTs after MDS error

If the MDS fails to mount due to LU-482, ensure that the clients
and OSTs are unmounted cleanly.  Otherwise, the normal unmount
will hang on the clients as they wait for the MDS to be recovered.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iccd8f81f714974063eaeec45ffbd80f9cef3242b
Reviewed-on: http://review.whamcloud.com/3019
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-827 tests: fix user ID for sanityn test_60
Andreas Dilger [Mon, 4 Jun 2012 20:58:42 +0000 (14:58 -0600)]
LU-827 tests: fix user ID for sanityn test_60

If sanityn.sh test_60() is run on a test system before any of the
quota users are configured, the test will fail.  The test does not
require that any specific user ID is used, so instead use RUNAS_ID
(which is a numeric value and does not even need to be defined in
/etc/passwd for this test to work).

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iaca88344e0dc615f44417e4175e5cddd146c2a7a
Reviewed-on: http://review.whamcloud.com/3021
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
11 years agoNew tag 2.2.55 2.2.55 v2_2_55 v2_2_55_0
Oleg Drokin [Thu, 14 Jun 2012 21:06:46 +0000 (17:06 -0400)]
New tag 2.2.55

Change-Id: I61c9699e1a557d3d5c0848c5e8628f70d256c280
Signed-off-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1286 obdclass: oops in lprocfs_remove_nolock
wangdi [Thu, 5 Apr 2012 06:21:31 +0000 (23:21 -0700)]
LU-1286 obdclass: oops in lprocfs_remove_nolock

Move lov /proc entry cleanup from lov_precleanup to
lov_cleanup, so the lov proc entry will be removed after
the pool has been removed, otherwise it will
cause oops during pool proc entry deletion.

Signed-off-by: Di Wang <di.wang@whamcloud.com>
Change-Id: I1d1b03d02348caaa1e421aa1de1535c8cf0ba3f4
Reviewed-on: http://review.whamcloud.com/2450
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1421 clio: check cp_owner of cl_page after the page is owned
Jinshan Xiong [Tue, 5 Jun 2012 04:32:02 +0000 (21:32 -0700)]
LU-1421 clio: check cp_owner of cl_page after the page is owned

After page writeback is supported in clio, there exists a short
period that page is unlocked but cp_owner is set. This confuses
the assertion of cl_page_assume(). This problem can be fixed by
checking cp_owner after the page is owned.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: Iafcdbb9b0dbb5f081a84c1da1b7ad8051d406988
Reviewed-on: http://review.whamcloud.com/3027
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-506 kernel: some other minor cleanup of BKL
Liu Xuezhao [Wed, 30 May 2012 12:44:05 +0000 (20:44 +0800)]
LU-506 kernel: some other minor cleanup of BKL

Remove lock_kernel/unlock_kernel from cfs_daemonize.
Remove HAVE_UNLOCKED_IOCTL check as it is defined after 2.6.12.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: I6e41c0c3fa76c33dae4f820a63bd3ba62ff704ec
Reviewed-on: http://review.whamcloud.com/2950
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-506 kernel: FC15 - support dcache scalability changes.
yangsheng [Fri, 25 May 2012 18:12:36 +0000 (02:12 +0800)]
LU-506 kernel: FC15 - support dcache scalability changes.

* dcache supports rcu-walk, but lustre doesn't, so
revalidate/check_acl/permission fall back to ref-walk.
* dcache_lock is completely removed in latest kernel, many
fields are protected by inode->i_lock now.
* remove super hack d_rehash_cond(), and treats
DCACHE_LUSTRE_INVALID similar to DCACHE_DISCONNECTED, therefore
dentry doesn't need to be dropped and rehashed frequently.
* .lookup(LOOKUP_CREATE) calls d_add() dentry directly, and
.create only needs to d_instantiate() this dentry.
* set sb->s_d_op to ll_d_ops, and d_alloc() will initialize
dentry with it, therefore it's no need to set dentry operations
explicitely for new kernel.
* .d_delete is called before dentry refcount decrease for new
kernel.
* other cleanups.

Change-Id: Ib08bd4a45dba8d3d5b6150e9a4df95728fdf33e2
Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Signed-off-by: yang sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1865
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoRevert "LU-1201 checksum: add libcfs crypto hash"
Oleg Drokin [Mon, 11 Jun 2012 19:45:59 +0000 (15:45 -0400)]
Revert "LU-1201 checksum: add libcfs crypto hash"

This reverts commit 84a3fd67356c8073a917ea6abd63928055e38156
that broke build for external ofed on rhel5 kernels.

Signed-off-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1373 ptlrpc: add flow control extension to ptlrpc req set
Johann Lombardi [Fri, 4 May 2012 15:53:44 +0000 (17:53 +0200)]
LU-1373 ptlrpc: add flow control extension to ptlrpc req set

This patch allows new requests to be added to a request set while
this latter has already requests in flight. This is done by adding a
callback function invoked by the request set to generate RPCs.
The request set will fire a new RPC each time one completes, keeping
the number of RPCs in flight equals to set->set_max_inflight.

Lock callbacks can thus be sent by the service thread again. This
avoids doing disk I/O from the ptlrpcd context.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: If95922fa8da1dfa7ce7c98d6bfe35e9a5f5bb34f
Reviewed-on: http://review.whamcloud.com/2650
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1461 vmalloc usage needs linux/vmalloc.h include
Oleg Drokin [Tue, 5 Jun 2012 22:14:00 +0000 (18:14 -0400)]
LU-1461 vmalloc usage needs linux/vmalloc.h include

It seems the ext4-vmalloc patch forgot to include linux/vmalloc.h
in mballoc.c

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Test Requirements: fortestonly
Test-Parameters: ostcount=2

Change-Id: I994b75353ce13bac41f45e6bf494191b43304e3c
Reviewed-on: http://review.whamcloud.com/3036
Tested-by: Hudson
Tested-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-56 lnet: move "match" functions to lib-ptl.c
Liang Zhen [Mon, 28 May 2012 06:40:00 +0000 (14:40 +0800)]
LU-56 lnet: move "match" functions to lib-ptl.c

This is still an intermediate patch for LNet SMP improvements,
it covered a few things:
- create a new file lib-ptl.c, all functions about portals
  are moved into this file.
- always precreate match hash-table for all portals, it's a little
  wasty for unique-portal, but it will save a lot of problems for
  upcoming patches
- instead of storing all portals in contiguous buffer, we allocate
  memory for each portal in turn, this change is also for upcoming
  patches.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I19e61c7f3a01f1c90a9f3f78d48d81dc00cd037d
Reviewed-on: http://review.whamcloud.com/2926
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-933 mdc: allow disabling the mdc_rpc_lock
Liang Zhen [Fri, 3 Feb 2012 15:43:55 +0000 (23:43 +0800)]
LU-933 mdc: allow disabling the mdc_rpc_lock

It is desirable to allow disabling the client mdc_{get,put}_rpc_lock()
in order to allow clients to send multiple filesystem-modifying RPCs
at the same time. While this would break MDS recovery (due to
insufficient transaction slots in the MDS last_rcvd file) it would
allow a smaller number of clients to generate a much higher RPC load
on the MDS. This is ideal for MDS/RPC load testing purposes, and can
also be used to help evaluate the potential benefits of implementing
the multi-slot last_rcvd feature.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I39be390ed37f0327bf7eb8015d528195b8568e0b
Reviewed-on: http://review.whamcloud.com/2084
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 lnet: allow to create EQ with zero eq_size
Liang Zhen [Mon, 28 May 2012 03:16:49 +0000 (11:16 +0800)]
LU-56 lnet: allow to create EQ with zero eq_size

Current LNet doesn't allow to create EQ with zero eq_size, it's
an unnecessary limit if the EQ has callback, because EQ callback
is guaranteed to get every event. EQ with no-zero eq_size is only
useful for LNetEQPoll, there is no Lustre use-case that is using
both EQ callback and LNetEQPoll on a same EQ.

Also, even with upcoming LNet SMP improvements, there has to be
a lock to protect enqueue/dequeue operations of event, which might
hurt performance because of lock contention.

This patch will allow user to create EQ with eq_size being zero,
which means events will only be delivered by callback and user
will get nothing by calling LNetEQPoll.

In multiple thread environment, we would suggest user to create EQ
only with callback (eq_size=0) if the EQ might get high rate events,
LNetEQPoll can still work fine if eq_size is non-zero and nothing
will be changed at all. User is still able to create EQ with both
callback and non-zero eq_size, and call LNetEQPoll to get event,
although it's deprecated.

Summary: still can support all current use-cases, although set
eq_size to zero and only rely on callback will be the better way.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ia984f2f65eb5fd064a36dfd7b399e46378013c57
Reviewed-on: http://review.whamcloud.com/2925
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 lnet: cleanup for LNet Event Queue
Liang Zhen [Mon, 28 May 2012 01:58:47 +0000 (09:58 +0800)]
LU-56 lnet: cleanup for LNet Event Queue

This patch is just a cleanup patch for LNet Event Queue (EQ),
it's an intermediate patch for upcoming LNet SMP improvement
for EQ.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I2761c0c4118fa709a921b4060a7bd5ec6700a3f0
Reviewed-on: http://review.whamcloud.com/2924
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 lnet: new internal object lnet_peer_table
Liang Zhen [Sun, 27 May 2012 09:34:00 +0000 (17:34 +0800)]
LU-56 lnet: new internal object lnet_peer_table

Instead of attaching all peers under the big global data of LNet, we
created lnet_peer_table which is just a simple container of peers.

We also made another small change in this patch, when the last
refcount of a peer is released, the dead peer wouldn't be released
anymore, instead it will be attached on deathrow list of peer-table
and released while destroying the peer-table. Peers on deathrow list
can be re-used if we need create a new peer.

Both changes in this patch are just preparations for other LNet SMP
improvements. For example, we will create instances of
lnet_peer_table for each CPT in following patches.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I60e6f2e564f9ab9c86278c9a08e24aaac114e89c
Reviewed-on: http://review.whamcloud.com/2923
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 ptlrpc: clean up ptlrpc svc initializing APIs
Liang Zhen [Wed, 23 May 2012 01:36:03 +0000 (09:36 +0800)]
LU-56 ptlrpc: clean up ptlrpc svc initializing APIs

Both ptlrpc_init_svc and ptlrpc_init_svc_conf have long parameter list
and difficult for user, also, it doesn't make too much sense to keep
both of them. This patch will merge them to a new one
ptlrpc_register_service, and improve the structured parameter
ptlrpc_service_conf and make it more clear for user.
This is the first patch for ptlrpc service SMP improvements, more
patches depending on this will come soon.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I7b2932d4235fb418bee8e9d65f8b3e4f62d2849e
Reviewed-on: http://review.whamcloud.com/2878
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1406 ofd: add object methods
Mikhail Pershin [Wed, 23 May 2012 09:42:12 +0000 (13:42 +0400)]
LU-1406 ofd: add object methods

OFD objects methods except IO

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I17d1e9fd462f8022b516f16df71e40be0879efea
Reviewed-on: http://review.whamcloud.com/2887
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-1406 ofd: transaction and last_rcvd functions
Mikhail Pershin [Wed, 23 May 2012 07:58:03 +0000 (11:58 +0400)]
LU-1406 ofd: transaction and last_rcvd functions

Add ofd transaction handlers and last_rcvd update function

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I14467658ea36d5b71eec7ce183190509a049eb09
Reviewed-on: http://review.whamcloud.com/2886
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1406 ofd: grant support
Mikhail Pershin [Tue, 22 May 2012 10:42:33 +0000 (14:42 +0400)]
LU-1406 ofd: grant support

Add grants functionality

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I660508681e1e33b01dfbdb44ac32705575ee85e1
Reviewed-on: http://review.whamcloud.com/2871
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1201 checksum: add libcfs crypto hash
Alexander.Boyko [Fri, 8 Jun 2012 07:45:59 +0000 (11:45 +0400)]
LU-1201 checksum: add libcfs crypto hash

Add libcfs crypto hash and cleanup all lustre hash checksumming.
Now lustre hash calculations base on linux kernel crypto api for kernel,
and base on libcfs crypto implementation for userlevel. So any improvement
at linux kernel for hashes would improve lustre.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Xyratex-bug-id: MRP-337, MRP-471
Change-Id: I71e7d4fbf45ee40c0c15e235eb71fb4aa86c17d1
Reviewed-on: http://review.whamcloud.com/2276
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1402 libcfs: CFS_ALLOC_HIGH is __GFP_HIGHMEM
Isaac Huang [Mon, 14 May 2012 20:23:04 +0000 (14:23 -0600)]
LU-1402 libcfs: CFS_ALLOC_HIGH is __GFP_HIGHMEM

According to the comment above CFS_ALLOC_HIGH, the flag
allows allocator to return memory in the highmen zone, but
cfs_alloc_flags_to_gfp() incorrectly converts it to
__GFP_HIGH, which enables the use of emergency pools.

__GFP_HIGHMEM should be used instead. Also it makes sense to rename
CFS_ALLOC_HIGH to CFS_ALLOC_HIGHMEM to avoid further confusion.

Signed-off-by: Isaac Huang <Isaac_Huang@xyratex.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I088f36faec1fe22902065d680730731f8deab5d2
Reviewed-on: http://review.whamcloud.com/2732
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
11 years agoLU-1467 ost: ASSERTION(lock->l_req_mode == lock->l_granted_mode)
yangsheng [Wed, 6 Jun 2012 08:15:47 +0000 (16:15 +0800)]
LU-1467 ost: ASSERTION(lock->l_req_mode == lock->l_granted_mode)

The lock may be cancel while ost_prolong_lock_one invoked, so
just return in this case.

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: Ica6ad9199e4b210145e99d2420925803b18a7edd
Reviewed-on: http://review.whamcloud.com/3042
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1406 ofd: add FMD support
Mikhail Pershin [Mon, 21 May 2012 09:36:42 +0000 (13:36 +0400)]
LU-1406 ofd: add FMD support

Add FMD code in OFD

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Iec1cd619cd99b49f5a40c21fc7e8815b5ae392e0
Reviewed-on: http://review.whamcloud.com/2863
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-1406 ofd: add procfs values
Mikhail Pershin [Mon, 21 May 2012 07:28:55 +0000 (11:28 +0400)]
LU-1406 ofd: add procfs values

Add procfs functionality to OFD

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: If2c0a66479d9df8d585cde651244b86bf78636ce
Reviewed-on: http://review.whamcloud.com/2862
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
11 years agoLU-1406 ofd: OFD device methods
Mikhail Pershin [Sun, 20 May 2012 16:55:56 +0000 (20:55 +0400)]
LU-1406 ofd: OFD device methods

Add capability code and basic device functions: statfs, set_info,
get_info, iocontrol, ping, sync, health_check

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I7a0a671ef45e8f4860344bd8c9a9a306d60bd780
Reviewed-on: http://review.whamcloud.com/2856
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1422 cray: Cray platforms set the initial connection value to high
James Simmons [Fri, 18 May 2012 17:44:07 +0000 (13:44 -0400)]
LU-1422 cray: Cray platforms set the initial connection value to high

While testing imperative recovery it was observed that cray systems
had a noticable delay that did not happen on other systems. It was
discovered that the INITIAL_CONNECTION_TIMEOUT has set to a value
for the old Cray catamount systems which is not long valid and
now impacts recovery performace for both the IR and SR case. Also
this patch removes some catamount hacks from liblustre.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia6ff6e22886c49c861ad9469e675f38fe42d6fb4
Reviewed-on: http://review.whamcloud.com/2835
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1378 fid: Add console info for super seq allocation
wangdi [Wed, 9 May 2012 05:18:20 +0000 (22:18 -0700)]
LU-1378 fid: Add console info for super seq allocation

Add console information for super sequence allocation. Because
one super sequence will include 1 billion sequences, it rarely
happens in reality, so it will not cause the flood of console
msg.

Signed-off-by: Di Wang <di.wang@whamcloud.com>
Change-Id: I5154ee2a03006680b6a08d588287bf3941149457
Reviewed-on: http://review.whamcloud.com/2701
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1282 lprocfs: free stat memory as it is cleared
Bobi Jam [Mon, 4 Jun 2012 15:38:17 +0000 (23:38 +0800)]
LU-1282 lprocfs: free stat memory as it is cleared

Clear stats also free the stat memory occupation.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ie0050eb716883aaeca08f4b3bacc772e83b87679
Reviewed-on: http://review.whamcloud.com/3026
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-80 tests: make lfs_migrate script executable
Andreas Dilger [Mon, 4 Jun 2012 17:39:38 +0000 (11:39 -0600)]
LU-80 tests: make lfs_migrate script executable

When running sanity.sh test_56w from the build tree, the test fails
because lfs_migrate does not have execute permission set.  Set the
execute permissions on this script in the build tree.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4af5d7ff4b41353d2e73937292020505ef8de0d9
Reviewed-on: http://review.whamcloud.com/3020
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 lnet: container for LNet message
Liang Zhen [Sun, 27 May 2012 07:58:14 +0000 (15:58 +0800)]
LU-56 lnet: container for LNet message

Adding a simple message container to LNet, and use it to replace
global message queues and message finalizers.
It's a work step of LNet SMP improvements, we will create instances
for each CPT in following patches.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I00f79d7598c4cd1274d7b72f7c4b482d9df75b7a
Reviewed-on: http://review.whamcloud.com/2922
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 lnet: abstract container for EQ/ME/MD
Liang Zhen [Sun, 27 May 2012 06:22:34 +0000 (14:22 +0800)]
LU-56 lnet: abstract container for EQ/ME/MD

This patch implemented a simple resource container for LNet,
it also created several instances of it to manage different LNet
objects (EQ/ME/MD), instead of putting resources in global
table/list.
It's a work step of LNet SMP improvements, we will create instances
for each CPT in following patches.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5e03d618a5c932f032b45b4acdbd4aec546cf9a9
Reviewed-on: http://review.whamcloud.com/2921
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 lnet: add lnet_*_free_locked for LNet
Liang Zhen [Sat, 26 May 2012 13:56:46 +0000 (21:56 +0800)]
LU-56 lnet: add lnet_*_free_locked for LNet

LNet has a set of internal functions lnet_*_free to release LNet
message or LNet descriptors (EQ, MD, ME), they require caller to
hold LNET_LOCK on calling, this is somehow confusing because
lnet_*_alloc should be called w/o any lock.
This patch add another set of functions named as lnet_*_free_locked
to replace original lnet_*_free which require caller to hold lock,
and still provide lnet_*_free which can be called w/o any lock.

This patch is just the first step work for LNet SMP improvements.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I4686d83fc08695f117bf6b3b53d5ad56fb03bcd1
Reviewed-on: http://review.whamcloud.com/2919
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1408 debug: initialize debug_msg_data if needed
Hongchao Zhang [Thu, 5 Apr 2012 20:22:31 +0000 (04:22 +0800)]
LU-1408 debug: initialize debug_msg_data if needed

initialize the libcfs_debug_msg_data only if needed

Change-Id: Id80ea290b982153c86dae2f71c3d42753fa8848f
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2889
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Hudson
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
11 years agoLU-1406 ofd: add connect functions
Mikhail Pershin [Sun, 20 May 2012 14:05:26 +0000 (18:05 +0400)]
LU-1406 ofd: add connect functions

Patch adds connect functions and export init/fini

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I6535d37845863c269dc279c6acc8be8bb8c54432
Reviewed-on: http://review.whamcloud.com/2855
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-957 scrub: OI scrub against ldiskfs
Fan Yong [Fri, 1 Jun 2012 01:56:52 +0000 (09:56 +0800)]
LU-957 scrub: OI scrub against ldiskfs

The OI scrub is totally implemented inside osd-ldiskfs,
and driven by the inode iterator. For each object with
normal fid, its fid in the LMA and related OI entry in
the OI file will be compared, if they are inconsistent,
then the OI entry will be updated.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I04d25b3433336fb63c312f795a8631328beb3aa7
Reviewed-on: http://review.whamcloud.com/2552
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-957 scrub: OSD layer cleanup for OI scrub
Fan Yong [Wed, 30 May 2012 15:08:06 +0000 (23:08 +0800)]
LU-957 scrub: OSD layer cleanup for OI scrub

1) Abstract some common functions in OSD layer.
2) Call lu_object_put() in osd_object_find() under failure case.
3) Do not release buffer_head until all the needed contents have
   been accessed in osd_oi_index_create_one().

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I3e6880271ee73ec2c4638b02c7e9c55fc827bdf3
Reviewed-on: http://review.whamcloud.com/2551
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-957 scrub: Ancillary work for LFSCK/OI scrub
Fan Yong [Wed, 30 May 2012 12:27:48 +0000 (20:27 +0800)]
LU-957 scrub: Ancillary work for LFSCK/OI scrub

1) New debug sub-system - 'D_LFSCK'
For Lustre fsck/scrub running trace.

2) New MDT mount option - 'noscrub'
To disable auto triggering OI scrub when MDT mounts up
or by RPC which accesses inconsistent OI mapping entry.

3) Any object which will be used when Lustre server mounts
   should not be added into OI files to guarantee that the
   Lustre server can start up even though OI files corrupt.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I8b1f1f1160db25ce05bd2d5688f9ff1507e3841b
Reviewed-on: http://review.whamcloud.com/2897
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1299 clio: a combo patch to fix cl_lock
Jinshan Xiong [Tue, 22 May 2012 21:02:33 +0000 (14:02 -0700)]
LU-1299 clio: a combo patch to fix cl_lock

In this patch, the following issues are fixed:
* redefine cl_lock_error(). There are two kinds of error when
  enqueuing a cl_lock: internal errors return from server side and
  local error for example interrupted by signal when waiting for a
  lock. Obviously the latter one is not fatal so we shouldn't err the
  lock out;
* for fault handling, only signal SIGKILL and SIGTERM is not allowed;
* handle return code of ll_fault correctly;
* redefine ->unuse() to release resource it held if enqueuing lock
  failed;
* in cl_lock_hold_release(), only CLS_CACHED mode lock will be cached,
  for locks in other states, they will be cancelled voluntarily;
* fix the problem that cfs_block_sigs() is wrongly used as
  cfs_restore_sigs() and implementation of cfs_block_sigs() was wrong
* other minor fixes.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: Ice6d237d367c54bfd84a4a33cea8a4c1081e7696
Reviewed-on: http://review.whamcloud.com/2654
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1438 quota: fix race in quota_chk_acq_common()
Niu Yawei [Fri, 1 Jun 2012 06:28:45 +0000 (23:28 -0700)]
LU-1438 quota: fix race in quota_chk_acq_common()

quota_check_common() & qctxt_adjust_qunit() uses different way
to check if quota is enforced on certain ID, which could result
in infinite loop in quota_chk_acq_common() when the QB/QI_SET
flag is cleared just after checking.

This patch used a non-instrusive way to fix this rare race.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ia8c5cb593abf515cf4dc041c63c8a247ebe0cd03
Reviewed-on: http://review.whamcloud.com/2996
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1461 Fix ppc64 build errors
Oleg Drokin [Thu, 31 May 2012 16:38:08 +0000 (12:38 -0400)]
LU-1461 Fix ppc64 build errors

Do not try to print u64 as %llu, on ppc64 it's translated into
long int, so we must use LP.64 macroses.

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I14d1051b9427abab5c44582d3230352937643fa5
Reviewed-on: http://review.whamcloud.com/2994
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
11 years agoLU-1030 clio: reimplement ll_fsync in clio way
Jinshan Xiong [Mon, 21 May 2012 19:12:59 +0000 (12:12 -0700)]
LU-1030 clio: reimplement ll_fsync in clio way

A new clio operation CIT_FSYNC is added to forcibly synchorize data
on local client and OST.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I047471f819c00901907d4312a353095b44dd96a1
Reviewed-on: http://review.whamcloud.com/2460
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1030 osc: move io data from lov_oinfo into osc_object
Jinshan Xiong [Mon, 20 Feb 2012 19:01:00 +0000 (11:01 -0800)]
LU-1030 osc: move io data from lov_oinfo into osc_object

Cleanup the code and move io related data structure into osc_object
so that it will be easier to manage pages with red-black tree.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I7cd8d6278bb70b221351d9db4193c679f93aeb21
Reviewed-on: http://review.whamcloud.com/2009
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1305 lu: fid_is_acct function
Alex Zhuravlev [Wed, 23 May 2012 07:54:00 +0000 (11:54 +0400)]
LU-1305 lu: fid_is_acct function

a helper introduced in orion for new quota componenent

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iee557fbd8f910c49b65e1b66330e6906d6efe975
Reviewed-on: http://review.whamcloud.com/2882
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1323 llite: find_vma() should be protected by mmap_sem
Jinshan Xiong [Fri, 13 Apr 2012 22:21:49 +0000 (15:21 -0700)]
LU-1323 llite: find_vma() should be protected by mmap_sem

It had been wrongly protected by page_table_lock since 2004.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: Id7b0341dc8c385ad57e195d94f4e76b6c5626515
Reviewed-on: http://review.whamcloud.com/2543
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1252 recovery: don't always swap nidtbl entries
Jinshan Xiong [Fri, 30 Mar 2012 00:31:40 +0000 (17:31 -0700)]
LU-1252 recovery: don't always swap nidtbl entries

Swap nidtbl entries only when server and client are using different
byte order.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I58ebe98e81040ec187f4ecd75020b7b84c7d8904
Reviewed-on: http://review.whamcloud.com/2410
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1274 osc: Do not grab mutex of cl_lock for glimpse
Jinshan Xiong [Fri, 30 Mar 2012 19:57:34 +0000 (12:57 -0700)]
LU-1274 osc: Do not grab mutex of cl_lock for glimpse

Otherwise this will cause client eviction if that lock is being
flushed and OST happens to be slow to finish the IO.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I8cbd9c38a317f4afaf3b0e237efe55e4c0f991c2
Reviewed-on: http://review.whamcloud.com/2426
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
11 years agoLU-1347 build: remove the vim/emacs modelines
Andreas Dilger [Wed, 9 May 2012 20:19:03 +0000 (14:19 -0600)]
LU-1347 build: remove the vim/emacs modelines

Remove the expandtabs directive from vim modelines from test scripts
and autoconf files as well.  In some cases where a very large test
script has been using 4-space indents for a long time, add directives
for 4-space tabstops so that the formatting is not too ugly.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I115abb1ceea5744495bc189bc5a83e734b845d36
Reviewed-on: http://review.whamcloud.com/2698
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1406 ofd: setup local files for OFD
Mikhail Pershin [Thu, 17 May 2012 09:55:06 +0000 (13:55 +0400)]
LU-1406 ofd: setup local files for OFD

Setup local files needed for OFD work, initialize server data
and clients data upon OFD setup

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I1ec1fc9c07889009f1404eb26830c63cbbc9f451
Reviewed-on: http://review.whamcloud.com/2854
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1213 tests: remove duplicate parallel-scale variables
Yu Jian [Fri, 6 Apr 2012 03:51:10 +0000 (11:51 +0800)]
LU-1213 tests: remove duplicate parallel-scale variables

Remove the duplicate variables from parallel-scale.sh and
parallel-scale-nfs.sh. Those variables are defined inside
the corresponding functions in functions.sh.

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I77ea061289296aeb8b138631f7a55090f52093c9
Reviewed-on: http://review.whamcloud.com/2306
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Cliff White <cliffw@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1282 misc: Use present cpu numbers to save memory.
Bobi Jam [Thu, 5 Apr 2012 06:25:40 +0000 (14:25 +0800)]
LU-1282 misc: Use present cpu numbers to save memory.

lprocfs stats data should allocated by the number of present cpus in
stead of by possible cpu number which wastes a lot of memory.

When new cpus are hot-plugged in, alloc necessary percpu array
elements on demand.

OSS minimum thread number also better be decided by online cpu
number.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Id1690f185f4f83fae75be7eddb756e413cbc4fba
Reviewed-on: http://review.whamcloud.com/2451
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1291 mds: Test failure on test suite replay-single 44c
yangsheng [Wed, 23 May 2012 07:42:20 +0000 (15:42 +0800)]
LU-1291 mds: Test failure on test suite replay-single 44c

The obd_notify needs protect by mds_notify_lock. Else LOV
maybe gone while __mds_lov_synchronize be called.

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: Icb856b2a2b135711fd5681b0b4ebef0a1d78b988
Reviewed-on: http://review.whamcloud.com/2708
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
11 years agoLU-56 libcfs: more common APIs in libcfs
Liang Zhen [Mon, 16 Apr 2012 17:38:17 +0000 (01:38 +0800)]
LU-56 libcfs: more common APIs in libcfs

Implementation of some common APIs:
- per-cpu-partition (percpt) data allocators
- implementation of per-cpu-partition lock
- a few other functions

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ib303b79ee9be87cc306da7e2feb20c8c296b8ac6
Reviewed-on: http://review.whamcloud.com/2558
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1384 mdd: validate incoming param to avoid crashing
Bobi Jam [Thu, 24 May 2012 03:30:18 +0000 (11:30 +0800)]
LU-1384 mdd: validate incoming param to avoid crashing

MDS get crashed when it is connected by old 1.8.x client, the crash
point is

kernel:LustreError: 3657:0:(mdd_object.c:635:mdd_big_lmm_get())
ASSERTION( ma->ma_lmm_size > 0 ) failed

We need validate the incoming @ma lest old client crashes the MDS.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Iebdce6396318a2c6ee413a03dc76c122e3f84146
Reviewed-on: http://review.whamcloud.com/2905
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-988 llite: add debugging for inode timestamps
Bobi Jam [Tue, 13 Mar 2012 07:08:54 +0000 (15:08 +0800)]
LU-988 llite: add debugging for inode timestamps

Intermittent test failures for sanity.sh test_39j are showing the
inode timestamps going backward. Add internal debugging to track the
LVB timestamps to see where the old timestamps are coming from, to
determine if this is a problem with the VM environment being used for
testing, or if it is a genuine problem in the Lustre code.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Iae09d2361e81c05ccc5883dd4418067d9a90abab
Reviewed-on: http://review.whamcloud.com/2274
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 libcfs: export a few symbols from libcfs
Liang Zhen [Sun, 27 May 2012 01:15:31 +0000 (09:15 +0800)]
LU-56 libcfs: export a few symbols from libcfs

Export cfs_trace_copyin_string and cfs_strncasecmp from libcfs,
These symbols will be used by LNet patches soon.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I8722adec822f9d90322a59ff71afff87a9062efd
Reviewed-on: http://review.whamcloud.com/2920
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1417 lov: device type finalized twice
Liang Zhen [Thu, 17 May 2012 10:35:49 +0000 (18:35 +0800)]
LU-1417 lov: device type finalized twice

lov_exit and osc_exit explicitly called lu_device_type_fini() right
before calling class_unregister_type().

However class_unregister_type() will call lu_device_type_fini() too,
which means lov_device_type and osc_device_type will be finalized
twice.

This can be fixed by removing lu_device_type_fini() calls from
lov_exit and osc_exit.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I12dea692f4d2bdd75bb1640b71c6dcc392b94478
Reviewed-on: http://review.whamcloud.com/2823
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1363 llite: Not held lock when calling security_d_instantiate
Bobi Jam [Thu, 3 May 2012 15:28:26 +0000 (23:28 +0800)]
LU-1363 llite: Not held lock when calling security_d_instantiate

security_d_instantiate() could allocate momory and/or yield cpu, don't
held spin lock.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I0a32e067e21ec5f3d8427db020ae5fda6ce96ba1
Reviewed-on: http://review.whamcloud.com/2641
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1364 utils: add lfs 'poollist' subcommand
Bob Glossman [Mon, 14 May 2012 20:02:42 +0000 (13:02 -0700)]
LU-1364 utils: add lfs 'poollist' subcommand

Instead of adding a new subcommand this updates
the lfs man page.  pool_list is the preferred usage,
we just make all the docs match.

Signed-off-by: Bob Glossman <bogl@whamcloud.com>
Change-Id: I3f0a6692923e4ff8359afc8a3558ee404ffc4365
Reviewed-on: http://review.whamcloud.com/2734
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Richard Henwood <rhenwood@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1347 style: removes obsolete EXPORT_SYMTAB macros
Liu Xuezhao [Sat, 14 Apr 2012 08:26:37 +0000 (16:26 +0800)]
LU-1347 style: removes obsolete EXPORT_SYMTAB macros

EXPORT_SYMTAB is obsolete after 2.4 kernel, this patch removes
all EXPORT_SYMTAB macros from source code.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: I5fe3a5544d0cdb45f8d40cd5828a18ea3e1ad26a
Reviewed-on: http://review.whamcloud.com/2671
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1347 lvfs: makes EXPORT_SYMBOL follows function body
Liu Xuezhao [Sat, 14 Apr 2012 08:29:52 +0000 (16:29 +0800)]
LU-1347 lvfs: makes EXPORT_SYMBOL follows function body

Makes EXPORT_SYMBOL macros immediately follow the function body,
to follow normal Linux kernel coding style.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: Icf63514a8ad173217b55f115b4e4ada695eb9295
Reviewed-on: http://review.whamcloud.com/2838
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-56 libcfs: NUMA allocator and code cleanup
Liang Zhen [Thu, 12 Apr 2012 19:08:49 +0000 (03:08 +0800)]
LU-56 libcfs: NUMA allocator and code cleanup

This patch covered a few things:
- Implementation of NUMA allocaters based on concept of CPU
  partition
- Add macros of NUMA allocators for both libcfs and OBD
- cleanup OBD allocaters
- A minor fix for llite, it's using GFP flag which should be
  CFS_ALLOC flag

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5441a504d5df217b59b5db59e37ddb4855e7ae49
Reviewed-on: http://review.whamcloud.com/2523
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-694 config: make 'jobid_var' global
Niu Yawei [Fri, 18 May 2012 06:09:11 +0000 (23:09 -0700)]
LU-694 config: make 'jobid_var' global

This patch makes the global parameter able to accept
string value, and move the 'jobid_var' from the llite
folder into the lustre proc root /proc/fs/lustre.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ibaac19d2f29f8631d4717c6b0f82853cff64fc96
Reviewed-on: http://review.whamcloud.com/2785
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-694 ptlrpc: Job Stats
Niu Yawei [Mon, 24 Oct 2011 15:14:09 +0000 (08:14 -0700)]
LU-694 ptlrpc: Job Stats

This feature is to collect filesystem operation stats for the jobs
running on Lustre.

When some job scheduler (SLURM, for instance) is running on lustre
client, the lustre client will pack the job id into each request
(open, unlink, write...), and server will collect those information
then expose them via procfs.

- A 'pb_jobid' is added in 'ptlrpc_body' for storing the jobID;
- The job stats can be accessed from the proc file 'job_stats' in
  each mdt & obdfilter proc entries;
- 'job_cleanup_interval' is created in each mdt & obdfilter proc
  entries for tuning stats cleanup interval;
- 'jobid_var' is created in llite proc entry for configuring the
  jobID environment variable name;

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I35c21f93d7ce1ce648504ce437fcfd374f891453
Reviewed-on: http://review.whamcloud.com/1397
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
11 years agoLU-1217 osc: to not check a cl_lock's state w/o protection
Jinshan Xiong [Mon, 26 Mar 2012 19:17:17 +0000 (12:17 -0700)]
LU-1217 osc: to not check a cl_lock's state w/o protection

osc_page_putref_lock() used to check cl_lock's refcount and
corresponding osc_lock's ols_hold without any protection, this
is racy because other process can change the lock state so as to
make the assertion be false.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I65fe1fa7fc55e8642fea6789784d7bb92a45d56f
Reviewed-on: http://review.whamcloud.com/2616
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>