Whamcloud - gitweb
fs/lustre-release.git
7 years agoLU-812 kernel: remove smp_lock.h
James Simmons [Fri, 13 Jul 2012 13:35:06 +0000 (09:35 -0400)]
LU-812 kernel: remove smp_lock.h

smp_lock.h has been removed from kernel since 2.6.39.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Wally Wang <wang@cray.com>
Change-Id: Ia15eb8befe4d7c8c1ab29b5860c7ec5bfffbf563
Reviewed-on: http://review.whamcloud.com/2000
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Peng Tao <tao.peng@emc.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1415 tests: Handle OFD procfs changes
Li Wei [Wed, 20 Jun 2012 02:41:34 +0000 (10:41 +0800)]
LU-1415 tests: Handle OFD procfs changes

This patch handles these procfs entry changes in OFD:

  - obdfilter.*.mntdev -> osd-*.*.mntdev
  - obdfilter.*.<cache_related> -> osd-*.*.<cache_related>

Change-Id: Ie9fac69446b5c183bc8eed44a86338ef62197529
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2934
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1481 build: quiet warnings due to := assignment
Andreas Dilger [Tue, 5 Jun 2012 18:53:31 +0000 (12:53 -0600)]
LU-1481 build: quiet warnings due to := assignment

When running automake to generate Makefiles, newer compilers will
warn about non-portable assignments:

  lvfs/autoMakefile.am:52: `:='-style assignments are not portable

The use of ":=" vs. "=" determines when conditional expressions
are evaluated, but have absolutely no use for constant assignments,
so there is no point in using them at all in that case.

Convert all ":=" to "=" for constant assignments, leaving the
decision to use ":=" for conditional assignments until later.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I76caedbc1368a0d2fe02137c46b46c36fc3a500c
Reviewed-on: http://review.whamcloud.com/3129
Tested-by: Hudson
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1209 tests: get dir size after rename in sanity 133d
wangdi [Sat, 7 Jul 2012 11:22:01 +0000 (04:22 -0700)]
LU-1209 tests: get dir size after rename in sanity 133d

Because rename will change the directory size, so we
should get dir size after rename.

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: I1fee9f75cbc0163d10e9de7f04fcf00a4d5c1d9d
Reviewed-on: http://review.whamcloud.com/3298
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1604 osd: don't call md_ucred() for OFD
Johann Lombardi [Thu, 5 Jul 2012 20:34:23 +0000 (22:34 +0200)]
LU-1604 osd: don't call md_ucred() for OFD

There are several places where md_ucred() is called in osd-ldiskfs.
However, this function is specific to the metadata stack (it ends
up calling a callback in mdd layer) and should not be called with OFD.

osd_{push,pop}_ctxt() have thus been modified to be no-op when the
parent is of DT type. __osd_oi_insert() has also been changed to call
md_ucred() only when dealing with a MDT device.

This hack is required to keep compatibility with current MDT stack,
but should be removed as soon as the LOD/OSP is landed.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I931f0376a78cb2aff8b00a99b1c20618d4093e7d
Reviewed-on: http://review.whamcloud.com/3286
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1342 test: sanity-quota test_29
Bobi Jam [Wed, 25 Apr 2012 13:44:09 +0000 (21:44 +0800)]
LU-1342 test: sanity-quota test_29

sanity-quota test_29() needs sleep enough seconds to take into account
both service time and network latency.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ide2b321fbd104eede17eee109998d40d98fb2d30
Reviewed-on: http://review.whamcloud.com/2601
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jay J Lan <jay.j.lan@nasa.gov>
7 years agoLU-549 llite: Improve statfs performance if selinux is disabled
Keith [Thu, 28 Jun 2012 20:24:54 +0000 (13:24 -0700)]
LU-549 llite: Improve statfs performance if selinux is disabled

Don't disable all security xattrs when selinux is disabled just
disable security.selinux. Disabling all security xattrs was
overreaching and causing issues at a customer site.

This patch is built ontop of the original patch:
commit 7cc542fd4c26ccb117ceb13a47ac8ced3107b9b3.

Signed-off-by: Keith Mannthey <keith@whamcloud.com>
Change-Id: I73482d2ff1f166a48dceb3737df0e9e1b187dc85
Reviewed-on: http://review.whamcloud.com/2955
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1370 obdecho: few fixes about md_echo clients
wangdi [Mon, 14 May 2012 23:45:07 +0000 (16:45 -0700)]
LU-1370 obdecho: few fixes about md_echo clients

1. Remove LU_NOREF flag from echo client ctxt, because it
conflicts with cl ctxt(which has no NOREF flag). So if
echo_client and normal lustre client share the same context
cache, this flag will cause echo module refcount not being
dropped.

2. Ma_lmm_size in ma will be changed during mdd_xxx,
so we can not use that to free the ma_lmm.

3. Remove lu_device from linkage on lu_site during cleanup
 phase.

Signed-off-by: Di Wang <di.wang@whamcloud.com>
Change-Id: I08fec689ef9c8b64284f7400ab0fc4dc362ebf87
Reviewed-on: http://review.whamcloud.com/2784
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1544 llog: remove lfs catinfo command
Gregoire Pichon [Wed, 20 Jun 2012 11:14:53 +0000 (13:14 +0200)]
LU-1544 llog: remove lfs catinfo command

This patch removes the llog catinfo code, both client and server
side, since it has not been maintained and may crash the client
system in some cases.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ifa627e61d73a2a416bb548fcc510fb9ca3f36e79
Reviewed-on: http://review.whamcloud.com/3151
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bogl@whamcloud.com>
Reviewed-by: Liu Xuezhao <xuezhao.liu@emc.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1415 utils: Process all {get,set}_param arguments
Andreas Dilger [Fri, 29 Jun 2012 09:04:27 +0000 (03:04 -0600)]
LU-1415 utils: Process all {get,set}_param arguments

Process all arguments to get_param and set_param before returning an
error, to allow accessing one of multiple parameters that have changed
during code reorganization.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Li Wei <liwei@whamcloud.com>
Change-Id: Icb3019c8bd6ed83ffbfeae1fa54084f34df00c1e
Reviewed-on: http://review.whamcloud.com/3245
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1603 ofd: Fix null pointer dereferences in ldiskfs_statfs()
Li Wei [Fri, 6 Jul 2012 13:12:38 +0000 (21:12 +0800)]
LU-1603 ofd: Fix null pointer dereferences in ldiskfs_statfs()

On an OSS running the OFD-OSD stack, unmounting all OSTs makes all OFD
and OSD keys (i.e., lu_context_key) quiescent.  Next time the OSTs are
remounted, ofd_stack_init() will try to refill the quiescent osd_key
into the context, resulting in a null pointer dereference in
ldiskfs_statfs().  Before the problem motivating LCT_QUIESCENT and
LCT_REMEMBER can be revisited, the solution is to refill contexts
after new device types have been "revived" via allocations of their
first device instances.

Change-Id: I4835afb88fae460c21501df95765afd29e570860
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/3353
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1595 build: add 70-column commit comment wrap
Andreas Dilger [Wed, 20 Jun 2012 18:59:47 +0000 (12:59 -0600)]
LU-1595 build: add 70-column commit comment wrap

Add a Vim modeline "textwidth=70" to force commit comments to wrap
at 70 characters, to match standard Git commit style guidelines.

Reduce the commit summary length to 62 characters, which is the
maximum that can fit after "Subject: [PATCH] " when formatting a
patch for email.  This is still above the recommended 50 character
summary limit, but with the leading ticket number and component,
there wouldn't be very much room left for a meaningful comment.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I04f8e0278a18a869b78acbf9dd1b55adf354500c
Reviewed-on: http://review.whamcloud.com/3284
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Bruce Korb <bruce_korb@xyratex.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1339 crc: remove #warning
Alexander.Boyko [Mon, 9 Jul 2012 08:16:11 +0000 (12:16 +0400)]
LU-1339 crc: remove #warning

Patch removes #warning from 32bit code path.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: I9183caf6bc1a98da0c78086ecd300e7cf6880eb0
Reviewed-on: http://review.whamcloud.com/3354
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1282 lprocfs: fix multi-thread contention glitch
Bobi Jam [Fri, 29 Jun 2012 07:33:04 +0000 (15:33 +0800)]
LU-1282 lprocfs: fix multi-thread contention glitch

* Revert clear stats patch committed on 8c831cb8, it is not multi-
  thread safe.
* Should protect the change of lprocfs_stats::ls_biggest_alloc_num
* Add a LPROCFS_STATS_FLAG_IRQ_SAFE flag, when a stat is non-percpu
  stats with this flag, lprocfs_stats_lock() should disable irq.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I42f29e97ff75bf3817249940c8bb491af123d1d9
Reviewed-on: http://review.whamcloud.com/3240
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-107 Add scripts for implementing heartbeat v1 failover
Ned Bass [Thu, 13 Oct 2011 17:56:03 +0000 (10:56 -0700)]
LU-107 Add scripts for implementing heartbeat v1 failover

/usr/sbin/ldev - list devices, determine validity, etc.
/usr/sbin/lhbadm - wrapper for heartbeat utils for failover/failback/status
/etc/ha.d/resource.d/Lustre - heartbeat resource agent (wraps init script)
/etc/init.d/lustre - lustre init script
/etc/init.d/lnet - lnet init script
/usr/sbin/haconfig - helper script for building heartbeat config files

The scripts use two configuration files:
 /etc/ldev.conf - maps hostnames to failover partners, devices, and labels
 /etc/nids - hostnames to lustre NIDS

In addition to heartbeat support, the ldev script enables parallel
execution of commands against all luns configured on a server.  The
lustre init script supports devices backed by Linux software RAID, ZFS,
or traditional block devices.

NOTE: these scripts presume the udev rules for persistent block device
naming are in place, in particular that lustre labels can be mapped to
block devices in /dev/disk/by-id.

Change-Id: I8391744ce6eed989c061f131aca4a2da7b5d51b2
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/290
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1347 lnet: makes EXPORT_SYMBOL follows function body
Liu Xuezhao [Tue, 10 Jul 2012 13:09:29 +0000 (21:09 +0800)]
LU-1347 lnet: makes EXPORT_SYMBOL follows function body

Makes EXPORT_SYMBOL macros immediately follow the function body,
to follow normal Linux kernel coding style.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: Ief3cc2a8e6bf6ad851287278a354fe542d330865
Reviewed-on: http://review.whamcloud.com/2836
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1378 fld: export fldb under /proc
wangdi [Fri, 22 Jun 2012 14:58:47 +0000 (07:58 -0700)]
LU-1378 fld: export fldb under /proc

Export fldb under /proc.

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: Ie4e877fb707cbd86ca83841fffa98425b8f66c00
Reviewed-on: http://review.whamcloud.com/3175
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1598 osd: bypass OI lookup for creation
Liang Zhen [Wed, 4 Jul 2012 05:50:30 +0000 (13:50 +0800)]
LU-1598 osd: bypass OI lookup for creation

It's unnecessary to call osd_oi_lookup() in osd_fid_lookup if it's a
file creation because FID shouldn't never be re-used so we expect it
to be not existed, which means we can bypass overhead of OI lookup at
here, if it's really a duplicate FID from unexpected reasons, we
should detect it later while calling do_create->osd_oi_insert().

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I87227a4e7d008c06876b1366734832886868de3b
Reviewed-on: http://review.whamcloud.com/3269
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1423 mdt: 16K pagesize clients error during ls
yangsheng [Sat, 2 Jun 2012 18:14:36 +0000 (02:14 +0800)]
LU-1423 mdt: 16K pagesize clients error during ls

The 1.8.x client need return entire page even partially
filled in readdir.

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: I762e63dafc511537e3f9e47782dc328a0d7c69de
Reviewed-on: http://review.whamcloud.com/3014
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1492 utils: cleanup wirecheck for ptlrpc_body
Niu Yawei [Thu, 7 Jun 2012 12:19:14 +0000 (05:19 -0700)]
LU-1492 utils: cleanup wirecheck for ptlrpc_body

Added check for the consistency of ptlrpc_body_v3 & ptlrpc_body_v2,
minor fix in ptlrpc_body checking & copyright for in wirehdr.c.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I84d756fcccd63400a144dd96b24df7c6364498e5
Reviewed-on: http://review.whamcloud.com/3057
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1331 changelog: allow changelog to extend record
Lai Siyao [Fri, 1 Jun 2012 11:46:44 +0000 (19:46 +0800)]
LU-1331 changelog: allow changelog to extend record

Previously rename is split into two records, this isn't natural and
makes replication hard because rename needs two records, but they
may not be consecutive.

To solve this, allow Changelog to extend record, add fields sfid/spfid
in changelog record to store source/parent fid, and pack both source
and target name into the record.

Added changelog flag CLF_RENAME_LAST: rename unlink of last hardlink
of the target.

Added macro HAVE_CHANGELOG_EXTEND_REC: liblustreapi receives all
changelog records in changelog_ext_rec format.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I97b990687726e45e661cfb11609f80132c84825d
Reviewed-on: http://review.whamcloud.com/2577
Tested-by: Hudson
Reviewed-by: Thomas LEIBOVICI - CEA <thomas.leibovici@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-181 ptlrpc: align fields of ptlrpc_request
Liang Zhen [Sun, 8 Jul 2012 16:11:08 +0000 (00:11 +0800)]
LU-181 ptlrpc: align fields of ptlrpc_request

Size of ptlrpc_request_buffer_desc is 1048 which means we need 2K
slab for allocating rqbd. This can be changed after we align
fields in ptlrpc_reques to decrease size of ptlrpc_requestt,
because each rqbd has an embedded ptlrpc_request.
We also aligned fields of lu_context because it's embedded in
ptlrpc_request.

With this simple change, size of rqbd is 1008 bytes, and we
can save at least tens of MBs on server.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Iac292e4375421843785fdae083b1aa667d43b3a7
Reviewed-on: http://review.whamcloud.com/3352
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
7 years agoLU-1607 tests: enable multiple CPU Partitions
Liang Zhen [Fri, 6 Jul 2012 04:43:11 +0000 (12:43 +0800)]
LU-1607 tests: enable multiple CPU Partitions

CPU partitions number of libcfs is explicitly set to 1 which means
most SMP features are disabled by default, the reason we didn't
enable it so far is because enabling it will break some tests
because change of /proc interfaces or modparameters.

This patch covered a few things:
- use 0 as default value of cpu_npartitions
  libcfs will estimate CPU partition number based on cores
  or NUMA nodes
- a small fix for LNet
  ni_last_alive should use second instead of jiffies as time stamp
- always enable multiple CPU partitions in test-framework
- fix test_215 for sanity
- fix test_52 for conf-sanity

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ib1a296e254a7b2e91e8d77337a9cd9ac877321e3
Reviewed-on: http://review.whamcloud.com/3288
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoNew tag 2.2.90 2.2.90 v2_2_90 v2_2_90_0
Oleg Drokin [Tue, 10 Jul 2012 02:20:56 +0000 (22:20 -0400)]
New tag 2.2.90

code freeze in preparations for 2.3 release.

Change-Id: Ib2a6fad2d1e1807a1950d0288961eccc446144ad
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 ofd: learn osd to be used from lmi
Alex Zhuravlev [Wed, 27 Jun 2012 12:10:19 +0000 (16:10 +0400)]
LU-1581 ofd: learn osd to be used from lmi

so that OSS can work with different type of backends (ldiskfs, zfs)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I20d7cdc7251881bc83c1d0e5af61a72f0dbca283
Reviewed-on: http://review.whamcloud.com/3282
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-169 clio: restart clio operations if layout changes
Jinshan Xiong [Mon, 18 Jun 2012 10:36:11 +0000 (18:36 +0800)]
LU-169 clio: restart clio operations if layout changes

For glimpse and setattr operations, the layout may change as they may
not open file while running, so we need to restart these operations
in case layout is changed during the IO time.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: Ia7ba1829164259e06af342be3f94e654ed0f89b8
Reviewed-on: http://review.whamcloud.com/2026
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-169 ldlm: add support for layout lock
Jinshan Xiong [Fri, 29 Jun 2012 07:52:19 +0000 (15:52 +0800)]
LU-169 ldlm: add support for layout lock

Add support for layout lock at client side.

Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: Icbdf691134bfb403b0e2019ed364da3e3a11bf5c
Reviewed-on: http://review.whamcloud.com/2025
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1182 ldiskfs-osd: space accounting support
James Simmons [Mon, 2 Jul 2012 15:47:10 +0000 (11:47 -0400)]
LU-1182 ldiskfs-osd: space accounting support

Also support space accounting for ldiskfs OSD on SLES11 SP1
platforms.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I6fd2f58f2fb04f11105a13dc4c9da423cd80fdcc
Reviewed-on: http://review.whamcloud.com/3258
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
7 years agoLU-1581 utils: mount to parse mountdata
Alex Zhuravlev [Wed, 27 Jun 2012 09:07:01 +0000 (13:07 +0400)]
LU-1581 utils: mount to parse mountdata

only for non-ldiskfs backends for a while

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I051489ff2e853f2fd07fc0b355c1e18278939a1d
Reviewed-on: http://review.whamcloud.com/3235
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1581 utils: introduce osd_tune()
Alex Zhuravlev [Wed, 27 Jun 2012 07:15:40 +0000 (11:15 +0400)]
LU-1581 utils: introduce osd_tune()

mount_lustre.c to use that

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If3e576c69c30625a079007cb93e5c447721118ab
Reviewed-on: http://review.whamcloud.com/3234
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1581 mount: minimal support to use zfs
Alex Zhuravlev [Mon, 21 May 2012 07:50:57 +0000 (11:50 +0400)]
LU-1581 mount: minimal support to use zfs

few changes to obd_mount.c to pass backend type

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iea621d089321e81453f937ad1a772a38b3b2a7f1
Reviewed-on: http://review.whamcloud.com/3232
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: add zfs support to wrappers
Alex Zhuravlev [Mon, 18 Jun 2012 06:04:41 +0000 (10:04 +0400)]
LU-1581 utils: add zfs support to wrappers

now mkfs can create lustre server with zfs backend

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ibbed024ec918f1ef76b6fc92951143cff04421b6
Reviewed-on: http://review.whamcloud.com/3228
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 utils: min zfs support
Alex Zhuravlev [Sun, 17 Jun 2012 07:33:25 +0000 (11:33 +0400)]
LU-1581 utils: min zfs support

set of osd_*() instance to support zfs backend

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I6733d6ad139c288d26e1a5f2617b07c9fdc6ed0f
Reviewed-on: http://review.whamcloud.com/3227
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
7 years agoLU-957 lfsck: misc fixes for lfsck
Fan Yong [Fri, 6 Jul 2012 05:28:28 +0000 (13:28 +0800)]
LU-957 lfsck: misc fixes for lfsck

Some fixes for typo, style, and code cleanup.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: Ia027486de26cf8ee7f41da1a33a4fc39d9656752
Reviewed-on: http://review.whamcloud.com/3289
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1594 ldlm: disable LUSTRE_TRACKS_LOCK_EXP_REFS
Liang Zhen [Tue, 3 Jul 2012 09:04:19 +0000 (17:04 +0800)]
LU-1594 ldlm: disable LUSTRE_TRACKS_LOCK_EXP_REFS

Disable lock export refcount, it's supposed to expose bug(20498)
but we didn't see it for two years already, disable it will
save 28 bytes for each ldlm_lock and decrease compute overhead.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I385c5e9d5052eb14b23bbf27003bf499e5b51576
Reviewed-on: http://review.whamcloud.com/3260
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1600 lnet: peer creation has race with shutdown
Liang Zhen [Thu, 5 Jul 2012 05:10:08 +0000 (13:10 +0800)]
LU-1600 lnet: peer creation has race with shutdown

lnet_nid2peer_locked()->lnet_find_peer_locked() will get NULL if
LNet's in progress of shutting down, then it will try to allocate
a new peer and insert it into peer table. If one thread is doing this
and another thread could have already finalized everything of LNet,
so the first thread will crash system after allocation.

The solution is add an extra refcount on peer-table (number of peers)
before allocating new peer, because the shutting down thread always
needs to wait until peers number to be zero before going to the
next step of finalization.

This bug is not introduced by new LNet, but it can be exposed
easily by new LNet.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5c8d26f08ce56092bee1b4bae5111fdfe1e9c12b
Reviewed-on: http://review.whamcloud.com/3274
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoNew tag 2.2.60 2.2.60 v2_2_60 v2_2_60_0
Oleg Drokin [Fri, 6 Jul 2012 16:23:58 +0000 (12:23 -0400)]
New tag 2.2.60

Change-Id: Ib962d669a8f3d0cbeeb92b430a0f86eb0d1b6283
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-56 ptlrpc: post rqbd with flag LNET_INS_LOCAL
Liang Zhen [Wed, 4 Jul 2012 04:21:38 +0000 (12:21 +0800)]
LU-56 ptlrpc: post rqbd with flag LNET_INS_LOCAL

LNet has a new flag LNET_INS_LOCAL which can be used by CPT
affinity threads while posting buffer.
(commit 279bbc81e03dc74d273ec12b4d9e703ca94404c4)
Buffer posted with this flag will be attached on local partition
only, and LND threads can find/match buffer by grabbing a local
partition lock which is good for performance.
This patch applied this flag to ptlrpc service.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I75cee15f125b033642195a71921dbc6ad4db5dfd
Reviewed-on: http://review.whamcloud.com/3268
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-957 lfsck: user space tools for LFSCK/scrub
Fan Yong [Wed, 4 Jul 2012 11:54:00 +0000 (19:54 +0800)]
LU-957 lfsck: user space tools for LFSCK/scrub

Control LFSCK/scrub by lctl commands:

1) lfsck_start: start LFSCK/scrub with specified parameters.

2) lfsck_stop: stop LFSCK/scrub on the specified MDT device.

3) The LFSCK/scrub status can be obtained through some special
lproc interface. For example: check OI scrub status by:
lctl get_param -n osd-ldiskfs.*.oi_scrub

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I5828c18453c92162fa0dc211324b69d15ecd9fbc
Reviewed-on: http://review.whamcloud.com/3170
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 mount: do not pass osd= option
Alex Zhuravlev [Thu, 5 Jul 2012 19:40:20 +0000 (23:40 +0400)]
LU-1581 mount: do not pass osd= option

until it's supported by the kernel part

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia3868e1c2dffb6a0ae8bbe4f8b38018ffcdcc6f4
Reviewed-on: http://review.whamcloud.com/3285
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
7 years agoLU-521 lnet: make LST support variable page size
Liang Zhen [Mon, 5 Sep 2011 15:12:32 +0000 (23:12 +0800)]
LU-521 lnet: make LST support variable page size

LNet selftest can't support variable page size because it sends
number of pages over the wire.
We have to change wire format to fix this but it will have
compatibility issue, so this patch also implemented "session features"
for LST to resolve compatibility issue, only new version LST
can understand new bulk RPC format, although new LST nodes still can
communicate with old LST nodes.

The Variable page size feature can be turned on by setting
LST_FEATURES to LST_FEAT_BULK_LEN (1). This feature is off by default.

Please see the Jira ticket (LU-521) for more details.

Change-Id: I4a552a3310cf0ed0a2f5ae29eaf789469e1c245a
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Signed-off-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1338
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1239 ldlm: cascading client reconnects
Vitaly Fertman [Tue, 26 Jun 2012 11:37:47 +0000 (15:37 +0400)]
LU-1239 ldlm: cascading client reconnects

It may happen that
- MDS is overloaded with enqueues, they consume all the threads on
  MDS_REQUEST portal and waiting for a lock a client holds;
- that client tries to re-connect but MDS is out of threads and
  re-connection fails;
- other clients are waiting for their enqueue completions, they try
  to ping MDS if it is still alive, but despite the fact it is a HP-rpc,
  there is no thread reserved for it. Thus, other clients get timed
  out as well.

Ensure each service which handles HP-rpc has an extra thread reserved
for them; make MDS_CONNECT and OST_CONNECT HP-rpc.

Change-Id: I295aec6a2d2fb614d4b5f037068a3dcdda1a8b09
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Xyratex-bug-id: MRP-455
Reviewed-on: http://review.whamcloud.com/2355
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1422 lnet: eliminate obsolete Cray Catamount support
James Simmons [Mon, 2 Jul 2012 17:18:17 +0000 (13:18 -0400)]
LU-1422 lnet: eliminate obsolete Cray Catamount support

Remove the bulk of code for the no longer supported Catamount
platform on Cray. This was conditionally compiled under CRAY_XT3.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I011058fb0bc74aaf01ec34ea6385e54bdee2356f
Reviewed-on: http://review.whamcloud.com/3064
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1576 llite: correct page usage count
Bobi Jam [Mon, 2 Jul 2012 08:56:07 +0000 (16:56 +0800)]
LU-1576 llite: correct page usage count

If kernel has add_to_page_cache_lru(), the ll_pagevec_add() is defined
as an empty function, while page_cache_get(page) only makes sense if
ll_pagevec_add() is defined.

This patch moves page_cache_get into ll_pagevec_add() macro
definition.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Iad98aacff43beec3e7a64fd1a778f549250aa5b8
Reviewed-on: http://review.whamcloud.com/3255
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-56 ptlrpc: CPT affinity ptlrpc RS handlers
Liang Zhen [Tue, 19 Jun 2012 07:56:45 +0000 (15:56 +0800)]
LU-56 ptlrpc: CPT affinity ptlrpc RS handlers

This patch covered a couple of things:
- reimplement RS handler by using CPU partition APIs
- Instead of always round-robin choose RS handler thread, this patch
  directly choose RS handler thread on partition of rs::rs_svcpt

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5fdebb116630d073d41b39fc4271c4cebb429965
Reviewed-on: http://review.whamcloud.com/3135
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 ofd: revert learn osd to be used from lmi
Johann Lombardi [Thu, 5 Jul 2012 11:16:18 +0000 (07:16 -0400)]
LU-1581 ofd: revert learn osd to be used from lmi

This reverts commit 92ff6ccdfb8b069d36840db97820d3ebe44dfd5b
This patch has broken builds:
ofd_dev.c>:106: error: 'struct lustre_sb_info' has no member
named 'lsi_osd_type'

Change-Id: Id3aa27c513575689c3b33754639b64f716d651f0
Reviewed-on: http://review.whamcloud.com/3279
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Johann Lombardi <johann@whamcloud.com>
7 years agoLU-1581 ofd: learn osd to be used from lmi
Alex Zhuravlev [Wed, 27 Jun 2012 12:10:19 +0000 (16:10 +0400)]
LU-1581 ofd: learn osd to be used from lmi

so that OSS can work with different type of backends (ldiskfs, zfs)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia630d010b77a8d7ca74d67122d77a3ceb98e95e1
Reviewed-on: http://review.whamcloud.com/3236
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: mount to recognize backend type
Alex Zhuravlev [Wed, 27 Jun 2012 05:35:50 +0000 (09:35 +0400)]
LU-1581 utils: mount to recognize backend type

using mount utils introduced earlier.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I18ed4d1840f0b9577b54bc9aa0e7d34e56287ed6
Reviewed-on: http://review.whamcloud.com/3233
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: mt_type macro
Alex Zhuravlev [Wed, 27 Jun 2012 05:33:18 +0000 (09:33 +0400)]
LU-1581 utils: mt_type macro

to convert ldd_mount_type into string

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I7ea21958584384b68ccde546c1472eb3053f6e22
Reviewed-on: http://review.whamcloud.com/3231
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: struct mount_opts
Alex Zhuravlev [Tue, 26 Jun 2012 17:37:03 +0000 (21:37 +0400)]
LU-1581 utils: struct mount_opts

a structure to pass mount options all around

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ide67df554c49752bb06c1f0ed81531925332808a
Reviewed-on: http://review.whamcloud.com/3230
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: mkfs's usage to show zfs support
Alex Zhuravlev [Mon, 18 Jun 2012 06:52:27 +0000 (10:52 +0400)]
LU-1581 utils: mkfs's usage to show zfs support

now everyone is aware of zfs support

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie03aa55360e61b7db1e45a27ea9f0c040827c4d2
Reviewed-on: http://review.whamcloud.com/3229
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1445 fld: Checking lsr_flage after gotten from the cache.
wangdi [Mon, 11 Jun 2012 21:32:59 +0000 (14:32 -0700)]
LU-1445 fld: Checking lsr_flage after gotten from the cache.

Checking lsr_flags after getting the FLD entry from the local
cache to make sure the the correct entry is gotten.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I42c01bdb6521b69d6ce6b79e2b5eeec512d0d657
Reviewed-on: http://review.whamcloud.com/3164
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
7 years agoLU-988 clio: use OSC object's m/a/ctime when build write RPC
Bobi Jam [Wed, 27 Jun 2012 13:29:05 +0000 (21:29 +0800)]
LU-988 clio: use OSC object's m/a/ctime when build write RPC

When building OST write RPC, inode's m/a/ctime are out-dated until
lov_merge_lvb() is called, so we need OSC object's m/a/ctime to set
the OST object.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ie4466ad646fd87c36577ea2aad5fa19ad97c5be7
Reviewed-on: http://review.whamcloud.com/3200
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1581 utils: introduce osd_init() wrapper
Alex Zhuravlev [Sun, 17 Jun 2012 07:33:00 +0000 (11:33 +0400)]
LU-1581 utils: introduce osd_init() wrapper

to load and initialize libraries/modules for different backends

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iae4ac1bbf87093731686f9362d7f6a71cc704030
Reviewed-on: http://review.whamcloud.com/3226
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: unknown fstype
Alex Zhuravlev [Sun, 17 Jun 2012 06:18:15 +0000 (10:18 +0400)]
LU-1581 utils: unknown fstype

scream and exit if specific fstype is not recognized

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib5e897bdfe9609be81d74a41fd65ec57b6cf5d37
Reviewed-on: http://review.whamcloud.com/3225
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: support for multiple underlying devices
Alex Zhuravlev [Thu, 7 Jun 2012 07:22:22 +0000 (11:22 +0400)]
LU-1581 utils: support for multiple underlying devices

will be used by zfs backend

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I90c3630d993811d702f81c838aae7344775cc115
Reviewed-on: http://review.whamcloud.com/3224
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: osd_write_ldd() wrapper
Alex Zhuravlev [Tue, 5 Jun 2012 08:54:41 +0000 (12:54 +0400)]
LU-1581 utils: osd_write_ldd() wrapper

used to write mountdata to backend

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I8cb0275372f00f12f54461ce9afb243283aff4f3
Reviewed-on: http://review.whamcloud.com/3223
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-445 lnet: Make previous change backwards compatible
Doug Oucharek [Wed, 27 Jun 2012 00:16:39 +0000 (17:16 -0700)]
LU-445 lnet: Make previous change backwards compatible

With the previous patch as is, a 2.3 system running against
a 2.2 or 2.1 system will have an invalid timestamp for doing
bandwidth calculations.

This patch checks the value of the timestamp field to
determine if it is a valid value (>= 2.3) or is too small
to be a timestamp (< 2.3). If it is a valid timestamp, it
will be used. If not, the local timestamp will be used.

Signed-off-by: Doug Oucharek <doug@whamcloud.com>
Change-Id: If655144de72d3e46ab32acfcfc35ed79f58d00b2
Reviewed-on: http://review.whamcloud.com/3192
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: osd_make_lustre() wrapper
Alex Zhuravlev [Tue, 5 Jun 2012 08:50:41 +0000 (12:50 +0400)]
LU-1581 utils: osd_make_lustre() wrapper

a wrapper to prepare backend (basically mkfs for given type)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5ba5c7b5905de524bd0089671c753c3a5c44a9ec
Reviewed-on: http://review.whamcloud.com/3222
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: osd_read_ldd() wrapper
Alex Zhuravlev [Tue, 5 Jun 2012 08:40:22 +0000 (12:40 +0400)]
LU-1581 utils: osd_read_ldd() wrapper

used to fetch mountdata from backend

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I785c43ff95ae3e8faecf9231495529e0d2da8246
Reviewed-on: http://review.whamcloud.com/3221
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1339 libcfs: add crc32 pclmulqdq implementation
Alexander.Boyko [Fri, 29 Jun 2012 08:51:56 +0000 (12:51 +0400)]
LU-1339 libcfs: add crc32 pclmulqdq implementation

Using hardware provided PCLMULQDQ instruction to accelerate the CRC32
disposal. This instruction present from Intel Westmere and
AMD Bulldozer CPUs.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyartex.com>
Xyratex-bug-id: MRP-314
Change-Id: Id6c88629f77cc5d389db49b7ee6e7111294c4a14
Reviewed-on: http://review.whamcloud.com/2586
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: introduce osd_is_lustre() wrapper
Alex Zhuravlev [Tue, 5 Jun 2012 07:55:36 +0000 (11:55 +0400)]
LU-1581 utils: introduce osd_is_lustre() wrapper

the purpose of that is to verify whether underlying device
can be used for lustre and identify backend filesystem

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie95cf8b88811be12feb4af67e15ff26ecae9a447
Reviewed-on: http://review.whamcloud.com/3220
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: osd_prepare_lustre
Alex Zhuravlev [Tue, 5 Jun 2012 07:26:38 +0000 (11:26 +0400)]
LU-1581 utils: osd_prepare_lustre

wrapper to prepare backed, load modules, etc

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9f841889cdb950aa9a6650369d73fdc1f0a57f55
Reviewed-on: http://review.whamcloud.com/3219
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: more common helpers
Alex Zhuravlev [Tue, 5 Jun 2012 05:40:19 +0000 (09:40 +0400)]
LU-1581 utils: more common helpers

check_mtab_entry() and update_mtab_entry() this time

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Idcb4d75c4fe202d41f468085f792519b1c431666
Reviewed-on: http://review.whamcloud.com/3218
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: move common funcs to mount_utils.c
Alex Zhuravlev [Tue, 5 Jun 2012 05:01:01 +0000 (09:01 +0400)]
LU-1581 utils: move common funcs to mount_utils.c

like functions to manipulate loop devices, etc

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I96c6347fbda130b81e7da267547af8a3a1c8b567
Reviewed-on: http://review.whamcloud.com/3217
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 utils: extract ldiskfs specifics from mkfs_lustre.c
Alex Zhuravlev [Mon, 4 Jun 2012 11:44:15 +0000 (15:44 +0400)]
LU-1581 utils: extract ldiskfs specifics from mkfs_lustre.c

into mount_utils_ldiskfs.c

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib63258604f9f38b37bb232bc48a33ff867ec1ef0
Reviewed-on: http://review.whamcloud.com/3216
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-56 ptlrpc: partitioned ptlrpc service
Liang Zhen [Sun, 17 Jun 2012 02:57:03 +0000 (10:57 +0800)]
LU-56 ptlrpc: partitioned ptlrpc service

Current ptlrpc service only have a per-service instance, all service
threads share locks and request queue of this instance, this causes
many perforamnce issues like heavy lock contention and data/threads
migration between CPUs/NUMA nodes.

This patch created per-partition data for ptlrpc service, each service
have locks/request-queues on each partition, also, service will have
CPT (CPU partition) affinity threads on each partition, threads
bond on a CPT will only access data on local partition, this feature
should decrease lock contention and data/thread migration and improve
server side performance.

Another change is we use cfs_hash to replace big array fo_iobuf_pool
in obdfilter, filter_iobuf can be found by from the hash by thread ID.
The reason we made this change is because we removed absolute limit
of ptlrpc threads number, which means we have no idea how big the
fo_iobuf_pool should be. Also, even we have obsolute limit of threads
number, it's still dangerous to use a pre-allocated array because
it's difficult to guarantee thread ID to be contiguous if we want
to shrink threads in the future.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5f8dce7bcf389f9f076f5ce2d4685a03f910260b
Reviewed-on: http://review.whamcloud.com/3133
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1581 defs: new param
Alex Zhuravlev [Sun, 17 Jun 2012 09:40:02 +0000 (13:40 +0400)]
LU-1581 defs: new param

identity_upcall param

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie5ed6fea5de4f7dd6ed1f8e2a8333e158d0d147b
Reviewed-on: http://review.whamcloud.com/3215
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1581 defs: zfs string
Alex Zhuravlev [Fri, 25 May 2012 11:56:42 +0000 (15:56 +0400)]
LU-1581 defs: zfs string

a string corresponding to ZFS type

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I491041e22680f051016e5f782843b88580bf841e
Reviewed-on: http://review.whamcloud.com/3214
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1305 lprocfs: osd-ldiskfs to use common helpers
Alex Zhuravlev [Wed, 27 Jun 2012 12:08:25 +0000 (16:08 +0400)]
LU-1305 lprocfs: osd-ldiskfs to use common helpers

introduced with osd-zfs

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I504fa697f6502ddfbac1e0ebc524337a02a74328
Reviewed-on: http://review.whamcloud.com/3198
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-56 o2iblnd: CPT affinity o2iblnd
Liang Zhen [Thu, 10 May 2012 13:44:51 +0000 (21:44 +0800)]
LU-56 o2iblnd: CPT affinity o2iblnd

this patch covered a few things:
- implement percpt scheduler threads for o2iblnd
- decrease overall threads number for fat core machine
- increase thread number only if there are more than one NIC

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ic4b72258f73baabed2e59746639e271cab4467fc
Reviewed-on: http://review.whamcloud.com/2725
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-56 lnet: re-finalize failed ACK or routed message
Liang Zhen [Sun, 1 Jul 2012 14:21:39 +0000 (22:21 +0800)]
LU-56 lnet: re-finalize failed ACK or routed message

lnet_finalize should restart finalizing process for failed ACK
or failed forwarding, because message could be committed for
sending then failed before delivering to LND, i.e: ENOMEM,
in that case we can't just continue to call lnet_msg_decommit():

- The rule is message must decommit for sending first if
  the it's committed for both sending and receiving

- CPT for sending can be different with CPT for receiving,
  so we should return back to lnet_finalize() to make
  sure we are locking the correct partition.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I0b35434762225fcb0dccad7d23bcd63740484e0a
Reviewed-on: http://review.whamcloud.com/3252
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1589 lnet: read peers of /proc could be endless
Liang Zhen [Sun, 1 Jul 2012 07:17:34 +0000 (15:17 +0800)]
LU-1589 lnet: read peers of /proc could be endless

There is a chance that reading /proc/sys/lnet/peers could be endless
because we didn't set correct condition for ending. This bug is
introduced by commit a07e9d350b3e500c7be877f6dcf54380b86a9cbe of LU-56

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I25f6dad4a926bb7c62a4b1b1d4c3a86c783e3f7a
Reviewed-on: http://review.whamcloud.com/3251
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1588 lnet: deadlock while shutting down router
Liang Zhen [Sun, 1 Jul 2012 06:12:24 +0000 (14:12 +0800)]
LU-1588 lnet: deadlock while shutting down router

Should release lock on exiting of lnet_prune_rc_data(), otherwise
we will get deadlock on later attempting on lnet_net_lock().
Also, there is a wrong condition check in lnet_prune_rc_data()
can prevent router checker from shutting down.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5292075453e61f300384043e2346df714c530303
Reviewed-on: http://review.whamcloud.com/3250
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-56 ksocklnd: CPT affinity socklnd
Liang Zhen [Fri, 11 May 2012 02:51:58 +0000 (10:51 +0800)]
LU-56 ksocklnd: CPT affinity socklnd

this patch covered a few things:
- implement percpt scheduler threads for socklnd
- decrease overall threads number for fat core machine
- create more threads only if there are more than one NIC
- remove IRQ affinity implementation from socklnd
  IRQ affinity is not very helpful because CPUs on modern computer
  are very powerful. Also, user can still setup IRQ affinity via
  /proc and cpu_pattern of libcfs

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Idfa19037a529fe96cb1432cbd7f55a5dfac89d29
Reviewed-on: http://review.whamcloud.com/2718
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-169 lov: add lsm refcounting
Jinshan Xiong [Tue, 19 Jun 2012 13:09:48 +0000 (21:09 +0800)]
LU-169 lov: add lsm refcounting

This patch adds a reference counter to the lsm structure.
Each time a lsm is used, a reference must be taken with lsm_get/addref
(lsm_addref() can be used when we already own a reference on the lsm).
This reference can be released via obd_free_memmd(). The lsm is freed
when the last reference is dropped.

This patch also moves lov_stripe_md into a private data of
lov_object.

Signed-off-by: Jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Change-Id: I156b4cc2dc82bb15ae8107cf9842f100048c03d4
Reviewed-on: http://review.whamcloud.com/1874
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-957 lfsck: LFSCK main engine
Fan Yong [Sun, 1 Jul 2012 02:16:10 +0000 (10:16 +0800)]
LU-957 lfsck: LFSCK main engine

Implement the main engine for Lustre online fsck. The kernel
thread "lfsck" scans the system object table by low layer DT
iteration APIs, and will drives registered scrub component(s)
to check/repair Lustre system.

It is the controller for the whole LFSCK, and controls the
speed, including the low layer OI scrub (for osd-ldiskfs).

For urgent mode, like MDT is restored from file-level backup
against ldiskfs, the OI files are invalided. Under such case,
we need to rebuild the OI files ASAP, then low layer OI scrub
inside osd-ldiskfs ignores the main engine speed control, and
runs with full speed.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: If1bd3cfac1f299e964c029e5e9c4cce6432edfa5
Reviewed-on: http://review.whamcloud.com/3169
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-957 scrub: Proc interfaces and tests for OI scrub
Fan Yong [Fri, 29 Jun 2012 11:34:50 +0000 (19:34 +0800)]
LU-957 scrub: Proc interfaces and tests for OI scrub

1) Control/trace OI scrub running.

2) Verify whether the OI scrub basic functions works or not.

3) Test OI scrub performance.

For autotest:
Test-Parameters: testlist=sanity-scrub,scrub-performance

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Change-Id: I5be3d1a521f5f7875f56e9455ff2010016e6a344
Reviewed-on: http://review.whamcloud.com/3168
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoNew tag 2.2.59 2.2.59 v2_2_59 v2_2_59_0
Oleg Drokin [Mon, 2 Jul 2012 16:37:50 +0000 (12:37 -0400)]
New tag 2.2.59

Change-Id: Ia3d5edcc5032e33ce4b90914a6c00f06ae73aa00
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1182 tests: run accounting tests when OFD is used
Johann Lombardi [Tue, 26 Jun 2012 14:46:31 +0000 (16:46 +0200)]
LU-1182 tests: run accounting tests when OFD is used

Skip all quota enforcement tests in sanity-quota.sh and only run
the accounting tests when OFD is used.

This patch also enables the ext4 quota feature when OFD is in use.
Although it is not done in a very efficient way, it still allows
us to run tests.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: Ie35bba9fb064acd8e878e4c31f1b43e90e164e15
Reviewed-on: http://review.whamcloud.com/3191
Tested-by: Hudson
Reviewed-by: Li Wei <liwei@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1182 osd-zfs: lu_quota.h was renamed to lquota.h
Johann Lombardi [Mon, 2 Jul 2012 07:35:29 +0000 (09:35 +0200)]
LU-1182 osd-zfs: lu_quota.h was renamed to lquota.h

Fix build issue due to lu_quota.h which was renamed to lquota.h.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I01f5b159eeae932f1985546b7ba6b6a064e77a01
Reviewed-on: http://review.whamcloud.com/3254
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
7 years agoLU-1182 osd-zfs: add accounting support
Johann Lombardi [Wed, 27 Jun 2012 14:36:34 +0000 (16:36 +0200)]
LU-1182 osd-zfs: add accounting support

Create QSD instance in ZFS OSD so that space accounting data
are exported via procfs.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: Ie44a8032d437beadcd0cf337f6271247538946c4
Reviewed-on: http://review.whamcloud.com/3201
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
7 years agoLU-56 lnet: wrong assertion for optimized GET
Liang Zhen [Fri, 29 Jun 2012 04:22:00 +0000 (12:22 +0800)]
LU-56 lnet: wrong assertion for optimized GET

Event type of optimized GET/REPLY will not match message type,
lnet_msg_decommit_rx() did correct assetion for optimzed REPLY,
but wrong assertion for optimized GET.
this patch also removed another wrong assertion in lnet_finalize()

Both wrong assertions are introduced by this commit:
75a8f4b4aa9ad6bf697aedece539e62111e9029a

Test-Parameters: no_virtualization=true nettype=o2ib
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I089c091819c05e27a8b13cf378a60d6fe64cb849
Reviewed-on: http://review.whamcloud.com/3238
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-56 lnet: tuning wildcard portals rotor
Liang Zhen [Wed, 27 Jun 2012 01:49:48 +0000 (09:49 +0800)]
LU-56 lnet: tuning wildcard portals rotor

By default, PUT messages arrived on wildcard Portals always prefer
to match buffers on current CPT, this patch added a few more
options for dispatching message match which's called portals rotor:

- OFF: Turn off message rotor
- ON: round-robin dispatch all PUT messages to different CPTS
- RR_RT: round-robin dispatch routed PUT message to different CPTs
- HASH_RT: dispatch routed PUT message to CPTs by hashing source NID

User can change portals rotor by /proc interface at runtime.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ie2f03255d9b00b005ca5365315fff350d45a6933
Reviewed-on: http://review.whamcloud.com/3193
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1182 ofd: add space accounting support
Johann Lombardi [Mon, 25 Jun 2012 23:57:54 +0000 (01:57 +0200)]
LU-1182 ofd: add space accounting support

Add space accounting support to OFD. This is done by calling into the
lquota library which looks up space usage via the accounting objects.
Report success on quotacheck, quotaon & quotaoff for the time being,
otherwise space accounting can't be tested with current MDT stack
which requires quotacheck to be run for space accounting to be
enabled.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I442f3d1bcc9e7c95bb3f505f7a00665419d0f517
Reviewed-on: http://review.whamcloud.com/3186
Tested-by: Hudson
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-56 lnet: SMP improvements for LNet selftest
Liang Zhen [Wed, 16 May 2012 06:53:43 +0000 (14:53 +0800)]
LU-56 lnet: SMP improvements for LNet selftest

LNet selftest is using a global WI threads pool to handle all RPCs,
it has performance problem on fat cores machine because all threads
will contend on global queues protected by a single spinlock.
This patch will fix this by creating WI scheduler for each CPT,
RPCs will be dispatched to WI schedulers on different CPTs, and there
is no contention between threads in different WI schedulers.

Another major change in this patch is create percpt data for LST
service. In current implementation each service has a global data
structure to store buffer list and RPC list etch, and all operations
on them are protected by per-service lock, again, this could be a
serious performance issue if the service is busy enough. Having
percpt service data would resolve this issue because threads running
in one CPT will only require lock lock and access local data.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I8035faf2e87d8e424a8c2fac903bf3b241668e00
Reviewed-on: http://review.whamcloud.com/2805
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1182 ldiskfs-osd: space accounting support
Mikhail Pershin [Tue, 19 Jun 2012 19:49:35 +0000 (23:49 +0400)]
LU-1182 ldiskfs-osd: space accounting support

Add space accounting support to ldiskfs OSD.

This patch also sets initial attributes in do_create().
mdd_attr_set_internal() from mdd_object_initialize() is kept until
EDQUOT is returned in lquota itself.
Attributes of new inodes are now initialized in osd_object_create().
All LA_MODE bits are now passed to ldiskfs_create_inode().
(original patch from LiWei, see ORI-46)

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I77a621c76343c2633810bb3cef9859ee30b7b23a
Reviewed-on: http://review.whamcloud.com/3160
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1201 checksum: add libcfs crypto hash
Alexander.Boyko [Thu, 28 Jun 2012 06:04:06 +0000 (10:04 +0400)]
LU-1201 checksum: add libcfs crypto hash

Add libcfs crypto hash and cleanup all lustre hash checksumming.
Now lustre hash calculations base on linux kernel crypto api for kernel,
and base on libcfs crypto implementation for userlevel. So any improvement
at linux kernel for hashes would improve lustre.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Xyratex-bug-id: MRP-337, MRP-471
Change-Id: I2fbf0f4d0c8ce7e7c3c7ea411c6ccd9dcfc7e03a
Reviewed-on: http://review.whamcloud.com/3098
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1305 build: support for osd-zfs
Alex Zhuravlev [Wed, 23 May 2012 08:17:44 +0000 (12:17 +0400)]
LU-1305 build: support for osd-zfs

changes to enable building of spl, zfs and osd-zfs.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I01dcc211b6943b3ed19eaf8756397a97095bbc59
Reviewed-on: http://review.whamcloud.com/2885
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1305 osd: Makefiles for osd-zfs
Alex Zhuravlev [Tue, 29 May 2012 11:26:11 +0000 (15:26 +0400)]
LU-1305 osd: Makefiles for osd-zfs

Makefiles ...

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I1fdf9eecf98306b0cfe0bbc0611df3a977f48b10
Reviewed-on: http://review.whamcloud.com/2972
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
7 years agoLU-1577 lnet: should export lnet_net2ni
Liang Zhen [Thu, 28 Jun 2012 01:52:38 +0000 (09:52 +0800)]
LU-1577 lnet: should export lnet_net2ni

We don't want to export lnet_net2ni_locked because it requires
an extra parameter CPT number, which is internal to LNet and
confusing to user, also, no LND is using it/should use it, however,
LND does need lnet_net2ni() which is an inline function and it is
referring lnet_net2ni_locked(), which means it forces us to export
lnet_net2ni_locked.
The way to fix should be simple, we can just dis-inline lnet_net2ni()
and export itself instead of exporting lnet_net2ni_locked.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I251d42d47770211ae6238ee56972fb9aacd63599
Reviewed-on: http://review.whamcloud.com/3207
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
7 years agoLU-1415 tests: Basic support for ZFS-based servers
Li Wei [Tue, 12 Jun 2012 13:54:47 +0000 (21:54 +0800)]
LU-1415 tests: Basic support for ZFS-based servers

This patch extends Test Framework to support formatting and mounting
ZFS-based servers.

Current global FSTYPE is no longer flexible enough, as different
facets can use different types of back ends.  Such "mixed-fstype"
configurations are firstly resulted from the incremental landing of
Orion changes.  For example, when we have OFD and ZFS OSD, although
OSTs will be able to use ZFS, MGS and MDTs can still only use LDiskFS.
Secondly, to reduce the test matrix, we may also want to have both
OSTs using LDiskFS and OSTs using ZFS in the same test cluster.
Hence, this patch makes back end file system type a per-facet
attribute.  The flexible configuration variables in local.sh are
reserved for users; all other scripts shall use facet_fstype()
instead.

The major differences between using an LDiskFS-based target and a
ZFS-based one are on the mkfs.lustre(8) command line:

  - ZFS-based targets shall have "--backfstype=zfs", while
    LDiskFS-based ones shall have "--backfstype=ldiskfs".

  - LDiskFS-specific "--mkfsoptions" arguments shall not be given to
    ZFS-based targets, and vice versa.

  - ZFS-based targets have different device specifications.  See
    mkfs.lustre(8).

In addition, we will make "--index" mandatory.  Naturally, format
options have to be generated for each facet dynamically, based on its
back end type, index, etc.  This patch changes mkfs_opts() to do
exactly that.  All test scripts shall use mkfs_opts() instead of
reading related environment variables directly.

Similarly, mount options should also be generated on a per-facet, or
at least per-fstype, basis.  Nevertheless, this patch takes a shortcut
by keeping current per-facet-type {MGS,MDS,OST}_MOUNT_OPTSs and
generating the "loop" mount option dynamically in mount_facet().  I
believe this solution introduces fewer changes comparing to a pure
per-facet one and is sufficient for current use cases.

This patch is based on Brian Behlendorf's work under ORI-155.  See
http://review.whamcloud.com/1417.

Change-Id: Ifcce9b10179dd1b4992a30d10df13ea10bc34548
Whamcloud-bug-id: ORI-155
Test-Parameters: testgroup=full
Test-Parameters: testgroup=full envdefinitions=USE_OFD=yes,LOAD_MODULES_REMOTE=true
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2907
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 years agoLU-1305 osd: dmu helpers
Alex Zhuravlev [Tue, 29 May 2012 10:56:48 +0000 (14:56 +0400)]
LU-1305 osd: dmu helpers

few simple functions to wrap native dmu funtions.
to be replaced with direct calls at some point.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ibf9331b5668b1e7db22670caf395391bca002061
Reviewed-on: http://review.whamcloud.com/2971
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
7 years agoLU-1561 tests: Fix ALWAYS_EXCEPT in mds-survey.sh
Li Wei [Mon, 25 Jun 2012 13:24:53 +0000 (21:24 +0800)]
LU-1561 tests: Fix ALWAYS_EXCEPT in mds-survey.sh

Modifying the ALWAYS_EXCEPT assignment in mds-survey.sh does not take
effect, because the test filters have already been built.  This patch
moves the build_test_filter() call below the ALWAYS_EXCEPT assignment.
Also moved downward is the check_and_setup_lustre() call, which
usually goes after environment checks.

Test-Parameters: testlist=mds-survey
Change-Id: I547310e4fccfaaa4fbe7010e14baefa2ab9bbdd4
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/3183
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1528 test: run_one_logged should return error when test fail
Minh Diep [Fri, 15 Jun 2012 11:14:22 +0000 (04:14 -0700)]
LU-1528 test: run_one_logged should return error when test fail

run_one_logged need to return error code even FAIL_ON_ERROR=false

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I1ac59ddfb58b25e7292aacc8c5a31fc218371520
Reviewed-on: http://review.whamcloud.com/3112
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1448 llite: Prevent NULL pointer dereference on disabled OSC
Jeremy Filizetti [Thu, 31 May 2012 12:26:28 +0000 (08:26 -0400)]
LU-1448 llite: Prevent NULL pointer dereference on disabled OSC

When a file system is mounted with a disabled OSC reading the import
information from the proc file system can result in a NULL pointer
dereference. The Lustre import on a disabled OSC with remain
in the LUSTRE_IMP_NEW state and imp_connection will remain NULL.

Signed-off-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ia6df51a36efbcd5a7fc7668bb23455b253ae4855
Reviewed-on: http://review.whamcloud.com/2995
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1479 build: enable srp initiator module with lbuild
Shuichi Ihara [Wed, 6 Jun 2012 15:08:20 +0000 (00:08 +0900)]
LU-1479 build: enable srp initiator module with lbuild

kernel-ib doesn't include ib_srp module if RPMs built by lbuild.
It's required for Infiniband based storage.

Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Change-Id: I16e940592f93abc9150f391316b8f55eb60c663a
Reviewed-on: http://review.whamcloud.com/3049
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-56 ldlm: SMP improvement for ldlm_lock_cancel
Liang Zhen [Tue, 19 Jun 2012 14:43:31 +0000 (22:43 +0800)]
LU-56 ldlm: SMP improvement for ldlm_lock_cancel

ldlm_del_waiting_lock() is always called twice in ldlm_lock_cancel
even the ldlm_lock is not on expiring waiter list, this is bad for
performance because it needs to grab a global lock and disable
softirq.
This patch add a new bit l_waited to ldlm_lock, it's set only if
ldlm_add_waiting_lock() is ever called, and we can bypass
ldlm_del_waiting_lock() in ldlm_lock_cancel() if w/o this bit.

I also adopted patch(32562) on BZ 20509 because it's necessary for
this improvement.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ie48d527b8da187a88646aa070be3aa4beb304b1d
Reviewed-on: http://review.whamcloud.com/3141
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1203 mdt: recognize old rootsquash and nosquash_nid params
Yu Jian [Tue, 8 May 2012 10:12:14 +0000 (18:12 +0800)]
LU-1203 mdt: recognize old rootsquash and nosquash_nid params

Change mdt_process_config() to make it capable of recognizing
old "mdt.rootsquash" and "mdt.nosquash_nid" parameters.

The new parameters are "mdt.root_squash" and "mdt.nosquash_nids".

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I9fc71e2a997a3b0835a3adad2ccfde6d2c64ca56
Reviewed-on: http://review.whamcloud.com/2680
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 years agoLU-1095 debug: no console message for long symlink
Andreas Dilger [Mon, 4 Jun 2012 22:03:28 +0000 (16:03 -0600)]
LU-1095 debug: no console message for long symlink

Some minor fixups from running local sanity.sh tests:
- don't print a console message for long-but-legal symlinks,
  which could be hit during normal usage as well

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I24e3e8ea9d8db0ca9c59597f3170f2b18359500c
Reviewed-on: http://review.whamcloud.com/3130
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>