Whamcloud - gitweb
fs/lustre-release.git
8 years agoORNL-10: Basic IR implementation
Jinshan Xiong [Fri, 19 Aug 2011 23:44:50 +0000 (16:44 -0700)]
ORNL-10: Basic IR implementation

To support imperative recovery, there is a target status table defined for
each file system defined on the MGS. When a target registers itself to the
MGS, the MGS will change this table correspondingly.

In the status table, one important field is target NID. This NID information
is used by clients locating server where target lives. By transferring this NID
to clients, clients can know the restarting of targets earlier. This is
so-called imperative recovery - the MGS notifies clients to do recovery
imperatively instead of timeout based standard recovery.

To implement imperative recovery, clients are asked to cache a NID table, which
contains the location information of all servers. Clients need to hold a read
mode ldlm plain lock - recover lock - to cache this table. Whenever the MGS
wants to change this table, it will enqueue an EXCL recover lock so that all
clients will be notified for this change. Clients will request for a new read
recover lock and then query for the MGS for NID table updates.

Change-Id: I3b38ba142b810df507805b71972feeb1bade1ac2
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1217
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-666 mdd depends on ldiskfs-only dynlocks feature
Brian Behlendorf [Wed, 24 Aug 2011 17:22:27 +0000 (10:22 -0700)]
LU-666 mdd depends on ldiskfs-only dynlocks feature

As discussed in Lustre bugzilla 22435 dynlocks are provided by
the ldiskfs and are not a generic facility.  They should be
removed from the mdd-level to allow building --without-ldiskfs
per Andreas's comment in 22435:

  https://bugzilla.lustre.org/show_bug.cgi?id=22435#c3

This change converts the instances of "struct dynlock_handle *"
to a "void *" pointer which is safe because they are never
dereferenced at this level.

It also wraps the "struct dynlock" entry in the mdd_object with
MDD_DISABLE_PDO_LOCK.  This effectively limits all direct dynlock
usage to the !MDD_DISABLE_PDO_LOCK case which is unconditionally
disabled in the code.  Presumable this code will simply be removed
at some point which will fully resolve the issue.

Change-Id: I7c8d2fb04b31736782e5f6ea2ff6b631b475fb91
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1348
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-499 grant/cancel_rate are static when OST is idle
Lai Siyao [Fri, 5 Aug 2011 03:43:20 +0000 (20:43 -0700)]
LU-499 grant/cancel_rate are static when OST is idle

ldlm_pool_recalc() shouldn't be skipped if namespace->ns_bref eqauls
zero, instead a flag ns_stopping is added to mark ldlm namespace is
being freed.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Ic6485c34ec3e9868ae531a4dc25aee969c374eb5
Reviewed-on: http://review.whamcloud.com/1185
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-690 test: wait_osc_import_state() fixes
Yu Jian [Sun, 9 Oct 2011 06:36:22 +0000 (14:36 +0800)]
LU-690 test: wait_osc_import_state() fixes

b=24498 wait_osc_import_state() fixes

i=Vladimir.Saveliev

-- increase maxtime to wait the timeout of 1st request;
   take into account at_min value;
-- cleanup wait_osc_import_state() to use _wait_import_state();
-- ost-pools test_1 fix: use local var instead of global NAME

Signed-off-by: Elena Gryaznova <elena.gryaznova@oracle.com>
Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I36b163a291d783b1d772f05898ccee783d815298
Reviewed-on: http://review.whamcloud.com/1499
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-731 ldlm: revise and export ldlm_lock_set_data
Jinshan Xiong [Mon, 3 Oct 2011 22:48:36 +0000 (15:48 -0700)]
LU-731 ldlm: revise and export ldlm_lock_set_data

We can only reset ->l_ast_data if it's NULL; otherwise, returning an
error if ->l_ast_data is trying to be changed to a different value.

Change-Id: I8f999a98101daa2c465256792285582f581909c5
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1466
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-580: update mgc llog process code
Jinshan Xiong [Tue, 9 Aug 2011 02:39:22 +0000 (19:39 -0700)]
LU-580: update mgc llog process code

This patch does the following work:
1. fix a bug about setting mgc lock callback data;
2. define log instance types;
3. simplify and cleanup mgc_requeue_thread code

Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Change-Id: Ic8d25eda9f0a7884b4a8cbeccc89c1e66c19f0ff
Reviewed-on: http://review.whamcloud.com/1198
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-573: conf-sanity test_22 failed with 41
Jinshan Xiong [Sat, 6 Aug 2011 02:32:22 +0000 (19:32 -0700)]
LU-573: conf-sanity test_22 failed with 41

Make sure recovery on OST is finished before trying to create file

Change-Id: I4a36685a5cd9c55de729906bff50c29b1108c931
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1192
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-433 remove jbd2-jcberr patch from kernel
Bobi Jam [Tue, 21 Jun 2011 01:07:57 +0000 (09:07 +0800)]
LU-433 remove jbd2-jcberr patch from kernel

In the upstream ext4 code there is a new functionality to allow a
per-commit callback to be set (j_commit_callback), which is used by
the mballoc code to manage free space bitmaps after deleted blocks
have been released. We expand it to contain multiple different
callbacks.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I6397ccabd8d729658cf2ee13c9c3731a9eb31219
Reviewed-on: http://review.whamcloud.com/983
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-553 build: Lustre style commit hooks for Git
Andreas Dilger [Thu, 25 Aug 2011 21:08:04 +0000 (15:08 -0600)]
LU-553 build: Lustre style commit hooks for Git

Adds recommended commit hooks for Lustre commit messages, that
can/should be installed into lustre/.git/hooks for all Lustre trees.

prepare-commit-msg: adds the output of checkpatch.pl as a
    comment to the commit message, to "soft" validate the new
    patch is matching the Lustre coding style. It doesn't check
    100% of the Lustre coding style, but at least finds most of
    the obvious errors so that they can be fixed locally before
    any manual inspection time is spent on the patch.

commit-msg: verifies the format of the commit message (somewhat).
    Checks that there is a valid-looking Signed-off-by: line.
    Adds the Gerrit Change-Id: line for change tracking.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I35fdfb8206d1f0efb3bb401522e8412c62b7c02a
Reviewed-on: http://review.whamcloud.com/1296
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoStart of 2.2 development, updated version to 2.1.50 2.1.50 v2_1_50_0
Oleg Drokin [Wed, 5 Oct 2011 08:40:41 +0000 (04:40 -0400)]
Start of 2.2 development, updated version to 2.1.50

Change-Id: Ibaca591c4db2b839ece2419b77404cdcd869d843
Signed-off-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-459 quiet too noisy console messages at mount
Andreas Dilger [Mon, 22 Aug 2011 22:56:45 +0000 (16:56 -0600)]
LU-459 quiet too noisy console messages at mount

Quiet a number of extra debug messages printed to the console after a
remount or recovery.  They provide no value and just add to the general
confusion of reading Lustre debug messages.

Change-Id: Ia0b43bbd337501169dec234ff66dd56f67f75131
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1275
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoORNL-14: debug: print recovery completion message
Andreas Dilger [Tue, 23 Aug 2011 21:31:58 +0000 (15:31 -0600)]
ORNL-14: debug: print recovery completion message

Print a proper "end of recovery" message to match the one printed
by 1.x so that it is clear when recovery has finished, how long it
took, and how many clients recovered.

This should have been part of the original patch committed for
eda8eab5abeef1117ae1082533b5888a91875e12.

Change-Id: I3a8793fdcb3c665e8df0d39ae82b4c56fb1bb527
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1278
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-258 tests: delete obsolete filter_survey script
Andreas Dilger [Wed, 21 Sep 2011 23:34:12 +0000 (17:34 -0600)]
LU-258 tests: delete obsolete filter_survey script

The filter_survey.sh was a very early version of obdfilter-survey.sh
script, but is long obsolete (last updated 2003) and can be removed.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2fa46d393355ab35352ac693272071424d1a57e1
Reviewed-on: http://review.whamcloud.com/1406
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-610 fix the wrong data type in find_value_cmp
hongchao.zhang [Fri, 2 Sep 2011 08:57:18 +0000 (16:57 +0800)]
LU-610 fix the wrong data type in find_value_cmp

in function find_value_cmp (lustre/utils/liblustreapi.c), the data type for
paramters "file" and "limit" is converted from "unsigned long long" type into
"unsigned int" type by mistake, and the file size should be defined as
"unsigned long long" type too.

Change-Id: Icacebccdf55d8ad761390bb03aa9aab0fdc67131
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1289
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-241 Support crc32c with hardware accelerated instruction as one of lustre checksums
Shuichi Ihara [Thu, 23 Jun 2011 13:08:39 +0000 (22:08 +0900)]
LU-241 Support crc32c with hardware accelerated instruction as one of lustre checksums

Adding CRC32C as new lustre checksum algorithm. Because of crc32c,
the Intel Nehalem based CPU supports h/w accelerated crc32c
instruction as one of SSE4.2 instruction set. The new crc32c codes
in the lustre also use this h/w instruction. The lustre also
automatically detects whether crc32c instruction is available, if
not, use adler (fastest checksum other than crc32c).

Change-Id: I764851a46a94a879239cd127eac411e98342e67f
Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-on: http://review.whamcloud.com/1009
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-110 Recognize old mdt.quota_type param
Niu Yawei [Wed, 23 Mar 2011 06:30:33 +0000 (23:30 -0700)]
LU-110 Recognize old mdt.quota_type param

Minor changes in mdt_process_config() to make it capable of recognizing old mdt.quota_type.

Issue:
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ie13851eee573eb633a68bf5ee3cd86dbbf1bd1a9
Reviewed-on: http://review.whamcloud.com/354
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-570: Add function to find connect uuid by nid
Jinshan Xiong [Wed, 10 Aug 2011 19:30:20 +0000 (12:30 -0700)]
LU-570: Add function to find connect uuid by nid

In this patch, two functions are added:
- class_find_uuid(), find conn uuid by peer nid
- client_import_find_conn(), find a conn uuid in import connection list

Also, a code cleanup is performed.

Change-Id: I50e8e9392a39ef78719504cf083c0c22f5d39dcb
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1189
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-641 obdclass: use rwlock for obd_dev_lock
Jinshan Xiong [Thu, 25 Aug 2011 20:32:02 +0000 (13:32 -0700)]
LU-641 obdclass: use rwlock for obd_dev_lock

In target_handle_connect() it grabs obd_dev_lock to find obd_device
by target uuid. This causes lock contention if there are two many connection
requests on the same time. Based on obd_dev_lock is a read-mostly lock, it
is much reasonable to revise it as a rwlock.

Change-Id: I7f936d6182df9e7ed46b8a1b6fb259226188acbc
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1294
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-574: conf-sanity test_41b failed with 1
Jinshan Xiong [Sat, 6 Aug 2011 03:18:45 +0000 (20:18 -0700)]
LU-574: conf-sanity test_41b failed with 1

Do not run this case if the MGS and MDS are on different devices.

Change-Id: I3b3839a0860dbf55ba1e066e02358abc131f643a
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1193
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-572 ptlrpc: make ptlrpc_recover_import() async
Jinshan Xiong [Fri, 5 Aug 2011 23:48:29 +0000 (16:48 -0700)]
LU-572 ptlrpc: make ptlrpc_recover_import() async

Add an async argument for ptlrpc_recover_import() and with async nonzero,
ptlrpc_recover_import() will return immediately after sending connecting
RPC.

This is needed for imperative recovery since it may recover multiple
targets meanwhile.

Change-Id: Iccc40fca4d3d5d3fb0b2965e62beda23e2fb3b4c
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1191
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-571 ldlm: Remove parallel AST limitation
Jinshan Xiong [Thu, 11 Aug 2011 23:56:06 +0000 (16:56 -0700)]
LU-571 ldlm: Remove parallel AST limitation

Imperative Recovery needs to notify clients as soon as possible.
However, limiting the AST count to 200 will definitely block it.
This may result in the recovery window has passed before the MGS
notifies all clients.

In this patch, I revised the above limitation to be configurable,
and it is no limit by default.

Change-Id: I141da9dbd8b282a76252da15b93c432ac5253bfd
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1190
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-569: Make lu_object cache size adjustable
Jinshan Xiong [Fri, 5 Aug 2011 19:24:33 +0000 (12:24 -0700)]
LU-569: Make lu_object cache size adjustable

lu_object cache is specified to consume 20% of total memory. This
limits 200 clients at most can be mounted on one node. We should make
it adjustable so that customers have a chance to configure it by
their needs.

Change-Id: I8dcb993c88a3abd9fc8ba11ff1578aa3897c3933
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1188
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-721 llite: Parallel writes to same file results in zero file
Bobi Jam [Mon, 3 Oct 2011 04:44:37 +0000 (12:44 +0800)]
LU-721 llite: Parallel writes to same file results in zero file

Canceling lock may contain data being sent to OSTs. Change
find_cbdata iterator to take that into account.

Author: Vladimir Saveliev <vladimir.saveliev@oracle.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1b30fe13ee5303fe3b98e45ad969f5273c41d676
Reviewed-on: http://review.whamcloud.com/1460
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoUpdate release date for 2.1 in Changelog to reflect RC2 2.1.0 2.1.0-RC2 v2_1_0_0 v2_1_0_0_RC2
Oleg Drokin [Fri, 16 Sep 2011 17:20:21 +0000 (13:20 -0400)]
Update release date for 2.1 in Changelog to reflect RC2

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: Ifda31666cac43cb964184903a3d3de20f671cb7d

8 years agoLU-683: handle short write in cl_io_commit_write
Jinshan Xiong [Thu, 15 Sep 2011 05:49:21 +0000 (22:49 -0700)]
LU-683: handle short write in cl_io_commit_write

CLIO has to handle short write case where no bytes were
actually copied from user space. In this case, we should
avoid adding that page into cache.

Change-Id: I78c870e741d7bdb2a9c4c0bfba1adc50a6e814e7
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1383
Tested-by: Hudson
Tested-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
8 years agoLU-653 i_version shouldn't be used for VBR
Mikhail Pershin [Wed, 7 Sep 2011 06:26:31 +0000 (10:26 +0400)]
LU-653 i_version shouldn't be used for VBR

RHEL6 plus EXT4 combination cause i_version is to be used to store
Lustre versions instead of i_fs_version. Meanwhile this field is used
by ext4 internally and may be changed breaking Lustre recovery.

Patch makes only i_fs_version to be used for versions.

Change-Id: Idf7cd257aa2d288abce2a7c18c4d8b139bc9c5c6
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1342
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-653 Ignore last transno from clients with no outstanding transactions
Oleg Drokin [Thu, 1 Sep 2011 20:19:11 +0000 (16:19 -0400)]
LU-653 Ignore last transno from clients with no outstanding transactions

Clients that did not have any unreplied transactions should not
artificially reduce next_recovery_transno since those transactions are
never coming anyway and would just elay all replay activity
until all clients are reconnected.

Change-Id: Iafa32b315211ada93838f4b16fe0f2c800d619c4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1318
Tested-by: Hudson
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-662 fix conflict between seq_client_flush and seq_client_alloc_fid
Hongchao Zhang [Fri, 9 Sep 2011 11:00:39 +0000 (19:00 +0800)]
LU-662 fix conflict between seq_client_flush and seq_client_alloc_fid

seq_client_flush modifies the lu_client_seq.lcs_space by holding the
lu_client_seq.lcs_sem, but seq_client_alloc_fid uses lcs_update to
exclude other concurrent modification.

Change-Id: Ibc3335841b9a045360283e15aab9b6c36b7a39af
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1364
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-650 Max possible file size is mutable
Bobi Jam [Sun, 2 Oct 2011 05:18:59 +0000 (13:18 +0800)]
LU-650 Max possible file size is mutable

Move maxbytes fields out of wire data part of lov_stripe_md,
it can change when OSTs change their status, e.g. disconnected
or inactivated.

Change-Id: I9fb1f2597df1e98d993792706ac94f469d23a78c
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1327
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-614 tests: speed up needlessly long tests
Andreas Dilger [Fri, 26 Aug 2011 08:26:51 +0000 (02:26 -0600)]
LU-614 tests: speed up needlessly long tests

Fix unlinkmany to use the original $next_id instead of the one
seen after createmany was run.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I816081ab1ee04fe66d6c14b76eb076262215bc32
Reviewed-on: http://review.whamcloud.com/1298
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-617 temporary fix for the setattr from open
Niu Yawei [Fri, 26 Aug 2011 02:58:36 +0000 (19:58 -0700)]
LU-617 temporary fix for the setattr from open

The setattr from open(open(O_TRUNC)) is serialized with
'cl_setattr_lock' on client and goes to a dedicate portal, which is
different with other reint operations, consequently, setattr RPC
can be parallel with other reint RPCs, and that result in the race of
updating last_transno/last_xid on server.

This temporary fix removes the LASSERT(req_is_replay(req)) in
mdt_last_rcvd_update().

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I7e07eaf9e718c2d75d9a503207523be9c700ea4a
Reviewed-on: http://review.whamcloud.com/1295
Tested-by: Hudson
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-594 Make sure sanity 27v fallout clears
Bobi Jam [Sat, 20 Aug 2011 10:51:33 +0000 (18:51 +0800)]
LU-594 Make sure sanity 27v fallout clears

We need wait long enough for OST to wake up from
OBD_FAIL_TGT_DELAY_PRECREATE sleep.

Change-Id: I8e4e8153a11a70b66a64f4e8a1deb3e741f58554
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1262
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-622 Alloc enough cfg buffer space
Bobi Jam [Wed, 24 Aug 2011 04:07:22 +0000 (12:07 +0800)]
LU-622 Alloc enough cfg buffer space

obd device name needs space enough for "<dev_name>-<instance_name>\0".
This fixes the glitch in commit cbc4ca2e8dc37d54bb7d3c9b02ab20b63e60f592.

Change-Id: Ib8c3c8db49baad8a9ca068a1c8009aee72aa1e62
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1280
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-614 test: speed up needlessly long tests
Andreas Dilger [Fri, 19 Aug 2011 21:58:43 +0000 (15:58 -0600)]
LU-614 test: speed up needlessly long tests

Some tests are running needlessly long compared to the value that
they provide, and can be made much faster without any loss in value.

conf-sanity test_53a: run llverdev in partial mode always
conf-sanity test_53b: run llverfs in partial mode always
sanity test_220: use unlinkmany instead of "rm -rf", but could likely
                 be sped up dramatically via CFS_FAIL_LOC() on OST

Change-Id: Ifbe66bffc051b6163e41a9bffb1d6b1ab6304e94
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1266
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoUpdate release date for 2.1.0 RC1 2.1.0-RC1 v2_1_0_0_RC1
Oleg Drokin [Tue, 23 Aug 2011 03:54:16 +0000 (23:54 -0400)]
Update release date for 2.1.0 RC1

Change-Id: Ia71c1f4a4d6867e0e605675a1d57a541ef5f9c92
Signed-off-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-365 Update lustre/Changelog
Oleg Drokin [Sat, 20 Aug 2011 16:52:43 +0000 (12:52 -0400)]
LU-365 Update lustre/Changelog

To indicate that the release is from Whamcloud.

Change-Id: I2e8f8e83c895ef1591d55b460727e04a695e1f29
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1269
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
8 years agoUpdate version to 2.1 as a start of RC process. 2.1.0-RC0 v2_1_0_0_RC0
Oleg Drokin [Sat, 20 Aug 2011 16:30:26 +0000 (12:30 -0400)]
Update version to 2.1 as a start of RC process.

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I46e6405953581b5cc10140e8f25e2b4b345e1a4d

8 years agoUpdate recommended e2fsprogs version
Oleg Drokin [Sat, 20 Aug 2011 16:29:52 +0000 (12:29 -0400)]
Update recommended e2fsprogs version

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I00a587a4ab4e8fecb6d3def33ee952b2f793873e

8 years agoLU-365 Update copyright for files modified by Whamcloud
Oleg Drokin [Fri, 19 Aug 2011 23:56:06 +0000 (19:56 -0400)]
LU-365 Update copyright for files modified by Whamcloud

I scoured through the commits logs since first whamcloud commit
and updated all the files that had non-trivial contributions by
Whamcloud.

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I293a82b69a4ce02c028517771bd172ed94c4e44b
Reviewed-on: http://review.whamcloud.com/1267
Reviewed-by: Robert Read <rread@whamcloud.com>
Tested-by: Hudson
8 years agoLU-598 open non-exist object should return ENOENT
nasf [Tue, 16 Aug 2011 13:53:19 +0000 (21:53 +0800)]
LU-598 open non-exist object should return ENOENT

When call ll_intent_file_open() against non-exist object, we cannot
obtain RPC request refcount after the call, so subsequent operation
should not use such internal (only) RPC request to avoid to access
freed (or re-allocated) memory.

Change-Id: I4c3763f460c6145e376538f392fde338d44e6c5e
Signed-off-by: nasf <yong.fan@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1242
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-473 llite: improve error for 2.x cli + 1.8 srv
Andreas Dilger [Wed, 29 Jun 2011 20:36:25 +0000 (14:36 -0600)]
LU-473 llite: improve error for 2.x cli + 1.8 srv

Improve the error message printed to the console when a 2.x client
is trying to mount a 1.8 filesystem.  More specifically, have the 2.x
client check whether the MDS supports the OBD_CONNECT_FID and
OBD_CONNECT_FULL20 flags, without which the client will not be able
to cope.  The 2.x client also does not understand the RPC structure
of 1.8 servers (mdt_body, mdt_rec, etc), but this is harder to check.

For a 2.x server, the client will configure an LMV device, so
exp_connect_flags will be zero, but in that case we also know the
server is running 2.x.

Change-Id: I935571aa0623f9311ddf0545904230721d2fad32
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1036
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-592 1.8 <-> 2.1 interop: missing LMV on client
Niu Yawei [Tue, 16 Aug 2011 09:38:20 +0000 (02:38 -0700)]
LU-592 1.8 <-> 2.1 interop: missing LMV on client

When the cluster is upgraded from 1.8, client will not have LMV layer,
and the "ll_md_exp" is pointing to MDC directly, so the code which
assuming "ll_md_exp" as LMV need be revised:

- ll_dir_ioctl(LL_IOC_GETOBDCOUNT) should return 1 for MDC case;
- mdc_iocontrol() needs to handle the OBD_IOC_QUOTACTL as well when the
  "ll_md_exp" is MDC;

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I3659a5f8076125c9b2012826521c44540e65a75e
Reviewed-on: http://review.whamcloud.com/1236
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-599 verify "mi_ginfo" before accessing its member
nasf [Wed, 17 Aug 2011 07:18:34 +0000 (15:18 +0800)]
LU-599 verify "mi_ginfo" before accessing its member

If the target user only belong to one group, then we just set
"identity->mi_gid", and does not set "identity->mi_ginfo", to
save unnecessary memory allocation and groups processing.

So before accessing member inside "identity->mi_ginfo", make
sure "identity->mi_ginfo" is valid.

Change-Id: I880ccadddd93425c301115328d57c552d11bda8e
Signed-off-by: nasf <yong.fan@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1252
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Tested-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-585 lower mdt reconstruct open debug message level
Bobi Jam [Sat, 13 Aug 2011 14:53:22 +0000 (22:53 +0800)]
LU-585 lower mdt reconstruct open debug message level

Reconstruct open is not an error case.

Change-Id: Ifb559616c148b01298196b4be7e398306a6b8e4c
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1233
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-325 using preallocated objects if OST has enough disk space
hongchao.zhang [Fri, 12 Aug 2011 13:25:13 +0000 (21:25 +0800)]
LU-325 using preallocated objects if OST has enough disk space

during creating files, if ENOSPC is returned by one OST, the preallocated
objects in the corresponding OSC should still can be used if there is enough
blocks but no inode available in the OST.

Change-Id: I9d57461698d1c0fb3a492188fb4ce1519d65efc7
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1051
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-586 1.8 <-> 2.1 interop: list_add corruption
Lai Siyao [Thu, 21 Jul 2011 21:04:42 +0000 (05:04 +0800)]
LU-586 1.8 <-> 2.1 interop: list_add corruption

In seq_client_alloc_fid(), client may list_add itself more than
once to waitqueue under race or error condition.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: Id7d35cc4ca3eafc4fc44d2d79a9ebc25fe56e62c
Reviewed-on: http://review.whamcloud.com/1216
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-15 Strange slow IO messages and bad performance
yangsheng [Wed, 20 Apr 2011 10:05:51 +0000 (18:05 +0800)]
LU-15 Strange slow IO messages and bad performance

b=24183 slow I/O on new files via mballoc.

upstream patch to avoid loading bitmaps from full groups

Change-Id: I9b4de1b4b1942b0f084b6199d5ab3e1267c9e8e3
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/442
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-544 speed up filesystem formatting time in conf-sanity.sh
Yu Jian [Thu, 11 Aug 2011 10:34:27 +0000 (18:34 +0800)]
LU-544 speed up filesystem formatting time in conf-sanity.sh

Pass "-E lazy_itable_init" to mke2fs to speed up Lustre
filesystem formatting time in conf-sanity.sh.

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I29070e6c4b95028b12573d552a32ca04189c2c1d
Reviewed-on: http://review.whamcloud.com/1210
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-455 Replace DIST_SOURCES with EXTRA_DIST
Christopher J. Morrone [Fri, 22 Jul 2011 20:28:06 +0000 (13:28 -0700)]
LU-455 Replace DIST_SOURCES with EXTRA_DIST

Resolve autoconf-2.63 warnings mainly by replacing DIST_SOURCES
with EXTRA_DIST.  Additionally, the non-portable $(shell) contruct
was removed from tests/module.mk and the sources simply enumated.
Finally, the incorrect EXTRA_SOURCES instance was removed from
the top level autoMakefile.am.

  Running automake-1.11 -a -c -W no-portability
  libcfs/libcfs/autoMakefile.am:92:
          variable `DIST_SOURCES' is defined but no program or
  libcfs/libcfs/autoMakefile.am:92:
          library has `DIST' as canonical name (possible typo)
  lnet/klnds/mxlnd/autoMakefile.am:44:
          variable `DIST_SOURCES' is defined but no program or
  lnet/klnds/mxlnd/autoMakefile.am:44:
          library has `DIST' as canonical name (possible typo)
  ...

Also, as a result of the above changes, libcfs/libcfs/autoMakefile.am had to be
modified in order to allow 'make dist' to succeed.

As it turns out, libcfsutils_a_SOURCES was incorrect. That list contained
references to nonexistent 'util/[parser|platform].h' files. The assumption is
it intended to reference libcfs/include/libcfs/util/[parser|platform].h.

To fix the issue, both [parser|platform].h references were removed from the
list. This produces a simple solution that maintains consistency with the rest
of the build system. The caveat being that libcfsutil.a won't automatically be
rebuilt if either of the intended [parser|platform].h files are modified.

Change-Id: Ia81eb1e3fc219f6dac4c7da234f7e736754c5440
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1092
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Richard Henwood <rhenwood@whamcloud.com>
Tested-by: Richard Henwood <rhenwood@whamcloud.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-550 incorrect quota hash function
Niu Yawei [Fri, 29 Jul 2011 04:00:09 +0000 (21:00 -0700)]
LU-550 incorrect quota hash function

Quota code uses 'uid' or '(1 << 32) + gid' as the hash key, however,
the quota hash function lqs_hash() always try to interpret such a
unsigned long long key as a 'quota_adjust_qunit' struct, then calculate
the hash value by quota_adjust_qunit->qaq_id.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I48dc7f276eb97f34b22097b4edc529d5f76acf37
Reviewed-on: http://review.whamcloud.com/1162
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-297 improve llverfs to display the current read/write performance
Yu Jian [Thu, 14 Jul 2011 03:07:34 +0000 (11:07 +0800)]
LU-297 improve llverfs to display the current read/write performance

This patch changes show_filename() to show_rate() in llverfs.c to
display the current read/write file name and performance, along with
an estimate of how long the whole read/write operation will continue.

The patch also changes "fast" to "partial" and fixes some typos in
the comments of llverdev.c and llverfs.c.

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I94cb112ae9a38b1128e53021aa0c1f1e8e5411af
Reviewed-on: http://review.whamcloud.com/1110
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-560 Fixup intent status on ENOENT in mdt_intent_reint
John Hammond [Mon, 1 Aug 2011 15:28:48 +0000 (10:28 -0500)]
LU-560 Fixup intent status on ENOENT in mdt_intent_reint

Clear -ENOENT from rep->lock_policy_res2 for non existent files so
that clients may cache negative dentries if they hold
MDS_INODELOCK_UPDATE on the parent directory.

Change-Id: I238f1001782923a595f9f7ee1c094d53ab1422ae
Signed-off-by: John Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/1173
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-574 Fix whitespace issues in conf-sanity.sh
Andreas Dilger [Tue, 9 Aug 2011 21:17:02 +0000 (15:17 -0600)]
LU-574 Fix whitespace issues in conf-sanity.sh

Move "run_one test_41b" to immediately follow declaration of test_41b.
Fix some whitespace problems (space before tab, space at end of line)
that I noticed while inspecting LU-574.

Change-Id: I18a80a55c9895cf8a6647a01ffa646f5ca499795
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1200
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-321 Don't assume file system name for some test
James Simmons [Fri, 13 May 2011 13:25:01 +0000 (09:25 -0400)]
LU-321 Don't assume file system name for some test

Some of the test assume the file system name is lustre
which is not always the case.

Change-Id: I582aa81fce5d1a881837dfa28bb0d6d84cfb5547
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/253
Tested-by: Hudson
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-556: Fix config_log_find() in mgc_request.c
Jinshan Xiong [Sat, 30 Jul 2011 07:24:51 +0000 (00:24 -0700)]
LU-556: Fix config_log_find() in mgc_request.c

This issue was imported by lu-411. Actually after checking
the code, I found we can make things a bit simpler.

Change-Id: Iad212782a5f0357b73014baf7766c77637fb47db
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1168
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
8 years agoLU-292 srv_n_queued_reqs is screwed up by ptlrpc_unregister_service
Liang Zhen [Mon, 1 Aug 2011 13:07:58 +0000 (21:07 +0800)]
LU-292 srv_n_queued_reqs is screwed up by ptlrpc_unregister_service

service::srv_n_queued_reqs is counter for pre-unpack request on
service::srv_req_in_queue, we shouldn't change it for unpacked request
returned by ptlrpc_server_request_get()

Change-Id: I770411fa8f986fa387a1790f87e6a265abc469f1
Signed-off-by: Liang Zhen <liang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1172
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-553 add checkpatch.pl for Lustre coding style
Andreas Dilger [Fri, 29 Jul 2011 10:59:12 +0000 (04:59 -0600)]
LU-553 add checkpatch.pl for Lustre coding style

Add a modified version of the Linux kernel checkpatch.pl script (from 2.6.38)
to Lustre for validating patches against the Lustre coding style. The Lustre
coding style is largely the same as the kernel, except using spaces instead of
tabs for indenting.

Change-Id: I2d116c6769f9211825dc8ddaead05914f2116c60
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1164
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-559 Keep the o_seq unchanged for 1.8 client
Niu Yawei [Thu, 4 Aug 2011 06:31:10 +0000 (23:31 -0700)]
LU-559 Keep the o_seq unchanged for 1.8 client

ost_validate_obdo() arbitrarily change the o_seq to 0 for the requests
from 1.8 client, which will make the 1.8 echo client use the 0 group
mistakenly. In fact, I think we needn't do anything special for the 1.8
client here.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I93cd3bdbf984f8aef0f5b68f5ee8640f31ded337
Reviewed-on: http://review.whamcloud.com/1182
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
8 years agoLU-540 misc patch for user identity upcall/downcall
nasf [Tue, 2 Aug 2011 16:26:18 +0000 (00:26 +0800)]
LU-540 misc patch for user identity upcall/downcall

1) l_getidentity forgets to allocate supplementary group info space, but access
   such non-allocated memoey without checking.
2) In structure "identity_downcall_data", "idd_gid" is same as "idd_groups[0]",
   remove such redundant information.
3) More sanity check for lprocfs_wr_identity_info().
4) Drop unnecessary memory allocation to simplify mdt_identity_do_upcall().
5) Add padding field in "identity_downcall_data" to resolve align issues.
6) Use new magic to distinguish old version/format downcall.

Change-Id: Ib0dfb37c39a7cb335e29557b68b885afa726789e
Signed-off-by: nasf <yong.fan@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1160
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-540 quota: add compatibility for 1.8 userspace
Andreas Dilger [Wed, 27 Jul 2011 08:30:19 +0000 (02:30 -0600)]
LU-540 quota: add compatibility for 1.8 userspace

The LL_IOC_QUOTACTL ioctl data structure if_quotactl changed size
in 2.x by adding new fields in the middle of the structure.  Add
compatibility for the old 1.8 quotactl userspace ioctl so that it
is possible for existing quotactl tools to run on both 1.x and 2.x.

This requires renumbering the LL_IOC_QUOTACTL ioctl, since it had
incorrectly used "struct if_quotactl *" as the "size" parameter,
but the _pointer_ size did not change.  ioctl() values should always
take a struct as the size parameter, not a pointer to the struct,
so that issues like this can be caught earlier.

Change-Id: I0bce98327a276c0c9a49ce78efd3f45e00df7168
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1152
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-533 Include posix-types.h in headers and distribution
Christopher J. Morrone [Mon, 8 Aug 2011 12:30:44 +0000 (08:30 -0400)]
LU-533 Include posix-types.h in headers and distribution

Include libcfs/posix/posix-types.h in the main rpm, and
add #include statements in lustre_user.h and ll_fiemap.h.
This will allow user-space applications that include these
files to compile.

Change-Id: Ibe391bf28afe91b3914ca1eb07fe66e408b21b4a
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/1138
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-429 Fix and quiet debug message in filter_connect
yangsheng [Fri, 5 Aug 2011 09:54:40 +0000 (17:54 +0800)]
LU-429 Fix and quiet debug message in filter_connect

The comment refers to mds_connect(), which no longer exits.  So I
just removed the comment.

The message says "Received MDS connection", but this function is
used when clients connect to OSTs as well.  We don't need to see
this message on the console all of the time, so I changed it to
D_INFO.

Change-Id: Iebfbaafe39df702862ade126b979528855173d5c
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/963
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-350 fix bug in mgs to allow starting OST before MDT
hongchao.zhang [Thu, 28 Jul 2011 05:43:59 +0000 (13:43 +0800)]
LU-350 fix bug in mgs to allow starting OST before MDT

in mgs_steal_llog_for_mdt_from_client, it forgets to add the "index" and
"svname" of the existing OST to the config log of the MDT's lov, then cause
Lustre fails to mount if the OSTs start before MDT.

Change-Id: I0e68bf51c88517dbd42790e7e4447a186cf268f3
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/595
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-333: Make read_bytes in llite/*/stats report bytes read.
John Hammond [Thu, 14 Jul 2011 00:48:40 +0000 (19:48 -0500)]
LU-333: Make read_bytes in llite/*/stats report bytes read.

Move ll_stats_ops_tally(..., LPROC_LL_{READ,WRITE}_BYTES, ...) from
vvp_io_init() to the end of ll_file_io_generic(), and tally by the
actual number of bytes read rather than the number requested.

Test 127b, client stats are correctly tallied with this patch.

Change-Id: Icad0bc09630b8c1775f2ec59326902aa88215498
Signed-off-by: John Hammond <jhammond@tacc.utexas.edu>
Signed-off-by: Richard Henwood <rhenwood@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1094
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-487 brw_[read|write] stats to correctly counts number of pages.
Richard Henwood [Wed, 6 Jul 2011 20:21:01 +0000 (16:21 -0400)]
LU-487 brw_[read|write] stats to correctly counts number of pages.

brw_[read|write] now correctly counts the number of pages. Before
this change, the number of pages multiplied by the page size was
counted.

Change-Id: I069c18cf46c67285de1ae2afc48f7fb4ab92df3c
Signed-off-by: Richard Henwood <rhenwood@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1056
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-530 group quotas not enforced
Niu Yawei [Thu, 28 Jul 2011 10:47:46 +0000 (03:47 -0700)]
LU-530 group quotas not enforced

In filter_quota_getflag(), we shouldn't break the loop when the file's owner
doesn't have quota limit, otherwise, the check for file's group will be skipped
and the out of quota flag will not be returned to client at the end.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I3e92ba4c061409d5a911d46916535c71f7843e0b
Reviewed-on: http://review.whamcloud.com/1156
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
8 years agoLU-541 Fix liblustreapi.c usage of errno and rc.
Bobi Jam [Thu, 28 Jul 2011 10:11:54 +0000 (18:11 +0800)]
LU-541 Fix liblustreapi.c usage of errno and rc.

* fix liblustreapi.c usage of errno and rc, llapi_xxx() functions should
  return negative value on error.
* enable sanity subtest 65.

Change-Id: I30d172c49b60f6f77ea57c3082d0c68f4c141dea
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1155
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Tested-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-540 remove file flags definition conflict between 1.8 and 2.x
nasf [Sun, 31 Jul 2011 16:26:46 +0000 (00:26 +0800)]
LU-540 remove file flags definition conflict between 1.8 and 2.x

There is file flags definition conflict between 1.8 "LL_FILE_LOCKED_DIRECTIO"
and 2.x "LL_FILE_RMTACL". Redefine the later one to erase such conflict.

Change-Id: I1f4b96382f865f8548bf1a54fb1c74d3649b0daf
Signed-off-by: nasf <yong.fan@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1159
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
8 years agoLU-451 Workaround for lockd (nfsv3) locking
Oleg Drokin [Tue, 2 Aug 2011 03:08:19 +0000 (23:08 -0400)]
LU-451 Workaround for lockd (nfsv3) locking

It seems that lockd works in a different way lock-wise compared
to everybody else. They actually don't use lock owner to determine
conflicts and install their own owner compare that compares specially
crafted pids (duh!).
As a workaround if a custom owner compare is detected, assign pid to
owner field (not intersecting with kernel pointer space anyway).
svclock is the only place using custom owner compare function ATM.

Change-Id: Ia89e2327705f2afbe200fdbadd0f86038fda936d
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1174
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
8 years agoLU-290 Reconnects are not throttled
Lai Siyao [Wed, 15 Jun 2011 02:52:55 +0000 (19:52 -0700)]
LU-290 Reconnects are not throttled

bz22423
Client should not try to reconnect to the same nid in a busy loop,
but instead rely on pinger to issue reconnect.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I878093030d62173071192ca816e037464cc9248d
Reviewed-on: http://review.whamcloud.com/944
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Tested-by: hongchao.zhang <hongchao.zhang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-407 Fix lustre-modules rpm name in spec file.
Christopher J. Morrone [Thu, 9 Jun 2011 20:14:11 +0000 (13:14 -0700)]
LU-407 Fix lustre-modules rpm name in spec file.

RHEL and Fedora kernels include ".%{_arch}" in the kernel's version
string.  Since Lustre includes the kernel version in its rpms, it
can appear that a double architecture name appears at the end.  Really
one is from the kernel, and one is from Lustre.  Trying to change that
is probably more trouble than it is worth.

This seems to cause problems with rpm's understanding of rpm names.  To
fix that this patch takes Andreas's suggestion of grep'ing the full
rpm name from "rpm -q kernel-modules".

We also add "|| true" to the last line of the %preun scriptlet so that
failure does not aport installation.

Finally, we also remove the redundant greps in the postun scriptlet.

Change-Id: I2c71e853e28ec6e0907eb4ea7c3205ca6e5dd873
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/934
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
8 years agoLU-327 cleanup the client import of mgc
hongchao.zhang [Thu, 28 Jul 2011 07:46:47 +0000 (15:46 +0800)]
LU-327 cleanup the client import of mgc

the client import will not have been cleaned if it is setup but never
connected, adding such cleanup code in mgc_precleanup.

define a new inline function "obd_cleanup_client_import" in "obd_class.h"
to be used by mgc, mdc and osc.

Change-Id: If033de3fe81b9a565a7af40a7b93e395fb0bd659
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/917
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-409 Oops: RIP: _spin_lock_irq+0x15/0x40
yangsheng [Wed, 27 Jul 2011 08:01:10 +0000 (16:01 +0800)]
LU-409 Oops: RIP: _spin_lock_irq+0x15/0x40

Skip sanity test_68a since llite_loop module disabled in rhel6.

Change-Id: I213b435e64ac3816f1a3d564522a8c8797fc387c
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1150
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Richard Henwood <rhenwood@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-539 obd_data_connect server side could be corrupted
Bobi Jam [Tue, 26 Jul 2011 17:36:18 +0000 (01:36 +0800)]
LU-539 obd_data_connect server side could be corrupted

In target_handle_connect(), we use assignment to copy client's connect
data to its server counterpart, and we define the pointers pointing to
a big buffer (struct obd_connect_data), while server's reply connect
data could be a smaller buffer (struct obd_connect_data_v1), the
assignment could corrupt the request's obd_connect_data reply buffer.

Change-Id: Ib72776d3dee8a2a836fda5b29ab49c3bbc026cab
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1144
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-207 adding stats for some operations in MDT
hongchao.zhang [Fri, 29 Jul 2011 03:33:13 +0000 (11:33 +0800)]
LU-207 adding stats for some operations in MDT

Adding stats for getattr, setattr, statfs and sync in MDT

Change-Id: I4790bb2eb6bc756a24942224506c7306429243bc
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/427
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-527 Update RHEL6 kernel to 2.6.32-131.6.1 security update
yangsheng [Fri, 22 Jul 2011 18:50:33 +0000 (02:50 +0800)]
LU-527  Update RHEL6 kernel to 2.6.32-131.6.1 security update

Change-Id: I5a84a5240ed8e4d119ae3094beb575666c69f7c6
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1135
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-540 lloop: fix LLOOP_INFO to return FID
Andreas Dilger [Wed, 27 Jul 2011 05:25:18 +0000 (23:25 -0600)]
LU-540 lloop: fix LLOOP_INFO to return FID

Instead of returning an inode number to userspace for the LLOOP_INFO
ioctl, instead return a FID value.

This does not pose compatibility problems, because the ioctl() number
was already different between Lustre 1.x and 2.x, and the lctl needed
for using lloop devices is always built with the kernel modules.

Change-Id: Ia3ee6dab830ad65e2a11d9248982c82ce76221f1
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1149
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-540 quota: move quota ioctl handling to helper
Andreas Dilger [Wed, 27 Jul 2011 07:54:20 +0000 (01:54 -0600)]
LU-540 quota: move quota ioctl handling to helper

The ll_dir_ioctl()::OBD_IOC_QUOTACTL handling is very large, and
is moved to a helper routine quotactl_ioctl() instead of being inline.
This will also facilitate the compatibility handling for the 1.x
QUOTACTL ioctl handling, by being able to share quotactl_ioctl() code.

Also, check for LL_IOC_QUOTACTL in ll_dir_ioctl(), since this is the
ioctl() number exported to userspace, and it also facilitates being
able to find the kernel-side ioctl handler for llapi_quotactl().

Add comments to show the duplicate user/kernel ioctl definitions.
Keep these separate for now, to allow the kernel/userspace values
to be defined differently for compatibility reasons.

There should be no functional differences with this change.

Change-Id: Iecfaf20ddbd78d77a546e1ba5a60b281732a85d9
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1151
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-20 remove unnecessary DMU export patch
Andreas Dilger [Wed, 13 Jul 2011 03:41:39 +0000 (21:41 -0600)]
LU-20 remove unnecessary DMU export patch

The patch for kernel symbol export for DMU back-ends was added to
facilitate upgrading servers.  However, the upstream ZFS code will
no longer be working on RHEL5 kernels, so this patch is not needed.

Change-Id: Icf443a366fd51c06cb5d02df27b7e70f476fcb63
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1086
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-228 kernel: remove more unused kernel patches
Andreas Dilger [Fri, 17 Jun 2011 20:01:44 +0000 (14:01 -0600)]
LU-228 kernel: remove more unused kernel patches

The filterdata patches have been unused and unnecessary for some
time already.  The jbd-slab-race and include-fixes patches were
merged into the upstream kernel and back-ported into RHEL5.

Change-Id: I01851a60ecf453f6eb5dbfc4f573444f4f220861
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/962
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-73 kernel: remove wrong jbd2 commit timer patch
Andreas Dilger [Fri, 17 Jun 2011 18:36:23 +0000 (12:36 -0600)]
LU-73 kernel: remove wrong jbd2 commit timer patch

The bug that this patch was originally fixing (incorrectly rounding
down the jiffies timer, which might cause it to be skipped) was
fixed in the upstream kernel by using round_jiffies_up(), which
always moves the timer into the future.

Remove this patch entirely, since it is no longer needed.

Change-Id: I3b4f30e0565cf64b95fc87babf1f54a9ab190754
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/961
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-528 Update RHEL5 kernel to 2.6.18-238.19.1 security update
yangsheng [Fri, 22 Jul 2011 18:47:43 +0000 (02:47 +0800)]
LU-528  Update RHEL5 kernel to 2.6.18-238.19.1 security update

Change-Id: I13974999954ae1806c48f75fa6d35f4e98cf2a75
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1134
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-278 build: Only warn for tag/version mismatch
Andreas Dilger [Thu, 14 Jul 2011 19:07:29 +0000 (13:07 -0600)]
LU-278 build: Only warn for tag/version mismatch

The configure process should NOT abort just because the most
recent tag is not of the form that upstream uses to tag Lustre.
Downstream developers may use their own tags, or just add
extensions to upsteam's version tags.

Change-Id: I9a98bfd4475d3df2694f536ba0352779a62650c7
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/510
Tested-by: Hudson
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-411 Kernel panic when running conf-sanity on RHEL5/i686
yangsheng [Tue, 14 Jun 2011 20:12:06 +0000 (04:12 +0800)]
LU-411 Kernel panic when running conf-sanity on RHEL5/i686

Reduce stack usage to avoid stack overflow on
RHEL5/i686 patchless client.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I54d3a19734ab089f44f43943b80a2e7ad1f3335f
Reviewed-on: http://review.whamcloud.com/950
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-372 add $TMP/mgsactive to indicate the active combined MGS/MDS node
Yu Jian [Wed, 8 Jun 2011 13:23:32 +0000 (21:23 +0800)]
LU-372 add $TMP/mgsactive to indicate the active combined MGS/MDS node

The current test-framework could not figure out the active MGS node
correctly while the MGS and MDS nodes are combined under a failover
configuration. We need add a $TMP/mgsactive file to indicate the
active MGS node like what we did for the MDS node.

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I74bbf9293f9a290573c4b9fcbf9fea0fcc0aad95
Reviewed-on: http://review.whamcloud.com/913
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Reviewed-by: Chris Gearing <chris@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-221 don't use a/c/m time for newly allocated object in OST
hongchao.zhang [Sat, 16 Jul 2011 22:16:55 +0000 (06:16 +0800)]
LU-221 don't use a/c/m time for newly allocated object in OST

in OST, the a/c/m time is unusable for newly allocated object for
it has not gotten valid time info from clients. by setting the a/c/m
time as LONG_MIN at creation will make the clients ignore the time.

Change-Id: I0d6094e4257626ffd1128b3540837deaf230e056
Signed-off-by: Hongchao Zhang <hongchao.zhang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1084
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-394: LND failure casued by discontiguous KIOV
Jinshan Xiong [Tue, 26 Jul 2011 23:42:34 +0000 (16:42 -0700)]
LU-394: LND failure casued by discontiguous KIOV

Fix the problem of assigning starting_offset twice. This is imported
in combining patchset of lu-394 and lu-523.

Change-Id: Ia4ea205f211c70965101507906b02799a65694be
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1146
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-502 don't allow to kill service threads by OOM killer.
Alexey Lyashkov [Wed, 13 Jul 2011 03:16:12 +0000 (07:16 +0400)]
LU-502 don't allow to kill service threads by OOM killer.

OOM can produce a many issues like requests processing counter.

Change-Id: I8d417d148b589ea6de84c5d16bc2607a59852564
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1088
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoNew version 2.0.66 2.0.66.0 v2_0_66_0
Oleg Drokin [Tue, 26 Jul 2011 15:50:15 +0000 (11:50 -0400)]
New version 2.0.66

Change-Id: Ic2aa5252ec569efefd00255440c5465cae633e1c
Signed-off-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-394: LND failure casued by discontiguous KIOV
Jinshan Xiong [Tue, 12 Jul 2011 17:55:19 +0000 (10:55 -0700)]
LU-394: LND failure casued by discontiguous KIOV

This issue was imported by bug 18881 where I moved the urgent
pages to front of lop_pending to fix a deadlock issue.
I reverted bug 18881 in this patch and came up with a new solution:
cl_page_gang_lookup() only blocks on the first page. This is also
for deadlock avoid since we should never grab multiple pages' lock
without try method.

Change-Id: I5dce35e3929e4f79a350e56ddc9e752269db060e
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/911

8 years agoLU-523: No prepare_write for lockless IO
Jinshan Xiong [Sat, 23 Jul 2011 03:52:46 +0000 (20:52 -0700)]
LU-523: No prepare_write for lockless IO

For page unaligned write, CLIO does prepare_write even for lockless IO
and then write full page back, this will cause data corruption since
data is not covered by lock.

In this patch, we don't do prepare_write for lockless IO, and then submit
exact bytes in the page to OST.

Change-Id: I4aa5afeb82cb717de499c8a8c004078b279302c7
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1130
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-456 Force commit to reuse the just-deleted blocks
Niu Yawei [Mon, 27 Jun 2011 10:55:10 +0000 (03:55 -0700)]
LU-456 Force commit to reuse the just-deleted blocks

In ext4 non-writeback journal mode, the just-deleted blocks are not
useable until the transaction committed, this could cause filter
return -ENOSPC mistakenly when the just-deleted blocks are not committed.

Whenever block allocation fails for -ENOSPC on filter, we should wait
for the previous deleted blocks committed, then retry the allocation.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I678b05ae9aa1b9e9d0aa29f48279bc7daedebbb2
Reviewed-on: http://review.whamcloud.com/1022
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
8 years agoLU-507 check_write_rcs() return -EPROTO instead of -ENOSPC
Johann Lombardi [Tue, 19 Jul 2011 12:15:58 +0000 (14:15 +0200)]
LU-507 check_write_rcs() return -EPROTO instead of -ENOSPC

remote_rcs[i] is an unsigned value so the check remote_rcs[i] < 0 is
always false.

Change-Id: I5468d9185e4c8de819f58693375e9d96722b9fbb
Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1115
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-476 Run run-llog.sh on MGS in sanity 60a
Li Wei [Tue, 5 Jul 2011 08:44:36 +0000 (16:44 +0800)]
LU-476 Run run-llog.sh on MGS in sanity 60a

sanity 60a had been effectively skipped during lab testing, in which the
MGS is not running on the host where sanity.sh is executed.  This patch
fixes the subtest by requesting run-llog.sh to be run on the MGS
explicitly.

Change-Id: I9dc47d856cd42bfe0cd5b5b71e3fd7d4f7c0e561
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1050
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-16 Allow objects larger than 2TB in size
Bobi Jam [Tue, 22 Mar 2011 04:41:17 +0000 (12:41 +0800)]
LU-16 Allow objects larger than 2TB in size

Use OST superblock sb->s_maxbytes telling LOV the maxmimum object size
it supports, and LOV records the minimum of the per-OSC maxbytes values
to determine the lov_stripe_md.lsm_maxbytes.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ice920dc59f5cc5855393bb7031fa089757b3314d
Bugzilla: 20128
Reviewed-on: http://review.whamcloud.com/241
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-477 allocate memory for s_group_desc and s_group_info by vmalloc()
Yu Jian [Thu, 14 Jul 2011 06:32:14 +0000 (14:32 +0800)]
LU-477 allocate memory for s_group_desc and s_group_info by vmalloc()

Add the patch to the RHEL6 ldiskfs patch series.

Large kmalloc() for sbi->s_group_desc and sbi->s_group_info can fail
for large filesystems, which will cause the "not enough memory" error
while mounting. This patch makes it fall back to vmalloc() if the
kmalloc() failed, as what was done for sbi->s_flex_groups.

To avoid colliding with an valid on-disk inode number, EXT4_BAD_INO
is used as the number of the buddy cache inode.

The patch also incorporates the following upstream kernel fix:

commit 32a9bb57d7c1fd04ae0f72b8f671501f000a0e9f
ext4: fix missing iput of root inode for some mount error paths
https://bugzilla.kernel.org/show_bug.cgi?id=26752

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: Ia263c90759e96710702e4afff3ba19e77455386f
Reviewed-on: http://review.whamcloud.com/1095
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-121 Fixup yaml.sh and test-framework.sh so that it only ever uses the first
Chris [Thu, 10 Mar 2011 11:05:37 +0000 (11:05 +0000)]
LU-121 Fixup yaml.sh and test-framework.sh so that it only ever uses the first
part of the host name for log files.
This means stripping of everything off the name after and including the first .

This change is only designed to make the .yml files consistent. The log files
will append the fully qualified name and appear to do this consitantly.

We can create a jira to make the log and yml files consistant, but the reality
is that the yml files have a short life and are only used to send to Maloo.

This change will allow automated posting of results and automated testing to
begin.

1. Carries out the above using hostname -s [Thanks Andreas]
2. Adds in a quick change so that LUSTRE_BUILD in yaml.sh can be a reference to
the source rather than just the lustre version string which is recorded and
written to the yaml anyway as LUSTER_VERSION

Additionally a couple of other changes sneaked in.

1. Allows the review information to be applied to the yaml output file
by way of exporting the variable CODE_REVIEW_YAML to be a yaml description
for maloo

2. The addition of a couple of fixes to make the permissions for yaml
files be allow-all. This permissions are bracketed and so do not change any
other parts of the code.

Change-Id: Ied9a2117cd1b68e048bbc2bf1bffe782485c6150
Signed-off-by: Chris Gearing <chris@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/312
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
8 years agoLU-484 Don't do error cleanup in ext4_add_dot_dotdot()
Niu Yawei [Mon, 11 Jul 2011 06:50:15 +0000 (23:50 -0700)]
LU-484 Don't do error cleanup in ext4_add_dot_dotdot()

The ext4_add_dot_dotdot() will do cleanup work (zeroing i_nlink, iput)
when ext4_bread() fails, that will mess up the i_nlink and i_count if
the ext4_add_dot_dotdot() is called from lustre.

The error cleanup work should be moved from ext4_add_dot_dotdot() to
it's caller: ext4_mkdir().

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I8e8a328c26bf5d603ca26c94a5a04cf8bd1c7b4f
Reviewed-on: http://review.whamcloud.com/1079
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-85 build: update main bug report URL
Andreas Dilger [Sun, 19 Jun 2011 06:39:16 +0000 (00:39 -0600)]
LU-85 build: update main bug report URL

Update bug reporting URL in autoconf and other documentation
to reflect source of release.

Change-Id: I7a4017bd55e9e3f8de6adca0e9a030c94f0db9ec
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/971
Tested-by: Hudson
Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 years agoLU-464 obdfilter-survey test 2a ASSERT(imp->imp_sec == NULL)
Lai Siyao [Wed, 13 Jul 2011 08:00:40 +0000 (16:00 +0800)]
LU-464 obdfilter-survey test 2a ASSERT(imp->imp_sec == NULL)

There is a race in echo client:
1. `lctl --device $echo_client_dev detach`
     -> class_detach() puts last refcount of export
       -> obd_zombie_add_export() adds this export in zombie list, and
            wake up zombie thread.
2. zombie thread culls this export:
     class_destroy_export() put last refcount of echo_client_dev
       -> obd_cleanup()
         -> echo_device_free()
           -> echo_client_cleanup()
             -> osc_disconnect()
               -> client_disconnect_export() will clear cli->cl_import
3. `lctl --device $osc_dev cleanup`
     -> osc_precleanup() find scli->cl_import is not NULL (this means
          import is never connected, but it's not true here)
       -> class_destroy_import()
         -> class_import_put()
           -> LASSERT(imp->imp_sec == NULL) fails

It's expected that step 2 is done before step 3, but this can't be
guaranteed currently. To ensure this, osc_precleanup() calls
obd_zombie_barrier() to ensure step 2 is finished.

Change-Id: I2d044c3ac45a72d305a369a1f31c315f36a7ce02
Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1093
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>