Whamcloud - gitweb
Bobi Jam [Mon, 28 Apr 2014 15:40:27 +0000 (23:40 +0800)]
LU-4943 obdclass: detach MGC dev on error
lustre_start_mgc() creates MGC device, if error happens later on
ll_fill_super(), this device is still attached, and later mount
fails by keep complaining that the MGC device's already in the
client node.
It turns out that the device was referenced by mgc config llog data
which is arranged in the mgc lock requeue thread re-trying to get its
mgc lock, and in normal case, this llog reference only released in
mgc_blocking_ast() when the system is umount.
This patch make mgc_precleanup() to wake up requeue thread to handle
the config llog data.
This patch also makes mgc_setup() wait for mgc_requeue_thread() start
before moving on.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I83df8c68c1dbe4ef4ee879e04ab20df46fea9062
Reviewed-on: http://review.whamcloud.com/11765
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ryan Haasken <haasken@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Emoly Liu [Mon, 25 Aug 2014 05:30:45 +0000 (13:30 +0800)]
LU-4441 test: improve run-llog.sh to print more information
Improve run-llog.sh script so that it can print more useful
information when "Module llog_test is in use" happens again.
This is a backport to b2_5.
Lustre-change: http://review.whamcloud.com/9966
Lustre-commit:
5d8de1fabf740eda724ad2e9eb40c3b57322b9e4
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I15594f34f26550204b525d8f532fd064f30d00a4
Reviewed-on: http://review.whamcloud.com/11573
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Nathaniel Clark [Sun, 23 Nov 2014 07:48:10 +0000 (23:48 -0800)]
LU-4839 tests: Give copytool more time to start
Copytool can take some time to start, and if the HSM archive directory
is on a busy NFS server, it can take a bit of time for the initial
opens to occur. This allows those actions more time to complete which
should give this test a better chance of passing correctly.
Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm
Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes,ONLY=60 \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
mdtcount=4 testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm
This patch is back-ported from the following one:
Lustre-commit:
6948e80bc149aa689e09334a70941340143fa2ce
Lustre-change: http://review.whamcloud.com/12682
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ib8828805f52c1f86bf01388606b78ad2c4085acb
Reviewed-on: http://review.whamcloud.com/12823
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Matt Ezell [Tue, 28 Oct 2014 20:49:23 +0000 (16:49 -0400)]
LU-5822 osd-ldiskfs: Correctly return OS_STATE_READONLY
Ensure that read-only devices correctly return their state so that
the health_check file will be accurate
Signed-off-by: Matt Ezell <ezellma@ornl.gov>
Change-Id: Iea38c924ca0d8cb543b77599019fc88fd8b58105
Reviewed-on: http://review.whamcloud.com/12463
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Hongchao Zhang [Mon, 1 Dec 2014 22:47:05 +0000 (14:47 -0800)]
LU-5650 mgc: check the import stat for lprocfs
in lprocfs_mgc_rd_ir_state, the import state should be checked
the validity before doing further work.
This patch is back-ported from the following one:
Lustre-commit:
4295402585605e1ba210ad96c12e3820fa523f7b
Lustre-change: http://review.whamcloud.com/12327
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I41a5f7547d24bf61ce83dd488aef699409be209b
Reviewed-on: http://review.whamcloud.com/12896
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Liang Zhen [Tue, 25 Nov 2014 19:21:34 +0000 (11:21 -0800)]
LU-5545 ptlrpc: false alarm in AT network latency measuring
If early reply of client RPC is lost and client RPC is expired and
resent, server will drop the resent RPC because it's already in
processing, server may also send reply or early reply to client,
which can still match reply buffer of the original request.
In this case, client is measuring time from resent time, but server
is reporting service time of original RPC, which is longer than
the time measured by client.
This patch is back-ported from the following one:
Lustre-commit:
c238be0bd02367b634c1061a30bbe161dfd9d8e9
Lustre-change: http://review.whamcloud.com/12018
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I5bdcdee9dd6b439d5d650fa812ea65527458c590
Reviewed-on: http://review.whamcloud.com/12855
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Niu Yawei [Mon, 28 Apr 2014 04:39:38 +0000 (00:39 -0400)]
LU-4810 utils: print messages when set tunables
When set scheduler and max_sectors_kb on mount, error messages should
be printed to notify user that these tunables are changed by lustre.
Lustre-commit:
9813961151e8f6e19eac9cf5a8db676f7aa70f50
Lustre-change: http://review.whamcloud.com/9865
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I6a3618b7e5eb0127e9aea18631397cd4dcbde546
Reviewed-on: http://review.whamcloud.com/9865
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Blake Caldwell <blakec@ornl.gov>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/12553
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Bobi Jam [Thu, 13 Feb 2014 15:27:41 +0000 (23:27 +0800)]
LU-4627 llite: deed taking lli_trunc_sem during file write
File write before io loop will take lli_trun_sem read semaphore to
protect osc_extent, while after generic_file_aio_write() done, it
could possible need to kill suid or sgid, which will call
ll_setattr_raw() to change the inode's attribute, and it does not
involve size.
So the ll_truc_sem write semaphore should be constrained
around ll_setattr_ost() to not come across the lli_trunc_sem read
semaphore get from the normal file write path.
Backported from master:
Lustre-change: http://review.whamcloud.com/9267
Lustre-commit:
1601e8591c1334ee7879bb6daac09342caa4a542
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic7f9cfb6d76dcf7ad08635efab154e7cdc392ce8
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/12268
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Liang Zhen [Tue, 16 Sep 2014 03:49:40 +0000 (11:49 +0800)]
LU-5639 lnet: portal spreading rotor should be unsigned
Portal spreading rotor should be unsigned, otherwise lnet may get
negative CPT number and access invalid addresses.
Lustre-commit:
ed5c72e96f7f951ad29284fccd1f878963a817cf
Lustre-change: http://review.whamcloud.com/11936
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Id7f40da241af3b01483fdedd366b09329f530163
Reviewed-on: http://review.whamcloud.com/11999
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Henri Doreau [Tue, 16 Sep 2014 10:34:07 +0000 (12:34 +0200)]
LU-5631 obdclass: Proper swabbing of llog_rec_tail.
A variable-length structure preceeds llog_rec_tail within an llog
block. Thus cr_tail shouldn't be accessed directly as a structure
member but its actual location should be computed dynamically.
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I2d244797d107cf52f647e19b2db780138e910925
Reviewed-on: http://review.whamcloud.com/11953
Tested-by: Jenkins
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Jian Yu [Wed, 22 Oct 2014 06:34:04 +0000 (23:34 -0700)]
LU-5606 tests: add version check codes to conf-sanity test 41c
This patch adds Lustre version check codes into conf-sanity test
41c to make the test interoperate with servers that do not have
commit
166c5ba95cb2a4771317e030a3649e4480c8cbad.
Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=41c \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=conf-sanity
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I605e43954ba7015002dabcaee1cd587c898fa6cb
Reviewed-on: http://review.whamcloud.com/12375
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Bruno Faccini [Tue, 21 Oct 2014 01:33:50 +0000 (18:33 -0700)]
LU-5573 obdclass: strengthen against concurrent server mounts
This patch is an add-on to previous fix for LU-5299, against same
issues/races scenarios during concurrent server devices mounts.
This patch is back-ported from the following one:
Lustre-commit:
91b68bc4f4d8f25e88a252ca0918f0a422fec27b
Lustre-change: http://review.whamcloud.com/12114
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,CONF_SANITY_EXCEPT=45 \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
ostcount=2 ostsizegb=8 testlist=conf-sanity,conf-sanity,conf-sanity
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I121ee3f14d4bee7973cd968d561e8fc03acb6386
Reviewed-on: http://review.whamcloud.com/12353
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Johann Lombardi [Mon, 1 Sep 2014 13:03:51 +0000 (15:03 +0200)]
LU-5556 target: limit bulk transfer time
Messages lost during bulk transfer are not resent, so there is no
point in waiting for a very long time (up to at_max/600s has been
seen). This patch adds a new static timeout for the bulk transfer
(100s by default).
Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I3926a7a8f2bce4cbd00b8fe54094a8e9cbec1508
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Reviewed-on: http://review.whamcloud.com/12242
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Liang Zhen [Tue, 19 Aug 2014 02:55:04 +0000 (10:55 +0800)]
LU-5548 ptlrpc: avoid list scan in ptlrpcd_check
ptlrpcd_check() always scan all requests on ptlrpc_request_set
and try to finish completed requests, this is low efficiency.
Even worse, l_wait_event() always checks condition for twice
before sleeping and one more time after waking up, which means
it will call ptlrpcd_check() for three times in each loop.
This patch will move completed requests at the head of list
in ptlrpc_check_set(), with this change ptlrpcd_check doesn't
need to scan all requests anymore.
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I770016d4a2bcd51603b390b41427eb64ff52f71c
Reviewed-on: http://review.whamcloud.com/12265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jinshan Xiong [Tue, 30 Sep 2014 18:44:54 +0000 (11:44 -0700)]
LU-5062 llite: Solve a race to access lli_has_smd in read case
In vvp_io_read_lock(), it used to decide if to add read lock by
checking lli_has_smd. Accessing lli_has_smd is racy when an empty
file is turned into raid0, therefore, it may result in read requests
are issued without corresponding lock.
Patch on master: http://review.whamcloud.com/10760
Test-Parameters: alwaysuploadlogs testlist=metadata-updates
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I70cb1888b42e0e9928f8346359e2b1f554fe4e8f
Reviewed-on: http://review.whamcloud.com/12139
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andriy Skulysh [Tue, 21 Oct 2014 07:06:43 +0000 (00:06 -0700)]
LU-4904 test: write_append_truncate failed with divide error
fix trunc_size calculation in case of append_size == 1
Don't allow to specify write_max and append_max < 2.
This patch is back-ported from the following one:
Lustre-commit:
85b037f5da6a022919783ef9a87ed6d2a4d102d2
Lustre-change: http://review.whamcloud.com/9948
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=write_append_truncate \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
mdtcount=4 mdssizegb=2 ostcount=7 ostsizegb=2 \
testlist=parallel-scale
Xyratex-bug-id: MRP-1779
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Change-Id: Ia2cf141b3cc78e4f1488a96fc602b78b16869678
Reviewed-on: http://review.whamcloud.com/12358
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Sat, 27 Sep 2014 18:22:55 +0000 (14:22 -0400)]
LU-4839 utils: fix bandwidth ctl in lhsmtool
Use nanosleep for bandwidth control because usleep fails
for times of a second or longer (usleep(2) EINVAL return value). Add
loop in case sleep is woken by signal.
This is a backport to b2_5 of
Lustre-change: http://review.whamcloud.com/#/c/12093/
Lustre-commit:
f74ace3919d60dc05664caa32f5a79abc3a2fe91
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ib58676b878678eb399bf58bb9873d8fb411b3316
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/12405
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
John L. Hammond [Wed, 24 Sep 2014 17:33:10 +0000 (12:33 -0500)]
LU-5659 hsm: add timestamps to lhsmtool log messages
Add epoch timestamps (secs.usecs) to the log messages emitted by
lhsmtool_posix. Change the single use of CT_DEBUG() to
CT_TRACE(). Remove the unused macro CT_PRINTF().
This is a backport to b2_5 of
Lustre-change: http://review.whamcloud.com/#/c/12042/3
Lustre-commit:
b23357064aba6022237dd146c33725529069770d
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I227d00c97f93a09ac9322213d255102b8a3cf612
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/12368
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Sebastien Buisson [Wed, 2 Oct 2013 06:43:08 +0000 (08:43 +0200)]
LU-4044 build: fix 'resource leak' errors
Fix 'resource leak' defects found by Coverity version 6.6.1:
Resource leak (RESOURCE_LEAK)
Variable going out of scope leaks the storage it points to.
Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I1039fa8bd5e82acab0ffcb082d35fca28dfb694c
Reviewed-on: http://review.whamcloud.com/7820
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/12394
James Nunez [Fri, 26 Sep 2014 20:24:09 +0000 (14:24 -0600)]
LU-3696 mdd: decref volatile object after creation
Drop the nlink count on a volatile file after it is created.
It is created with a single reference and then added to the PENDING
directory, which adds a second reference to the inode. When it is
closed it is left with an nlink count=1 and not unlinked as it should
be.
Do not insert volatile files into the ChangeLog, since they are
intended to be temporary files that cannot be accessed after creation
and should not be synched to a backup filesystem.
If llapi_create_volatile_idx() is used on a non-Lustre filesystem then
it should also unlink the temporary filename after opening it, so that
the behaviour is the same. Improve the comments for this function.
Minor cleanups of related functions found during investigation.
This patch is based on Andreas Dilger's patch:
Lustre-change: http://review.whamcloud.com/#/c/10179
Lustre-commit:
1ebc8dc473dd9f948837b9d5b258194637d46525
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ief906362ab2f6f92342c8eabcb9b090a340a5003
Reviewed-on: http://review.whamcloud.com/12076
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Gregoire Pichon [Wed, 18 Jun 2014 09:18:00 +0000 (11:18 +0200)]
LU-1778 llite: fix inconsistencies of root squash feature
Root squash exhibits inconsistent behaviour on a client when
enabled. If a file is not cached on the client, then root will get
a permission denied error when accessing the file. When
the file has recently been accessed by a regular user and is
still in cache, root will be able to access the file without error
because the permission check is only done by the client that
isn't aware of root squash.
While the only real security benefit from root squash is to deny
clients access to files owned by root itself, it also makes sense
to treat file access on the client in a consistent manner
regardless of whether the file is in cache or not.
This patch adds root squash settings to llite so that client is able
to apply root squashing when it is relevant.
Configuration of MDT root squash settings will automatically be
applied to llite config log as well.
Update cfs_str2num_check() routine by removing any modification
of the specified string parameter. Since string can come from ls_str
field of a lstr structure, this avoids inconsistent ls_len field.
Lustre-change: http://review.whamcloud.com/5700
Lustre-commit:
3064318df90ce7019b83c8a7f80e93a9634c00fa
Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ib94a1a4a63c62c434a76bb935faad0fbdfddfec3
Reviewed-on: http://review.whamcloud.com/10744
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Gregoire Pichon [Tue, 17 Jun 2014 14:10:40 +0000 (16:10 +0200)]
LU-1778 libcfs: add a service that prints a nidlist
The libcfs already provides services to parse a string into a nidlist
and to match a nid into a nidlist. This patch implements a service
that prints a nidlist into a buffer.
This is required for instance to print the nosquash_nids parameter
of the MDT procfs component.
Additionally, this patch fixes a bug in return code of
parse_addrange() routine, so that parsing of nids including
a * character works fine ('*@elan' for instance).
Lustre-change: http://review.whamcloud.com/9221
Lustre-commit:
917b0a17d4e50db9f966763925a2dc860d932a3b
Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ifd5a9c224d43c815104eb170c5b854a1951ab47a
Reviewed-on: http://review.whamcloud.com/10743
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Jian Yu [Fri, 14 Nov 2014 03:29:00 +0000 (19:29 -0800)]
LU-5079 tests: fix service_time in max_recovery_time()
This patch fixes the calculation of service_time in
max_recovery_time() to use the new method in
check_and_start_recovery_timer() and new values of
CONNECTION_SWITCH_MAX and CONNECTION_SWITCH_INC.
The patch also fixes replay-dual sub-tests:
- to call wait_clients_import_state() instead of sleeping
uncertain time in test_11()
- to add some margin into the recovery time comparison
in test_20()
Backport to b2_5 from master
Lustre-change: http://review.whamcloud.com/12724
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs mdtcount=1 \
testlist=replay-dual,replay-dual
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs \
ostfilesystemtype=zfs mdtcount=1 \
testlist=replay-dual,replay-dual
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs mdtcount=4 \
testlist=replay-dual,replay-dual
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I8ff0385c508dc4aea883f4159f3f9e55a216527c
Reviewed-on: http://review.whamcloud.com/12714
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Li Wei [Wed, 24 Sep 2014 04:12:37 +0000 (12:12 +0800)]
LU-5654 osd-ldiskfs: Handle holes in osd_ldiskfs_read()
Current osd_ldiskfs_read() incorrectly returns zero and leaves the
corresponding portion of the buffer untouched when a block to be read
is not allocated.
Test-Parameters: testlist=conf-sanity,conf-sanity,conf-sanity mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs
Test-Parameters: testlist=conf-sanity,conf-sanity,conf-sanity mdsfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Change-Id: I51c5fcc5d9e258b536e5d2f070eb9ca6cd67d5aa
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/12319
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Dmitry Eremin [Thu, 9 Oct 2014 20:59:06 +0000 (00:59 +0400)]
LU-4588 code: replace semaphores with mutexes
It's just optimization. The mutex subsystem is slightly faster
and has better scalability for contended workloads.
Remove the lustre_lock and it's accessor functions l_lock(),
l_unlock(), l_lock_init(), and l_has_lock() since they have
not been used by the code since Lustre 1.6.
Lustre-Commit:
34b9fe2f703d91e7ad5b315aaac696d3a314cf0b
Lustre-Review: http://review.whamcloud.com/9294
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Iea30c0853faf1ff18821b1facd2020ef91df1764
Reviewed-on: http://review.whamcloud.com/12258
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jian Yu [Mon, 27 Oct 2014 07:44:44 +0000 (00:44 -0700)]
LU-4669 type: change "%llx" to "LPX64”
This patch fixes the following compile errors by using "LPX64”:
error: format '%llx' expects type 'long long unsigned int',
but argument 5 has type '__u64'
error: format '%#llx' expects type 'long long unsigned int',
but argument 3 has type '__u64'
This patch is back-ported from the following one:
Lustre-commit:
f1eaf5e99aa9093b7a0fac7ae11e741560f64c31
Lustre-change: http://review.whamcloud.com/9811
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I0637f2c0a1e57cf8ab89d8fe9c905145724fded3
Reviewed-on: http://review.whamcloud.com/12432
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Jian Yu [Mon, 27 Oct 2014 07:33:46 +0000 (00:33 -0700)]
LU-4669 ldiskfs: do not export static inline functions
In ldiskfs/ldiskfs_extents.h, both ldiskfs_ext_store_pblock()
and ldiskfs_ext_pblock() are defined as static inline functions.
However, commit
49b06fba39e (http://review.whamcloud.com/4804)
exported these two functions in ldiskfs/extents.c, which is not
allowed by gcc and causes the following compile errors:
error: __ksymtab_ldiskfs_ext_pblock causes a section type conflict
error: __ksymtab_ldiskfs_ext_store_pblock causes a section type conflict
This patch fixes the above errors by not exporting the two functions.
This patch is back-ported from the following one:
Lustre-commit:
e387b0e21c093ebbfb7571cd9e65a98d8e7e5718
Lustre-change: http://review.whamcloud.com/9401
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I9ba2aa39d9d9dc97e74bd29b8513b86eed7fe175
Reviewed-on: http://review.whamcloud.com/12431
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Li Wei [Wed, 17 Sep 2014 05:55:33 +0000 (13:55 +0800)]
LU-4749 mgs: Recognize colons in failover.node values
Current mgs assumes that all the NIDs in a single failover.node
parameter belong to one node. This is not the case, even prior to
http://review.whamcloud.com/11161, because mkfs.lustre options like
--servicenode=<nid1>,<nid2>:<nid3>,<nid4>
are allowed.
Lustre-commit:
bbe5f8ec92c63cc4fcb9ab0b316e92fb2a3d9683
Lustre-change: http://review.whamcloud.com/11956
Change-Id: If3e2ebc0e81093e9c8304e496afdca24edf456ef
Signed-off-by: Li Wei <wei.g.li@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-on: http://review.whamcloud.com/12196
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
James Nunez [Fri, 18 Jul 2014 15:06:36 +0000 (09:06 -0600)]
LU-4298 utils: do not create file with no striping info
'lfs setstripe' should not create a file with no striping
information if bad values are given. The input parameter
check in llapi_file_open_pool is moved before the file open
to catch bad parameters before the file is created.
A check is added to sanity test 27f to make sure
'lfs setstripe -s' does not create a file with no
striping information.
Lustre-commit:
823147fa760752c8c2ff11a5bd74f71f819f9e4e
Lustre-change: http://review.whamcloud.com/8375
Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I19beef973dc3a62bf2cb443e02195896772702fc
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: http://review.whamcloud.com/12335
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bob Glossman [Thu, 23 Oct 2014 16:35:01 +0000 (09:35 -0700)]
LU-5795 kernel: kernel update [SLES11 SP3 3.0.101-0.40]
Update target and config files for new version
Test-Parameters: mdsdistro=sles11sp3 ossdistro=sles11sp3\
clientdistro=sles11sp3 mdsfilesystemtype=ldiskfs\
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iac6e9dc76a6d2bdeb5615bffaeaa5e69652f31ba
Reviewed-on: http://review.whamcloud.com/12402
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Mikhail Pershin [Sat, 25 Oct 2014 05:23:41 +0000 (22:23 -0700)]
LU-2059 mgs: fix a bug in mgs_llog_open
- don't calculate len for NULL ptr
- check the logname is PARAMS_FILENAME inside 'if (ptr == NULL ...)
clause so all cases with NULL ptr will end there.
This patch is back-ported from the following one:
Lustre-commit:
8c21986e79f50131b0f381e5fe0311294328d660
Lustre-change: http://review.whamcloud.com/10589
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ic69866e91acad7920085848faf12c69bf3d17f12
Reviewed-on: http://review.whamcloud.com/12428
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Mikhail Pershin [Sat, 25 Oct 2014 04:58:05 +0000 (21:58 -0700)]
LU-2059 mgs: don't fail on missing params log
The mgc_process_cfg_log() fails if 'params' log is not
accessinble and has no local copy. In fact that log is
optional and may be just empty.
Patch allows 'params' log to be empty and checks also for
'params' log in other places to avoid wrong error messages.
There was also deprecated code in mgs_write_log_direct_all()
which creates lustre-params log which is not used by anyone now.
Third change is removal of exceptions in conf-sanity.sh and
insanity.sh related to LU-2059
The only exception is insanity test_1 which is still failed due
to other reason.
This patch is back-ported from the following one:
Lustre-commit:
864fc9daac267819f5e3bdebef6cdac4c6325626
Lustre-change: http://review.whamcloud.com/10311
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I081ca3920f3c2d05d4e966ed5234369ddc0eaac0
Reviewed-on: http://review.whamcloud.com/12427
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Alexander Boyko [Wed, 29 Oct 2014 21:59:12 +0000 (14:59 -0700)]
LU-5079 ptlrpc: fix early reply timeout for recovery
Commit
8b2f9c0e408 http://review.whamcloud.com/9100 changed
the deadline calculation from current time to request arrival.
During recovery, the new deadline could be less than calculated
at request arrival time. And even worse, the deadline may be
in past. For the first case unneeded early reply would be sent
to client. For the second, client requests would be dropped by
timeout and client reconnect happend.
Do at_measured() for recovery in the same way like general early
reply base on the current time. And set new timeout to the end
of recovery.
Test to check recovery deadline bug.
The patch also speeds up replay-vbr test 7* by decreasing the
at_max value in test_7_cycle().
This patch is back-ported from the following one:
Lustre-commit:
1d889090f2e2902d861d1fab0227c4343127cc42
Lustre-change: http://review.whamcloud.com/11213
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 \
mdtcount=1 mdssizegb=2 ostcount=7 ostsizegb=2 \
testlist=replay-vbr,replay-vbr
Xyratex-bug-id: MRP-1988
Change-Id: I7a5d60ca8e52a805546cc9934eb42bddeaf8ff4e
Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/12365
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Niu Yawei [Thu, 11 Sep 2014 10:57:52 +0000 (06:57 -0400)]
LU-5287 export: hold exp_lock when modify exp_flags
There are few places that changing the exp_flags without holding
exp_lock, that could overwrite other concurrent exp_flags updating.
Lustre-commit:
999fedbd5c18a021064b23933e84e4c92a7f48ac
Lustre-change: http://review.whamcloud.com/11871
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I9c0421ffeacd6bf6feaa31dca9f9fc0e7963bc8b
Reviewed-on: http://review.whamcloud.com/12162
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Christopher J. Morrone [Fri, 19 Sep 2014 01:51:59 +0000 (18:51 -0700)]
LU-5150 mdc: Handle empty but non-zero acl xattr
We have found that posix_acl_access can have a value
of \002\000\000\000. In that case body->aclsize is
non-zero, but the there are no actuall acls stored
in the xattr.
In mdc_unpack_acl(), it only checks IS_ERR() on the
pointer returned by posix_acl_from_xattr(), it does not
check for NULL. Because of the above situation, the
xattr aclsize can be non-zero, but posic_acl_from_xattr()
still returns NULL. Passing NULL to posix_acl_valid()
crashes the kernel.
We add a check to properly handle the NULL return value.
This patch is back-ported from the following one:
Lustre-commit:
867e301972278e397b57094810821ff884092fd8
Lustre-change: http://review.whamcloud.com/10620
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I76fb2198e7233261ee65c6527d4f2b78f27022bc
Reviewed-on: http://review.whamcloud.com/11989
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Tue, 9 Sep 2014 23:24:54 +0000 (17:24 -0600)]
LU-5018 libcfs: don't depend on configure for lustre_user.h
Don't depend on configure to detect the presence of <linux/types.h>
in order to define the __u8, __u16, __u32, and __u64 types. Instead,
just define these types directly if <linux/types.h> is not included
already.
The <linux/types.h> header is available on Linux systems with GCC
installed, which is required to build Lustre anyway. On non-Linux
systems they should create an appropriate <linux/types.h> compat
header in the include path.
Lustre-change: http://review.whamcloud.com/10258
Lustre-commit:
241cdb8ba008d47fd96f0b6529797ce922e2b635
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ida0915fe1b5829f3378e459c782844cf9df060e6
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/11840
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Niu Yawei [Sun, 28 Sep 2014 08:18:15 +0000 (04:18 -0400)]
LU-4629 osd-ldiskfs: Fix use after free
Object 'dir' was dereferenced at line 3343 after being freed
by calling 'kfree' at line 3332.
Lustre-commit:
f4842b027ee57969fa1b01b90d5f23aed18b5eb5
Lustre-change: http://review.whamcloud.com/10609
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I5e27f4555bda6144be5e5fcb9d0f00916512020a
Reviewed-on: http://review.whamcloud.com/12096
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Nunez [Wed, 8 Oct 2014 23:46:28 +0000 (17:46 -0600)]
LU-4768 tests: Update ost-survey script
Currently, ost-survey hangs due to calling
'lfs setstripe' in an old (positional) style and
setting max_cached_mb to zero.
The call to 'lfs setstripe' is updated to use the
'-S', '-i' and '-c' flags. max_cached_mb is now
set to pagesize * 256 (in MB). The patch also gets
parameters for the correct file system if more
than one Lustre file system is mounted, and
corrects a few typos in comments.
In ll_max_cached_mb_seq_write(), the number of
pages requested is set to the max of pages requested
or PTLRPC_MAX_BRW_PAGES to allow the client to make
well formed RPCs.
This is a backport to b2_5
Lustre-change: http://review.whamcloud.com/#/c/11971/4
Lustre-commit:
46bec835ac726aa2d06024b197002be7b112e192
with some changes to ost-survey from
Lustre-change: http://review.whamcloud.com/#/c/10534/8
Lustre-commit:
fe3ddff77475dd487873e2f873835455aab7aa38
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ia9f22c19f95b85d14687bdaf70cad05ca3a3f455
Reviewed-on: http://review.whamcloud.com/12240
Tested-by: Jenkins
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Li Wei [Wed, 22 Oct 2014 13:31:33 +0000 (21:31 +0800)]
LU-5768 target: Revert the test part of the LU-5579 fix
This patch reverts the test part, including the tgt_enqueue() fault
injection, of the LU-5579 fix (
d29c0438). These changes cause
replay-single 52 and 73b to fail frequently.
Change-Id: I63c9210ddd4165cf587272e4216cd39dc608f976
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/12390
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Christopher J. Morrone [Fri, 21 Feb 2014 22:46:47 +0000 (14:46 -0800)]
LU-4624 llite: Avoid statahead thread start/stop deadlocks
The statahead and statahead agl threads blindly set their
thread state to SVC_RUNNING without checking the state first. If, for
instance, another thread sets the state to SVC_STOPPING that
stop signal will now have been lost. Deadlock ensues.
We also partly improve the sai reference counting, because a race exists
where the ll_stop_statahead thread can drop the default reference, and
the statahead thread can exit and drop its reference as well. With no
references on the sai, the final put will poison and free the buffer. The
original do_statahead_enter() function may then continue to access
the buffer after it is freed because it did not take a reference of its
own. We add a local reference to address that.
Lustre-commit:
373d937086e5cede6fc1a3b3f859622c0173c1de
Lustre-change: http://review.whamcloud.com/9358
Change-Id: I531f17966d49ba1e6ebd99abe9cb8b128eeac4f9
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/12182
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Bobi Jam [Mon, 12 May 2014 14:04:23 +0000 (22:04 +0800)]
LU-3188 osc: shorten IO calling path
By using osc_io_unplug_aync() for osc_queue_sync_pages() to shorten
the IO calling path, to reduce the chance of stack overflow.
This is revive of git commit
83ae17df2bdce837e62473aec27c03d67312c8ea.
Lustre-commit:
5bc6be85c7b5434917791f97a199b16f6c8bf533
Lustre-change: http://review.whamcloud.com/10292
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I2ac32866f7adbc4701370704612c849a18a5d1ac
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/11612
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Nathaniel Clark [Sat, 20 Sep 2014 06:50:48 +0000 (23:50 -0700)]
LU-5240 obdclass: Increase minimum LU Cache for ZFS
If the lu cache is too small threads will thrash for data including
the OI tables and other important metadata.
This patch is back-ported from the following one:
Lustre-commit:
a9c51c03eaecb5234ba754b738877323a35bed51
Lustre-change: http://review.whamcloud.com/10802
Test-Parameters: alwaysuploadlogs \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
testgroup=review-zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ia6c85f523219706b5d41054c869e73ed5bf3c2f5
Reviewed-on: http://review.whamcloud.com/12002
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Brian Behlendorf [Sat, 20 Sep 2014 06:43:56 +0000 (23:43 -0700)]
LU-5164 osd: Limit lu_object cache
For OSDs like ZFS to perform optimally it's import that they be allowed
to manage their own cache. This maximizes the likelyhood that the ARC
will prefetch and cache the right buffers. In the existing ZFS OSD
code a cached LU object pins buffers in the ARC preventing them from
being dropped. As the LU cache grows it can consume the entire ARC
preventing buffers for other objects, such as the OIs, from being
cached and severely impacting the performance for FID lookups.
By default this patch will only limit the LU cache for ZFS OSDs.
NOTES:
* Setting LU_CACHE_NR_ZFS_LIMIT to 0 results in an LBUG on the MDS.
This may be because an object is being used without a reference.
Setting a minimum value of 256 was arbitrary, ideally we would
set this value to 0.
* In order to be able to quickly determine the number of objects in
the hash table the CFS_HASH_COUNTER flag is added. This adds an
atomic_inc/dec to the hash insert/remove paths but is not expected
to have any measurable impact of performance.
This patch is back-ported from the following one:
Lustre-commit:
0123baecc4e2050447f8c4f48f5b33a6d3c524a8
Lustre-change: http://review.whamcloud.com/10237
Test-Parameters: alwaysuploadlogs \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
testgroup=review-zfs
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: If573f02175a02abeeb100ed74f3109e629d4c392
Reviewed-on: http://review.whamcloud.com/12001
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Mikhail Pershin [Thu, 9 Jan 2014 16:01:19 +0000 (20:01 +0400)]
LU-4349 target: count disconnected exports as stale
If export is disconnected during recovery then stale client
counter need to be increased.
Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity,conf-sanity,conf-sanity,conf-sanity,conf-sanity
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Id4f2a668dbe8d5cd2fa3d392b7df7b4d67a4265a
Reviewed-on: http://review.whamcloud.com/8785
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Vitaly Fertman [Mon, 29 Sep 2014 23:42:32 +0000 (19:42 -0400)]
LU-5579 ldlm: re-sent enqueue vs lock destroy race
upon lock enqueue re-send, lock is pinned by ldlm_handle_enqueue0,
however it may race with client eviction or even lcok cancel (if
a reply for the original RPC finally reached the client) and the
lock cann be found by cookie anymore:
ASSERTION( lock != NULL ) failed: Invalid lock handle
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: I9d8156bf78a1b83ac22ffaa1148feb43bef37b1a
Xyratex-bug-id: MRP-2094
Oleg Drokin [Mon, 29 Sep 2014 23:42:02 +0000 (19:42 -0400)]
LU-5530 mdt: Properly match open lock and unlock
It seems that when request resend with a corresponding lock match is
in play (thanks to large striping + llnl patch for the client to send
small requests only), after all the suffering and fixing coming from
LU-2827, here is another casualty in in open/lease locking.
mdt_reint_open() and mdt_open_by_fid_lock() might match the lock
on resend and not call mdt_object_open_lock(), yet call
mdt_object_open_unlock() before exit.
Since mdt_object_open_lock/unlock also plays with a semaphore,
hilarity ensues usually most visible as a rw_sem lockup
in mdt_object_open_lock.
This patch adds tracking whenever we actually called mdt_object_open_lock
or not and only calls mdt_object_open_unlock if we did.
Change-Id: I73c529229acec98cac4ad73f7b487e759ad9a763
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Vitaly Fertman [Mon, 29 Sep 2014 23:25:47 +0000 (19:25 -0400)]
LU-5496 ldlm: reconstruct proper flags on enqueue resend
otherwise, waiting lock may get granted as no BLOCKED_GRANTED
flag is returned
Lustre-commit:
175ce732b881e99d77d3f5c11fd1d0d5f604218a
Lustre-change: http://review.whamcloud.com/11644
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Xyratex-bug-id: MRP-1944
Change-Id: I5e938ff0454d5e8694b09f9fff3c1f82d746360d
Reviewed-on: http://review.whamcloud.com/11644
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Vitaly Fertman [Mon, 29 Sep 2014 23:21:21 +0000 (19:21 -0400)]
LU-5266 ldlm: granting the same lock twice on recovery
ASSERTION( lock->l_granted_mode != lock->l_req_mode ) is hit
when resending LDLM_FL_REPLAY.
Don't add lock to waiting list as it is already added or granted.
Lustre-commit :
63851b5816bb30687fbf3750380d6b448e9400f1
Lustre-change: http://review.whamcloud.com/10903
Change-Id: Ib8e5d2c7588f6cacd1723529e70d29f63742caad
Xyratex-bug-id: MRP-1944
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/10903
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Vitaly Fertman [Mon, 29 Sep 2014 23:13:25 +0000 (19:13 -0400)]
LU-2827 ldlm: wrong handling of ldlm resends
If a successful reply is lost for an intent
lock request, MDS will not correctly recover
from this situation on resend. Instead, a new
lock is created.
Lustre-commit: I9a23c13313d5168d0c232fdc0acbdab330089356
Lustre-change: http://review.whamcloud.com/5978
This patch also adds handling of resend during layout-lock
Lustre-commit: Ideb551779a884d77ded8036c513f62557b2dfcde
Lustre-change: http://review.whamcloud.com/#/c/10378
Change-Id: I6745461b2a1370837be7bcbff12dc021cd1021cf
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Frank Zago [Thu, 22 May 2014 18:27:56 +0000 (13:27 -0500)]
LU-5051 hsm: do not reallocate hsm receive buffer
Instead of allocating 1MB for every call to
llapi_hsm_copytool_recv, allocate that buffer once in
llapi_hsm_copytool_register. The application must not call
llapi_hsm_copytool_recv until it has cleared the data from
the previous call, which lhsmtool_posix already does. This
also make llapi_hsm_action_list_free unnecessary.
Lustre-change: http://review.whamcloud.com/10299
Lustre-commit:
76dca841869d87df2b625cb9d186d531c406fc9b
Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: Frank Zago <fzago@cray.com>
Change-Id: Ie9e15a5cb4e3ba6d3fe23d40ee2bb47330280abf
Reviewed-on: http://review.whamcloud.com/10299
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit
76dca841869d87df2b625cb9d186d531c406fc9b)
Reviewed-on: http://review.whamcloud.com/10716
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Frank Zago [Sat, 19 Jul 2014 14:21:10 +0000 (09:21 -0500)]
LU-5385: HSM: do not call the JSON log function if no log is open
llapi_hsm_log_ct_registration() and llapi_hsm_log_ct_progress() are
very expensive (fid2path+allocations). Do not let them do anything
if llapi_hsm_write_json_event() is going to discard the JSON record.
Make llapi_hsm_log_ct_registration() and llapi_hsm_log_ct_progress()
static too.
This is a backport to b2_5.
Lustre-change: http://review.whamcloud.com/11164
Lustre-commit:
35d122fd99c961b0b9fd500b3a78bdc0f5334a97
Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ib10023968a1bd021694bca6338c0a962f58da19a
Reviewed-on: http://review.whamcloud.com/11164
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit
35d122fd99c961b0b9fd500b3a78bdc0f5334a97)
Reviewed-on: http://review.whamcloud.com/11617
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Bruno Faccini [Wed, 12 Mar 2014 13:12:33 +0000 (14:12 +0100)]
LU-4086 tests: strengthen sanity-hsm/test_33
Instead of waiting+checking both Restore and Cancel HSM operations
sequentially, wait for both to be finished and then check each
results. This will allow to handle conditions where Restore
terminates before the Cancel is started.
Also added to wait_all_done() the capability to only work on
requests for a specific fid.
Lustre-change: http://review.whamcloud.com/9600
Lustre-commit:
adc6a5e76348d9bb1c8355103f419405a14decca
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I480ce074513d6829ecb3078c273bac1fdffa6256
Reviewed-on: http://review.whamcloud.com/9600
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit
adc6a5e76348d9bb1c8355103f419405a14decca)
Reviewed-on: http://review.whamcloud.com/10712
Frank Zago [Mon, 11 Aug 2014 23:24:07 +0000 (18:24 -0500)]
LU-5195: hsm: delete HSM records not found on disk
After a MDS crash, it is possible the file containing an hsm record is
not present anymore. When the MDS restarts, we get traces like this:
(mdt_hsm_cdt_actions.c:104:cdt_llog_process()) tas01-MDT0000:
failed to process HSM_ACTIONS llog (rc=-2)
(mdt_hsm_cdt_actions.c:104:cdt_llog_process())
Skipped 600 previous similar messages
(llog_cat.c:192:llog_cat_id2handle()) tas01-MDD0000:
error opening log id 0x1c:1:0: rc = -2
(llog_cat.c:192:llog_cat_id2handle()) Skipped 600 previous similar messages
(llog_cat.c:556:llog_cat_process_cb()) tas01-MDD0000:
cannot find handle for llog 0x1c:1: -2
(llog_cat.c:556:llog_cat_process_cb()) Skipped 600 previous similar messages
(mdt_hsm_cdt_actions.c:104:cdt_llog_process()) tas01-MDT0000:
failed to process HSM_ACTIONS llog (rc=-2)
(mdt_hsm_cdt_actions.c:104:cdt_llog_process())
Skipped 600 previous similar messages
(llog_cat.c:192:llog_cat_id2handle()) tas01-MDD0000:
error opening log id 0x1c:1:0: rc = -2
(llog_cat.c:192:llog_cat_id2handle()) Skipped 600 previous similar messages
No HSM operation can happen, and the only way to clean is to unmount
the MDT and delete hsm_actions.
If the record can't be found, let the MDS delete it instead.
This is a backport to b2_5.
Lustre-change: http://review.whamcloud.com/11419
Lustre-commit:
3a83b4b93373db46a4ee60a7388775fa0be2eb9a
Change-Id: I61f625d4c18750c8044909ff56d53042cf0b6d86
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/11419
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ryan Haasken <haasken@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit
3a83b4b93373db46a4ee60a7388775fa0be2eb9a)
Reviewed-on: http://review.whamcloud.com/11619
Reviewed-by: Patrick Farrell <paf@cray.com>
Bruno Faccini [Fri, 18 Jul 2014 11:34:33 +0000 (13:34 +0200)]
LU-5299 obdclass: avoid race during Server device start
Handle concurrent starts for same device (multiple mounts, ...).
But allows for separate nosvc and nomgs case.
Also add a specific test of concurent MDT/OST start with an
artificial delay to verify.
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I442819a5b865ed3e98477f9d2602efc4d09d7860
Reviewed-on: http://review.whamcloud.com/11139
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Wei [Fri, 25 Jul 2014 16:14:51 +0000 (00:14 +0800)]
LU-1279 kernel: Fix concurrent module loading deadlocks
Concurrently starting multiple OSTs on a single OSS frequently
triggers 30s deadlocks on module_mutex. This RHEL 6 kernel bug
applies to any module that results in additional request_module()
calls in its init callback. In Lustre, at least ptlrpc and libcfs are
affected. (RHEL 7 should have enough fixes in this area, but testing
needs to be done.) This patch adds a fix adapted from a number of
upstream commits to the RHEL 6 kernel patch series.
Change-Id: Ibdd384fd7622a0b4fcbf4cb5fdb864de87fcc25e
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/11229
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Liang Zhen [Tue, 12 Aug 2014 14:36:42 +0000 (22:36 +0800)]
LU-5428 libcfs: false alarm of libcfs watchdog
lc_watchdog_disable will not delete kernel timer, benefit of this
is no overhead of del_timer in each serivce loop, however, there
is a corner case that lc_watchdog_touch can race with lcw_cb when
service thread is actually idle:
1. service thread armed a timer before processing request, after
it handled request, it called lc_watchdog_disable() and set
lc_watchdog as disabled, but kernel timer is still outstanding.
2. service thread slept for many seconds because there is no
request to handle
3. it's waken up by incoming request
4. it called lc_watchdog_touch and set watchdog status to enabled.
now because timer is still alive, if timer is expired right
before cfs_timer_arm(), then lcw_cb will set a disabled watchdog
as "expired", which is wrong.
5. the next call of lc_watchdog_disable will complain for expired
watchdog.
There are two options to fix this issue:
1. always del_timer in lc_watchdog_disable, however, it may increase
system overhead when service threads are busy.
2. set watchdog as "enabled" after cfs_timer_arm, so there is no
window for lcw_cb to see false status of watchdog.
This patch chooses the second approach to fix the problem
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: Iac24082e2d63de8330285cf243ed585da6524ab9
Reviewed-on: http://review.whamcloud.com/11415
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Mon, 25 Aug 2014 08:56:29 +0000 (16:56 +0800)]
LU-5405 llog: add newly opened llog at tail of handle list
Add newly opened llog handle at the tail of handle list to increase
lookup speed, especially for cancel operation, because the canceled
log will be removed from the list, and lookup is from the beginning.
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I4c5c22901ea5818a8ee50ef97d6dabe4839b9a74
Reviewed-on: http://review.whamcloud.com/11575
Tested-by: Jenkins
Reviewed-by: Ian Costello <costello.ian@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Conflicts:
lustre/obdclass/llog_cat.c
Li Xi [Wed, 2 Jul 2014 03:07:20 +0000 (11:07 +0800)]
LU-4933 osc: Automatically increase the max_dirty_mb
When RPC size or the max RPCs in flight is increased, the actual
limit might be max_dirty_mb. This patch automatically increases
the max_dirty_mb value at connection time and when the related
values are tuned manually by proc file system.
This patch also changes the unit of "cl_dirty" and "cl_dirty_max"
in client_obd from byte to page.
Lustre-change: http://review.whamcloud.com/10446/
Lustre-commit:
bdc5bb52c55470cf8020933f80e327c397810603
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I8480122b7370247b17de81d731c2f2b5f67892ce
Reviewed-on: http://review.whamcloud.com/10937
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
wang di [Wed, 2 Apr 2014 16:57:00 +0000 (09:57 -0700)]
LU-4855 fld: refer to MDT0 for fld lookup in some cases
It is possible that when fld client is trying to lookup seq
on one of MDT, but the connection between the client and the MDT
is not being initialized yet, especially during striped dir creation,
because client will only send create req to the master MDT, then
master MDT will distribute the operation to all of other MDT, instead
of client distributing these requests, which will usually trigger
the connection.
In this case, we will send the fld request to MDT0, since it has
all of location information.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I577ada2a782ecbdc863d05e8d007c0914a1e7837
Reviewed-on: http://review.whamcloud.com/9877
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/11780
Rahul Deshmukh [Sat, 30 Aug 2014 00:21:59 +0000 (17:21 -0700)]
LU-5144 mdt: rename vs link deadlock
it may happen that tgt child for rename is a directory, which is used
as tgt dir for link in parallel, as the result rename locks src child,
tgt child what may deadlock with tgt dir, src file locking of link.
As not dir cannot replace dir, check it before locking the tgt child.
Lustre-commit:
b2c2cfc24fa461b3e54cd204a697eeee9a89d49c
Lustre-change: http://review.whamcloud.com/10570/
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=55 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanityn
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Signed-off-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Change-Id: I09e96b132d9069d4818e8dc2628c903eff38e53d
Reviewed-on: http://review.whamcloud.com/10918
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Rahul Deshmukh [Sat, 30 Aug 2014 00:11:07 +0000 (17:11 -0700)]
LU-4725 mdt: child-parent lock ordering in rename
change rename so that it always has parent-child lock ordering,
otherwise it may deadlock with other operations.
Lustre-commit:
4e308ef74f271ec7e19917e3c0f88586537582c3
Lustre-change: http://review.whamcloud.com/9538/
LU-4725 obd: lu_object_find_at hung
lu_object_find_at hungs if called 2 times and object is removed
in between. It makes mdt_rename_sanity not workable for rename.
Change mdt_rename_sanity to work on existing object.
Lustre-commit:
b7c72ec1ddeda2cf94ea151f974d3f94e3a7d1ed
Lustre-change: http://review.whamcloud.com/10484/
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=54 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanityn
Signed-off-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Change-Id: Ic9ce52bfcd8788834347fba155cc8c6be674dcd8
Reviewed-on: http://review.whamcloud.com/11615
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Mon, 22 Sep 2014 12:58:55 +0000 (08:58 -0400)]
LU-5114 utils: Fix autoconf warning
Fix mount_osd_ldiskfs_LDADD autoconf warning. This corrects a fix
left out of commit
0d77156572c2b77581f01d960c0e5738b369de6a.
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2815f093951c215a8c92678348444cbd18297e25
Reviewed-on: http://review.whamcloud.com/12005
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bobi Jam [Mon, 14 Jul 2014 05:51:05 +0000 (13:51 +0800)]
LU-5040 osd: fix osd declare credit for quota
osd_attr_set() always calls ll_vfs_dq_init() to initialize dquot for
the inode, while in some cases osd_declare_attr_set() does not reserve
credit for it.
This patch fixes this issue. This patch also corrects the quota credit
accounting in osd_declare_qid() by judging whether the inode's
i_dquot[quota_type] has been allocated, while old code judgement is
based on the file uid/gid existence, which is not correct.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I16555cb1097e1a3e75cdcb4852a2c5e1772ddd88
Reviewed-on: http://review.whamcloud.com/11097
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Fri, 26 Sep 2014 01:02:55 +0000 (21:02 -0400)]
Revert "LU-3489 osd-zfs: varsize index support"
This seems to be causing LU-5649
This reverts commit
d9e627c09ccc5c22aa10704020fb9f45bcf26ed0.
Bruno Faccini [Wed, 3 Sep 2014 09:09:34 +0000 (11:09 +0200)]
LU-5349 include: use __vmalloc() to avoid __GFP_FS default
When possible, try to Direct use of __vmalloc() which allows
for protection flag specification, and particularly to not
set __GFP_FS, which can cause some deadlock situations in our
code.
This patch is the b2_5 back-port of :
Lustre-commit:
635421641bfa8a412edebd4ffbbdf05fd0c47410
Lustre-change: http://review.whamcloud.com/11190/
Also corrects a typo in __OBD_VMALLOC_VERBOSE name.
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I588ebf27d1ff7538745b732d174696c379107ea3
Reviewed-on: http://review.whamcloud.com/11739
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Stephen Champion [Sat, 13 Sep 2014 03:23:45 +0000 (20:23 -0700)]
LU-5610 tests: Handle quoted module options
When test-framework.sh translates module options to environment
variables for remote nodes, quotes should be escaped to the subshell.
This patch is back-ported from the following one:
Lustre-commit:
4597789df0b6b9fcc7bcbe5529adab3522f0e6ac
Lustre-change: http://review.whamcloud.com/11887
Signed-off-by: Stephen Champion <schamp@sgi.com>
Change-Id: I02daf5ca36170ea901f6b0e4e5152cf4835945dc
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11907
Tested-by: Jenkins
Reviewed-by: Stephen Champion <that.schamp@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bob Glossman [Tue, 9 Sep 2014 19:59:10 +0000 (12:59 -0700)]
LU-5600 kernel: kernel update RHEL6.5 [2.6.32-431.29.2.el6]
Update RHEL6.5 kernel to 2.6.32-431.29.2.el6
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4832cf12cb49eebd1c81fc4e07a53d0a1315500d
Reviewed-on: http://review.whamcloud.com/11838
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Niu Yawei [Fri, 12 Sep 2014 03:30:05 +0000 (23:30 -0400)]
LU-5612 osd: typo in osd_declare_write()
Some bits are lost when porting the fix of LU-4611
3902ff4c54925b2f1fcb732a32ed7ee5428e9f77
That could result in insufficient credits in certain situation.
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I828a75c674978278b5adaf210586baa177ea0dc9
Reviewed-on: http://review.whamcloud.com/11889
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Ned Bass [Thu, 31 Jul 2014 22:07:32 +0000 (15:07 -0700)]
LU-5434 mdd: disregard empty POSIX ACLs
Some files may have a POSIX ACL consisting of only a struct
posix_acl_xattr_header, resulting in unexpected permission errors.
Update mdd_check_acl() to return -EAGAIN when such ACLs are
encountered so the caller falls back to standard UNIX
permissions.
These empty ACLs originate from tools like cp. In some cases (e.g.
'cp -rp') just POSIX_ACL_XATTR_VERSION is given as the value of the
system.posix_acl_default extended attribute. Lustre accepts and
stores this value on disk, and the bad default ACL then propagates
from directories to newly created directory entries. Separate
patches will prevent the storage of these bad ACLs. This patch
mitigates problems caused by the ones already stored.
Other minor updates:
- Use the kernel function posix_acl_xattr_count() to count the
number of ACL entries.
- Convert remainder of mdd_check_acl() to TAB indentation.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: Ib4f7b3874696b7645b9de3f1f10ef45dc4105646
Reviewed-on: http://review.whamcloud.com/11300
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Lai Siyao [Mon, 30 Jun 2014 13:33:25 +0000 (21:33 +0800)]
LU-5150 acl: 'cp -rp' set empty default ACL
'cp -rp' will set an empty default ACL on target, MDD should treat
this as removing specified ACL.
add sanity 103c to verify this.
Signed-off-by: Lai Siayo <lai.siyao@intel.com>
Change-Id: I2b6947e241675c291a43999e6324bbb295d3c88d
Reviewed-on: http://review.whamcloud.com/11158
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Lai Siyao [Thu, 26 Jun 2014 16:20:46 +0000 (00:20 +0800)]
LU-3660 acl: support mount option "noacl" for zfs
zfs parse mount option in zpl layer, but osd-zfs calls dmu
directly, since ldiskfs parse "noacl" at mount time, osd-zfs
should do the same in osd_mount().
Similar to ldiskfs, osd-zfs should check whether ACL is enabled
for ACL operations, if not -EOPNOTSUPP is returned.
add sanity test 103b for "noacl" mount option.
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I11075a9de8759fde956e7ae8b715840a56d0428e
Reviewed-on: http://review.whamcloud.com/11157
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Mon, 21 Jul 2014 13:31:11 +0000 (09:31 -0400)]
LU-4334 utils: Only set a single property for nodes
For servicenode, failnode and mgsnode only set a single property but
concatenate data. ZFS can only set a single property for a given
name so this prevents erroneously overwritting previous entries.
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ia0316e138b0d68dcda1d14811e43db3bbed64457
Reviewed-on: http://review.whamcloud.com/11161
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Alex Zhuravlev [Sun, 28 Jul 2013 07:33:20 +0000 (11:33 +0400)]
LU-3489 osd-zfs: varsize index support
LFSCK needs indices with keys/values of different size.
add a support for this to osd-zfs. key/value size can't
vary within a single index object so far.
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6faa730ce1f778563c1fcf43d3b50c5b62c899d9
Reviewed-on: http://review.whamcloud.com/7152
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Fri, 29 Aug 2014 02:38:24 +0000 (22:38 -0400)]
New tag 2.5.3-RC1
Change-Id: I386568268b31090e3909519ff1cfcc304400a436
Emoly Liu [Mon, 25 Aug 2014 05:53:04 +0000 (13:53 +0800)]
LU-4167 tests: correct version check to enable ff_convert
In conf-sanity.sh test_32d, filter fid convertion can be made only
when both of the following conditions are satisfied:
- client img version < 2.3.64 (struct filter_fid_old has been
introduced since 2.3.64)
- ost server version >= 2.5
Also, this patch fix the ofd_iocontrol() message to print the ioctl()
CMD argument as hex instead of a signed integer.
Test-Parameters: testlist=conf-sanity envdefinitions=SLOW=yes,ONLY=32 ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 ossdistro=el6 mdsdistro=el6
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I43b2fc9b62b84b37f6fc8245ede584547a53c878
Reviewed-on: http://review.whamcloud.com/11574
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Frank Zago [Wed, 2 Jul 2014 16:35:18 +0000 (11:35 -0500)]
LU-5095 hsm: Allow receiving messages to be non-blocking
When using the HSM API for a copy tool, it's impossible to
properly exit an application because one thread might be
stuck in llapi_hsm_copytool_recv(), which in turn blocks
forever on a read.
So this patch expands the existing Lustre userspace HSM API
to be able to apply flags to the read side of the
communication pipe, so it can be opened with O_NONBLOCK,
and gives access to the pipe file descriptor used. (read side)
So now a program can do a poll/select, and then call
llapi_hsm_copytool_recv without blocking.
Notes: The flags passed in from the copy tool to
llapi_hsm_copytool_register are applied to the read side of
the pipe using fcntl, because we do not want to apply
the same flags to the read and write sides of the pipe.
No functionality is added for setting flags on the write
side of the pipe, since the kernel expects particular
behavior from that side of the pipe, and it seems unwise
to let user space modify it.
Additionally, this patch changes llapi_hsm_copytool_recv to
not return -EAGAIN when it finds a message for an archive
not serviced by that copytool.
Instead of returning -EAGAIN to the copy tool, which just
restarts the receive operation, this patch just loops inside
the recv function.
This is because this -EAGAIN conflicts with -EAGAIN
returned from the call to libcfs_ukuc_msg_get
which can occur when O_NONBLOCK is set.
This is a backport to b2_5
Lustre-change: http://review.whamcloud.com/10393
Lustre-commit:
35b3a429d1915ec147f01a42c4ec0526b887d1c7
Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I3e611d3c259de9a8c30f3939fb5e48ab88210c2e
Reviewed-on: http://review.whamcloud.com/11220
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jian Yu [Thu, 21 Aug 2014 05:46:41 +0000 (22:46 -0700)]
LU-5388 tests: add version check code into replay-dual test 24
This patch adds Lustre version check code into replay-dual test 24
to make the test interoperate with servers that do not have change
of http://review.whamcloud.com/11025 .
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=24 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=replay-dual
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Id4bf872ee4e2d478c2b6cc809711692084e990fd
Reviewed-on: http://review.whamcloud.com/11539
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bob Glossman [Thu, 17 Jul 2014 18:26:58 +0000 (11:26 -0700)]
LU-5363 kernel: kernel update [SLES11 SP3 3.0.101-0.35]
Update target and config files for new version.
Back out change from LU-4351 that skips sanity test 54c.
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I66713caa53d078f62a345a0cb3e32f0d426a7e36
Reviewed-on: http://review.whamcloud.com/11140
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Tue, 19 Aug 2014 22:39:14 +0000 (18:39 -0400)]
Revert "LU-3929 lwp: don't connect LWP to old MDT"
This causes LU-5298
This reverts commit
220c4ec0eb7b1f2145c55b923fe1461383f81a77.
Change-Id: Ifffa567650e0da3408625a1f4cca2dac473db5a3
Frank Zago [Mon, 18 Aug 2014 15:48:17 +0000 (08:48 -0700)]
LU-5252 hsm: llapi_hsm_event_fd is local, so make it static.
This is a backport to b2_5.
Lustre-change: http://review.whamcloud.com/10808
Lustre-commit:
9feb80c65b077204f7091e90f314098266fb7e82
Change-Id: I9905d0251702a11a9c2546ba4bd78fd00658d28a
Signed-off-by: Frank Zago <fzago@cray.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11362
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Mon, 18 Aug 2014 16:55:24 +0000 (09:55 -0700)]
LU-4090 kernel: a kernel patch for jbd2 hung
patch get from linux vanilla kernel commit
0ef54180e0187117062939202b96faf04c8673bc (v3.10-rc2)
jbd2:drop checkpoint mutex when waiting in __jbd2_log_wait_for_space()
While trying to debug an an issue under extreme I/O loading
on preempt-rt kernels, the following backtrace was observed
via SysRQ output:
rm D
ffff8802203afbc0 4600 4878 4748 0x00000000
ffff8802217bfb78 0000000000000082 ffff88021fc2bb80 ffff88021fc2bb80
ffff88021fc2bb80 ffff8802217bffd8 ffff8802217bffd8 ffff8802217bffd8
ffff88021f1d4c80 ffff88021fc2bb80 ffff8802217bfb88 ffff88022437b000
Call Trace:
[<
ffffffff8172dc34>] schedule+0x24/0x70
[<
ffffffff81225b5d>] jbd2_log_wait_commit+0xbd/0x140
[<
ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
[<
ffffffff81223635>] jbd2_log_do_checkpoint+0xf5/0x520
[<
ffffffff81223b09>] __jbd2_log_wait_for_space+0xa9/0x1f0
[<
ffffffff8121dc40>] start_this_handle.isra.10+0x2e0/0x530
[<
ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
[<
ffffffff8121e0a3>] jbd2__journal_start+0xc3/0x110
[<
ffffffff811de7ce>] ? ext4_rmdir+0x6e/0x230
[<
ffffffff8121e0fe>] jbd2_journal_start+0xe/0x10
[<
ffffffff811f308b>] ext4_journal_start_sb+0x5b/0x160
[<
ffffffff811de7ce>] ext4_rmdir+0x6e/0x230
[<
ffffffff811435c5>] vfs_rmdir+0xd5/0x140
[<
ffffffff8114370f>] do_rmdir+0xdf/0x120
[<
ffffffff8105c6b4>] ? task_work_run+0x44/0x80
[<
ffffffff81002889>] ? do_notify_resume+0x89/0x100
[<
ffffffff817361ae>] ? int_signal+0x12/0x17
[<
ffffffff81145d85>] sys_unlinkat+0x25/0x40
[<
ffffffff81735f22>] system_call_fastpath+0x16/0x1b
What is interesting here, is that we call log_wait_commit, from
within wait_for_space, but we are still holding the checkpoint_mutex
as it surrounds mostly the whole of wait_for_space. And then, as we
are waiting, journal_commit_transaction can run, and if the
JBD2_FLUSHED bit is set, then we will also try to take the same
checkpoint_mutex.
It seems that we need to drop the checkpoint_mutex while sitting in
jbd2_log_wait_commit, if we want to guarantee that progress can be
made by jbd2_journal_commit_transaction(). There does not seem to be
anything preempt-rt specific about this, other then perhaps increasing
the odds of it happening.
This patch is back-ported from the following one:
Lustre-commit:
f8e80f01d6883c3e55841528ac16811f6588e260
Lustre-change: http://review.whamcloud.com/11041
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I9adc35d316d7ea0379d0df2d38276297f682f8f7
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11492
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Thu, 5 Jun 2014 02:14:11 +0000 (22:14 -0400)]
LU-4579 ldlm: Properly display time on lock timeout
When lock timeout happens, need to printtime since last lock
activity, not since when the lock was granted.
As such set l_last_activity to current time when sending callbacks too.
Lustre-change: http://review.whamcloud.com/10601
Lustre-commit:
f3864905b26a90ed3a9bb6f99357264fa40015c4
Change-Id: I31d0de6d9add67865cfcb16af903647f7cb932a0
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10601
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-on: http://review.whamcloud.com/10997
Niu Yawei [Wed, 13 Aug 2014 18:45:46 +0000 (11:45 -0700)]
LU-4345 osp: store valid bits in setattr record
Backport a series of 4 patches to resolve the random ID on OST
objects problem.
1: store valid bits in setattr record (LU-4345)
We'd store LA_UID/LA_GID bit along with the UID/GID in the setattr
llog record, otherwise, osp could set a random uid/gid to the OST
object.
Lustre-commit:
80f90fcde73e2faff8b7b0ffc7c19bc52982e027
Lustre-change: http://review.whamcloud.com/10223
2: Correctly check for invalid setattr record (LU-5188)
Patch for LU-4345 (commit
80f90fcde73e2faff8b7b0ffc7c19bc52982e027 )
has a correct comment about lsr_valid member being either 0 or
having UID and GID fields set, but the check has a typo causing
it to check for lsr_valid to be both 0 and have the bits set which
is impossible.
The osp_sync_new_setattr_job() should return 0 for invalid record,
so that sync thread can continue processing on other records.
Lustre-commit:
79dd530f1352e6b57fcb870a1e0f2c2a05a0648d
Lustre-change: http://review.whamcloud.com/10706
3: don't skip attr_set for osp objects (LU-5296)
The lsr_valid handling in osp_sync_add_rec() was got a problem in
commit
80f90fcde73e because it stored all of the passed attr flags
in struct llog_setattr64_rec, even though there are only fields for
storing the UID and GID. Since the time stamps do not need to be
propagated to the OSTs (the MDT values are good enough), this is
not a problem. Also, osp_sync_add_rec stored LA_UID/LA_GID flags
on disk, but they are not part of lustre_idl.h or lustre_disk.h and
are not guaranteed to be constant over time. Instead, use the flags
OBD_MD_FLUID/OBD_MD_FLGID that are in lustre_idl.h.
Lustre-commit:
2d5a5e81660adbcd05f14a10bc6c246d6f108d10
Lustre-change: http://review.whamcloud.com/10989
4: return 1 if osp_sync_xxx_job issue RPC (LU-5188)
Return 1 if osp_sync_new_xxx_job() issue RPC, so
sp_sync_process_record() can decrease opd_syn_rpc_in_flight
and opd_syn_rpc_in_progress correctly if RPC is not
being sent, otherwise the opd_sync_thread will not be
stopped, caused LBUG (see LU-5244)
Lustre-commit:
73f47cdda42305df0be57e0ea6252eb6fd36db55
Lustre-change: http://review.whamcloud.com/10828
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,ONLY=34 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanity-quota
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ie231465b274627561e7eb58f30b868472751bd71
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11435
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Brian J. Murrell [Thu, 29 May 2014 11:49:46 +0000 (07:49 -0400)]
LU-5120 Don't really need the default-base rpm
The kernel-default-base is not actually needed to build Lustre
on SLES11 so get rid of it.
Clean up the fetching of needed packages.
Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Change-Id: Ieee2b01f9a8c280de5d2cc4c43ff672539eea7ef
Reviewed-on: http://review.whamcloud.com/10485
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Frank Zago [Fri, 20 Jun 2014 16:58:50 +0000 (11:58 -0500)]
LU-5252 hsm: make path argument to fifo event a const
Neither llapi_hsm_register_event_fifo() nor
llapi_hsm_unregister_event_fifo() take a const as their argument.
This leads to the following compilation warning when a caller is
using a const char *:
... warning: passing argument 1 of ‘llapi_hsm_register_event_fifo’
discards ‘const’ qualifier from pointer target type
[enabled by default]
rc = llapi_hsm_register_event_fifo(lustre_fifo_event);
^
... note: expected ‘char *’ but argument is of type ‘const char *’
extern int llapi_hsm_register_event_fifo(char *path);
Making path a const fixes this warning.
This is a backport to b2_5
Lustre-change://review.whamcloud.com/10807
Lustre-commit:
8264927b92b82e5ec6d223523db8680ee8df1da4
Change-Id: Ia2544cafefef8d5d1e4cf4cfc6408aac373c447b
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/10807
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit
8264927b92b82e5ec6d223523db8680ee8df1da4)
Reviewed-on: http://review.whamcloud.com/11363
Bobi Jam [Thu, 14 Aug 2014 21:54:59 +0000 (14:54 -0700)]
LU-5263 llite: update ras stride offset
When a read ahead does not reach the end of the region reserved from
ras, we'd set ras::ras_next_readahead back to where we left off; For
stride read ahead, it needs to make sure that the offset is no less
than ras_stride_offset, so that the stride read ahead can work
correctly.
This patch is back-ported from the following one:
Lustre-commit:
9e280281b8daf8ad69d04a5371121224ace28fbc
Lustre-change: http://review.whamcloud.com/11181
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I5d319a94435833153c52da6ccc1b20c4952789e9
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11455
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Tue, 12 Aug 2014 14:58:53 +0000 (22:58 +0800)]
LU-5415 ldlm: high load because of negative timeout
When the time of LRU resizing exceeds waiting period of
recalculation, the ldlm daemon will keep on resizing without any
interval of rest. That will cause high CPU load.
This patch fixes the problem by setting the recalculation timestamp
after LRU resizing finishes rather than before it. What is more, an
interval of one second is enforced between each recalculation.
Lustre-change: http://review.whamcloud.com/11227/
Lustre-commit:
381492e1fc3f93873bf495885859c4f63ce0f4fe
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I989255eca64d74dfcb2087060f627c2e72615248
Reviewed-on: http://review.whamcloud.com/11414
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Nunez [Tue, 12 Aug 2014 17:46:50 +0000 (11:46 -0600)]
LU-5252 hsm: Do not unconditionally delete the HSM event fifo
The lustreapi HSM library will use a pipe if it already exists.
However the deregistration will unconditionally delete that pipe,
which is not correct.
An admin may want to create the pipe with certain rights/ownership,
which the llapi_hsm_register_event_fifo() doesn't allow. In that case,
llapi_hsm_unregister_event_fifo() should not delete it.
This is a back port to b2_5.
Lustre-change: http://review.whamcloud.com/10809
Lustre-commit:
7b7e6e2fb692eb3fa8d386addbc33763acf1a397
Signed-off-by: frank zago <fzago@cray.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I66e44f21217228c5f425e5a9312d8e381d564c7f
Reviewed-on: http://review.whamcloud.com/11421
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Wu Libin [Thu, 14 Aug 2014 21:27:21 +0000 (14:27 -0700)]
LU-5128 obdclass: fix a race in recovery
in "class_export_recovery_cleanup", the check of the flag
"exp->exp_req_replay_needed" should be protected by "exp_lock".
Lustre-commit:
5a7e4480fc519c8df4ca0b11e6d5239956316bd3
Lustre-change: http://review.whamcloud.com/10849
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Wu Libin <lwu@ddn.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ib3c980101f400ce1b9cb0b3d2a4418fbec31b94b
Reviewed-on: http://review.whamcloud.com/11102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Thu, 14 Aug 2014 21:10:35 +0000 (14:10 -0700)]
LU-5126 libcfs: fix illegal page access of tracefiled()
After failure happens and put_pages_back() returns the pages,
tracefiled() should not go on itering on the page list.
Otherwise, some pages might be accessed illegally.
This patch is back-ported from the following one:
Lustre-commit:
f111fac3f785400f892ca03bfb9078c6bd091f7d
Lustre-change: http://review.whamcloud.com/10524
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I9f2953e9e0a6473112c7d1ce16375b84c66d622c
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11454
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Matt Ezell [Wed, 26 Mar 2014 15:00:16 +0000 (11:00 -0400)]
LU-4824 utils: lfs find should continue after errors
'lfs find' stops as soon as it encounters an error, such as EACCES.
Change the behavior to match the regular find command: just note the
failure but continue on, returning a non-zero exit code at the end.
Lustre-change: http://review.whamcloud.com/9794
Lustre-commit:
785e7dc56613e79bdfd89fab762eeb82deb7564c
Signed-off-by: Matt Ezell <ezellma@ornl.gov>
Change-Id: I16ca35ec193e36134e7b93d91a20ff1d0778fe60
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/11093
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Hongchao Zhang [Tue, 12 Aug 2014 20:55:15 +0000 (13:55 -0700)]
LU-3326 tests: sync after file deleted in run_*.sh
In run_*.sh, call sync to flush any dirty pages to OST
if MDT has not sent the object destroy request to it,
which will shrink the available disk space gotten before
running the tests and cause -EIO error.
This patch is back-ported from the following one:
Lustre-commit:
58c2a357202ba83d26ba9d8f1d9c093e3272e9a0
Lustre-change: http://review.whamcloud.com/8071
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,REQFAIL=100,DURATION=28800 \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 \
ossarch=x86_64 mdsarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
mdssizegb=10 ostsizegb=10 austeroptions=-R failover=true iscsi=1 \
testlist=recovery-mds-scale
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I922774dcbf5546a9f5e3fe345bac954fd73d50a7
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/11425
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Yang Sheng [Mon, 28 Jul 2014 13:57:32 +0000 (21:57 +0800)]
LU-5403 kernel: kernel update RHEL6.5 [2.6.32-431.23.3.el6]
Update RHEL6.5 kernel to [2.6.32-431.23.3.el6].
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I798991bbce4b30860b7b35096b430144eb2a1a9c
Reviewed-on: http://review.whamcloud.com/11246
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Dmitry Eremin [Mon, 21 Jul 2014 09:36:57 +0000 (13:36 +0400)]
LU-5334 lnet: fix build with OFED-3.5-2-MIC-beta1
The stuff for RHEL6.4 should be guarded by CONFIG_COMPAT_RHEL_6_4
also in RHEL6.5. So, we need both _COMPAT_ defines in RHEL6.5.
Lustre-Review: http://review.whamcloud.com/11104
Lustre-Commit:
fa4a67d5b4871e20acd07076e7fa15613695cae5
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I322dd2ef167c7b3c272b766ee2cd3f0d464e0a13
Reviewed-on: http://review.whamcloud.com/11153
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Nunez [Thu, 24 Jul 2014 16:42:05 +0000 (10:42 -0600)]
LU-5332 hsm: Release XATTR lock on exiting
This should be a typo where it should jump to out_unlock instead of
out_ucred.
This is a backport to b2_5.
Lustre-change: http://review.whamcloud.com/11083
Lustre-commit:
393350b37cae30efc03ffc076c77e390999c2a2d
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I03242e40f7e33a115934081375be7c86044fa254
Reviewed-on: http://review.whamcloud.com/11215
Tested-by: Jenkins
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Wu Libin [Wed, 25 Jun 2014 08:04:51 +0000 (16:04 +0800)]
LU-5224 lnet: build failed with MLNX_OFED-2.2
Build lustre client can fail on the SLES11 SP3 with
MLNX_OFED-2.2 against the kernel-3.0.76-0.11, which will
cause the "'pm_qos_req' has incomplete type" error.This patch
put "#include <linux/compat-2.6.h>" in front of
"#include <net/sock.h>" to solve this problem.
This error come with the configure option
"--with-o2ib=/usr/src/ofa_kernel/default/", it affect the
search path of the compiler, which will search the header
files in this directory before the kernel header files
directory.
This patch change this way because "net/sock.h"(exactly is
"linux/netdevice.h") include header file "linux/pm_qos_params.h",
the pm_qos_params.h is exist both in ofa_kernel include
directory and kernel include directory, so we need to put the
linux/compat-2.6.h in front of net/sock.h to solve the
compatible problem.
Signed-off-by: Wu Libin <lwu@ddn.com>
Change-Id: I98df2a5faaecbd44528d3bd8a7ba796cc1d14cdc
Reviewed-on: http://review.whamcloud.com/10819
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Thu, 23 Jan 2014 20:02:51 +0000 (15:02 -0500)]
LU-5203 zfs: Update spl/zfs version to 0.6.3
Update SPL and ZFS version to official 0.6.3.
This also fixes the "search path" for libzfs.so.
Lustre-commit:
1f5fa56a0b824e684c32d382d0108c92d9125c5a
Lustre-change: http://review.whamcloud.com/8979
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I710d254d93ce9c3db64ab4bf250f8bf556c684aa
Reviewed-on: http://review.whamcloud.com/11298
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Nathaniel Clark [Thu, 1 May 2014 21:13:45 +0000 (17:13 -0400)]
LU-4606 utils: Mount uses so for backing fstype
Make mount/mkfs/tunefs functionality a per backing FSTYPE shared
object that is loadable, and seperately buildable and installable
in the appropriate lustre-osd-FSTYPE rpm.
For ZFS this allows the shared object to be directly linked to libzfs
and thus remove the hardcoding of a libzfs.so version in the lustre
source. This also has the benefit of making lustre-osd-zfs explicitly
requires libzfs (which it needs to do mount/mkfs operations).
Adjust build system to allow building of loadable module. Including
requiering libtool as part of the build.
Lustre-change: http://review.whamcloud.com/10193
Lustre-commit:
3e8c354c09725a97b90e477a77411cc6fb306f09
LU-5091 build: Support alternate locations of libzfs.so
Due to the change for with-zfs-devel (LU-3497), also add support to
search for libzfs.so along with the alternate locations supplied with
the header files.
Also need to unpack zfs{,-devel}. The zfs RPM is needed because it
has the actual libraries in it and the zfs-devel RPM only has un-
versioned symlinks to the libraries in the zfs RPM.
This will all change one day when we have a libzfs RPM per
https://github.com/zfsonlinux/zfs/issues/2329 and it looks like it
could be one day soon: https://github.com/zfsonlinux/zfs/pull/2341.
Lustre-change: http://review.whamcloud.com/10401
Lustre-commit:
b2659030edb7d34c91430ff924bab23881b6c230
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I1d69b64eec1589f63e24f20cc624778a4be2ec60
Reviewed-on: http://review.whamcloud.com/11297
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Li Xi [Wed, 30 Jul 2014 07:58:56 +0000 (15:58 +0800)]
LU-4920 quota: always clear lqe_lockh on lock cancel
In the qsd_id_blocking_ast(), lqe_lockh should always be cleared,
otherwise, quota code will mistakenly assume the id lock is still
valid.
Lustre-change: http://review.whamcloud.com/9965/
Lustre-commit:
73ef5dcd8bcf32e4127e0011357b8bc39472cdba
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I653e263176a5415ec32324479fed86d948958ef0
Reviewed-on: http://review.whamcloud.com/11278
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jinshan Xiong [Mon, 23 Jun 2014 21:20:52 +0000 (14:20 -0700)]
LU-4786 osc: to not pick busy pages for ELC
This is a combination of commit
154fb1f7 from LU-3321 and commit
bfae5a4e from LU-4300. This patch is only required in b2_5 as the
code is already in 2.6.
Use weigh_ast to decide if a lock covers any pages.
In recovery, weigh_ast will be used to decide if a DLM read lock
covers any locked pages, or it will be canceled instead being
recovered.
The problem with the original implementation is that it attached
each osc_page to an osc_lock also changed lock state to add every
pages for readahead.
Bugzilla-bug-Id: b=16774
Change the policy of ELC to pick locks that have no dirty pages,
no page in writeback state, and no locked pages.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie857a4dd4e235f0c3273f53cbfab9592c2793ac9
Reviewed-on: http://review.whamcloud.com/10795
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
git://git.whamcloud.com / fs / lustre-release.git / log