Whamcloud - gitweb
Amir Shehata [Tue, 11 Jun 2019 18:25:27 +0000 (11:25 -0700)]
LU-12424 lnet: prevent loop in LNetPrimaryNID()
If discovery is disabled locally or at the remote end, then attempt
discovery only once. Do not update the internal database when
discovery is disabled and do not repeat discovery.
This change prevents LNet from getting hung waiting for
discovery to complete.
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I4543b0f71e6cf297a1a5f058ebcc6bf74b8ac328
Reviewed-on: https://review.whamcloud.com/35191
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Tested-by: Jenkins
Reviewed-by: Chris Horn <hornc@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38890
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
James Nunez [Fri, 5 Jun 2020 15:15:01 +0000 (09:15 -0600)]
LU-13149 tests: change sanityn 103 facet value
The facet name input to lustre_version_code() in sanityn
test 103 should be 'ost1' not a variable '$ost1'. Let's
replace this call with the $OST1_VERSION variable.
Fixes:
2548cb9e32bfca ("LU-11670 osc: glimpse - search for active lock")
Test-Parameters: trivial
Test-Parameters: serverversion=2.10.8 serverdistro=el7.6 env=ONLY=103 testlist=sanityn
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ib7426f78210c9b32ba53c46ba5f08faeb3ea8ec5
Reviewed-on: https://review.whamcloud.com/38847
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
James Nunez [Fri, 5 Jun 2020 17:21:09 +0000 (11:21 -0600)]
LU-11782 tests: add version check to conf-sanity 117
conf-sanity test 117 was added to check error returns from
read_param(). This test will fail when run with servers
with Lustre version less than 2.12.0 and, thus, should be
skipped for all Lustre servers earlier than 2.12.0.
Fixes:
6ca2425ccf6b ("LU-11198 utils: propagate errors for read_param")
Test-Parameters: trivial
Test-Parameters: serverversion=2.10.8 serverdistro=el7.6 env=ONLY=117 testlist=conf-sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ia0889584d9c1a6c09ea2a99fa11c7abfd1474de4
Reviewed-on: https://review.whamcloud.com/38851
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
James Nunez [Fri, 5 Jun 2020 16:55:44 +0000 (10:55 -0600)]
LU-13640 tests: add version check to conf-sanity 125
In Lustre 2.12.3, the l_tunedisk utility was modified to
skip tuning devices on the MDS and MGS and conf-santity
test 125 was added to check this functionality. Thus, this
test should be skipped for all Lustre server versions prior
to 2.12.3.
Fixes:
bab0570ce3081 ("LU-12387 tests: Validate l_tunedisk max_sectors_kb tuning")
Test-Parameters: trivial
Test-Parameters: serverversion=2.10.8 serverdistro=el7.6 env=ONLY=125 testlist=conf-sanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I89c2900c2430ff3e76bee297809957380404aa31
Reviewed-on: https://review.whamcloud.com/38850
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 19 Dec 2019 05:55:35 +0000 (16:55 +1100)]
LU-13088 ldlm: Fix sleeping function called in atomic
target_recovery_overseer() can sleep while holding a spinlock, which
triggers a BUG warning.
It is easily fixed by dropping the spinlock before waiting. In the
case where the task waits, no useful information that could be
protected by the spinlock is held, so nothing can be lost by dropping
it.
Lustre-change: https://review.whamcloud.com/#/c/37063/
Lustre-commit:
b29b9310dafe17ba78e1db490b79b89d2d6fdcd1
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I8bb3d02523b5dcfadac19f01ccb736d7b7f28239
Reviewed-on: https://review.whamcloud.com/37063
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39283
Hongchao Zhang [Wed, 24 Jun 2020 09:53:55 +0000 (17:53 +0800)]
LU-13653 mdt: ignore quota when creating slave stripe
When creating striped directory, the quota limit has been checked
on master MDT, the quota should be ignored when creating the slave
stripe object.
Lustre-change: https://review.whamcloud.com/#/c/38875/
Lustre-commit:
f762acebfcc6a88c3f4ba6296cbd6f1696bff530
Change-Id: Ia53b1975a8d66c78725feb313659f7a9b889e735
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38875
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/39282
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Lai Siyao [Wed, 24 Jun 2020 12:01:08 +0000 (20:01 +0800)]
LU-13709 utils: 'lfs mkdir -i -1' doesn't work
'lfs mkdir -i -1 -c...' is to create directory on MDTs by space usage,
when stripe count is more than 1, the target MDT list is not correctly
initialized, which will cause command fail.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id4584940cec390a9245e888c96c7873f5afa209e
Reviewed-on: https://review.whamcloud.com/39165
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Fri, 12 Jun 2020 14:14:50 +0000 (17:14 +0300)]
LU-13600 ptlrpc: limit rate of lock replays
Clients send all lock replays at once and that may overwhelm
server with huge amount of replays in recovery queue causing
OOM effects.
Patch adds rate control for lock replays on client.
Patch includes also later fix for signal_completed_replay()
race.
Lustre-change: https://review.whamcloud.com/38920
Lustre-commit:
3b613a442b8698596096b23ce82e157c158a5874
Lustre-change: https://review.whamcloud.com/39140
Lustre-commit:
dc654756af63bd30802ebd86074019d1533a4d8f
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ie557f8481c5facb690468d7136cf5feebe4e8f11
Reviewed-on: https://review.whamcloud.com/39111
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Tue, 7 Jul 2020 18:13:05 +0000 (11:13 -0700)]
LU-13657 kernel: kernel update RHEL8.2 [4.18.0-193.6.3.el8_2]
Update RHEL8.2 kernel to 4.18.0-193.6.3.el8_2 for Lustre client.
Test-Parameters: trivial clientdistro=el8.2
Change-Id: Id9eb16b9277bf2157905eb38a23a3250a0033560
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38903
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 10 Jun 2020 21:34:03 +0000 (14:34 -0700)]
LU-13503 mdc: allow setting max_mod_rpcs_in_flight larger
Allow setting mdc.*.max_mod_rpcs_in_flight > mdc.*.max_rpcs_in_flight
by increasing the latter value, rather than returning an error and
telling the user to do that. This matches the similar behavior if
mdc.*.max_rpcs_in_flight is reduced lower than max_mod_rpcs_in_flight.
If there are multiple MDTs, the "mdc.*.max_mod_rpcs_in_flight" param
may be set from e.g. the MDT0000 config log before MDT0001 is fully
configured, catching MDT0001 with ocd_maxmodrpcs = 0 before the OCD
from the MDT has been filled in, and incorrectly trigger an error.
If seen during setup, allow ocd_maxmodrpcs = (max_rpcs_in_flight - 1),
since this will be fixed up later if mdc.*.max_rpcs_in_flight is set
smaller in the config log (if set larger it doesn't matter).
Test-Parameters: env=ONLY=90 testlist=conf-sanity
This patch is back-ported from the following one:
Lustre-commit:
6d314902e6d19229379577aab60d4b20a5b4d2ea
Lustre-change: https://review.whamcloud.com/38455
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I4b20163e9e212db451738169ebdc361ab8c1c15e
Reviewed-on: https://review.whamcloud.com/38893
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Nathaniel Clark [Tue, 19 Nov 2019 14:52:45 +0000 (09:52 -0500)]
LU-12100 tests: Use least qunit to set limit
Use least qunit to set lower limit for inodes in sanity-quota/2
This ensures that the limit is set at or above the minimum size.
Lustre-change: https://review.whamcloud.com/36797
Lustre-commit:
33e500cfb33406b8dddac46e1dfb5a3d59ff01c5
Test-Parameters: trivial
Test-Parameters: env=ONLY=2 testlist=sanity-quota
Test-Parameters: env=ONLY=2 testlist=sanity-quota fstype=zfs
Test-Parameters: env=ONLY=2,ONLY_REPEAT=20 fstype=zfs testlist=sanity-quota
Test-Parameters: mdtcount=2 mdscount=4 env=ONLY=2,ONLY_REPEAT=20 fstype=zfs testlist=sanity-quota
Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I80e2c3cb66870d11f74f34c435e266a46630479b
Reviewed-on: https://review.whamcloud.com/36797
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/38769
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Wed, 22 Apr 2020 05:28:54 +0000 (13:28 +0800)]
LU-13473 llite: don't check mirror info for page discard
The CIT_MISC is used for locks/pages manipulation, it will not
go with full io procedure, i.e. cl_io_loop() will not be called
for it. So don't check it for plain file since the mirror info
is not initialized/set in this case.
Lustre-change: https://review.whamcloud.com/38307
Lustre-commit:
d0dd744ed6ae002f34bdade993428b635b23d072
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I723d18260629b8f7c470d350d6d899d3bb88018a
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38856
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 17 Oct 2019 07:19:26 +0000 (16:19 +0900)]
LU-12865 tests: fix sanity 160f to be more robust
The sanity test_160f test was failing intermittently because the first
Changelog user ("cl6") was being unregistered in some cases when it
set changelog_max_idle_time=10, but the test slept for 9s and then did
some operations that could be slow. In rare cases the test runs too
long and the MDS evicts the "good" user along with the bad user:
MDD0000: Force deregister of ChangeLog user cl7 idle more than 35s
MDD0000: Force deregister of ChangeLog user cl6 idle more than 11s
Change the test sleep interval to be half of the max_idle limit so
that there is no risk of the "good" Changelog user being evicted.
Add some logging to the test so that it is easier to correlate test
script actions with events in the MDS debug log.
Lustre-change: https://review.whamcloud.com/36468
Lustre-commit:
4b0f0164c6ed761897409186376e9edc989323c9
Fixes:
31fef6845e8b ("LU-10680 mdd: create gc thread when no current transaction")
Test-Parameters: trivial envdefinitions=ONLY=160 testlist=sanity,sanity
Test-Parameters: envdefinitions=ONLY=160 mdscount=2 testlist=sanity,sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0e4c9c271d98a2716f848e75676780b0383ebbe5
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38833
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Thu, 28 May 2020 07:44:34 +0000 (00:44 -0700)]
LU-13421 kernel: kernel update RHEL8.1 [4.18.0-147.8.1.el8_1]
Update RHEL8.1 kernel to 4.18.0-147.8.1.el8_1 for Lustre client.
Test-Parameters: trivial clientdistro=el8.1
Change-Id: I4c8d925f295927ed7b7fd8fd5d17754d720bfc4d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38227
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Mon, 23 Sep 2019 12:39:48 +0000 (08:39 -0400)]
LU-12761: tests: make version_code() accept two number versions too
There's now a user in sanity test 103a that calls version_code with
2.6. Andreas rightfully points instead of fixing the caller we can
just update the code to accept this usage.
Lustre-change: https://review.whamcloud.com/36275
Lustre-commit:
6521dda6f4377c9c688ce4905cd94adf9f99013f
Change-Id: I5915cd08a36946c6d26f2e231aa7a820a3eef46a
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38715
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Amir Shehata [Wed, 20 May 2020 05:21:10 +0000 (22:21 -0700)]
LU-13553 lnd: gracefully handle unexpected events
When a tx completes kiblnd_tx_complete() callback is invoked.
We ensure:
LASSERT (tx->tx_sending > 0);
However this assert is being triggered in some rare scenarios.
The reason tx_sending would be 0 at this point is because:
1. ib_post_send() failed but OFED stack is still sending
a tx complete event.
2. We're getting two different events for the same tx
Instead of asserting, ignore that tx_complete event and print
the tx pointer and its status.
Lustre-change: https://review.whamcloud.com/38669
Lustre-commit:
60f9f539e686fc19b080a3cda15ade7111bbd4a7
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I8cd192538c0c80abaef23a4b6e6906936043060b
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38752
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 8 May 2020 23:28:39 +0000 (17:28 -0600)]
LU-13225 utils: fix install path for bash-completion
Fix the default install path for bash-completion if the package is
not installed at build time. This avoids BASH_COMPLETION_DIR being
badly formatted in the lustre.spec file.
Lustre-change: https://review.whamcloud.com/38548
Fixes:
dfb4afc24102 ("LU-13225 utils: bash completion for lfs and lctl")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie50071c4ff86f57bc9dd53409ae339da2a3ebbe5
Reviewed-on: https://review.whamcloud.com/38670
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sat, 8 Feb 2020 08:25:29 +0000 (01:25 -0700)]
LU-13225 utils: bash completion for lfs and lctl
Add a bash completion for "lfs" and improve completion for "lctl".
Rename the "lctl" completion script to "lustre" since the two
commands share helper routines for fsnames, pools, etc. and install
"lfs" and "lctl" symlinks to the common command file.
The completion prints available sub-commands and their options,
and for some sub-commands it completes available arguments
(e.g. mount points, pool names, and MDT/OST names).
A couple of minor changes to "lfs" and "lctl" usage messages to make
the sub-command options easier to parse. More needs to be done to
make all sub-commands have proper long options.
There is definitely more that could be added to the completions,
but this is a good start and provides a framework for the future.
Lustre-change: https://review.whamcloud.com/37483
Lustre-commit:
dfb4afc24102ee305d4901dc76944f4c91887633
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie989b2ef4c0d6d8565e5c6753205bb6ed83ebbe5
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/38519
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Mon, 25 May 2020 18:27:22 +0000 (11:27 -0700)]
LU-11986 libcfs: lnet_remove_debugfs() compat for RHEL6
Unloading libcfs module on RHEL 6.10 Lustre client with
kernel 2.6.32-754.24.3 hit kernel panic issue. The issue
doesn't exist in Lustre b2_10 where RHEL 6.10 is supported
and debugfs_remove_recursive() is called directly from
lnet_remove_debugfs(). This patch adds compat changes to
lnet_remove_debugfs() to resolve the issue.
Fixes:
9d42660e173e ("LU-11986 lnet: properly cleanup lnet debugfs files")
Fixes:
ae93a9f21752 ("LU-11986 libcfs: add compat for d_hash_and_lookup()")
Test-Parameters: trivial
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Ib63a40afe8926f56cd1d2873975855c226098418
Reviewed-on: https://review.whamcloud.com/38716
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Tue, 18 Feb 2020 15:04:44 +0000 (18:04 +0300)]
LU-10395 osd: stop OI at device shutdown
and not at obd_cleanup(). otherwise a race is possible:
umount <MDT> stopping OI vs MGS accessing same OSD which
results in the assertion:
ASSERTION( osd->od_oi_table != NULL && osd->od_oi_count >= 1 )
Lustre-change: https://review.whamcloud.com/37615
Lustre-commit:
2789978e1192dbf6d90399c96b5594e0dc049cd9
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I24fccea718f2e2663166cfb0ff26571039357535
Reviewed-on: https://review.whamcloud.com/38153
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 20 May 2020 23:56:37 +0000 (16:56 -0700)]
LU-11669 tests: add project in yml_test_group()
This patch fixes yml_test_group() in yaml.sh to
add test project name, which is required in
results.yml for Maloo to parse.
This patch is back-ported from the following one:
Lustre-commit:
054bb02880d26bacc4e7080869955c2039bbf986
Lustre-change: https://review.whamcloud.com/33658
Test-Parameters: trivial
Change-Id: I0ae563d855dc2d28eaea85e86b1cb23d2428988b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Leonel Ochoa <lochoa@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38684
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Zhuravlev [Tue, 25 Feb 2020 16:44:18 +0000 (19:44 +0300)]
LU-11269 ptlrpc: request's counter in import
which is separate from imp_refcount as the latter can be
used for other purposes and it's hard to use to track
requests.
to verify the theory that imp_refcount should be checked.
Lustre-change: https://review.whamcloud.com/37722
Lustre-commit:
b09afdf57643cbc1c437a42b4babb0837dd19e65
Change-Id: I7c273a73e2b1bb43059c7ed003ee2b7d09273bfe
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38340
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Tue, 6 Nov 2018 00:26:44 +0000 (19:26 -0500)]
LU-11623 llite: hash just created files if lock allows
If open|creat (and other intent operations later) returned a lookup
bit as part of the lock, hash the resultant dentry under this lock,
not to trigger further RPCs in subsequent lookups.
Lustre-change: https://review.whamcloud.com/33584
Lustre-commit:
fc42cbe0e2e5d1d87d0edca73986b831ac718301
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: Id5140d1042af7f5ab9052922e11a7eda8f92a29a
Reviewed-on: https://review.whamcloud.com/38763
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Lai Siyao [Thu, 12 Mar 2020 00:35:20 +0000 (08:35 +0800)]
LU-13357 lod: implement striped directory .dio_lookup
Add function lod_striped_lookup() for
lod_striped_index_ops.dio_lookup to allow name lookup under striped
directory.
Currently this is used by subdir mount, which needs to lookup FID
of the subdir on server side.
Function lfsck_namespace_repair_dirent() should call dt_lookup() with
bottom object, because child may be shard.
Add sanity 247f.
Lustre-change: https://review.whamcloud.com/37903
Lustre-commit:
42b0304e2571a80effe5bc4ab6fb58acfabb361d
Change-Id: Iba844d1a34a318bcbd42b00186ed6fa9d165effc
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38691
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Mon, 8 Jun 2020 13:18:56 +0000 (09:18 -0400)]
New release 2.12.5
Change-Id: I2bd2c42ba57730856fe454999f67b870b41330e8
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Wed, 27 May 2020 22:50:20 +0000 (18:50 -0400)]
New Rc 2.12.5-RC1
Change-Id: I2db2a133d8d8fc8479cc36e3714f4f62b2ea2dd5
Alexey Lyashkov [Mon, 20 Apr 2020 09:45:52 +0000 (12:45 +0300)]
LU-13416 ldiskfs: don't corrupt data on journal replay
Journalled write want a special attention on blocks release,
revoke records must added to avoid replace new write blocks
with stale data. Mark inode as "journal write" to generate
valid revoke records. Large EA inode updates affected
with this bug also.
large ea fix is
Linux-commit:
ddfa17e4adc4bd19c32216aaa6250dc38b0579df
Author: Tahsin Erdogan <tahsin@google.com>
Date: Wed Jun 21 21:36:51 2017 -0400
ext4: call journal revoke when freeing ea_inode blocks
Lustre-change: https://review.whamcloud.com/38281
Lustre-commit:
a23aac2219047cb04ed1fa555f31fa39e5c499dc
Change-Id: I605128c4ba70331a48715dc95546430909efb893
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38705
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 20 May 2020 18:19:32 +0000 (12:19 -0600)]
LU-13589 utils: fix lfs setstripe unit parsing
The "size_units" variable was not being reset while parsing different
"lfs setstripe" arguments, so e.g. "lfs setstripe -E 1M -S 65536 ..."
ended up using the 'M' unit for the stripe size, which resulted in a
stripe_size of 65536MiB = 64GiB, which resulted in an error.
This only appeared with PFL or other layout patterns which had more
than one unit being parsed, and was already fixed in master via SEL.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib3f9be86f5104aaea4f77d87853255a518cbc3a0
Reviewed-on: https://review.whamcloud.com/38680
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Tue, 12 May 2020 20:32:22 +0000 (23:32 +0300)]
LU-13535 lfsck: fix possible PFL layout corruption
While checking lmm_oi in composite layout the pointer to 'lmm'
is re-assigned to component entry but the same pointer is used
for LOV EA buffer to update EA. Therefore if lmm_oi was fixed in
some component then just current entry is saved as new layout.
Lustre-change: https://review.whamcloud.com/38584
Lustre-commit:
be009cb4a73b3bef7302083bec7d1d6289d515b7
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ifbd984a71b383ab4ca35ad59ed9cd8cf57b6d7cc
Reviewed-on: https://review.whamcloud.com/38585
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Jian Yu [Sun, 17 May 2020 07:39:39 +0000 (00:39 -0700)]
LU-13111 kernel: new kernel [SLES12 SP5 4.12.14-122.20.1]
This patch makes changes to support new SLES12 SP5 release
for Lustre client.
Test-Parameters: trivial clientdistro=sles12sp5 \
env=SANITY_EXCEPT="56oc 817" testlist=sanity
Change-Id: Ia4b856b03801e02da9a2e584efeb8759b4dd30c3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38640
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Sat, 16 May 2020 23:56:34 +0000 (16:56 -0700)]
LU-13556 kernel: kernel update RHEL7.8 [3.10.0-1127.8.2.el7]
Update RHEL7.8 kernel to 3.10.0-1127.8.2.el7.
Test-Parameters: trivial clientdistro=el7.8 serverdistro=el7.8
Change-Id: If7ac6f4b5f1fe32a15c63f51589a2e320001b4a5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38635
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Fri, 22 May 2020 18:06:34 +0000 (11:06 -0700)]
LU-13488 kernel: new kernel [RHEL 8.2 4.18.0-193.1.2.el8]
This patch makes changes to support new RHEL 8.2 release
for Lustre client.
Test-Parameters: trivial clientdistro=el8.2 \
env=SANITY_EXCEPT="130" testlist=sanity
Change-Id: Icb1db3afd2e94423a45354acfdd559f8f1e294cb
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38461
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 4 May 2020 22:03:38 +0000 (15:03 -0700)]
LU-12904 o2ib: ib_destroy_cq() returns void
Kernel destroy CQ flows can't fail and the returned value of
ib_destroy_cq() is not interested in those flows.
kernel-commit:
890ac8d97e6722a9e4a66a0bd836d1b028d075fe
This patch is back-ported from the following one:
Lustre-commit:
7d2ea1e5bbd80f23e6935174c969b34b58048443
Lustre-change: https://review.whamcloud.com/36578
Test-Parameters: trivial
Cray-bug-id: LUS-8042
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I873bf76a33bd80d5e6de4d1b16a79ff5ea930f3a
Reviewed-on: https://review.whamcloud.com/38489
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 4 May 2020 21:59:47 +0000 (14:59 -0700)]
LU-12634 lnet: for_ifa removed. Use in_dev_for_each_ifa_rtnl
Linux 5.3 removed for_ifa and replaced it with an _rntl and _rcu
versions for use with their respective locking primitives.
kernel-commit:
ef11db3310e272d3d8dbe8739e0770820dd20e52
This patch is back-ported from the following one:
Lustre-commit:
6e0d0146276353559c821916e193c90d167b14e0
Lustre-change: https://review.whamcloud.com/35744
Test-Parameters: trivial
Cray-bug-id: LUS-7689
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Iea07222b9abb3f9c219d28fe2c660d9eaf21af80
Reviewed-on: https://review.whamcloud.com/38487
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 4 May 2020 20:16:00 +0000 (13:16 -0700)]
LU-12400 ptlrpc: Sun RPC changes for RCU locking
In kernel 4.20 SUNRPC cache_detail->hash_lock changed to spinlock_t
Now that the reader functions are all RCU protected, use a regular
spinlock rather than a reader/writer lock.
Linux-commit:
1863d77f15da0addcd293a1719fa5d3ef8cde3ca
This patch is back-ported from the following one:
Lustre-commit:
77d53777e32c80047cb75293d5f9a4c0d23bbea8
Lustre-change: https://review.whamcloud.com/35499
Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If0df38337d5a2bb0ac4b8cb645dbe89f65e0f352
Reviewed-on: https://review.whamcloud.com/38483
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Mon, 4 May 2020 20:01:34 +0000 (13:01 -0700)]
LU-12355 llite: include file linux/selinux.h removed
In kernel 5.1 linux/selinux.h was removed with
SELinux: Remove unused selinux_is_enabled
Linux-commit:
3d252529480c68bfd6a6774652df7c8968b28e41
This patch is back-ported from the following one:
Lustre-commit:
39e5bfa73414d18738001761b42ea0e3264c2983
Lustre-change: https://review.whamcloud.com/35035
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: If963e6b22b7b07899de5b970f934bb157c5f7cec
Reviewed-on: https://review.whamcloud.com/38480
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 22 May 2020 17:59:40 +0000 (10:59 -0700)]
LU-12355 llite: Lustre specific iov_for_each broken (removed)
Kernel 4.20 introduced iov_iter_type and broke iov_for_each
As iov_for_each is only used once so drop the macro entirely.
When iov_iter_type is available ignore invalid iter types.
Linux-commit:
8a363970d1dc38c4ec4ad575c862f776f468d057
Kernel 3.15 added type to iov_iter. Use the type to provide
a sensible replacement for iov_iter_type when it is available.
Linux-commit:
71d8e532b1549a478e6a6a8a44f309d050294d00
This patch is back-ported from the following one:
Lustre-commit:
d93aa0171a25f8ffca51bed35a2d477a45fda0f3
Lustre-change: https://review.whamcloud.com/35024
Cray-bug-id: LUS-6962
Change-Id: I97cdce1c85803ac2d4436d4eedf67a545ea2cdb8
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-on: https://review.whamcloud.com/38479
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 22 May 2020 17:49:45 +0000 (10:49 -0700)]
LU-12400 llite: Use the new vm_fault_t type
Linux 4.17 created the new vm_fault_t type
Linux-commit:
1c8f422059ae5da07db7406ab916203f9417e396
Linux 5.1 changed the vm_fault_t type to bitwise unsigned int
which changes the interfaces registered to struct vm_operations_struct
Linux-commit:
3d3539018d2cbd12e5af4a132636ee7fd8d43ef0
Prefer to match the upstream API and fallback to 'int'
where vm_fault_t is not available.
This patch is back-ported from the following one:
Lustre-commit:
f2b224a48cb00f885b9df2cc56e349dae5f27f9e
Lustre-change: https://review.whamcloud.com/35500
Test-Parameters: trivial
Cray-bug-id: LUS-7600
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I7122fb0d4af3ee9a19c1a5d0b77c4f13f6850181
Reviewed-on: https://review.whamcloud.com/38478
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 30 Apr 2020 22:20:01 +0000 (16:20 -0600)]
LU-13168 tests: verify truncated xattr is handled
Verify that a truncated trusted.lov xattr is handled properly,
for both plain and PFL layouts.
Add a test case that verifies this is fixed for both layout types.
Lustre-change: https://review.whamcloud.com/38434
Lustre-commit:
cb74546354201434a6fd3d53acd1a0808fbfcb1c
Fixes:
f2d06d3c76 ("LU-12911 llite: Don't access lov_md fields before size check")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I11d420c7fdc2362f64689a545b95c76e893ebbe5
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38604
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Oleg Drokin [Wed, 1 Apr 2020 05:38:16 +0000 (01:38 -0400)]
LU-13131 osc: Ensure immediate departure of sync write pages
Except for the case of direct-io and server-lock, we are
hold potentially multiple locks that are next to impossible
to find and cross reference.
So instead just send it all right away - should only
be a factor in rare cases of out of quota or close to out
of space.
Lustre-change: https://review.whamcloud.com/38453
Lustre-commit:
13b7cf4fabdd55977b68eb856bfdc82f0a349e73
Change-Id: I961cd9ba7f3266d22dfc5eff758c2f4ebbe148a4
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Oleg Drokin [Thu, 19 Mar 2020 19:24:28 +0000 (15:24 -0400)]
LU-13131 osc: Do not wait for grants for too long
obd_timeout is way too long considering we are holding a lock
that might be contended. If OST is slow to respond, we might
get evicted, so limit us to a half of the shortest possible
max wait a server might have before switching to synchronous IO.
Lustre-change: https://review.whamcloud.com/38283
Lustre-commit:
1eee11c75ca13745d083410e1ced3a1a8b088ee9
Change-Id: I36653194c1b8b95ba3cc2ed9240df7b0888cf7ed
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38672
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Oleg Drokin [Tue, 12 May 2020 00:05:32 +0000 (20:05 -0400)]
LU-9679 osc: convert cl_cache_waiters to a wait_queue.
cli->cl_cache_waiters is a list of tasks that need
to be woken when grant-space becomes available. This
means it is acting much like a wait queue.
So let's change it to really be a wait queue.
The current implementation adds new waiters to the end of the list,
and calls osc_enter_cache_try() on each in order. We can provide the
same behaviour by using an exclusive wait, and having each waiter wake
the next task when it succeeds.
If a waiter notices that success has become impossible, it wakes all
other waiters.
If a waiter times out, it doesn't wake other - just leaves them to
time out themselves.
Note that the old code handled -EINTR from the wait function. That is
not a possible return value when wait_event_idle* is used, so that
case is discarded.
As we need wait_event_idle_exclusive_timeout_cmd(), we should fix the
bug in that macro - the "might_sleep()" is wrong, as a spinlock might
be held at that point.
Linux-Commit:
31f45f56ecdf ("lustre: osc_cache: convert
cl_cache_waiters to a wait_queue.")
Lustre-change: https://review.whamcloud.com/37605
Lustre-commit:
b2ede01d1ed77ddc512c013220f6ea8b509e9541
Change-Id: Ib7622ea2daea8f6e59bef95d3b6c5a80d209b81e
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/38575
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Fri, 7 Feb 2020 01:21:43 +0000 (20:21 -0500)]
LU-13131 osc: Always send all HP RPCs requests
that are resulting from lock cancel activity.
Lustre-change: https://review.whamcloud.com/38057
Lustre-commit:
0f1743916be6605fcd8f57993d6ce7d8d06ce12c
Change-Id: I4007167f0f39b0699e977c14bd160f475d8288ad
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37468
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
James Simmons [Fri, 19 Jul 2019 12:38:44 +0000 (08:38 -0400)]
LU-9859 libcfs: initialize bit_wait_table
With older platforms wait_event_var() is missing so we needed to
provide are own version. This included creating a local
bit_wait_table like the kernel has. That table was never properly
initialized so it can cause failures under the right conditions.
Test-Parameters: trivial
Lustre-commit: https://review.whamcloud.com/35567
Lustre-change:
916643c4b8d136c9c96afa5cd21d1eb6030d148c
Fixes:
372ef85512 ("LU-11089 obd: use wait_event_var() in lu_context_key_degister()")
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: I310d37da7c1b54166224b367446cc905c02ab8bc
Reviewed-on: https://review.whamcloud.com/38574
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
NeilBrown [Tue, 21 May 2019 14:22:39 +0000 (10:22 -0400)]
LU-11089 obd: use wait_event_var() in lu_context_key_degister()
lu_context_key_degister() has an open coded loop which calls
schedule() without setting a new task state. This is generally
a bad idea - it could easily just spin.
Instead, use wait_event_var() to wait for ->lct_used to be zero,
and arrange to get a wakeup when that happens.
Previously ->lct_used would only fall down to 1. Now we decrement
it an extra time so that wake_up, which only happens when the
count reaches zero, will only happen when lu_context_key_degister()
is actually waiting for it.
Note that this patch removes key_fini() from protection by
lu_keys_guard. key_fini() calls are not always protected
by a lock, and there seems to be no need here. Nothing else
can be acting on the given key in that context at this point,
so no race is possible.
Linux-commit:
ef84c07364211bb4e398a9de45d1c13a32059cee
Lustre-commit:
372ef85512dd2a722415fba9a3df66f81029508b
Lustre-change: https://review.whamcloud.com/33667
Change-Id: I9514bd21916f75fce00e393612967fb197e3a1c4
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/38573
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr NeilBrown [Mon, 28 Oct 2019 01:24:26 +0000 (12:24 +1100)]
LU-12911 llite: Don't access lov_md fields before size check
When 'struct lov_user_md' is passed in via setxattr, it comes with
a size. If thatt size is too small, some function that check exactly
what version is present might access beyond the end of allocation
memory, which can have undesirable effects, such as triggering
a KASAN warning (and possibly worse).
So check that the size is sane before looking inside the structure
at all.
Lustre-change: https://review.whamcloud.com/36589
Lustre-commit
f2d06d3c76a1d69447e7bd6fd29d8165be558d73
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib3f071a3ff77a039fdfa38c903d87999108b3322
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/38433
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Boyko [Mon, 16 Dec 2019 13:24:16 +0000 (08:24 -0500)]
LU-13102 llog: fix processing of a wrapped catalog
The logic for rereading a llog buffer had an exception
for a full catalog, when lgh_last_idx = llh_cat_idx and a first
processing index is a llh_cat_idx+1. This check is based on
a value lh_last_idx, which stays unchanged between buffer read.
But llh_cat_idx could go forward, and this lead to a wrong check
where reread doesn't happen. As a result Lustre got ENOENT for
a record and stoped osp processing.
llog_cat_set_first_idx())
catlog [0x6:0x1:0x0] first idx 34730, last_idx 34731
osp_sync_process_queues()) 1 changes, 0 in progress, 0 in flight
llog_process_thread())
stop processing plain 0x76941:1:0 index 64767 count 1
llog_process_thread())
index: 34731, lh_last_idx: 34730 synced_idx: 34730 lgh_last_idx: 34731
llog_cat_process_common()) processing log [0x2780f:0x1:0x0]:0
at index 34731 of catalog [0x6:0x1:0x0]
llog_cat_id2handle()) snx11281-OST0001-osc-MDT0001:
error opening log id [0x2780f:0x1:0x0]:0:rc = -2
The patch fixes logic and also adds/changes debugging for
llog and osp.
Lustre-change: https://review.whamcloud.com/37102
Lustre-commit:
a4f049b96562fd502b1948fb082767351e040a1c
Cray-bug-id: LUS-8193
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I9463223a1ea904b96643b19e1580f5894142c12b
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38541
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 7 May 2020 00:08:14 +0000 (18:08 -0600)]
LU-13274 uapi: fix build on older kernels
The recent changes to lustre_user.h broke building on older kernels,
because it resulted in the user tools including both <linux/quota.h>
and <sys/quota.h>, which results in conflicts for the declaration of
the quotactl() function.
Also, restore the compat declaration of __ALIGN_KERNEL(), though it
only in lustre_user.h, since it is included in other headers anyway.
Test-Parameters: trivial
Fixes:
0417dce9fc75 ("LU-13274 uapi: make lustre UAPI headers C99 compliant")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9b31cabaf2732eb5872e88686e6af7d12e7d3564
Reviewed-on: https://review.whamcloud.com/38520
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Thu, 28 Feb 2019 04:40:37 +0000 (23:40 -0500)]
LU-12030 tests: Properly detect debug kernel use on rhel7.6
kmalloc-128 slab seems to be gone, so let's use dma-kmalloc-128
instead.
Lustre-commit:
673e8b0b3c404b14a02e7e874c3aed991bc52eb3
Lustre-change: https://review.whamcloud.com/34342
Test-Parameters: trivial testlist=conf-sanity env=ONLY=63
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: Ice7f350ba2bc6cc733c0a98b0037e6f0980216c9
Reviewed-on: https://review.whamcloud.com/38572
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Yang Sheng [Thu, 16 Apr 2020 15:22:54 +0000 (23:22 +0800)]
LU-12846 mdd: return error while delete failed
Since we use a global buffer, So avoid to replace the
index name while iterate the orphan directory. Also
return error code in mdd_orphan_destroy while dt_delete
failed. Else will cause a dead loop.
Fixes:
e1ace3751f ("LU-8514 mdd: transaction failure should be checked")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I6fc3e992333ffa61900074309223555264cfe66b
Reviewed-on: https://review.whamcloud.com/37570
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Andreas Dilger [Thu, 7 May 2020 11:19:58 +0000 (05:19 -0600)]
LU-11986 libcfs: add compat for d_hash_and_lookup()
Add compatibility for older kernels that do not export the
d_hash_and_lookup() function, even though it exists forever.
Fixes:
9d42660e173e ("LU-11986 lnet: properly cleanup lnet debugfs files")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I10d6547f6b17665880cacfbf87d4dbfd386b92ff
Reviewed-on: https://review.whamcloud.com/38529
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Mon, 4 May 2020 18:58:40 +0000 (11:58 -0700)]
LU-12236 gss: remove unused code in gss_svc_upcall.c
Delete rsc_flush() related functions which are never
used.
This patch is back-ported from the following one:
Lustre-commit:
25b0bf5a23032394055b7b94b3169f5cf4068570
Lustre-change: https://review.whamcloud.com/34794
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small,sanity-sec
Change-Id: Iedd6339b5fafdea81147c83e5f0499fa3ad60251
Signed-off-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-on: https://review.whamcloud.com/38476
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexander Zarochentsev [Fri, 20 Dec 2019 23:19:44 +0000 (02:19 +0300)]
LU-13228 clio: mmap write when overquota
Flagging client by overquota flag should not
cause mmap write access to sigbus the app.
Lustre-change: https://review.whamcloud.com/37495
Lustre-commit:
b651089f859e8269af7272b91f5e60aa25f24226
Cray-bug-id: LUS-8221
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Change-Id: I29d5901fa5078b5cfca40391a02531cf27efce93
Reviewed-on: https://review.whamcloud.com/38292
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Sat, 25 Apr 2020 17:19:32 +0000 (10:19 -0700)]
LU-12402 lnet: handle recursion in resend
When we're resending a message we have to decommit it first. This
could potentially result in another message being picked up from the
queue and sent, which could fail immediately and be finalized, causing
recursion. This problem was observed when a router was being shutdown.
This patch uses the same mechanism used in lnet_finalize() to limit
recursion. If a thread is already finalizing a message and it gets
into path where it starts finalizing a second, then that message
is queued and handled later.
Lustre-change: https://review.whamcloud.com/35431
Lustre-commit:
ad9243693c9a5a5b2c34165ad853ddf5ceec4617
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Iace64c7ddb1f56a0a63b030df6a5ab103ae6c645
Reviewed-on: https://review.whamcloud.com/38367
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 13 Jun 2019 22:51:38 +0000 (17:51 -0500)]
LU-12436 lov: return error if cl_env_get fails
When cl_env_get() fails with an error return the error.
Lustre-change: https://review.whamcloud.com/35229
Lustre-commit:
a7997c836bbfe2a0674007f1c23b9593e596e0ba
Test-Parameters: trivial
Cray-bug-id: LUS-7310
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: Ia065aeb142a772f4d620b84111af423e27c06b90
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38360
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Tue, 21 Apr 2020 01:08:02 +0000 (18:08 -0700)]
LU-13369 kernel: kernel update RHEL7.7 [3.10.0-1062.18.1.el7]
Update RHEL7.7 kernel to 3.10.0-1062.18.1.el7.
Test-Parameters: trivial clientdistro=el7.7 serverdistro=el7.7
Change-Id: Ifc00fca35a0ad28ba8326e56e693ea39360a8114
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38241
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Wed, 15 Apr 2020 05:23:14 +0000 (22:23 -0700)]
LU-13347 kernel: RHEL 7.8 server support
This patch makes changes to support new RHEL 7.8 release
for Lustre server (kernel 3.10.0-1127.el7).
Test-Parameters: trivial clientdistro=el7.8 serverdistro=el7.8
Change-Id: I4817fd2f9512111aa7d26109454104945fd2778f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38144
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Ann Koehler [Mon, 14 Oct 2019 16:30:56 +0000 (11:30 -0500)]
LU-12816 ptlrpc: ptlrpc_register_bulk LBUG on ENOMEM
Another path through ptl_send_rpc() can cause the assert reported
in LU-10643. The assertion in ptlrpc_register_bulk() on
!desc->bd_registered fails when an rpc is resent and the first
send attempt failed to successfully attach the reply buffer. The
bulk error cleanup in ptl_send_rpc() does not reset the
bd_registered flag.
Lustre-change: https://review.whamcloud.com/36309
Lustre-commit:
e6225c07ce4c0037a127a41b2bc539364dfd1f4d
Cray-bug-id: LUS-7946
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I474211f196ea9bd83a036747e25c91c37c85ffbb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <stancheff@cray.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38266
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Nunez [Sat, 18 Apr 2020 09:11:14 +0000 (02:11 -0700)]
LU-13063 tests: stop running sanity test 411
sanity test 411 hits a kernel bug for RHEL 8.1. Since this
is an issue with the kernel and not Lustre, let's stop
running this test until the kernel is patched. Thus, we
need to add sanity test 411 to the ALWAYS_EXCEPT list.
Also change the ALWAYS_EXCEPT condition for test smoke for
lnet-selftest to be based on kernel version and not
architecture, so that the custom test for this patch can
pass.
This patch is back-ported from the following one:
Lustre-commit:
34e4c37474b3d9328aac1dd9228a018ac7d4f47e
Lustre-change: https://review.whamcloud.com/37270
Test-Parameters: trivial clientdistro=el8.1
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I60174dcd4776b53ac5b44be6c208d40e1f022445
Reviewed-on: https://review.whamcloud.com/38260
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Sat, 18 Apr 2020 08:42:06 +0000 (01:42 -0700)]
LU-12661 tests: skip sanity 817 if kernel version >= 4.14
sanity test_817 is in the ALWAYS_EXCEPT list for aarch64,
however it's failing because the test was done on kernel-alt
which is 4.14.x, it's not related with the architecture.
On new kernels nfsd is not releasing the file after write,
it will fail with ETXTBSY regardless of whether the nfs export
is backed by a lustre mount or not.
Skip the test on new kernels for now.
This patch is back-ported from the following one:
Lustre-commit:
4fed33473ca2964ff19f61fdb8501b2210f923de
Lustre-change: https://review.whamcloud.com/36712
Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ie18ceb961eee2313fca7d60a35159a7496075029
Reviewed-on: https://review.whamcloud.com/38259
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Fri, 6 Dec 2019 02:08:07 +0000 (10:08 +0800)]
LU-12852 pfl: restrict the stripe count correctly
In function lod_get_stripe_count(), when restricting the stripe
count to the maximum xattr size, the xattr overhead should be
taken into count correctly.
Lustre-change: https://review.whamcloud.com/36947
Lustre-commit:
6dc37759cfb22727ac5d776c38b72e8638563fd8
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ief548e47ce4d375f2e189860ccfe05d0f3c7e890
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37512
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Gian-Carlo DeFazio [Tue, 11 Feb 2020 21:08:18 +0000 (13:08 -0800)]
LU-13224 utils: expose llapi_param* functions
Added an interface to find files in sysfs,procfs,etc.,
and read them into a file or buffer.
The interface is meant to return the same results as
the "lctl list_param" and "lctl get_param" commands,
but doesn't require starting an lctl process.
The interface is being used by lctl for parts of
its "list_param" and "get_param" commands.
The output of lctl get_param is altered slightly.
Previously, for some multi-line files lctl would
print the param name and first line of the file on
the same line such as:
param=line 1
line 2
line 3
Now multi-line files consistently put a newline
after the param name and before the content such as:
param=
line 1
line 2
line 3
Added debug output to failing test sanity 300d.
Lustre-commit:
9b44cf70a95b2ee97a17697dc97fbe462ad1f5b9
Lustre-change: https://review.whamcloud.com/37545
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Change-Id: I2726b27b0042d58c97284f8348970deb6efc43d1
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38239
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Tue, 10 Mar 2020 13:12:53 +0000 (21:12 +0800)]
LU-13324 llite: page delete race
Page could be raced by truncate during its deletion by
another thread, check vmpage mapping before accessing it.
Lustre-change: https://review.whamcloud.com/37793
Lustre-commit:
8ccaed513006f0582578ff42c9917b5aa76f2f4b
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I3172f5ffd3928926b16ab6fd7362b05da0c7cfd5
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38111
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 10 Apr 2020 05:50:25 +0000 (22:50 -0700)]
LU-12904 gss: struct cache_detail readers changed to writers
Linux 5.3 changed struct cache_detail readers to writers
SUNRPC: Track writers of the 'channel' file to improve ...
kernel-commit:
64a38e840ce5940253208eaba40265c73decc4ee
This patch is back-ported from the following one:
Lustre-commit:
81fb8bc7d214394bbc504379a5a84441e871b60a
Lustre-change: https://review.whamcloud.com/36580
Test-Parameters: trivial
Cray-bug-id: LUS-8042
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I7750303937cd6fc560e458efa79f25e521fefec7
Reviewed-on: https://review.whamcloud.com/38193
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 10 Apr 2020 16:20:17 +0000 (18:20 +0200)]
LU-13355 crypto: crypto engine wrappers in libcfs
libcfs has wrappers in order to be able to use crypto engines as
standard crypto modules. But libcfs should not be considered as the
owner of these implementations.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1e22907c10837042606095ed6089c3c675fffe27
Reviewed-on: https://review.whamcloud.com/38205
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 10 Apr 2020 05:54:56 +0000 (22:54 -0700)]
LU-13347 kernel: new kernel [RHEL 7.8 3.10.0-1127.el7]
This patch makes changes to support new RHEL 7.8 release
for Lustre client.
Test-Parameters: trivial clientdistro=el7.8
Change-Id: I82f89495d5ab1d46a539a016a899307d7c8f37b7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38143
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andriy Skulysh [Tue, 25 Feb 2020 16:04:32 +0000 (11:04 -0500)]
LU-13101 llite: eviction during ll_open_cleanup()
On error ll_open_cleanup()Â is called while
intent lock remains pinned. So eviction can
happen while close request waits for a mod rpc slot.
Release intent lock before ll_open_cleanup()
Lustre-change: https://review.whamcloud.com/37096
Lustre-commit:
6d5d7c6bdb4f19f9db485a774d9259d452cf220e
Change-Id: Ia422351f3f54fc652078f742f2ead0bf278c9d17
Cray-bug-id: LUS-8055
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-on: https://review.whamcloud.com/37096
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38147
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Mikhail Pershin [Wed, 15 Jan 2020 11:55:18 +0000 (14:55 +0300)]
LU-13136 dom: check read-on-open buffer presents in reply
The ll_dom_finish_open() uses req_capsule_has_field() wronly,
it check only format but not buffer presence in reply, that
causes unneeded console errors about missing buffer later in
req_capsule_server_get()
Patch replaces that with req_capsule_field_present() to check
if server pack that field in reply or not and properly skip
responses from an old server.
Lustre-change: https://review.whamcloud.com/37249
Lustre-commit:
58bea527100b50abf3df2dbab0ed6d6b42b69d86
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ia6114879c90e3e6b8c5020c4912e988cad90df30
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38188
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Oleg Drokin [Sat, 14 Mar 2020 07:11:29 +0000 (03:11 -0400)]
LU-13131 osc: Make sure we don't accidentally deprioritize hp requests
Looks like in some cases HP requests could migrade off HP list back
onto urgent list
Lustre-change: https://review.whamcloud.com/37967
Lustre-commit:
f1c55f97d2d4b21fe87031b2bddfb9813f7df488
Change-Id: I96bf0a3a005b166f34dba215463c0806dfe2526a
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37918
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Oleg Drokin [Tue, 17 Sep 2019 05:23:26 +0000 (01:23 -0400)]
LU-12773 tests: sanity test_805 Use do_facet
do_node cannot really work with $SINGLEMDS, that's the
facet name.
This fixes error message below (and a following syntax error):
mds1: ssh: Could not resolve hostname mds1: Name or service not known
Lustre-change: https://review.whamcloud.com/36204
Lustre-commit:
fa8b581aea27acc00f6ac48e76ec261dddf8631a
Fixes:
106abc184d8b ("LU-8856 osd: mark specific transactions netfree")
Test-Parameters: trivial fstype=zfs
Change-Id: I0d842dbccbfd934c524ae01cca7399dd52158064
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37901
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Sat, 21 Mar 2020 01:58:09 +0000 (09:58 +0800)]
LU-13377 llite: fix dead loop for short write
|->vvp_io_write_start()
|->__generic_file_write_iter()
|->iov_iter_advance() if write succeed()
|->vvp_io_write_commit()
|->update ci_nob
The problem is we will move forward iov iter inside
__generic_file_write_iter(), but @ci_nob will be updated
after vvp_io_write_commit().
If out of quota or some other problems happen, this could
cause a mismatch with @ci_nob and @vui_iter.
And @vui_iter->count will be reset using @ci_nob in
iov_iter_reexpand(), this will make @vui_iter->count
more than what it really left, and we could dead loop
in vvp_mmap_locks() if IO need be retried or restarted:
vvp_io_write_lock+0x45/0x80 [lustre]
cl_io_lock+0x5f/0x3d0 [obdclass]
cl_io_loop+0x92/0x190 [obdclass]
ll_file_io_generic+0x7b3/0xc90 [lustre]
ll_file_aio_write+0x12d/0x1f0 [lustre]
ll_file_write+0xce/0x1e0 [lustre]
vfs_write+0xc0/0x1f0
SyS_write+0x7f/0xf0
system_call_fastpath+0x22/0x27
Lustre-change: https://review.whamcloud.com/38018
Lustre-commit:
13dfe0df4956afb50b323a11615b0b34ed014e53
Change-Id: I5fb4c18cf02fb17bf50122b63decacef678caa01
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-on: https://review.whamcloud.com/38163
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 28 Nov 2019 12:04:25 +0000 (15:04 +0300)]
LU-12988 ldiskfs: skip non-loaded groups at cr=0/1
cr=0 is supposed to be an optimization to save CPU cycles,
but if buddy data (in memory) is not initialized then all
this makes no sense as we have to do sync IO taking a lot
of cycles. also, at cr=0 mballoc doesn't store any avaibale
chunk. cr=1 also skips groups using heruistic based on avg.
fragment size.
it's more useful to skip such groups and switch to cr=2 where
groups will be scanned for available chunks.
using sparse image and dm-slow virtual device of 120TB was
simulated. then the image was formatted as OST and filled
using debugfs to mark ~85% of available space as busy.
mount as OST w/o the patch couldn't complete in half an hour
(according to vmstat it would take ~10-11 hours). with the
patch applied mount took ~20 seconds.
Lustre-change: https://review.whamcloud.com/36891
Lustre-commit:
6a7a700a1490dfde6b60c2fb36df92a052059866
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I88c8c1b01b386af0fa438bfeb97acb6110bd00ec
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Mon, 20 Jan 2020 04:23:45 +0000 (12:23 +0800)]
LU-13157 mdd: migrate readlink from wrong place
In osd_read(), if symlink name buf length is smaller than i_size,
return -EOVERFLOW, and compare inline data length with i_size
instead of buflen.
Updated sanity.sh 230b.
Lustre-change: https://review.whamcloud.com/37285
Lustre-commit:
a3b30423c6067b3e8644ecfb3269f8837af7e4cd
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia13b1f1079efc4ebd22ec400f1a909ff7ec2095d
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38112
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Artem Blagodarenko [Tue, 28 May 2019 16:51:21 +0000 (19:51 +0300)]
LU-12345 ldiskfs: optimize nodelalloc mode
We found performance regression when using bigalloc with "nodelalloc"
(1MB cluster size):
1. mke2fs -C 1048576 -O ^has_journal,bigalloc /dev/sda
2. mount -o nodelalloc /dev/sda /test/
3. time dd if=/dev/zero of=/test/io bs=1048576 count=1024
The "dd" will cost about 2 seconds to finish, but if we mke2fs without
"bigalloc", "dd" will only cost less than 1 second.
The reason is: when using ext4 with "nodelalloc", it will call
ext4_find_delalloc_cluster() nearly everytime it call
ext4_ext_map_blocks(), and ext4_find_delalloc_range() will also scan
all pages in cluster because no buffer is "delayed". A cluster has
256 pages (1MB cluster), so it will scan 256 * 256k pags when creating
a 1G file. That severely hurts the performance.
Therefore, we return immediately from ext4_find_delalloc_range() in
nodelalloc mode, since by definition there can't be any delalloc
pages.
The same optimization also added for ldiskfs_find_delayed_extent()
function that improve performance dromaticaly.
Here is results of testing on two node system.
Without the patch:
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00   56.30    0.06    0.00   43.63
Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sds               0.00     0.00    0.00 1174.00     0.00     4.59
8.00     0.84    0.71    0.00    0.71   0.01   1.20
With patch:
08/29/2018 01:13:22 AM
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 4.13 30.37 0.00 65.50
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await r_await w_await svctm %util
sds 0.00 0.00 0.00 54117.82 0.00 211.43
8.00 152.59 2.82 0.00 2.82 0.02 99.01
Lustre-change: https://review.whamcloud.com/34982
Lustre-commit:
af48ae8bff289b2bc083a888efeafa3c48df91e2
Cray-bug-id: LUS-5835
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: Ie33410d4481778ee4f76a054ab8cfc11cc19a0ed
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37538
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Olaf Faaland [Tue, 26 Nov 2019 23:20:11 +0000 (15:20 -0800)]
LU-11114 llite: Update mdc and lite stats on open|creat
Increment "create" counter in mdc/<instance>/md_stats, and
"mknod" counter in llite/<instance>stats when an open with
the CREAT flag results in a newly created file.
The mknod counter is chosen for consistency with
patch http://review.whamcloud.com/20246
"LU-8150 mdt: Track open+create as mknod"
but the mdc counter set does not include mknod.
Lustre-change: https://review.whamcloud.com/36948
Lustre-commit:
4b8518ee4fa542f45fcdaeaec580d858dfcaf137
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: If082b911e415c0bc46248728e47ce0f37b9efa83
Reviewed-on: https://review.whamcloud.com/38158
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Li Dongyang [Fri, 7 Feb 2020 12:16:26 +0000 (23:16 +1100)]
LU-12580 lov: fix out of bound usercopy
When handling ioctl LL_IOC_LOV_GETSTRIPE, the user
could pass a limited buffer which is bigger than
lov_comp_md_size(), it will crash the client because
we are doing the usercopy with the user provided buffer
size.
Make sure the copy works, also for the PFL file,
we should only copy the chosen component.
Lustre-change: https://review.whamcloud.com/37469
Lustre-commit:
2f1beb33144523467b596f4b6fab882b0a839187
Change-Id: I92bcf6d7b7f7a4387a9936a0b58332e50a88e542
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38051
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Dominique Martinet [Thu, 13 Feb 2020 13:36:32 +0000 (13:36 +0000)]
LU-12198 libcfs: always copy ioctl header back to user
lnetctl_get_peer_list fills back the required size in header if the
given buffer was too small. Userspace needs the info back to grow
the buffer and try again.
Note we only replace err on failure if err was previously not set
Lustre-change: https://review.whamcloud.com/37559
Lustre-commit:
9e02ef474f8caa833d6a1b5e0068d5323a57e8c4
Fixes:
fba98579efc4 ("LU-6202 libcfs: replace libcfs_register_ioctl with a blocking notifier_chain")
Signed-off-by: Dominique Martinet <dominique.martinet@cea.fr>
Change-Id: I2b6e319aceeb00d488572053d27023891afe1928
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37720
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andrew Perepechko [Fri, 14 Feb 2020 02:20:09 +0000 (05:20 +0300)]
LU-13294 libcfs: incorrect rotor behaviour
Signed int cpt rotor is set to -1 on initialization.
cfs_cpt_spread_node() improperly handles this value
via "if (!rotor--)" check. The condition is never true
with negative rotor values, so for_each_node_mask()
only exits with node = MAX_NUMNODES.
kmalloc_node() attempts to determine the zonelist based
on the passed node id and maps MAX_NUMNODES to some
random pointer. Crash.
BUG: unable to handle kernel paging request at
0000000100002007
IP: [<
ffffffff847c0da7>] __alloc_pages_nodemask+0x97/0x420
Lustre-change: https://review.whamcloud.com/37709
Lustre-commit:
f8aa86dd1622804d81020a7dbb1116f276b340f3
Change-Id: I4df74e394bdfc2a918d66aa12e6852ff0f6738ab
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-8492
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38049
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 27 Mar 2020 08:08:26 +0000 (02:08 -0600)]
LU-11986 libcfs: provide QSTR_INIT compat macro
Provide a compat macro for QSTR_INIT() for older kernels.
Fixes:
9d42660e173e ("LU-11986 lnet: properly cleanup lnet debugfs files")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ice19a4dad8456551ba398034a8d3942068006512
Reviewed-on: https://review.whamcloud.com/38090
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sun, 22 Mar 2020 00:17:14 +0000 (18:17 -0600)]
LU-13148 tests: clean up sanity 56ra add debugging
Consolidate duplicate code from sanity.sh test_56ra() into a
helper function to make it easier to see what is being run.
Print out the before and after values for each test.
Skip test_56ra for versions older than 2.12.4, since it was
backported in commit v2_12_3-24-gd55982d842.
Test-Parameters: trivial testlist=sanity env=ONLY=56ra,ONLY_REPEAT=50
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia28c1b556f53ea88643805cbf4ada725a53ebbe5
Reviewed-on: https://review.whamcloud.com/38021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Mon, 10 Feb 2020 04:32:58 +0000 (15:32 +1100)]
LU-12580 lov: fix typo in lov_comp_md_size
If the component of a PFL file is not initialized,
we should use 0 as the stripe size when calculating
the LOVEA size.
Lustre-change: https://review.whamcloud.com/37493
Lustre-commit:
d41716533682ed88b8a77654f9b5b050ef5c672c
Change-Id: I4ff5f4a78bc1d432cc1ac6fa3733461bd6b762e6
Fixes:
62f64a1077 ("LU-9489 lod: keep minimum LOVEA size")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38050
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Wed, 19 Feb 2020 11:08:33 +0000 (19:08 +0800)]
LU-13260 lov: fix size check when stripe is zero
Set correct max size while stripe is zero.
Lustre-change: https://review.whamcloud.com/37623
Lustre-commit:
fe57ce6adf1a00e14269b230d07a4548a58d77c3
Fixes:
f3f6515562 (LU-8998 lov: add composite layout unpacking)
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I9b76283fcc65f58e3be6adf49f035236687ac85c
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38048
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sergey Cheremencev [Fri, 6 Jul 2018 19:51:14 +0000 (22:51 +0300)]
LU-11758 osp: remove assertion from statfs
Sequence can't be changed or overflowed
in case of IDIF. Thus don't tigger kernel
panic for below case:
last_created [0x100000001:0x15:0x0], next_fid [0x100000000:0xfffffff6:0x0]
The same assertion that excepts IDIFs exists
in osp_fid_diff.
Also the patch is adding several optimizations
in osp_precreate_send.
Lustre-commit:
bcfd0e040d1536410ba6c301f64d4f8ea6a8797a
Lustre-change: https://review.whamcloud.com/33832
Change-Id: I3966dfc621999d065c9b485d387938085fccb140
Cray-bug-id: LUS-2386
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-on: https://review.whamcloud.com/37993
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Wed, 15 May 2019 01:52:37 +0000 (09:52 +0800)]
LU-12299 libcfs: fix panic for too large cpu partions
If cpu partions larger than online cpus, following calcuation
will be 0:
num = num_online_cpus() / ncpt;
And it will trigger following panic in cfs_cpt_choose_ncpus()
LASSERT(number > 0);
We actually did not support this, instead of panic
it, return failure is better.
Also fix a invalid pointer access if we failed to init @cfs_cpt_table,
as it will be converted to ERR_PTR() if error happen.
Lustre-change: https://review.whamcloud.com/34864
Lustre-commit:
77771ff24c03a59fc96a7f41199a6b73530a418a
Change-Id: I49daadd8f0c7d22aa78d08248d8c085781740768
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37332
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Dongyang [Mon, 17 Feb 2020 02:53:16 +0000 (13:53 +1100)]
LU-13160 tests: fix sanity-hsm monitor setup
On RHEL8, even we are using pdsh -R ssh,
the ssh still waits for the remote cat process
to finish.
Use the subshell to avoid the time out.
Lustre-change: https://review.whamcloud.com/37595
Lustre-commit:
6724d8ca58e9b8474a180b013a4723cbdd8900d9
Change-Id: Id5b8d492b5ce9a235da73448ade475ade145bbed
Test-Parameters: trivial clientdistro=el8.1 testlist=sanity-hsm
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37773
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Tue, 3 Dec 2019 11:43:28 +0000 (19:43 +0800)]
LU-12775 test: reorder 'tar' command options
'tar' in RHEL8 is stricter in command option order.
Test-Parameters: trivial \
envdefinitions=ONLY="32c" \
clientdistro=el8.1 serverdistro=el8.1 \
mdscount=2 mdtcount=4 testlist=conf-sanity
Lustre-change: https://review.whamcloud.com/36907
Lustre-commit:
f3e101a36310c0c2b9d516c09ec0166eb24524d2
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I814203808efae4a746166abd3ba08f2bc5fce8f7
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Sat, 29 Feb 2020 01:49:42 +0000 (20:49 -0500)]
LU-13274 uapi: make lustre UAPI headers C99 compliant
Attempting to compile strict C99 user land applications or
libraries with the Lustre UAPI headers will fail. These same
errors can be seen by enabling CONFIG_UAPI_HEADER_TEST as well.
Update the Lustre UAPI headers to be compilable with -std=c99.
Enhance our current test covering UAPI header handling.
For OpenSFS branch we can't include <linux/stat.h> since we support
kernels before struct statx existed and they will collide with the
special definitions in lustre_user.h.
Lustre-change: https://review.whamcloud.com/37678
Lustre-commit:
7a7309fa849577ddd5a4f6bb5bfb69e84a7fec89
Signed-off-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Change-Id: Ifb0da33180dc3c7e116d6bf2b7f603ad0528277a
Reviewed-on: https://review.whamcloud.com/37973
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Wed, 26 Feb 2020 14:12:59 +0000 (22:12 +0800)]
LU-11510 lfs: migrate a composite layout file correctly
The patch fixes the following issues:
- in function migrate_open_files(), "layout" pointer should be used
instead of "param" pointer to tell whether a comp file should be
created or not, because "param" pointer is always not null and
the composite layout file will never be created;
- make --copy and --yaml options work correctly in lfs_migrate tool;
- when a composite layout file is migrated, "--copy" option will be
added to preserve its layout in both lfs_migrate and "lfs migrate",
and in such situation, pool name will be saved as well;
- when a file is restriped with -R option by lfs_migrate, the file
will be set with its parent's stripe by default, by adding
"--copy $parent_dir" option;
- do some code cleanup in lfs_migrate and sanity.sh test_56wb/c
sanity.sh test_56xd/xe are added to verify this patch.
Lustre-change: https://review.whamcloud.com/36082
Lustre-commit:
8bedfa377fbd0c9f1b6ea2c40d36fdcaa52137df
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I85779c69e74444eb869f28add4363ad3a6835b97
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37814
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 19 Dec 2019 11:51:41 +0000 (04:51 -0700)]
LU-13090 utils: fix lfs_migrate -p for file with pool
If "lfs_migrate -p <pool>" is run to migrate a file with an existing
pool, the given pool is overridden by the existing pool from the file
during migration. Fix this to use the OST pool requested by the user.
Don't print a warning about deprecated -n option if --dry-run is used.
If a pool is specified, use it with "lfs df" to find OST free space.
Change temp filename to work better with new DNE "crush" hash.
Don't return an error if falling back to rsync and no links are found.
Add test for "lfs_migrate -p" and update man page and usage to match.
Clean up debug-level helpers in test-framework.sh.
Lustre-change: https://review.whamcloud.com/37067
Lustre-commit:
128137adfc539dd2dd92040c14a63ff27f969820
Test-Parameters: trivial testlist=ost-pools
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ief69a620fc969aeff24ec0633a3314c3b83ebbe5
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37813
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 4 Apr 2019 02:26:37 +0000 (20:26 -0600)]
LU-12157 utils: fix lfs_migrate output and testing
Don't pass the "-v" option through to "lfs migrate", as this causes
the filename to be printed twice when run with the "-v" option.
Don't use "echo -e" to process escape characters in filenames unless
this is needed, but add it where it is needed. Don't add ANSI escape
characters to the output. The output previously looked like:
/mnt/testfs/l0: stripe count=1,size=1048576,pool=/mnt/testfs/l0
done migrate
nr[K/mnt/testfs/l1: /mnt/testfs/l1: already migrated ...
nr[K/mnt/testfs/l2: /mnt/testfs/l1: already migrated ...
nr[K/mnt/testfs/l3: /mnt/testfs/l1: already migrated ...
Print out the "pool=" and "mirror_count=" parameters only if needed.
Fix "-A" option to round up the number of stripes when the migrated.
Skip sanity test_56xc 1GB test if there is not enough space on OSTs.
Fixes:
60c5bc2502 ("LU-8235 scripts: pass unrecognized options to lfs migrate")
Fixes:
80a2ff7137 ("LU-6051 utils: allow lfs_migrate to handle hard links")
Fixes:
99d7a8ed43 ("LU-8207 scripts: add auto-stripe option to lfs_migrate")
Lustre-change: https://review.whamcloud.com/34592
Lustre-commit:
9b8e8e0e54e5055c02469cb16d186c94fa2040e0
Test-Parameters: trivial fstype=zfs testlist=sanity envdefinitions=ONLY=56
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I059e7daeb2fa82e7607fd9d862797433053ebbe5
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37812
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 24 Jan 2020 09:20:38 +0000 (02:20 -0700)]
LU-13169 tests: add ONLY_REPEAT parameter to repeat subtests
Add the ONLY_REPEAT environment variable, to allow tests specified
by ONLY to be run multiple times, to ensure that the test is passing
consistently (or fixing an intermittent bug). This is faster than
restarting the test session multiple times for only a few subtests.
Have the iteration around the subshell started for run_one() so that
any registered stack_trap EXIT calls are triggered between iterations,
the fail_loc is reset, grant/health/error checks are done, and so on.
Remove $tdir and $tfile files after each iteration to avoid failures
with the subsequent subtest runs. For tests that do not follow the
standard naming convention for test directories and files, they need
to be updated to use $tdir and $tfile, which is good in any case.
YAML output splits each iteration into a separate subtest for Maloo.
The output from run_one() is appended to a single output file for all
iterations so all output is captured instead of just the last one.
The iterations will continue until $ONLY_REPEAT loops pass, or until
the subtest hits an error. Trying to continue for all iterations in
the face of errors would likely end up with all of later iterations
failing also due to leftover state from the previous failure, and the
goal is for the subtests to pass consistently. If we are trying to
determine rates of intermittent failures, this can be computed using
1/num_passes about the same as num_failures/ONLY_REPEAT iterations.
Rename variables in subtests to avoid clash with testnum, testname,
and TESTNAME, and use them consistently in functions and subtests.
Lustre-change: https://review.whamcloud.com/37321
Lustre-commit:
e16e3d46ee8c44e691c5cd3d25161f2f297fa0fd
Test-Parameters: testlist=sanity envdefinitions=ONLY=27l,ONLY_REPEAT=100
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Change-Id: I5449590dc3e25c113b059974fb7b96c892434380
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Charlie Olmstead <charlie@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37586
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexey Lyashkov [Thu, 27 Feb 2020 14:48:48 +0000 (17:48 +0300)]
LU-13296 obd: make statfs cache working again
Once statfs raced on mutex, lets read a cached data instead
of trash.
Lustre-change: https://review.whamcloud.com/37753
Lustre-commit:
7281635521a823548d497bce2f19acfa3318dfe9
Test-Parameters: testlist=sanity envdefinitions=ONLY=423,ONLY_REPEAT=500
Fixes:
1c41a6ac390b ("LU-12368 obdclass: don't send multiple statfs RPCs")
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: I268782875c30c078f239c194f69cdf7506d66169
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37819
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Sun, 19 Jan 2020 06:26:10 +0000 (01:26 -0500)]
LU-13261 mdt: PFL layout changed while accessing
The PFL layout EA could be enlarged when the corresponding layout
of some IO range is started to be written, which can cause other
thread to get incorrect layout size at "mdt_intent_layout" and cause
"mdt_lvbo_fill" to fail checking the real layout size.
In Lustre, "ldlm_handle_enqueue0" has processed the error "-ERANGE"
and it will retry after expanding the layout buffer size, then it
only needs to decrease the debug level of the log in "mdt_lvbo_fill"
Lustre-change: https://review.whamcloud.com/37684
Lustre-commit:
35d01a0fc7b2933d589f5a6bc4878382cbc15b52
Change-Id: Iad722d1dac187f57ae77606a4d4587525412cd68
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37821
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Sun, 27 Jan 2019 18:23:48 +0000 (11:23 -0700)]
LU-11891 utils: getstripe use --mdt-index consistently
LU-10856 fixed most usages of "warning: '-M' deprecated,
use '--mdt-index' or '-m' instead" but missed a few in
cases in sanity test_271d, test_271e, and test_271f.
Fix those tests to use "--mdt-index".
Also, lfs has a few places were the usage of "--mdt-index"
and "--mdt" is inconsistent. Fix those options to be used
consistently across all commands.
Lustre-change: https://review.whamcloud.com/34116
Lustre-commit:
88d8f0f86bd4994e07aa12bd00cbc7ad3192205c
Fixes:
6c617a3d56 ("LU-10856 tests: remove deprecated lfs ...")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I013b2198f3a39533da9a0067a0bf5846604b3052
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37728
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Tue, 18 Dec 2018 19:01:04 +0000 (12:01 -0700)]
LU-11810 misc: allow Fixes: tag in commit signoff block
Allow the "Fixes:" tag in the signoff block of the commit message.
The Fixes: tag should contain a valid git commit hash, and be
followed by a description of the original patch.
Lustre-change: https://review.whamcloud.com/33888
Lustre-commit:
4d6bff6be51ff6e336f73b2822ac5100254b9431
Fixes:
5760b34c48b4 ("LU-1145 test: add Test-Parameters tag")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I49f712dd8da173510e5941b66eb050e53a1cab07
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-on: https://review.whamcloud.com/37724
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Thu, 12 Dec 2019 19:19:48 +0000 (11:19 -0800)]
LU-13071 lnet: reduce log severity for health events
No need to print an error when the health of an interface is
reduced. Changed it to debug level.
Lustre-change: https://review.whamcloud.com/37002
Lustre-commit:
5567aaba1086217e021938e5f9543c640b2d007c
Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ia60ade12efab732ea4b0388a3803976bf65938ab
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37718
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Alex Zhuravlev [Mon, 10 Feb 2020 21:06:07 +0000 (00:06 +0300)]
LU-11269 ptlrpc: do not expose transient IDLE state
to avoid cases when anyone sending an RPC observes the connection
in this state while it's going to reconnect right away.
Lustre-change: https://review.whamcloud.com/37523
Lustre-commit:
ea8d2ecc783fbaff12c581935ac426b9b8567031
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I9ca89051c4176fe321262f8b2f52969c382e401e
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37649
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Fri, 14 Feb 2020 19:15:56 +0000 (11:15 -0800)]
LU-13110 kernel: kernel update SLES12 SP4 [4.12.14-95.45.1]
Update SLES12 SP4 kernel to 4.12.14-95.45.1 for Lustre client.
Test-Parameters: trivial clientdistro=sles12sp4 \
envdefinitions=LNET_SELFTEST_EXCEPT=smoke,SANITY_EXCEPT="103a 817"
Change-Id: I1f7024465b4b6334488b7314f1073fafa10331d6
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37124
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Sat, 25 Jan 2020 21:23:28 +0000 (05:23 +0800)]
LU-13191 osp: handle -EROFS in osp_sync_interpret()
Upon OST disk failure, osp_sync_interpret() may get -EROFS,
which is a valid errno.
Lustre-change: https://review.whamcloud.com/37404
Lustre-commit:
868089cd309506719b814afecebf825effc6c93f
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I5c3cff3019aa47c6d5803f0f0b373bc704f18118
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37516
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mikhail Pershin [Wed, 29 Jan 2020 21:22:07 +0000 (00:22 +0300)]
LU-10198 llog: keep llog handle alive until last reference
Llog handle keeps related dt_object pinned until llog_close()
call, meanwhile llog handle can still have other users which
took llog handle via llog_cat_id2handle()
Patch changes llog_handle_put() to call lop_close() upon last
reference drop. So llog_osd_close() will put dt_object only
when llog_handle has no more references.
The llog_handle_get() checks and reports if llog_handle has
zero reference.
Also patch modifies checks for destroyed llogs, llog handle
has new lgh_destroyed flag which is set when llog is destroyed,
llog_osd_exist() checks dt_object_exist() and lgh_destroyed
flag, so destroyed llogs are considered as non-existent too.
Previously it uses lu_object_is_dying() check which is not
reliable because means only that object is not to be kept in
cache.
Lustre-change: https://review.whamcloud.com/37367
Lustre-commit:
d6bd5e9cc49b3bb9901ada503107e8b0eca44f7e
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: If7df41646c243c0d40b20a30a33e86c688d24508
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37514
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log