Whamcloud - gitweb
Andreas Dilger [Tue, 24 Mar 2020 13:57:39 +0000 (09:57 -0400)]
LU-9091 procfs: remove old string parsing routines
Remove the llprocfs_str_with_units_to_s64(), lu_str_to_s64(),
llprocfs_str_with_units_to_u64(), and lu_str_to_u64() functions.
All new code should use sysfs_memparse() to parse sizes from
userspace.
Mark patch forbuildonly so that it does not land before we have
a chance to remove any patches with users of these functions.
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6dc321e60eb587ef25652f4b5ae8fa478d3ebbe5
Reviewed-on: https://review.whamcloud.com/37275
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 21 Nov 2019 04:26:04 +0000 (15:26 +1100)]
LU-13004 net: discard LNET_MD_IOVEC
This flag is now never set, so discard it. lnet_md memory descriptors
are either a single virtual address, or a list of bvec.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9b2280f801108e5d9fc718c90c83692042991f12
Reviewed-on: https://review.whamcloud.com/36974
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Tue, 3 Dec 2019 17:22:03 +0000 (09:22 -0800)]
LU-12292 lnet: keep health even if recovery failed
Don't decrement the interface's health value when recovery
message fails. If we've already determined that an interface
is unhealthy, there is no need to continue decrementing
it's health every 1 second. It'll take too long to come back
into service when it becomes healthy.
Clean up where health is decremented in order not to have
repetitive decrements. No need to decrement in lnet_notify()
because in order for the LND to call this an existing transmit
must've failed. This means a message has already failed which
will result in the health being decremented.
When a recovery send fails make sure to flag the recovery as
failed because there is no reply expected in this case.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ifb3500a77a5a5be51e7079269c8ddba85ed0c2a7
Reviewed-on: https://review.whamcloud.com/36921
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Tue, 3 Dec 2019 01:09:07 +0000 (17:09 -0800)]
LU-12303 lnet: recover health at same rate as dec
When there is a failure to send to or over an interface the
interface's health value is decremented by lnet_health_sensitivity.
Originally, when it recovers the health value goes up by 1.
This patch makes incrementing and decrementing health value
symmetrical, by lnet_health_sensitivity.
In this way if a site wants to stop using and start using an interface
quickly, then it can set lnet_health_sensitivity to a large value
compared to the health value maximum, which is hard coded to 1000.
For example setting lnet_health_sensitivity to 500 will reduce the
health value of an interface to 0 after two failed sends and
reset it to maximum health value after two successful sends.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ib5cedb063d9ccf79c574edac291551f8c94bcce4
Reviewed-on: https://review.whamcloud.com/36920
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Mon, 2 Dec 2019 01:05:24 +0000 (17:05 -0800)]
LU-13028 lnet: advertise discovery when toggled
When discovery is toggled send a push message to all peers.
When a node receives a push notification that discovery is
turned off while previously it was on, then delete the peer
information. If the peer is a router, recreate the routes.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I58f9f42542e4c05763128d7c9d23108c3e7f13a3
Reviewed-on: https://review.whamcloud.com/36919
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 18 Mar 2020 02:55:19 +0000 (13:55 +1100)]
LU-8130 lu_object: use RCU to free lu_object_header
Any struct containing an lu_object_header is now freed using
call_rcu() or rcu_kfree() when freed by loo_object_free.
This will allow RCU protocols to protect lookups once we
switch the lu_object cache to rhashtable in a future patch.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib0e1134ae2ac73532d6f953ff4510076f1d8a78a
Reviewed-on: https://review.whamcloud.com/37965
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Oct 2019 00:30:51 +0000 (11:30 +1100)]
LU-12780 quota: don't use ptlrpc_thread for qsd_upd_thread()
Instead of ptlrpc_thread, use native kthread functionality.
- for startup, perform allocations before starting the thread,
and use a completion to ensure thread function runs before
kthread_stop() is ever called, so cleanup is guaranteed to
happen.
- for shutdown, use kthread_stop/kthread_should_stop
- for signalling the thread, use wake_up_process. The thread
sets TASK_IDLE while checking if there is anything to do,
and TASK_RUNNING when it finds something to work on, so
the schedule_timeout() only blocks if there was nothing to do.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I2ec2e28320e14251f342b8749a1a738f136f3575
Reviewed-on: https://review.whamcloud.com/36267
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Elena Gryaznova [Wed, 5 Jun 2019 11:46:18 +0000 (14:46 +0300)]
LU-12390 tests: customise mdsrate tests
Patch adds the following customizations:
set custom layout for filesystem root
directory by specifying fs_STRIPEPARAMS
and
set the custom layout for mdsrate test's
working directory BASEDIR by specifying
mdsrate_STRIPEPARAMS.
Test-Parameters: trivial testlist=performance-sanity
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-7249
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I28524c203f86e7e94a4db47a8d8c4b3518883600
Reviewed-on: https://review.whamcloud.com/35068
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Wed, 11 Mar 2020 04:52:35 +0000 (22:52 -0600)]
LU-11596 tests: wait for grant to stop changing
Wait for the client and server grant to stop changing in sanity
test-42d, test_64[acd] to avoid intermittent failures.
Test-Parameters: trivial testlist=sanity env=ONLY=64a,ONLY_REPEAT=100
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3ef3a93d04e73ffb0098f712373c1462c53ebbe5
Reviewed-on: https://review.whamcloud.com/37876
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Sat, 15 Feb 2020 02:48:59 +0000 (10:48 +0800)]
LU-11025 lmv: simplify name to stripe mapping
Handle layout change internally when mapping name to stripe:
* Move layout changing related code into lmv name to stripe mapping
so callers doesn't need to take care of the internals.
* lmv_name_to_stripe_index() maps name in new layout, and
lmv_name_to_stripe_index_old() in old layout.
* rename lmv_migrate_existence_check() to lmv_old_layout_lookup()
to support directory restripe in the future.
* support layout changing directory in LFSCK.
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Icf8bda5db884784f761a2d373a6f81d7e13f525f
Reviewed-on: https://review.whamcloud.com/37711
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Fri, 14 Feb 2020 09:37:58 +0000 (17:37 +0800)]
LU-11025 dne: change dir layout via dt_layout_change
This patch includes following changes:
* Use existing mo_layout_change()/dt_layout_change() interface to
change directory layout, which is cleaner and easier to add new
features. The supported layout change operations are:
1. MD_LAYOUT_DETACH: detach stripes from directory master object.
2. MD_LAYOUT_ATTACH: attach stripes after target's.
3. MD_LAYOUT_SHRINK: destroy stripes from specific offset.
* Previously layout change is done in MDD layer if striped directory
is remote, however LOD can handle remote striped directory the same
as local one, NB, remember to initialize index operations before
altering directory layout. In this way layer violation can be
avoided.
* Use setxattr(XATTR_NAME_LMV, LU_XATTR_CREATE) to create stripes
for directories, and '0' or 'LU_XATTR_REPLACE' to set or replace
directory LMV instead of setxattr(XATTR_NAME_LMV".set").
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Iac226f9e0e504f19837e14e93cee2542a2cb5e3d
Reviewed-on: https://review.whamcloud.com/37634
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Lai Siyao [Wed, 31 Jul 2019 02:30:15 +0000 (10:30 +0800)]
LU-11025 dne: introduce new directory hash type: "crush"
Current directory striping strategy is fixed striping, i.e., it
calls (hash(filename) % stripe_count) to decide stripe for file.
The problem with this approach is that if stripe_count changes,
most of the files will need to be relocated between MDTs. This
makes directory split/merge quite expensive.
This patch introduces consistent hash striping strategy:
it calls (hash(filename) % LMV_CRUSH_PG_COUNT) to locate PG_ID
(placement group index), and then calls
crush_hash(PG_ID, stripe_index) to get a straw for each stripe,
and the stripe with the highest staw will be used to place this
file.
As we can see, it uses the CRUSH algorithm, but it only uses it
to map placement group pseudo-randomly among all stripes, while
doesn't use it to choose MDTs if MDT is not specified. The latter
is done by MDT object QoS allocation in LMV and LOD (LMV decides
the starting stripe MDT, while LOD decides the rest stripes).
This implementation contains below changes:
* new hash type "crush", and lmv_name_to_stripe_index() will take
care of it in mapping name to stripe index.
* add "mdt_hash" sysfs tunable for LOD, which will be the default LMV
hash type used in mkdir.
* if 'lfs setdirstripe' doesn't set hash type, server will choose
the default LMV hash type.
* place temp file on MDT where target is: map temporary file created
by rsync or mpifileutils dstripe on the same MDT where target is
located, so later rename won't make target a remote entry. And the
same for backup file ends with .bak, .sav, .orig or ~.
Compatibility:
* if server doesn't support OBD_CONNECT2_CRUSH, client will switch
to "fnv_1a_64" hash type if "crush" is specified.
* if client doesn't know "crush" hash type, it will try all stripes
in lookup.
* client will set hash type to LMV_HASH_TYPE_UNKNOWN if setdirstripe
doesn't set it explicitly, and server will use the default hash
type to create striped directory, NB, for backward compatibility,
client will set it to LMV_HASH_TYPE_DEFAULT if server < 2.14.
test_mkdir() in sanity test will create striped directory with
random hash type if not specified.
sanity test 160g, 160h and 160i are fragile, which require a certain
amount of files created under each stripe, use 'fnv_1a_64' hash type
to fullfill this requirement.
sanity-lfsck test 31b has the same limit, create more subdirs to
make it more robust.
Add sanity 33h for temp file mapping.
Test-Parameters: trivial
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I669e561c667f926c35cf1338f4c6604249e1ee51
Reviewed-on: https://review.whamcloud.com/36775
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Oleg Drokin [Sat, 14 Mar 2020 07:11:29 +0000 (03:11 -0400)]
LU-13131 osc: Make sure we don't accidentally deprioritize extents
Looks like in some cases high priority extents could migrate off
oo_hp_exts list back onto urgent list.
Change-Id: I96bf0a3a005b166f34dba215463c0806dfe2526a
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37967
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Dongyang [Fri, 7 Feb 2020 12:16:26 +0000 (23:16 +1100)]
LU-12580 lov: fix out of bound usercopy
When handling ioctl LL_IOC_LOV_GETSTRIPE, the user
could pass a limited buffer which is bigger than
lov_comp_md_size(), it will crash the client because
we are doing the usercopy with the user provided buffer
size.
Make sure the copy works, also for the PFL file,
we should only copy the chosen component.
Change-Id: I92bcf6d7b7f7a4387a9936a0b58332e50a88e542
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/37469
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Li Dongyang [Mon, 10 Feb 2020 04:32:58 +0000 (15:32 +1100)]
LU-12580 lov: fix typo in lov_comp_md_size
If the component of a PFL file is not initialized,
we should use 0 as the stripe size when calculating
the LOVEA size.
Change-Id: I4ff5f4a78bc1d432cc1ac6fa3733461bd6b762e6
Fixes:
62f64a1077 ("LU-9489 lod: keep minimum LOVEA size")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/37493
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Mon, 9 Mar 2020 02:20:41 +0000 (13:20 +1100)]
LU-11591 llog: remove partial_chunk variable
This variable is set but never used, so remove it.
Test-Parameters: trivial
Fixes:
ec4194e4e78c ("LU-11591 llog: add synchronization for the last record")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I59ec75ba66a3ca37990d3ffec1d1b8f80350ae52
Reviewed-on: https://review.whamcloud.com/37830
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Wang Shilong [Sun, 15 Mar 2020 14:06:40 +0000 (22:06 +0800)]
LU-12748 readahead: limit async ra requests
Currently async readahead is limited by following factors:
1) @ra_max_pages_per_file
2) @ra_max_read_ahead_whole_pages;
3) @ra_async_pages_per_file_threshold
If admin change a large value 4G to @ra_max_read_ahead_whole_pages,
with 16M RPC we could have 256 async readahead requests
flighting at the same time, this could consume all CPU
resources for readahead without limiting.
Even though we could set @max_active for workqueue,
RA requests still kept in the workqueue pool which help
prevent from CPU busying, the problem is RA still try to
use CPU later, we might still submit too many requests
to workqueue, so instead of limiting it in the workqueue,
we could limit it earlier, if there has been too many
async RA requests in the system(let's say default is 1/2
of CPU cores), we just fallback to sync RA, which limit
read threads using all CPU resources.
Change-Id: I370c04e014f24c795c1a28effca9c51b1db2a417
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37927
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Wed, 11 Mar 2020 13:28:15 +0000 (16:28 +0300)]
LU-13354 capa: remove remained lustre capa code
Remove remaining unused lustre capability structures and
defines.
Test-Parameters: trivial
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Iebd5bf87537c19fb30804adaf623c185983c351b
Reviewed-on: https://review.whamcloud.com/37895
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mikhail Pershin [Wed, 11 Mar 2020 08:59:10 +0000 (11:59 +0300)]
LU-13354 ptlrpc: update wirecheck with missed checks
- add missed values and defines in wirecheck.c
- prefer compile-time checks with CVALUE/CDEFINE for new
blocks of checks, one-line updates are added with the
same checks style as checks around it.
- update wirehdr.c to generate proper wiretest.c. Changes
were made to wiretest.c directly, change wirehdr.c to
reflect these changes.
- remove obsoleted and not used capa enum and defines found
while updating checks
Test-Parameters: trivial
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ia1ea4ce30d61c58b20e3f20bed083812391f5998
Reviewed-on: https://review.whamcloud.com/37877
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Mr NeilBrown [Tue, 10 Mar 2020 20:34:41 +0000 (16:34 -0400)]
LU-13344 libcfs: adler32: don't set CRYPTO_TFM_RES_BAD_KEY_LEN
This flag was not being checked by most callers and has been removed since
Commit
674f368a952c ("crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN")
so remove from lustre too.
Change-Id: Ia616430a98d05c7bb3b22a0be543e83855272d2f
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/37870
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 12 Mar 2020 19:52:56 +0000 (15:52 -0400)]
LU-9859 libcfs: open code cfs_trace_max_debug_mb() into cfs_trace_set_debug_mb()
This function in used twice.
In libcfs_debug_init() the usage is pointless as
the only place that set libcfs_debug_mb ensures
that it does not exceed the maximum. So checking
again can never help.
So open-code the small function into the only
other place that it is used - in cfs_trace_set_debug_mb(),
which is used to set libcfs_debug_mb.
HPC machines are being deployed with 24 TiB or more memory.
Change the variables uses to calculate the max_debug_mb
to unsigned long to avoid a potential overflow and change
80 / 100 scaling factors to 4 / 5.
Linux-commit:
5b5df84f3582b0e3a2ff70d49867bc3e03f67502
Change-Id: I497502bb7418e0b3d24b24c507029652bc9a9e55
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/37829
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Mon, 9 Mar 2020 01:35:57 +0000 (21:35 -0400)]
LU-9859 libcfs: always range-check libcfs_debug_mb setting.
When the libcfs_debug_mb module parameter is set
at module-load time it isn't range-checked. When
it is set via sysfs it is. This inconsistency
makes the code harder to follow.
It is quite safe to call cfs_trace_set_debug_mb()
and cfs_trace_get_debug_mb() before the module
is initialized as cfs_tcd_for_each() does nothing
before initializtion.
So change cfs_trace_set_debug_mb() - which does
range checking - to returned the ranged checked number (it currently
always returns zero) and use that as the new value, unless
cfs_trace_get_debug_mb() now returns a non-zero value.
Linux-commit:
205b154f3bd6fa3e95ceca8bba8d00122bb8fb04
Change-Id: I495097bbae138625e54a012bf06c96e62d206228
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/37828
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Fri, 6 Mar 2020 21:37:47 +0000 (15:37 -0600)]
LU-13120 build: Fix ZFS dependancies for osd-zfs-mount
lustre-osd-zfs-mount depends on zfs
lustre-osd-zfs-mount depends on kmod-lustre-osd-zfs
SuSE packaging style prefers kmp package naming so prepare
for adopting a kmp named zfs package
Test-Parameters: fstype=zfs
Cray-bug-id: LUS-7077
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ifcee1fa90bf95925cbcff0975d95f12ae04f670d
Reviewed-on: https://review.whamcloud.com/37809
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Sat, 29 Feb 2020 14:02:55 +0000 (22:02 +0800)]
LU-13312 ldlm: fix to stop iterating tree early in ldlm_kms_shift_cb()
It is very possible that old_kms is equal (last extent end + 1),
and if we fail last one check, we will have to iterate all PR extent locks
in tree which is O(N).
This is very likely to happen in IO500 hard mode, because firstly
we try to write which generates a lot of PW locks, after that read
is triggered which need cancel all PW locks to PR locks.
Be careful to avoid overflow, as @l_extent.end could be OBD_OBJECT_EOF,
if that is case, it just means we could stop iterating.
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ie142fb738482a8abcc7d0c9d67d6e6eb520459db
Reviewed-on: https://review.whamcloud.com/37762
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Andrew Perepechko [Fri, 14 Feb 2020 02:20:09 +0000 (05:20 +0300)]
LU-13294 libcfs: incorrect rotor behaviour
Signed int cpt rotor is set to -1 on initialization.
cfs_cpt_spread_node() improperly handles this value
via "if (!rotor--)" check. The condition is never true
with negative rotor values, so for_each_node_mask()
only exits with node = MAX_NUMNODES.
kmalloc_node() attempts to determine the zonelist based
on the passed node id and maps MAX_NUMNODES to some
random pointer. Crash.
BUG: unable to handle kernel paging request at
0000000100002007
IP: [<
ffffffff847c0da7>] __alloc_pages_nodemask+0x97/0x420
Change-Id: I4df74e394bdfc2a918d66aa12e6852ff0f6738ab
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-8492
Reviewed-on: https://review.whamcloud.com/37709
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Mon, 10 Feb 2020 20:11:49 +0000 (14:11 -0600)]
LU-13278 lnet: Reconcile discovery push and reply handling
Reconcile the logic for updating the multi-rail flag of a peer when
processing a discovery PUSH with the logic used when processing a
discovery REPLY.
Cray-bug-id: LUS-8516
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Idfb4c3729822d03b71f9440ac66176ae6b886022
Reviewed-on: https://review.whamcloud.com/37674
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Stephen Champion <stephen.champion@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Elena Gryaznova [Wed, 26 Feb 2020 16:05:11 +0000 (19:05 +0300)]
LU-13280 tests: customize run_mdtest for several mount points
Patch adds the possibility to run mdtest on $mdtest_Nmntp
number of Lustre mount points.
Test-Parameters: trivial testlist=parallel-scale \
envdefinitions="mdtest_Nmntp=3" \
envdefinitions=ONLY=mdtestfpp envdefinitions=ONLY=mdtestssf
Cray-bug-id: LUS-7686
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I9bc68d4f296970290cf460bf25cc26537bcc3281
Reviewed-on: https://review.whamcloud.com/37665
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Yang Sheng [Wed, 19 Feb 2020 11:08:33 +0000 (19:08 +0800)]
LU-13260 lov: fix size check when stripe is zero
Set correct max size while stripe is zero.
Fixes:
f3f6515562 (LU-8998 lov: add composite layout unpacking)
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I9b76283fcc65f58e3be6adf49f035236687ac85c
Reviewed-on: https://review.whamcloud.com/37623
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 10 May 2019 21:09:41 +0000 (15:09 -0600)]
LU-9091 obdclass: allow bare KMGTPE param suffix
Allow sysfs_memparse() to parse bare "KMGTPE" suffixes as base-2.
This simplifies the code to not append the unnecessary "iB" if
the suffix is one of KMGTPE already, since these are assumed to
be binary units already.
Don't append a bare defunit of "B", since this unnecessarily copies
the input string and takes the slow path for parsing the string, but
doesn't actually change the resulting value.
The test_string_to_size_one() may only have printed an error and not
caused module load to actually fail. Print a message and return an
error if the tests fail. Some checks expect to fail, so add
test_string_to_size_err() to verify that an error was returned.
Mark the obd_init_checks() function with __init so it is dropped
after the module is loaded.
Fixes:
d9e0c9f346d0 ("LU-9091 sysfs: use string helper like functions for sysfs")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3cdf5f8f0aeca458ed1989366102c33ae83ebbe5
Reviewed-on: https://review.whamcloud.com/37620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Fri, 6 Mar 2020 01:40:47 +0000 (20:40 -0500)]
LU-12477 ldiskfs: remove obsolete ext4 patches
Drop support for ldiskfs kernels earlier than RHEL7.6.
Test-Parameters: trivial
Change-Id: I30450904c508ec8aa5388cbfd9bd967028f88b28
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/37352
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 21 Nov 2019 05:53:57 +0000 (16:53 +1100)]
LU-13004 modules: replace lnet_kiov_t with struct bio_vec
lnet_kiov_t has the same structure and general purpose
as struct bio_vec - only the names a different.
The difference brings no value - so let's remove it.
sed -e 's/lnet_kiov_t/struct biovec/g'
-e 's/kiov_page/bv_page/g'
-e 's/kiov_len/bv_len/g'
-e 's/kiov_offset/bv_offset/g'
plus some cleaning up.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia718e5880d0fbbb9aa9aed0edf48a2b2c42b54d3
Reviewed-on: https://review.whamcloud.com/37852
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Sun, 1 Dec 2019 22:49:31 +0000 (14:49 -0800)]
LU-13041 lnet: fix router primary_nid deletion
For MR routers it is possible to update the primary_nid. This will
result in a push to update the peers of the changes. The peer should
be able to update the primary_nid of the router.
When the peer receives the push notifications and deletes the primary
NID of the router, it should update the router's primary NID to be
the next peer NI in the list.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I5928882dde513cb885301f632535282695429bb0
Reviewed-on: https://review.whamcloud.com/36918
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Sun, 1 Dec 2019 20:18:38 +0000 (12:18 -0800)]
LU-13027 lnet: fix route consolidation
After discovery one or more peers can be merged together. Some
of those peers might be gateways. The routes from the gateways
should be transferred to the MR peer. However, if the MR peer
is already a gateway, then we shouldn't add routes to a remote
net for which a route already exists, unless that route is
higher priority or has less number of hops, in which case we
should replace the route with the better one.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I8790f39a604f4e3d222c1f98c3d28d3dfb03a237
Reviewed-on: https://review.whamcloud.com/36917
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Fri, 29 Nov 2019 05:22:30 +0000 (21:22 -0800)]
LU-13035 lnet: fix remote peer ni selection
When selecting the NI of a remote peer we can only round robin
over the NIs. Remote peer NIs have no credits and no health.
However, we'd like to still spread the traffic over the remote
peer NIs as much as we can.
Remote peer NIs sequence number was not being incremented and
therefore the round robin algorithm wasn't working.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I83dba3597987844b7bddc980c53e8c6d90bcd3b2
Reviewed-on: https://review.whamcloud.com/36915
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Thu, 28 Nov 2019 23:44:27 +0000 (15:44 -0800)]
LU-13029 lnet: fix asym routing with multi-hop
When a system is setup with multi-hop routing the next-hop
gateway, one off the nodes, will not have an interface
on the remote network. The route should still be considered
alive even if lnet_avoid_asym_routing is set
The code sets a route to be multi-hop if the gateway it uses
does not have an NI on the remote net of the route. The underlying
assumption here is that routing configuration is correct
To aid in fault detection a warning message is printed out if
the route is set to be multi-hop but the route hop count is less
than 1.
Added a YAML field in the:
lnetctl route show -v
output to print the type of the route: single-hop or multi-hop.
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I0a96263e2903974ec50d46bd56e550cde7d86e8c
Reviewed-on: https://review.whamcloud.com/36914
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 20 Nov 2019 02:16:04 +0000 (13:16 +1100)]
LU-13005 ptlrpc: use percpu refcount to track requests.
ptlrpc needs to wait for outstanding RPC requests to complete before
ptlrpc_ni_fini() finishes.
It currently does this using a refcount in the event-queue, but that
refcount is otherwise unused, and it will shortly be removed.
So add a percpu refcount that explicitly tracks these requests, and
wait for it with a completion.
Also move the declaration of ptlrpc_eq to ptlrpc_internal.h which is a
more appropriate location.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I479df29d1ddfb4e3d82a26b5fb567f3432dad8af
Reviewed-on: https://review.whamcloud.com/36845
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Thu, 26 Dec 2019 20:22:42 +0000 (15:22 -0500)]
LU-12546 lfsck: print logs for LLIT_UNMATCHED_PAIR
Print the information of MDT/OST objects for LLIT_UNMATCHED_PAIR
to help track the affected files.
Change-Id: Ifa94ef9365fc6fa9875bbcdb9ad377150a52700d
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36744
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Oct 2019 00:30:51 +0000 (11:30 +1100)]
LU-12780 quota: don't use ptlrpc_thread for qsd_reint_main()
Instead of ptlrpc_thread, use native kthread operations.
- perform allocation before starting the thread, and don't
bother with synchronization at thread start
- use kthread_stop() and kthread_should_stop() to signal
shutdown
- as the thread can stop independently, use xchg() on
qqi->qqi_reint_task to determine if there is a waiter, and
if so, the thread should synchronize with the waiter.
- use wake_up_var and wait_var_event for event signalling.
Also use kthread's thread-name formatting rather than
allocating a 'name' for formating the name separately.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia1334a3e865b67ba5ae78c7b8342704a340229f7
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/36268
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Oct 2019 00:30:50 +0000 (11:30 +1100)]
LU-12780 osp: don't use ptlrpc_thread for lwp_notify_main()
Instead of ptlrpc_thread, use native kthread functionality.
- There is no need to synchronise on start-up. But we do
need to ensure class_export_get/put are balanced, and we
cannot be certain that the thread function will run.
So call class_export_get() before starting the thread,
and class_export_put() after it has finished.
- As the thread exits independantly, waiting for it to finish
is a little more complex than just using kthread_stop().
If the thread stops before lwp_device_fini() runs, it sets
->lpd_notify_task to NULL, and lwp_device_fini() doesn't wait.
If lwp_device_fini() runs first, it set ->lpd_notify_task to
NULL and calls kthread_stop(). The thread notices this, and
deliberately waits for kthread_should_stop() to avoid any
races in kthread code.
xchg() is used to ensure exactly one of the thread and
lwp_device_fini() sets lpd_notify_task to NULL.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I45621f325d14194af5a44f86f7e605bdafbbb9be
Reviewed-on: https://review.whamcloud.com/36266
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sat, 7 Mar 2020 16:26:35 +0000 (10:26 -0600)]
LU-11310 ldiskfs: Support for SUSE 15 GA and SP1
Add support for ldiskfs SUSE 15 SP1 and late GA:
GA 150 series kernels and SP1 195, and 197 series
Test-Parameters: trivial
Cray-bug-id: LUS-7572
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2525acc475503166fb71bed59663edeee603d38c
Reviewed-on: https://review.whamcloud.com/36094
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Shaun Tancheff [Mon, 2 Dec 2019 18:14:30 +0000 (12:14 -0600)]
LU-12670 osd-ldiskfs: Shrink rcu critical section
Drop ENTRY/EXIT macros to exclude them rcu critical section
scope without introducing additional work around logic.
Cray-bug-id: LUS-7771
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Change-Id: I70486ba137592d1a57bc712837b8a300b8017a36
Reviewed-on: https://review.whamcloud.com/35805
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
NeilBrown [Wed, 11 Mar 2020 13:01:09 +0000 (09:01 -0400)]
LU-9859 libcfs: improve API and implementation of blocking signals.
According to comment for set_current_blocked() in
kernel/signal.c, changing ->blocked directly is wrong.
sigprocmask() should be called instead.
So change cfs_block_sigsinv() and cfs_restore_sigs()
to use sigprocmask().
For consistency, change them to pass the sigset_t by reference
rather than by value.
Also fix cfs_block_sigsinv() so that it correctly blocks
signals above 32 on a 32bit host.
Linux-commit:
84e07b9d0ac8728b1865b23498d746861a8ab4c2
Change-Id: Iffb8b39ee2b988e9909ceaaba50446eecdf7f249
Signed-off-by: NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/35411
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Chris Horn [Fri, 13 Mar 2020 19:34:23 +0000 (14:34 -0500)]
LU-13362 lnet: Disc reply race with finalize and routed recv
A race exists between a thread handling a discovery reply, and
another thread in the lnet_finalize() call path, or in any of
the code paths that result in lnet_post_routed_recv_locked().
The discovery reply handler takes the lp_lock, and while holding
that lock, tries to acquire the lpni_lock for each lpni associated
with the lnet_peer object.
In lnet_return_rx_credits_locked() (lnet_finalize() code path) and
lnet_post_routed_recv_locked() (called via a couple different code
paths) the lpni_lock is taken, and then the lp_lock is taken for the
associated lnet_peer object.
Thread A: spin_lock(lp_lock)
Thread B: spin_lock(lpni_lock)
Thread B: spin_lock(lp_lock)
Thread A: spin_lock(lpni_lock)
This results in deadlock. The lp_lock and lpni_lock do not need to be
held at the same time in lnet_return_rx_credits_locked() nor in
lnet_post_routed_recv_locked().
Cray-bug-id: LUS-8607
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Ie4e9a172b4705d9f5723a6da1ff251b380ad47ac
Reviewed-on: https://review.whamcloud.com/37937
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sun, 1 Mar 2020 11:40:22 +0000 (04:40 -0700)]
LU-6142 llite: Fix style issues for llite/statahead.c
Fix style issues for llite/statahead.c.
Test-Parameters: trivial testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie6d8507f35cbd586ca51ad6f1c5dc1ef7abc3305
Reviewed-on: https://review.whamcloud.com/37765
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Wed, 4 Mar 2020 14:07:19 +0000 (09:07 -0500)]
LU-12506 changelog: support large number of MDT
At client, the changelog of each MDT is associated with
one miscdevice, but the number of miscdevice is limited
to 64 in Linux kernel, then it will fail if there are
more than 64 MDTs.
This patch replaces miscdevice with dynamic devices to
support more MDTs.
Change-Id: Ie3ce76cbe1c613bf17d6350ea95546524b6d66b8
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37759
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Feb 2020 04:28:50 +0000 (15:28 +1100)]
LU-6142 lustre: remove remaining users of ldebugfs_register
ldebugfs_register() is just a call to debugfs_create_dir() and
ldebugfs_add_vars() if the list option is set. Fix up the remaining
users of this function to just call these two functions instead, and
delete the now unused ldebugfs_register() call.
This ends up cleaning up more code and making things smaller, always a
good thing.
Linux-Commit
00905f006608 ("staging: lustre: remove last two
users of ldebugfs_register()")
Test-Parameters: trivial
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I08f4cdcdd3367f50c37d899a5dc4d753e72c0fda
Reviewed-on: https://review.whamcloud.com/37743
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Mr NeilBrown [Thu, 27 Feb 2020 03:58:57 +0000 (14:58 +1100)]
LU-6142 lustre: unwrap some ldebugfs_register() calls
When the third option (list) to ldebugfs_register() is NULL, it's the
same as just calling debugfs_create_dir(). So unwind this and call
debugfs_create_dir() directly.
This ends up saving lots of code as we do not need to do any error
checking of the return value (because it does not matter).
Note: this depends on Commit
a59d6293e537 ("debugfs: change parameter
check in debugfs_remove() functions") which landed in v3.6-rc1.
Linux-Commit
b145f49f233d ("staging: lustre: unwrap some
ldebugfs_register() calls")
Test-Parameters: trivial
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6bff3b1d416de1f15bb95b22fa6075b1ae30a534
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/37742
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Feb 2020 02:59:27 +0000 (13:59 +1100)]
LU-6142 fld: remove function typedefs
typedefs a generally discouraged in upstream linux.
fld_hash_func_t and fld_scan_func_t are only used
once immediately after the definition, so having them
doesn't help readability or maintainability at all.
So remove them.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I37288238f375d5673b7cb8d68d30102f43e44107
Reviewed-on: https://review.whamcloud.com/37740
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Feb 2020 02:47:38 +0000 (13:47 +1100)]
LU-12511 fld: use list_last_entry in fld_cache_shrink
fld_cache_shrink can be substantially simplified by using
list_last_entry().
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I0dcbd8ffefbd113786f422b7662298605020687d
Reviewed-on: https://review.whamcloud.com/37739
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Feb 2020 02:43:25 +0000 (13:43 +1100)]
LU-12511 lustre: introduce LUSTRE_MDT_MAXNAMELEN
Various char arrays which can contain a target name are
declared as [80].
Instead of a "magic constant" introduce LUSTRE_MD_MAXNAMELEN
and use that instead.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I029b8086d081297f4018ba9752c484b1ebd596db
Reviewed-on: https://review.whamcloud.com/37738
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Feb 2020 02:35:51 +0000 (13:35 +1100)]
LU-12511 fld: remove unused functions from fld_cache.c
fld_cache_delete and fld_cache_entry_lookup{,nolock}
are never used, so they can be removed.
fld_cache_entry_delete is only used locally to fld_cache.c
so it can be made static.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ied370420c348c95e14897187daaaec6708b5c96a
Reviewed-on: https://review.whamcloud.com/37737
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Mr NeilBrown [Fri, 7 Feb 2020 02:05:56 +0000 (13:05 +1100)]
LU-10391 lnet: change various function to return pointer
Rather than returning an error code, and using a by-ref
arg to return the socket pointer, the functions
lnet_connect()
lnet_sock_listen()
lnet_sock_connect()
now return the socket pointer or an ERR_PTR() with the
error encoded.
This is more in-keeping with kernel style.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic09d3bf38a89920a1298848d5c8a9017428cb752
Reviewed-on: https://review.whamcloud.com/37701
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Fri, 7 Feb 2020 02:05:56 +0000 (13:05 +1100)]
LU-10391 lnet: remove 'fatal' arg from lnet_{sock_}connect
The 'fatal' arg for indicating if an error is fatal doesn't
help make the code easier to read. In each case where it is
tested, it would be more clear to directly tested for the
particular errors that are not fatal.
So remove the arg and change the tests.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I905e0c2652b3148d93219a32a6b93515af323159
Reviewed-on: https://review.whamcloud.com/37700
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
NeilBrown [Fri, 14 Dec 2018 03:48:45 +0000 (14:48 +1100)]
LU-9679 osc: convert cl_cache_waiters to a wait_queue.
cli->cl_cache_waiters is a list of tasks that need
to be woken when grant-space becomes available. This
means it is acting much like a wait queue.
So let's change it to really be a wait queue.
The current implementation adds new waiters to the end of the list,
and calls osc_enter_cache_try() on each in order. We can provide the
same behaviour by using an exclusive wait, and having each waiter wake
the next task when it succeeds.
If a waiter notices that success has become impossible, it wakes all
other waiters.
If a waiter times out, it doesn't wake other - just leaves them to
time out themselves.
Note that the old code handled -EINTR from the wait function. That is
not a possible return value when wait_event_idle* is used, so that
case is discarded.
As we need wait_event_idle_exclusive_timeout_cmd(), we should fix the
bug in that macro - the "might_sleep()" is wrong, as a spinlock might
be held at that point.
Linux-Commit:
31f45f56ecdf ("lustre: osc_cache: convert
cl_cache_waiters to a wait_queue.")
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: Ib7622ea2daea8f6e59bef95d3b6c5a80d209b81e
Reviewed-on: https://review.whamcloud.com/37605
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
NeilBrown [Wed, 12 Dec 2018 03:06:35 +0000 (14:06 +1100)]
LU-9679 osc: discard oe_intree
An rbnode knows if it is in the tree or not, using RB_EMPTY_NODE().
There is no need for an extra flag.
Linux-Commit:
a25a82301800 ("lustre: osc_cache: discard oe_intree")
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: Iddd4ef623d96c1cf2e0789343c02ecb095f7b6a1
Reviewed-on: https://review.whamcloud.com/37600
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Amir Shehata [Fri, 29 Nov 2019 04:51:27 +0000 (20:51 -0800)]
LU-12955 lnet: fix routed final dest selection
The final hop gateway should rely on the message initiator to
select the final destination NID. IE if gateway is handing off
the message to the final destination, always use the NID provided
in the message.
If the initiator has discovery off, then it'll always pick the
same destination NID.
Otherwise if the initiator has discovery on && there exists a route to
each of the interfaces on the destination, the initiator will select
these interfaces in round robin
This will work for both single-hop and multi-hop cases. The logic
on the routers will be:
if destination NID in the message is local then
send the message to the destination NID
else if destination NID is remote && there exists a route
if router is MR then
select NI according to selection criteria
Test-parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ifd8dd65fe7695d69609ac4c1af5102429d8aa379
Reviewed-on: https://review.whamcloud.com/36916
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Fri, 13 Mar 2020 13:54:31 +0000 (09:54 -0400)]
LU-6174 osd-ldiskfs: perform proper division
The used value calculated for osd_inode_iteration() and
sf_items_updated_prior are both 64 bit values. The code
wrongly using do_div() which could truncate sf_items_updated_prior
to a 32 bit value. Use div64_u64() instead.
Change-Id: I3d76e399e6cf50460d98b1174de83263d9c2d3cc
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/36750
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Andreas Dilger [Wed, 11 Mar 2020 22:46:13 +0000 (16:46 -0600)]
LU-13156 tests: skip sanity-selinux test_21a+21b
These tests are failing most of the time, but are the last failures
for both review-dne-selinux and review-dne-selinux-ssk sessions of
this script. Add them to ALWAYS_EXCEPT so that these sessions can be
enforced to prevent further regressions. They can be fixed afterward.
Test-Parameters: trivial testlist=sanity-selinux
Test-Parameters: testlist=sanity-selinux env=SHARED_KEY=true
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib51e3749b1e262336ae1db53278909dd78d2be63
Reviewed-on: https://review.whamcloud.com/37899
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 7 Jan 2020 17:22:17 +0000 (02:22 +0900)]
LU-13116 tests: properly clean keyring in sanity-sec test_30
sanity-sec test_30 makes use of a bogus key to test Lustre behaviour
in case of invalid shared key. This bogus key must be cleaned and
removed properly so that it does not interfere with subsequent tests.
Test-Parameters: trivial envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I32d6b70948b2fedfb60f2750f56ef772a404d08c
Reviewed-on: https://review.whamcloud.com/37151
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Wed, 15 Jan 2020 16:17:17 +0000 (17:17 +0100)]
LU-13116 ptlrpc: return error for conn with NULL export
In import_select_connection(), instead of asserting when export on
chosen connection is NULL, return an error.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I60dcb26693f2ba177d612ded7efff9721febf601
Reviewed-on: https://review.whamcloud.com/37251
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Fri, 7 Feb 2020 17:16:02 +0000 (02:16 +0900)]
LU-13216 ptlrpc: sptlrpc_req_refresh_ctx's timeout semantic
sptlrpc_req_refresh_ctx() is only called with two possible timeouts
in the current code: either we do not wait at all, or we wait
indefinitely.
So change semantic of sptlrpc_req_refresh_ctx()'s timeout parameter,
to be more consistent with wait* primitives. timeout == 0 means
'do not wait at all', and timeout == MAX_SCHEDULE_TIMEOUT means
'wait indefinitely'. Other timeout values are unsupported.
Consequently, remove code that handles the case of a non zero and
non infinite timeout. And remove the unsed ctx_refresh_timeout()
callback.
It solves a problem in the current code, because in case of indefinite
wait, sptlrpc_req_refresh_ctx() is not supposed to try to refresh the
request, so it must not call the ctx_refresh_timeout() callback.
Fixes:
c1fad6a9a5 ("LU-10467 ptlrpc: convert waiting in sptlrpc_req_refresh_ctx()")
Test-Parameters: trivial envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1b20457c86863d57f97f4d998f7c5fa8ec928b31
Reviewed-on: https://review.whamcloud.com/37473
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 19 Dec 2019 14:30:11 +0000 (23:30 +0900)]
LU-12992 gss: retry in case of short computed shared key
Lustre uses OpenSSL's DH_compute_key() to compute shared secret key.
There is around 1 chance out of 256 that the returned shared key
is shorter than expected.
https://www.qacafe.com/articles/router-vpn-implementation-pitfalls/
If the returned key is short by 1 or 2 bytes, we prepend it with 0s.
Otherwise, we retry twice as it should finally be the expected length.
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small
Test-Parameters: envdefinitions=SHARED_KEY=true testlist=sanity,recovery-small
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I519194ef52539ca9a6305120fbf00b60ca985b1f
Reviewed-on: https://review.whamcloud.com/37064
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Tue, 28 Jan 2020 12:00:49 +0000 (21:00 +0900)]
LU-13116 mgc: do not lose sptlrpc config lock
Multiple targets on the same node can make use of the same lock on
sptlrpc config log. So if a target is being stopped, sptlrpc
config log should not be directly marked as stopping.
Fixes:
0ad54d5977 ("LU-11185 mgc: config lock leak")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I09c2b0c276daadb66721df88c33f734ebba86114
Reviewed-on: https://review.whamcloud.com/37344
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Fri, 6 Mar 2020 04:59:17 +0000 (15:59 +1100)]
LU-4198 clio: Remove pl_owner
The pl_owner field is only used for invariant testing, but standard
tests to enable that testing, so it doesn't really serve any purpose.
If you do enable it, you will finx the invariant in cl_page_list_del()
fails.
As async-io allows completion to happen in a different thread to
submission, this might not be a surprise.
So let's remove the field.
Sample stack trace:
LustreError: 2813:0:(cl_io.c:823:cl_page_list_del()) ASSERTION(
plist->pl_owner == get_current() ) failed:
LustreError: 2813:0:(cl_io.c:823:cl_page_list_del()) LBUG
Pid: 2813, comm: ptlrpcd_01_01 3.10.0-7.7-debug #1 SMP Fri Oct 25
00:19:37 EDT 2019
Call Trace:
[<
ffffffffa017de4c>] libcfs_call_trace+0x8c/0xc0 [libcfs]
[<
ffffffffa017defc>] lbug_with_loc+0x4c/0xa0 [libcfs]
[<
ffffffffa0318e21>] cl_page_list_del+0x1c1/0x200 [obdclass]
[<
ffffffffa031aeee>] cl_aio_end+0x5e/0x180 [obdclass]
[<
ffffffffa031a531>] cl_sync_io_note+0xa1/0x280 [obdclass]
[<
ffffffffa0315bf0>] cl_page_completion+0xd0/0x410 [obdclass]
[<
ffffffffa0822d79>] osc_ap_completion.isra.29+0x169/0x4c0 [osc]
[<
ffffffffa082a1c2>] osc_extent_finish+0x3a2/0xb10 [osc]
[<
ffffffffa080eb53>] brw_interpret+0x1f3/0xd60 [osc]
[<
ffffffffa05debc0>] ptlrpc_check_set+0x520/0x1ec0 [ptlrpc]
[<
ffffffffa060d0fb>] ptlrpcd_check+0x48b/0x580 [ptlrpc]
[<
ffffffffa060d4f3>] ptlrpcd+0x303/0x6c0 [ptlrpc]
[<
ffffffff810b8254>] kthread+0xe4/0xf0
[<
ffffffff817e5ddd>] ret_from_fork_nospec_begin+0x7/0x21
[<
ffffffffffffffff>] 0xffffffffffffffff
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Icc3af742140ddc5dd31426154d578da0ba4300fe
Reviewed-on: https://review.whamcloud.com/37824
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Thu, 27 Feb 2020 02:21:07 +0000 (13:21 +1100)]
LU-9859 libcfs: prepare for switch to container_of_safe()
Upstream Linux uses container_of_safe() rather than
container_of0().
So provide container_of_safe() for all kernels, and use spelling.txt
to suggest new code uses it. We can then start switching code
over gradually.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie938b76d569de6bb10b51fb0b12d79fcd0f43e7e
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/37736
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Wang Shilong [Tue, 25 Feb 2020 10:51:43 +0000 (18:51 +0800)]
LU-13096 llite: serialize max_cached_mb write operation
It is possible that two threads try to change max_cached_mb concurrently,
if they both try to reclaim LRU pages, it will be possible
that one thread finish, but another thread loop forever to reclaim
enough LRU pages.
Morever, this operations try to change two value, they should be
protected together to guarantee atomic, the whole operations could
be slow, this patch introduce another mutex lock to serialize operation.
Patch also fix to avoid reclaim too many LRU pages with large memory.
Change-Id: I1e88e37607f9e8db2ba67f1f7e5670f25c136465
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37710
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Elena Gryaznova [Thu, 20 Feb 2020 18:22:55 +0000 (21:22 +0300)]
LU-13273 tests: run_ior() parameters order fix
IOR requires particular options to be specified
before script file option. Otherwise those options
make no effect.
After script file processing any options which
change initialTestParams values have no effect
because test set is already created based on
initialTestParams initialized and modified with
preceding options.
To have initialTestParams->testFileName equal to
<testFile> the parameters order should be:
-o <testFile> -f <scriptFile>
With the order
-f <scriptFile> -o <testFile>
initialTestParams->testFileName is equal to default
value "testFile" set in init_IOR_Param_t.
Fixes:
2d7419478e79 (LU-11078 tests: customise run_ior())
Test-Parameters: trivial
Test-Parameters: testlist=parallel-scale envdefinitions=ONLY=iorssf
Cray-bug-id: LUS-7734
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Change-Id: I16a797466355c69e655259aaf8d9a3ae69d4a31d
Reviewed-on: https://review.whamcloud.com/37651
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Elena Gryaznova [Thu, 20 Feb 2020 13:47:12 +0000 (16:47 +0300)]
LU-13271 tests: skip 87, 115 tests if FILESET set
When FILESET is set the tests require Lustre root mounted
on $MOUNT are to be skipped.
Test-Parameters: trivial testlist=conf-sanity envdefinitions="FILESET=/subdir" envdefinitions=ONLY=87
Test-Parameters: testlist=conf-sanity envdefinitions="FILESET=/subdir" envdefinitions=ONLY=115
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I8931d0419d3613caf53497c99ad8185dc2b56fad
Reviewed-on: https://review.whamcloud.com/37647
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Mr NeilBrown [Thu, 7 Nov 2019 05:58:00 +0000 (16:58 +1100)]
LU-13255 lnet: introduce wait_var_event_warning.
LNet has a recurring pattern of waiting for some variable to reach a
particular value, and generating a warning every second that it
hasn't. In many cases the warning has a higher priority if the wait
has been for a power-of-2 seconds.
This patch embodies that pattern in a new macro
wait_var_event_warning()
and uses wake_up_var() to cause the wait to complete as soon as
possible.
This patch does not include any change to gnilnd - that is left for
a separate patch.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I557a64713cbe379a566a775944f58ddf93dbd800
Reviewed-on: https://review.whamcloud.com/37593
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexey Lyashkov [Fri, 8 Nov 2019 07:58:29 +0000 (10:58 +0300)]
LU-12214 build: add right kernel detect
add kabi package dependence and define to use it.
add ability to use an 'epoch' in kernel version
(confuse a kmodtool)
don't replace a requested kernel version with
kernel source provided
(remove an epoch)
Cray-bug-id: LUS-6033, LUS-7204
Test-parameters: trivial
Change-Id: Ia3dbca4520119ac7b6e83800b862b4f29f763796
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/36428
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Oct 2019 00:30:49 +0000 (11:30 +1100)]
LU-12780 lod: don't use ptlrpc_thread for recovery thread
rather than ptlrpc_thread, use native kthreads functionality.
- This saves a memory allocation - and associated error handling
and memory freeing - for a 'struct ptlrpc_thread'
- startup-up code is moved out of the thread to before the
thread is started. This make error handling clearer,
though it does require putting a 'struct lu_env' in
struct lod_recovery_data.
- As freeing lod_recovery_data and the env is most easily done
in the thread, add a completion to ensure the thread actually
starts. Without this it is possible for a kthread_stop() to
stop the thread before it calls the thread function at all.
- As the thread exits independently but is also waited for, we
need a handshake around kthread_stop(). Both thread_exit and
code that stops the thread use xchg() to store a NULL in the
task pointer ltd_recovery_task or lod_child_recovery_thread.
If the thread reads out a non-NULL pointer, it can exit because
nothing will wait for it.
If it gets NULL, it must wait for kthread_should_stop(). The waiter
will call kthread_shop() immediately after the xchg, so the wait
will be short.
We use wait_var_event() for this wait even though there is no
wake_up_var(). wait_var_event() is just a convenient macro that
handles thread state change for us - the wait queue is irrelevant.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I95403fc38bac99f21a4353ec7814816b813184d0
Reviewed-on: https://review.whamcloud.com/36261
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alex Zhuravlev [Thu, 5 Mar 2020 23:36:32 +0000 (15:36 -0800)]
LU-12988 ldiskfs: port ext4-mballoc-prefetch.patch to RHEL 8.1
ext4: mballoc to prefetch groups ahead of scanning
Prefetching is done in 8 * flex_bg groups, so it should be 8 read-ahead
reads for a single allocating thread. At the end of allocation the
thread waits for read-ahead completion and initializes buddy information
so that read-aheads are not lost in case of memory pressure.
At cr=0 the number of prefetching IOs is limited per allocation context
to prevent a situation when mballoc loads thousands of bitmaps looking
for a perfect group and ignoring groups with good chunks.
Change-Id: I34e3e3dd604379fb65d711b620e8b30ae5b7df6e
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Lustre-bug-id: https://jira.whamcloud.com/browse/LU_12988
Reviewed-on: https://review.whamcloud.com/37811
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sergey Cheremencev [Mon, 10 Sep 2018 19:41:37 +0000 (22:41 +0300)]
LU-12642 ofd: change last_id output in proc
Don't show IDIF 32:48 bits in sequence like below:
lctl get_param obdfilter.lustre-OST0002.last_id
0x100020001:
4295098401
The sequence should be the same with seq on MDS:
lctl get_param osp.lustre-OST0002-osc-MDT0000.prealloc_last_seq
0x100020000
lctl get_param osp.lustre-OST0002-osc-MDT0000.prealloc_last_id
4295098401
Change-Id: Ie3b7db3d69305a26c06325869628e5b38196e5d7
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Cray-bug-id: LUS-6454
Reviewed-on: https://es-gerrit.dev.cray.com/155308
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/35730
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Bruno Faccini [Thu, 7 Jan 2016 09:11:30 +0000 (10:11 +0100)]
LU-8221 osc: fix for lu_ref feature in osc_extent_put()
Trying to enable and use the USE_LU_REF/lu_ref feature, a previous
clio related changes have been found to silently (as feature
has not been tested since!) break it.
In osc_extent_put(), there is no need to remove an
implicit "handle" reference, by using LDLM_LOCK_PUT() macro, since
it has never been acquired before. This was introduced by
3f3a24d LU-3259 clio: cl_lock simplification
And the "osc_extent" reference must be deleted instead added.
This patch makes the necessary changes to fix all of these.
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I86599ff97310b16c9599b6a52a88693872ae6d30
Reviewed-on: https://review.whamcloud.com/20519
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
NeilBrown [Sun, 16 Dec 2018 23:49:53 +0000 (10:49 +1100)]
LU-9679 osc: convert while to for in get_write_extents()
This 'while' loop is really a 'for' loop in disguise.
So change it to 'for'.
Linux-Commit:
d3ba9dc5cc66 ("lustre: osc_cache: convert while to for
in get_write_extents()")
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: Ifd4295cbdad400d2c49d83e0c54ddd0e4c5ae548
Reviewed-on: https://review.whamcloud.com/37608
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
NeilBrown [Wed, 12 Dec 2018 05:09:33 +0000 (16:09 +1100)]
LU-9679 osc: simplify osc_extent_wait()
Taking a spinlock to check the current value of the state is
unnecessary. The wake_up() and wait_event() calls have sufficient
barriers to ensure that the value will be seen and the wait will abort
properly.
In most cases, osc_extent_wait() is followed by osc_object_lock()
before any shared data is touched - in those cases there is no need
for osc_extent_wait() to wait for the spinlock to be released.
The one case where osc_object_lock() does not immediately follow is in
osc_cache_truncate_start(). The extra locking was introduced in a
patch which fixed a problem with truncation, so it is likely that this
is the call that was thought to be relevant.
In that case, following osc_extent_wait(), an extent that had been
detached from the per-object list (oe_link linkage) is worked on it
without any locking.
In this case the code is waiting for OES_TRUNC, so any changes that
happen after the osc_extent_state_set(ext, OES_TRUNC) and when the
lock is dropped, might not be seen by the woken code. The only thing
changed is ->oe_trunc_pending, and the woken code doesn't look at
that.
The only remaining possible need for extra synchronization is if some
other value was changed before the wakeup and is needed after the
wait. According to memory-barriers.txt, a barrier might be needed to
ensure that is visible. Such a barrier is most clearly presented by
used smp_store_release() to set the state before wakeup, and
smp_load_acquire() to view it after waiting.
Also use a simple wake_up() instead of wake_up_all() - the latter is
only needed when exclusive waiting is being used.
Linux-Commit:
2a7e8a40cb11 ("lustre: osc: simplify osc_extent_wait()")
Signed-off-by: Mr NeilBrown <neilb@suse.com>
Change-Id: I0945902b55d598af263779787b9641a86e9476c3
Reviewed-on: https://review.whamcloud.com/37598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <farr0186@gmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Thu, 5 Mar 2020 16:34:14 +0000 (11:34 -0500)]
LU-6174 nrs: perform proper division
For NRS TBF both tr_nsecs and tr_rpc_rate are 64 bit values. The
code is wrongly using do_div() which could truncate tr_rpc_rate
to a 32 bit value. With current hardware setups the rpc rate is
less than the 4 billion limit of u32, currently I see 64K of rpcs
with a single IB port. Since this is the case we can change
tr_rpc_rate to a 32 bit value.
Change-Id: I62eabf3a8a30f5c9062aaf4c32fa878710a788c8
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/36749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Chris Horn [Tue, 18 Feb 2020 20:31:38 +0000 (14:31 -0600)]
LU-13265 lnet: Modify push MD to have single threshold
Drop LNET_MD_MANAGE_REMOTE from the push target buffer and adjust
lnet_peer_push_event() handler accordingly.
Set the lazy flag on the LNET_RESERVED_PORTAL.
Set md.threshold to 1 on the MD for push target buffer. This forces
the buffer to unlink when there is any operation on it, thus
preventing any chance of concurrent access.
Refactor lnet_push_target_resize() into two functions:
- lnet_push_target_post(): Posts the specified buffer
- lnet_push_target_resize() allocates new ping buffer and calls
lnet_push_target_post() to post it.
Add flag to lnet_ping_buffer struct to indicate if the buffer needs
to be posted. Flag is set by the push target event handler when it
has finished processing the buffer. The discovery thread checks this
flag and takes appropriate action.
Cray-bug-id: LUS-8461
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: If7f2fe26eb31bc2ad5e7023a809ef107ada9e99c
Reviewed-on: https://review.whamcloud.com/37677
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Stephen Champion <stephen.champion@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Wed, 26 Feb 2020 08:00:17 +0000 (16:00 +0800)]
LU-12811 ptlrpc: do some cleanups in swabber_dumper_helper()
This patch combines the process of dump/swab/dump calls for both
array and non-array cases in function swabber_dumper_helper(),
also does some code cleanups.
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Iccd487999ad998068b4fae6a614bad3b17737d07
Reviewed-on: https://review.whamcloud.com/36848
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Wang Shilong [Thu, 27 Feb 2020 09:21:53 +0000 (17:21 +0800)]
LU-13300 ldiskfs: port patches to improve extent status shrink
We see serious extent status shrink problem on some customer sites,
the backtrace could be:
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [kswapd0:106]
[<
ffffffffae1958c2>] merge+0x62/0xc0
[<
ffffffffc123fc50>] ? ldiskfs_init_inode_table+0x410/0x410 [ldiskfs]
[<
ffffffffae1959bb>] list_sort+0x9b/0x250
[<
ffffffffc124014e>] __ldiskfs_es_shrink+0x1ce/0x2a0 [ldiskfs]
[<
ffffffffc12402d4>] ldiskfs_es_shrink+0xb4/0x130 [ldiskfs]
[<
ffffffffadfcf805>] shrink_slab+0x175/0x340
[<
ffffffffae03e1d7>] ? vmpressure+0x87/0x90
[<
ffffffffadfd3538>] balance_pgdat+0x3a8/0x5e0
Backport following Linux upstrem commits since v3.18 to RHEL7:
Linux-commit:
eb68d0e2fc5a4e5c06324ea5f485fccbae626d05
ext4: track extent status tree shrinker delay statictics
Linux-commit:
50460fe8c6d1d95b16427936e351f277a1c72d43
ext4: remove extent status procfs files if journal load
Linux-commit:
edaa53cac8fd4b96ed4b8f96c4933158ff2dd337
ext4: change LRU to round-robin in extent status tree shrinker
Linux-commit:
b0dea4c1651f3cdb6d17604fa473e72cb74cdc6b
ext4: move handling of list of shrinkable inodes into extent status code
Linux-commit:
dd4759255188771e60cf3455982959a1ba04f4eb
ext4: limit number of scanned extents in status tree shrinker
Linux-commit:
624d0f1dd7c80d2bac4fc3066b2ff3947f890883
ext4: cleanup flag definitions for extent status tree
Linux-commit:
2be12de98a1cc21c4de4e2d6fb2bf5aa0a279947
ext4: introduce aging to extent status tree
Test-Parameters: fstype=ldiskfs serverdistro=el7.7
Change-Id: Idd97872b1663bc001a63274a430eaade66efd37d
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 11 Mar 2020 10:05:48 +0000 (11:05 +0100)]
LU-13355 crypto: adler32 wrapper in libcfs
libcfs has wrappers in order to be able to use adler32 as any other
crypto module. But libcfs should not be considered as the owner of
this implementation.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I66da322b7deae7aeb7f2fb95ede9ee1eff43c70c
Reviewed-on: https://review.whamcloud.com/37878
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
James Simmons [Tue, 3 Mar 2020 00:59:43 +0000 (19:59 -0500)]
LU-9859 ptlrpc: change imp_refcount to refcount_t
The lustre portals handle was removed from the OBD import so
it missed the move to refcount_t with the LU-12452 work. Moving
to refcount_t also gives us security protection due to overflow
issues with atomic_t. Lastly refcount_t gives use the same
equivalent as LASSERT_ATOMIC_* with CONFIG_REFCOUNT_FULL thus
allowing us to remove the LASSERT_ATOMIC_* wrappers in time.
Change-Id: Ifdbd51c39bd3921e3f5d18d60efed6d1ff58c5c6
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/37733
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Wed, 19 Feb 2020 22:56:28 +0000 (16:56 -0600)]
LU-13252 mgs: Handle possible failure from name_create_mdt
name_create_mdt may not succeed if memory allocation fails.
Cray-bug-id: LUS-8503
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I692cfd962e39b7f41d21bcfff1d7577a722ab80d
Reviewed-on: https://review.whamcloud.com/37584
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Wang Shilong [Wed, 4 Mar 2020 07:41:58 +0000 (15:41 +0800)]
LU-13227 ofd: update lvb before dropping server lock
If this is a local lock on a server namespace and this was
the last reference, lock will be canceled directly instead
of staying at LRU, this might be problem that we will lose
O_DIRECT write size expanding without LVB updating.
This patch try to fix the problem by updating LVB just after
writing but before lock is dropped.
Fixes:
6bce536725 ("LU-4198 clio: turn on lockless for some kind of IO")
Change-Id: I9d1fbac30c8599d677358e8720fe9b3ca81fe818
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37790
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
NeilBrown [Mon, 2 Mar 2020 17:37:35 +0000 (12:37 -0500)]
LU-9859 lustre: remove current_pid() and current_comm()
Just use current->pid and current->comm directly, instead
of having wrappers.
Linux-commit:
63fd7d04580b6345ff1e0aab906c034f973d493e
Test-Parameters: trivial
Change-Id: I278f32d6dd8c370a7ab211c5147ee8d246ea1893
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/37776
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Andreas Dilger [Tue, 25 Feb 2020 23:56:30 +0000 (16:56 -0700)]
LU-930 utils: fix lfs setstripe option description
Document the "-o mode" for the file create mode in the usage
message instead of the deprecated "-m mode" for setdirstripe.
Fixes:
fdad38781ccc ("LU-11376 lmv: new foreign LMV format")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I49259c06be57e234083c3d0f981ab674931b25d2
Reviewed-on: https://review.whamcloud.com/37730
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Sat, 29 Feb 2020 01:49:42 +0000 (20:49 -0500)]
LU-13274 uapi: make lustre UAPI headers C99 compliant
Attempting to compile strict C99 user land applications or
libraries with the Lustre UAPI headers will fail. These same
errors can be seen by enabling CONFIG_UAPI_HEADER_TEST as well.
Update the Lustre UAPI headers to be compilable with -std=c99.
Enhance our current test covering UAPI header handling.
For OpenSFS branch we can't include <linux/stat.h> since we support
kernels before struct statx existed and they will collide with the
special definitions in lustre_user.h.
Change-Id: Ifb0da33180dc3c7e116d6bf2b7f603ad0528277a
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/37678
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Thu, 6 Feb 2020 22:41:28 +0000 (16:41 -0600)]
LU-13209 build: SUSE 15 SP2 fix for KBUILD_SRC removed
Linux v5.2-rc4-45-g051f278e9d81 changed
KBUILD_SRC := $(abs_srctree)
to
building_out_of_srctree := 1
So adjust the build to handler either case
Test-Parameters: trivial
Cray-bug-id: LUS-8472
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I385d84cfe0b557f18d17d5732ab943b16b1e1d9c
Reviewed-on: https://review.whamcloud.com/37465
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Tue, 10 Dec 2019 00:03:08 +0000 (11:03 +1100)]
LU-9855 lustre: rename obd_device variables to obd
Most variables of type 'struct obd_device *' are named 'obd' -
655 of them.
81 are named 'dev'
29 are named 'obddev'
The remainder (< 100) have various names specific to the used (e.g
disk_obd, src, tgt etc), none used more than 11 times (mgc).
As 'obd', 'dev', and 'obddev' are all simple generic names, it would
be nice to be consistent and only use one of these.
This patch changes all 'dev' and 'obddev' variables to 'obd'.
Also change some "sprintf" to "scnprintf .. PAGE_SIZE" to avoid
warnings.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I640c4f121d7db6aec1fef848a84463d46f68f262
Reviewed-on: https://review.whamcloud.com/36969
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Chris Horn [Tue, 22 Oct 2019 02:36:57 +0000 (21:36 -0500)]
LU-12756 lnet: Use info cached in lnet_msg on resend
The src_nid and rtr_nid supplied to lnet_send() are cached in the
lnet_msg object. Use this information when resending messages via
lnet_resend_pending_msgs_locked()
Test-Parameters: trivial
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I7a9f54d6d28f26492508f86e447fd01814874f35
Reviewed-on: https://review.whamcloud.com/36541
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Shaun Tancheff [Sun, 1 Mar 2020 07:08:45 +0000 (23:08 -0800)]
LU-11310 kernel: new kernel [SLES15 SP1 4.12.14-197.29]
This patch makes changes to support new SLES15 SP1 release
with kernel 4.12.14-197.29 for Lustre client.
Test-Parameters: trivial \
envdefinitions=SANITY_EXCEPT="100 130 136 817" \
clientdistro=sles15sp1 serverdistro=el7.7 \
testlist=sanity
Cray-bug-id: LUS-6001
Signed-off-by: Chris Horn <hornc@cray.com>
Signed-off-by: Shaun Tancheff <stancheff@cray.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I07d4dce16f7a01744c7776c826ba738ace83e58f
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32693
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Tue, 25 Feb 2020 22:51:21 +0000 (09:51 +1100)]
LU-10467 libcfs: remove l_wait_event() and related macros.
Now that all usage of l_wait_event() has been converted to standard
wait_event* macros, we can remove l_wait_event and related macros.
Also clean up a few comments that mentioned these removed macros.
Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie5fdc21ac94520e52021aa870d351230d39b0bc7
Reviewed-on: https://review.whamcloud.com/37729
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Fri, 7 Feb 2020 02:05:57 +0000 (13:05 +1100)]
LU-10391 lnet: remove 'accept' variable.
lnet/acceptor.c has two variables, "accept" and "accept_type",
which have identical content and purpose.
Remove "accept" and just use "accept_type".
As the module parameter "accept" will now represent the value stored
in the variable "accept_type", we need to use module_param_named() to
allow the module parameter to have a different name to the kernel
variable.
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I938418c971dfdd9f6e5c8c22c03e992432082334
Reviewed-on: https://review.whamcloud.com/37694
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Emoly Liu [Wed, 27 Nov 2019 14:35:56 +0000 (22:35 +0800)]
LU-12811 ptlrpc: pass buflen to lustre_swab_object_update_*()
Pass the buffer size to the swabbing functions
lustre_swab_object_update_reply() and
lustre_swab_object_update_request() to avoid invalid access,
especially for small buffer.
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I229c1946ec3c088c7804762655f143dc419b2ea0
Reviewed-on: https://review.whamcloud.com/36867
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Fri, 6 Mar 2020 23:58:46 +0000 (16:58 -0700)]
LU-12207 tests: allow some margin for sanity/76
With newer slab allocators it is possible that the kernel may keep
some inodes in the per-cpu cache and not release all of them. This
was resulting sanity test_76 failures with a few (often 12) inodes
not being freed of the inodes that are created by the test, like:
inode slab grew from 3224 to 3236
inode slab grew from 70368 to 70380
inode slab grew from 68878 to 68890
Allow some small number of inodes (8 per core) cached on the client
without considering it a test failure failure.
Clean up test_76 code style to current standards.
Test-Parameters: trivial testlist=sanity env=ONLY=76,ONLY_REPEAT=100
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia3f60de2fb471bb32da27d36665a3a0fd43ebbe5
Reviewed-on: https://review.whamcloud.com/37827
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Mon, 9 Mar 2020 09:37:22 +0000 (03:37 -0600)]
LU-10447 tests: remove use of SETSTRIPE from sanity
The recently landed test_136() was using $SETSTRIPE instead of
$LFS setstripe, but that was removed from test-framework.sh in a
later patch (written earlier and didn't take it into account).
This test doesn't fail during normal testing because it is skipped
due to SLOW testing. Change it to use $LFS setstripe instead.
Fixes:
cc1092291932 ("LU-13069 obdclass: don't skip records for wrapped catalog")
Fixes:
be3b7e772d3a ("LU-10447 tests: deprecate use of $SETSTRIPE/$GETSTRIPE")
Test-Parameters: trivial testlist=sanity env=ONLY=136,ONLY_REPEAT=10
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibb1e8220e2de711e7034d91baa329bdc04687c72
Reviewed-on: https://review.whamcloud.com/37831
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 9 Mar 2020 08:47:13 +0000 (17:47 +0900)]
LU-13343 tests: skip recovery-small test_140 with SSK
recovery-small test_140a and test_140b are using a 'local client',
ie a client mounted on a server node.
This is not compatible with SSK keys installed by the test framework,
so just skip these tests when running with SSK.
Test-Parameters: trivial
Test-Parameters: env=SHARED_KEY=true,ONLY=140 testlist=recovery-small
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I420deed9dbc50da622766648a51f885ed01203c6
Reviewed-on: https://review.whamcloud.com/37832
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Andrew Perepechko [Fri, 28 Feb 2020 13:18:40 +0000 (16:18 +0300)]
LU-13309 osd-ldiskfs: remove per-page object_get/put in brw
According to profiling data, object_get/put calls consume a lot
of CPU ticks:
99.99% [kernel.vmlinux]
|
|--28.82%--_atomic_dec_and_lock
| |
| --28.82%--lu_object_put
| |
| --28.73%--osd_bufs_put
| ofd_commitrw
| tgt_brw_read
| tgt_request_handle
| ptlrpc_server_handle_request
| ptlrpc_main
| kthread
| ret_from_fork
|
|--26.51%--osd_bufs_get
| |
| --26.51%--ofd_preprw
| tgt_brw_read
| tgt_request_handle
| ptlrpc_server_handle_request
| ptlrpc_main
| kthread
| ret_from_fork
|
|--18.09%--lu_object_put
| |
| --18.01%--osd_bufs_put
| ofd_commitrw
| tgt_brw_read
| tgt_request_handle
| ptlrpc_server_handle_request
| ptlrpc_main
| kthread
| ret_from_fork
ofd_preprw_read() and ofd_preprw_write() pin the corresponding
ofd object, later ofd_commitrw_read() and ofd_commitrw_write()
unpin it. When the ofd object is pinned, its underlying
osd object cannot go away, so object_get/object_put
in osd_bufs_get()/osd_bufs_put() are basically no-op.
Change-Id: Ic48f793de5ef3e62505f44879c91050922160000
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-4388
Reviewed-on: https://review.whamcloud.com/37758
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Tue, 25 Feb 2020 20:59:56 +0000 (07:59 +1100)]
LU-13258 libcfs: fixes for cfs_arch_init()
The introduction of cfs_arch_init() brought two problems.
1/ wait_bit_init() wasn't known due to a missing include file.
2/ cfs_arch_init() was not marked __init, but it called
a function (wait_bit_init) that was.
This patch fixes both of these.
Fixes:
3453c95f513c ("LU-13258 libcfs: make apply_workqueue_attrs() available for Lustre")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6f19e11e68f52ca8071332364d369ed3a717d5c9
Reviewed-on: https://review.whamcloud.com/37727
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Tue, 25 Feb 2020 20:53:38 +0000 (07:53 +1100)]
LU-12477 lustre: restore time_after32()
time_after32 is needed when compiling with kernels earlier than v4.14,
which haven't had it backported. E.g. SLE12-SP3-LTSS needs it.
Fixes:
8e88bbfef579 ("LU-12477 lustre: remove obsolete config checks")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ifb51b9ed0945ad28f8e8aa34dada3860107b95df
Reviewed-on: https://review.whamcloud.com/37725
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log