Whamcloud - gitweb
Andreas Dilger [Fri, 24 Mar 2023 02:31:59 +0000 (20:31 -0600)]
LU-16661 build: improve lustre.spec.in Requires
Add Suggests: bash-completion for lustre-client and lustre for
lctl and lfs sub-command completion.
Move perl from Requires to Recommends, since there are only some
uncommonly used tools (llstat, llobdstat) that are using perl.
Remove a couple of ancient obsolete test scripts that used perl.
lustre-iokit incorrectly Required perl instead of python3.
Set minimum kernel version for client to be 3.10 or later.
Change "netstat" to "ss" in tests to avoid dependency issues.
Fix sanity.sh and conf-sanity.sh tests for sles12sp5 issues.
Lustre-change: https://review.whamcloud.com/50397
Lustre-commit:
621e0bc2f9d4e7d9aed0ec14e939eae4451a8f1e
Test-Parameters: trivial testlist=runtests clientdistro=el9.1
Test-Parameters: trivial testlist=runtests clientdistro=sles15sp3
Fixes:
7521473bdd ("LU-16382 spec: add more dependencies for lustre-tests")
Fixes:
fd734cffb3 ("b=18443 tests: remove obsolete tests scripts")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I48c6819596c81cb044e983bc64f1edf1ee3ebbe5
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51751
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 24 Mar 2023 00:13:20 +0000 (18:13 -0600)]
LU-16661 build: add Recommends and Suggests for Debian
Add Suggests: bash-completion for lustre-client-tools and
lustre-server-tools for lctl and lfs completion.
Move perl from Depends to Recommends, since there are only some
uncommonly used tools (llstat, llobdstat) that are using perl.
Add python3 to Recommends for lustre-server-tools for lljobstat.
Remove python3 from lustre-iokit since it isn't used anywhere.
Change Maintainer for Debian packages to the lustre-devel mailing
list, instead of someone who hasn't worked on Lustre for 6 years.
Increase minimum kernel version for client from 2.6.32 to 3.10.
Improve package descriptions slightly.
Lustre-change: https://review.whamcloud.com/50396
Lustre-commit:
73ed44854a740b78523ed1f050960038c8adf552
Test-Parameters: trivial testlist=runtests clientdistro=ubuntu2204
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I43248cc78ae6a47ad77817c27ba11de25b3ebbe5
Reviewed-by: Thomas Stibor <thomas@stibor.net>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lei Feng [Wed, 26 Jul 2023 09:08:21 +0000 (17:08 +0800)]
LU-16984 tests: replay-dual/31 checks file from DIR2
Move replay-dual/test_31 to correct place.
In replay-dual/test_31, check file existence from DIR2.
Add more messages for diagnosis.
Lustre-change: https://review.whamcloud.com/51762
Lustre-commit: TBD (from
025aa6e78ca6d3f4fc872b2b616034ec5b28952a)
Fixes:
07764c4eeb ("LU-16953 tests: wait longer in replay-dual/test_31")
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=replay-dual env=ONLY=31,ONLY_REPEAT=100
Change-Id: Iee679ee94ac2cb51baad1651bfaddf452fafdbd1
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51764
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Chris Horn [Tue, 1 Nov 2022 20:33:18 +0000 (14:33 -0600)]
LU-16290 lnet: Signal completion on ping send failure
Call complete() on the ping_data::completion if we get
LNET_EVENT_SEND with non-zero status. Otherwise the thread which
issued the ping is stuck waiting for the full ping timeout.
A pd_unlinked member is added to struct ping_data to indicate whether
the associated MD has been unlinked. This is checked by lnet_ping() to
determine whether it needs to explicitly called LNetMDUnlink().
Lastly, in cases where we do not receive a reply, we now return the
value of pd.rc, if it is non-zero, rather than -EIO. This can provide
more information about the underlying ping failure.
Lustre-change: https://review.whamcloud.com/49020
Lustre-commit:
48c34c71de65e8a251a218bc9ecb7c5ed522d786
HPE-bug-id: LUS-11317
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I1bc573cf7397e319993fa8aabb31c5f3b59768e7
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51700
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Chris Horn [Mon, 13 Feb 2023 21:15:31 +0000 (15:15 -0600)]
LU-16551 tests: Ensure all peer credits used in MR
sanity-lnet test_254 needs to ensure that all peer credits are
consumed. Because of the raciness of the round robin code in LNet,
we cannot rely on just issuing the appropriate number of pings.
Instead we should use the --source argument to lnetctl ping to ensure
that we send the correct number of pings over each interface.
To simplify matters, only perform this test, and the other tests that
call setup_health_test(), in non-routed configurations.
Lustre-change: https://review.whamcloud.com/49979
Lustre-commit:
aa66cb46761d72ec2c861ff9665ea6b6dbbe76a8
Test-Parameters: trivial testlist=sanity-lnet
Fixes:
52db11cdce ("LU-16303 lnet: Drop LNet message if deadline exceeded")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I05a7ffec37d16c14711fe696232708f927357b1c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51699
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Chris Horn [Mon, 7 Nov 2022 22:06:32 +0000 (15:06 -0700)]
LU-16303 lnet: Drop LNet message if deadline exceeded
The LNet message deadline is set when a message is committed for
sending. A message can be queued while waiting for send credit(s)
after it has been committed. Thus, it is possible for a message
deadline to be exceeded while on the queue. We should check for this
when posting messages to LND layer.
Lustre-change: https://review.whamcloud.com/49078
Lustre-commit:
52db11cdceef0851b972410cf8f7014d20fd194e
HPE-bug-id: LUS-11333
Test-Parameters: trivial testlist=sanity-lnet env=ONLY=253,ONLY_REPEAT=100
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I1315b2351536e63b9d4f22d9336a57415031e0c7
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51698
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Chris Horn [Wed, 14 Sep 2022 01:23:37 +0000 (20:23 -0500)]
LU-15595 tests: Router test interop check and aarch fix
setup_router_test() executes load_lnet() on remote nodes, but
this function was only added in 2.15. Add a version check for it.
Enabling routing may fail on nodes with small amount of memory (like
aarch config). Define small number of router buffers to work around
this issue. Modify the functions which calculate the number of buffers
to allow small sizes to be specified via parameters.
Lustre-change: https://review.whamcloud.com/48578
Lustre-commit:
1aba6b0d9b661d3699cbd4624e9db334a13fc647
Test-Parameters: trivial testlist=sanity-lnet serverversion=2.12.9
Test-Parameters: testgroup=review-ldiskfs-arm testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: If0b76747fe09e883546f18da9f3322c72263e29d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51664
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Chris Horn [Wed, 27 Oct 2021 20:10:17 +0000 (20:10 +0000)]
LU-15595 lnet: Always use ping reply to set route lr_alive
We currently process discovery ping replies in different ways
depending on whether the gateway has discovery enabled or disabled
(or the local peer doing the processing has discovery enabled or
disabled).
When DD is disabled we process the ping reply to set the lr_alive
field of lnet_route because the peer objects for non-MR routers do
not contain all the information needed to calculate the route
aliveness when a message is being sent.
When DD is enabled then we don't do any special processing of the
ping reply. We simply let discovery update the NI status for the
GW's peer NIs and then we calculate the route aliveness on every
send.
We issue discovery pings to routers every alive_router_check_interval
seconds (default 60), but we calculate route aliveness on every send
to a remote network (1000s of times per seconds). Thus, it is better
to slightly duplicate the effort expended when we receive a discovery
reply so that we can avoid calculating route aliveness on every send.
Since both lr_alive and hop type are being set on each ping reply, for
both DD enabled and disabled cases, we can remove the code for
updating lr_alive and hop type from lnet_router_discovery_complete().
If discover encounters a fatal error, we still set the status of each
peer NI, as well as all routes, to down in
lnet_router_discovery_complete().
Lustre-change: https://review.whamcloud.com/46624
Lustre-commit:
1ea6c87d415144522a1c50c08063c9d30d585325
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: If4838c269a89885ba3763f62847e294804edf62e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51663
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Chris Horn [Mon, 7 Feb 2022 23:20:37 +0000 (23:20 +0000)]
LU-15595 tests: Add various router tests
Add test cases to exercise LNet routing.
Lustre-change: https://review.whamcloud.com/46622
Lustre-commit:
8ee85e15412d32fbe60f70c474c0a28ff15b8351
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I4a077937b3e3b8b07707afeb0c5c23ec1c9074f4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51661
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
James Simmons [Wed, 30 Mar 2022 13:10:28 +0000 (09:10 -0400)]
LU-15363 tests: don't use lustre module to test lnet
Currently sanity-lnet.sh loads the lustre modules to properly
initialize the lnet modules. This doesn't work with the native
Linux client since it only starts up LNet after mounting the
file system. We shouldn't be using lustre to test lnet so
load lnet modules with config_on_load option to properly setup
the default LNet configuration.
Also fix ksocklnd-config to use bash so sanity-lnet.sh can
pass on Ubuntu.
Lustre-change: https://review.whamcloud.com/45834
Lustre-commit:
e41f91dc90a0977f7ea85b199b7e5809c56b810e
Test-Parameters: trivial testlist=sanity-lnet
Change-Id: Ifffc51625f5c2ffbb3ab811b75739c0e6407a821
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51666
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Chris Horn [Tue, 11 Jan 2022 22:19:16 +0000 (16:19 -0600)]
LU-15440 lnet: lnet_peer_data_present() memory leak
If the ping buffer has nnis <= 1 then the ref on the ping buffer does
not get dropped. This causes a memory leak.
Lustre-change: https://review.whamcloud.com/46052
Lustre-commit:
56384a4fc39ff99c8abb3538f93d303f2be6ab45
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I5e3c651ffecbe4f8860afb86770cecef23ebe862
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51684
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
hxing [Fri, 28 Jul 2023 07:36:12 +0000 (15:36 +0800)]
LU-16911 sec: quiet messages from identity upcall retry mech
Do not use CERROR to print messages about failed identity acquire
upcalls. And make a difference between initial attempt before retry,
and final failure.
Lustre-change: https://review.whamcloud.com/51355
Lustre-commit:
765e59d37dc2b4c85e9bdaea2765c9023226778b
Fixes:
61c3b3a9bb ("LU-16165 sec: retry mechanism for identity cache")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: I35e04ca31b623d6037bb49e4ded4ea96d653f074
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51797
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Zhuravlev [Tue, 25 Jul 2023 14:55:53 +0000 (17:55 +0300)]
EX-7965 utils: resync-cmd option to lamigo
so we can change default "lfs mirror resync" to something else,
add extra options, etc.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Idf445128e811297651e09babc51fb5b0f2b9b20d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51757
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Wed, 21 Jun 2023 06:33:21 +0000 (14:33 +0800)]
LU-16831 lfs: limit stripe count for component size
If stripe count is larger than component_size/stripe_size, some
allocated OST objects are created but inaccessible. This patch
reduces the number of stripes in that case to avoid this.
Lustre-change: https://review.whamcloud.com/51143
Lustre-commit:
a250ecb959a98c2ec0a01bbca9d943a19b8fa078
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I117ed8a7696c6c6adcdd0c2c6531a958cc53bd51
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51719
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Qian Yingjin [Fri, 28 Jul 2023 08:40:16 +0000 (16:40 +0800)]
LU-17001 pcc: fix the build failures for mmap_conv
PCC has implemented mmap page conversion for performance reason.
However, it is not stable, and disabled by default.
The Lustre build meets some failures against the newer kernel such
as Ubuntu 2204 kernel 6.2.0-26-generic.
The reason is that @pagevec_lookup and @add_to_page_cache_locked
are removed and no longer exported by the kernel.
This patch gives a simple solution. It disables the mmap_conv
option for PCC and does not perform mmap page conversion for
the newer kernel without @add_to_page_cache_locked exported.
Change-Id: Ie314830f225848b7f888709ad000fd7d36d3c0b2
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51798
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Tue, 25 Jul 2023 22:09:31 +0000 (01:09 +0300)]
LU-16983 mdc: check errcode prior mdc_fill_lvb() call
The mdc_enqueue_fini() can be called with negative
errcode parameter if request processing was failed.
In that case the mdc_fill_lvb() shouldn't be called.
Issue may occur with DoM files, old server (<2.14) and
new client. The problem is in new client code.
Lustre-change: https://review.whamcloud.com/51761
Lustre-commit: TBD (from
c422797b3949e0b3b7fac7164f03f01dfa6aca0b)
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I884398beada4286bc07875247e15b41120f73a3e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51769
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Sun, 7 May 2023 02:11:27 +0000 (21:11 -0500)]
LU-16667 build: kernel_cap_t contains u64
linux kernel v6.2-13111-gf122a08b197d
capability: just use a 'u64' instead of a 'u32[2]' array
Add configure test for kernel_cap_t as u64 and provide
and accessor for the least significant 32 bits.
As of linux commit v3.6-10973-g607ca46e97a1 lustre implicitly
started to ignore some capabilities, see:
include/uapi/linux/capability.h
The last capability flag was added by:
linux commit v5.8-rc5-1-g124ea650d307
The capabilities the Lustre currently ignores are:
- CAP_MAC_OVERRIDE
- CAP_MAC_ADMIN
- CAP_SYSLOG
- CAP_WAKE_ALARM
- CAP_BLOCK_SUSPEND
- CAP_AUDIT_READ
- CAP_PERFMON
- CAP_BPF
- CAP_CHECKPOINT_RESTORE
None of which appear to be important to Lustre operations
and should be fine to continue ignore.
Lustre-change: https://review.whamcloud.com/50421
Lustre-commit:
ea9532fb731bbfe041010e2224219479c2c0d71b
Test-Parameters: trivial
HPE-bug-id: LUS-11557
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I48ad7b1a34fff378c260dc73ea91b22aaa0d7469
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51726
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr. NeilBrown [Wed, 14 Jul 2021 16:15:26 +0000 (12:15 -0400)]
LU-9859 libcfs: discard cfs_cap_t, use kernel_cap_t
lustre only sends 32bits of capabilities in on-the-wire RPC calls.
It current strips off higher bits and uses a 32bit cfs_cap_t
throughout.
Though there is a small memory cost, it is cleaner to use
kernel_cap_t throughout and only truncate when marshalling
data for RPC calls.
So this patch replaces cfs_cap_t with kernel_cap_t throughout,
and where a cfs_cap_t was previous stored in a __u32, we now
store cap.cap[0] instead.
With this, we can remove include/linux/libcfs/curproc.h
Linux-commit:
18f92a6e3d6bd00941ddfb5837835348f72d39dc
Lustre-change: https://review.whamcloud.com/43171
Lustre-commit:
0a6beb2a50f7a4f953a258b63c2ea1fbfd373092
Change-Id: If7dd7a16c218dfc0d520e189f021ed6bda3b93fd
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51743
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
NeilBrown [Mon, 1 Mar 2021 14:15:45 +0000 (09:15 -0500)]
LU-9859 libcfs: simplify capability dropping.
Lustre has a 'squash credentials' concept similar to the "anon_uid"
for nfsd. When accessing a file with squashed credentials, we
need to also drop capabilities.
Linux has cap_drop_fs_set() and cap_drop_nfsd_set(). Rather than
taking a completely different approach, this patch changes lustre
to use this same cap_drop_*_set() approach.
With this change we also drop CAP_MKNOD and CAP_MAC_OVERRIDE
which are probably appropriate, and don't drop
CAP_SYS_ADMIN or CAP_SYS_BOOT which should be irrelevant for
file permission checking
Calling both cap_drop_*_set() seems a bit clumsy, but gets
the job done.
Linux-commit:
f497115d4cf8a430c5d9902ce35716ba5f9c21ef
Lustre-change: https://review.whamcloud.com/41957
Lustre-commit:
ee06281d89871c46310ee4b0585ee4489d2c2f3f
Change-Id: I2f4f691bc4ad090f6abaa4e13eb473bf8d904b23
Signed-off-by: NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51745
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr. NeilBrown [Thu, 18 Mar 2021 21:57:03 +0000 (17:57 -0400)]
LU-9859 libcfs: remove linux-curproc.c
The only real functionality remaining here is
cfs_curproc_cap_pack(),
and it can be trivially implemented as an inline
in curproc.h.
So do that and remove the file.
The rest can be moved to jobid.c
Linux-commit:
37d3b407dc14a13ec8bba3a4d7737c92f996e9c0
Lustre-change: https://review.whamcloud.com/41938
Lustre-commit:
f6cd6a880cd03fd1d3fd3d8e03133f0835ad275d
Change-Id: I3546841fa44accb19d0867099c17b16ede48228e
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51742
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Peng Tao [Mon, 11 Jan 2021 15:49:38 +0000 (10:49 -0500)]
LU-9859 libcfs: remove cfs_capable
Use capable() directly.
Linux-commit:
2eb90a757e9d953c9e2a8fce530422189992fb1b
Lustre-change: https://review.whamcloud.com/41783
Lustre-commit:
e34fdc5479037bf7b6efed3239362585a1c1ff3b
Change-Id: Iadaa3c743a350def37558b23d954f5dfd4e0844a
Signed-off-by: Peng Tao <bergwolf@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51741
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr. NeilBrown [Thu, 10 Sep 2020 13:49:30 +0000 (09:49 -0400)]
LU-9859 libcfs: replace all CFS_CAP_* macros with CAP_*
Lustre defines a few CFS_CAP_* macros which are exactly the
same as the corresponding CAP_* macro, with one exception.
CFS_CAP_SYS_BOOT is 23
CAP_SYS_BOOT is 22.
CFS_CAP_SYS_BOOT is only used through CFS_CAP_FS_MASK and
causes capability 23 (CAP_SYS_NICE) to be dropped in certain
circumstances.
It is probable that the intention was to drop CAP_SYS_BOOT,
and this is what is now done.
CFS_CAP_CHOWN_MASK and CFS_CAP_SYS_RESOURCE_MASK are never
used, so they have been removed.
Linux-commit:
5ebaa2d14850205e44757c4d5fdd4097712d01ef
Lustre-change: https://review.whamcloud.com/39875
Lustre-commit:
a076975f9fed44e2b3a8b516aa7fe2ee6fbdb2bb
Change-Id: Ifb90c0a36e204c76b90ff23ac609345d11b878da
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51740
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sun, 23 Jul 2023 01:42:07 +0000 (19:42 -0600)]
RM-620 build: New tag 2.14.0-ddn94
New tag 2.14.0-ddn94
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie82ae2d2198bdfd780acf1c2050c64287bc0b7ec
Bobi Jam [Thu, 4 May 2023 02:39:29 +0000 (10:39 +0800)]
LU-16637 llite: call truncate_inode_pages() in inode lock
In some cases vvp_prune()->truncate_inode_pages() is get called
without IO context, we need protect it with inode lock as well.
So we add ll_inode_info::lli_inode_lock_owner and set it according to
vfs lock rules (Documentation/filesystems/Locking or
Documentation/filesystems/locking.rst), so before calling
truncate_inode_pages(), we'd lock the inode if it's not locked in
vfs.
Lustre-commit:
51d62f2122fee14fbb3ff8333b5a830e1181e4e5
Lustre-change: https://review.whamcloud.com/50857
Fixes:
ef9be34478 ("LU-16637 llite: call truncate_inode_pages() under inode lock")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I84d7d999a49325810062a9a7337e184d35467820
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51643
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Shaun Tancheff [Sun, 2 Apr 2023 09:15:58 +0000 (04:15 -0500)]
LU-16594 build: get_random_u32_below, get_acl with dentry
Linux commit v6.1-13825-g3c202d14a9d7
prandom: remove prandom_u32_max()
Use get_random_u32_below() and provide a replacement
when get_random_u32_below is not available.
Linux commit v6.1-rc1-2-g138060ba92b3
fs: pass dentry to set acl method
Linux commit v6.1-rc1-4-g7420332a6ff4
fs: add new get acl method
get_acl() and set_acl() have new signatures
Lustre-change: https://review.whamcloud.com/50193
Lustre-commit:
3ef773db80fc346455c9939ad108f3f56990ee9c
Test-Parameters: trivial
HPE-bug-id: LUS-11556
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I1de02f86fd2719fc75de4f014f51d73736d83c33
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51734
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mr NeilBrown [Fri, 13 May 2022 02:16:12 +0000 (12:16 +1000)]
LU-15759 libcfs: debugfs file_operation should have an owner
If debugfs a file is open when unloading the libcfs/lnet module, it
produces a kernel Oops (debugfs file_operations callbacks no longer
exist).
Crash generated with routerstat (/sys/kernel/debug/lnet/stats):
[ 1449.750396] IP: [<
ffffffffab24e093>] SyS_lseek+0x83/0x100
[ 1449.750412] PGD
9fa14067 PUD
9fa16067 PMD
d4e5d067 PTE 0
[ 1449.750428] Oops: 0000 [#1] SMP
[ 1449.750883] [<
ffffffffab7aaf92>] system_call_fastpath+0x25/0x2a
[ 1449.750897] [<
ffffffffab7aaed5>] ?
system_call_after_swapgs+0xa2/0x13a
This patch adds an owner to debugfs file_operation for libcfs and
lnet_router entries (/sys/kernel/debug/lnet/*).
The following behavior is expected:
$ modprobe lustre
$ routerstat 10 > /dev/null &
$ lustre_rmmod
rmmod: ERROR: Module lnet is in use
Can't read statfile (ENODEV)
[1]+ Exit 1 routerstat 10 > /dev/null
$ lustre_rmmod
Note that the allocated 'struct file_operations' cannot be freed until
the module_exit() function is called, as files could still be open
until then.
Lustre-change: https://review.whamcloud.com/47335
Lustre-commit:
b2dfb4457f0f1e56f3df448cf67ac97e728f4417
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia0920313e0c2a4b6cdc875fed08221e174a12a73
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51652
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Zhuravlev [Fri, 10 Mar 2023 17:47:05 +0000 (20:47 +0300)]
LU-16633 obdclass: fix rpc slot leakage
obd_get_mod_rpc_slot() can race with obd_put_mod_rpc_slot():
finishing wait_woken() resets WQ_FLAG_WOKEN (which is set
when the corresponding thread gets a slot incrementing
cl_mod_rpcs_in_flight. then another thread execting
__wake_up_locked_key() may find that wq_entry again and call
claim_mod_rpc_function() one more time again incrementing
cl_mod_rpc_in_flight. thus it's incremented twice for a
single obd_get_mod_rpc_slot().
#1: obd_get_mod_rpc_slot() #2: obd_put_mod_rpc_slot()
flags &= ~WQ_FLAG_WOKEN
list_add()
wait_woken()
schedule claim_mod_rpc_function()
cl_mod_rpcs_in_flight++
wake_up()
flags &= ~WQ_FLAG_WOKEN
#3: obd_put_mod_rpc_slot()
claim_mod_rpc_function()
cl_mod_rpcs_in_flight++
wake_up()
list_del()
the patch introduces a replacement for WQ_FLAG_WOKEN which is never
reset once set.
Lustre-change: https://review.whamcloud.com/50261
Lustre-commit:
91a3726f313df33e099320d171039f8371fec27f
Fixes:
5243630b09 ("LU-15947 obdclass: improve precision of wakeups for mod_rpcs")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I29371c8c85414413c5a8e41dec3632f64ad127bb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Mr NeilBrown [Mon, 21 Jun 2021 03:25:42 +0000 (13:25 +1000)]
LU-15947 obdclass: improve precision of wakeups for mod_rpcs
There is a limit of the number of in-flight mod rpcs with a
complication that a 'close' rpc is always permitted if there are no
other close rpcs in flight, even if that would exceed the limit.
When a non-close-request complete, we just wake the first waiting
request and assume it will use the slot we released. When a
close-request completes, the first waiting request may not find a slot
if the close was using the 'extra' slot. So in that case we wake all
waiting requests and let them fit it out. This is wasteful and
unfair.
To correct this we revise the wait/wake approach to use a dedicated
wakeup function which atomically checks if a given task can proceed,
and updates the counters when permission to proceed is given. This
means that once a task has been woken, it has already been accounted
and it can proceed.
To minimise locking, cl_mod_rpcs_lock is discarded and
cl_mod_rpcs_waitq.lock is used to protect the counters. For the
fast-path where the max has not been reached, this means we take and
release that spinlock just once. We call wake_up_locked while still
holding the lock, and if that woke the process, then we don't drop the
spinlock to wait, but proceed directly to the remainder of the task.
When the last 'close' rpc completes, the wake function will iterate
the whole wait queue until it finds a task waiting to submit a close
request. When any other rpc completes, the queue will only be
searched until the maximum is reached.
Lustre-change: https://review.whamcloud.com/44041
Lustre-commit:
5243630b09d22e0b576d81390d604774881f63f7
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Iff094c3188a3bd8a04edc1d5d98ec3014e2b059b
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51657
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Chris Horn [Tue, 4 Jan 2022 20:42:26 +0000 (14:42 -0600)]
LU-15398 tests: Use remote peers for health tests
LNet health may take different action depending on whether a NID
belongs to the local host or a remote peer. As such, the test cases
need to be careful to use remote or local NIs appropriately.
Introduce helper functions to create and cleanup LNet peers that are
needed for these tests. Convert existing test cases to use the new
helpers.
New function, lnet_if_list(), is added to test-framework.sh to
facilitate configuration of remote interfaces. do_rpc_nodes() modified
to recognize '--quiet' flag to ease parsing of lnet_if_list() output.
Tests 204 and 206 were re-worked to check the health state after each
simulated error. lnet_health_post() modified to reset peer and local
NI health so they are at max value when each error condition is
simulated.
Test 214, 215, and 250 were using hardcoded "eth0" names. These were
switched to use the INTERFACES variable.
The lnet_recovery_limit parameter is deprecated so remove lines that
were setting that parameter.
Lustre-change: https://review.whamcloud.com/45975
Lustre-commit:
3166a201e0a5cbc173ca110f64dc21f32ec10c8c
Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-10661
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I685fda8a84bcce024a765ddfc81c085acf24607a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51682
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Fri, 18 Jun 2021 21:35:29 +0000 (15:35 -0600)]
LU-14773 tests: quiet down some verbose messages
Don't print anything into the test logs for normal background
operations that are run as part of run_one(), so that they
don't clutter the test output with repeated/useless messages.
Lustre-change: https://review.whamcloud.com/44034
Lustre-commit:
86f16910645d9d9cad17c0f53ca1a375121e3f4c
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib6a49fc268e4cd0ad92c71a391865ce2d73ebbe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51686
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Jian Yu [Sat, 15 Jul 2023 14:36:13 +0000 (22:36 +0800)]
LU-16916 tests: fix client_evicted() not to ignore EOPNOTSUPP
After RHEL 9.x or Ubuntu 22.04 client is evicted, "lfs df" returns
error code 95 (EOPNOTSUPP), which is ignored in check_lfs_df_ret_val()
and then causes client_evicted() to ingore that error.
This patch fixes client_evicted() to check the return value
from "lfs df" directly so as not to ignore EOPNOTSUPP.
Lustre-change: https://review.whamcloud.com/51667
Lustre-commit:
a5a9ded43b72238c2df8e0a74f03151ea3d4ce99
Test-Parameters: trivial clientdistro=el9.2 testlist=replay-vbr
Test-Parameters: trivial clientdistro=el8.8 testlist=replay-vbr
Test-Parameters: trivial clientdistro=ubuntu2204 testlist=replay-vbr
Change-Id: I633ae8769fc563b8068f433e2afae29463ac5553
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51691
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Arshad Hussain [Sat, 15 Jul 2023 14:25:54 +0000 (22:25 +0800)]
LU-8962 lfs: Handle non-lustre and multiple args
This patch addresses:
01: Handle multiple filesystems provided to 'lfs df'
02: Correctly report 'EOPNOTSUPP' for filesystems which
are non-Lustre.
03: Make changes to test-framework.sh to handle modified
return value from 'lfs df'. This changes For compatibility
reason, ignores and masquerades EOPNOTSUPP as success.
The final return value is 0 for _all_ success or
value of the first failure for even a single failure
seen during the argument processing
sanity/56e Test-case added.
Lustre-change: https://review.whamcloud.com/42126
Lustre-commit:
2d714041ba718853be700960b76769a8fb44cf51
LU-15465 tests: conf-sanity failed with code 95
conf-sanity tests 27b, 47 and 84 (the tests execute 'fail mds1' and
then 'cleanup' at the end of test) failed with code EOPNOTSUPP because
of 'set -e' and lfs df <non_lustre> return code 95.
The scenario:
test_27b () {
facet_failover $SINGLEMDS
change_active mds1
...
cleanup -> umount_client $MOUNT
}
formatall
stopall
activemds=`facet_active mds1`
if [ $activemds != "mds1" ]; then
fail mds1
clients_up
lfs_df_check
+ local clients=fre0111,fre0112
+ local rc
+ [ -z fre0111,fre0112 ]
+ pdsh -S -w fre0111,fre0112
/usr/bin/lfs df /mnt/lustre << lustre not mounted
pdsh@fre0111: fre0111: ssh exited with exit code 95
pdsh@fre0111: fre0112: ssh exited with exit code 95
To reproduce the issue just run:
ONLY="27b" sh conf-sanity.sh or:
ONLY="47" sh conf-sanity.sh or:
ONLY="84" sh conf-sanity.sh
Lustre-change: https://review.whamcloud.com/46236
Lustre-commit:
242fc2ccbacaf171159a20d59c9633707d8fbf66
Fixes:
2d714041ba ("LU-8962 lfs: Handle non-lustre and multiple args")
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-10680
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I73287d21792d89b8cde672acdaf9c9caf829522f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51690
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Qian Yingjin [Tue, 21 Mar 2023 08:53:00 +0000 (04:53 -0400)]
LU-16651 llite: hold invalidate_lock when invalidate cache pages
The newer kernel (such as Ubuntu 2204) introduces a new member:
invalidate_lock in the structure @address_space.
The filesystem must exclusively acquire invalidate_lock before
invalidating page cache in truncate / hole punch (and thus calling
into ->invalidatepage) to block races between page cache
invalidation and page cache filling functions (fault, read, ...)
However, current Lustre client does not hold this lock when remove
pages from page cache caused by the revocation of the extent DLM
lock protecting them.
If a client has two overlapped PR DLM extent locks, i.e:
- L1 = <PR, [1M, 4M - 1]
- L2 = <PR, [3M, 5M - 1]
A reader process holds L1 and reads data in range [3M, 4M - 1].
L2 is being revoken due to the conflict access.
Then the page read-in by the reader may be invalidated and deleted
from page cache by the revocation of L2 (in lock blocking AST).
The older kernel will check each page after read whether it was
invalidated and deleted from page cache. If so, it will retry the
page read.
In the newer kernel, it removes this check and retry.
Instead, it introduces a new rw_semaphore in the address_space -
invalidate_lock - that holding the shared lock to protect adding
of pages to page cache for page faults / reads / readahead, and
the exclusive lock to protect invalidating pages, removing them
from page cache for truncate / hole punch.
Thus, in this patch it holds exclusive invalidate_lock in newer
kernels when remove pages from page cache caused by the revocation
of a extent DLM lock protecting them. Otherwsie, it will result in
-EIO error or partial reads in the new added test case sanity/833.
Lustre-change: https://review.whamcloud.com/50371
Lustre-commit:
bba59b1287c9cd8c30a85fafb4fd5788452bd05c
Test-parameters: clientdistro=ubuntu2204 testlist=sanity env=ONLY=833,ONLY_REPEAT=10
Change-Id: If3a27002b89636b9fd4d7b5ea50afa9aeac5d121
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50353
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sergey Cheremencev [Tue, 11 Jul 2023 14:28:12 +0000 (18:28 +0400)]
LU-16913 quota: fix ASSERTION(lqe->lqe_gl)
It is possible to add in a 2nd time lqe into qmt_reba_list while
handling of the 1st from the 1st time is not finished. There is a
small window in qmt_id_lock_glimpse when lqe_link is empty but
lqe_gl is not set.
Lustre-change: https://review.whamcloud.com/51629
Lustre-commit:
5df0459712fe1af2bc9459b4ce1b5a1220682c26
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I1168903bff88df7e5106186b082e8065a6480367
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51685
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Deiter [Wed, 19 Jul 2023 18:52:35 +0000 (22:52 +0400)]
EX-7423 tests: skip recovery-mds-scale/failover_mds
Put recovery-mds-scale/failover_mds to the always_except
list until LU-16671 has been fixed.
Test-Parameters: trivial testlist=recovery-mds-scale
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Change-Id: Id6bf929a71c3e7d6a190d4c971120f0b93159393
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51718
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 22 Jul 2023 17:27:49 +0000 (11:27 -0600)]
EX-7938 tests: skip sanity-compr sanity/100
Skip test_100 due to constant failures and unclear relation to
anything related to PFL/compression.
Test-Parameters: trivial testlist=sanity-compr
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic426498c120f8d97def44fd2531930e0a183e74f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51746
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Lei Feng [Tue, 11 Jul 2023 03:13:01 +0000 (11:13 +0800)]
EX-6826 tests: wait before unmounting pcc device if busy
wait at most 10 seconds before unmounting pcc device
if it is busy.
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc env=ONLY=25,ONLY_REPEAT=100
Change-Id: I77ec018d33d14af99bdc5d5c5c94c8fa0dafdb61
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51623
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Lei Feng [Mon, 10 Jul 2023 07:02:32 +0000 (15:02 +0800)]
LU-16953 tests: wait longer in replay-dual/test_31
Wait until file was created in replay-dual/test_31.
Lustre-change: https://review.whamcloud.com/51621
Lustre-commit: TBD (from
eed10c2ef36c7e1aebec27ce943b80bd0174ddf0)
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=replay-dual env=ONLY=31,ONLY_REPEAT=100
Change-Id: I847beb51d53e667f1599c9693aa5eb099dcf9435
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51622
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Tue, 4 Jul 2023 19:13:58 +0000 (22:13 +0300)]
EX-7786 test: get size before fallback to generic
If ll_file_seek() falls back to generic_file_llseek_size()
then make sure inode size is reliable
Test-Parameters: testlist=sanity env=ONLY=460f,ONLY_REPEAT=4 clientdistro=el9.1
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ieb8c90e66fb19675ba41f0147c0d9cdaf29ea20a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51564
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Fri, 14 Jul 2023 06:22:25 +0000 (14:22 +0800)]
LU-16943 tests: fix replay-single/135 under hard failure mode
This patch fixes replay-single test_135() to load libcfs module
on the failover partner node to avoid 'fail_val' setting error.
It also fixes the issue that not all of the OSTs are mounted after
failing back ost1.
Lustre-change: https://review.whamcloud.com/51574
Lustre-commit: TBD (from
1b73b6465b77744992bb1f6d782362bf0cf7f409)
Test-Parameters: trivial testlist=replay-single
Test-Parameters: trivial env=FAILURE_MODE=HARD \
clientcount=4 mdtcount=1 mdscount=2 osscount=2 \
austeroptions=-R failover=true iscsi=1 \
testlist=replay-single
Change-Id: Id46c722a6db9d832829a739f41f7462b32a6d9d9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51607
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Alex Deiter [Fri, 30 Jun 2023 18:02:49 +0000 (22:02 +0400)]
LU-16868 tests: add check for replace_nid
Added check for replace_nid operations and return an
error to prevent module reload errors and timeout
when unmounting targets.
Lustre-change: https://review.whamcloud.com/51524
Lustre-commit: TBD (from
a822c82326821c5c30e14d9620cd2976d5438714)
Test-Parameters: trivial
Test-Parameters: testlist=conf-sanity env=ONLY=32a serverdistro=el7.9
Test-Parameters: testlist=conf-sanity env=ONLY=32a serverdistro=el8.7
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Change-Id: I29a5de826ac8f0040dd671e502d30bac4a082c43
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51604
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Deiter [Sun, 9 Jul 2023 12:35:53 +0000 (16:35 +0400)]
LU-15046 tests: skip sanity-flr/test_200c for old MDS
Skip sanity-flr test_200c for old MDS missing the fix
for synchronized replicas and its corresponding test.
Lustre-change: https://review.whamcloud.com/51649
Lustre-commit: TBD (from
f152fe6e313843067c4c32299acf066d41896d61)
Fixes:
b7ec0d2390 ("LU-15046 osp: precreate thread vs connect race")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-flr env=ONLY=200c serverversion=2.14.0-ddn85
Test-Parameters: testlist=sanity-flr env=ONLY=200c serverversion=2.14.0-ddn89
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Change-Id: I87cd7d6b767086f993a27ce6905b05f87e325474
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51613
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Deiter [Sun, 9 Jul 2023 14:01:43 +0000 (18:01 +0400)]
LU-16341 tests: skip sanity-quota/test_1b for old MDS
Skip sanity-quota test_1b for old MDS missing the fix
for LU-16341 kernel NULL in qmt_site_recalc_cb.
Lustre-change: https://review.whamcloud.com/51648
Lustre-commit: TBD (from
a47128600fce1dd5135af610d0b31dafe1baa9d0)
Fixes:
d965d63415 ("LU-16341 quota: fix panic in qmt_site_recalc_cb")
Test-Parameters: trivial
Test-Parameters: testlist=sanity-quota env=ONLY=1b serverversion=2.14.0-ddn85
Test-Parameters: testlist=sanity-quota env=ONLY=1b serverversion=2.14.0-ddn87
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Change-Id: I1b1bc3fdfa8f36b0c20a9a06721735c8e02c034c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51612
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Fri, 27 Jan 2023 06:54:42 +0000 (00:54 -0600)]
LU-16351 llite: Linux 6.1 prandom, folios_contig, vma_iterator
Linux commit v4.10-rc3-6-gc440408cf690
random: convert get_random_int/long into get_random_u32/u64
Linux commit v6.0-11338-gde492c83cae0
prandom: remove unused functions
prandom_u32 is a wrapper around get_random_u32, change users
of prandom_u32 to get_random_u32 and provide a fallback
to prandom_u32 when get_random_u32 is not available.
Linux commit v6.0-rc1-2-g25885a35a720
Change calling conventions for filldir_t
Add a test for the new filldir_t signature
Provide wrappers for transition from int (error code) to bool
Linux commit v6.0-rc3-94-g35b471467f88
filemap: add filemap_get_folios_contig()
Provide a wrapper and fallback to find_get_pages_contig
Linux commit v6.0-rc3-225-gf39af05949a4
mm: add VMA iterator
Use vma_iterator and for_each_vma when available.
Lustre-change: https://review.whamcloud.com/49232
Lustre-commit:
ca992899d55fd13e65b75ace02931daaa29c18bd
Test-Parameters: trivial
HPE-bug-id: LUS-11377
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I23dc23d0252e1995555b6685f5cf7c207edf642b
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51628
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Shaun Tancheff [Sun, 2 Apr 2023 11:47:51 +0000 (06:47 -0500)]
LU-16328 llite: migrate_folio, vfs_setxattr
Linux commit v5.19-rc3-392-g5490da4f06d1
fs: Add aops->migrate_folio
Linux commit v5.19-rc4-52-ge33c267ab70d
mm: shrinkers: provide shrinkers with names
From Linux commit v5.19-rc5-17-g0c5fd887d2bb
acl: move idmapped mount fixup into vfs_{g,s}etxattr()
Until Linux commit v6.0-rc3-6-g6344e66970c6
xattr: constify value argument in vfs_setxattr()
Cast away const when required.
Linux commit v5.19-10313-geba2d3d79829
get rid of non-advancing variants
iov_iter_get_pages_alloc2() replaces iov_iter_get_pages_alloc()
Lustre-change: https://review.whamcloud.com/49265
Lustre-commit:
0006eb36440dcb4dc06aa61c35db40bf7dec0ddc
Test-Parameters: trivial
HPE-bug-id: LUS-11358
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Id1fa6db94172c0a61008ba4b066907950bdd6473
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51624
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Serguei Smirnov [Tue, 20 Jun 2023 19:21:42 +0000 (12:21 -0700)]
LU-16709 lnet: fix locking multiple peer NIDs
If Lustre identifies the same peer with multiple NIDs,
as a result of peer discovery it is possible that
the discovered peer is found to contain a NID which is locked
as primary by a different existing peer record.
In this case it is safe to merge the peer records,
but the NID which got locked the earliest should be
kept as primary.
This allows for the first of the two locked NIDs
to stay primary as intended for the purpose of communicating
with Lustre even if peer discovery succeeded
using a different NID of MR peer.
This patch adds updates to the original port because master
version of of this moment evolved after it was landed.
Lustre-change: https://review.whamcloud.com/50530
Lustre-commit:
3b7a02ee4d656b7b3e044713681da2f56dddb152
Test-parameters: trivial testlist=sanity-lnet
Fixes:
1a2db3e14b78 ("EX-7251 lnet: fix locking multiple NIDs")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I3303e618b37a76c30be6426972e7853bb31ae497
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51384
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sebastien Buisson [Fri, 4 Mar 2022 15:45:59 +0000 (16:45 +0100)]
LU-13343 gss: no sec flavor on loopback connection
When using a local client, i.e. a client mounted on a server node,
there is no benefit from a security standpoint to enforce an SSK or
KRB flavor, since the data does not go over the network.
So force the 'null' security flavor for connections on 0@lo,
independently of the currently defined srpc flavor.
Lustre-change: https://review.whamcloud.com/46704
Lustre-commit:
e3e91ea95fd96a5eafc598e3812390b4cbac05c3
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: If25d69bb1e67735cb0544ca954e49175f7471248
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Etienne AUJAMES [Sat, 15 Jul 2023 11:29:30 +0000 (19:29 +0800)]
LU-16747 llapi: fix race in get_root_path_slow()
The patch
bdf7788d ("LU-8585 llapi: use open_by_handle_at in
llapi_open_by_fid") caches the Lustre root fd to avoid re-openning
it each time an ioctl() is needed on the fs.
For now, only 1 entry is stored. If a llapi call is performed on
another mountpoint, llapi needs to close the old root fd and open a
new one.
A race condition exists at startup, when root_cached.fd is not
initialized yet. Several threads try to determine root information at
the same time (in get_root_path_slow()). Those threads will close(),
open() and update different "root_cached.fd".
The usage of a closed root fd will return EBADFD (e.g: in
llapi_open_by_fid(), llapi_hsm_request() or llapi_fid2path()).
This patch checks if the fs is the same before updating the root
entry. If so, the root entry (and cached root fd) will not be changed.
Add the regresion test sanityn 85 (llapi_root_test).
Lustre-change: https://review.whamcloud.com/50682
Lustre-commit:
9ef1e097d53000233f9ba23319268f467c276173
Test-Parameters: trivial testlist=sanityn env=ONLY=85,ONLY_REPEAT=20
Test-Parameters: testlist=sanityn
Test-Parameters: testlist=sanity
Fixes:
bdf7788d ("LU-8585 llapi: use open_by_handle_at in llapi_open_by_fid")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I681aac7d5715022e700cdb092db94deaa6bf6a8f
Reviewed-by: Guillaume Courrier <guillaume.courrier@cea.fr>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51689
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Wed, 5 Jul 2023 20:32:10 +0000 (14:32 -0600)]
LU-16101 tests: skip sanity/27J for more kernels
This is a bug in the kernel that is not present in older kernels
before commit v5.11-10234-gcbd59c48ae2b (5.12), and is fixed with
commit v6.2-rc4-61-g5956592ce337 (6.2).
Move this from ALWAYS_EXCEPT (bug that needs to be fixed) to skip
(test that is known to fail in some configs but has been fixed).
Lustre-change: https://review.whamcloud.com/51567
Lustre-commit:
b711af7d243f3773cec3a37f64c0e0aa8bbc363f
Fixes:
af6f49698a18 ("LU-16101 tests: add sanity/27J to always_except")
Test-Parameters: trivial testlist=sanity clientdistro=el9.2 env=ONLY=27J
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8ec0a6d25a90e05672b039cd6c2b2fbf8a3ebbe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51583
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Arshad Hussain [Mon, 14 Feb 2022 08:36:47 +0000 (14:06 +0530)]
LU-15519 quota: fallocate does not increase projectid usage
fallocate() was not accounting for projectid quota usage.
This was happening due to two reasons. 1) the projectid
was not properly passed to md_op_data in ll_set_project()
and 2) the OBD_MD_FLPROJID flag was not set receive the
projctid.
This patch addresses the above reasons.
Test-case: sanity-quota/78a added
Lustre-change: https://review.whamcloud.com/46676
Lustre-commit:
5fc934ebbbe665f24e2f11fe224065dd8e9a08ba
Fixes:
48457868a02a ("LU-3606 fallocate: Implement fallocate preallocate operation")
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I3ed44e7ef7ca8fe49a08133449c33b62b1eff500
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51639
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Thu, 6 Jul 2023 05:21:01 +0000 (23:21 -0600)]
RM-620 build: New tag 2.14.0-ddn93
New tag 2.14.0-ddn93
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic6530f389228232cf1f763810d2abcbebc9b51d7
Andreas Dilger [Fri, 30 Jun 2023 10:29:49 +0000 (04:29 -0600)]
LU-16937 utils: avoid lctl shmget() if not needed
lctl is dynamically allocating an IPC shared memory segment
during every startup, even though it is only needed for a
small number of uncommon debug commands:
shmget(IPC_PRIVATE, 65680, 0600) = 196641
shmat(196641, NULL, 0) = 0x7f752b1c5000
shmctl(196641, IPC_RMID, NULL) = 0
This setup can be moved to sub-commands that actually need it.
Lustre-change: https://review.whamcloud.com/51526
Lustre-commit: TBD (from
309713169fde9e162c26e909bc83cb43cccd67ba)
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I41c790ce7cba2d9c48c1ec06eb23eb94aa548242
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51516
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Fri, 30 Jun 2023 11:32:03 +0000 (19:32 +0800)]
LU-16934 kernel: update RHEL 8.8 [4.18.0-477.15.1.el8_8]
Update RHEL 8.8 kernel to 4.18.0-477.15.1.el8_8.
Lustre-change: https://review.whamcloud.com/51517
Lustre-commit: TBD (from
7174f706328cb4e6a52c898c1cd7719b81e26c0d)
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Change-Id: I66365dce63065a0a07958a182a3c705e9948d424
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51519
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Tue, 27 Jun 2023 15:50:03 +0000 (15:50 +0000)]
EX-7683 utils: always try to use our own lz4/lz4hc
lz4/lz4hc provided by the kernel do not grok a compression level.
The built-in lz4/lz4hc do, so always build them as dedicated kernel
modules llz4.ko and llz4hc.ko, with the same .cra_name but with a
slightly higher .cra_priority = 110, so that they are preferred over
the in-kernel modules if any.
And try to manually load the llz4/llz4hc kernel modules when a file
requires compression with the corresponding alg. This is a "one-shot"
try that allows us to prefer our modules that has level support, but
continues to at least compress/decompress files even if our own
modules are not available.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0bdf267f998e21df81e460250a653aed34e3215d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51474
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Wed, 5 Jul 2023 01:04:10 +0000 (19:04 -0600)]
LU-8582 tests: skip sanity/905 for old OSTs
The fail_loc used in sanity test_905 does not exist in older OSTs.
Skip this subtest for older OSTs.
Lustre-change: https://review.whamcloud.com/51568
Lustre-commit: TBD (from
2ced1e0898aacd741c95c25d44350dfefa953853)
Fixes:
566edb8c43 ("LU-8582 target: send error reply on wrong opcode")
Test-Parameters: trivial testlist=sanity serverversion=2.12.9 env=ONLY=905
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8fa50ec0f66afd9f24d562e0be57a416c04d8ba8
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51569
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Wed, 5 Jul 2023 20:13:42 +0000 (14:13 -0600)]
LU-15235 tests: skip sanity/56od in interop
Sanity test_56oc and test_56od were using the btime_supported()
function to check it "lfs find" supported file birth time, but
this did not properly check whether the MDS supported this option.
Remove the btime_supported() check and just use the version, since
this has been around a few releases already.
Lustre-change: https://review.whamcloud.com/51580
Lustre-commit: TBD (from
8add332bda0c58d9908478b9263e8aea46edc135)
Fixes:
186b97e68abb ("LU-11971 utils: Send file creation time to clients")
Test-Parameters: trivial testlist=sanity serverversion=2.12.9 env=ONLY=56
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0c85103c843d3b993e3e112bf5d0da976d3ebbe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51581
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Alex Deiter [Tue, 4 Jul 2023 10:30:57 +0000 (14:30 +0400)]
EX-7814 tests: fixed shell syntax error
Fixed a regression introduced in LU-16399.
Auster assumes that all scripts are written in bash,
so this commit removes hardcoded sh calls.
Test-Parameters: trivial testlist=parallel-scale-nfsv4
Fixes:
ea13f42719 ("LU-16399 tests: add subtests setup/cleanup records")
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Change-Id: I20d136db3b763143df038d25a167eb84f646444f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51556
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wei Liu [Mon, 26 Jun 2023 20:33:21 +0000 (13:33 -0700)]
EX-7758 test: Add sanity-compr.sh to run sanity with PFL layout
Add sanity-compr.sh to run sanity and sanityn with PFL layout
Also fix sanity subtests problem of 56ba,57b,65e,65g,65n,204e
Test-Parameters: trivial testlist=sanity-compr
Signed-off-by: Wei Liu <sarah@whamcloud.com>
Change-Id: Iefdc7757697629eb5c57d7694456249d62a2049e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51577
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Yang Sheng [Thu, 29 Dec 2022 17:46:56 +0000 (01:46 +0800)]
LU-15934 lod: clear up the message
Print out the precise info while llog context error.
Lustre-change: https://review.whamcloud.com/49528
Lustre-commit:
9882d4e933fd8cdbc4a9bc8bf6b29655009f7e03
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I492201cd3ae5eb39ad34f3a873d7bb346b52430f
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Bobi Jam [Wed, 28 Jun 2023 16:51:25 +0000 (00:51 +0800)]
EX-6856 utils: add -Z option for 'lfs getstripe/find'
Add support for "lfs getstripe -Z" to get the last instantiated
component compression information.
Add support for "lfs find -Z <type>[:[+-]<level>]" to keep consistent
options with "lfs setstripe -Z".
Fixes:
093bd2f343 ("EX-6856 utils: add 'lfs find' support for compressed file")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Id3e788761656d05604bc9a72fb1e51c5f2a0ad3b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51497
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Yang Sheng [Sat, 3 Jun 2023 18:47:30 +0000 (02:47 +0800)]
LU-15934 tests: add a test case for update llog
A test case to simulate the update llog corruption
situation. It replaces the catalog file with a
random data. The recovery of mdt will be blocked
if without the fixing patch.
Lustre-change: https://review.whamcloud.com/51208
Lustre-commit:
54301fe4f598eef5aebdbdb0c7f3dddea9541c4e
Fixes:
814691bcff ("LU-15934 lod: renew the update llog")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I0ade8d0ff33ddc06b622e5e67cf4b4775dfff129
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51570
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Yang Sheng [Fri, 6 Jan 2023 13:10:35 +0000 (21:10 +0800)]
LU-15934 lod: renew the update llog
Skip and renew the update llog file while it was
corrupted.
Lustre-change: https://review.whamcloud.com/49569
Lustre-commit:
814691bcffab0a19240121740fb85a1912886a3c
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I3491858dce42b4a8ed11db55ebbf8a12ef5f521d
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51552
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Sat, 1 Jul 2023 23:57:26 +0000 (17:57 -0600)]
EX-6127 tests: test compression without bzip2/HDF5
Run as much of sanity test_460a compression tests as possible,
even if bunzip2 or HDF5 file are unavailable.
Print a clear message in test_84 if bunzip2 unavailable.
Fixes:
f43b9ce9af ("EX-6127 osc: osc brw request compression")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I36251834f636600eb9b0194ccd14c8b203da32e5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51532
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Sergey Cheremencev [Mon, 25 Apr 2022 18:49:55 +0000 (21:49 +0300)]
LU-16340 quota: notify only global lqe
Don't notify slaves with new limits when set new
limits to the pools. Do this only for lqes that
belong to the global pool.
The fix helps to avoid a case when slaves do not
apply new limit because slaves` data version is
greater than the one comes from the MDT. It was
possible, if set a lot of times different limits to PQ.
After that new limits from the global pool could not
be be applied:
qsd_upd_schedule()) lustre-OST0000: discarding glb
update from glimpse ver:7 local ver:203
For details about the problem see "check indexes versions"
test in sanity-quota.sh.
Add test 25 "check indexes versions" into sanity-quota.
Without the fix it reproduces above problem.
Fix checkpatch to don't print "Invalid vsprintf pointer
extension" for %px.
HPE-bug-id: LUS-10705
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Lustre-change: https://review.whamcloud.com/c/fs/lustre-release/+/49239/
Lustre-commit:
513b1cdbca58913249eb524a37374c418fdec44f
Change-Id: Idb091a10894e9db9f67d215baef2926723d6c65d
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51551
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Patrick Farrell [Tue, 4 Jul 2023 18:44:22 +0000 (14:44 -0400)]
EX-7818 osc: don't do decomp. with compression disabled
decompress_request likely has a significant performance
cost for checking if data is compressed, so we should not
call it when compression is disabled.
This is a stop-gap solution for the preview until we can
have the server tell the client if data is compressed, as
described in EX-7818.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iba8feba3ab0fe620d0594f59c2c6ddea25faeb4f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51563
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Artem Blagodarenko [Sat, 1 Jul 2023 22:13:20 +0000 (23:13 +0100)]
EX-7807 osc: don't discard decompress_request error
The error handling for decompress_request is unusual - non-zero
returns are just discarded. And rc2 is just discarded.
The read() doesn't fail or get a short read.
Fix this so if decompression fails with an error.
Reported-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Fixes:
f43b9ce9af ("EX-6127 osc: osc brw request compression")
Change-Id: Idd01947c7375c9586a64f064dd6ee0ac2308ea86
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51531
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Patrick Farrell [Mon, 3 Jul 2023 23:57:17 +0000 (19:57 -0400)]
EX-7812 llite: use CIT_MISC in check_compression
ll_mmap_check_compression uses a CIT_FAULT io type and
initializes it with cl_io_rw_init. This doesn't
initialize the IO correctly and this sometimes results
in the following crash during io_fini, because the ft_page
pointer is initialized to something else by cl_io_rw_init:
[ 1512.276302] BUG: unable to handle kernel NULL pointer dereference at
000000000000004a
[ 1512.280778] RIP: 0010:cl_pagevec_put+0x9f/0x3a0 [obdclass]
[ 1512.288693] vvp_io_fault_fini+0x21/0x40 [lustre]
[ 1512.289294] cl_io_fini+0x7a/0x230 [obdclass]
[ 1512.289876] ll_mmap_check_compression+0x403/0x540 [lustre]
Fixes:
95df962a92 ("EX-6265 llite: disable mmap on compressed files"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ib1f270f071370d6c045bb9d799ab5b7b41a6c6be
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51553
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Thu, 29 Jun 2023 16:06:47 +0000 (18:06 +0200)]
EX-6127 llite: fix chunk_bits usage in readahead
For the minimum compression chunk size, chunk bits is zero,
so we cannot use if (chunk_bits) to determine if we're
doing compression.
This also fixes two other things:
1. A rounding error when rounding to chunk
2. Move rounding of end_idx to before first usage of
end_idx, so calculation of number of pages is correct
Without this, when the user reads 1 page or less, readahead
will calculate the readahead page count as 0 and will exit
without reading the chunk.
Fixes:
c05d5990f4 ("EX-6127 llite: getting stripe info optimization")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I273506fd4f6ed5f0b8b5020357fd7caf0531e61c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51504
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Wed, 28 Jun 2023 18:37:22 +0000 (02:37 +0800)]
EX-6856 utils: support 'lfs find --printf %LZ'
Add support for "lfs find --printf %LZ" to print the compression
type:level of the last instantiated component of a file.
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Iaf1b6c031b06c70e7b5be51354697aa6bdcc9850
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51498
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 1 Jul 2023 23:06:17 +0000 (17:06 -0600)]
RM-620 build: New tag 2.14.0-ddn92
New tag 2.14.0-ddn92
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Icc54de2ab22bff722ec19c3293767706458f684b
Patrick Farrell [Fri, 30 Jun 2023 20:40:51 +0000 (16:40 -0400)]
EX-6127 osc: add compression to can_merge_pages
Some BRW flags are OK to have on only some pages in a BRW,
others are not. can_merge_pages has a whitelist of the
flags which are safe to have on only some pages in a BRW,
and prints a warning if other flags are seen.
Add compression to the white list, because while all pages
in an niobuf must be compressed, it is normal to have only
some pages in a BRW compressed.
Prior to this patch, this warning was printing during
normal usage of compression.
Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ia262d4fc878e5328bd956865047e997aa77946f0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51528
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Fri, 23 Jun 2023 11:58:39 +0000 (13:58 +0200)]
EX-6206 lustre: add lgzip kernel module
lgzip kernel module implements compression according to the
deflate/zlib algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'deflate':
* deflate-lustre-generic of type compression
* deflate-lustre-scomp of type scomp
Note the 'deflate' name is identical to the in-kernel module, but
lgzip registers it with a slightly higher .cra_priority = 110, so that
it is preferred over the in-kernel module.
Our 'deflate' is also different in that it accepts a compression level
as explained below.
lgzip kernel module sources are copied from linux v6.2-rc5 and renamed
to gzip.c to avoid name collisions. It implements the Crypto API
interface, and rely on the deflate/zlib kernel library for compression
implementation. It has been modified to grok a compression level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The deflate/zlib library sources are also copied from linux v6.2-rc5
and built statically. Headers have also been copied from linux
v6.2-rc5 for consistency, and source files modified to include the
copied headers instead of the system headers.
All aforementioned sources are located in the lustre/gzip directory.
The lgzip module is always built with Lustre.
This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'deflate' module.
It also tries to manually load the lgzip kernel module when a file
requires compression with the 'delfate' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I2b0386457bc781d91816172dea6b52ce3dd273f4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51422
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Artem Blagodarenko [Thu, 29 Jun 2023 13:49:11 +0000 (14:49 +0100)]
EX-7784 tests: skip sanity 460 on 64k PAGE_SIZE
Skip this test on the system with 64k PAGE_SIZE.
Test-Parameters: trivial testlist=sanity clientdistro=el8.7 clientarch=aarch64
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I9324ae975bfc3c4f08d5048d7c977447ef62cc78
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51501
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 1 Jul 2023 10:22:53 +0000 (04:22 -0600)]
LU-14301 revert: client: use EOPNOTSUPP instead of ENOTSUPP
This reverts commit
3deafa8a39a964c67f533173a5b2f90d0d2d7730.
Accidentally landed and is likely to cause problems.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 1 Jul 2023 10:09:49 +0000 (04:09 -0600)]
EX-7386 build: Update lipe tag to 2.24
New version tag.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I711231b7ec063ed9bf8a0976e18360bebdc84d3b
Andreas Dilger [Fri, 30 Jun 2023 00:19:06 +0000 (18:19 -0600)]
LU-14301 client: use EOPNOTSUPP instead of ENOTSUPP
Don't return NFS-specific error code ENOTSUPP back to userspace,
instead use EOPNOTSUPP.
Lustre-change: https://review.whamcloud.com/51511
Lustre-commit: TBD (from
1a0d553000d5a869f9039bab74dbdbb20d4259b0)
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iabd07b31069737e8ee7ca2382fd8cff6143ebbe5
Patrick Farrell [Thu, 29 Jun 2023 17:52:01 +0000 (13:52 -0400)]
EX-6265 llite: allow mmap reads of compressed files
mmap reads of compressed files work, so we should only
block writes.
We cannot block the actual fault operations because that
will cause the application to get a SIGBUS, so we check
the file open mode when we go to create the memory mapping.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I636f398fd247ddcd153f94bf8116440540e8469c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51506
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Zhuravlev [Thu, 22 Jun 2023 08:36:51 +0000 (11:36 +0300)]
EX-6265 llite: disable mmap on compressed files
disable mmap(2) on compressed files until well tested.
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I3464a03b16708edcd0692bc9db337eb8473ea047
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51413
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alex Deiter [Mon, 9 Jan 2023 16:19:26 +0000 (20:19 +0400)]
LU-16399 tests: add subtests setup/cleanup records
* Added setup/cleanup records for subtests
Lustre-change: https://review.whamcloud.com/49582
Lustre-commit:
d0ae0079d747d05f74f733fb594d8edb512f8b16
Change-Id: Icb203a864fa8785e423a073b4ee0f02ea3d3ac77
Signed-off-by: Alex Deiter <adeiter@tintri.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51406
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Serguei Smirnov [Fri, 19 May 2023 02:12:19 +0000 (19:12 -0700)]
LU-16836 lnet: ensure dev notification on lnd startup
Look up device and link state on lnd startup so that
the initial NI state may be set properly.
Reduce code duplication by adding lnet_set_link_fatal_state() and
lnet_get_link_status() functions which are shared across LNDs.
LND-specific versions of these are removed.
This fixes the issue with adding LNet NI using an interface with
cable unplugged which results in the NI state initialized as "up".
Lustre-change: https://review.whamcloud.com/51057
Lustre-commit:
09c6e2b872287c847d15620788f6cf50b3a9f30b
Fixes:
c4df48116d ("LU-16563 lnet: use discovered ni status")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I16084092cc21a4e42dfef4624adfbf57eb4fdecb
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51310
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Jian Yu [Wed, 28 Jun 2023 09:36:19 +0000 (17:36 +0800)]
EX-7509 tests: enable neterror logging in sanity-benchmark/iozone
This patch enables LNet error logging in sanity-benchmark test_iozone()
to gather network errors while connection issue occurs.
Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes \
clientdistro=el8.6 serverdistro=el7.9 testlist=sanity-benchmark
Change-Id: I398779abc95525fe5579fc7505e6e6221c32bf90
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51483
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Tue, 21 Jun 2022 12:06:22 +0000 (20:06 +0800)]
LU-8582 target: send error reply on wrong opcode
Unknown opcode does not necessarily means insane client. A new client
might send RPCs with new opcodes to an old server. The client might
desperately stuck there waiting for a reply. So, send an error back
when RPC has a wrong opcode.
This patch returns the EOPNOTSUPP to client instead of block. ENOTSUPP
is not used here since strerror() does not understand ENOTSUPP.
OBD_FAIL_OST_OPCODE=0x253 is added for fault injection test of opcode.
To test whether an invalid opcode is handled properly on OST, use the
following command:
lctl set_param fail_val=${opcode} fail_loc=0x253
Lustre-change: https://review.whamcloud.com/47761
Lustre-commit:
03c1ddf19c83891683e1726f240a2449941e8b22
Change-Id: I46ca62bc532b92368e06a4f883b102c7186c453c
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51513
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
John L. Hammond [Wed, 20 Jan 2021 15:22:54 +0000 (09:22 -0600)]
LU-14301 lustre: add ENOTSUPP to spelling.txt
Add a spelling check for ENOTSUPP to suggest use of EOPNOTSUPP
instead. Note:
ENOTSUPP (524) and defined only in the kernel errno.h and is a NFSv3
specific errno. If ENOTSUPP is returned to userspace then strerror()
will print "Unknown error 524".
EOPNOTSUPP (95) is defined in kernel and userspace errno.h.
ENOTSUP is defined in userspace errno.h as an alias for EOPNOTSUPP.
Lustre-change: https://review.whamcloud.com/41280
Lustre-commit:
e00733f0f87659c936039a58ea738cfb070638bc
Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I13b0389c9ec0853f43d8ab4a8f6538eb24c8a2ad
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51512
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Sebastien Buisson [Wed, 28 Jun 2023 15:05:18 +0000 (17:05 +0200)]
EX-7683 man: document supported compression types
Update lfs-setstripe to mention the currently supported compression
types: gzip, lz4, lz4fast and lzo.
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9b587516d27b882bec5855da4948f489e5a0041f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51485
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 28 Jun 2023 09:34:22 +0000 (09:34 +0000)]
EX-7775 utils: fix cp_comp_type size
cp_comp_type should be 8 bits, as llch_compr_type and all associated
variables are declared as u8.
So remove useless cp_comp_enabled and fix code to test for compressed
component with cp_comp_type against LL_COMPR_TYPE_NONE.
And update LL_COMPR_TYPE_MAX value to 255 to avoid conflicts with
future compression types.
Fixes:
f43b9ce9af ("EX-6127 osc: osc brw request compression")
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ia15868ac0ac003b62942540a57f782226ae8c141
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51481
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Tue, 27 Jun 2023 14:42:20 +0000 (16:42 +0200)]
EX-7775 utils: fix LL_COMPR_TYPE wire checks
Fix LL_COMPR_TYPE* wire checks to avoid duplication.
The value of LL_COMPR_TYPE_UNCHANGED is also declared as 15 so that
is does not conflict with other potential real compression types in
the future.
Fixes:
67d4601737 ("EX-6249 csdc: set compress component for file")
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iab9830f09f0778e1e1f3b1ea4c9878ce1017de8d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51473
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Tue, 20 Jun 2023 15:49:14 +0000 (15:49 +0000)]
EX-7683 utils: update handling of compr level for lz4
The way lz4 compression level or acceleration factor is handled needs
to be adapted in order to match what is provided by the lz4 userspace
tool:
- any level between 0 and 2 is interpreted as the default lz4
acceleration factor of 1;
- any level from 3 and up to 16 is interpreted as a compression level
for internal lz4hc. Increasing the compression level trades CPU
time for improved compression ratio;
- acceleration factor can be specified for the lz4fast compression
type, from 1 to 26. This acceleration factor trades compression
ratio for faster speed.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4711217c1a6601f29f78d262567da5998f657fc9
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51380
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Wed, 21 Jun 2023 12:00:07 +0000 (12:00 +0000)]
EX-7715 utils: error for invalid compression level
Not all compression types accept a compression level (e.g. lzo in its
kernel implementation). For those, return an error if a compression
level is provided on 'lfs setstripe' command-line.
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6c367c4bfd76cc81c890a89ba9f994f4fd9f4f80
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51394
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alena Nikitenko [Wed, 28 Jun 2023 13:28:44 +0000 (16:28 +0300)]
EX-6931 tests: add version check in sanity test_904
The patch that added hide virtual projid xattr functionality
to b_es6_0 branch was submitted under a tag of 2.14.0-ddn55.
In case of interop testing with older servers test 904 fails.
At the same time the rest of the test should still run on
older servers, so I moved parts of the code related to hide
virtual projid xattr functionality under version check
condition.
Sanity test 904 was modified.
Fixes:
8acf413647 ("LU-15548 osd-ldiskfs: hide virtual projid xattr")
Test-Parameters: trivial testlist=sanity env=ONLY="904"
Test-Parameters: testlist=sanity env=ONLY="904" serverversion=2.14.0-ddn54
Signed-off-by: Alena Nikitenko <anikitenko@ddn.com>
Change-Id: Ie3bc67cb8a7a83954f3b2048f009f84ab77bf53d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51484
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Thu, 29 Jun 2023 15:01:35 +0000 (23:01 +0800)]
EX-7752 test: hot-pools test_59: not expected flag 'stale'
Make the 1st mirror preferable for write for the test purpose.
Test-Parameters: trivial env=ONLY=59 testlist=hot-pools
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I06c33badfc10ff9743df83a6e421e27afc9b6dbd
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51502
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 24 Jun 2023 18:15:33 +0000 (12:15 -0600)]
RM-620 build: New tag 2.14.0-ddn91
New tag 2.14.0-ddn91
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8634b3650bf43aadaf303e48b9ebe2931e6af6ab
Jian Yu [Fri, 23 Jun 2023 03:10:35 +0000 (11:10 +0800)]
LU-16923 kernel: update RHEL 8.8 [4.18.0-477.13.1.el8_8]
Update RHEL 8.8 kernel to 4.18.0-477.13.1.el8_8.
Lustre-change: https://review.whamcloud.com/51411
Lustre-commit: TBD (from
a04c2ff950a2a6b65e55184113b1513fc9bf0058)
Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity
Change-Id: I23c6439aedd6f8e9473ddb629ff7e01c50d9c8fc
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51420
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Mikhail Pershin [Fri, 23 Jun 2023 22:21:28 +0000 (01:21 +0300)]
EX-7680 llite: skip hole/data lseek() on compressed file
Doesn't execute real lseek() on compressed file with
SEEK_HOLE/SEEK_DATA origin but consider file always
non-sparsed and do generic lseek() only
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I2cd587cc0205e85758e06bbaafafe0e2959e0ade
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Thu, 18 May 2023 21:41:47 +0000 (15:41 -0600)]
LU-15671 mds: do not send OST_CREATE transno interop
Send OST_CREATE RPCs from the MDS with no_resend and no_delay
when communicating with an old OST that does not support the
OBD_CONNECT2_REPLAY_RESEND. Likewise, the OST should not reply
to the MDS RPC with rq_transno set, or this will trigger:
osp_precreate_send() ASSERTION(req->rq_transno == 0) failed
This can be avoided if the MDS is upgraded before the OSS, but
will always be hit if OSS is upgraded first.
After 2.20.53 the MDS/OSS assume that this is always true, since
rolling upgrades are unsupported for larger version differences.
Lustre-change: https://review.whamcloud.com/51056
Lustre-commit:
9ee1281060d0a00a9c5d715a9a6d9b99c27123ff
Test-Parameters: testgroup=rolling-upgrade-oss
Fixes:
63e17799a3 ("LU-8367 osp: enable replay for precreation request")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I1ab601a2f55540dd75cf24838f7cdb7f823ed42c
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51425
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Artem Blagodarenko [Wed, 14 Jun 2023 21:10:28 +0000 (22:10 +0100)]
EX-6127 llite: getting stripe info optimization
ll_lov_getstripe_ea_info() is expensive call and should be avoided
if possible. Let's use cached chunk size rather than get it from
stripe info every time.
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: Id08487ec782f797e242e3f673c4a4dd8d526c9cc
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51321
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Patrick Farrell [Tue, 13 Jun 2023 17:26:09 +0000 (13:26 -0400)]
EX-7601 llite: restrict overwrites during preview
EX-7601 is an issue where when modifying a compressed file
we do not correctly read-up and re-write existing
compressed data.
To avoid this, we can only allow writes which are not
aligned to compression chunk size when they are not
overwriting existing data, ie, when they are extending the
file.
This returns EINVAL for all writes to compressed files
which are not either chunk aligned or extending the file.
This should prevent users from hitting the data corruption
issue but still allows some basic usage. This is intended
just for the preview period.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0eea01e2249866a074afd0d0642fe6dce9a49664
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51259
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Patrick Farrell [Fri, 2 Jun 2023 02:24:15 +0000 (22:24 -0400)]
EX-6127 llite: DIO fallback on compressed files
Fully supporting direct I/O on compressed files is tricky
because we cannot pull the full chunk in to the page cache
(because there is no page cache for DIO).
So instead we fall back to buffered I/O for DIO on
compressed files.
This patch adds the check and a test for this.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I8224ef9b8ad1d912d8a11eccad37d3dff8dd8498
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51200
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Artem Blagodarenko [Fri, 19 May 2023 09:27:36 +0000 (10:27 +0100)]
EX-7528 lov: fallocate is not allowed for compressed files
Client Side Data Compression allocates blocks after a compression.
It is impossible to preallocate blocks for the whole file, so
fallocate should be disabled in case of compression.
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: Ie834ace183fdcec0d7d6f747237e0964c3c4120b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51059
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Artem Blagodarenko [Wed, 30 Nov 2022 14:54:57 +0000 (14:54 +0000)]
EX-6127 osc: osc brw request compression
This patch adds client-side compression/decompression.
The client-side data compression project (CSDC) reduces storage and network
utilization by leveraging the more plentiful memory and CPU resources on the
local client. Data is sent compressed over the network, saved directly to
storage on the server side, and decompressed back on the client side.
Uncompressed data is kept in client page cache, all while being functionally
transparent to the end user and application.
As an example, a test file is compressed and decompressed.
The resulting file is compared with the original one.
The test case shows 2.5x compression ratio:
356K /mnt/lustre/d460.sanity/sanity.sh
884K /tmp/cmp-46ofie/decompressed_sanity.sh
Compression should read whole chunk even if offset
and size differ.
Let's modify readahead to force reading data from the
offset and size multiple to the chunk size.
Test-Parameters: testlist=sanity env=ONLY=460
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49511
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Bobi Jam [Wed, 24 May 2023 00:25:05 +0000 (08:25 +0800)]
LU-16837 csdc: reserve connect bits for compressed layout
Add connect data bit for compressed layout (OBD_CONNECT2_COMPRESS)
and another connect data bit to be used (OBD_CONNECT2_LARGE_NID).
Also reserve obd_connect_data::ocd_compr_type which is a bitmask of
supported compression type to be negotiated between client and MDS.
Lustre-change: https://review.whamcloud.com/51108
Lustre-commit:
83189aef3b23f18cb8c1deae34994a00f8582039
Test-Parameters: trivial
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I21029c6c3e8a7e690ecc8d489bbb95aec3ab1fa8
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51409
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log