Whamcloud - gitweb
fs/lustre-release.git
2 weeks agoLU-16480 lov: fiemap improperly handles fm_extent_count=0 08/52308/2
Andrew Perepechko [Mon, 16 Jan 2023 13:13:34 +0000 (08:13 -0500)]
LU-16480 lov: fiemap improperly handles fm_extent_count=0

FIEMAP calls with fm_extent_count=0 are supposed only to
return the number of extents.

lov_object_fiemap() attempts to initialize stripe_last
based on fiemap->fm_extents[0] which is not initialized
in userspace and not even allocated in kernelspace.

Eventually, the call exits with -EINVAL and "FIEMAP does
not init start entry" kernel log message.

Lustre-change: https://review.whamcloud.com/49645
Lustre-commit: 829af7b029d8e4e391b93792bf5214611b0193bd

Fixes: 409719608c ("LU-11848 lov: FIEMAP support for PFL and FLR file")
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Change-Id: I65e706b5dd5c8a6db90a539c2602af839b4da823
HPE-bug-id: LUS-11443
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52308
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 weeks agoLU-17883 kernel: update SLES15 SP5 [5.14.21-150500.55.65.1] 28/55228/4
Jian Yu [Tue, 28 May 2024 23:32:47 +0000 (16:32 -0700)]
LU-17883 kernel: update SLES15 SP5 [5.14.21-150500.55.65.1]

Update SLES15 SP5 kernel to 5.14.21-150500.55.65.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/55227
Lustre-commit: 530f49f628bad9b8bb3c2a87a79009b735998938

Test-Parameters: trivial mdtcount=4 mdscount=2 \
  clientdistro=sles15sp5 testlist=sanity

Test-Parameters: optional clientdistro=sles15sp5 testgroup=full-part-1
Test-Parameters: optional clientdistro=sles15sp5 testgroup=full-part-2
Test-Parameters: optional clientdistro=sles15sp5 testgroup=full-part-3

Change-Id: Ie0601c190e52d6192bf389338be51c77db03a9c2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55228
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 weeks agoLU-17404 kernel: update RHEL 9.4 [5.14.0-427.20.1.el9_4] 66/55366/3
Jian Yu [Wed, 12 Jun 2024 17:13:40 +0000 (10:13 -0700)]
LU-17404 kernel: update RHEL 9.4 [5.14.0-427.20.1.el9_4]

Update RHEL 9.4 kernel to 5.14.0-427.20.1.el9_4 for Lustre client.

Lustre-change: https://review.whamcloud.com/54712
Lustre-commit: TBD (from 000ac2084bef80ed9b0610245ab7552f678d3e39)

Test-Parameters: trivial \
  mdtcount=4 mdscount=2 clientdistro=el9.4 testlist=sanity
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-3

Change-Id: Ieee88a5a9f8e58f8445e126d21e45228e7b5ca64
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55366
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
2 weeks agoLU-17942 kernel: update RHEL 8.10 [4.18.0-553.5.1.el8_10] 11/55411/2
Jian Yu [Wed, 12 Jun 2024 17:08:24 +0000 (10:08 -0700)]
LU-17942 kernel: update RHEL 8.10 [4.18.0-553.5.1.el8_10]

Update RHEL 8.10 kernel to 4.18.0-553.5.1.el8_10.

Lustre-change: https://review.whamcloud.com/55410
Lustre-commit: TBD (from 2cc06472d975dfe224d09bac0fd54316a721a122)

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el8.10 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.10 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.10 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.10 testlist=sanity

Test-Parameters: optional clientdistro=el8.10 serverdistro=el8.10 \
  testgroup=full-part-1

Test-Parameters: optional clientdistro=el8.10 serverdistro=el8.10 \
  testgroup=full-part-2

Test-Parameters: optional clientdistro=el8.10 serverdistro=el8.10 \
  testgroup=full-part-3

Change-Id: I9e36b42c56f0d5e45077350d0afc32f207e3d8b7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55411
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 weeks agoLU-17197 obdclass: preserve fairness when waiting for rpc slot 32/53232/3
Shaun Tancheff [Wed, 18 Oct 2023 03:54:59 +0000 (22:54 -0500)]
LU-17197 obdclass: preserve fairness when waiting for rpc slot

When obd_get_mod_rpc_slot() waits for an available slot it places the
waiting thread at the HEAD of the queue, so it will be woken before
anything else that is already queued.  This is clearly unfair and can
hurt performance.

So change to always add to the tail to ensure a FIFO ordering (except
that CLOSE might sometimes be woken a bit early).

This regression was introduced in a rewrite that was supposed to make
waiting more fair - by avoiding a broadcast wakeup for "close"
requests.

Also fix some stale comments and expose __add_wait_queue_entry_tail

Running mdtest with the patch applied shows about a 3% improvement:

                             master            patched
  mdtest-easy-write      350.585906 kIOPS   353.783545 kIOPS
   mdtest-easy-stat     1320.329353 kIOPS  1408.320419 kIOPS
 mdtest-easy-delete      285.084103 kIOPS   289.625900 kIOPS
            [SCORE]      509.115803 kiops   524.516113 kiops

Lustre-change: https://review.whamcloud.com/52738
Lustre-commit: b5fde4d6c02324a8511afe30d02eb2cf46ea799d

Fixes: 5243630b09d2 ("LU-15947 obdclass: improve precision of wakeups for mod_rpcs")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If767c4299bcbab71589b0f3c01e85bf461686ca5
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53232
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 weeks agoLU-16633 obdclass: fix rpc slot leakage 39/51539/3
Alex Zhuravlev [Fri, 10 Mar 2023 17:47:05 +0000 (20:47 +0300)]
LU-16633 obdclass: fix rpc slot leakage

obd_get_mod_rpc_slot() can race with obd_put_mod_rpc_slot():
finishing wait_woken() resets WQ_FLAG_WOKEN (which is set
when the corresponding thread gets a slot incrementing
cl_mod_rpcs_in_flight. then another thread execting
__wake_up_locked_key() may find that wq_entry again and call
claim_mod_rpc_function() one more time again incrementing
cl_mod_rpc_in_flight. thus it's incremented twice for a
single obd_get_mod_rpc_slot().

    #1: obd_get_mod_rpc_slot() #2: obd_put_mod_rpc_slot()
    flags &= ~WQ_FLAG_WOKEN
    list_add()
    wait_woken()
schedule claim_mod_rpc_function()
cl_mod_rpcs_in_flight++
wake_up()

flags &= ~WQ_FLAG_WOKEN

#3: obd_put_mod_rpc_slot()
claim_mod_rpc_function()
cl_mod_rpcs_in_flight++
wake_up()
    list_del()

the patch introduces a replacement for WQ_FLAG_WOKEN which is never
reset once set.

Lustre-change: https://review.whamcloud.com/50261
Lustre-commit: 91a3726f313df33e099320d171039f8371fec27f

Fixes: 5243630b09 ("LU-15947 obdclass: improve precision of wakeups for mod_rpcs")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I29371c8c85414413c5a8e41dec3632f64ad127bb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51539
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 weeks agoLU-15595 tests: Router test interop check and aarch fix 46/51546/3
Chris Horn [Wed, 14 Sep 2022 01:23:37 +0000 (20:23 -0500)]
LU-15595 tests: Router test interop check and aarch fix

setup_router_test() executes load_lnet() on remote nodes, but
this function was only added in 2.15. Add a version check for it.

Enabling routing may fail on nodes with small amount of memory (like
aarch config). Define small number of router buffers to work around
this issue. Modify the functions which calculate the number of buffers
to allow small sizes to be specified via parameters.

Lustre-change: https://review.whamcloud.com/48578
Lustre-commit: 1aba6b0d9b661d3699cbd4624e9db334a13fc647

Test-Parameters: trivial testlist=sanity-lnet serverversion=2.12.9
Test-Parameters: testgroup=review-ldiskfs-arm testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: If0b76747fe09e883546f18da9f3322c72263e29d
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51546
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
4 weeks agoNew tag 2.15.5-RC1 2.15.5-RC1 v2_15_5-RC1
Oleg Drokin [Thu, 30 May 2024 22:10:17 +0000 (18:10 -0400)]
New tag 2.15.5-RC1

Change-Id: I2d863fb40f635e5193660d03b3a5f57ae694f336
Signed-off-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17402 kernel: RHEL 8.10 client and server support 93/55193/4
Jian Yu [Tue, 28 May 2024 23:45:13 +0000 (16:45 -0700)]
LU-17402 kernel: RHEL 8.10 client and server support

This patch makes changes to support RHEL 8.10 release
with kernel 4.18.0-553.el8_10 for Lustre client and server.

Lustre-change: https://review.whamcloud.com/54800
Lustre-commit: TBD (from 6748f47fac79e557ae21eb790b597be6449c926a)

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el8.10 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.10 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.10 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.10 testlist=sanity

Test-Parameters: optional clientdistro=el8.10 serverdistro=el8.10 \
  testgroup=full-part-1

Test-Parameters: optional clientdistro=el8.10 serverdistro=el8.10 \
  testgroup=full-part-2

Test-Parameters: optional clientdistro=el8.10 serverdistro=el8.10 \
  testgroup=full-part-3

Change-Id: I0a9a262d13e0b0de3607da0982468fd8b5f6a7aa
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55193
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17404 kernel: update RHEL 9.4 [5.14.0-427.18.1.el9_4] 04/55204/3
Jian Yu [Tue, 28 May 2024 23:42:16 +0000 (16:42 -0700)]
LU-17404 kernel: update RHEL 9.4 [5.14.0-427.18.1.el9_4]

Update RHEL 9.4 kernel to 5.14.0-427.18.1.el9_4 for Lustre client.

Lustre-change: https://review.whamcloud.com/55203
Lustre-commit: TBD (from 07a23833999207c336532bcf75aa9d5a954f1b07)

Test-Parameters: trivial \
  mdtcount=4 mdscount=2 clientdistro=el9.4 testlist=sanity
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-3

Change-Id: If18027650ff953733f2e57727b71d2daa61d249c
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55204
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17749 kernel: update RHEL 8.9 [4.18.0-513.24.1.el8_9] 57/55157/2
Jian Yu [Mon, 20 May 2024 19:52:09 +0000 (12:52 -0700)]
LU-17749 kernel: update RHEL 8.9 [4.18.0-513.24.1.el8_9]

Update RHEL 8.9 kernel to 4.18.0-513.24.1.el8_9.

Lustre-change: https://review.whamcloud.com/54821
Lustre-commit: TBD (from 23a99efd9104b328ce1edb5fc9094bce2c06e9b9)

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
  testgroup=full-part-1

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
  testgroup=full-part-2

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
  testgroup=full-part-3

Change-Id: I94b5a95e9e85f2f5e0cddb1dbb519ef92520ad0b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55157
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17641 kernel: update RHEL 9.3 [5.14.0-362.24.1.el9_3] 37/55137/2
Jian Yu [Fri, 17 May 2024 07:18:21 +0000 (00:18 -0700)]
LU-17641 kernel: update RHEL 9.3 [5.14.0-362.24.1.el9_3]

Update RHEL 9.3 kernel to 5.14.0-362.24.1.el9_3 for Lustre client.

Lustre-change: https://review.whamcloud.com/54820
Lustre-commit: TBD (from 3448e237b53797044e5b25544667a31ac761a9e9)

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
  clientdistro=el9.3 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el9.3 testlist=sanity

Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-3

Change-Id: Ifafb3fbbfdfcd82506daed44d3601a0d4357331e
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55137
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Alex Deiter
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17404 kernel: new kernel [RHEL 9.4 5.14.0-427.16.1.el9_4] 17/54717/5
Jian Yu [Sat, 11 May 2024 07:27:49 +0000 (00:27 -0700)]
LU-17404 kernel: new kernel [RHEL 9.4 5.14.0-427.16.1.el9_4]

This patch makes changes to support new RHEL 9.4 release
for Lustre client.

Lustre-change: https://review.whamcloud.com/54712
Lustre-commit: TBD (from 177846a0aa58b35d43696b3c3c5d71df0109ab14)

Test-Parameters: trivial \
  mdtcount=4 mdscount=2 clientdistro=el9.4 testlist=sanity
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.4 testgroup=full-part-3

Change-Id: Ic292c01ad16dc06e8dee966c4a211896fea284c0
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54717
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17850 build: prefer LINUXRELEASE over uname -r 15/55115/3
Jian Yu [Wed, 15 May 2024 04:52:41 +0000 (21:52 -0700)]
LU-17850 build: prefer LINUXRELEASE over uname -r

In a container or chroot environment, "uname -r" reports
the host instead of the target kernel version. We should
use the LINUXRELEASE variable which is configured in
config/lustre-build-linux.m4 with the value from UTS_RELEASE.

Lustre-change: https://review.whamcloud.com/55108
Lustre-commit: 0c46ba62efb35b31bb826e5898ffa6e52768e7fa

Test-Parameters: trivial
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Iaa48027f5ae873e1298695a264db1c351d9eac5c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55115
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
4 weeks agoLU-16819 build: use mofed path based on target kernel 17/55117/4
Ake Sandgren [Wed, 15 May 2024 05:09:13 +0000 (22:09 -0700)]
LU-16819 build: use mofed path based on target kernel

Instead of using "uname -r", which limits builds to the currently
running kernel, use the target kernel which is available in
LINUXRELEASE, if the directory is available.
Building for a specific kernel is common practice when using DKMS.

Lustre-change: https://review.whamcloud.com/50937
Lustre-commit: 0e9708016b9948676484d290326c1fe8a269eb80
Test-Parameters: trivial
Signed-off-by: Ake Sandgren <ake.sandgren@hpc2n.umu.se>
Change-Id: Ifce912061a74fc5b7435cd940105190f0c3cd544
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55117
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
4 weeks agoLU-17034 quota: tmp fix against memory corruption 35/55035/2
Sergey Cheremencev [Mon, 8 Apr 2024 11:43:53 +0000 (14:43 +0300)]
LU-17034 quota: tmp fix against memory corruption

Change QMT_INIT_SLV_CNT from 64 to 2000 to avoid accessing
memory out of array lqeg_arr. It could happen when at least
one of OSTs has index larger than the whole number of OSTs.
It is a temporary solution and maximum supported OST index
is 0x7d0. Later it will be changed with the longterm
solution.

Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I8d9444017fa9847142f3df77c63368282ff134c4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55035
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-15163 osd: osd_obj_map_recover() to restart transaction 25/55125/2
Alex Zhuravlev [Tue, 26 Oct 2021 08:38:50 +0000 (11:38 +0300)]
LU-15163 osd: osd_obj_map_recover() to restart transaction

osd_obj_map_recover() stops transaction when need to call
vfs_link() and it has to start a new transaction to modify
filesystem.

Lustre-commit: 7bf0e557a2b3a463e4d78e81b6ab93987d3dc8af
Lustre-change: https://review.whamcloud.com/45368

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I6efe5444ddc959b19092bebc6e3c7dc25a29cea1
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55125
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-16831 lfs: limit stripe count for component size 75/53775/6
Bobi Jam [Mon, 13 May 2024 18:06:37 +0000 (11:06 -0700)]
LU-16831 lfs: limit stripe count for component size

If stripe count is larger than component_size/stripe_size, some
allocated OST objects are created but inaccessible. This patch
reduces the number of stripes in that case to avoid this.

Lustre-change: https://review.whamcloud.com/51143
Lustre-commit: a250ecb959a98c2ec0a01bbca9d943a19b8fa078

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I117ed8a7696c6c6adcdd0c2c6531a958cc53bd51
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53775
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-16297 ptlrpc: don't panic during reconnection 40/55040/2
Alexander Boyko [Thu, 3 Nov 2022 11:23:20 +0000 (07:23 -0400)]
LU-16297 ptlrpc: don't panic during reconnection

ptlrpc_send_rpc() could race with ptlrpc_connect_import_locked()
in the middle of assertion check and this leads to a wrong panic.
Assertion checks

(AT_OFF || imp->imp_state != LUSTRE_IMP_FULL ||

reconnect changes import state and flags
and second part

(imp->imp_msghdr_flags & MSGHDR_AT_SUPPORT) ||
!(imp->imp_connect_data.ocd_connect_flags & OBD_CONNECT_AT)))

MSGHDR_AT_SUPPORT is disabled during client reconnection.
It is not good to use locking at this hot part, so fix changes
assertion to a report.

Lustre-change: https://review.whamcloud.com/49029
Lustre-commit: df31c4c0b39b8845911344e6fadc008bcba40bb1

HPE-bug-id: LUS-10985
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ifc9e413c679c3e8a4c8f4f541251bebabae41c82
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55040
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-16623 tests: ignore sanity-pfl stripe-count off-by-1 83/55083/2
Andreas Dilger [Sun, 14 Apr 2024 05:54:24 +0000 (23:54 -0600)]
LU-16623 tests: ignore sanity-pfl stripe-count off-by-1

In some cases the MDS may not create all stripes on a file, if the
MDT-OST connection does not have precreated objects.  This is OK,
so the tests should not fail the stripe-count check if trying to
create a fully-striped file and one of the stripes is missing.

Lustre-change: https://review.whamcloud.com/54778
Lustre-commit: e715a8c2a616f6d4158decfed5dec2fa444f0c67

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie482fdf86f82e7a2292c021761885249a6c551f1
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55083
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-16515 tests: disable sanity test_118c/118d 77/55077/2
Andreas Dilger [Sat, 11 May 2024 07:20:43 +0000 (00:20 -0700)]
LU-16515 tests: disable sanity test_118c/118d

Temporarily disable sanity test_118c and test_118d until there is
a fix available, since this is failing a large fraction of tests.

Lustre-change: https://review.whamcloud.com/50470
Lustre-commit: 7c52cbf65218d77c0594f92981173aa7d78f6758

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I16ebbc470a126bb99b5c3ecdf93407d6b73ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/55077
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-17362 build: Update ZFS version to 2.1.15 91/54791/3
Jian Yu [Fri, 10 May 2024 00:12:44 +0000 (17:12 -0700)]
LU-17362 build: Update ZFS version to 2.1.15

Update ZFS version to 2.1.15. The changes are listed in:
https://github.com/openzfs/zfs/releases/tag/zfs-2.1.15

Lustre-change: https://review.whamcloud.com/54769
Lustre-commit: 01103eba35e88638d8860457fbdf89b101d4ab67

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el8.9 serverdistro=el8.9 testlist=sanity

Test-Parameters: optional fstype=zfs mdtcount=4 mdscount=2 \
  testgroup=full-dne-zfs-part-1

Test-Parameters: optional fstype=zfs mdtcount=4 mdscount=2 \
  testgroup=full-dne-zfs-part-2

Test-Parameters: optional fstype=zfs mdtcount=4 mdscount=2 \
  testgroup=full-dne-zfs-part-3

Change-Id: I51532dbf9dbcadf64bb9dbd3b10e88d0cab38ffd
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54791
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Olaf Faaland <faaland1@llnl.gov>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
4 weeks agoLU-16350 osd-ldiskfs: no_llseek removed, dquot_transfer 32/54732/3
Shaun Tancheff [Thu, 11 Apr 2024 07:52:06 +0000 (00:52 -0700)]
LU-16350 osd-ldiskfs: no_llseek removed, dquot_transfer

Linux commit v5.19-rc2-6-g868941b14441
  fs: remove no_llseek

With the removal of no_llseek, leaving .llseek set to NULL
is functionally equivalent. Only provide no_llseek if it exists.

Linux commit v5.19-rc3-6-g71e7b535b890
 quota: port quota helpers mount ids

dquot_transfer adds a user namespace argument. Provide an
osd_dquot_transfer() wrapper to discard the additional
argument for older kernels.

Lustre-change: https://review.whamcloud.com/49266
Lustre-commit: 2de1dbd440e2b26ea1bdf663b92a3e8c62a95ee7

Test-Parameters: trivial
HPE-bug-id: LUS-11376
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If3165aed0d7b827b90e26d9f0174137d087ce57a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54732
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
4 weeks agoLU-17587 build: use kernel version from dkms for client 65/54965/3
snehring [Wed, 17 Apr 2024 16:09:17 +0000 (11:09 -0500)]
LU-17587 build: use kernel version from dkms for client

The current behavior of the dkms build for clients is to only build
for the running kernel. This is fine if the other kernels are ABI
compatible with the running kernel because we tell dkms to run
weak-updates as part of the install process. However, if kernels that
are not ABI compatible with the running kernel are installed they
won't be targeted and weak-updates won't add in the modules. This
could be worked around by running 'dkms install' once booted into the
new kernel, but that's additional administrator overhead and not the
assumed behavior for a dkms module.

This modifies the dkms build script to accept the kernel version from
dkms and configure for that version. It also changes the behavior of
dkms wrt lustre to disable weak module updates since we're now
building for individual kernel versions. This will likely result in
longer times to install the client since we're building for each
installed version of the kernel, but it _should_ mean the client is
actually installed for each version.

Lustre-change: https://review.whamcloud.com/54830
Lustre-commit: aa76ae0c21a7e315ecb1fded82f2c793aa26f78c

Signed-off-by: snehring <snehring@iastate.edu>
Change-Id: I55fb1bb7159772d7ecd9d1837e870c7097c02d78
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54965
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
4 weeks agoLU-17062 lnet: Update lnet_peer_*_decref_locked usage 04/54904/2
Shaun Tancheff [Sat, 16 Sep 2023 05:54:54 +0000 (00:54 -0500)]
LU-17062 lnet: Update lnet_peer_*_decref_locked usage

Move decref's to occur after last reference to prevent
use after free.

Lustre-change: https://review.whamcloud.com/52184
Lustre-commit: 60cfceb8c59364f786b31ac36c2c245b9a1e495a

HPE-bug-id: LUS-11799
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2382ece560039383f644b6aee73a9481d6bb5673
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54904
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 weeks agoLU-15515 contrib: epython scripts to python3 conversion 22/54522/4
Oleg Drokin [Thu, 21 Mar 2024 19:52:21 +0000 (12:52 -0700)]
LU-15515 contrib: epython scripts to python3 conversion

This is to allow them to work with newer pykdump versions
that are actually current and therefore desirable.

Lustre-change: https://review.whamcloud.com/46442
Lustre-commit: 4f5eb6cfeae20dd14a1eb441515892ac8c27d196

Test-Parameters: trivial
Change-Id: I84a1be5f24c04b61d2ab1f8d1744e3ef752ce8c6
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54522
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 weeks agoLU-16382 config: ensure lutf.sh is included in dist 99/53299/4
Mr NeilBrown [Thu, 21 Mar 2024 19:39:11 +0000 (12:39 -0700)]
LU-16382 config: ensure lutf.sh is included in dist

The official 2.15.1 source distribution does not contain
lutf.sh.  As lustre.spec lists it (when LUTF is enabled) this causes a
build error.

It is likely not included because "./configure --enable-dist" was run
in a context where swig was not installed.

So when determining whether to enable lutf, first check for
enable_dist and in the case for enable_lutf="yes"

Lustre-change: https://review.whamcloud.com/49382
Lustre-commit: b486347abdb2f0c12f01deb5424c41a532c952a8

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: If5f856985a6d642822baba4b6ee301c04f851217
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53299
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 weeks agoLU-16502 lutf: fix bugs in bash scripts 19/54519/3
Timothy Day [Thu, 21 Mar 2024 19:06:51 +0000 (12:06 -0700)]
LU-16502 lutf: fix bugs in bash scripts

Addressed some issues I found when running
LUTF. The "rm" fails without a file to remove.
A Lustre wiki led me to source the script, but this
will log out of the current shell. Adding a warning
against doing this.

Also, fix shellcheck errors and a few warnings in
LUTF related scripts. Added bash shebangs, since
shellcheck requires these to lint the scripts.

Lustre-change: https://review.whamcloud.com/49728
Lustre-commit: 3cd0bb6968fc3432b7d1641bf0489e4e4b1e809f

Test-Parameters: trivial env=SANITY_EXCEPT="101j" testlist=sanity
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I501d58d25bfcd6564755485b9a1afa2277848b96
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54519
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 weeks agoLU-17010 lfsck: don't dump stack repeatedly 53/54853/2
Andreas Dilger [Wed, 13 Sep 2023 05:12:18 +0000 (23:12 -0600)]
LU-17010 lfsck: don't dump stack repeatedly

If there are transactions started with LFSCK in dry-run mode, don't
dump the stack repeatedly, as this can spam the console logs and
significantly hurt performance.

Lustre-commit: dc360cd3eff20618f243ab89097a62f8ecf2c929
Lustre-change: https://review.whamcloud.com/52356

Test-Parameters: trivial testlist=sanity-lfsck
Fixes: 0c1ae1cb9c ("LU-13124 scrub: check for multiple linked file")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0b0d64911453dc8ab947e284656311b5d0300c1e
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54853
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 weeks agoLU-15851 lnet: Adjust niov checks for large MD 48/54848/2
Chris Horn [Sat, 16 Apr 2022 16:01:57 +0000 (10:01 -0600)]
LU-15851 lnet: Adjust niov checks for large MD

An LNet user can allocate a large contiguous MD. That MD can have >
LNET_MAX_IOV pages which causes some LNDs to assert on either niov
argument passed to lnd_recv() or the value stored in
lnet_msg::msg_niov. This is true even in cases where the actual
transfer size is <= LNET_MTU and will not exceed limits in the LNDs.

Adjust ksocklnd_send()/ksocklnd_recv() to assert on the return value
of lnet_extract_kiov().

Remove the assert on msg_niov (payload_niov) from kiblnd_send().
kiblnd_setup_rd_kiov() will already fail if we exceed ko2iblnd's
available scatter gather entries.

Lustre-change: https://review.whamcloud.com/47319
Lustre-commit: 105193b4a147257a0f9332053a16eb676dc99623

HPE-bug-id: LUS-10878
Test-Parameters: trivial
Fixes: 857f11169f ("LU-13004 lnet: always put a page list into struct lnet_libmd")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Iaa851d90f735d04e5167bb9c07235625759245b2
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54848
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 weeks agoLU-17510 obdclass: fix wake up when queuing close request. 89/54689/3
Mr NeilBrown [Mon, 4 Mar 2024 02:15:17 +0000 (13:15 +1100)]
LU-17510 obdclass: fix wake up when queuing close request.

The waitqueue for requests that need to be sent but that haven't been
allocated a slot is kept ordered by request arrival for fairness.  So
new requests are added to the end.

For requests other than 'close' there is a limit to the number of
active requests (slots) and requests are assigned to slot on a
first-come-first-served basis, so they are simply removed from the
head of the list.

For 'close' requests it is important that these not block indefinitely
behind other other requests so there is one slot that can only be used
by a close request - and only if no other slots are used by a close
request.  These requests do not follow a strict FIFO order.

When a non-"close" request completes we wake the first request on the
list.  There is no point searching all the way down the list for a
close request that could also be woken.  We only do that when a
"close" request completes.  This optimises the common case.

However: when a request is first queued we add it to the end of the
queue and then wake up the first deserving request if there is one.
When there are free slots, this is expected to wake the request just
queued.  When there are no free slots, nothing is woken.

When a "close" request is queued and added to the end of the queue
after other non-close requests, we need to potentially search to the
end of the queue for a close request to wake, just as we do when a
close request completes.  Unfortunately we don't.  This can result in
a close request blocking indefinitely.

So: change the wakeup in obd_get_mod_rpc_slot() to match the wakeup in
obd_put_mod_rpc_slot().  This ensure consistent handling and in
particular will handle a close request immediately if there are no
other close requests in flight.

Clarify comment in claim_mod_rpc_function() to make and perform minor
code cleanup there.

Lustre-change: https://review.whamcloud.com/54259
Lustre-commit: 7a2296a397381a5f6f9473b297f0062e8ff15948

Fixes: b5fde4d6c023 ("LU-17197 obdclass: preserve fairness when waiting for rpc slot")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I7b658efc0298a091166f0f18ce460fc3148047eb
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54689
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 weeks agoLU-15947 obdclass: improve precision of wakeups for mod_rpcs 38/51538/4
Mr NeilBrown [Mon, 21 Jun 2021 03:25:42 +0000 (13:25 +1000)]
LU-15947 obdclass: improve precision of wakeups for mod_rpcs

There is a limit of the number of in-flight mod rpcs with a
complication that a 'close' rpc is always permitted if there are no
other close rpcs in flight, even if that would exceed the limit.

When a non-close-request complete, we just wake the first waiting
request and assume it will use the slot we released.  When a
close-request completes, the first waiting request may not find a slot
if the close was using the 'extra' slot.  So in that case we wake all
waiting requests and let them fit it out.  This is wasteful and
unfair.

To correct this we revise the wait/wake approach to use a dedicated
wakeup function which atomically checks if a given task can proceed,
and updates the counters when permission to proceed is given.  This
means that once a task has been woken, it has already been accounted
and it can proceed.

To minimise locking, cl_mod_rpcs_lock is discarded and
cl_mod_rpcs_waitq.lock is used to protect the counters.  For the
fast-path where the max has not been reached, this means we take and
release that spinlock just once.  We call wake_up_locked while still
holding the lock, and if that woke the process, then we don't drop the
spinlock to wait, but proceed directly to the remainder of the task.

When the last 'close' rpc completes, the wake function will iterate
the whole wait queue until it finds a task waiting to submit a close
request.  When any other rpc completes, the queue will only be
searched until the maximum is reached.

Lustre-change: https://review.whamcloud.com/44041
Lustre-commit: 5243630b09d22e0b576d81390d604774881f63f7

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Iff094c3188a3bd8a04edc1d5d98ec3014e2b059b
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51538
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 weeks agoLU-17409 scripts: correct ldev MGS handling 67/54667/2
Olaf P. Faaland [Tue, 9 Jan 2024 05:36:50 +0000 (21:36 -0800)]
LU-17409 scripts: correct ldev MGS handling

ldev was incorrectly parsing the line specifying the hosts that can
run the MGS, when of the form

    gopher1  gopher2  MGS  gopher1/mgs

as it assumed every target specified included a filesystem name,
like 'lustre3-MDT0000'.

This corrects that, assuming that an MGS may not be related to a
specific file system.

When such an input line is found, assume that MGS is used by any file
systems included in the ldev.conf.  When user includes option '-F
<fsname>' as well as '-R MGS', include that MGS, in the targets
reported.

Lustre-change: https://review.whamcloud.com/53619
Lustre-commit: bb6a2d2e80f04645b488ecca6ba14cb628e3eeb3

Test-Parameters: trivial
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: Ifab5db1dfb094755e29747ec6b90d1566b16c18c
Reviewed-by: Cameron Harr <charr@llnl.gov>
Reviewed-by: Eric Carbonneau <carbonneau1@llnl.gov>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54667
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 weeks agoLU-10680 tests: fix interop for sanity test_160h 73/53773/5
Xing Huang [Thu, 27 Oct 2022 11:41:11 +0000 (19:41 +0800)]
LU-10680 tests: fix interop for sanity test_160h

Add a check sanity test_160h whether /sbin/umount.lustre is installed
on the MDS, since this subtest is checking whether the MDS unmount
process has completed, and otherwise fails during interop testing.

Test-Parameters: testlist=sanity env=ONLY=160 serverversion=2.12
Fixes: 31fef6845e8b ("LU-10680 mdd: create gc thread when no current transaction")
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: I6720b9e27a3a92e543ed877453802d23c0eef36d
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53773
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 weeks agoLU-16351 llite: Linux 6.1 prandom, folios_contig, vma_iterator 00/54700/3
Shaun Tancheff [Wed, 10 Apr 2024 07:45:22 +0000 (00:45 -0700)]
LU-16351 llite: Linux 6.1 prandom, folios_contig, vma_iterator

Linux commit v4.10-rc3-6-gc440408cf690
  random: convert get_random_int/long into get_random_u32/u64
Linux commit v6.0-11338-gde492c83cae0
  prandom: remove unused functions

prandom_u32 is a wrapper around get_random_u32, change users
of prandom_u32 to get_random_u32 and provide a fallback
to prandom_u32 when get_random_u32 is not available.

Linux commit v6.0-rc1-2-g25885a35a720
  Change calling conventions for filldir_t
Add a test for the new filldir_t signature
Provide wrappers for transition from int (error code) to bool

Linux commit v6.0-rc3-94-g35b471467f88
  filemap: add filemap_get_folios_contig()
Provide a wrapper and fallback to find_get_pages_contig

Linux commit v6.0-rc3-225-gf39af05949a4
  mm: add VMA iterator
Use vma_iterator and for_each_vma when available.

Lustre-change: https://review.whamcloud.com/49232
Lustre-commit: ca992899d55fd13e65b75ace02931daaa29c18bd

Test-Parameters: trivial
HPE-bug-id: LUS-11377
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I23dc23d0252e1995555b6685f5cf7c207edf642b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54700
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 months agoLU-17664 lnet: LNetPrimaryNID returns wrong NID 37/54537/2
Chris Horn [Fri, 22 Mar 2024 21:02:55 +0000 (15:02 -0600)]
LU-17664 lnet: LNetPrimaryNID returns wrong NID

This is a regression introduced by the b2_15 port of

Lustre-change: https://review.whamcloud.com/50159
Lustre-commit: fc7a0d6013b46ebc17cdfdccc04a5d1d92c6af24

This port assigned the return value to the wrong variable. As a
result, LNetPrimaryNID() will always return the same value that
was passed in as an argument. Fix the issue by assigning the
return value to the correct variable.

Test-Parameters: trivial
Fixes: 6cfc8e55a2 ("LU-14668 lnet: add 'lock_prim_nid" lnet module parameter")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I10e334d38e8c257239ae3fa82d0aea4f87d8ba69
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54537
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 months agoLU-17226 build: create config option for l_getsepol 90/54190/2
Gian-Carlo DeFazio [Thu, 16 Nov 2023 23:05:45 +0000 (15:05 -0800)]
LU-17226 build: create config option for l_getsepol

Add a configuration option for l_getsepol.
l_getsepol is build by default unless the --disable-l_getsepol
option is given to configure.
lustre.spec.in builds l_getsepol by default and has its
dependencies as build requirements.

The implicit configuration check for the dependency
openssl-devel is removed and replaced by a BuildRequires.

Lustre-change: https://review.whamcloud.com/52849
Lustre-commit: 2777adcabd1032ddb886f913fa04d82a292ab379

Test-Parameters: trivial
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Change-Id: If71a2a4a524047edbd2b31e6fac7a42f36a030bf
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54190
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Olaf Faaland <faaland1@llnl.gov>
2 months agoLU-14992 tests: sanity/replay-vbr mkdir on MDT0 04/49404/9
James Nunez [Mon, 13 Sep 2021 16:35:30 +0000 (10:35 -0600)]
LU-14992 tests: sanity/replay-vbr mkdir on MDT0

Replace mkdir with mkdir_on_mdt0() for sanity test 133a
and relay-vbr test 7a.  These tests expect the newly
created directory is on MDT0.

Lustre-change: https://review.whamcloud.com/44902
Lustre-commit: f0324c5c2f4390d6d7e93ed799e95d8eef4704f4

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=sanity
Test-Parameters: env=SLOW=yes mdscount=2 mdtcount=4 testlist=replay-vbr
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Icea2923a8d8d3a3aa0ddf0401f0a025480b2f6f0
Reviewed-by: Kevin Zhao <kevin.zhao@linaro.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49404
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-17175 tests: fix sanity-krb5 test_5 in interop mode 03/54103/3
Sebastien Buisson [Tue, 20 Feb 2024 09:58:40 +0000 (10:58 +0100)]
LU-17175 tests: fix sanity-krb5 test_5 in interop mode

With newer servers, the lsvcgssd daemon is automatically restarted
when the server needs to process a gss request. So do not error out
if a file system access succeeds whereas the daemon was explicitly
killed.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I52ec49ca584f07100699274fe3a52ea28d227c17
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54103
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 months agoLU-16101 tests: skip sanity/27J for more kernels 36/54036/2
Andreas Dilger [Tue, 4 Jul 2023 23:08:03 +0000 (17:08 -0600)]
LU-16101 tests: skip sanity/27J for more kernels

This is a bug in the kernel that is not present in older kernels
before commit v5.11-10234-gcbd59c48ae2b (5.12), and is fixed with
commit v6.2-rc4-61-g5956592ce337 (6.2).

Move this from ALWAYS_EXCEPT (bug that needs to be fixed) to skip
(test that is known to fail in some configs but has been fixed).

Lustre-change: https://review.whamcloud.com/51567
Lustre-commit: b711af7d243f3773cec3a37f64c0e0aa8bbc363f

Fixes: 63dd644 ("LU-16101 tests: add sanity/27J to always_except")
Test-Parameters: trivial testlist=sanity clientdistro=el9.2 env=ONLY=27J
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8ec0a6d25a90e05672b039cd6c2b2fbf8a3ebbe5
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54036
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 months agoLU-15655 contrib: update branch_comm to python3 47/53947/2
John L. Hammond [Thu, 17 Mar 2022 13:55:32 +0000 (08:55 -0500)]
LU-15655 contrib: update branch_comm to python3

Update the branch_comm script from to python2 to python3.

Lustre-change: https://review.whamcloud.com/46856
Lustre-commit: c6b257254878387976247fea29bd2b142cd03e5e

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Icbb747c01e15cbe77c6dd6df3e913654ded84239
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53947
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-16019 llite: fully disable readahead in kernel I/O path 19/48219/4
Qian Yingjin [Mon, 15 Aug 2022 18:15:25 +0000 (11:15 -0700)]
LU-16019 llite: fully disable readahead in kernel I/O path

In the new kernel (rhel9 or ubuntu 2204), the readahead path may
be out of the control of Lustre CLIO engine:

generic_file_read_iter()
  ->filemap_read()
    ->filemap_get_pages()
      ->page_cache_sync_readahead()
        ->page_cache_sync_ra()

void page_cache_sync_ra()
{
if (!ractl->ra->ra_pages || blk_cgroup_congested()) {
if (!ractl->file)
return;
req_count = 1;
do_forced_ra = true;
}

/* be dumb */
if (do_forced_ra) {
force_page_cache_ra(ractl, req_count);
return;
}
...
}

From the kernel readahead code, even if read-ahead is disabled
(via @ra_pages == 0), it still issues this request as read-ahead
as we will need it to satisfy the requested range. The forced
read-ahead will do the right thing and limit the read to just
the requested range, which we will set to 1 page for this case.

Thus it can not totally avoid the read-ahead in the kernel I/O
path only by setting @ra_pages with 0.
To fully disable the read-ahead in the Linux kernel I/O path, we
still need to set @io_pages to 0, it will set I/O range to 0 in
@force_page_cache_ra():
void force_page_cache_ra()
{
...
max_pages = = max_t(unsigned long, bdi->io_pages,
    ra->ra_pages);
nr_to_read = min_t(unsigned long, nr_to_read, max_pages);
while (nr_to_read) {
...
}
...
}

After set bdi->io_pages with 0, it can pass the sanity/101j.

Lustre-change: https://review.whamcloud.com/47993
Lustre-commit: f0cf7fd3cccb2313fa94a307cf862afba256b8d8

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I859a6404abb9116d9acfa03de91e61d3536d3554
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48219
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 months agoLU-13135 quota: improve checks in OSDs to ignore quota 76/51776/4
Alex Zhuravlev [Tue, 14 Jan 2020 19:38:51 +0000 (22:38 +0300)]
LU-13135  quota: improve checks in OSDs to ignore quota

for root-owned files.

sanity/60a:
  zfs before 80s, after 66s
  ldiskfs before 65s, after 38s

ave.write declaration in sanity/60a:
  zfs before 3.21 usec, after 1.16 usec
  ldiskfs before 4.06 usec, after 0.66 usec

Note that this patch is related to bug LU-15726.
Lustre-change: https://review.whamcloud.com/37232
Lustre-commit: ef90a02d126312b0d2f28b8c6eb2d299c68eb936

Change-Id: Ib9ba50d260eac408f1f5e43c4d722ff5024135cf
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51776
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-16772 quota: protect lqe_glbl_data in qmt_site_recalc_cb 84/53284/2
Sergey Cheremencev [Tue, 25 Apr 2023 18:10:21 +0000 (22:10 +0400)]
LU-16772 quota: protect lqe_glbl_data in qmt_site_recalc_cb

lqe_glbl_data should be protected with lqe_glbl_data_lock in
qmt_site_recalc_sb like it did in other places. Otherwise it
may cause following panic:

  BUG: unable to handle kernel NULL pointer at 00000000000000f8
  qmt_site_recalc_cb+0x2f8/0x790 [lquota]
  cfs_hash_for_each_tight+0x121/0x310 [libcfs]
  qmt_pool_recalc+0x372/0x9f0 [lquota]

Also protect lqe_glbl_data access with lqe_glbl_data_lock in
qmt_lvbo_free().

Lustre-change: https://review.whamcloud.com/50748
Lustre-commit: 50ff4d1da63e8bc1dba4b6b52219fb7024f8d66f

Fixes: 1dbcbd70f8 ("LU-15021 quota: protect lqe_glbl_data in lqe")
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I030f14b02062151f1708a03ac7414a9991f798f6
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53284
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
2 months agoLU-16057 obdclass: set OBD_MD_FLGROUP for ladvise RPC 01/53501/2
Li Dongyang [Fri, 29 Jul 2022 06:35:41 +0000 (16:35 +1000)]
LU-16057 obdclass: set OBD_MD_FLGROUP for ladvise RPC

ladvise RPC doesn't have OBD_MD_FLGROUP set, when RPC
reaches server, tgt_validate_obdo() will corrupt the FID
if it's seq is in FID_SEQ_NORMAL range.

Do not mess with seq in obdo_to_ioobj() and tgt_validate_obdo(),
since 2.0 all RPCs should have OBD_MD_FLGROUP set.

Add OBD_MD_FLGROUP for ladvise RPC to fix new client talking
to old servers.

Lustre-change: https://review.whamcloud.com/48080
Lustre-commit: bee803c6e440ba6b55e0ff356e5324f44cfa63eb

Change-Id: I373b7f32458b18e29d9bb716a912fe4a54eccac5
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53501
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-16076 utils: enhance 'lfs check' command 65/51365/3
Lei Feng [Mon, 8 Aug 2022 02:59:25 +0000 (10:59 +0800)]
LU-16076 utils: enhance 'lfs check' command

Add optional argument to 'lfs check' command so that only the
servers related to the specified lustre file system is checked.

Lustre-change: https://review.whamcloud.com/48155
Lustre-commit: f5ca6853b8d8b918b0228af31fa8249be49d3000

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanityn env=ONLY=113
Change-Id: I826a8e822af0a290f06ffaadadf1bb7f86899d99
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51365
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-15207 libcfs: reset hs_rehash_bits 83/53283/2
Alex Zhuravlev [Thu, 11 Nov 2021 08:19:46 +0000 (11:19 +0300)]
LU-15207 libcfs: reset hs_rehash_bits

if rehash work is cancelled, then nobody resets
hs_rehash_bits and the first iterator asserts
at LASSERT(!cfs_hash_is_rehashing(hs)) in
cfs_hash_for_each_relax().

Lustre-change: https://review.whamcloud.com/45533
Lustre-commit: 9257f24dfdf9f0a68512fce52d79064f78d9dc88

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I1a567f6be77ca6c45e5d4f256722206b12588554
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53283
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-14824 test: sanity 413a/b unlink timeout v2 61/51561/2
Etienne AUJAMES [Tue, 4 Jul 2023 15:16:31 +0000 (17:16 +0200)]
LU-14824 test: sanity 413a/b unlink timeout v2

Unlinking remote/striped directories is slow on ZFS system, limit
total directory number for 1-stripe directory test in 413a/b on ZFS
system, and don't test striped directory to avoid timeout.

Also limit total stripe object count to avoid timeout.

Use fallocate to fill the MDTs if it is supported.  Add a new helper
function check_fallocate() that just determines if fallocate support
is available on the OST without trying to change the mode.  This is
cached across calls to avoid repeated SSH calls to get information
that is the same for the entire test run.

Lustre-change: https://review.whamcloud.com/49799
Lustre-commit: 09fc9ccb1acc534b3fb074433dcbcecebb175384

The backport use parts of 37c1ddc ("LU-15850 lmv: always space-balance
r-r directories") to merge sanity.sh.

Test-Parameters: trivial
Test-Parameters: mdscount=2 mdtcount=4 fstype=ldiskfs testlist=sanity env=ONLY="413a 413b",ONLY_REPEAT=50
Test-Parameters: mdscount=2 mdtcount=4 fstype=zfs testlist=sanity env=ONLY="413a 413b",ONLY_REPEAT=50
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ie116e6df5aee3877ed9f093f58e7bd71f63ebbe5
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51561
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-14073 ldiskfs: don't test LDISKFS_IOC_FSSETXATTR 83/52283/3
Mr NeilBrown [Fri, 9 Dec 2022 05:31:13 +0000 (16:31 +1100)]
LU-14073 ldiskfs: don't test LDISKFS_IOC_FSSETXATTR

EXT4_IOC_FSSETXATTR was removed upstream in Linux 5.9, Commit
cb29a02d3a9d ("ext4: use generic names for generic ioctls").
So we cannot use it to test if project quotas are supported.

Instead test if EXT4_MAXQUOTAS is 3.  This was changed to 3 upstream
in the commit immediately before EXT4_IOC_FSSETXATTR was added, so it
is effectively the same test.

Lustre-change: https://review.whamcloud.com/49353
Lustre-commit: 40389067f5645fed903304d19dd6c58de72d0b88

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I88c51c03959ebe98cd5066596f5158fac570a625
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52283
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
2 months agoLU-15595 tests: Add various router tests 43/51543/2
Chris Horn [Mon, 7 Feb 2022 23:20:37 +0000 (23:20 +0000)]
LU-15595 tests: Add various router tests

Add test cases to exercise LNet routing.

Lustre-change: https://review.whamcloud.com/46622
Lustre-commit: 8ee85e15412d32fbe60f70c474c0a28ff15b8351

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I4a077937b3e3b8b07707afeb0c5c23ec1c9074f4
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51543
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-16894 lod: fix stripe_count limit in lod_qos_set_pool 27/51527/2
Etienne AUJAMES [Wed, 14 Jun 2023 08:02:01 +0000 (10:02 +0200)]
LU-16894 lod: fix stripe_count limit in lod_qos_set_pool

For a conflicting pool name and stripe offset parameter, the MDS
should not set the pool on the file layout in favor of the stripe
offset (LU-15658). But lod_qos_set_pool() still limit the
stripe_count to the number of OSTs in the specified pool.

Let's fix this.

Lustre-change: https://review.whamcloud.com/51314
Lustre-commit: d6df160877cda3ff1e0b3ce5c4df46fa17e1468a

Fixes: 06dd5a4 ("LU-15658 lod: ost list and pool name conflict")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ic47f9aadd8feea01367e526aaf0ea41a69ade9fa
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51527
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
2 months agoLU-15658 lod: ost list and pool name conflict 85/51285/2
Vitaly Fertman [Wed, 30 Mar 2022 19:11:42 +0000 (22:11 +0300)]
LU-15658 lod: ost list and pool name conflict

If the OST list is given on setstripe with the -o option, the pool is
unconditionally dropped even if all the OSTs are in. Let the pool stay
in this case.

Also, if the start index given on setstripe with the -i option is out
of the pool, make it similar to the -o option - drop the pool.

lustre-change: https://review.whamcloud.com/46967
Lustre-commit: 06dd5a4638dd36640b146d4388c09a322873760b

HPE-bug-id: LUS-10868
Fixes: b384ea39e5 ("LU-14480 pool: wrong usage with ost list")
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Change-Id: I718c237e273689048eb74044eea73de6c212395e
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51285
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoNew release 2.15.4 2.15.4 v2_15_4
Oleg Drokin [Sat, 23 Dec 2023 03:15:24 +0000 (22:15 -0500)]
New release 2.15.4

Change-Id: I5de619560b371d4aec58caa777f67bdb8c2c4c64
Signed-off-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13791 mdt: parameter to tune capabilities 39/53539/4
Andreas Dilger [Fri, 22 Dec 2023 18:59:59 +0000 (11:59 -0700)]
LU-13791 mdt: parameter to tune capabilities

Add mdt.*.enable_cap_mask to allow specific capabilities to
be enabled and disabled individually.

Fixes: f05edf8e2b ("LU-13791 sec: enable FS capabilities")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6fc0130a90693d673d8c2158e7e31c2de951553d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53539
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-17366 kernel: update SLES15 SP5 [5.14.21-150500.55.39.1] 08/53508/3
Jian Yu [Tue, 19 Dec 2023 18:21:08 +0000 (10:21 -0800)]
LU-17366 kernel: update SLES15 SP5 [5.14.21-150500.55.39.1]

Update SLES15 SP5 kernel to 5.14.21-150500.55.39.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/53467
Lustre-commit: TBD (from 7084f80ec256f6a7335fe4d5981db1e8bcbed440)

Test-Parameters: trivial mdtcount=4 mdscount=2 env=SANITY_EXCEPT="101j" \
clientdistro=sles15sp5 testlist=sanity

Change-Id: Id9476e8726728b00d4079cdaf31b081f89190eb1
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53508
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-17132 kernel: update RHEL 8.8 [4.18.0-477.27.1.el8_8] 91/53491/3
Jian Yu [Mon, 18 Dec 2023 17:40:30 +0000 (09:40 -0800)]
LU-17132 kernel: update RHEL 8.8 [4.18.0-477.27.1.el8_8]

Update RHEL 8.8 kernel to 4.18.0-477.27.1.el8_8.

Lustre-change: https://review.whamcloud.com/52422
Lustre-commit: d1fadf8e8a865ce094b90a7691db58ed0d3963fe

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity

Change-Id: I4edd823b273c75618bc6dea236be8d64ed7c13ed
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53491
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-17338 kernel: update RHEL 8.9 [4.18.0-513.9.1.el8_9] 61/53361/2
Jian Yu [Thu, 7 Dec 2023 07:57:48 +0000 (23:57 -0800)]
LU-17338 kernel: update RHEL 8.9 [4.18.0-513.9.1.el8_9]

Update RHEL 8.9 kernel to 4.18.0-513.9.1.el8_9.

Lustre-change: https://review.whamcloud.com/53357
Lustre-commit: TBD (from 5574088906d813c8a17237edc85e55c5d54f10f5)

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
testgroup=full-part-1

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
testgroup=full-part-2

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
testgroup=full-part-3

Change-Id: Ied0d2873974a3c8cc6e346373457c8ebc09740d6
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53361
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoNew RC 2.15.4-RC2 2.15.4-RC2 v2_15_4-RC2
Oleg Drokin [Wed, 20 Dec 2023 04:18:20 +0000 (23:18 -0500)]
New RC 2.15.4-RC2

Change-Id: Iec09eb04a1d97d5115c18776a66f92310f4a02b2
Signed-off-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-16430 ptlrpc: racy rq_obsolete bit modification 38/52338/2
Andriy Skulysh [Thu, 24 Nov 2022 13:18:04 +0000 (15:18 +0200)]
LU-16430 ptlrpc: racy rq_obsolete bit modification

Racy bit modification causes assertion failure in
ptlrpc_at_remove_timed():
ASSERTION( !list_empty(&req->rq_srv.sr_timed_list) )

rq_obsolete is a bit field, so it's modification
isn't atomic and should be modified under rq_lock.

Lustre-change: https://review.whamcloud.com/49505
Lustre-commit: 14ac768fd9633c5cf4474555170e5042c71a135b

Change-Id: Ib1d3ad189a78b71ecf5b01585478922e984c9568
HPE-bug-id: LUS-11368
Fixes:  23773b32bf ("LU-11444 ptlrpc: resend may corrupt the data")
Signed-off-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52338
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
6 months agoLU-16662 autoconf: fix configure test compile for CONFIG_KEYS 68/50568/3
Xinliang Liu [Fri, 7 Apr 2023 06:24:04 +0000 (23:24 -0700)]
LU-16662 autoconf: fix configure test compile for CONFIG_KEYS

This fixes below configure error on Linux v5.19+:
$ ./configure --disable-server
...
checking whether to enable gss keyring backend... yes
checking if Linux kernel was built with CONFIG_KEYS in or as module...
no
configure: WARNING: GSS keyring backend requires that CONFIG_KEYS be
enabled in your kernel.
checking for keyctl_search in -lkeyutils... yes
configure: error: Cannot enable gss_keyring. See above for details.
$ grep CONFIG_KEYS -rn /boot/config-*
6884:CONFIG_KEYS=y

For in-tree IB support and without passing Linux src path when run
./configure, the LINUX_OBJ maybe just a soft link to O2IBPATH, they
are pointing to the same dir. E.g.:
O2IBPATH='/usr/src/kernels/6.1.8-3.0.0.7.oe1.aarch64'
LINUX_OBJ='/lib/modules/6.1.8-3.0.0.7.oe1.aarch64/build'
$ ls -l /lib/modules/6.1.8-3.0.0.7.oe1.aarch64/build
lrwxrwxrwx 1 root root 42 Feb  7 00:00
/lib/modules/6.1.8-3.0.0.7.oe1.aarch64/build ->
/usr/src/kernels/6.1.8-3.0.0.7.oe1.aarch64
In this case, current configure will put kernel's Module.symvers to
variable KBUILD_EXTRA_SYMBOLS. This should be avoided after kernel
v5.19 which contains commit "b8422711080f modpost: make multiple export
error". This making multiple export symbol as an error from a warning
which can be seen in the config.log:
...
ERROR: modpost: vmlinux: 'init_uts_ns' exported twice. Previous export
was in vmlinux
...

Lustre-change: https://review.whamcloud.com/50399
Lustre-commit: 321a533b868908f37d01a4b787f5a463a02e427c

Test-Parameters: trivial
Change-Id: I35295b3acc7fffb93716362f5d8c659eb922afcb
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50568
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-15816 tests: use correct ost host to manage failure 66/52066/3
Mr NeilBrown [Fri, 25 Nov 2022 05:13:20 +0000 (16:13 +1100)]
LU-15816 tests: use correct ost host to manage failure

sanity test_398m sets up striping across 2 OSTs.  It ensures that
failing IO to either OST individually will fail the total IO.

However it sends the command to fail IO for the second OST (OST1) to
the host managing the first OST (ost1).  If the first 2 OSTs are on
the same host, this works.  If not, it fails.

Also there error messages when testing the second stripe say "first
stripe".

Lustre-change: https://review.whamcloud.com/49248
Lustre-commit: 6e66cbdb5c8c08193c36262649667747127b6d90

Test-Parameters: trivial env=ONLY=398m
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic7085dab2610fa2c044a966fd8de40def0438ca4
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52066
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-16950 git: Fix git review branch for b2_15 11/51611/4
Xinliang Liu [Mon, 10 Jul 2023 08:33:00 +0000 (08:33 +0000)]
LU-16950 git: Fix git review branch for b2_15

The push branch for b2_15 should be b2_15.

Change-Id: Ib6bcafaee3f1dc2934b8591f253a34a663f4ec50
Test-Parameters: trivial
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51611
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14651 build: fix build for el7.9 kernels 04/53504/4
Andrew Perepechko [Mon, 18 Dec 2023 18:19:26 +0000 (11:19 -0700)]
LU-14651 build: fix build for el7.9 kernels

Handle extra setattr_prepare() argument added in Linux 5.12 kernels
when building on older kernels.

Lustre-change: https://review.whamcloud.com/53503
Lustre-commit: 7815835d21a5c0b6dbc58d9bc9dd823d4952f86f

HPE-bug-id: LUS-12059
Signed-off-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Change-Id: Ie7fd1c4d51b7a9b086cfca0db941321cbcce7057
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53504
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-17360 kernel: update RHEL 9.3 [5.14.0-362.13.1.el9_3] 36/53436/2
Jian Yu [Wed, 13 Dec 2023 08:41:19 +0000 (00:41 -0800)]
LU-17360 kernel: update RHEL 9.3 [5.14.0-362.13.1.el9_3]

Update RHEL 9.3 kernel to 5.14.0-362.13.1.el9_3 for Lustre client.

Lustre-change: https://review.whamcloud.com/53433
Lustre-commit: TBD (from 3662949bcd342a96f8dddcb6663872e870f9871b)

Test-Parameters: trivial env=SANITY_EXCEPT="27J 101j" \
  mdtcount=4 mdscount=2 clientdistro=el9.3 testlist=sanity
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-3

Change-Id: I35863d298a612d7913d39f9031e792808f204ad4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53436
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14992 tests: add more mkdir_on_mdt0 calls 65/52065/4
Mr NeilBrown [Sun, 27 Nov 2022 20:49:50 +0000 (07:49 +1100)]
LU-14992 tests: add more mkdir_on_mdt0 calls

A previous patch changed some mkdir calls in test_133a to
mkdir_on_mdt0. This allows stats collected from mdt0 to
reflect the mkdir.

However two mkdir calls were missed, so "crossdir_rename" stats can be
wrong.

Lustre-change: https://review.whamcloud.com/49252
Lustre-commit: d56ea0c80a959ebd9b393f2da048cc179cb16127

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=sanity env=ONLY=133a

Fixes: f0324c5c2f ("LU-14992 tests: sanity/replay-vbr mkdir on MDT0")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4e5c2e5504307462bff4012a13ef9deb24f8da8c
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52065
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-16168 tests: fix get_slave_nr in sanity-quota 76/51576/2
Sergey Cheremencev [Mon, 17 Apr 2023 14:44:22 +0000 (17:44 +0300)]
LU-16168 tests: fix get_slave_nr in sanity-quota

Redirect output of wait_update_facet in get_slave_nr
to /dev/null/. Otherwise if wait_update_cond prints
something while waiting, get_slave_nr returns this
output together with slave number causing test_68
to fail.

Lustre-change: https://review.whamcloud.com/50659
Lustre-commit: 2e5941f94e496559235fa568b48ed03954db2dba

Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota fstype=zfs
Fixes: 83dd308db5 ("LU-15460 test: wait quota pool to be prepared")
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I813ed31db864897372c7eb6aab4e1f5a4b955f49
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51576
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-16541 tests: Improve test 64f 96/52096/2
Patrick Farrell [Tue, 22 Aug 2023 16:32:52 +0000 (12:32 -0400)]
LU-16541 tests: Improve test 64f

The buffered IO part of test 64f has several timing related
holes and other oddities.  The use of multiop in the
background does not guarantee the RPC will not be sent, AND
the test doesn't kill it correctly.

Clean this up and make a more reliable version of the test.
Hopefully this will resolve the failure issues, if not, a
better version of the test will allow debugging.

Lustre-commit: 33e4d86a480b860e0a3b4b51c7c6da6ec0159e51
Lustre-change: https://review.whamcloud.com/52040

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I25b825e1d9d516635ef8cbd26dd12809625c34df
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52096
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-15495 tests: fixed dbench test 84/51284/3
Alex Deiter [Wed, 9 Nov 2022 17:06:37 +0000 (21:06 +0400)]
LU-15495 tests: fixed dbench test

* Using awk to get list shared libraries
* Fixed shellcheck warnings

Lustre-change: https://review.whamcloud.com/49088
Lustre-commit: 01fb7bda971ee9d5dcd3950f40668131f306b8c5

Test-Parameters: trivial testlist=sanity \
clientdistro=el7.9 clientarch=x86_64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=el8.7 clientarch=x86_64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=el8.7 clientarch=aarch64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=el8.7 clientarch=ppc64le \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=el9.0 clientarch=x86_64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=sles12sp5 clientarch=x86_64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=sles15sp4 clientarch=x86_64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial testlist=sanity \
clientdistro=ubuntu2004 clientarch=x86_64 \
env=SLOW=yes,ONLY=71

Test-Parameters: trivial env=SLOW=yes,ONLY=26 testlist=replay-dual

Test-Parameters: trivial env=SLOW=yes,ONLY=70b testlist=replay-single

Test-Parameters: trivial env=SLOW=yes,ONLY=8 testlist=sanity-pfl

Test-Parameters: trivial env=SLOW=yes,ENABLE_QUOTA=yes,ONLY=8 \
testlist=sanity-quota

Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: Ic28bd67dcfb5ff24e65e33278ac867409a2c1cc6
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51284
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
7 months agoNew RC 2.15.4-RC1 2.15.4-RC1 v2_15_4-RC1
Oleg Drokin [Thu, 23 Nov 2023 04:29:37 +0000 (23:29 -0500)]
New RC 2.15.4-RC1

Change-Id: I2f767990e7ccf1de1b03036ad53678d9e4af2141
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-16284 utils: lfs getstripe follows symlink 62/53162/2
Lei Feng [Tue, 1 Nov 2022 02:57:39 +0000 (10:57 +0800)]
LU-16284 utils: lfs getstripe follows symlink

'lfs getstripe' prints the information of symlink target by default.
With '--no-follow' option it prints the information of symlink itself.

Lustre-change: https://review.whamcloud.com/49003
Lustre-commit: af32b516593dbf2a8e7a85d885c33fd017926ada

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I6cef01af5bb2235bdcbf0b5c99af4b9ed5869515
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53162
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-17109 kernel: new kernel [SLES15 SP5 5.14.21-150500.55.36.1] 69/52769/5
Jian Yu [Wed, 22 Nov 2023 20:05:55 +0000 (12:05 -0800)]
LU-17109 kernel: new kernel [SLES15 SP5 5.14.21-150500.55.36.1]

This patch makes changes to support new SLES15 SP5 release
with kernel 5.14.21-150500.55.36.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/52340
Lustre-commit: 3df9e032db94d6d8bd63aadf36f60b5409aeba94

Test-Parameters: trivial mdtcount=4 mdscount=2 \
clientdistro=sles15sp4 testlist=sanity

Test-Parameters: trivial mdtcount=4 mdscount=2 \
clientdistro=sles15sp5 testlist=sanity

Change-Id: I278017a5c996a8cf4e3d604aa928e968ca007312
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52769
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-17095 build: avoid modules.order nonexistence failure 45/52545/2
Jian Yu [Thu, 28 Sep 2023 17:44:51 +0000 (10:44 -0700)]
LU-17095 build: avoid modules.order nonexistence failure

The modules.order is a temporary output file generated by
kbuild while running "make" command. Sometimes, there is
a race condition that causes the file not created and makes
make command fail as follows:

cat: ...//modules.order: No such file or directory

This patch creates an empty modules.order file to avoid
the error.

Lustre-change: https://review.whamcloud.com/52323
Lustre-commit: dbe4f860977455a9abe50165645a025bb6c46350

Test-Parameters: trivial

Change-Id: If779a727731f18e9409c35c0cd0deddd79559d3a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52545
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-17275 kernel: update RHEL 8.9 [4.18.0-513.5.1.el8_9] 87/53187/2
Jian Yu [Mon, 20 Nov 2023 22:48:28 +0000 (14:48 -0800)]
LU-17275 kernel: update RHEL 8.9 [4.18.0-513.5.1.el8_9]

Update RHEL 8.9 kernel to 4.18.0-513.5.1.el8_9.

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Change-Id: Id0396fc42d7d25bae86ff7dc2a38220d02220e19
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53187
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-17275 kernel: RHEL 8.9 client and server support 91/53091/5
Jian Yu [Tue, 14 Nov 2023 08:18:53 +0000 (00:18 -0800)]
LU-17275 kernel: RHEL 8.9 client and server support

This patch makes changes to support RHEL 8.9 release
with kernel 4.18.0-513.1.1.el8_9 for Lustre client and server.

Lustre-change: https://review.whamcloud.com/53071
Lustre-commit: TBD (from 3f741e2e986e648c868974d4161cc317b49fffe5)

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el8.9 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=ldiskfs mdtcount=4 mdscount=2 \
clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: trivial fstype=zfs mdtcount=4 mdscount=2 \
clientdistro=el8.8 serverdistro=el8.9 testlist=sanity

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
testgroup=full-part-1

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
testgroup=full-part-2

Test-Parameters: optional clientdistro=el8.9 serverdistro=el8.9 \
testgroup=full-part-3

Change-Id: Ia3672d134534b877bb6aaffb4cea0339bc55974f
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53091
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-16802 build: compatibility for 6.4 kernels 28/53128/3
Shaun Tancheff [Tue, 14 Nov 2023 07:30:26 +0000 (23:30 -0800)]
LU-16802 build: compatibility for 6.4 kernels

linux kernel v6.3-rc4-32-g6eb203e1a868
  iov_iter: remove iov_iter_iovec()

Provide a replacement iov_iter_iovec() when one is not provided.

linux kernel v6.3-rc4-34-g747b1f65d39a
  iov_iter: overlay struct iovec and ubuf/len

This renames iov_iter member iov to __iov and provides the
iov_iter() accessor.
Define __iov as iov when __iov not present.
Provide an iov_iter() for older kernels.

linux kernel v6.3-rc1-13-g1aaba11da9aa
  driver core: class: remove module * from class_create()

Provide an ll_class_create() to pass THIS_MODULE, or not,
as needed by class_create().

Linux commit v6.2-rc1-20-gf861646a6562
  quota: port to mnt_idmap

Update osd_dquot_transfer to use mnt_idmap and fallback
to user_ns, if needed, by dquot_transfer.

Linux commit v6.3-rc7-2433-gcf64b9bce950
  SUNRPC: return proper error from get_expiry()

Updated get_expiry() requires a time64_t pointer to be passed
to hold the expiry time. A non-zero return value indicates an
error, nominally -EINVAL. Provide a wrapper for kernels that
return a time64_t and return -EINVAL on error.

Lustre-change: https://review.whamcloud.com/c/fs/lustre-release/+/50875
Lustre-commit: 2559c3a7a2c64b31baae1cba2ede6c4415ddc04d

Test-Parameters: trivial
HPE-bug-id: LUS-11614
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I765d6257eec8b5a9bf1bd5947f03370eb9df1625
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53128
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-16328 llite: migrate_folio, vfs_setxattr 26/53126/2
Shaun Tancheff [Tue, 14 Nov 2023 05:09:25 +0000 (21:09 -0800)]
LU-16328 llite: migrate_folio, vfs_setxattr

Linux commit v5.19-rc3-392-g5490da4f06d1
 fs: Add aops->migrate_folio

Linux commit v5.19-rc4-52-ge33c267ab70d
  mm: shrinkers: provide shrinkers with names

From Linux commit v5.19-rc5-17-g0c5fd887d2bb
  acl: move idmapped mount fixup into vfs_{g,s}etxattr()
Until Linux commit v6.0-rc3-6-g6344e66970c6
  xattr: constify value argument in vfs_setxattr()
Cast away const when required.

Linux commit v5.19-10313-geba2d3d79829
  get rid of non-advancing variants
iov_iter_get_pages_alloc2() replaces iov_iter_get_pages_alloc()

Lustre-change: https://review.whamcloud.com/49265
Lustre-commit: 0006eb36440dcb4dc06aa61c35db40bf7dec0ddc

Test-Parameters: trivial
HPE-bug-id: LUS-11358
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Id1fa6db94172c0a61008ba4b066907950bdd6473
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53126
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-16327 llite: read_folio, release_folio, filler_t 25/53125/3
Shaun Tancheff [Tue, 14 Nov 2023 04:57:43 +0000 (20:57 -0800)]
LU-16327 llite: read_folio, release_folio, filler_t

Linux commit v5.18-rc5-221-gb7446e7cf15f
  fs: Remove aop flags parameter from grab_cache_page_write_begin()
flags have been dropped from write_begin() and
grab_cache_page_write_begin()

Linux commit v5.18-rc5-241-g5efe7448a142
  fs: Introduce aops->read_folio
Provide a ll_read_folio handler around ll_readpage

Linux commit v5.18-rc5-280-ge9b5b23e957e
  fs: Change the type of filler_t
Affects read_cache_page, provides a wrapper for read_cache_page
and wrappers for filler functions

Linux commit v5.18-rc5-282-gfa29000b6b26
  fs: Add aops->release_folio
Provide an ll_release_folio function based on ll_releasepage

Lustre-change: https://review.whamcloud.com/49199
Lustre-commit: 133ed0cf6f0c84d2b5b84e1de3ff2c54b1fb902d

Test-Parameters: trivial
HPE-bug-id: LUS-11357
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ibd4ec1133c80cd0eb8400c4cd07b50e421dd35c5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53125
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-13485 kernel: Parallel core configure tests 98/49098/4
Shaun Tancheff [Mon, 13 Nov 2023 21:15:24 +0000 (13:15 -0800)]
LU-13485 kernel: Parallel core configure tests

Transform the compile tests in lustre-core to run in parallel

Lustre-change: https://review.whamcloud.com/38361
Lustre-commit: a346cf6cf22c285e05177a5a5e70a7d8e5bd9fa8

LU-16586 build: Remove old check for linux_selinux_enabled

LC_SRC_HAS_LINUX_SELINUX_ENABLED is used and not defined, it
should be removed.

LC_PROG_LINUX_SRC and LC_PROG_LINUX_RESULTS are defined twice,
remove the unused ones.

Lustre-change: https://review.whamcloud.com/50111
Lustre-commit: 2bdff20f97707cf05118e822adf202ccb209e3a5

Fixes: a346cf6cf22 ("LU-13485 kernel: Parallel core configure tests")
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I46fa852659eb4db86a12ec4ad3efddd0fdd3a655
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49098
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-16534 build: Prefer timer_delete[_sync] 90/53090/3
Shaun Tancheff [Mon, 13 Nov 2023 23:36:41 +0000 (15:36 -0800)]
LU-16534 build: Prefer timer_delete[_sync]

Linux commit v6.1-rc1-7-g9a5a30568697
  timers: Get rid of del_singleshot_timer_sync()
Linux commit v6.1-rc1-11-g9b13df3fb64e
  timers: Rename del_timer_sync() to timer_delete_sync()
Linux commit v6.1-rc1-12-gbb663f0f3c39
  timers: Rename del_timer() to timer_delete()

Prefer timer_delete_sync() to del_singleshot_timer_sync()
Prefer timer_delete_sync() to del_timer_sync()
Prefer del_timer() to timer_delete()

Provide del_timer and del_timer_sync when
timer_delete[_sync] is not available

Lustre-change: https://review.whamcloud.com/49922
Lustre-commit: 0ec89529ce14a1bb5af0c01ed86424a10e0e373c

Test-Parameters: trivial
HPE-bug-id: LUS-11470
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4c946c315a83482dd0bd69e5e89f0302a67bf81c
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53090
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-17274 kernel: new kernel [RHEL 9.3 5.14.0-362.8.1.el9_3] 55/53055/6
Jian Yu [Tue, 14 Nov 2023 07:55:32 +0000 (23:55 -0800)]
LU-17274 kernel: new kernel [RHEL 9.3 5.14.0-362.8.1.el9_3]

This patch makes changes to support new RHEL 9.3 release
for Lustre client.

Lustre-change: https://review.whamcloud.com/53054
Lustre-commit: TBD (from 9146471f862d6c6fae6c1f6ac99f55d8280a2891)

Test-Parameters: trivial env=SANITY_EXCEPT="27J 101j" \
  mdtcount=4 mdscount=2 clientdistro=el9.3 testlist=sanity
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-1
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-2
Test-Parameters: optional clientdistro=el9.3 testgroup=full-part-3

Change-Id: I9cce1a7d2249cb4df39106c44ba4417411ee0757
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53055
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
7 months agoLU-13031 tests: avoid new user xattr in interop 32/52332/4
Xing Huang [Sun, 10 Sep 2023 08:07:16 +0000 (16:07 +0800)]
LU-13031 tests: avoid new user xattr in interop

The xattr test of test_103a is expecting that there are no other
user xattrs, but the patch(#50982) is adding a new user xattr to
every file.  This new xattr can be disabled when running with new
clients and old servers, but this is not easily possible with old
clients (where the test scripts run) and new servers
(that have that patch #50982).

We can have this change to avoid the above problem.

Test-Parameters: testlist=sanity serverjob=lustre-b_es-reviews serverbuildno=12671 env=ONLY=103
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: Iaf4bd81be7e43d946ec184e7d8d65326983c351d
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52332
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
7 months agoLU-17218 ofd: improve filter_fid upgrade compatibility 08/52708/5
Bobi Jam [Mon, 16 Oct 2023 10:02:54 +0000 (18:02 +0800)]
LU-17218 ofd: improve filter_fid upgrade compatibility

filter_fid could be expanded in later Lustre version, and with
upgrade then downgrade process, the filter_fid EA on disk
could has been expanded during upgrade, and won't work after
the downgrade.

This patch improves this process by allocating bigger buffer to
hold the expanded filter_fid EA then trims the unrecognizable
fileds off.

Lustre-change: https://review.whamcloud.com/52798
Lustre-commit: TBD (from 3dfe2d3abc6bff161ebce62736990fabe540c1de)

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I4c99f1d9f3962d46ebf9e9b799988ff3dba4f919
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52708
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-16966 osd: take trunc_lock for fallocate 10/52710/2
Alex Zhuravlev [Mon, 16 Oct 2023 13:59:08 +0000 (16:59 +0300)]
LU-16966 osd: take trunc_lock for fallocate

as fallocate may need few transactions (or transaction restarted)
we have to avoid any concurrent writes/truncates on this object
until fallocate supports 'restart-from-beginning' - first stop the
transaction, then release the lock, then repeat again (like
the write path does).

Lustre-change: https://review.whamcloud.com/52264
Lustre-commit: 51529fb57f85210e292a15c882cf25a4689ea77d

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0bf38b1886fbf24656b45fe0f87fcbad2227672a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52710
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-15564 osd: add allocation time histogram 68/52768/3
Alex Zhuravlev [Fri, 18 Feb 2022 08:39:12 +0000 (11:39 +0300)]
LU-15564 osd: add allocation time histogram

add block mapping/allocation histogram to brw stats to debug
mballoc related issues.

$ lctl get_param osd*.*OST*.brw_stats
                           read      |     write
block maps msec        maps  % cum % |  maps        % cum %
1:    1522360 100 100   | 49272  99  99
2:          0   0 100   |    1   0  99
4:          0   0 100   |    1   0  99
8:          0   0 100   |    0   0  99
16:          0   0 100   |    0   0  99
32:          0   0 100   |    0   0  99
64:          0   0 100   |    1   0 100

Lustre-change: https://review.whamcloud.com/46550
Lustre-commit: f97ce54357bc91f7b1285febfc50d6087dd94c13

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I1185386adc64e844de71e25a4e439e493e5e5bc5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52768
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-15707 lod: force creation of a component without a pool 77/49477/2
Etienne AUJAMES [Wed, 30 Mar 2022 16:43:44 +0000 (18:43 +0200)]
LU-15707 lod: force creation of a component without a pool

This patch add the pool option "lfs setstripe -p ignore" to force
the creation of component without a pool set by inheritance (from
parent or root).

e.g:
$ lfs setstripe -p pool tdir
$ lfs setstripe -E1M -p ignore -E-1 -p '' -c2 -S2M tdir/tfile
$ lfs getstripe -I1 -p tdir/tfile
(no pool set)
$ lfs getstripe -I2 -p tdir/tfile
pool
(inherited from tdir)

This patch add the test "ost-pools test_32" to verify this behavior.

The poorly-named "-p none" keyword, which indicates the pool name
should be inherited from the root or parent dir layout, will be
eventually replaced by the new "-p inherit" keyword.

Lustre-change: https://review.whamcloud.com/46955
Lustre-commit: 6b69d22e4cb738f4f9ff5454a6f9ae17a3a2d6fa

Test-Parameters: serverdistro=el7.9 serverversion=2.12.8 testlist=ost-pools env=ONLY=32,ONLY_REPEAT=50
Test-Parameters: testlist=ost-pools env=ONLY=32,ONLY_REPEAT=50
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I782cbafe209cff6857162303a4650f5e3b438be5
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-17108 nodemap: make map_mode available for default nm 35/52535/2
Sebastien Buisson [Mon, 11 Sep 2023 15:31:09 +0000 (17:31 +0200)]
LU-17108 nodemap: make map_mode available for default nm

The map_mode property lets control the way mapping is carried out. It
is already available on regular nodemaps, to decide whether uids, gids
and/or projids will be mapped.
On the default nodemap, where it is not possible to define mappings,
the map_mode property will be taken into account when trusted is 0 and
deny_unknown is 0. Unmapped IDs will be left unchanged.

Lustre-change: https://review.whamcloud.com/52336
Lustre-commit: d9df39505530c5f224427103290b60828c11e81c

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I16a2f5cfda11a8435b56a00f3e97bdc70741c156
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52535
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-17111 kernel: update RHEL 9.2 [5.14.0-284.30.1.el9_2] 66/52366/6
Jian Yu [Thu, 9 Nov 2023 18:37:42 +0000 (10:37 -0800)]
LU-17111 kernel: update RHEL 9.2 [5.14.0-284.30.1.el9_2]

Update RHEL 9.2 kernel to 5.14.0-284.30.1.el9_2 for Lustre client.

Lustre-change: https://review.whamcloud.com/52358
Lustre-commit: 4f9957c1e9a0d61087a63b523f3d29ee17904baa

Test-Parameters: trivial env=SANITY_EXCEPT="27J 101j" clientdistro=el9.2 testlist=sanity

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Id80dbba6b4434a83cf925d6961d727941274edf4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52366
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-16167 obdclass: fix lctl llog_print with skipped records 21/51221/2
Etienne AUJAMES [Mon, 19 Sep 2022 10:23:47 +0000 (12:23 +0200)]
LU-16167 obdclass: fix lctl llog_print with skipped records

The 2a5b50d ignores the skipped records in configuration llog.
But if ioctl OBD_IOC_LLOG_PRINT return 0 record to display,
jt_llog_print_iter() will stop the processing and ignore the
non-skipped records at the end of the llog.

This patch returns to user space if the last index processed
(by llog_print_cb) is the last of llog file. If true,
jt_llog_print_iter() stops the processing.

Add regression test "conf-sanity test_123ai" for this issue.

Lustre-change: https://review.whamcloud.com/48586
Lustre-commit: c6da54aa7546440339265c644538d3d109e46bde

Fixes: 2a5b50d ("LU-15142 lctl: fixes for set_param -P and llog_print")
Test-Parameters: testlist=conf-sanity env=ONLY=123ai,SLOW=yes,ONLY_REPEAT=10
Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: I78395268c57555e4fd2a4048ccf5b6132ca2877f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander <alexander.boyko@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51221
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 months agoLU-16042 tests: can not get cache size on Arm64 79/51179/4
Kevin Zhao [Mon, 25 Jul 2022 07:53:44 +0000 (15:53 +0800)]
LU-16042 tests: can not get cache size on Arm64

This fix the test fail on Arm64, the cache size can not be
display on /proc/cpuinfo. And even in the VM and somee
older Arm64 CPU, we can not get the cachesize. So it's
better to fallback to a pre-set value here if we don't get
the cache size.

Lustre-change: https://review.whamcloud.com/48030
Lustre-commit: f276f1cb0859e8718448e69bd99ee305f5e62d42

Test-Parameters: trivial
Test-Parameters: clientarch=aarch64 clientdistro=el8.7 \
    testlist=sanity env=ONLY=155

Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
Change-Id: I17ce1d8accc69d1489db2071a2741b3927fff302
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51179
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 months agoLU-15660 statahead: statahead thread doesn't stop 00/52300/2
Yang Sheng [Fri, 17 Jun 2022 12:30:34 +0000 (20:30 +0800)]
LU-15660 statahead: statahead thread doesn't stop

Add a barrier to ensure sai_task changing can be seen
when access it without locking. Else the statahead
thread could sleep forever since wake_up was lost.

Lustre-change: https://review.whamcloud.com/47673
Lustre-commit: b977caa2dc7dddcec9e20d393ee79dfa9fe31c0d

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I211e99f1bdddaaaf028a205658f603fda034d389
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52300
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 months agoLU-16585 build: remove python2 dependencies 76/52176/2
Alex Deiter [Tue, 21 Feb 2023 22:27:47 +0000 (02:27 +0400)]
LU-16585 build: remove python2 dependencies

Fixed packaging issue casued by scripts and control files.

Lustre-change: https://review.whamcloud.com/50084
Lustre-commit: bea3f81f84fd16d2d403682ef25b8abe314acd0f

Test-Parameters: trivial
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: I6c9b24bf811269928494af17c15627902e5fe27b
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52176
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 months agoLU-16943 tests: use primary ost1 server in replay-single/135 59/52059/3
Jian Yu [Thu, 24 Aug 2023 01:01:20 +0000 (18:01 -0700)]
LU-16943 tests: use primary ost1 server in replay-single/135

This patch fixes replay-single test_135() to make sure
the primary ost1 server is used at the beginning of the test.

Lustre-change: https://review.whamcloud.com/52058
Lustre-commit: cdd8b056bff0d48155eaf4b7732d1d8880ceda55

Test-Parameters: trivial testlist=replay-single

Test-Parameters: trivial env=FAILURE_MODE=HARD \
    clientcount=4 mdtcount=1 mdscount=2 osscount=2 \
    austeroptions=-R failover=true iscsi=1 \
    testlist=replay-single,mmp

Fixes: 81418be83ed8 ("LU-16943 tests: fix replay-single/135 under hard failure mode")
Change-Id: Ia25314255c9f00ba71687e1f757517f37031caed
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52059
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 months agoLU-16626 build: remove python2 dependencies 26/51426/3
Alex Deiter [Thu, 9 Mar 2023 14:09:19 +0000 (18:09 +0400)]
LU-16626 build: remove python2 dependencies

Fixed packaging issue caused by zfsobj2fid script.

Lustre-change: https://review.whamcloud.com/50241
Lustre-commit: 404a1e827b0a9d86864695c8699e1ca076be6c9d

Test-Parameters: trivial
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: I4375038b0d2c2b42ac4080fe834d35bdd3ef54f8
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51426
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 months agoLU-16916 tests: fix client_evicted() not to ignore EOPNOTSUPP 68/51668/4
Jian Yu [Fri, 14 Jul 2023 05:22:18 +0000 (13:22 +0800)]
LU-16916 tests: fix client_evicted() not to ignore EOPNOTSUPP

After RHEL 9.x or Ubuntu 22.04 client is evicted, "lfs df" returns
error code 95 (EOPNOTSUPP), which is ignored in check_lfs_df_ret_val()
and then causes client_evicted() to ingore that error.

This patch fixes client_evicted() to check the return value
from "lfs df" directly so as not to ignore EOPNOTSUPP.

Lustre-change: https://review.whamcloud.com/51667
Lustre-commit: a5a9ded43b72238c2df8e0a74f03151ea3d4ce99

Test-Parameters: trivial clientdistro=el9.2 testlist=replay-vbr
Test-Parameters: trivial clientdistro=el8.8 testlist=replay-vbr

Change-Id: I633ae8769fc563b8068f433e2afae29463ac5553
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51668
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
8 months agoLU-15193 quota: expand QUOTA_MAX_TRANSIDS to 12 11/49611/3
Lei Feng [Thu, 4 Nov 2021 11:41:06 +0000 (19:41 +0800)]
LU-15193 quota: expand QUOTA_MAX_TRANSIDS to 12

In some rare cases 12 quota ids are needed.
Usually (user, group) * (block, inode) * (inode, parent) = 8 qids
are needed. But with project id,
(user, group, project) * (block, inode) * (inode, parent) = 12 qids
are needed.

Lustre-change: https://review.whamcloud.com/45456
Lustre-commit: 61481796ac85e9ab2469b8d2f4cc75088c65d298

Change-Id: I4b3ee197f6e274abda06edf60b246f089fe28d10
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-quota
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49611
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Stephane Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
8 months agoLU-16517 build: pass extra configure options to "make debs" 78/51178/4
Jian Yu [Wed, 31 May 2023 06:40:09 +0000 (23:40 -0700)]
LU-16517 build: pass extra configure options to "make debs"

While running "make debs", the configure command in debian/rules
ignores some user defined configure options. This patch fixes
the issue by adding the detection of the extra options into
debian/rules.

Lustre-change: https://review.whamcloud.com/50464
Lustre-commit: 3989529f22f5c54a98e445674b4b3cc443a3af5f

Test-Parameters: trivial clientdistro=ubuntu2004

Change-Id: Ia9db4e05abf33834cb3c853f4f0829dadc8d7400
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51178
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
8 months agoLU-16943 tests: fix replay-single/135 under hard failure mode 08/51608/4
Jian Yu [Fri, 14 Jul 2023 06:04:42 +0000 (14:04 +0800)]
LU-16943 tests: fix replay-single/135 under hard failure mode

This patch fixes replay-single test_135() to load libcfs module
on the failover partner node to avoid 'fail_val' setting error.
It also fixes the issue that not all of the OSTs are mounted after
failing back ost1.

Lustre-change: https://review.whamcloud.com/51574
Lustre-commit: 74140e5df4c094f7f0e923e1b82c464b18e8a7cc

Test-Parameters: trivial testlist=replay-single
Test-Parameters: trivial fstype=zfs testlist=replay-single

Test-Parameters: trivial env=FAILURE_MODE=HARD \
    clientcount=4 mdtcount=1 mdscount=2 osscount=2 \
    austeroptions=-R failover=true iscsi=1 \
    testlist=replay-single

Change-Id: Id46c722a6db9d832829a739f41f7462b32a6d9d9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51608
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
8 months agoLU-15740 tests: scale fs_log_size by OSTCOUNT 06/51606/2
Andreas Dilger [Fri, 24 Mar 2023 23:09:44 +0000 (17:09 -0600)]
LU-15740 tests: scale fs_log_size by OSTCOUNT

The fs_log_size "free space skew" was being scaled by MDSCOUNT,
but in fact this parameter is only ever used to compare the OST
free space usage, so the OSTCOUNT should be used when scaling it.

It is likely that the skew is actually caused by blocks allocated
by OST object directories and not llogs (no llogs used on OSTs for
many years), but it isn't worthwhile to rename the function.

Lustre-change: https://review.whamcloud.com/50419
Lustre-commit: fabec6f2cb39950a2f208567dac716e21880fa9f

Test-Parameters: trivial testlist=runtests
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I97f05b10fa7ec367534b5bdce09feae5e93ebbe5
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51606
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 months agoLU-16934 kernel: update RHEL 8.8 [4.18.0-477.15.1.el8_8] 18/51518/3
Jian Yu [Sat, 29 Jul 2023 03:36:03 +0000 (20:36 -0700)]
LU-16934 kernel: update RHEL 8.8 [4.18.0-477.15.1.el8_8]

Update RHEL 8.8 kernel to 4.18.0-477.15.1.el8_8.

Lustre-change: https://review.whamcloud.com/51517
Lustre-commit: 830bf7a1f8de73a4f46248e6b8d2bbcd944a1f09

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.8 serverdistro=el8.8 testlist=sanity

Change-Id: I66365dce63065a0a07958a182a3c705e9948d424
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51518
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
8 months agoLU-16060 osd-ldiskfs: copy nul byte terminator in writelink 56/51356/3
Alexander Zarochentsev [Wed, 20 Jul 2022 16:05:53 +0000 (19:05 +0300)]
LU-16060 osd-ldiskfs: copy nul byte terminator in writelink

memcpy() call in osd_ldiskfs_writelink() doesn't copy the nul
terminator byte from the source buffer, leaving the space
after target link name uninialized which is ok for the kernel
code and debugfs but not e2fsck.

HPE-bug-id: LUS-11103

Lustre-change: https://review.whamcloud.com/48092
Lustre-commit: 907dc0a2d333f2df2d654a968fc50f8cc05b779d

Signed-off-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Change-Id: I914f2c78e1a6571bf360a23b0ede8c70502bf0df
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51356
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>