git://git.whamcloud.com - fs/lustre-release.git/log

LU-11596 osc: Fix and re-enable sanity grant test for ARM

If both OST and OSC support OBD_CONNECT_GRANT_PARAM, OST side will not
change client side claimed grant (a.k.a. o_grant_used) regardless of
the client page size. So no grant loss in this case.

Lustre-change: https://review.whamcloud.com/40758
Lustre-commit: 7d3edce0650f0b66bdd280373a54a16cb28b8469

Test-Parameters: trivial clientarch=aarch64 clientdistro=el8.6 testlist=sanity
Fixes: bd1e41672c97 ("LU-2049 grant: add support for OBD_CONNECT_GRANT_PARAM")
Change-Id: Ia0d3da587cb551400fec0c054dc65b116e6bd95b
Signed-off-by: James Simmons <jsimmons@infradead.org>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50279
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15129 tests: sanity-quota_75_dom fix

oflag=sync causes dd to write page by page instead
of sending several pages in RPC. Furthermore, when
granted space becomes closer to soft_limit(i.e. over
9MB if soft_limit is 10MB), OST can not preacquire
space anymore. Also OST could acquire only requested
amount of space - see qmt_alloc_expand. Thus OST has
to send quota acquire request at MDT for each BRW request
from the client. Sometimes 20 seconds is not enough to write
10MB. Change oflag=sync with conv=fsync to reduce the number
of RPCs between the client and OST and between QSDs and QMT.
One fsync at close should help to avoid timeout failure.

Lustre-change: https://review.whamcloud.com/50164
Lustre-commit: 7d05a687ee5d4f4b95585244a7f60394475fe0ba

Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota env=ONLY=75,ONLY_REPEAT=50
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Iad363fdc8a0984861055c295ea9cc3f23110fd9f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50336
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-7013 lipe: printf formats added to lipe_find3

Printf formats added to lipe_find3: fid, mirror-count,
stripe-count. Sanity-lipe-find3 updated with printf
format tests

Test-Parameters: trivial testlist=sanity-lipe-find3,sanity-lipe-scan3
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: Ic5b0d6786c705e51d809eba19e123287d2794740
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50313
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-7065 build: add Rocky distro to lipe

Test-Parameters: trivial

Change-Id: I83298fee9da925a698246d3f52e2c9164cb5cf4b
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50294
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16589 tests: fix hard-link failure in sanityn/55d

Since coreutils version 8.31, the stat() and lstat()
operations were removed from ln by commit 571f63f5010b,
which caused the following dir hard-link failure in
sanityn/55d:

ln: failed to create hard link '/mnt/lustre2/d55d.sanityn/d55d.sanityn/'
=> '/mnt/lustre2/d55d.sanityn/f1': No such file or directory

This actually reveals a kernel issue which is fixed by commit
v5.18-rc2-188-gb3d4650d82c7.

To avoid the kernel issue and keep the test effective,
this patch appends the target filename to the $tdir/
so as to fix the hard-link failure.

Lustre-change: https://review.whamcloud.com/50127
Lustre-commit: TBD (from 6dea4fabe02ebd90bc6e4dbb8641003a9a66c967)

Test-Parameters: trivial env=ONLY=55d testlist=sanityn
Test-Parameters: trivial clientdistro=el9.1 env=ONLY=55d testlist=sanityn
Test-Parameters: trivial clientdistro=el9.0 env=ONLY=55d testlist=sanityn
Test-Parameters: trivial clientdistro=sles15sp4 env=ONLY=55d testlist=sanityn
Test-Parameters: trivial clientdistro=sles15sp3 env=ONLY=55d testlist=sanityn

Change-Id: I42313e43eaea3d94007d534bf38efdeacf2ede43
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50350
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn81

New tag 2.14.0-ddn81

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I189f73281dfe12d4835c3d54b10e41099fe47af8

LU-16285 ldlm: BL_AST lock cancel still can be batched

The previous patch makes BLAST locks to be cancelled separately.
However the main problem is flushing the data under the other batched
locks, thus still possible to batch it with those with no data.
Could be optimized for not yet CANCELLING locks only, otherwise it is
already in the l_bl_ast list.

Lustre-change: https://review.whamcloud.com/50158
Lustre-commit: 9d79f92076b6a9ca735dbe4420c122f47d240263

Fixes: b65374d9 ("LU-16285 ldlm: send the cancel RPC asap")
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: Ie4a7c7f3e0f5462290f72af7c3b2ff410a31f5e7
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50274
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16221 kernel: update RHEL 9.1 [5.14.0-162.18.1.el9_1]

Update RHEL 9.1 kernel to 5.14.0-162.18.1.el9_1 for Lustre client.

Lustre-change: https://review.whamcloud.com/50177
Lustre-commit: TBD (from ae128e3a29f10fc9bdb5711c1c27e2edaa35a82b)

Test-Parameters: trivial clientdistro=el9.1 testlist=sanity

Change-Id: I032f69f1ecba60248729bb856a3aad78e5f05680
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50204
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16509 lnet: memcpy false positive in brw_test

The flexible array at the end of srcp_bulk is triggering a
false positive in fortified memcpy(). Quash it with
unsafe_memcpy().

Lustre-change: https://review.whamcloud.com/50185/
Lustre-commit: 299bd671a9f7c0955b2be59e36bb8239797c1b82 (tbd)

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I13386c0a8e73b04af8d398aa49361bfdf6a05ad8
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50218
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn80

New tag 2.14.0-ddn80

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia899d8b47c4df16020bcfd8c20cc60d59f761daa

LU-16509 lnet: quash memcpy WARN_ONCE false positive

Linux v6.1-rc1-4-g6f7630b1b5bc
fortify: Capture __bos() results in const temp var

In lnet_peer_push_event() the memcpy triggers a WARN_ONCE
due to the flexible array at the end of
struct lnet_ping_info contained in struct lnet_ping_buffer

Use unsafe_memcpy() to avoid this false positive warning.

Lustre-change: https://review.whamcloud.com/49801
Lustre-commit: 18e24419599666bd536d072705052a8c64b681ab (tbd)

Test-Parameters: trivial
HPE-bug-id: LUS-11455
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4aa8f38678cd1522004d98b58a3f440d8a38589c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50217
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16363 build: fiemap flexible array

Linux commit v5.19-rc2-1-g94dfc73e7cf4
treewide: uapi: Replace zero-length arrays with flexible-array
members
Adjust wiretest to handle flexible array when
sizeof(fiemap->fm_extents) is undefined.

Lustre-change: https://review.whamcloud.com/49305
Lustre-commit: fedf1e8bd70ccb2aaa64cb90111a7298d9bb2bf7

Test-Parameters: trivial
HPE-bug-id: LUS-11388
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ia2692d126a871b43e9144e5d151215166604702d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50216
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16557 client: -o network needs add_conn processing

Mount option -o network restricts client import to use
only selected network. It processes connection UUID/NIDs
during 'setup' config command handling but skips any
'add_conn' command if its UUID has no mention about that
network. Meahwhile connection UUID is just a name and may
have many NIDs configured including those on restricted
network which are skipped as well. Therefore client import
configuration misses failover NIDs on restricted network.

Patch makes import to save restricted network information
after 'setup' command processing, so it is applied to any
client_import_add_conn() call. The 'add_conn' command is
always processed now and its NIDs will be filtered in the
same way as for 'setup'.
Test 31 in sanity-sec.sh is extended to check imports
failover_nids has all and only NIDs on restricted network

Lustre-change: https://review.whamcloud.com/49986
Lustre-commit: c508c9426838f16256223ab0bbd648bfbec25e46

Test-Parameters: env=ONLY=31 testlist=sanity-sec
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id70ebd836f061f154e3779b07b52f1baea9a1776
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50172
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9162 lod: option to set max stripe count per filesystem

Add an option to set max default stripe count when the stripe count
is set to -1.

Lustre-change: https://review.whamcloud.com/45532
Lustre-commit: 3c9580931d1a7753e6c03900df01fd05ab3ca73d

Change-Id: I02634a02a6f6579750fe964662b7e644af1689d6
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50231
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-6169 lipe: debug sanity-lipe-find3 problem

Added test print for files/FIDs left behind in filesystem.

Test-Parameters: trivial testlist=sanity-lipe-find3 env=ONLY="90 350"
Test-Parameters: testlist=sanity-lipe-scan3,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-pcc,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-pcc,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-lipe-scan3,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-pcc,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Test-Parameters: testlist=sanity-quota,sanity-lipe-find3
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: I2ad8dc24f6825075f4089c1759c148bee26bfbfe
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50142
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11170 tests: add debugging to sanity/415

Add a loop of renames without the concurrent 'touch' operation to
measure the test time, and then a second loop that has the 'touch'
so that we can see whether slow renames are because of COS (which
would make the test failure a kernel bug to be fixed), or because
the test is running in a VM and the server/disk is slow (which is
something to be fixed in the test (e.g. by making "slow" relative
to the non-touch baseline time).

Lustre-change: https://review.whamcloud.com/49724
Lustre-commit: 6594babc73851fab335c514cd1fee018425e7bb3

Test-Parameters: trivial testlist=sanity env=ONLY=415,ONLY_REPEAT=120 mdscount=2 mdtcount=4
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic1a952be0b861005b46da3e673216e455f3ebbe5
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50203
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13417 tests: fix interop version check for tests

Fix version check in tests so that they actually run.

Add a spelling.txt rule for wrong version checks used in tests.

Test-Parameters: trivial testlist=sanity-hsm env=ONLY=12s
Fixes: ca1f7f6899 ("LU-15132 hsm: Protect against parallel HSM restore requests")
Fixes: b1b3c1413a ("LU-15727 lod: honor append_pool with default composite layouts")
Fixes: f1a5ef423e ("LU-15938 llog: more checks in llog_reader")
Fixes: 4e43fe8d0c ("LU-14719 lod: distributed transaction check space")
Fixes: 56ea02d192 ("LU-15850 lmv: always space-balance r-r directories")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5241d4b419876b8f9cc6799757d6f205ee7b9e82
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50256
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16349 o2iblnd: Fix key mismatch issue

If a pool memory region (mr) is mapped then unmapped without being
used, its key becomes out of sync with the RDMA subsystem.

At pool mr map time, the present code will create a local
invalidate work request (wr) using the mr's present key and then
change the mr's key.  When the mr is first used after being mapped,
the local invalidate wr will invalidate the original mr key, and
then a fast register wr is used with the modified key.  The fast
register will update the RDMA subsystem's key for the mr.

The error occurs when the mr is never used.  The next time the mr
is mapped, a local invalidate wr will again be created, but this
time it will use the mr's modified key.  The RDMA subsystem never
saw the original local invalidate, so now the RDMA subsystem's
key for the mr and o2iblnd's key for the mr are out of sync.

Fix the issue by tracking if the invalidate has been used.
Repurpose the boolean frd->frd_valid.  Presently, frd_valid is
always false.  Remove the code that used frd_valid to conditionally
split the invalidate from the fast register.  Instead, use frd_valid
to indicate when a new invalidate needs to be generated.  After a
post, evaluate if the invalidate was successfully used in the post.

These changes are only meaningful to the FRWR code path.  The failure
has only been observed when using Omni-Path Architecture.

Lustre-change: https://review.whamcloud.com/49714
Lustre-commit: 0c93919f1375ce16d42ea13755ca6ffcc66b9969

Signed-off-by: Cyril Bordage <cbordage@whamcloud.com>
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: I532a11f10ae6a5917a4c054f37747d08eb4d6331
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50211
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16555 obdclass: quote jobid with ':' char

quote jobid with ':' char even though ':' is printed literally.

Lustre-change: https://review.whamcloud.com/50150
Lustre-commit: TBD (from e3f53a5fa5c086b685eb1f0e8d3c96780faabd49)

Fixes: 12604ac4de (LU-16555 obdclass: print more special chars for jobid)
Test-Parameters: trivial
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I8bf642180b00f068921598c3a0ea30b4f5b99c1e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50151
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16591 build: use vendor tag for LUSTRE_FIX

Improve the regexp for the LUSTRE_FIX (fourth) component of the
LUSTRE_VERSION parsing so that if there is an additional component
in the version like "2.15.2-abc123" the "123" will be reported as
the fourth component of the version in obd_connect_data.ocd_version.
If there is already a fix version specified (e.g. 2.15.2.4-abc123)
then it will continue to be used for LUSTRE_FIX instead of the extra
"-abcNN" value.

The build version is shown in "{mdt,obdfilter}.*.exports.*.export"
param on servers for connected clients, and in "{mdc,osc}.*.import"
param on clients for connected servers. Displaying the full version
improves debuggability of remote peers to know their specific build
instead of showing the first three digits and always ".0" at the end.

Since ocd_version is a numeric field it is not possible to include
the "abc" part of the version string on the peer nodes.

Lustre-change: https://review.whamcloud.com/50136
Lustre-commit: ee67a78cf2b102230dc3a7beea2274866e643224

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I96b38f99b2522c5ea3f3b3e2ddd7cd64f1ce7057
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50137
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>

LU-16601 kernel: update SLES15 SP4 [5.14.21-150400.24.46.1]

Update SLES15 SP4 kernel to 5.14.21-150400.24.46.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/50179
Lustre-commit: TBD (from c1e032746e318daba275eec8995136a187de4a3c)

Test-Parameters: trivial clientdistro=sles15sp4 testlist=sanity

Change-Id: I5b9e39359e61e929adaeddece60f4d247996a00a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50206
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16600 kernel: update RHEL 8.7 [4.18.0-425.13.1.el8_7]

Update RHEL 8.7 kernel to 4.18.0-425.13.1.el8_7.

Lustre-change: https://review.whamcloud.com/50178
Lustre-commit: TBD (from 0fbefb1c42e98d0323a8febe104ec4b8afc4ad22)

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.7 serverdistro=el8.7 testlist=sanity

Change-Id: I4e4d7024e31597566d936bdec4b65a1d4ed925a0
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50205
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15524 mdd: trigger changelog GC by free space

if amount of space consumed by changelog become comparable
with system free space then start emergency GC for changelog
by purging the oldest user

Such behavior is enabled by default and can be disabled via
mdd_changelog_free_space_gc parameter

Test 160t is added to sanity.sh

Lustre-change: https://review.whamcloud.com/46467
Lustre-commit: bec1334954a73ed668fad409e8c728f9dfd6bb99

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ia63cc71e708b0f10cdf54f45f0809c0e86950101
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50243
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-6792 tests: Added support of UTF-8 in lipe_find3

- Set locale UTF-8 before running shell command
- Use %s format when SCM script is produced to protect
UTF-8 symbols
- Use fnmatch when compare UTF-8 strings with no
case sensitivity
- Added test for UTF-8 file name test and path test
- Added test name in path name in test_109

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3,sanityn
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: I6c311ac0b2725f5414a5828d5577343ac8a22dee
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50066
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-7013 lipe: exclude some printf formats

Exclude some printf formats: fid, mirror-count, stripe-count
until they are implemented.

Test-Parameters: trivial testlist=sanity-lipe-find3
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: I3156c63172f25c0e7ea57ee15836aa4d005f6bbb
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50251
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14760 llite: restart clio for AIO if necessary

If the clio needs to be restarted from where it left off,
do it for AIO as well, so we don't end up with short IO.
Limit thr number of retries to 1000, to avoid potential
issues if the loop is stuck forever.

Lustre-change: https://review.whamcloud.com/43995
Lustre-commit: TBD (from cf20b7fffa211b5093a2d698df6162f5ddf575a4)

Change-Id: Iccca31b032b01b940656864bfff22a821ff5061d
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/47407
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-6169 lipe: prevent lipe_find3 interop testing

Some tests in sanity-lipe-find3 may fail when devices are left
with not removed files from previous test sets.

Test-Parameters: trivial testlist=sanity-lipe-find3
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: I8f99c0b34f343128a1ce2b9d71572f6d1a58b79a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50245
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn79

New tag 2.14.0-ddn79

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I532d7747da3604b0c40828c21d606d2bca4ffab1

LU-14719 lod: ignore space check error in recovery

statfs may fail in recovery, ignore this error in
lod_trans_space_check().

Fix syntax error in replay-single 111g version check.

Lustre-change: https://review.whamcloud.com/49249
Lustre-commit: e44489f2f29a2e50883f9bbdec491b65ca92a692

Fixes: 6aee406c84 ("LU-14719 lod: distributed transaction check space")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I6c7934ca242a639d996d0ab5a4d7648cec8a53de
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50107
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16335 tests: modify the version_code in test_832 of sanity

Modify the version_code in test_832() of sanity.sh

Test-Parameters: testlist=sanity env=ONLY=832
Fixes: 485953b0eb ("LU-16335 mdt: skip target check for rm_entry")
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: I5cd641f4e8ae830099242d1cd49cdab6d341199c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50128
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14668 lnet: don't delete peer created by Lustre

Peers created by Lustre have their primary NIDs locked.
If that peer is deleted, it'll confuse lustre. So when manually
deleting a peer using:
lnetctl peer del --prim_nid ...
We must continue to preserve the primary NID. Therefore we delete
all the constituent NIDs, but keep the primary NID. We then
flag the peer for rediscovery.

Lustre-change: https://review.whamcloud.com/43565
Lustre-commit: TBD (83f66616e303039974e7ed6ec07140dcc17f0cea)

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I34eef9b0049435a01fde87dc8263dd50f631c551
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50117
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14668 lnet: Peers added via kernel API should be permanent

The LNetAddPeer() API allows Lustre to predefine the Peer for LNet.
Originally these peers would be temporary and potentially re-created
via discovery. Instead, let's make these peers permanent. This allows
Lustre to dictate the primary NID of the peer. LNet makes sure this
primary NID is not changed afterwards.

Lustre-change: https://review.whamcloud.com/43788
Lustre-commit: TBD (f5008e02d8a49525b5d85887a13aad27e4a23cfd)

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I3f54c04719c9e0374176682af08183f0c93ef737
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50115
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14668 lnet: Lock primary NID logic

If a peer is created by Lustre make sure to lock that peer's
primary NID. This peer can be discovered in the background.
There is no need to block until discovery is complete, as Lustre
can continue on with the primary NID it provided.

Discovery will populate the peer with other interfaces the peer has
but will not change the peer's primary NID. It can also delete
peer's NIDs which Lustre told it about (not the Primary NID).

If a peer has been manually discovered via
lnetctl discover <nid>
command, then make sure to delete the manually discovered
peer and recreate it with the Lustre NID information
provided for us.

Lustre-change: https://review.whamcloud.com/50106
Lustre-commit: TBD (551867989ac96d50f66eb63c5be3f3b4d9b5f3a0)

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I8fc8a69caccca047e3085bb33d026a3f09fb359b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50114
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14668 lnet: peer state to lock primary nid

Introduce the following two peer states:

LNET_PEER_LOCK_PRIMARY, set by Lustre to lock the primary NID
of a peer to the NID Lustre is configured with

LNET_PEER_BAD_CONFIG, set by LNet if Lustre attempts to set
a peer's Primary NID to a NID used as the primary NID of another
peer

Lustre-change: https://review.whamcloud.com/43562
Lustre-commit: 684943e2d0c2ad095e3521586d61d007b4f49abd

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I8c55e90ad2abd083c2fc902a04d4cd06a3412bfa
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49966
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-6917 lamigo: replicate stripe size

along with stripe count to match the documentation.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If5733fac7f92b821cdec4ca91e4e1d32656cd5b0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50071
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16585 build: remove python2 dependencies

Fixed packaging issue casued by scripts and control files.

Lustre-change: https://review.whamcloud.com/50084
Lustre-commit: TBD (from a9af6b4a1c3a97f5e41074d6a05e87908daf8cf9)

Test-Parameters: trivial
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Change-Id: I6c9b24bf811269928494af17c15627902e5fe27b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50078
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn78

New tag 2.14.0-ddn78

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibc992821d8788f57aa75278a7af72ad2241900d3

LU-16546 kernel: kernel update SLES15 SP4 [5.14.21-150400.24.41.1]

Update SLES15 SP4 kernel to 5.14.21-150400.24.41.1 for Lustre client.

Lustre-change: https://review.whamcloud.com/50004
Lustre-commit: TBD (from a1a3810ef6d37666d8d9b7441ca18e75d77c48b2)

Test-Parameters: trivial clientdistro=sles15sp4 testlist=sanity

Change-Id: Ie2461a2aa92b66b5d501b45a5ee2195cdc2cb2c4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50015
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16544 kernel: kernel update RHEL 7.9 [3.10.0-1160.83.1.el7]

Update RHEL 7.9 kernel to 3.10.0-1160.83.1.el7.

Lustre-change: https://review.whamcloud.com/49969
Lustre-commit: TBD (from 0e47caa49100a1f2e49e56c6853d6e21f3de97fc)

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I8f2e67c27e80e7ac852fc367f0bd83f2c91016ce
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49983
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14607 osp: separate buffer for large XATTR

Once XATTR is too large to fit into PAGE_SIZE, allocate value in a
separate buffer for osp_xattr_entry.

Lustre-change: https://review.whamcloud.com/43736
Lustre-commit: a1c5adf7f466cce5b9abae46704c126b1f11d6da

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ied090ff73e2e5cdeaf2d91a3670067210f2ab1d7
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50008
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16532 sec: session key bad keyring

At initialization, the session key created for GSS context is linked
to a keyring from userpace, and then unlinked from kernelspace if it
is for root, as we want to share it across all root sessions.
Sometimes initialization fails (expired token, unresponsive server,
etc.) so the key cannot be unlinked. Survive this use case gracefully.

Lustre-change: https://review.whamcloud.com/49909
Lustre-commit: 1f034cf610deeb6b070338a9d005b75ce964f945

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8ad2afa0e51e50640620e36211e5db1253d85e08
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50002
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16221 kernel: new kernel [RHEL 9.1 5.14.0-162.12.1.el9_1]

This patch makes changes to support new RHEL 9.1 release
for Lustre client.

Lustre-change: https://review.whamcloud.com/48938
Lustre-commit: a05d02ea0e43bc656b0c25b8cd821323857e6cc2

Test-Parameters: trivial clientdistro=el9.1 testlist=sanity
Test-Parameters: serverdistro=el8.7 clientdistro=el9.1 testlist=sanity
Change-Id: Ied1bb5a3015e7cbb9156a0ccd748fd8c3069da37
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50060
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16510 build: fortified memcpy from linux 6.1

The fortified memcpy() from Linux v5.11-11104-ga28a6e860c6c
through v5.18-rc5-1405-g43213daed6d6 incorrectly reports
a false positive out of bounds check.

In function 'memcpy' ...
'__read_overflow2' declared with attribute error: detected
read beyond size of object passed as 2nd parameter

Lustre-change: https://review.whamcloud.com/49811
Lustre-commit: 919b93b951d4a9aa0400b9c882a1f68b79d8f118

Test-Parameters: trivial
HPE-bug-id: LUS-11459
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I3a59d8b647833c05ff4b51e327ed8bce894141fe
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49880
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13485 libcfs: Parallel configure tests for libcfs

Transform the compile tests in libcfs to run in parallel

Lustre-change: https://review.whamcloud.com/38349
Lustre-commit: 182fa9be075f5866aba2f37fbc3434cd0292ac0e

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I76ab65558dd456dc08d6ef4a1985455ce1f17913
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49879
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15280 llog: fix processing of a wrapped catalog

Several issues were found with "lfs changelog --follow" for a wrapped
catalog (llog_cat_process() with startidx):

1/ incorrect lpcd_first_idx value for a wrapped catalog (startcat>0)

The first llog index to process is "lpcd_first_idx + 1". The startidx
represents the last record index processed for a llog plain. The
catalog index of this llog is startcat.
lpcd_first_idx of a catalog should be set to "startcat - 1"
e.g:
llog_cat_process(... startcat=10, startidx=101) means that the
processing will start with the llog plain at the index 10 of the
catalog. And the first record to process will be at index 102.

2/ startidx is not reset for an incorrect startcat index

startidx is relevant only for a startcat. So if the corresponding llog
plain is removed or if startcat is out of range, we need to reset
startidx.

This patch remove LLOG_CAT_FIRST, that was really confusing
(LU-14158). And update osp_sync_thread() with the
llog_cat_process() corrected behavior.

It modifies also llog_cat_retain_cb() to zap empty plain llog directly
in it (like for llog_cat_size_cb()), the current implementation is not
compatible with this patch.

The test "conf-sanity 135" verify "lfs changelog --follow" for a
wrapped changelog_catalog.

Lustre-change: https://review.whamcloud.com/45708
Lustre-commit: TBD (from 0087f5428484dc1fbab66fabb661d8700b1cac9a)

Test-Parameters: testlist=conf-sanity env=ONLY=135,ONLY_REPEAT=10
Test-Parameters: testlist=sanity env=ONLY=60a,ONLY_REPEAT=20
Test-Parameters: testlist=conf-sanity env=SLOW=yes,ONLY=106,ONLY_REPEAT=10
Fixes: a4f049b9 ("LU-13102 llog: fix processing of a wrapped catalog")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Signed-off-by: Xing Huang <hxing@ddn.com>
Change-Id: Iaf46ddd4a6ec1e06cec0d17aa9bde766bd793abc
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50017
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16433 llite: check vvp_account_page_dirtied

This patch removes duplicated codes from vvp_set_pagevec_dirty()
and check vvp_account_page_dirtied to determine if falling back
to call __set_page_dirty_nobuffers().

HAVE_ACCOUNT_PAGE_DIRTIED_EXPORT also needs to be checked because
vvp_account_page_dirtied is not defined if account_page_dirtied
is exported.

Lustre-change: https://review.whamcloud.com/49512
Lustre-commit: 61c4c2b5e5d7d919149921d913322586ba5ebcab

Test-Parameters: trivial clientdistro=sles15sp4 env=SANITY_EXCEPT="27J 101j 244a" testlist=sanity
Fixes: 3f0bee2502 ("LU-15959 kernel: new kernel [SLES15 SP4 5.14.21-150400.24.18.1]")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I272033d7494a157145224b1b8ce999a80958aa6c
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50025
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-6916 build: fix default_groups handling in ofd

The ofd_procfs_add_brw_stats_symlink() function was missed during
the original kobj_type default_groups change on master because it
is disabled by a LUSTRE_VERSION_CODE > 2.14.53 check.  This causes
a crash at mount time due to accessing the (now) uninitialized
default_attrs fields that are still available in RHEL8.7 kernels.

   Call Trace:
    ofd_device_alloc+0xcb/0x180 [ofd]
    obd_setup+0x10f/0x2c0 [obdclass]
    class_setup+0x575/0x750 [obdclass]
    class_process_config+0xe68/0x1a40 [obdclass]
    class_config_llog_handler+0x92e/0x1410 [obdclass]
    llog_process_thread+0xf16/0x1a30 [obdclass]
    llog_process_thread_daemonize+0x9b/0xe0 [obdclass]
    kthread+0x10b/0x130
    ret_from_fork+0x1f/0x40

Update to using the interop accessor functions provided by LU-16120.

Test-Parameters: trivial testlist=runtests serverdistro=el8.7
Fixes: db7be91e0a ("LU-16120 build: Add support for kobj_type default_groups")
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Change-Id: I0117dc4541230ffc9ff6d366773d859d2fce7057
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50055
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12275 sec: disable bio functions on client

The reference to bio_alloc() on the client is causing compile
issues with different kernels and adds unnecessary portability
complexity to the client code.

Disable llcrypt_zeroout_range() for client builds because it is
not needed since direct block device access is not used by llite.

Lustre-change: https://review.whamcloud.com/50023
Lustre-commit: TBD (from 0ebb7e37b85f8e881985bdcb45d3e16ace37c1f0)

Test-Parameters: trivial
Fixes: a813e8187 ("LU-12275 sec: add llcrypt as file encryption library")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I97642dfd85053b9ea4196374f2002ffb6a2540e5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50022
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16221 revert: new kernel [RHEL 9.1 5.14.0-162.12.1.el9_1]

This reverts commit a7a6107dbe17be5a5f416bd2366b8bf75f6506e6.
This inadvertently triggered el9.1 to be built for all patches,
but the el9.1 support is not fully landed yet.

Change-Id: Ic51d0e65af9b2e57814738872fbb48a04db33738
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50043

LU-16221 kernel: new kernel [RHEL 9.1 5.14.0-162.12.1.el9_1]

This patch makes changes to support new RHEL 9.1 release
for Lustre client.

Lustre-change: https://review.whamcloud.com/48938
Lustre-commit: a05d02ea0e43bc656b0c25b8cd821323857e6cc2

Test-Parameters: trivial clientdistro=el9.1 \
env=SANITY_EXCEPT="130 244a" testlist=sanity

Change-Id: I8af730f84c9ddf9dcb7e3ddfbd24a68173f51e8d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49881
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-16555 obdclass: print more special chars for jobid

Print more YAML compatible special chars for jobid.
Currently they are any of ".@-_:/".

Lustre-change: https://review.whamcloud.com/49998
Lustre-commit: TBD (from 6117dedfc8823d7b0de0f5912cda18ea16b0d6a7)

Test-Parameters: trivial
Fixes: 338381574b ("LU-11407 tgt: cleanup job_stats output printing")
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: Ic3272b73c95b76ad3171cc7a368a18f804b9aa3e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/50001
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16292 llite: delete_from_page_cache not exported

Linux commit v5.16-rc4-44-g452e9e6992fe
filemap: Add filemap_remove_folio and __filemap_remove_folio

Directly removing a folio/page from the page cache is not
available.

Fallback to generic_error_remove_page for regular files,
and truncate_inode_pages_range as appropriate.

Lustre-change: https://review.whamcloud.com/49069
Lustre-commit: 738e69d4b97d28ef037fe50f4146aabead9a2528

Test-Parameters: trivial
HPE-bug-id: LUS-11198
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I634e7d7719d497ce035a78b424be8e9e8c5a8104
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49877
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16118 build: Workaround __write_overflow_field errors

Linux commit v5.17-rc3-1-gf68f2ff91512
   fortify: Detect struct member overflows in memcpy() at compile-time

memcpy and memset of collections of struct members
will trigger:

error: call to ‘__write_overflow_field’ declared with attribute
   warning: detected write beyond size of field (1st parameter);
   maybe use struct_group()?
   [-Werror] __write_overflow_field(p_size_field, size);

Lustre-change: https://review.whamcloud.com/48364
Lustre-commit: a3a51806ef361f55421a1bc07f64c78730ae50d5

Test-Parameters: trivial
HPE-bug-id: LUS-11194
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iacd1ab03d1b90ce62b5d7b65e1cd518a5f7981f2
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49876
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-16113 build: Fix configure tests for lock_page_memcg

Linux commit v5.15-12273-gab2f9d2d3626
mm: unexport {,un}lock_page_memcg

Fails when lock_page_memcg exists but is not exported.

Adjust usage of [un]lock_page_memcg() to vvp_[un]lock_page_memcg() and
define the mapping accordingly to avoid the compile error.

Lustre-change: https://review.whamcloud.com/49144
Lustre-commit: 61e83a6f13064aa9ae0d0e8ce8244493bad1f1bd

Test-Parameters: trivial
HPE-bug-id: LUS-11189
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I18029d078a00a0b21a14721bcdf953939b4118a1
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49857
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16342 mdt: not copy pool_name to quotactl in reply

Don not copy pool_name in mdt reply to avoid out-of-bounds:
BUG: KASAN: slab-out-of-bounds in mdt_quotactl+0x13ff/0x1430 [mdt]

Lustre-change: https://review.whamcloud.com/49242
Lustre-commit: 7e5f927458544bd2681027bfc3df6136d059121f

HPE-bug-id: LUS-10579
Change-Id: I34c4cd8aaccd938c95005dca06644e02132def34
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://es-gerrit.dev.cray.com/160899
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Tested-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49856
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-16115 build: Linux 5.17 external module support

Linux commit v5.16-rc3-26-g129ab0d2d9f3

Added quotes around "$(CONFIG_CC_VERSION_TEXT)", however
.config stores CONFIG_CC_VERSION_TEXT with quotes thus
breaking the GNU make Makefile for external modules.

We need to supply a non-quoted value and override the
value in .config before it is used.

Lustre-change: https://review.whamcloud.com/48360
Lustre-commit: 945ab61176213e082dfd6b3a2a35aa4687e31129

LU-16058 build: proc_ops check fails with SUBARCH undefined

During configure with config.cache enabled SUBARCH may not
be defined.

Move the definition to a location that must be traversed.

Lustre-change: https://review.whamcloud.com/48101
Lustre-commit: 8a7aa8d5902fcffe6c8b56d0c92ee36380a26e88

Test-Parameters: trivial
Fixes: a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
HPE-bug-id: LUS-11190
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I997b6987ef37a8c5b9d8f0984e81d9402a2ea705
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49855
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-14776 zfs: fix Ubuntu 20 HWE build issues

With newer Ubuntu systems using ZFS dkms have the following build
errors:

    In file included from zfs/2.0.2/source/include/sys/arc.h:32,
                 from lustre/osd-zfs/osd_internal.h:50,
                 from lustre/osd-zfs/osd_handler.c:51:
    zfs/2.0.2/source/include/sys/zfs_context.h:45:10:
                 fatal error: sys/types.h: No such file or directory
    45 | #include <sys/types.h>
       |          ^~~~~~~~~~~~~
    compilation terminated.

This is due to layout of the tree containing the needed headers.
Include those paths in build system.

Lustre-change: https://review.whamcloud.com/44749
Lustre-commit: b1723622b88e693cac1d65bc317188f256a82558

Test-Parameters: trivial
Change-Id: I453830c4111ad88ec655d3d7d0ee51627331cb0b
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49854
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-13485 build: Parallel build configure cache fixes

This fixes the infrastructure for parallel builds when cached
enabled results are enabled, the critical fix being proper
usage of AC_CACHE_CHECK in the LB2_LINUX_TEST_RESULT macro.

Lustre-change: https://review.whamcloud.com/49149
Lustre-commit: 24c9f1f4c56d266656572e7bd97197d690d49369

Test-Parameters: trivial
Fixes: b0209c2d4d ("LU-13485 build: Enable 2 stage configure tests")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I94d7ff291dfd4f2dc5e218acc811329b986f8fbf
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49853
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16302 llite: Use alloc_inode_sb() to allocate inodes

linux-commit: v5.17-49-g8b9f3ac5b01d
fs: introduce alloc_inode_sb() to allocate filesystems specific
inode

Filesystems are expected to use alloc_inode_sb to allocate inodes
for proper lru handling.

Lustre-change: https://review.whamcloud.com/49070
Lustre-commit: a883fec55694491a26b664c2cff7fb904f72b7af

Test-Parameters: trivial
HPE-bug-id: LUS-11332
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ie6f091a01df33738ed2ef6f7fef9c1f9c1a51e03
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: jsimmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49852
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-16121 llite: invalidate_folio and dirty_folio

linux commit v5.17-rc4-10-g128d1f8241d6
fs: Add invalidate_folio() aops method

A struct folio is often analogous to a struct page however
a struct folio can represent (contain) multiple pages.

linux commit v5.17-rc4-38-g6f31a5a261db
fs: Add aops->dirty_folio

__set_page_dirty_nobuffers() is replaced with filemap_dirty_folio()

Lustre-change: https://review.whamcloud.com/48366
Lustre-commit: f1604e680ba2b919c0c52c76c75a7e0dc27ec3c5

Test-Parameters: trivial
HPE-bug-id: LUS-11197
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iefe67615b333e066c49c4b884dad5bea3b3ae226
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49850
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16117 build: Avoid excessive modpost warnings

To avoid modpost warnings about duplicate symbols do not add
the LINUX_OBJ kernel symbols to the KBUILD_EXTRA_SYMBOLS list

Lustre-change: https://review.whamcloud.com/48362
Lustre-commit: 445f85de2baddbaa8a84a709b98a9d0b8ed4231a

Test-Parameters: trivial
HPE-bug-id: LUS-11192
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I85fc90661efcb66e4aa39c9bd3393dbe4f7ba5eb
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49849
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16116 build: Configure tests for rhltable, bitmap_alloc...

rhel8.6 with kernel 5.18 breaks a couple of compile tests

struct rhltable test fails with:
... error: ‘hlt’ is used uninitialized in this function
    [-Werror=uninitialized]

rdma_wr() test failes with:
... error: assignment discards ‘const’ qualifier from pointer
    target type [-Werror=discarded-qualifiers]
   wr = rdma_wr(NULL);

Lustre-change: https://review.whamcloud.com/48361
Lustre-commit: 009faf132dc47dcfb604b95c93211da7d2c644e2

Test-Parameters: trivial
HPE-bug-id: LUS-11191
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ib2b1d223ac809cea157158fe35fd2535b04367df
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49848
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16118 build: Use pde_data() when available

Linux commit v5.16-11573-g6dfbbae14a7b
introduce pde_data() and
Linux commit v5.16-11574-g359745d78351
remove PDE_DATA()

Use PDE_DATA() when pde_data is not available.

Lustre-change: https://review.whamcloud.com/48363
Lustre-commit: d54e8e95de88e13f193bd44eb43662e75b2091c1

Test-Parameters: trivial
HPE-bug-id: LUS-11193
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ida570462acd466a251adc81a14bc1fbf35d96b00
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49847
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16120 build: Add support for kobj_type default_groups

Linux commit v5.1-rc3-29-gaa30f47cf666
kobject: Add support for default attribute groups to kobj_type

Linux commit v5.18-rc1-2-gcdb4f26a63c3
kobject: kobj_type: remove default_attrs

Switch to using kobj_type default_groups when it is available.
Provide support for default_attrs for older kernels.

Lustre-change: https://review.whamcloud.com/48365
Lustre-commit: 62e9d055d9516ec6a2fe2f0962b23d478a45dad0

HPE-bug-id: LUS-11196
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I43b03c67c22307293a2abc444aa1a73889ca09ee
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49838
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-8066 lov: move stripesize to sysfs

One lonely simple proc file exist for lov. Move stripesize to
sysfs.

Lustre-change: https://review.whamcloud.com/43212
Lustre-commit: 28bac98fcb44aabb08258121fcfd39eb46cff0d7

Change-Id: I4db660f0e7af4d69c697f8c73547a245108adb9b
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49837
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16359 build: RHEL use Module.symvers during find-provides

find-provides fails to find module versions on newer
kernels.

The generated Module.symvers is always generated and
correct. Install it to the well known location BUILDROOT
use it to generate provides and ignore it for installation.

Create a new find-provides and find-provides.ksyms for
lustre based on the one provided by the redhat-rpm-config
package using Module.symvers to supply the symbol versions
instead of extracting symbol versions from the .ko files.

Lustre-change: https://review.whamcloud.com/49302
Lustre-commit: bb951b90268bb013b55166e1c0bcae1c2a66b165

Test-Parameters: trivial
HPE-bug-id: LUS-11383
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I01c3b3692e6a2a6be86a6930eaead9df75147f90
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49836
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16291 build: make kobj_type constant

Kernel v5.16-rc2-28-gee6d3dd4ed48:
commit ee6d3dd4ed48ab24b74bab3c3977b8218518247d
driver core: make kobj_type constant.

This patch makes struct kobj_type constant to fix
the following build failure against kernel 5.16:

lustre/obdclass/obd_config.c: In function 'class_modify_config':
lustre/obdclass/obd_config.c:1639:13: error: assignment discards
'const' qualifier from pointer target type [-Werror=discarded-qualifiers]
1639 | typ = get_ktype(kobj);
| ^

Lustre-change: https://review.whamcloud.com/49043
Lustre-commit: d1dbf26afd6676e02a2a00e635b9ad1fe14cf68e

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I19e0d1f4e3cf97f6871e038487cda9294ac1f67b
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49835
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>

LU-16112 build: ki_complete removed unused argument

Linux commit v5.15-rc6-145-g6b19b766e8f0
fs: get rid of the res2 iocb->ki_complete argument

Prior to 4.1 Linux provided an aio_complete(iocb, res, res2)
which propagated res2 to io_event.res2. This functionality
migrated to iocb->ki_complete().

Provide a wrapper around iocb->ki_complete() to use
aio_complete() or iocb->ki_complete() as appropriate.

Lustre-change: https://review.whamcloud.com/48357
Lustre-commit: c13eccf71ddee4bc8bc8cbee8f927bdba8768735

HPE-bug-id: LUS-11187
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I11d1ee61528d4d89e2a316fd71066824b202dac7
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49834
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16111 build: Fix include of stddef.h

In kernel builds include the linux/stddef.h

Lustre-change: https://review.whamcloud.com/48367
Lustre-commit: 95f7ef609467a9534b0b1639eba0ef5a2c02f21f

Test-Parameters: trivial
HPE-bug-id: LUS-11185
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0db81e01fadd01445515f96b3d04a2ec51f43044
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49781
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13903 utils: separate out server code for lctl

The utility lctl is used by both client and server management but
the code is not cleanly separated for each case. This breaks
building lctl for the native Linux Lustre client since it only
contains client related work. Fix up lctl to be slimed down for
client only targets.

Lustre-change: https://review.whamcloud.com/42141
Lustre-commit: 0b7ddf32e336b06d459880faf9831db43ffae9cf

Change-Id: I0c8c037087de1155e478a2580fc952629c341447
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49779
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13903 utils: move userland only nidstr.h handling

The function cfs_expand_nidlist() no longer exist for kernel
internals. We can move the function prototype from the UAPI
header to string.h which is a libcfs user land header.
The structure netstrfns that is defined in a UAPI header
has been adding user land only handling. Additional its
use struct list_head which will confuse reviewers since
kernel developers see this as a kernel only thing.

Lustre-change: https://review.whamcloud.com/39115
Lustre-commit: 062809b1313ac7000e3476c96cbbe5d017f10875

Test-Parameters: trivial

Change-Id: Ifc3c87f6d3237a94d282d009455ff389278e73ea
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49777
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13903 build: make lustre-devel buildable for Linux client

Recently a new lustre-devel rpm was created which always has a
dependency on the kernel detected by the kmod macros. This doesn't
work in the case of running a custom Linux kernel. Addreess this
by not making use of the kmod macros when only building the Lustre
utilies i.e configure --disable-modules.

Lustre-change: https://review.whamcloud.com/41188
Lustre-commit: d498d1b9cc1bd199e97af3817396fdcada44d655

Test-Parameters: trivial
Fixes: 16af4e5ed634 ("LU-9215 build: Re-add the lustre-devel package")
Signed-off-by: James Simmons <jsimmons@infradead.org>
Change-Id: I324fbaa0f5b03e2095b493f3d8e00b74ca64298a
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49773
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15393 tests: check QoS hang with OST failover

Patch adds recovery small test 152, to reproduce situation
where MDT object allocation sleeps on OST failover at
lod_ost_alloc_rr under lq_rw_sem read. And all other creation threads
hang at lod_ost_alloc_qos at down_write(lq_rw_sem).

Lustre-change: https://review.whamcloud.com/47715
Lustre-commit: 52057d85eaef8c7b5262f0718629fabff919ff1d

HPE-bug-id: LUS-10388
Test-Parameters: trivial testlist=recovery-small env=ONLY=152
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I7b9c5a5c9870a559e673a5fd253dcaea40d9fe63
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49758
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15282 lod: less spinlock on the alloc rr

Don't need to hold spinlock for so much time, anyway it's released
in middle of loop, so RR cannot be perfect in multithreaded case.

Fix small bug in RR precession for stripecount=4+OSTCOUNT=6.

Lustre-change: https://review.whamcloud.com/45694
Lustre-commit: fa34a15f619c49535e214eec0d076872ac9a6b9a

Fixes: 665e36b780f ("OST pools on HEAD")
HPe-bug-id: LUS-10627
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I66eded451c8256de0e5a9a0eb862af8b306da9e1
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49757
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn77

New tag 2.14.0-ddn77

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4dfa64520bda390dd7334fa0ec9bf847306e140b

LU-13703 utils: fix lfs_migrate with PFL arguments

Pass the '-c', '-S', and '--pool' options to "lfs migrate" when
they are part of a PFL component (after -E), rather than using
them to set the stripe_count and stripe_size of the whole file.

This precludes using '-A' and '-R' with explicitly specified PFL
file layouts, but that didn't make sense in the first place.

Fix the handling of "--pool <pool>" to use "-p <pool>" since
the script later only strips "-p " from the pool name.

Lustre-change: https://review.whamcloud.com/39145
Lustre-commit: 28597c233c980f53732ab7628c2ec38efbff5364

Test-Parameters: trivial
Fixes: 60c5bc25025 ("LU-8235 scripts: pass unrecognized options to lfs migrate")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib7fb6e08d81dbae77e8348fc5f09837c612540e5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49868
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13475 utils: disable lfs_migrate rsync and warning

The --rsync option is no longer enabled by default for fallback if
'lfs migrate' fails for some reason, and is mandatory for rsync usage.
The warning message and "-y" option of lfs_migrate is no longer needed
if rsync is not used, and is only shown if --rsync is used.

Remove the LFS_MIGRATE_RSYNC_MODE variable that was used for tests
and instead pass the "--rsync" option directly when needed.

Lustre-change: https://review.whamcloud.com/40614
Lustre-commit: 1be889a5bc713ab8db6c3d2cbf83aef8df813293

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I70b70d969f2dc8b4836c6c7692e6a73a0e2540e5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49867
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16536 osp: don't cleanup ldlm in precleanup phase

instead do this in cleanup phase so that all OSPs have chance
to abort in-flight RPCs which can block MDT thread holding
LDLM locks.

Lustre-change: https://review.whamcloud.com/49925
Lustre-commit: eed4d4c7523c26cfc5bc230986d96b2acf152dee

Fixes: 226fd401f9 ("LU-7660 dne: support fs default stripe")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib3714b29c514a7fa938d47717dc36525654407d6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49926
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15880 quota: fix insane grant quota

Fix the insane grant value in quota master/slave index,
the logs often contain the content similar to the following,

LustreError: 39815:0:(qmt_handler.c:527:qmt_dqacq0())
$$$ Release too much! uuid:work-MDT0000-lwp-MDT0002_UUID
release:18446744070274413724 granted:18446744070291193856,
total:4118877744 qmt:work-QMT0000 pool:0-dt id:40212 enforced:1
hard:128849018880 soft:12884901888 granted:4118877744 time:0
qunit: 16777216 edquot:0 may_rel:0 revoke:0 default:no

It could be caused by chgrp, which reserves quota before changing
GID for some file at MDT, then release the reserved quota after
the file GID has been changed on the corresponding OST, (this issue
is tracked at LU-5152 and LU-11303)

In some case, some quota could be released even the quota was not
reserved correctly, which cause the grant quota to be some negative
value, which is regarded as some insane big value because the type
of grant is "__u64", then the normal grant release will fail and
the grant field of some quota ID in the quota file (both at QMT and
QSD) contain insane value, but can't be reset correctly.

This patch resets the affected quota by clear the quota limits and
grant, and the grant will be reported by each QSD when the quota ID
is enforced again, then rebuild the grant at QMT.

Lustre-change: https://review.whamcloud.com/48981
Lustre-commit: a2fd4d3aee9739dcb23ac3bf46d221a978808463

Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I083afa3b6648db5a1ccca0235667da022ff27e65
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49832
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13482 utils: bandwidth limit for lfs migrate

Add an option -W to control how much bandwidth
an lfs migrate job can consume. The migrate job
will periodically sleep to meet the bandwidth
restrictions.

This patch also adds a --stats option. The option
produces regular logs entries tracking the progress
of the migrate job. The logs are output in YAML
format. The frequency of the logs is controlled
by --stats-interval. This interval defaults to 5
seconds.

Also included are two tests, 56xh and 56xi. The
first verifies the functionality of the bandwidth
control. The second checks that the output is in
valid YAML and that the stats get printed without
using -W.

Lustre-change: https://review.whamcloud.com/49620
Lustre-commit: 55968bfabe351ad37ee810bf69748ffa56d28037

Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ic71cceb2434a737e3ad8bd325f719e37a70b0047
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49924
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14508 lfs: make mirror operations preserve timestamps

Save and try to restore the file timestamps around the various mirror
operations. Add sanity-flr tests 61[abc] to verify this.

Lustre-change: https://review.whamcloud.com/42009
Lustre-commit: 9f672d8a71d8f77d2e807f1d4df926239ade8ebd

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I5ef754e46cfbe82c731a709209576bbfcc73af3d
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49928
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-6807 tests: sync MDT for sanity-lipe-find3/109

Added sync all data to ensure that new created directory
flushed on MDT and lipe_scan3 works reliably
when scans inodes

Test-Parameters: trivial mdscount=2 mdtcount=4 ostcount=8 env=ONLY=109 testlist=sanity-lipe-find3
Test-Parameters: trivial testgroup=full-dne-part-2
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: If0e8b7de024a1c8c7e1124173e9b3472ea8616b0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49860
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16159 osp: destroy should not overtake writes

use transaction versioning for object destroy so that
destroy doesn't overtake writes, so writes don't hit
non-existing objects.

Lustre-change: https://review.whamcloud.com/49787
Lustre-commit: e3367a423ae09fcf133ecb7d9b21abfe549e22c6

Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Test-Parameters: mdscount=2 mdtcount=4 testlist=replay-single env=ONLY="70b 71a 119",ONLY_REPEAT=10
Fixes: b054fcd785 ("LU-16159 lod: cancel update llogs upon recovery abort")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iec2a5c72f27825820d36ebbe20d55fa303358982
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49830
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14909 test: mkdir_on_mdt0 to mkdir on MDT0

Many sub tests in recovery-small.sh and replay-single.sh need to mkdir
on MDT0, use mkdir_on_mdt0() to create such directories.

Lustre-change: https://review.whamcloud.com/44544
Lustre-commit: 96fd8f03c57d778fd40055b58f54b7310b704adc

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=replay-single.sh
Test-Parameters: mdscount=2 mdtcount=4 testlist=recovery-small.sh
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id4a44da062350ea284f51c8c821302aebbfe9dee
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49946
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16101 tests: add sanity/27J to always_except

This patch adds sanity/27J to always_except for SLES15 SP4
and 5.16.0+ kernels before the issue introduced by upstream
commit 8c8387ee3f55
("mm: stop filemap_read() from grabbing a superfluous page")
is resolved.

Lustre-change: https://review.whamcloud.com/49970
Lustre-commit: TBD (from a537ec478649579ab66866bae85083c68c82e96b)

Test-Parameters: trivial clientdistro=sles15sp4 testlist=sanity

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Iafde656530fcdc1de9265aacaa9266435c9d5c47
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49972
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Xing Huang <hxing@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn76

New tag 2.14.0-ddn76

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iac23eaba4042970458eac24a01ce180d303fd5ff

LU-16374 enc: align Base64 encoding with RFC 4648 base64url

Lustre encryption uses a Base64 encoding to encode no-key filenames
(the filenames that are presented to userspace when a directory is
listed without its encryption key).
Make this Base64 encoding compliant with RFC 4648 base64url. And use
'+' leading character to distringuish digested names.

This is adapted from kernel commit
ba47b515f594 fscrypt: align Base64 encoding with RFC 4648 base64url

To maintain compatibility with older clients, a new llite parameter
named 'filename_enc_use_old_base64' is introduced, set to 0 by
default. When 0, Lustre uses new-fashion base64 encoding. When set to
1, Lustre uses old-style base64 encoding.

To set this parameter globally for all clients, do on the MGS:
mgs# lctl set_param -P llite.*.filename_enc_use_old_base64={0,1}

Lustre-change: https://review.whamcloud.com/49581
Lustre-commit: 583ee6911b6cac7f2867a37101cc069b4011b73f

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaa2256da7fb591d842b5bb7aa474b2ee6de9899d
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49900
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16415 quota: enforce project quota for root

Patch adds an option to enforce project quotas for root.
It is disabled by default, to enable set
osd-ldiskfs.*.quota_slave.root_prj_enable to 1
at each target where you need this option.

Patch also adds sanity-quota_1j to test new feature.

Lustre-change: https://review.whamcloud.com/49460
Lustre-commit: f147655c33ea61450105b602c82da900fd1417b5

Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I978dc8442235149794f85110309f63bc560defdc
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49812
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-12275 tests: skip new nodemap params on old MGS

Skip setting forbid_encryption and readonly_mount parameters on old
MGSes that do not support these options. Otherwise test_61 failures
are seen during interop testing. Running test_36 would also fail in
this case, except that it is already skipped due to encryption checks.

Lustre-change: https://review.whamcloud.com/49828
Lustre-commit: TBD (from 74c9b1f6e4d5b1fbbd615b87fb7c62c0fcb1a727)

Test-Parameters: trivial testlist=sanity-sec
Fixes: 598c48707c ("LU-12275 tests: exercise file content encryption/decryption")
Fixes: e7ce67de92 ("LU-15451 sec: read-only nodemap flag")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I94f2e2f609927fea618a3a22f103bd32ae3ebbe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49829
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9859 libcfs: add "default" keyword for debug mask

Allow "lctl set_param debug=default" to reset the debug mask to
the default value. This is useful if the debug needs to be set
to a higher value temporarily, but should be easily reset back
to the original value afterward.

Lustre-change: https://review.whamcloud.com/46251
Lustre-commit: 4c9a5762413638cc630b1facfb565dcd765fce1e

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7d0d8fb81e51afb5ea6f29abea0d0814de3ebbe5
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49875
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16191 socklnd: limit retries on conns_per_peer mismatch

If connection initiator has a higher conns-per-peer setting than
its peer, don't try to create extra connections forever as the
peer will keep rejecting them. A few retries should suffice to
resolve a valid race.

Lustre-change: https://review.whamcloud.com/48664
Lustre-commit: da893c6c9707ca3b2e7532d05f754fccf1cffc74

Test-Parameters: trivial
Fixes: 71b2476e ("LU-12815 socklnd: add conns_per_peer parameter")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I7d04d4ac41e98a738b6c85c3d323608038f5c51e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49914
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16125 tests: make sanity-sec more robust with SSK

Encryption related tests in sanity-sec carry out unmount and mount of
clients in order to exercise code with and without the encryption key.
In case SSK is in use, we need to make sure flavors are properly
applied before carrying on.

Lustre-change: https://review.whamcloud.com/48386
Lustre-commit: bee889e87584aa3bd2e6819db73d6adf129460ee

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I92e85dc6dcef43f70a7fe05db94cd18fe66a3a24
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49892
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-6808 tests: allow run_lfsck() to skip error

If e2fsck or lfsck are repairing any corruption it means
the filesystem was corrupted by something, and that
shouldn't happen during testing. Running run_lfsck() will
fail and report an error.

However, after running sanityn test 70b, there will be
some known corruptions need to be repaired. We have to
make run_lfsck() support skipping some allowed errors
so as to make sanity-lipe-find3 and sanity-lipe-find3
tests proceed.

Test-Parameters: trivial testlist=sanityn,sanity-lipe-scan3
Test-Parameters: trivial testlist=sanityn,sanity-lipe-find3
Test-Parameters: trivial mdscount=2 mdtcount=4 \
testlist=sanityn,sanity-lipe-scan3
Test-Parameters: trivial mdscount=2 mdtcount=4 \
env=SANITY_LIPE_FIND3_EXCEPT=109 \
testlist=sanityn,sanity-lipe-find3

Fixes: f9ba28af02 ("EX-6692 tests: run LFSCK in sanity-lipe-scan3/find3")
Change-Id: I686820a70b1a393611d90e286febfd3512fd40f7
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49903
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16457 tests: wait for remote sleep in sanity-pcc/101a

Wait longer for the remote sleep command to start on the agent node.

Lustre-change: https://review.whamcloud.com/49587
Lustre-commit: 4b47c233b308dcfefe77a6a493c01d3b4fc59bbe

Test-Parameters: trivial testlist=sanity-pcc env=ONLY=101a,ONLY_REPEAT=200
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5dcbd6a7127b3e17aa658c87f5c75874432dc353
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49919
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16500 utils: set default ost index for lfs migrate

Running "lfs migrate <file>" without any SETSTRIPE arguments
to balance space usage keeps the PFL file layout, but preserves
the OST selection exactly, which makes the migration virtually
useless for space balancing.

This patch fixes the above issue by clearing the specific
OST indices from the source layout before using the layout to
create the volatile file in lfs_migrate().

Lustre-change: https://review.whamcloud.com/49819
Lustre-commit: TBD (from 58ce6f2c8276df8b1a3b38db016fc301334d589c)

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I82e1dc0a11fdda7d555df994cf4e5f6e3dbdcb5c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49865
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16501 tgt: skip free inodes in OST weights

In lu_tgt_qos_weight_calc() calculate the target weight consistently
with how the per-OST and per-OSS penalty calculation is done in
ltd_qos_penalties_calc(). Otherwise, the QOS weighting calculations
combine two different units, which incorrectly weighs allocations on
OST with more free inodes over those with more free space.

Lustre-change: https://review.whamcloud.com/49890
Lustre-commit: TBD (from ab24f031908d100146b2f2900ab88e99e689d236)

Fixes: d3090bb2b486 ("LU-11213 lod: share object alloc QoS code with LMV")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1ccc52d7ad5dc440ae48403ba129efd6a0a51c33
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49904
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn75

New tag 2.14.0-ddn75

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I70de832ee204c96afd2359b03530303b3282d942

LU-16412 llite: check read page past requested

Due to a kernel bug introduced in 5.12 in commit:
cbd59c48ae2bcadc4a7599c29cf32fd3f9b78251
("mm/filemap: use head pages in generic_file_buffered_read")
if the page immediately after the current read is in cache,
the kernel will try to read it.

This attempts to read a page past the end of requested
read from userspace, and so has not been safely locked by
Lustre.

For a page after the end of the current read, check wether
it is under the protection of a DLM lock. If so, we take a
reference on the DLM lock until the page read has finished
and then release the reference.  If the page is not covered
by a DLM lock, then we are racing with the page being
removed from Lustre.  In that case, we return
AOP_TRUNCATED_PAGE, which makes the kernel release its
reference on the page and retry the page read.  This allows
the page to be removed from cache, so the kernel will not
find it and incorrectly attempt to read it again.

NB: Earlier versions of this description refer to stripe
boundaries, but the locking issue can occur whether or
not the page is on a stripe boundary, because dlmlocks
can cover part of a stripe.  (This is rare, but is
allowed.)

Lustre-change: https://review.whamcloud.com/49723
Lustre-commit: TBD (from ebfe62b8e8f4db0555a7a39ce5c764059422a260)

Change-Id: Ib93bd0624fda0ed1c2b89f609d15208c86e21c29
Signed-off-by: Qian Yingjin <qian@ddn.com>
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49658
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>