git://git.whamcloud.com - fs/lustre-release.git/log

EX-9230 lipe: Add device name in file report name

This patch adds the device name (eg MDT-xxxx) to the
report name when automatically generating the name.
It also corrects the end time in the file name
(when scanning is completed) to the initial time
(when scanning began). Only for lipe_scan3.

Example of a new file name for a report:
files_sizes_report_lustre-MDT0000.2024-03-26-09:54:25.out

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I2e79404e459b5717858b92a0783fe3f1bad552ab
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54574
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9459 lipe: Fix behavior when getting attributes

This improvement is important and is intended for the
--collect-fsize-stats output policy in lipe_scan3.

This patch prevents the scanning process from stopping and
completing if any LOV attribute is not received correctly.
Instead of halting the scan, the patch adds additional error
counters, and all types of reports will now include new
error statistics.

Also add a counters for objects that have no size/allocate size.

An example of a new block with error information from a report
with the .out extension which will contain the following fields:

Error counters:
Allocated blocks is empty: 11101
Size is empty: 0
Without size (all size value empty): 59
Failed to get LOV attr: 0
Failed to get mirror count: 0
Failed to get stripe count: 0
Failed to get stripe size: 0

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I1817ea189f3d554894822ad8d12a8514546b13b0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54583
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17611 utils: fix wrong static declarations

Revert wrong changes made to zfs mount utils

Lustre-change: https://review.whamcloud.com/54293
Lustre-commit: f45a0288b00597bc797963f7aa01cae5167b024e

Test-Parameters: trivial
Fixes: c7e9bdf8d4 ("LU-8191 utils: remove unused, fix non-static functions")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I162d349ebadbf93a89abf49bd41465979d561423
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 utils: remove unused, fix non-static functions

Remove several functions which are never called.

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various Lustre utils static.

Some missing headers caused some functions being
incorrectly marked as possible candidates for
being made static. These missing headers have
been added.

Lustre-change: https://review.whamcloud.com/51439
Lustre-commit: c7e9bdf8d4bb5e1127eb87472fbf0414823d5461

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Id51f922be57c33c011ee2f9e509ca164cc480edf
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54629
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 llverfs: fix non-static functions

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in llverfs.c static.

Making functions new_file() and new_dir() static
causes new format truncation errors. Check the
return of snprintf() to silence these.

Lustre-change: https://review.whamcloud.com/53754
Lustre-commit: 986219131215e44a98703a6fb29d941b5f181aa3

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ieccf1e40c1da627571a7a95adbb85599185f1342
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54628
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert lmv,lod,lov functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in lmv, lod, and lov static.

Also, remove one unused function: lov_lsm_entry()

Another function is intentionally unused for
debugging purposes. It was detected by static
analysis, but it has been left untouched.

Lustre-change: https://review.whamcloud.com/51479
Lustre-commit: 78605b74a56a5338aa93bbf394fe35d0f7c17c5b

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If409226ea201587c7f95d4a65ffaef72671b5ac2
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54627
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 liblustre: add missing functions to header

A number of functions were missing from lustreapi.h,
causing them to be marked incorrectly as functions that
could be made static. They have been added to the
header so applications can use them.

Static analysis shows that a number of functions
could be made static. This patch also declares
several functions in liblustre static.

liblustreapi_nodemap.c and liblustreapi_ioctl.c were
missing an internal header, causing some functions
to be incorrectly flagged. This patch also adds that
header.

Initialize a previously uninitialized variable in
llapi_fid_to_handle().

Lustre-change: https://review.whamcloud.com/51434
Lustre-commit: 25523e5a35138a0534b01ff561169e501cc30787

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I67b9c59418b62602ffe36eb4284eb1e8d4a3b19b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54626
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 libcfs: convert functions to static, removed function

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in libcfs static.

Remove cfs_expr_list_values_free in string.c, since
it is not used.

A header was missing in param.c, causing a number of
functions to be missing declarations.

Lustre-change: https://review.whamcloud.com/51428
Lustre-commit: 067dfd8d2701572ebe7246726c14a6e72a78cb33

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia580881efa806bde49d532e5b2d8f5097f2294e0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lnet: convert functions in utils to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various LNet utils and lnetconfig to
static.

In LNet selftest (lst), one unused function was
removed entirely. Some declarations were moved to
made static.

Lustre-change: https://review.whamcloud.com/51427
Lustre-commit: d9cd9992b9e04bfad1ebd755f78d3e96850eaa32

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia4528281b3c87d77e46abb95f47ab0bdc72168c0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54619
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 ptlrpc: add missing headers

Missing headers for several c-files in ptlrpc
cause functions to be incorrectly marked as only
being used within their respective c-files. This
patch adds those missing headers. It also addresses
a couple minor style issues.

Lustre-change: https://review.whamcloud.com/51480
Lustre-commit: fba78a416621343aadeeb4d28df86e422b653bfe

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Idd8fa747a671079aba2b691ef23cc7564e5e2430
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54618
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert mdc,mdd,mdt,mgc functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in mdc, mdd, mdt, and mgc static.

Also, remove mgs_client_add() since it was unused, and
move a declaration from a c-file to the proper header file.

Lustre-change: https://review.whamcloud.com/51478
Lustre-commit: 38d151f2f65d76fc392d356519227a648e114b5f

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia23f62465c27c83a9a0260bb45e8c8b710491558
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert osp,osd,osc,ofd functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in osp, osd, osc, and ofd static.

Also, fix a few minor style issues.

Lustre-change: https://review.whamcloud.com/51477
Lustre-commit: 2fa075d10f72f1400e4da9bdcaec905858b0a264

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I3d7af7ec0fa2978bfdd0cb490f18f485a78f81f6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert ec,fid,ldlm,quota functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in ec, fid, ldlm, and quota static.

Lustre-change: https://review.whamcloud.com/51476
Lustre-commit: bcc828bdf88f8a19e56535ff3840e06065b606b2

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ic64bdf0d802fd4c963b7b7d3a654575ebde5c07d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54615
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 target: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in target static.

Also, remove an unused function tgt_obd_log_cancel(),
and add some headers where they were missing.

Lustre-change: https://review.whamcloud.com/51475
Lustre-commit: bcbf31a9e1bd6399053a5a757b406152f0d65a42

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I1823df3562cb181b275788560166c92b63483637
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54614
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 llite: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in llite static.

Also, conserve more * in comments.

Lustre-change: https://review.whamcloud.com/51441
Lustre-commit: a586a2bbc37879dc22382793d1704e7708b80887

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Iafa3bb84de158e31b27b7784243bc15e78187f10
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54613
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 obdclass: add static and remove functions

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in obdclass static.

There are a few functions which are never called
anywhere. These are removed. Additionally, there
is some debugging code (added 15 years ago) that
has also been removed.

Lustre-change: https://review.whamcloud.com/51440
Lustre-commit: b3f0995b772877f0bea5d3d790db441244c3c821

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I5f1d438c9663e62789d26093ec9bdd5d76a3060a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54605
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 tests: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various test helpers static.

Lustre-change: https://review.whamcloud.com/51433
Lustre-commit: 629d6bca95f96bd307d6a9da9b04d73d3fe7c68f

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I065fb4398ed1670ce6ad58cf946054f6bd1ec282
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54603
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 mdt: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in mdt_coordinator.c static.

mdt_coordinator.c:2145:9: warning: Should this function be static?
ssize_t loop_period_show(struct kobject *kobj,
^

Further patches will follow to clean up the
remaining non-static functions in other subsystems.

Lustre-change: https://review.whamcloud.com/51345
Lustre-commit: a1d332f613ac41f4a565c3aeca6633fbd618467e

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I0350b0d5c88c0a8d1f1748d1d429cdf90afb96b7
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54602
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 ptlrpc: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in ptlrpc static.

Lustre-change: https://review.whamcloud.com/51353
Lustre-commit: 41d94aa9c9c01fa88ae152625d5ffd97256bab73

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If0d92f7f4e625c146833f360806ae80b8914cc20
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54601
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16520 build: Move strscpy to libcfs common header

Ensure strscpy is available to lustre

Lustre-change: https://review.whamcloud.com/49863
Lustre-commit: 7fe7f4ca06b9c8d128f7ba36988e36f8141ed53d

Test-Parameters: trivial
Fixes: 0b406c91d17 ("LU-13642 lnet: modify lnet_inetdev to work with large NIDS")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0c3673c2aa7e6b61671521a8cabde8a364f7f6f8
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54570
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15074 build: Use strlcpy if strscpy is not available

Linux commit v4.2-rc1-2-g30035e45753b
    string: provide strscpy()

    The strscpy() API is intended to be used instead of strlcpy(),
    and instead of most uses of strncpy().

Unfortunatley strscpy is not always available.

Test for strscpy and fallback to strlcpy when strscpy is
not available.

Lustre-change: https://review.whamcloud.com/45175
Lustre-commit: 7861d0754d42ed2a02a330eb730cb43f21dd30f2

Test-Parameters: trivial
Fixes: b77a6d86936 ("LU-14665 lnet: simplify lnet_ni_add_interface")
HPE-bug-id: LUS-9546
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I43038e4a6260dafb57195ec3417ce009f5a3fad4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54569
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14665 lnet: simplify lnet_ni_add_interface

Remove an unnecessary counter and move the comment before
the relevant code. Improve error messages.

Lustre-change: https://review.whamcloud.com/43525
Lustre-commit: b77a6d86936c32bb5f36e9806323ba00a18b0f4b

Test-parameters: trivial

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: Iffc7a128b16bc1b2be7a44413a5972c97b12a5fa
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54568
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17609 sec: nodemap readonly_mount for remount

The readonly_mount property on nodemaps forces read-only mount from
clients. Clients trying rw remount (via mount -o remount,rw) should
also be forced to read-only.

Also improve sanity-sec test_61 to exercise client remount.

Lustre-change: https://review.whamcloud.com/54282
Lustre-commit: 27cf3e0ac8576841106b3fcbd58fd5d7d419197d

Fixes: e7ce67de92 ("LU-15451 sec: read-only nodemap flag")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I61f8141001d2ff9e832e5c93d8f5997479af98a6
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54561
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17605 obdclass: do not wait forever acquiring entry

The process of refreshing an entry via refresh_entry() goes through
an upcall/downcall. If the upcall succeeds, we enter a wait queue.
If after that the downcall is never called, we hit the expiry timeout,
and we get removed from the wait queue.
But if the entry is not new, the expiry time will be
MAX_SCHEDULE_TIMEOUT == LONG_MAX, which means an infinite wait.
So avoid waiting forever if an entry could not be refreshed, and call
wake_up_all() if the wait for the ACQUIRING state failed.

Lustre-change: https://review.whamcloud.com/54269
Lustre-commit: d3157bc43390b56e2de2c7251802a67ccebe4952

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I50ee59654adc221027c79cb68fa182b9abed50fa
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54551
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17612 gss: always try to unlink key in error

In case of error in context negotiation carried out in userspace,
always try to unlink key to avoid leaking it.

Lustre-change: https://review.whamcloud.com/54316
Lustre-commit: 21aa8404a42b79f7e0434cfe75411f85d7ee063a

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic771f1e4f1b6474caaa89f63c3b02678e163d3d3
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54557
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17612 sec: return keyring errors to userspace

In current code, Linux keyring errors, when using GSS Kerberos,
are all masked under a generic ECONNREFUSED error. That makes
it hard to understand the root cause of the problem
for the I/O caller.

Update the code to propagate errors from request_key() up to
the application.

struct ptlrpc_cli_ctx * gss_sec_lookup_ctx_kr(...) is modified
to now returns a NULL pointer or -errval. This is tested by callers
and propagated. NULL values are still converted to ECONNREFUSED.

Lustre-change: https://review.whamcloud.com/54296
Lustre-commit: cd8625792f10d51fceca4717544ff8016609c3be

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Change-Id: I13792f141a961036bc9f7629a4a2db692e245c41
Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54556
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17057 tests: Fix "...endpoint shutdown" under sanity-sec

This patch fixes test_0 failing with "Cannot send after
transport endpoint shutdown" by introducing wait_ssk()
in sec_setup() to deterministicly applied SSK flavor.

Lustre-change: https://review.whamcloud.com/54311
Lustre-commit: 24ae2519acd15691d9e319ffc8675cee60529b95

Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ia14021ab82913507df02dbb5a12c8596663f15d9
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-8981 csdc: execute tests if compression is enabled

"csdc: Provide finer grained enable_compression control" patch
enables CSDC by default on all archs except aarch64 and ppc64le.
No need to execute enable_compression/disable_compression for
each test, so this test changes them to compression_enabled
check that skips test if compression is disabled for the arch.

This patch forces to skip compression tests for ppc64le and
aarch64 so they stop adding noise to the test results.

Test-Parameters: testlist=sanity-compr env=SANITY_ONLY="460",ONLY="0 1001-1080" serverversion=2.15
Test-Parameters: testlist=hot-pools env=ONLY=80 serverversion=2.15
Test-Parameters: testlist=sanity-flr env=ONLY=43 serverversion=2.15
Test-Parameters: testlist=sanity-sec env=ONLY="66 67" serverversion=2.15
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I2b5deba0672f3bad028f383cf1205e66c2b05529
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54105
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9108 obdclass: disable T10PI guard from disk when decompressing

When server handles chunk unaligned read, we need to read
the chunk from disk and decompress the raw data.
This means we should not use the guard tags from the storage,
as they only match the raw data.
Disable the guard from disk and the T10PI checksum will be
recalculated later using decompressed data.

Change-Id: I49964e1412e9d02769797c64aab290b17851e26f
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>

EX-9462 build: fix the json-c-devel build error on sles12sp5

The build is failed on SLES12 SP5 system:
error: Failed build dependencies:
json-c-devel is needed by lustre-client-2.14.0_ddn107-1.x86_64

SLES12SP5 uses package "libjson-c-devel", similar to SLES15.
This patch update the version in lustre.spec.in to fix this build
failure on SLES12 SP5.

Test-Parameters: trivial
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iffeef43fa04275857ca38b7f16e5376b24c3e9bf
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54573
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-8450 tests: skip sanity-lipe-scan3/306 on el7.9

The sanity-lipe-scan3 test_309 is failing consistently with el7.9
*clients*. Exclude it until fixed or we drop this client version.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-lipe-scan3 clientdistro=el7.9 serverdistro=el7.9
Test-Parameters: testlist=sanity-lipe-scan3 clientdistro=el8.8 serverdistro=el7.9
Fixes: bf29ca2c68 ("EX-8450 tests: skip sanity-lipe-find3/306 on el7.9")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iceca83a3b85df95fe45482076170d77a6a3ebbe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54546
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>

RM-620 build: New tag 2.14.0-ddn140

New tag 2.14.0-ddn140

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I488a4d1ae1f95a90e7e1fbc088481d35ceb95555

RM-620 build: New tag lipe-2.45

New tag lipe-2.45

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I065ea0ca483e354dbaf5d30d6c204f72ce0b9609

EX-9452 tests: skip sanity-pcc pccro=1 tests

Skip subtests when running interop on MDS without PCC-RO support.
Avoid specifying "pccrw=1" for older systems without this parameter.

Fixes: 69594ef7cd ("EX-4006 pcc: make 'pccro=1' default")
Test-Parameters: trivial testlist=sanity-pcc
Test-Parameters: testlist=sanity-pcc serverversion=2.15
Test-Parameters: testlist=sanity-pcc serverbuildno=4512 serverjob=lustre-master serverdistro=el8.9
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3915ce001cf1ca821d62ba18798d8132a3909a57
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54541
Tested-by: jenkins <devops@whamcloud.com>

EX-9196 csdc: do not round lnb size up to the PAGE_SIZE

lnb size is successfully updated to the right one in case the chunk
has been decompressed but left equual to the PAGE_SIZE in case it
hasn't. This leads to the wrong file size change.

This patch set lnb size before the decompression, so if lnb hasn't
decompressed this size is left as is.

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Test-Parameters: trivial testlist=sanity-compr env=ONLY=1003
Change-Id: I4b8149cc3aab1973a1a261b2e164c321019a437e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54518
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15117 ofd: no lock for dt_bufs_get() in read path

osd_bufs_get() allocates the pages and can cause new transactions
as part of memory release procedure. this would break Lustre's
"start a transaction, then do locking" rule.

Lustre-change: https://review.whamcloud.com/48209
Lustre-commit: 85941b9fb9ef5c27870550469f2e088c4e690603

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I782f0cc6c96251ad88d5fb8d15c9ac91d382bf7e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54535
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15102 lnet: Reset ni_ping_count only on receive

The lnet_ni:ni_ping_count is currently reset on every (healthy) tx.
We should only reset it when receiving a message over the NI. Taking
net_lock 0 on every tx results in a performance loss for certain
workloads.

Lustre-change: https://review.whamcloud.com/45235
Lustre-commit: 9cc0a5ff5fc8f45aa60cd0407ae9893d5c116ccd

Test-Parameters: trivial testlist=sanity-lnet
Fixes: 8fdf2bc62a ("LU-13569 lnet: Recover local NI w/exponential backoff interval")
HPE-bug-id: LUS-10427
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I67ea3aa977cb5d67b04f7957120c29e9985c83e6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54407
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14654 tests: Ensure recovery_limit zero works as expected

When lnet_recovery_limit is set to zero (the default) peer NIs are
eligible for recovery pings indefinitely. Verify this functionality
by modifying sanity-lnet test_211 to use recovery_limit 0 to make
a peer NI re-eligible for recovery.

Lustre-change: https://review.whamcloud.com/43502
Lustre-commit: 8d1895f2f69bd2eec3ff6af5eb356740fa2c8766

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-9953
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I00cb0940133e15ec73491e875d08b6db2bff3fe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54449
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14654 lnet: Correct peer NI recovery age out calculation

The calculation to age a peer NI out of recovery is only valid if
lnet_recovery_limit is non-zero. When set to zero, we allow peer NIs
to be in recovery indefinitely.

Lustre-change: https://review.whamcloud.com/43501
Lustre-commit: 8f3f0e1219724d6e0ed727e46b28ab28203aef9f

Test-Parameters: trivial
HPE-bug-id: LUS-9953
Fixes: cc27201a76 ("LU-13569 lnet: Age peer NI out of recovery")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I6bb40ca3a9affa0eaaae9deb1cecdb03e4bb42c5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54408
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15398 lnet: Avoid peer NI recovery for local interface

If a MR peer has a MR peer entry for itself (can happen if manually
created or discovery is run on itself for some reason), then it is
possible for it to put its own interfaces into peer recovery. Problems
with local interfaces should be handled via local NI recovery.

Lustre-change: https://review.whamcloud.com/45933
Lustre-commit: fb5d7036ec356c825f2aadece68cd7c4af487680

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-10661
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I5b28195979a6113fa863b5795a4528b072610891
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51683
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 tests: Check LNet Health recovery logic

Add test cases to validate LNet Health recovery of local and peer
NIs.

The new test cases are added to the except list for aarch64 due to
an unresolved issue with the LNet drop functionality on that
architecture.

A few style issues are also addressed by this patch.

An asterisk was being supplied to the lctl net_drop_del commands when
this should have been the '-a' flag.

A bug in cleanup_testsuite is addressed where we were using the
wrong filename for the tmp files created by the subtests.

Lustre-change: https://review.whamcloud.com/39723
Lustre-commit: aa7391445519b46752b5b0adcbe5baa368750e70

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I965df2449770631caa03ced7726abb0ea76c17e6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54406
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15791 tests: Get health before removing drop rules

lnet_health_post() can race with recovery pings, so we should
wait to delete the drop rules until after we've gathered the
health and resend values.

Lustre-change: https://review.whamcloud.com/47998
Lustre-commit: 8caec97d5e89eefe250edb64e6f7ad61e12a9d71

Test-Parameters: trivial testlist=sanity-lnet
Fixes: 79ab053562 ("LU-13569 lnet: Deprecate lnet_recovery_interval")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ia7595e015809f796cafcc40382d98ab66a708a49
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54439
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Add health ping stats

Add the NI and peer NI ping count and next ping timestamp to
detailed output of lnetctl peer and net output.

Lustre-change: https://review.whamcloud.com/40314
Lustre-commit: 4c7e4aa57629660386ae2849151a0639b6177200

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I208cb3ea0b08a2984572cf0ec9874dbd09f6168e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54405
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Deprecate lnet_recovery_interval

We no longer use a static recovery interval, so remove its remaining
uses and add warning that it has been deprecated.

Lustre-change: https://review.whamcloud.com/39722
Lustre-commit: 79ab0535622782c82636cee47918dc4b19983144

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Iedc79803ef5b7ba041a531961eb77acd338abfb5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54404
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Recover local NI w/exponential backoff interval

Use an exponential backoff algorithm to determine the interval at
which unhealthy local NIs are ping'd

Introduce lnet_ni_add_to_recoveryq_locked() which handles checking
pre-conditions for whether the NI should be added to the recovery
queue, and takes a ref on the NI as appropriate.

Lustre-change: https://review.whamcloud.com/39721
Lustre-commit: 8fdf2bc62ac9c418bd8e326112da9a2835f09ccb

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Idb3789366b2e450837989f9a12eb2d598f80081c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54403
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Recover peer NI w/exponential backoff interval

Perform LNet recovery pings of peer NIs with an exponential backoff
interval.
- The interval is equal to 2^(number failed pings) up to a maximum
   of 900 seconds (15 minutes).
- When a message is received the count of failed pings for the
   associated peer NI is reset to 0 so that recovery can happen more
   quickly.

Lustre-change: https://review.whamcloud.com/39720
Lustre-commit: 917553c537a8860f57a50dc9752e3ac69d06c11c

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ic7e60455015a0236a96010c07fc0ddd02078cf92
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54402
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Only recover known good peer NIs

A peer NI should not be eligible for recovery if we've never
received a message from it.

Lustre-change: https://review.whamcloud.com/39719
Lustre-commit: 39a169cd02738a13866f3b88fbe3304dc20565d6

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Iec2fd015f6410ab91c6ef7c222cbed0204243106
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54401
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Age peer NI out of recovery

No longer send recovery pings to a peer NI that has been in recovery
for the recovery time limit. A peer NI will become eligible for
recovery again once we receive a message from it.

The existing lpni_last_alive field is utilized for this new purpose.

A check for NULL lpni is removed from
lnet_handle_remote_failure_locked() because all callers of that
function already ensure the lpni is non-NULL.

lnet_peer_ni_add_to_recoveryq_locked() now takes the recovery queue
as an argument rather than using the_lnet.ln_mt_peerNIRecovq. This
allows the function to be used by lnet_recover_peer_nis().
lnet_peer_ni_add_to_recoveryq_locked() is also modified to take a ref
on the peer NI if it is added to the recovery queue. Previously, it
was the responsibility of callers to take this ref.

Lustre-change: https://review.whamcloud.com/39718
Lustre-commit: cc27201a76574b51dc3ffb37f039b3364cab386d

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ib4676540ac4bb040690a4fb047236c54eea0e752
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54400
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10465 test: fix interop test for 4M default stripe size

New servers could use 4MiB default stripe size, so some of
the tests need to use bigger component extent or specify stripe size
explicitly to accommodate enough stripe count.

Patch includes several test fixes:
- sanity-pfl: takes into account stripe size in some tests
- sanity-flr: use bigger component size and amount of data to
saturate all stripes as expected by test
- sanity: 130g to use 1M stripe prior FIEMAP calcs
- sanity-lfsck: 36[a-c] to use 1M stripe as expected by calcs

This change for test scripts comes from:

Lustre-change: https://review.whamcloud.com/37318
Lustre-commit: ea18d7da59d369f093e340e150544f51b2f229a1

Test-Parameters: testlist=sanity-flr serverversion=2.15 env=ONLY="0 208"
Test-Parameters: testlist=sanity-pfl serverversion=2.15 env=ONLY="0 1 14 19 20 24"
Test-Parameters: testlist=sanity serverversion=2.15 env=ONLY="27 130"
Fixes: ee7dfc5ad1 ("LU-17025 llapi: Verify stripe pool name")
Fixes: 0396310692 ("LU-15727 lod: honor append_pool with default composite layouts")
Fixes: b384ea39e5 ("LU-14480 pool: wrong usage with ost list")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3cef8805247fc5253e0a0ac05157b9d609054df9
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54444
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9372 lpurge: rid of dup layout get

Delete the unnecessary lpurge_mirror_delete() repeat layout getting
code, and update fd/fdv open flags align with LU-14677 to allow
layout operations on encrypted files, even when the encryption key
is not available.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: If4e6e3a9c46ab782f08e5bc5b3710339af051719
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54527
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Alexandre Ioffe <aioffe@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14535 quota: improve quota output format

Make the output of the quota proc files more readable by removing
needless whitespace that causes it to wrap over a single line.

Lustre-change: https://review.whamcloud.com/43099
Lustre-commit: cd1847e73e5990ef797280846dc27fc0b0a876e9

[NOTE: missing changes to procfs output that change field names]

Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota serverversion=EXA5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I80fbd42dea865dff1d106724dbf69946d23ebbe5
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/46332
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9387 lipe: Update tests in lipe_scan3

This short patch fixes the lipe_scan3 and lipe_find3
automated tests. Adds new attributes to tests to
run lipe_scan3

Fixes: 9093217a72 ("EX-8130 lipe: Link a new statistics module")
Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ibe00b2eb75da00ce4a8dbfd7a4f4d74b365104de
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54440
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9125 tests: get stats_compr param in sanity-compr/1007

This patch adds getting stats_compr param in sanity-compr
test 1007 and 1008 to track data compression statistics.

Test-Parameters: trivial testlist=sanity-compr env=ONLY="1007 1008",ONLY_REPEAT=3

Change-Id: I7df25d36c8c89ec5a568bf1e8d5694a97dd18ecc
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53927
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>

LU-17434 lmv: add exclude list for remote dir

Apache Spark creating a _temporary subdirectory for staging files, and
it should be created on the same MDT as its parent directory. Add a
tunable lmv.*.qos_exclude_prefixes, if directory prefix is in this
list, lmv_create() should put it on its parent MDT.

This prefix list follows the same rule of shell environment PATH: use
':' as separator for prefixes. And for convenience '+/-' can be used
to add/remove prefixes.

Add sanity 413k.

Lustre-change: https://review.whamcloud.com/53780
Lustre-commit: 5b07dce19b1830769d7a1f7bba8b559d3ead9dfb

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I4c8a118f0630c19054934a87bee3599bdb1fe7bb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54462
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17632 o2iblnd: graceful handling of CM_EVENT_CONNECT_ERROR

There were examples in the field with RoCE setups which demonstrate
that RDMA_CM_EVENT_CONNECT_ERROR may be received when conn state
is neither IBLND_CONN_ACTIVE_CONNECT nor IBLND_CONN_PASSIVE_WAIT.
Handle this in a more gracious manner: report the event as unexpected
and allow the flow to continue.

Lustre-change: https://review.whamcloud.com/54353
Lustre-commit: 7f27a2fceef9a03d3ada74e258e774c8f5d420f0

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I58b2482207cfd821f6eac142bdefc8f5bc50f8b4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54362
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn139

New tag 2.14.0-ddn139

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I464d0618296d59093c0f85075daf3cdb653665e8

RM-620 build: New tag lipe-2.44

New tag lipe-2.44

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ife8844a38d83a63175c5cf0182e1d23405143459

EX-6247 lfsck: support compression in filter_fid

The OST object compression info has been stored in filter_fid
by EX-8038, the LFSCK also needs to handle the compression info
during processing the filter_fid.

Fixes: bf4f5295e594 ("EX-8038 csdc: expand filter_fid")
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: Id050214ba0057776a05c194dc9222117c5d7fd8b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54158
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14869 test: improve sanity-flr/200a

Make sure "flock -x" successfully returned before running mirror
resync so that it won't get into running read holding shared flock.

Lustre-change: https://review.whamcloud.com/54345
Lustre-commit: TBD (from 2ff02fec932ab6e00a4942517db76f48412db31a)

Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I6383af5d5761980d24af19efd4a4ac899f369a7d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14971 test: align mirror_io resync implementation

Align the mirror_io resync implementation with
llapi_mirror_resync_many().

Lustre-change: https://review.whamcloud.com/45202
Lustre-commit: 9898ae5347616a097193f69811a74af3ddd88349

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Icf11c4c2302f36fc0f9682e0a310058081e1214f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54327
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9371 lipe: print FID after llapi_lease_set() error

Investigate 'cannot get UNLOCK lease, ext 8: Invalid argument (22)'
Add error message and print FID when llapi_lease_set() fails.

Test-Parameters: trivial testlist=hot-pools
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: Id498f2017ccdf8d8896ce731cc65ca45ed692d62
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54339
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17566 mdt: remove duplicate call to mdt_init_ucred_reint()

Remove duplicate call to mdt_init_ucred_reint() from
mdt_reint_setxattr().

mdt_init_ucred_reint() is called in mdt_reint_internal() which is
covering all actual reinters. However, SETXATTR was converted to
reinters framework in fd908da and this call was not removed.
So mdt_init_ucred_reint() is called first in mdt_reint_internal() then
again in the specific mdt_reint_setxattr() handler, without anything
special being done on the ucred between them.

Also merge __mdt_init_ucred() and mdt_init_cred() which was
called only once, and with the same prototype.

Lustre-change: https://review.whamcloud.com/54111
Lustre-commit: 65e0802f2ada98f802d01b5672bb9349ad0dde8c

Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Change-Id: I90fed1d2709edf7337a27dd9c3cb0f75f7625135
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54368
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17175 gss: start lsvcgssd from l_getauth

If l_getauth detects it cannot connect to the socket supposed
to be opened by lsvcgssd, it tries to launch the daemon, with
predefined default values.

Lustre-change: https://review.whamcloud.com/53142
Lustre-commit: 414467762f8a034c72903bab8ebfce6e1feb8e79

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I3961ce0f548fb6ea23458edcb01a03fb8b3a617f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54369
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9400 sec: refresh entry directly

We can process entry directly for INTERNAL entry. It avoid
to release & take lock frequently.

Fixes: fb0082bba1 ("EX-4333 sec: support supplementary groups from client")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I2af9c3964978c842dac8f70ad814adb529dff39f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54273
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9280 lipe: lpurge: Add periodical stats on scanned files

- Report INFO message on each lpurge period.
- Include in the message number of purged files, number of files
for which mirror deleting failed, number of files
which are not purged due to a stale component.
- Reset the periodic counter every period.

Test-Parameters: trivial testlist=hot-pools
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: Ib70604c990801b79b0a0356991d45e83ec62db6c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54337
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17578 lnet: fix &the_lnet.ln_mt_peerNIRecovq race

To avoid race &the_lnet.ln_mt_peerNIRecovq must always be
accessed with lnet_net_lock(0) protection.

Lustre-change: https://review.whamcloud.com/54163
Lustre-commit: 0a0e881d8884a220c485c0384351da12dc8aed9f

Test-Parameters: trivial
Fixes: da23037 ("LU-16563 lnet: use discovered ni status to set initial health")
Change-Id: Ic5e0194020200afdecba4cbf5afed274b14da388
Signed-off-by: Bruno Faccini <bfaccini@nvidia.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54382
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17522 build: Distribute clang build infrastructure

Macro files:
lustre-toolchain.m4 lustre-compiler-plugins.m4
and directory:
cc-plugins

Should be included in distributed files, unconditionally.

Lustre-change: https://review.whamcloud.com/53991
Lustre-commit: 881cc6384a5e37f0e64b56c1b34563b87dcc210d

Test-Parameters: trivial
Fixes: d684885098 ("LU-16961 clang: plugins and build system integration")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I6ddedd82c6180ffd1c4134fda6af6df6bd23dd34
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54351
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16961 clang: plugins and build system integration

Clang has a plugin system. Compiler extensions can be created
by making a shared library and loading it via the "-fplugin"
options. This makes it simple to implement custom warnings
and static analyzers.

This patch adds a plugin to detect functions that should have
been made static. This plugin has been run over the majority
of the Lustre tree and patches have been submitted for all
warnings. The plugin did not return any false positives in
my testing.

It also add the "--enable-compiler-plugins" configure option,
which automatically builds and sets up the in-tree C compiler
plugins. The option force-enables the plugin regardless of
which compiler is in use. This behavior could be changed if
there is ever a need to support GCC specific plugins.

Also, add the configure checks needed to support building C++
in the Lustre tree. Clang and GCC plugins (and the compilers
themselves) are written in C++.

The license for the plugin mirrors that of the LLVM project
itself. This leaves the door open for contributing this
plugin upstream in the future. This isn't being upstreamed
now because it lacks any significant user community. Hence,
the plugin does not appear to meet the requirements for
upstreaming based on https://clang.llvm.org/get_involved.html.

Lustre-change: https://review.whamcloud.com/51659
Lustre-commit: d684885098c40fee2951feb410bec739717ac9bc

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I747ed91b53e765cc58e91a3eb9ec6c12b9908a96
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54350
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17243 pcc: replace i_mtime with inode_get_mtime()

This patch replaces i_mtime with inode_get_mtime(), and
i_mtime.tv_sec with inode_get_mtime_sec() in pcc codes.

Test-Parameters: trivial testlist=sanity-pcc

Fixes: 3c586ca ("LU-17243 build: compatibility updates for kernel 6.6")
Change-Id: I756451ac9d38c3b434bb511b33d5d891b2b914ae
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54357
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17627 build: fix new mofed version

Allow multi-digit MOFED version numbers.
Fix compare_version function to return what it should

Lustre-change: https://review.whamcloud.com/54336
Lustre-commit: TBD (from ec967a35b2ac09e780772bdfd4365f6fc7308417)

Change-Id: I0f585cb355bb34270003ae1139688080c301186a
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54289
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Deiter
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17179 tests: check the system is clean

Main part of tests cannot work correctly if the system
is not clean. So check this in the beginning of sanity-quota.

Lustre-change: https://review.whamcloud.com/52630
Lustre-commit: 7e1fb1a296ec7ab21be7ec39e2b6a38fbca76b6c

Test-Parameters: trivial testlist=sanity-quota,sanity-quota,sanity-quota
Test-Parameters: testlist=sanity-quota,sanity-quota mdscount=2 mdtcount=4
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Ibfbe4663dee8476486e96eb99ccbcea13216861b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54392
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17602 mdd: use correct fid in mdd_rename

mdd_rename() can re-insert target name back as a part of error
handling. use correct fid for that, not own target directory fid.

Lustre-change: https://review.whamcloud.com/54260
Lustre-commit: TBD (from 61a389ea659ce62790219b0d6edf72730284b007)

Fixes: 1c03346731 ("LU-17016 mdd: no EXDEV for parent dir projid mismatch")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0662fa005459416b070157a2d049fcf5ed08ae91
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54344
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn138

New tag 2.14.0-ddn138

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8320f67d93cc082c82597d6591eedf54a147b6be

RM-620 build: New tag lipe-2.43

New tag lipe-2.43

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I787e91686d0a10e34bbf855db7f02faa6a75e281

LU-15274 llite: whole file read fixes

There are two significant issues with whole file read.

1. Whole file read does not interact correctly with fast
reads - specifically, whole file read is not recognized by
the fast read code so files below the
"max_read_ahead_whole_mb" limit will not use fast reads.
This has a significant performance impact.

2. Whole file read does not start from the beginning of the
file, it starts from the current IO index. This causes
issues with unusual IO patterns, and can also confuse
readahead more generally (I admit to not fully understanding
what happens here, but the change is reasonable regardless.)
This is particularly important for cases where the read
doesn't start at the beginning of the file but still reads
the whole file (eg, random or backwards reads).

Performance data:
max_read_ahead_whole_mb defaults to 64 MiB, so a 64 MiB
file is read with whole file, and a 65 MiB file is not.

Without this fix:
rm -f file
truncate -s 64M file
dd if=file bs=4K of=/dev/null
67108864 bytes (67 MB, 64 MiB) copied, 7.40127 s, 9.1 MB/s

rm -f file
truncate -s 65M file
dd if=file bs=4K of=/dev/null
68157440 bytes (68 MB, 65 MiB) copied, 0.0932216 s, 630 MB/s

Whole file readahead: 9.1 MB/s
Non whole file readahead: 630 MB/s

With this fix (same test as above):
Whole file readahead: 994 MB/s
Non whole file readahead: 630 MB/s (unchanged)

Lustre-change: https://review.whamcloud.com/54011
Lustre-commit: f8b276734f2457880888ce2bfa2349d22062dd64

Fixes: 7864a68 ("LU-12043 llite,readahead: don't always use max RPC size")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I72f0b58e289e83a2f2a3868ef0d433a50889d4c0
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54276
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16307 tests: fix sanity-sec test_31 better

Patch "LU-16307 tests: fix sanity-sec test_31" was landed to this
branch before the master version, which got further improvements.
In particular, the test has been updated to handle IPv6 and numeric
NIDs, and it has been tweaked to run out of tree.

Lustre-change: https://review.whamcloud.com/53818
Lustre-commit: 28b4d02161c38e624efb10d4815856cb9df3dc07

Fixes: 331a0e20dd ("LU-16307 tests: fix sanity-sec test_31")
Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Idd657c7555e598d0ebc08387eac537b1c73e35be
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54279
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17476 lnet: use bits only to match ME in all cases

If NIDs belong to the same peer and matchbits are matching,
declare a match even if matchbits are matched as not available
or ignored

Lustre-change: https://review.whamcloud.com/54082
Lustre-commit: a7ae2e5515879dc31e87106314d35dc439a2c50d

Test-Parameters: testlist=sanity env=ONLY=350,ONLY_REPEAT=10
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I394c492381a2d069b34516c473220192df05fbd2
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54277
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17175 gss: background speedtest for Miller-Rabin rounds

The number of rounds used for Miller-Rabin testing of the prime
provided as input parameter to DH_check() is evaluated when the
lsvcgssd daemon starts. This speed test takes between 5 and 10 seconds
so it makes sense to run it in the background.
Any prime tested before the right number of rounds has been determined
would use the default from OpenSSL. This can lead to longer request
processing time, but this is only for a temporary and short period of
time.

Lustre-change: https://review.whamcloud.com/54088
Lustre-commit: d753dc75ad2a919e5fff3bc51c20b4569cd86a86

Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: If77f4374c5af463fdadd15979a594af1786af1df
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54278
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9007 lipe: Fix getting client mount path

This patch fixes a "Segmentation fault" issue related
to the generation of size statistics report in '*.json'
format when the client is unmounted

Test-Parameters: trivial testlist=sanity-lipe-scan3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ibae2fa2428ae7e202b08acae9354f303bbdbd739
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54284
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13805 revert "osd: Implement unaligned DIO connect flag"

This reverts commit 88d324be08a44364aea9ff73c362a5e4ed4aaf6e.

There are further compatibility issues with UDIO on master and
this flag may be used for more than just ZFS compatibility, so
should not be advertised by servers for compatibility just yet.

Change-Id: Id357fa1a735ef4b8d6d90218250888f4ee04e5af
Test-Parameters: trivial
Fixes: 88d324be08 ("LU-13805: osd: Implement unaligned DIO connect flag")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54287
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>

LU-17317 sec: fix sanity-sec test_28

Improve sanity-sec test_28 to verify that srpc_contexts is valid
YAML output.
Also remove the ctx information from the output, as printing out a
kernel pointer is not ideal.

Lustre-change: https://review.whamcloud.com/54280
Lustre-commit: TBD (from 7ff01038c81a813cd122b28d21964316e7afdf28)

Fixes: 45f2b1ca36 ("LU-17317 sec: add srpc_serverctx proc file")
Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ie48dc61adfd5017a2313981f27407c9d3b69dd71
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54281
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17317 sec: add srpc_serverctx proc file

GSS srpc contexts for client connections can already be dumped via
proc file <mdc,osc>.*.srpc_contexts.
This patch adds a new proc file to dump server side GSS srpc contexts,
e.g.:
mgs.MGS.gss.srpc_serverctx
mdt.testfs-MDT0000.gss.srpc_serverctx
obdfilter.testfs-OST0000.gss.srpc_serverctx

The GSS context information is dumped as YAML, with one line per
context, like this:
- 0000000013221bdf: { peer_nid: 192.168.56.206@tcp, uid: 0, ctxref: 1,
expire: 1707934985, delta: 3401, flags: [uptodate, cached], seq: 0,
win: 2048, key: 00000000, keyref: 0,
hdl: "0x5ae1a771fd57043:0x65a64972fda4e200",
mech: "krb5 (aes256-cts-hmac-sha1-96)" }

Because of this new syntax, sanity-sec test_28 needs to be fixed.

Lustre-change: https://review.whamcloud.com/53376
Lustre-commit: f6687bafcb296aa7c152774de65bc865c774c464

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I37da9ffe6dd5884006b36271185a4d7155ead65b
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54161
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-17545 lnet: use unsafe_memcpy() when flexible array

To avoid <memcpy: detected field-spanning write (size 64)
of single field "&lp->lp_data->pb_info" at
.../lnet/lnet/peer.c:2456 (size 16)> false positive
msgs/error.

Lustre-change: https://review.whamcloud.com/54069
Lustre-commit: 1b936f5b545f804f3ed21a18f90ceafd705291b2

Signed-off-by: Bruno Faccini <bfaccini@nvidia.com>
Change-Id: I4e2fc58e31f60b434a9050393cd65b89c54f0798
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54290
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-7729 osc: Add counters for compressed data

This patch is the first of two patches that add counters
to track client/server-side data compression statistics.
This patch add new compr_stats file in osc.*.compr_stats.

From added counters:
1. Size of compressed/uncompressed chunks written/read by
client to compressed files, in chunk/bytes;
2. Compressed page counter;

Test-Parameters: trivial testlist=sanity-compr
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I091153480e53309c641d39f271bef536296dc09e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53737
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-7729 osc: Do not iterate over chunk pages

If osc_decompress() knows chunk size, no need to iterate
other all pages in chunk. Thay can be skipped.

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: Ib84b060075c55c97eba9f74ef017c0a956e85b12
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54270
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17392 build: compatibility updates for kernel 6.7

Linux commit v6.6-rc4-53-gc42d50aefd17
mm: shrinker: add infrastructure for dynamically allocating
shrinker

Users of struct shrinker must dynamically allocate shrinker objects
to avoid run-time warnings.

Provide a wrapper for older kernels to alloc+register shinkers
and unregister+free.

Use get_group_info() and put_group_info() wrappers instead of
open coding the reference counting on group_info.usage

Lustre-change: https://review.whamcloud.com/53621
Lustre-commit: TBD (from 7e8165620e1943189d63ce4770c7722fa309a58e)

Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ie07bdb7fe3eb6060bd84f95f860f1b53d120a605
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54323
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-6142 osc: tidy up osc_init()

A module_init() function that registers the services
of the module should do that last, after all other
initialization has succeeded.
This patch moves the class_register_type() call to the
end and ensures everything else that might have been
set up, is cleaned up on error.

Linux-commit: e67f133d02e ("staging: lustre: osc: tidy up osc_init()")

Lustre-change: https://review.whamcloud.com/49458
Lustre-commit: f66b0c3b22bfcf0d7ac9383df5d87317f831a03d

Change-Id: I2a5ffb116c6d7c33a4530bab6e89a5ffe6117cea
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54322
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-6142 lustre: remove module_vars arg to class_register_type()

The module_vars arg to class_register_type() is always NULL. So it
can be removed.

Lustre-change: https://review.whamcloud.com/39383
Lustre-commit: 8d8e87a5ac7e9d072383019228270eb4681a597e

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie8d7e79075ba068dec606cc9dfcc38a90e371e5b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54321
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17243 build: compatibility updates for kernel 6.6

linux kernel v5.19-rc1-4-gc4f135d64382
  workqueue: Wrap flush_workqueue() using a macro
linux kernel v6.5-rc1-7-g20bdedafd2f6
  workqueue: Warn attempt to flush system-wide workqueues.
If __flush_workqueue(system_wq) is not available fall back to
flush_scheduled_work()

linux kernel v6.5-rc1-92-g13bc24457850
  fs: rename i_ctime field to __i_ctime
Use accessors for ctime. Provide replacements for older
kernels.

linux kernel v6.5-rc1-95-g0d72b92883c6
  fs: pass the request_mask to generic_fillattr
Provide request_mask argument where needed.

Linux commit v6.5-rc2-20-g2ddd3cac1fa9
  nsproxy: Convert nsproxy.count to refcount_t
Provide a wrapper for inc/dec of nsproxy.count

linux kernel v6.5-rc4-110-gcf95e337cb63
  mm: delete mmap_write_trylock() and vma_try_start_write()
Use down_write_trylock directly mmap_write_trylock

In preparation for kernel 6.7 the remaining inode time
accessors will be preferred:

linux kernel v6.6-rc5-86-g12cd44023651
  fs: rename inode i_atime and i_mtime fields
Use accessors for atime and mtime. Provide replacements for
older kernels.

Lustre-change: https://review.whamcloud.com/52908
Lustre-commit: TBD (from 223377dea7029118dd9c0deb1958bf7222117009)

Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ide6c2e3e8db532449850b145c2d61b972d21f649
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54122
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17081 build: Prefer folio_batch to pagevec

Linux commit v5.16-rc4-36-g10331795fb79
  pagevec: Add folio_batch

Linux commit v6.2-rc4-254-g811561288397
  mm: pagevec: add folio_batch_reinit()

Linux commit v6.4-rc4-438-g1e0877d58b1e
  mm: remove struct pagevec

Use folio_batch and provide wrappers for older kernels to use
pagevec handling, conditionally provide a folio_batch_reinit

Add macros to ease adding pages to folio_batch(es) as well
as unwinding batches of struct folio where struct page is
needed.

Lustre-change: https://review.whamcloud.com/52259
Lustre-commit: TBD (from 81c567481b7be1d9d4655a47027918f7a8d16ff8)

HPE-bug-id: LUS-11811
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ie70e4851df00a73f194aaa6631678b54b5d128a1
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54074
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Link a new statistics module

This patch links the collection of statistics for
directories with the main collection of statistics
about file sizes. This way we collect all the statistics
at one time.

This patch also adds 2 new options for collecting statistics
via lipe_find3.

The -depth option serves as a limiter on the collection and
output of directory statistics to a file.
The -top-rating option allows adjustment of the size of the
table ranking the largest directories by size.

A mechanism has also been added that aallows a copy of the
path from the filter for correct data processing.

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ia61f57536575fd17e40b6e34a0c4b9b5db9111c5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53990
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add output for dir sizes stats

This patch adds functions for displaying size statistics
for directories in the general report.
This patch adds support for *.out format only.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Iaf70aa4d84295f1a1a297b00fa45f12fb98c7625
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53983
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add entry point for dirs stats

This pr adds a function that is an entry point for
collecting statistics about directory sizes.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ide0c6006e287f69a1de99a5578ceab0070ea383e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53982
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add key func for work with tree

This patch adds two key functions to collect directory size
statistics, which contain the basic logic for adding
directories to memory and incrementing size counters.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I06e03a6be1052b7178274835169cc41d044ca1ab
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53963
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add helper functions for stats

This patch adds several helper functions for working with
directory size statistics. Also add ls3_stats_rm_first_dir()
which remove the directory from the list to increase counters.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I04807846b49d6fb0e476b8bf146ba337f80e3d5e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53962
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add func for working with paths

This patсh adds directory path processing helper functions
that will be used later to collect directory size statistics.

These functions set the stage for working on increasing the
size counters for each directory, along the entire chain in
the file or directory path.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I6e7302e9771dce2933c6730a1117fec3bc2b0fda
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53961
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Directory scan size stats

This patch adds functionality for creating new
directories and expanding memory for new child
directories in memory. Adds a function to
initialize the starting directory.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I3ff6a62ffd9d6535ed4434f517d1c93d6ae01b34
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53960
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add functions to create a rating

This patch adds new functionality to directory statistics
for working with a ranking table for the largest
directories (like TOP 100).

The creation of a structure for storing the rating occurs
after the lipe_scan3 scan is completed. The number of
objects in the structure is determined before lipe_scan3
is launched by the default value or by the user in lipe_find3
via the -top-rating option and is not expanded while lipe_scan3
is running. Adding new objects to the heap works by the logic
of replacing the object with the smallest size in the heap with
a new object if its size is larger. Adding objects to the heap
occurs when printing the results about the directory sizes,
since only in this case do we know the final sizes of the directories.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ie4a449fe69022716232638e0f856a10850403831
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53959
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-8130 lipe: Add directory scan structs

This patch adds new structures to lipe_scan3 for collecting
and storing directory statistics, as well as initialization
and destroy functions. This patch is the first in a series
of patches that add functionality for collecting directory
statistics in lipe_find3.

Test-Parameters: trivial
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ib14ce13677d93d1a53299501138e78c7b290793c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53958
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>