git://git.whamcloud.com - fs/lustre-release.git/log

LU-17034 quota: tmp fix against memory corruption

Change QMT_INIT_SLV_CNT from 64 to 2000 to avoid accessing
memory out of array lqeg_arr. It could happen when at least
one of OSTs has index larger than the whole number of OSTs.
It is a temporary solution and maximum supported OST index
is 0x7d0. Later it will be changed with the longterm
solution.

Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: I8d9444017fa9847142f3df77c63368282ff134c4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54696
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

DDN-4905 revert: "quota: lqeg_arr memmory corruption"

This reverts commit 7c6d08994b23cc3ef112e3626f9402dbccf0bc2c
("LU-17034 quota: lqeg_arr memmory corruption")
as it causes following panic:

qmt_map_lge_idx()) ASSERTION( k < lgd->lqeg_num_used ) failed: Cannot map ostidx 32 for 0000000072ee3f23
qmt_map_lge_idx()) LBUG
Call Trace TBD:
libcfs_call_trace+0x6f/0xa0 [libcfs]
lbug_with_loc+0x3f/0x70 [libcfs]
qmt_map_lge_idx+0x7f/0x90 [lquota]
qmt_seed_glbe_all+0x17f/0x770 [lquota]
qmt_revalidate_lqes+0x213/0x360 [lquota]
qmt_dqacq0+0x7d5/0x2320 [lquota]
qmt_intent_policy+0x8d2/0xf10 [lquota]
mdt_intent_opc+0x9a9/0xa80 [mdt]
mdt_intent_policy+0x1fd/0x390 [mdt]
ldlm_lock_enqueue+0x469/0xa90 [ptlrpc]
ldlm_handle_enqueue0+0x61a/0x16c0 [ptlrpc]
tgt_enqueue+0xa4/0x200 [ptlrpc]
tgt_request_handle+0xc9c/0x1950 [ptlrpc]
ptlrpc_server_handle_request+0x323/0xbd0 [ptlrpc]
ptlrpc_main+0xbf1/0x1510 [ptlrpc]
kthread+0x134/0x150
ret_from_fork+0x1f/0x40
Kernel panic - not syncing: LBUG

Fixes: 7c6d08994b ("LU-17034 quota: lqeg_arr memmory corruption")
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Iff377529d2862c869b751b4c942b476262951570
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54695
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn142

New tag 2.14.0-ddn142

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6f94ce2c50e7be19c386d214dde67f1455aebeb7

RM-620 build: New tag lipe-2.47

New tag lipe-2.47

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1d96d0f7049eeafd353e60d6a79ec19cf6554c0d

EX-9523 csdc: fix ofd_preprw_write() for sanity 819b test

Sanity 819b asserts:
tgt_brw_write()) ASSERTION( npages_local == npages_remote )

The test triggers fault inject in ofd_preprw_write():
if (OBD_FAIL_CHECK(OBD_FAIL_OST_2BIG_NIOBUF))
rnb[i].rnb_len += PAGE_SIZE;

ofd_preprw_write() calculates npages_local taking in account
additional len from fault inject, BUT npages_remote is calulated
BEFORE the fault inject. So npages_remote was not adjusted.

To solve the problem it is enough to move range_to_page_count() call.

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Fixes: 217341228f ("EX-7601 tgt: add remote_pages for writes")
Change-Id: Ifd659985a78c7630049a17622aff2eb7f4525fb1
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54681
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17566 mdt: move squash code in new/old_init_ucred

Move the uid/gid squashing code at the same place,
at the bottom of the function, to make code refactoring
simpler later.

The squashing code is mostly clearing suppgids from ucred,
and no code was using between the old and new position in
the function. So that should be pretty safe.

Handle suppgids clearing the same way for both function
and for both UID or GID squashing.

Lustre-change: https://review.whamcloud.com/54194
Lustre-commit: 1730d8093fb36e7957414d314755ae5208da1011

Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Change-Id: I29669af26cf68491bf1b6020548116acf318c0c7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54558
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17056 tests: force osc import reconnect in sanity-sec 30b

In sanity-sec test_30b, force reconnect of idle osc imports
so that security flavor is correctly updated.
In case of failure, dump more information about state of the imports
and the srpc connections.

Lustre-change: https://review.whamcloud.com/54349
Lustre-commit: fa2cfb49decf3d897f63023c998a23fd98c5c3ea

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaecc7321b12e61a266e97d3640a3288f0e7ec9dd
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54657
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17666: configure lnet before add net in sanity-sec:31

If "options lnet config_on_load=1" is not configured in
modprobe.d, the lnet will not be configured when trying to
add a network. The command will hit problem.

/usr/sbin/lnetctl net add --if eth1 --net tcp999
add:
    - net:
          errno: -22
          descr: "cannot add network: Invalid argument"

Test-Parameters: trivial testlist=sanity-sec env=ONLY=31

Lustre-change: https://review.whamcloud.com/54543
Lustre-commit: e163883f76dac45a516b7d89671513d31063b7d6

Change-Id: If65b7cb372d4f04a10ea066d62f3ae43029fcf65
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54654
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14911 osp: release thandle if it was created

osp_statfs_update() could leak thandle if transaction couldn't
start for a reason.

Lustre-change: https://review.whamcloud.com/44504
Lustre-commit: c807e3f33b39409a061fa997cac57ac394c503ba

Change-Id: I541a5e4a7860008eb179d905ac57997b737f178c
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54670
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17634 hsm: serialize HSM restore for a file on a client

For a file in HSM released, exists, archived status, start tens of
processes to read it in parallel on a client, and one read process
may report "No data available" error.

After analyzed the error, we found the following bug in HSM code:
Reading a released file already granted LAYOUT lock on a client:
P1:
->vvp_io_init()
->lov_io_init_released(): io->ci_restore_needed = 1;
->vvp_io_fini()
  ->ll_layout_restore()
    ->mdc_ioc_hsm_request()
      ->mdc_hsm_request_lock_to_cancel()
        ->ldlm_cancel_resource_local()
          remove LAYOUT lock from resource into cancel list
          NOT yet cancel the LAYOUT lock on the client via ELC...

P2:
->vvp_io_init()
->lov_io_init_released(): io->ci_restore_needed = 1;
->vvp_io_fini()
  ->ll_layout_restore()
    ->mdc_ioc_hsm_request()
      ->mdc_hsm_request_lock_to_cancel()
      SKIP: No any conflict LAYOUT lock on resource lock list as P1
      has already move it (if any) into its cancel list
    ->mdt_hsm_request()
      ->cdt_restore_handle_add()
        ->cdt_restore_handle_find()
        ->list_add_tail(): add @crh to restore handle list
        NOT yet obtain EX LAYOUT lock to cancel cached LAYOUT
        locks on client side...

P3:
->ll_file_read_iter()
->ll_do_fast_read(): => return -ENODATA;
->vvp_io_init()
->lov_io_init_released(): io->ci_restore_needed = 1;
->vvp_io_fini()
  ->ll_layout_restore()
    ->mdc_ioc_hsm_request()
      ->mdc_hsm_request_lock_to_cancel()
      SKIP as P1 has already move the conflict LAYOUT lock
      (if any) into its cancel list
    ->mdt_hsm_request()
      ->cdt_restore_handle_add()
        ->cdt_restore_handle_find()
        SKIP as found a restore handle with same FID in the
        the restore handle list added by P2.
  ->ll_layout_refresh()
  ->io->ci_need_restart = vio->vui_layout_gen != gen;
  ->LAYOUT gen does not have any change as the LAYOUT lock on
    the client is not revoken yet, will not restart I/O...
->return -ENODATA; =>from fast read

We can fix this bug by serializing the HSM restore operation on a
client by using the @lli->lli_layout_mutex simply.

Add sanity-hsm/test_12{t, u} to verfiy it.

Lustre-change: https://review.whamcloud.com/54366
Lustre-commit: a6b3faffeaea7abbef389ad5296880a522a13460

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Idc2a8c1818386c64798d7e28500c20c80ff369f1
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54653
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17499 llite: inode lock in ll_migrate()

should be taken after data version check as this is the
correct locking order used in another paths like lseek.

Lustre-change: https://review.whamcloud.com/54041
Lustre-commit: 133fd8b4b11a0228f71d60e3d145d93be16014c9

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0bafb8db215a2ea004928ff36049d8f053507c6f
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54651
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17448 lod: don't skip uninited components

don't skip uninitialized component during declaration as we need
to declare potential records to llogs if the component is created
in this transaction later.

Lustre-change: https://review.whamcloud.com/54302
Lustre-commit: 35b1076aef8fbb2840de2b831765a20ec937d034

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia1cbfaae9b28e40fd68fa125d748ec0b5319f512
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54655
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17546 osd: use __vfs_removexattr

as otherwise vfs_removexattr() taking inode's lock confict with
osd_execute_truncate() while we don't really need inode's lock
because another per-object lock has been already taken.

Lustre-change: https://review.whamcloud.com/54072
Lustre-commit: b9ef5d1e7f7dd1055a6ea6d3dc9f176fa910a372

Fixes: dcd5607ce0 ("LU-13430 vfs: add ll_vfs_getxattr/ll_vfs_setxattr compat macro")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I43c1c60d2a9f911b6395e1b7546507074a90b1cf
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54656
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17497 obdclass: check upcall incorrect values

Identity upcall is set via lctl set_param mdt.*.identity_upcall=xxx,
and rsi upcall is set via lctl set_param sptlrpc.gss.rsi_upcall=xxx.
Possible values are a valid path to an executable, and also INTERNAL
to enable support of supplementary groups from client, or NONE to
disable identity upcall.
Add an upcall cache function that checks the user provided string, to
make sure we do not store an invalid value. And print a message to
stdout to explain the accepted values.

Lustre-change: https://review.whamcloud.com/53878
Lustre-commit: 2153e86541884ef7a5c1697a5d00daf6fa6461a4

Add sanity-sec test_69 to exercise this.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaf59e72aa1612f5579db175d8999dcf0053308ed
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53879
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17446 revert: "ldlm: Do not wait for BL AST RPC completion on cancel"

This reverts commit cfd5411db998c2b0427e310a19b8741b1ec3644e.
There can be LASSERT triggered due to blocking callbacks on one lock.
This will be fixed to handle this in a more generic manner.

Change-Id: I5bdd59e3668de0f1db02f3654c73531712a77c72
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54643
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

EX-9136 lipe: Update the display format for fstats

This patch affects only .out format report types.
The data output format has been updated according to
the request in the ticket.

Also:
1. Fixed incorrect information display in some tables;
2. Expanded additional information for each table;
3. The size in tables is now displayed in various formats,
   not just in KB;
4. Except for the “File Size” table, the size obtained from
   block_size is now used everywhere;
5. Fixed an issue with displaying the allocated size
   for directories;
6. Fixed the total size calculation for all directories;
7. Removed tables that are not yet available;
8. Added additional information about the number of missed
   files for each table;
9. All txt information for working with reports in the .out
   format is combined into a single array to
   simplify code maintenance.

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Id75b3af12ea00761850a9009848621539c016446
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17696 llite: remove LASSERT from ll_ddelete()

On Linux kernel 6.8, the changes in commit 2f42f1eb9093
("Call retain_dentry() with refcount 0") made d_delete()
instances called for dentries with ->d_lock held and
refcount equal to 0, which caused the following assertion
failure on Lustre client:

(dcache.c:136:ll_ddelete()) ASSERTION( d_count(de) == 1 ) failed

The value of d_count(de) became 0 instead of 1. Since
retain_dentry() was called either with refcount 0 or 1,
we can simply remove the LASSERT(ll_d_count(de) == 1)
from ll_ddelete() to avoid the above failure.

Lustre-change: https://review.whamcloud.com/54676
Lustre-commit: TBD (from 50bd3822d2977cd45e56521d137aec2ce5829529)

Change-Id: Ic4a39d9328326634190cd0719b4c0637e1bf315c
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54679
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17504 build: fix array-index-out-of-bounds warning

On Linux kernel 6.5, due to commit 2d47c6956ab3
("ubsan: Tighten UBSAN_BOUNDS on GCC"), flexible
trailing arrays declared like 'lc_array_sum[1];'
will generate warnings when CONFIG_UBSAN & co. is
enabled:

UBSAN: array-index-out-of-bounds in lprocfs_status.c:1609:17
index 1 is out of range for type '__s64 [1]'

Since LPROCFS_STATS_FLAG_IRQ_SAFE flag is only used
in one place - obd_memory() counter, we can just
remove it and change obd_memory over to a regular
percpu_counter. This would both simplify the
lprocfs_counter() code, move over to using more
kernel functionality instead of libcfs, as well as
reduce overhead slightly for the memory accounting code.

Lustre-change: https://review.whamcloud.com/54365
Lustre-commit: TBD (from 21505a19d671868171de2ad0f94120b1ca779695)

Change-Id: Ic461c4b30317bfd2b1e9f5b6be84c4a7fb4e3eb9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54660
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17592 build: compatibility updates for kernel 6.8

Linux commit v4.9-12227-g7b737965b331 introduced
  staging/lustre/libcfs: Convert to hotplug state machine
Linux commit v4.10-rc1-5-g4205e4786d0b
  cpu/hotplug: Provide dynamic range for prepare stage
Linux commit v6.7-rc2-1-g15bece7bec0d
  cpu/hotplug: Remove unused CPU hotplug states

CPUHP_LUSTRE_CFS_DEAD was introduced in 4.9 and removed in 6.8
CPUHP_BP_PREPARE_DYN was introduced in 4.10

With no distro kernels between 4.10 and 4.11 switch to
CPUHP_BP_PREPARE_DYN

Linux commit v6.7-rc1-3-gda549bdd15c2
  dentry: switch the lists of children to hlist
Provide trival wrappers to abstract the changed members

Linux commit v6.7-rc4-79-gaf7628d6ec19
  fs: convert error_remove_page to error_remove_folio
Proved a generic_error_remove_folio() for older kernels.

Lustre-change: https://review.whamcloud.com/54229
Lustre-commit: TBD (from 2036974a891ffac3ecffc7b2a21ca50bc6c94f78)

HPE-bug-id: LUS-12181
Fixes: ce98bfe5f72 ("LU-10499 pcc: add readonly mode for PCC")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ib2e85c2acd3d0934e1c4712dad53b80f0ddb1b08
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54586
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17592 build: kernel 6.8 -Werror=missing-prototypes

Linux commit v6.7-rc4-156-g0fcb70851fbf
Makefile.extrawarn: turn on missing-prototypes globally

With -Wmissing-prototypes and -Werror cleanup some additional
funtions that are implicitly static and provide declarations
for those that are exported.

Add SERVER_ONLY and SERVER_ONLY_EXPORT_SYMBOL to wrap functions
that are only exported for and used by server components.

Lustre-change: https://review.whamcloud.com/54228
Lustre-commit: TBD (from 1da648a24984c94cccdf6686ab9c3aed28d32a47)

HPE-bug-id: LUS-12181
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ice5219df5463effe964d2cd2114f003d185337da
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54584
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17592 build: kernel 6.8 removed strlcpy()

Linux commit v6.7-11707-gd26270061ae6
string: Remove strlcpy()

strlcpy() is removed, use strscpy() and provide a strscpy()
for kernels that do not have one.

Lustre-change: https://review.whamcloud.com/54227
Lustre-commit: TBD (from 1861e4ce8ec6d66d17ed73042f39bacb6496685c)

HPE-bug-id: LUS-12181
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ieab872f20e08d17a4842bc944fa38f9867de81f9
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54576
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn141

New tag 2.14.0-ddn141

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5951e02d762098a10026e744a3894d2dd77b2c0a

RM-620 build: New tag lipe-2.46

New tag lipe-2.46

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I484423fbdfcd8ef6c5162e79fcb040a4039e20e0

LU-8191 lnet: remove unused, fix non-static functions

Static analysis shows that a number of functions
could be made static. This patch also declares
several functions in lnet static.

Lustre-change: https://review.whamcloud.com/51436
Lustre-commit: 43cbc93f1edc493e47fe5c4059bf0bae6a20c207

It is wrong to remove lnet_selftest_structure_assertion()
since it contained BUILD_BUGs used to ensure different LNet
Selftest versions can interoperate.

Add a dummy user for lnet_selftest_structure_assertion() in
LNet Selftest init. This should prevent analyzers from picking
this up as an unused function.

Lustre-change: https://review.whamcloud.com/54635
Lustre-commit: TBD (from ed2a2286d17a7d23b86a87094d1eb2abac8ea015)

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ie1b49c5652553715cd9f96b56090d33a95e3b438
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54604
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17334 lmv: exclude newly added MDT in mkdir

Exclude newly added MDT in QoS mkdir for 30 seconds in case
connections between MDTs are not ready, which may cause lookup fail.

Lustre-change: https://review.whamcloud.com/53860
Lustre-commit: a2b08583a1dc8ab18c4ea4a4b900870761a5c252

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ibb5e6eda29ddfff8f66708d72e33453a96f5e7ef
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54608
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9212 lipe: Fix for 'stripe_count = -1' in stats reports

This patch corrects the display of the range for the table by the
number of stripes. Now the range will only contain one position
and will support a value of -1.

Also corrects the display for a range in other tables by
removing the fractional part.

For example:
[        -1 ] ...
[         1 ] ...
[         2 ] ...
[        10 ] ...

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Iff1933f69b713c7e5dff9145c5516fa050294d2e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54611
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9230 lipe: Add device name in file report name

This patch adds the device name (eg MDT-xxxx) to the
report name when automatically generating the name.
It also corrects the end time in the file name
(when scanning is completed) to the initial time
(when scanning began). Only for lipe_scan3.

Example of a new file name for a report:
files_sizes_report_lustre-MDT0000.2024-03-26-09:54:25.out

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I2e79404e459b5717858b92a0783fe3f1bad552ab
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54574
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9459 lipe: Fix behavior when getting attributes

This improvement is important and is intended for the
--collect-fsize-stats output policy in lipe_scan3.

This patch prevents the scanning process from stopping and
completing if any LOV attribute is not received correctly.
Instead of halting the scan, the patch adds additional error
counters, and all types of reports will now include new
error statistics.

Also add a counters for objects that have no size/allocate size.

An example of a new block with error information from a report
with the .out extension which will contain the following fields:

Error counters:
Allocated blocks is empty: 11101
Size is empty: 0
Without size (all size value empty): 59
Failed to get LOV attr: 0
Failed to get mirror count: 0
Failed to get stripe count: 0
Failed to get stripe size: 0

Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: I1817ea189f3d554894822ad8d12a8514546b13b0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54583
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17611 utils: fix wrong static declarations

Revert wrong changes made to zfs mount utils

Lustre-change: https://review.whamcloud.com/54293
Lustre-commit: f45a0288b00597bc797963f7aa01cae5167b024e

Test-Parameters: trivial
Fixes: c7e9bdf8d4 ("LU-8191 utils: remove unused, fix non-static functions")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I162d349ebadbf93a89abf49bd41465979d561423
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 utils: remove unused, fix non-static functions

Remove several functions which are never called.

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various Lustre utils static.

Some missing headers caused some functions being
incorrectly marked as possible candidates for
being made static. These missing headers have
been added.

Lustre-change: https://review.whamcloud.com/51439
Lustre-commit: c7e9bdf8d4bb5e1127eb87472fbf0414823d5461

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Id51f922be57c33c011ee2f9e509ca164cc480edf
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54629
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 llverfs: fix non-static functions

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in llverfs.c static.

Making functions new_file() and new_dir() static
causes new format truncation errors. Check the
return of snprintf() to silence these.

Lustre-change: https://review.whamcloud.com/53754
Lustre-commit: 986219131215e44a98703a6fb29d941b5f181aa3

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ieccf1e40c1da627571a7a95adbb85599185f1342
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54628
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert lmv,lod,lov functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in lmv, lod, and lov static.

Also, remove one unused function: lov_lsm_entry()

Another function is intentionally unused for
debugging purposes. It was detected by static
analysis, but it has been left untouched.

Lustre-change: https://review.whamcloud.com/51479
Lustre-commit: 78605b74a56a5338aa93bbf394fe35d0f7c17c5b

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If409226ea201587c7f95d4a65ffaef72671b5ac2
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54627
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 liblustre: add missing functions to header

A number of functions were missing from lustreapi.h,
causing them to be marked incorrectly as functions that
could be made static. They have been added to the
header so applications can use them.

Static analysis shows that a number of functions
could be made static. This patch also declares
several functions in liblustre static.

liblustreapi_nodemap.c and liblustreapi_ioctl.c were
missing an internal header, causing some functions
to be incorrectly flagged. This patch also adds that
header.

Initialize a previously uninitialized variable in
llapi_fid_to_handle().

Lustre-change: https://review.whamcloud.com/51434
Lustre-commit: 25523e5a35138a0534b01ff561169e501cc30787

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I67b9c59418b62602ffe36eb4284eb1e8d4a3b19b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54626
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 libcfs: convert functions to static, removed function

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in libcfs static.

Remove cfs_expr_list_values_free in string.c, since
it is not used.

A header was missing in param.c, causing a number of
functions to be missing declarations.

Lustre-change: https://review.whamcloud.com/51428
Lustre-commit: 067dfd8d2701572ebe7246726c14a6e72a78cb33

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia580881efa806bde49d532e5b2d8f5097f2294e0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lnet: convert functions in utils to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various LNet utils and lnetconfig to
static.

In LNet selftest (lst), one unused function was
removed entirely. Some declarations were moved to
made static.

Lustre-change: https://review.whamcloud.com/51427
Lustre-commit: d9cd9992b9e04bfad1ebd755f78d3e96850eaa32

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia4528281b3c87d77e46abb95f47ab0bdc72168c0
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54619
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 ptlrpc: add missing headers

Missing headers for several c-files in ptlrpc
cause functions to be incorrectly marked as only
being used within their respective c-files. This
patch adds those missing headers. It also addresses
a couple minor style issues.

Lustre-change: https://review.whamcloud.com/51480
Lustre-commit: fba78a416621343aadeeb4d28df86e422b653bfe

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Idd8fa747a671079aba2b691ef23cc7564e5e2430
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54618
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert mdc,mdd,mdt,mgc functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in mdc, mdd, mdt, and mgc static.

Also, remove mgs_client_add() since it was unused, and
move a declaration from a c-file to the proper header file.

Lustre-change: https://review.whamcloud.com/51478
Lustre-commit: 38d151f2f65d76fc392d356519227a648e114b5f

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia23f62465c27c83a9a0260bb45e8c8b710491558
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert osp,osd,osc,ofd functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in osp, osd, osc, and ofd static.

Also, fix a few minor style issues.

Lustre-change: https://review.whamcloud.com/51477
Lustre-commit: 2fa075d10f72f1400e4da9bdcaec905858b0a264

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I3d7af7ec0fa2978bfdd0cb490f18f485a78f81f6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 lustre: convert ec,fid,ldlm,quota functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in ec, fid, ldlm, and quota static.

Lustre-change: https://review.whamcloud.com/51476
Lustre-commit: bcc828bdf88f8a19e56535ff3840e06065b606b2

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ic64bdf0d802fd4c963b7b7d3a654575ebde5c07d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54615
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 target: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in target static.

Also, remove an unused function tgt_obd_log_cancel(),
and add some headers where they were missing.

Lustre-change: https://review.whamcloud.com/51475
Lustre-commit: bcbf31a9e1bd6399053a5a757b406152f0d65a42

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I1823df3562cb181b275788560166c92b63483637
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54614
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 llite: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in llite static.

Also, conserve more * in comments.

Lustre-change: https://review.whamcloud.com/51441
Lustre-commit: a586a2bbc37879dc22382793d1704e7708b80887

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Iafa3bb84de158e31b27b7784243bc15e78187f10
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54613
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 obdclass: add static and remove functions

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in obdclass static.

There are a few functions which are never called
anywhere. These are removed. Additionally, there
is some debugging code (added 15 years ago) that
has also been removed.

Lustre-change: https://review.whamcloud.com/51440
Lustre-commit: b3f0995b772877f0bea5d3d790db441244c3c821

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I5f1d438c9663e62789d26093ec9bdd5d76a3060a
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54605
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 tests: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in various test helpers static.

Lustre-change: https://review.whamcloud.com/51433
Lustre-commit: 629d6bca95f96bd307d6a9da9b04d73d3fe7c68f

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I065fb4398ed1670ce6ad58cf946054f6bd1ec282
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54603
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 mdt: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in mdt_coordinator.c static.

mdt_coordinator.c:2145:9: warning: Should this function be static?
ssize_t loop_period_show(struct kobject *kobj,
^

Further patches will follow to clean up the
remaining non-static functions in other subsystems.

Lustre-change: https://review.whamcloud.com/51345
Lustre-commit: a1d332f613ac41f4a565c3aeca6633fbd618467e

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I0350b0d5c88c0a8d1f1748d1d429cdf90afb96b7
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54602
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-8191 ptlrpc: convert functions to static

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in ptlrpc static.

Lustre-change: https://review.whamcloud.com/51353
Lustre-commit: 41d94aa9c9c01fa88ae152625d5ffd97256bab73

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If0d92f7f4e625c146833f360806ae80b8914cc20
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54601
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16520 build: Move strscpy to libcfs common header

Ensure strscpy is available to lustre

Lustre-change: https://review.whamcloud.com/49863
Lustre-commit: 7fe7f4ca06b9c8d128f7ba36988e36f8141ed53d

Test-Parameters: trivial
Fixes: 0b406c91d17 ("LU-13642 lnet: modify lnet_inetdev to work with large NIDS")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I0c3673c2aa7e6b61671521a8cabde8a364f7f6f8
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54570
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15074 build: Use strlcpy if strscpy is not available

Linux commit v4.2-rc1-2-g30035e45753b
    string: provide strscpy()

    The strscpy() API is intended to be used instead of strlcpy(),
    and instead of most uses of strncpy().

Unfortunatley strscpy is not always available.

Test for strscpy and fallback to strlcpy when strscpy is
not available.

Lustre-change: https://review.whamcloud.com/45175
Lustre-commit: 7861d0754d42ed2a02a330eb730cb43f21dd30f2

Test-Parameters: trivial
Fixes: b77a6d86936 ("LU-14665 lnet: simplify lnet_ni_add_interface")
HPE-bug-id: LUS-9546
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I43038e4a6260dafb57195ec3417ce009f5a3fad4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54569
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14665 lnet: simplify lnet_ni_add_interface

Remove an unnecessary counter and move the comment before
the relevant code. Improve error messages.

Lustre-change: https://review.whamcloud.com/43525
Lustre-commit: b77a6d86936c32bb5f36e9806323ba00a18b0f4b

Test-parameters: trivial

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: Iffc7a128b16bc1b2be7a44413a5972c97b12a5fa
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54568
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17609 sec: nodemap readonly_mount for remount

The readonly_mount property on nodemaps forces read-only mount from
clients. Clients trying rw remount (via mount -o remount,rw) should
also be forced to read-only.

Also improve sanity-sec test_61 to exercise client remount.

Lustre-change: https://review.whamcloud.com/54282
Lustre-commit: 27cf3e0ac8576841106b3fcbd58fd5d7d419197d

Fixes: e7ce67de92 ("LU-15451 sec: read-only nodemap flag")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I61f8141001d2ff9e832e5c93d8f5997479af98a6
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54561
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17605 obdclass: do not wait forever acquiring entry

The process of refreshing an entry via refresh_entry() goes through
an upcall/downcall. If the upcall succeeds, we enter a wait queue.
If after that the downcall is never called, we hit the expiry timeout,
and we get removed from the wait queue.
But if the entry is not new, the expiry time will be
MAX_SCHEDULE_TIMEOUT == LONG_MAX, which means an infinite wait.
So avoid waiting forever if an entry could not be refreshed, and call
wake_up_all() if the wait for the ACQUIRING state failed.

Lustre-change: https://review.whamcloud.com/54269
Lustre-commit: d3157bc43390b56e2de2c7251802a67ccebe4952

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I50ee59654adc221027c79cb68fa182b9abed50fa
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54551
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17612 gss: always try to unlink key in error

In case of error in context negotiation carried out in userspace,
always try to unlink key to avoid leaking it.

Lustre-change: https://review.whamcloud.com/54316
Lustre-commit: 21aa8404a42b79f7e0434cfe75411f85d7ee063a

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ic771f1e4f1b6474caaa89f63c3b02678e163d3d3
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54557
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17612 sec: return keyring errors to userspace

In current code, Linux keyring errors, when using GSS Kerberos,
are all masked under a generic ECONNREFUSED error. That makes
it hard to understand the root cause of the problem
for the I/O caller.

Update the code to propagate errors from request_key() up to
the application.

struct ptlrpc_cli_ctx * gss_sec_lookup_ctx_kr(...) is modified
to now returns a NULL pointer or -errval. This is tested by callers
and propagated. NULL values are still converted to ECONNREFUSED.

Lustre-change: https://review.whamcloud.com/54296
Lustre-commit: cd8625792f10d51fceca4717544ff8016609c3be

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Change-Id: I13792f141a961036bc9f7629a4a2db692e245c41
Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54556
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17057 tests: Fix "...endpoint shutdown" under sanity-sec

This patch fixes test_0 failing with "Cannot send after
transport endpoint shutdown" by introducing wait_ssk()
in sec_setup() to deterministicly applied SSK flavor.

Lustre-change: https://review.whamcloud.com/54311
Lustre-commit: 24ae2519acd15691d9e319ffc8675cee60529b95

Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Test-Parameters: trivial mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 testlist=sanity-sec env=SHARED_KEY=true,ONLY=0
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ia14021ab82913507df02dbb5a12c8596663f15d9
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-8981 csdc: execute tests if compression is enabled

"csdc: Provide finer grained enable_compression control" patch
enables CSDC by default on all archs except aarch64 and ppc64le.
No need to execute enable_compression/disable_compression for
each test, so this test changes them to compression_enabled
check that skips test if compression is disabled for the arch.

This patch forces to skip compression tests for ppc64le and
aarch64 so they stop adding noise to the test results.

Test-Parameters: testlist=sanity-compr env=SANITY_ONLY="460",ONLY="0 1001-1080" serverversion=2.15
Test-Parameters: testlist=hot-pools env=ONLY=80 serverversion=2.15
Test-Parameters: testlist=sanity-flr env=ONLY=43 serverversion=2.15
Test-Parameters: testlist=sanity-sec env=ONLY="66 67" serverversion=2.15
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I2b5deba0672f3bad028f383cf1205e66c2b05529
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54105
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9108 obdclass: disable T10PI guard from disk when decompressing

When server handles chunk unaligned read, we need to read
the chunk from disk and decompress the raw data.
This means we should not use the guard tags from the storage,
as they only match the raw data.
Disable the guard from disk and the T10PI checksum will be
recalculated later using decompressed data.

Change-Id: I49964e1412e9d02769797c64aab290b17851e26f
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54567
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>

EX-9462 build: fix the json-c-devel build error on sles12sp5

The build is failed on SLES12 SP5 system:
error: Failed build dependencies:
json-c-devel is needed by lustre-client-2.14.0_ddn107-1.x86_64

SLES12SP5 uses package "libjson-c-devel", similar to SLES15.
This patch update the version in lustre.spec.in to fix this build
failure on SLES12 SP5.

Test-Parameters: trivial
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iffeef43fa04275857ca38b7f16e5376b24c3e9bf
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54573
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-8450 tests: skip sanity-lipe-scan3/306 on el7.9

The sanity-lipe-scan3 test_309 is failing consistently with el7.9
*clients*. Exclude it until fixed or we drop this client version.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-lipe-scan3 clientdistro=el7.9 serverdistro=el7.9
Test-Parameters: testlist=sanity-lipe-scan3 clientdistro=el8.8 serverdistro=el7.9
Fixes: bf29ca2c68 ("EX-8450 tests: skip sanity-lipe-find3/306 on el7.9")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iceca83a3b85df95fe45482076170d77a6a3ebbe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54546
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>

RM-620 build: New tag 2.14.0-ddn140

New tag 2.14.0-ddn140

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I488a4d1ae1f95a90e7e1fbc088481d35ceb95555

RM-620 build: New tag lipe-2.45

New tag lipe-2.45

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I065ea0ca483e354dbaf5d30d6c204f72ce0b9609

EX-9452 tests: skip sanity-pcc pccro=1 tests

Skip subtests when running interop on MDS without PCC-RO support.
Avoid specifying "pccrw=1" for older systems without this parameter.

Fixes: 69594ef7cd ("EX-4006 pcc: make 'pccro=1' default")
Test-Parameters: trivial testlist=sanity-pcc
Test-Parameters: testlist=sanity-pcc serverversion=2.15
Test-Parameters: testlist=sanity-pcc serverbuildno=4512 serverjob=lustre-master serverdistro=el8.9
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3915ce001cf1ca821d62ba18798d8132a3909a57
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54541
Tested-by: jenkins <devops@whamcloud.com>

EX-9196 csdc: do not round lnb size up to the PAGE_SIZE

lnb size is successfully updated to the right one in case the chunk
has been decompressed but left equual to the PAGE_SIZE in case it
hasn't. This leads to the wrong file size change.

This patch set lnb size before the decompression, so if lnb hasn't
decompressed this size is left as is.

Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Test-Parameters: trivial testlist=sanity-compr env=ONLY=1003
Change-Id: I4b8149cc3aab1973a1a261b2e164c321019a437e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54518
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15117 ofd: no lock for dt_bufs_get() in read path

osd_bufs_get() allocates the pages and can cause new transactions
as part of memory release procedure. this would break Lustre's
"start a transaction, then do locking" rule.

Lustre-change: https://review.whamcloud.com/48209
Lustre-commit: 85941b9fb9ef5c27870550469f2e088c4e690603

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I782f0cc6c96251ad88d5fb8d15c9ac91d382bf7e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54535
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15102 lnet: Reset ni_ping_count only on receive

The lnet_ni:ni_ping_count is currently reset on every (healthy) tx.
We should only reset it when receiving a message over the NI. Taking
net_lock 0 on every tx results in a performance loss for certain
workloads.

Lustre-change: https://review.whamcloud.com/45235
Lustre-commit: 9cc0a5ff5fc8f45aa60cd0407ae9893d5c116ccd

Test-Parameters: trivial testlist=sanity-lnet
Fixes: 8fdf2bc62a ("LU-13569 lnet: Recover local NI w/exponential backoff interval")
HPE-bug-id: LUS-10427
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I67ea3aa977cb5d67b04f7957120c29e9985c83e6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54407
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14654 tests: Ensure recovery_limit zero works as expected

When lnet_recovery_limit is set to zero (the default) peer NIs are
eligible for recovery pings indefinitely. Verify this functionality
by modifying sanity-lnet test_211 to use recovery_limit 0 to make
a peer NI re-eligible for recovery.

Lustre-change: https://review.whamcloud.com/43502
Lustre-commit: 8d1895f2f69bd2eec3ff6af5eb356740fa2c8766

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-9953
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I00cb0940133e15ec73491e875d08b6db2bff3fe5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54449
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14654 lnet: Correct peer NI recovery age out calculation

The calculation to age a peer NI out of recovery is only valid if
lnet_recovery_limit is non-zero. When set to zero, we allow peer NIs
to be in recovery indefinitely.

Lustre-change: https://review.whamcloud.com/43501
Lustre-commit: 8f3f0e1219724d6e0ed727e46b28ab28203aef9f

Test-Parameters: trivial
HPE-bug-id: LUS-9953
Fixes: cc27201a76 ("LU-13569 lnet: Age peer NI out of recovery")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I6bb40ca3a9affa0eaaae9deb1cecdb03e4bb42c5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54408
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15398 lnet: Avoid peer NI recovery for local interface

If a MR peer has a MR peer entry for itself (can happen if manually
created or discovery is run on itself for some reason), then it is
possible for it to put its own interfaces into peer recovery. Problems
with local interfaces should be handled via local NI recovery.

Lustre-change: https://review.whamcloud.com/45933
Lustre-commit: fb5d7036ec356c825f2aadece68cd7c4af487680

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-10661
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I5b28195979a6113fa863b5795a4528b072610891
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51683
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 tests: Check LNet Health recovery logic

Add test cases to validate LNet Health recovery of local and peer
NIs.

The new test cases are added to the except list for aarch64 due to
an unresolved issue with the LNet drop functionality on that
architecture.

A few style issues are also addressed by this patch.

An asterisk was being supplied to the lctl net_drop_del commands when
this should have been the '-a' flag.

A bug in cleanup_testsuite is addressed where we were using the
wrong filename for the tmp files created by the subtests.

Lustre-change: https://review.whamcloud.com/39723
Lustre-commit: aa7391445519b46752b5b0adcbe5baa368750e70

Test-Parameters: trivial testlist=sanity-lnet
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I965df2449770631caa03ced7726abb0ea76c17e6
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54406
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15791 tests: Get health before removing drop rules

lnet_health_post() can race with recovery pings, so we should
wait to delete the drop rules until after we've gathered the
health and resend values.

Lustre-change: https://review.whamcloud.com/47998
Lustre-commit: 8caec97d5e89eefe250edb64e6f7ad61e12a9d71

Test-Parameters: trivial testlist=sanity-lnet
Fixes: 79ab053562 ("LU-13569 lnet: Deprecate lnet_recovery_interval")
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ia7595e015809f796cafcc40382d98ab66a708a49
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54439
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Add health ping stats

Add the NI and peer NI ping count and next ping timestamp to
detailed output of lnetctl peer and net output.

Lustre-change: https://review.whamcloud.com/40314
Lustre-commit: 4c7e4aa57629660386ae2849151a0639b6177200

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I208cb3ea0b08a2984572cf0ec9874dbd09f6168e
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54405
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Deprecate lnet_recovery_interval

We no longer use a static recovery interval, so remove its remaining
uses and add warning that it has been deprecated.

Lustre-change: https://review.whamcloud.com/39722
Lustre-commit: 79ab0535622782c82636cee47918dc4b19983144

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Iedc79803ef5b7ba041a531961eb77acd338abfb5
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54404
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Recover local NI w/exponential backoff interval

Use an exponential backoff algorithm to determine the interval at
which unhealthy local NIs are ping'd

Introduce lnet_ni_add_to_recoveryq_locked() which handles checking
pre-conditions for whether the NI should be added to the recovery
queue, and takes a ref on the NI as appropriate.

Lustre-change: https://review.whamcloud.com/39721
Lustre-commit: 8fdf2bc62ac9c418bd8e326112da9a2835f09ccb

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Idb3789366b2e450837989f9a12eb2d598f80081c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54403
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Recover peer NI w/exponential backoff interval

Perform LNet recovery pings of peer NIs with an exponential backoff
interval.
- The interval is equal to 2^(number failed pings) up to a maximum
   of 900 seconds (15 minutes).
- When a message is received the count of failed pings for the
   associated peer NI is reset to 0 so that recovery can happen more
   quickly.

Lustre-change: https://review.whamcloud.com/39720
Lustre-commit: 917553c537a8860f57a50dc9752e3ac69d06c11c

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ic7e60455015a0236a96010c07fc0ddd02078cf92
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54402
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Only recover known good peer NIs

A peer NI should not be eligible for recovery if we've never
received a message from it.

Lustre-change: https://review.whamcloud.com/39719
Lustre-commit: 39a169cd02738a13866f3b88fbe3304dc20565d6

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Iec2fd015f6410ab91c6ef7c222cbed0204243106
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54401
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13569 lnet: Age peer NI out of recovery

No longer send recovery pings to a peer NI that has been in recovery
for the recovery time limit. A peer NI will become eligible for
recovery again once we receive a message from it.

The existing lpni_last_alive field is utilized for this new purpose.

A check for NULL lpni is removed from
lnet_handle_remote_failure_locked() because all callers of that
function already ensure the lpni is non-NULL.

lnet_peer_ni_add_to_recoveryq_locked() now takes the recovery queue
as an argument rather than using the_lnet.ln_mt_peerNIRecovq. This
allows the function to be used by lnet_recover_peer_nis().
lnet_peer_ni_add_to_recoveryq_locked() is also modified to take a ref
on the peer NI if it is added to the recovery queue. Previously, it
was the responsibility of callers to take this ref.

Lustre-change: https://review.whamcloud.com/39718
Lustre-commit: cc27201a76574b51dc3ffb37f039b3364cab386d

Test-Parameters: trivial
HPE-bug-id: LUS-9109
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: Ib4676540ac4bb040690a4fb047236c54eea0e752
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54400
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10465 test: fix interop test for 4M default stripe size

New servers could use 4MiB default stripe size, so some of
the tests need to use bigger component extent or specify stripe size
explicitly to accommodate enough stripe count.

Patch includes several test fixes:
- sanity-pfl: takes into account stripe size in some tests
- sanity-flr: use bigger component size and amount of data to
saturate all stripes as expected by test
- sanity: 130g to use 1M stripe prior FIEMAP calcs
- sanity-lfsck: 36[a-c] to use 1M stripe as expected by calcs

This change for test scripts comes from:

Lustre-change: https://review.whamcloud.com/37318
Lustre-commit: ea18d7da59d369f093e340e150544f51b2f229a1

Test-Parameters: testlist=sanity-flr serverversion=2.15 env=ONLY="0 208"
Test-Parameters: testlist=sanity-pfl serverversion=2.15 env=ONLY="0 1 14 19 20 24"
Test-Parameters: testlist=sanity serverversion=2.15 env=ONLY="27 130"
Fixes: ee7dfc5ad1 ("LU-17025 llapi: Verify stripe pool name")
Fixes: 0396310692 ("LU-15727 lod: honor append_pool with default composite layouts")
Fixes: b384ea39e5 ("LU-14480 pool: wrong usage with ost list")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3cef8805247fc5253e0a0ac05157b9d609054df9
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54444
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9372 lpurge: rid of dup layout get

Delete the unnecessary lpurge_mirror_delete() repeat layout getting
code, and update fd/fdv open flags align with LU-14677 to allow
layout operations on encrypted files, even when the encryption key
is not available.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: If4e6e3a9c46ab782f08e5bc5b3710339af051719
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54527
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Alexandre Ioffe <aioffe@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandre Ioffe <aioffe@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14535 quota: improve quota output format

Make the output of the quota proc files more readable by removing
needless whitespace that causes it to wrap over a single line.

Lustre-change: https://review.whamcloud.com/43099
Lustre-commit: cd1847e73e5990ef797280846dc27fc0b0a876e9

[NOTE: missing changes to procfs output that change field names]

Test-Parameters: trivial testlist=sanity-quota
Test-Parameters: testlist=sanity-quota serverversion=EXA5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I80fbd42dea865dff1d106724dbf69946d23ebbe5
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/46332
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9387 lipe: Update tests in lipe_scan3

This short patch fixes the lipe_scan3 and lipe_find3
automated tests. Adds new attributes to tests to
run lipe_scan3

Fixes: 9093217a72 ("EX-8130 lipe: Link a new statistics module")
Test-Parameters: trivial testlist=sanity-lipe-scan3,sanity-lipe-find3
Signed-off-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Change-Id: Ibe00b2eb75da00ce4a8dbfd7a4f4d74b365104de
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54440
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-9125 tests: get stats_compr param in sanity-compr/1007

This patch adds getting stats_compr param in sanity-compr
test 1007 and 1008 to track data compression statistics.

Test-Parameters: trivial testlist=sanity-compr env=ONLY="1007 1008",ONLY_REPEAT=3

Change-Id: I7df25d36c8c89ec5a568bf1e8d5694a97dd18ecc
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/53927
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>

LU-17434 lmv: add exclude list for remote dir

Apache Spark creating a _temporary subdirectory for staging files, and
it should be created on the same MDT as its parent directory. Add a
tunable lmv.*.qos_exclude_prefixes, if directory prefix is in this
list, lmv_create() should put it on its parent MDT.

This prefix list follows the same rule of shell environment PATH: use
':' as separator for prefixes. And for convenience '+/-' can be used
to add/remove prefixes.

Add sanity 413k.

Lustre-change: https://review.whamcloud.com/53780
Lustre-commit: 5b07dce19b1830769d7a1f7bba8b559d3ead9dfb

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I4c8a118f0630c19054934a87bee3599bdb1fe7bb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54462
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17632 o2iblnd: graceful handling of CM_EVENT_CONNECT_ERROR

There were examples in the field with RoCE setups which demonstrate
that RDMA_CM_EVENT_CONNECT_ERROR may be received when conn state
is neither IBLND_CONN_ACTIVE_CONNECT nor IBLND_CONN_PASSIVE_WAIT.
Handle this in a more gracious manner: report the event as unexpected
and allow the flow to continue.

Lustre-change: https://review.whamcloud.com/54353
Lustre-commit: 7f27a2fceef9a03d3ada74e258e774c8f5d420f0

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I58b2482207cfd821f6eac142bdefc8f5bc50f8b4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54362
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn139

New tag 2.14.0-ddn139

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I464d0618296d59093c0f85075daf3cdb653665e8

RM-620 build: New tag lipe-2.44

New tag lipe-2.44

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ife8844a38d83a63175c5cf0182e1d23405143459

EX-6247 lfsck: support compression in filter_fid

The OST object compression info has been stored in filter_fid
by EX-8038, the LFSCK also needs to handle the compression info
during processing the filter_fid.

Fixes: bf4f5295e594 ("EX-8038 csdc: expand filter_fid")
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: Id050214ba0057776a05c194dc9222117c5d7fd8b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54158
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14869 test: improve sanity-flr/200a

Make sure "flock -x" successfully returned before running mirror
resync so that it won't get into running read holding shared flock.

Lustre-change: https://review.whamcloud.com/54345
Lustre-commit: TBD (from 2ff02fec932ab6e00a4942517db76f48412db31a)

Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Test-Parameters: trivial testlist=sanity-flr env=ONLY=200a,ONLY_REPEAT=10
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I6383af5d5761980d24af19efd4a4ac899f369a7d
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14971 test: align mirror_io resync implementation

Align the mirror_io resync implementation with
llapi_mirror_resync_many().

Lustre-change: https://review.whamcloud.com/45202
Lustre-commit: 9898ae5347616a097193f69811a74af3ddd88349

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Icf11c4c2302f36fc0f9682e0a310058081e1214f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54327
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9371 lipe: print FID after llapi_lease_set() error

Investigate 'cannot get UNLOCK lease, ext 8: Invalid argument (22)'
Add error message and print FID when llapi_lease_set() fails.

Test-Parameters: trivial testlist=hot-pools
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: Id498f2017ccdf8d8896ce731cc65ca45ed692d62
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54339
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17566 mdt: remove duplicate call to mdt_init_ucred_reint()

Remove duplicate call to mdt_init_ucred_reint() from
mdt_reint_setxattr().

mdt_init_ucred_reint() is called in mdt_reint_internal() which is
covering all actual reinters. However, SETXATTR was converted to
reinters framework in fd908da and this call was not removed.
So mdt_init_ucred_reint() is called first in mdt_reint_internal() then
again in the specific mdt_reint_setxattr() handler, without anything
special being done on the ucred between them.

Also merge __mdt_init_ucred() and mdt_init_cred() which was
called only once, and with the same prototype.

Lustre-change: https://review.whamcloud.com/54111
Lustre-commit: 65e0802f2ada98f802d01b5672bb9349ad0dde8c

Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Change-Id: I90fed1d2709edf7337a27dd9c3cb0f75f7625135
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54368
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17175 gss: start lsvcgssd from l_getauth

If l_getauth detects it cannot connect to the socket supposed
to be opened by lsvcgssd, it tries to launch the daemon, with
predefined default values.

Lustre-change: https://review.whamcloud.com/53142
Lustre-commit: 414467762f8a034c72903bab8ebfce6e1feb8e79

Test-Parameters: trivial
Test-Parameters: kerberos=true testlist=sanity-krb5
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I3961ce0f548fb6ea23458edcb01a03fb8b3a617f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54369
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9400 sec: refresh entry directly

We can process entry directly for INTERNAL entry. It avoid
to release & take lock frequently.

Fixes: fb0082bba1 ("EX-4333 sec: support supplementary groups from client")
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I2af9c3964978c842dac8f70ad814adb529dff39f
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54273
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-9280 lipe: lpurge: Add periodical stats on scanned files

- Report INFO message on each lpurge period.
- Include in the message number of purged files, number of files
for which mirror deleting failed, number of files
which are not purged due to a stale component.
- Reset the periodic counter every period.

Test-Parameters: trivial testlist=hot-pools
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: Ib70604c990801b79b0a0356991d45e83ec62db6c
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54337
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17578 lnet: fix &the_lnet.ln_mt_peerNIRecovq race

To avoid race &the_lnet.ln_mt_peerNIRecovq must always be
accessed with lnet_net_lock(0) protection.

Lustre-change: https://review.whamcloud.com/54163
Lustre-commit: 0a0e881d8884a220c485c0384351da12dc8aed9f

Test-Parameters: trivial
Fixes: da23037 ("LU-16563 lnet: use discovered ni status to set initial health")
Change-Id: Ic5e0194020200afdecba4cbf5afed274b14da388
Signed-off-by: Bruno Faccini <bfaccini@nvidia.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54382
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17522 build: Distribute clang build infrastructure

Macro files:
lustre-toolchain.m4 lustre-compiler-plugins.m4
and directory:
cc-plugins

Should be included in distributed files, unconditionally.

Lustre-change: https://review.whamcloud.com/53991
Lustre-commit: 881cc6384a5e37f0e64b56c1b34563b87dcc210d

Test-Parameters: trivial
Fixes: d684885098 ("LU-16961 clang: plugins and build system integration")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I6ddedd82c6180ffd1c4134fda6af6df6bd23dd34
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54351
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16961 clang: plugins and build system integration

Clang has a plugin system. Compiler extensions can be created
by making a shared library and loading it via the "-fplugin"
options. This makes it simple to implement custom warnings
and static analyzers.

This patch adds a plugin to detect functions that should have
been made static. This plugin has been run over the majority
of the Lustre tree and patches have been submitted for all
warnings. The plugin did not return any false positives in
my testing.

It also add the "--enable-compiler-plugins" configure option,
which automatically builds and sets up the in-tree C compiler
plugins. The option force-enables the plugin regardless of
which compiler is in use. This behavior could be changed if
there is ever a need to support GCC specific plugins.

Also, add the configure checks needed to support building C++
in the Lustre tree. Clang and GCC plugins (and the compilers
themselves) are written in C++.

The license for the plugin mirrors that of the LLVM project
itself. This leaves the door open for contributing this
plugin upstream in the future. This isn't being upstreamed
now because it lacks any significant user community. Hence,
the plugin does not appear to meet the requirements for
upstreaming based on https://clang.llvm.org/get_involved.html.

Lustre-change: https://review.whamcloud.com/51659
Lustre-commit: d684885098c40fee2951feb410bec739717ac9bc

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I747ed91b53e765cc58e91a3eb9ec6c12b9908a96
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54350
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17243 pcc: replace i_mtime with inode_get_mtime()

This patch replaces i_mtime with inode_get_mtime(), and
i_mtime.tv_sec with inode_get_mtime_sec() in pcc codes.

Test-Parameters: trivial testlist=sanity-pcc

Fixes: 3c586ca ("LU-17243 build: compatibility updates for kernel 6.6")
Change-Id: I756451ac9d38c3b434bb511b33d5d891b2b914ae
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54357
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17627 build: fix new mofed version

Allow multi-digit MOFED version numbers.
Fix compare_version function to return what it should

Lustre-change: https://review.whamcloud.com/54336
Lustre-commit: TBD (from ec967a35b2ac09e780772bdfd4365f6fc7308417)

Change-Id: I0f585cb355bb34270003ae1139688080c301186a
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54289
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Deiter
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17179 tests: check the system is clean

Main part of tests cannot work correctly if the system
is not clean. So check this in the beginning of sanity-quota.

Lustre-change: https://review.whamcloud.com/52630
Lustre-commit: 7e1fb1a296ec7ab21be7ec39e2b6a38fbca76b6c

Test-Parameters: trivial testlist=sanity-quota,sanity-quota,sanity-quota
Test-Parameters: testlist=sanity-quota,sanity-quota mdscount=2 mdtcount=4
Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com>
Change-Id: Ibfbe4663dee8476486e96eb99ccbcea13216861b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54392
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17602 mdd: use correct fid in mdd_rename

mdd_rename() can re-insert target name back as a part of error
handling. use correct fid for that, not own target directory fid.

Lustre-change: https://review.whamcloud.com/54260
Lustre-commit: TBD (from 61a389ea659ce62790219b0d6edf72730284b007)

Fixes: 1c03346731 ("LU-17016 mdd: no EXDEV for parent dir projid mismatch")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0662fa005459416b070157a2d049fcf5ed08ae91
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/54344
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>