LU-17191 osc: only call xa_insert for new entries

The handling of updating the Xarray entry is incorrect. If the
Xarray already exist and we call xa_insert() it will return -EBUSY
and not continue examining the entry. The correct approach is
if xa_load() returns a value never call xa_insert(). With the
return value of xa_load() we test if type bit was set and if not
set it.

We add ll_xa_insert to spelling.txt since the return value changed
with kernel versions. This will avoid mistakes in the future for
cases we look at the return value of xa_insert().

Test-Parameters: testlist=sanity-quota
Test-parameters: clientdistro=ubuntu2204 testlist=sanity-quota
Fixes: ac8c28f959 ("LU-8130 osc: convert osc_quota hash to xarray")
Change-Id: Icd88f5e47fb24f669bf362c1daec500b743b447b
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52713
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17115 quota: fix race of acquiring locks in qmt

In qmt_delete_qid and qmt_reset_qid, the order to require
the lock of lquota_entry and journal is different from that
in qmt_dqacq0, which could cause deadlock in some cases.

Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: Ic439f2c5d6ca22429422b87f0dde65e0d2e6113d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52371
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17071 o2iblnd: IBLND_REJECT_EARLY condition causes Oops

The message printed when kiblnd_passive_connect recognizes
IBLND_REJECT_EARLY condition introduced by LU-16393 is trying
to derefence a NULL pointer in the parameter list. Fix this.

Test-parameters: trivial
Fixes: 673ff86a84a ("LU-16393 o2iblnd: add IBLND_REJECT_EARLY reject reason")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I711e5855383c140b9f7c35b27f48995f3f0e25ee
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52202
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-17232 build: fix ext4-misc for el7.6 server

rhel7.6/ext4-misc.patch was partially updated
however the intended addition of:

+EXPORT_SYMBOL(ext4_chunk_trans_blocks);

is not present.

Test-Parameters: trivial
HPE-bug-id: LUS-11954
Fixes: 9e5040a304 ("LU-16847 ldiskfs: do not copy ldiskfs_chunk_trans_blocks")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I28c3f1cc52af61b8b1b5036cf8c7cbce75b5c895
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52850
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-9859 libcfs: migrate libcfs_mem.c to lnet/lib-mem.c

Move the libcfs_mem.c code to the LNet core. The prototypes are declared in libcfs_cpu.h
but we don't move them yet since the CPT code depends on the libcfs_mem.c work. This can
end up in a modular cyclic dependency if we move the CPT work right away so limit what is
changed at this point.

Test-Parameters: trivial
Change-Id: I6bf5cd9f20033f988dde1989f0fc5f89ea74b5a2
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52701
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17131 ldiskfs: el9.2 encdata and filename-encode

Add encryption support for el9.2

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I03a42777eb7e23ab0934452461d3581d0b670af1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52412
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>

LU-16097 quota: release preacquired quota when over limits

The pre-acquired quota on each MDT or OST should be released when
the whole quota is over limits, for instance, after the quota limits
had been decreased for some quota ID by Administrator.

Test-Parameters: testlist=sanity-quota ossversion=2.15.3
Test-Parameters: testlist=sanity-quota mdsversion=2.15.3
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I6263b835d4ae6a3fd03f9a2bc4f463949cbc74d4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48576
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>

LU-14361 statahead: add tunable for fname pattern statahead

This patch adds a tunable parameter for fname pattern statahead.
Currenty fname pattern statahead is disabled.
It will be enabled by default once the patch series of fname
pattern statahead work is finished and stable.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I7a207e66df10b45bff1b3993e8724116489365b7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51572
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>

LU-4974 lod: Change pool_desc to "[lod|lov]_pool_desc"

This patch changes 'struct pool_desc' under lov and lod
to 'lov_pool_struct' and 'lod_pool_struct' respectively.
This is the first step to check if there is anything
common and can be unify. Although both layer uses
'struct pool_desc' to define the pool_desc struct
respectively. 'struct pool_desc' under lod has changed
and grown. Therefore to remove ambiguity, prefix lod/lov
is added to pool_desc struct for respective layer.

This patch also adds function description wherever
applicable

This patch also changes space/tabs reported by
checkpatch

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I3fee3f2e9c321145779d9177a8e4582d123f1e8d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/11114
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10391 ksocklnd: use ksocknal_protocol v4 for IPv6

During testing of IPv6 I encountered the following error:

LNetError: 11c-c: Protocol error connecting to ...

This was due to the ksocklnd protocol not being set to the correct
version when using IPv6 addresses. Test if we have a IPv6 peer and
set the proper protocol.

Test-Parameters: trivial testlist=sanity-lnet
Change-Id: I19183e904acee4cb8b9d3b7e77284c81f6cdc2b4
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52911
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17235 o2iblnd: adding alias ib interface causes crash

Commit 09c6e2b872 (LU-16836) causes o2iblnd startup routine to crash
when alias ib interface is used:

        ifconfig ib0:0 10.1.0.52 up
        modprobe lnet
        lnetctl lnet configure
        lnetctl net add --net o2ib --if ib0:0

Fix the code which attempts to set the NI status on startup to deal
with the case when corresponding net_device is not found gracefully.

Test-Parameters: trivial testlist=sanity-lnet
Fixes: 09c6e2b872 ("LU-16836 lnet: ensure dev notification")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Iaef9280a10f27ac28b872d9f4bc119c4d459ef40
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52894
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17225 utils: check_iam print records

Fix adds ability to print records with -r option.
Format is FID ino/igen.

[0x200000001:0x3:0x0] 88/2116153859
[0x200000001:0x4:0x0] 89/182164966

If a root block has wrong sizes, stop processing.
Also fixing a test.

Test-Parameters: trivial testlist=conf-sanity env=ONLY=134
HPE-bug-id: LUS-11905
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: I2e11d1a8d675e6be9ea79023348a18370a0bb5a0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52815
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17197 obdclass: preserve fairness when waiting for rpc slot

When obd_get_mod_rpc_slot() waits for an available slot it places the
waiting thread at the HEAD of the queue, so it will be woken before
anything else that is already queued.  This is clearly unfair and can
hurt performance.

So change to always add to the tail to ensure a FIFO ordering (except
that CLOSE might sometimes be woken a bit early).

This regression was introduced in a rewrite that was supposed to make
waiting more fair - by avoiding a broadcast wakeup for "close"
requests.

Also fix some stale comments and expose __add_wait_queue_entry_tail

Running mdtest with the patch applied shows about a 3% improvement:

                             master            patched
  mdtest-easy-write      350.585906 kIOPS   353.783545 kIOPS
   mdtest-easy-stat     1320.329353 kIOPS  1408.320419 kIOPS
 mdtest-easy-delete      285.084103 kIOPS   289.625900 kIOPS
            [SCORE]      509.115803 kiops   524.516113 kiops

Fixes: 5243630b09d2 ("LU-15947 obdclass: improve precision of wakeups for mod_rpcs")
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: If767c4299bcbab71589b0f3c01e85bf461686ca5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52738
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-10391 lnet: handle discovery with Netlink

Move the LNet discover feature to the Netlink API. This change
enables the detection of remote LNet setups using large NID
addresses. We treat LNet discover as a ping doit function since
the output is nearly identical to pings. Returned are successes
as well as failed attempts to discover the requested NIDs.

Test-Parameters: trivial testlist=sanity-lnet
Change-Id: Id0eb4adcb4561cfae96040086aae85d6ff804259
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50253
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16356 hsm: add running ref to the coordinator

This patch replaces the fe5706e by adding a reference "cdt_ref" when
the coordinator is running (it does not trust HSM state).
This avoids to de-init the coordinator while still in use (e.g:
thread to add an hsm request) and avoids complex locking on HSM state.

It also causes the coordinator thread to exit if
cdt_start_pending_restore() fails. Otherwise, this can produce a lot
of unexpected behavior (hang, crash).

The patch modifies mdc_kuc_reregister() to register the hsm agent in
background. This make independent reconnect and the agent
registration. It enables to re-activate resend for HSM_CT_REGISTER
without the LU-13455. The coordinator returns EINPROGRESS if not
ready and the client will resend the request for that case. So the
copytools can wait the coordinator to be ready.

Add regression test sanity-hsm 409a.

Fixes: fe5706e ("LU-16235 hsm: check CDT state before adding actions llog")
Fixes: 3d58403 ("LU-13455 ptlrpc: connect to MDT stucks")
Test-Parameters: testlist=sanity-hsm
Test-Parameters: testlist=sanity-hsm env=ONLY=107,ONLY_REPEAT=20
Test-Parameters: testlist=sanity-hsm env=ONLY=409a,ONLY_REPEAT=20
Test-Parameters: testlist=conf-sanity env=ONLY=132,ONLY_REPEAT=20
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Signed-off-by: Nikitas Angelinas <nikitas.angelinas@hpe.com>
Change-Id: I14302d1053abbe76eeaaa1a63c6fd6d9b530baa9
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51256
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-16032 osd: move unlink of large objects to separate thread

Final unlink and freeing of blocks for large objects can lead to
a thread hung with this call stack:

  Net: Service thread pid 1739 was inactive for 200.16s.
  The thread might be hung, or it might only be slow and will
  resume later.
  Dumping the stack trace for debugging purposes:
    __wait_on_buffer+0x2a/0x30
    ldiskfs_wait_block_bitmap+0xe0/0xf0 [ldiskfs]
    ldiskfs_read_block_bitmap+0x31/0x60 [ldiskfs]
    ldiskfs_free_blocks+0x329/0xbb0 [ldiskfs]
    ldiskfs_ext_remove_space+0x8a9/0x1150 [ldiskfs]
    ldiskfs_ext_truncate+0xb0/0xe0 [ldiskfs]
    ldiskfs_truncate+0x3b7/0x3f0 [ldiskfs]
    ldiskfs_evict_inode+0x58a/0x630 [ldiskfs]
    evict+0xb4/0x180
    iput+0xfc/0x190
    osd_object_delete+0x1f8/0x370 [osd_ldiskfs]
    lu_object_free.isra.30+0x68/0x170 [obdclass]
    lu_object_put+0xc5/0x3e0 [obdclass]
    ofd_destroy_by_fid+0x20e/0x500 [ofd]
    ofd_destroy_hdl+0x267/0x9f0 [ofd]
    tgt_request_handle+0xaee/0x15f0 [ptlrpc]
    ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc]
    ptlrpc_main+0xb34/0x1470 [ptlrpc]
    kthread+0xd1/0xe0

Let's move final unlink to workqueue if inode size > 1GB.  The size
threshold be configured by setting the minimum async truncate size
with the "osd-ldiskfs.*.delay_unlink_mb" parameter.

Writes to "osd-ldiskfs.*.force_sync" parameter will flush pending
delayed unlinks so that space can be reclaimed as needed.

Change-Id: Id535ae4c58732769effabee42835bc2da8cb5cc1
Signed-off-by: Artem Blagodarenko <ablagodarenko@whamcloud.com>
DDN-bug-id: DDN-3144
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47995
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17181 misc: don't block reclaim threads

memory reclaim threads may blocked by lustre reclaim
process, but lustre don't have any benifit from parallel
reclaim.

Test-Parameters: trivial
HPe-bug-id: LUS-11872
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I624edbb8833975864706ec51537d2954f5a9cea4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52627
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17103 lnet: use workqueue for lnd ping buffer updates

Introduce workqueue for handling lnd-initiated ping buffer
update requests.

This is done to avoid the possibility of monitor thread
lock up waiting for the "old" ping buffer refcount to get
decremented during the update, while the message which
triggers the decrement is on the monitor thread's own queue
waiting to be processed.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-lnet env=ONLY="207 500",ONLY_REPEAT=50
Fixes: 7ac399c5 ("LU-16949 lnet: get monitor thread to update ping buffer")
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I5176581703e52f4adbfff417040bebcc2489b79e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52522
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>

LU-17198 tests: running_in_vm to recognize qemu

qemu is reported in dmidecode's system-manufacturer field,
not in system-product-name

Test-Parameters: trivial
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Id7e01ae2825835080d29ebec1750e53b87f5cf04
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52704
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17000 coverity: Fix Resource Leak(2)

This patch fixes error reported by coverity run.

CoverityID: 397140 ("Resource leak"): lfs.c
CoverityID: 397370 ("Resource leak"): dir.c
CoverityID: 397378 ("Resource leak"): obd.c
CoverityID: 397406 ("Resource leak"): obd.c

Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I92bbf6749d667631b92868bcaf78af558c250441
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52686
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17000 coverity: Fix leak under mgc_request.c

This patch fixes resource leak error reported
by coverity run.

CoverityID: 403113 ("Resource leak"): mgc_request.c

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ie569a18cbacdb48c186d38ccc466ce86eeb1b28f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52656
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17149 tbf: nrs_tbf_id_cli_set should not modify the fmt

nrs_tbf_id_cli_set() needs to parse LDLM_ENQUEUE request to find
uid/gid.

It calls req_capsule_extend() that change the request format (rc_fmt).
If rc_fmt was not null, the function will not restore the initial
request format.

The following crash will occur the 2sd time that nrs_tbf_id_cli_set()
is called (1st: o_cli_find(), 2sd: o_cli_init()):

LustreError: 8949:0:(req_capsule_extend())
ASSERTION( fmt->rf_fields[i].nr >= old->rf_fields[i].nr ) failed:
Call Trace TBD:
[<0>] libcfs_call_trace+0x6f/0xa0 [libcfs]
[<0>] lbug_with_loc+0x3f/0x70 [libcfs]
[<0>] req_capsule_extend+0x174/0x1b0 [ptlrpc]
[<0>] nrs_tbf_id_cli_set+0x1ee/0x2a0 [ptlrpc]
[<0>] nrs_tbf_generic_cli_init+0x50/0x180 [ptlrpc]
[<0>] nrs_tbf_res_get+0x1fe/0x430 [ptlrpc]
[<0>] nrs_resource_get+0x6c/0xe0 [ptlrpc]
[<0>] nrs_resource_get_safe+0x87/0xe0 [ptlrpc]
[<0>] ptlrpc_nrs_req_initialize+0x58/0xb0 [ptlrpc]
[<0>] ptlrpc_server_request_add+0x248/0xa20 [ptlrpc]
[<0>] ptlrpc_server_handle_req_in+0x36a/0x8c0 [ptlrpc]
[<0>] ptlrpc_main+0xb97/0x1530 [ptlrpc]
[<0>] kthread+0x134/0x150
[<0>] ret_from_fork+0x1f/0x40

Test-Parameters: testlist=sanityn env=ONLY=77
Test-Parameters: testlist=sanityn env=ONLY=77
Test-Parameters: testlist=sanityn env=ONLY=77
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Ia762936262e8cde891ae2a9daf4ce691c6a6f616
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52528
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17171 test: improve sanity-quota test_41

On zfs backend, df result of project quota may print be a slightly
larger block used than quota result because the former is calcuated
in filesystem block size which is 4K.

Update sanity-quota test_41 to make it more robust.

Test-Parameters: trivial testlist=sanity-quota fstype=zfs
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ide51d9aaeb8907eb77acc30fa4fc76dcc16e8de0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52591
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-14361 statahead: Add test for statahead advise

This patch adds a test program "aheadmany" and sanity/test_123g to
verfiy that statahead advise works as expected.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I751313ecb790c66f70a09bf8e9d13846b3c0032d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51730
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12610 obd: delete OBD_ -> CFS_ redefinitions

With all consumers of the OBD macros fixed, finally
remove OBD macros that are simply redefinitions of CFS
macros.

Remove:
OBD_FAIL_PRECHECK(id)
OBD_FAIL_CHECK(id)
OBD_FAIL_CHECK_VALUE(id, value)
OBD_FAIL_CHECK_ORSET(id, value)
OBD_FAIL_CHECK_RESET(id, value)
OBD_FAIL_RETURN(id, ret)
OBD_FAIL_TIMEOUT(id, secs)
OBD_FAIL_TIMEOUT_MS(id, ms)
OBD_FAIL_TIMEOUT_ORSET(id, value, secs)
OBD_RACE(id)
OBD_FAIL_ONCE
OBD_FAILED

Avoid losing the unlikely optimization with OBD_FAIL_PRECHECK by
adding unlikely to CFS_FAIL_PRECHECK. For libcfs_private.h not
all callers of CFS_FAIL_PRECHECK had unlikely so this is fixed
as well.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Signed-off-by: Ben Evans <beevans@whamcloud.com>
Change-Id: I6620bae389a9e29da2c0258b07f0ca2a7f67c14a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/35640
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16837 lov: NULL dereference in lov_delete_composite

commit 14ed4a6f8f retroduced the issue fixed by commit
5da049d9ef ("LU-14389 lov: avoid NULL dereference in cleanup), this
patch makes the fix cover the new case added by 14ed4a6f8f.

Fixes: 14ed4a6f8f ("LU-16837 llite: handle unknown layout component")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I4a2b72e21139b60519ed523b4851723c91f523c1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52826
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vitaliy Kuznetsov <vkuznetsov@ddn.com>

LU-11912 tests: fix racing in force_new_seq_all

We run force_new_seq in parallel to reduce time spent
on consuming precreated objects.

However this could be racy when multiple MDTs are on
the same MDS, a task could finish for one MDT early
and reset the fail_loc to 0 on MDS while other tasks
are still working on other MDTs.

Replace OBD_FAIL_OSP_FORCE_NEW_SEQ with a new param
prealloc_force_new_seq for osp, so we can control
the seq rollover individually for each osp device.

Change-Id: I52dbd550564ca628a8a85c42951694d58b2b93a9
Fixes: 656fc937cf ("LU-11912 tests: consume precreated objects in parallel")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52801
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 tests: Add missing /tmp/target2 under cleanup_src_tgt

cleanup_src_tgt() is called after each test case
under lustre-rsync-test.sh. The cleanup function
was missing /tmp/target2/... cleanup. This patch
adds the missing cleanup of folder /tmp/target2/...
It is good to have a clean start for next test case
as the diff (comparison) is performed with these
folders.

This patch also fixes space/tabs missmatch reported
by checkpatch

Test-Parameters: trivial testlist=lustre-rsync-test
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ieb7cfa60d894f43f1aa7b2510d03bd07eeb90a1e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52770
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11912 tests: force new seq in runtests

If seq rollover happens during runtests/1,
the new seq on OST will consume some space and
this will fail the free space check.

Force a new seq before running test case to
prevent this.

Change-Id: I7bb1156127eb423889626bf84bc6c87dd68e6ece
Test-Parameters: trivial testlist=runtests
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52741
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13805 llite: add flag to disable unaligned DIO

As with any new IO feature, it's a good idea to have the
option to turn off unaligned DIO support if needed.

It would be reasonable to merge this patch with the core
patch implementing the feature; I have kept it separate for
ease of review.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ibc86d84704151a7f30afcc538d9c03e3fdf1c38a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51125
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-10003 lnet: migrate old route API to Netlink / YAML API

With LNet route management now supported with the Netlink / YAML
API we can move the pre-MR route handling away from the
old ioctls. This allows use of large NIDs as well for IPv6
support.

Test-Parameters: trivial
Test-Parameters: testlist=conf-sanity env=ONLY="67"
Signed-off-by: James Simmons <jsimmons@infradead.org>
Change-Id: I1526d3e27967a10f556b8e31f15803659c0164bd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50440
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-10391 lnet: migrate router management to Netlink

Add the doit Netlink handling for routes. The creation and
deletion of routes can now work with large NIDs using the
Netlink API.

Test-Parameters: trivial testlist=sanity-lnet
Change-Id: Ia9fc64785962ce3e74abeef9c77e8b2df0789dd7
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50254
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17103 lnet: Avoid deadlock when updating ping buffer

lnet_peer_send_push() adds a reference to the the_lnet.ln_ping_target
lnet_ping_buffer. This reference is dropped by
lnet_discovery_event_handler. When the LNet configuration is modified
the ln_api_mutex is held and lnet_ping_target_update() is called to
update the ln_ping_target to reflect the new configuration.
While holding the ln_api_mutex, lnet_ping_target_update() will wait
until all refs on the old ping buffer are dropped. This can result
in a deadlock if the ln_api_mutex is required to complete the push.

Here is one scenario where this can happen:

1. PUSH is sent by discovery thread
2. LNet configuration is modified. lnetctl process is holding
ln_api_mutex and waiting in lnet_ping_target_update()
3. Local NI goes into recovery
4. Monitor thread wakes and attempts to send ping to local NI. If this
is the first ping sent to this NI then monitor thread needs
ln_api_mutex to create peer NI object for local NI.
(LNetGet->
 lnet_send->
lnet_select_pathway->
lnet_peerni_by_nid_locked->
mutex_lock(&the_lnet.ln_api_mutex))
5. PUSH (1) fails with local timeout. It is placed on monitor thread
resend queue.
6. monitor thread cannot process resend queue until it acquires
ln_api_mutex. ln_api_mutex cannot be acquired until monitor thread
processes resend queue. Deadlock.

Fix is to drop ln_api_mutex before calling lnet_ping_md_unlink() in
lnet_ping_target_update(). This should ensure that updates to the
ping target are still synchronized via ln_api_mutex as intended, but
we're able to clear refs on the old ping buffer as needed.

Test-Parameters: trivial
Test-Parameters: testlist=sanity-lnet env=ONLY=207,ONLY_REPEAT=50
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I20cda185a865192f1ad162eaef1b8b4e5d751b2c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52479
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>

LU-17188 mdt: remove n from LDLM_DEBUG

LDLM_DEBUG() doesn't need n in an extra message

Test-Parameters: trivial
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5a62cccb0a17b3f878206e8bbec6c1fbe07c4753
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52673
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17000 coverity: Fix 'Extra argument' under lst.c

This patch fixes 'Extra argument to printf' error reported
by coverity run.

CID: 397265 ("Extra argument to printf"): lst.c

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Iae039e9c6a27a71a852e7d44baab840eac17795e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52660
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17000 coverity: Fix use before null under pcc.c

This patch fixes 'dereference before null' error reported
by coverity run.

CoverityID: 397538 ("Dereference before null"): pcc.c

Test-Parameters: trivial testlist=sanity-pcc
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ic125f74b61e500e2184bde894aade7119b7a2899
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52658
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17182 utils: pool_add send OSTs in one batch

'lctl pool_add' command sends all requests in one batch
then checks results. In this way, the command won't take
too long time if the OSTs are specified in command line
one by one.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: Ibd6e7ed5104e100d44c5f4288a25e7378cd9cfe8
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52654
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17151 tests: increase sanity/411b memory limit

This test fails most of the time when run using
arm clients. It seems like the cgroup memory limit
was increased in a past revision for a similar issue.
Increase it a bit more for aarch64. Increase it a
smaller amount for x86.

Also, add some better logging for some other issues.

There's likely a better fix for this, but hopefully
this will let the test pass and provide some value
without having to do a full revert.

Fixes: 8aa231a99 ("LU-16713 llite: writeback/commit pages under memory pressure")
Test-Parameters: trivial
Test-Parameters: testgroup=review-ldiskfs-arm testlist=sanity env=ONLY=411b,ONLY_REPEAT=50
Test-Parameters: clientdistro=el8.7 testlist=sanity env=ONLY=411b,ONLY_REPEAT=50
Test-Parameters: clientdistro=el9.1 testlist=sanity env=ONLY=411b,ONLY_REPEAT=50
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If850077c0d7f6466082433776d370d24eee9736c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17000 coverity: llverfs: Add check for -n option

CID: 397833 (Uninitialized scalar variable): llverfs.c
This patch also add missing "-n" opiton under man page

Test-Parameters: trivial testlist=sanity,conf-sanity
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I8c795edba0cfde80402db0f877caeb91663629a3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52583
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10026 tests: Fix sanity test_56ab for CSDC

Use /dev/urandom in sanity test_56ab so the data cannot be compressed

Test-Parameters: trivial testlist=sanity env=ONLY=56ab

Signed-off-by: Wei Liu <sarah@whamcloud.com>
Change-Id: Ib0108f4507a7b46ad5ca973ef43351e005232edf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52572
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17146 tests: avoid sanity-lfsck/test_38 failure

This regression has been introduced in kernels after commit
v5.11-10234-gcbd59c48ae2b (5.12), and is fixed with
commit v6.2-rc4-61-g5956592ce337 (6.2).
The issue has been introduced by upstream
commit 8c8387ee3f55
("mm: stop filemap_read() from grabbing a superfluous page").
Skip sanity-lfsck/test_38 for this range of kernels.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ic6066e43959c913c2f225d229927803471f06cee
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52537
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17152 tests: unmount NFS clients with zconf_umount_clients

This patch fixes cleanup_nfs() to unmount NFS clients by running
zconf_umount_clients(), which can find and kill active processes
that are accessing the NFS mount point so as to avoid the
"device is busy" failure.

The patch also adds racer_on_nfs test into always_except list for
parallel-scale-nfsv4 due to LU-17154.

Test-Parameters: trivial testlist=parallel-scale-nfsv4

Change-Id: I37a38502362399540c28e78d1343e768b490ce8b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52533
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17138 enc: prefer specific crypto engines

Some ciphers provided by external accelerators might register under
the generic cipher name. To avoid using them with Lustre, prefer the
AES-NI variant implemented directly in the CPU. And fallback to the
generic cipher if AES-NI is not available.

Introduce a new libcfs kernel module parameter named
'client_encryption_engine' to give the ability to choose the cipher.
By default its value is 'aes-ni', which makes Lustre look for the
AES-NI cipher first. This parameter can be set to 'system-default'
whic makes Lustre pick the generic cipher.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8b00f1c3c8dcf11c58e9f40a410b57b2f255e642
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Shuichi Ihara <sihara@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17015 gss: support large kerberos token for rpc sec init

If the current Kerberos setup is using large token, like when PAC
feature is enabled for Kerberos, authentication can fail due to server
side unable to exchange token between kernel and userspace.
This limitation is inherent to the sunrpc cache mechanism, that can
only handle tokens up to PAGE_SIZE.

For RPC sec init phase, use Lustre's upcall cache mechanism
instead of deprecated kernel's sunrpc cache. The upcall calls a new
userspace command 'l_getauth', that fowards the sec init request to
the lsvcgssd daemon via Unix domain sockets.

Test-Parameters: kerberos=true testlist=sanity-krb5
Change-Id: I709cd79894a5a13fc4cdfab2109c86f2230db3b8
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52224
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>

LU-17000 misc: Fix Unused Value error(0)

This patch fixes unused value error reported
by coverity run.

CoverityID: 397913 ("Unused value"): lquota_disk.c
CoverityID: 397912 ("Unused value"): libmount_utils_zfs.c

Test-Parameters: trivial fstype=zfs testlist=sanity-quota,conf-sanity,sanity
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I509592ddc9b8fa3e9a6a7dcef4e476ad4fc9d9fc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52216
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17065 build: Remove snmp support

Last patched in 2012 and not well tended to before that,
snmp support can probably be removed.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I1831aed42e560531e57a6ff8aa978f3e5286fd44
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52194
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9859 libcfs: move kernel specific code out of libcfs core

Over time kernel version specific code has leaked into the libcfs
core code. Move that code to the linux subdirectory code so in
the future code cleanup is not missed.

Test-Parameters: trivial
Change-Id: I38a00c377334066160083edd3932d4a718198497
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52010
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17193 build: fix gcc-12 compiler warnings

Building on el9.2 hit a couple of new errors in configure, ex:
  ((struct inode_operations *)1)->fileattr_get()
hits:
  error: array subscript 0 is outside array bounds
         of ‘struct inode_operations[0]’

A few instances of QCTL_COPY() should be QCTL_COPY_NO_PNAME()
as the zero-length array to hold the pool name is not
allocated in these cases.

Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I72bda8b46c51dbd42fb42bf569ba29572526acfe
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52687
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17024 build: Make o2ib defines consistent

Move all O2IB specific configure tests to use HAVE_O2IB_
as the #define prefix.

Making the defines consistent removes some complexity
when cloning the o2ib sources to create in-kernel-o2iblnd

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I82e72a993abc51ad260d880317da32a75929583b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51937
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-930 doc: add lnet man page

Add a simple lnet man page which provides a short description of
lnet and points to all lnet related commands, like lnetctl and lst.

This page will also help users find more lnet related pages using
apropos.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia3c1be4e60118e00ab9378ef7cd2d53b25196cf2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51085
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8802 obd: remove MAX_OBD_DEVICES

Remove this arbitrary limit by reimplementing the array as an
Xarray. Xarray can grow and shink dynamically, hence saving
memory and allow for many more OBD devices. There is still
technically a limit OBD_MAX_INDEX, which is xa_limit_31b.max
or around 2 billion. This is far more than is practically
useful.

This patch also adds various iterators for OBD devices, which
are used to simplify code in various places.

Removing class_obd_list() since it is unused. Rename
class_dev_by_str() to class_str2obd() to keep the pattern.
Several class_* functions have been refactored to improve
locking. The larger issue of OBD device locking will be
addressed separately.

Update the OBD device lifecycle test to try loading
more devices (about 24,000 for now).

Currently, adding an additional OBD device is an O(n^2)
operation due to the class_name2dev calls in
class_register_device(). This will be addressed in a future
patch adding a hash table for OBD device name lookups.

Further, OBD life cycle management could likely be simplified
by using Xarray marks. Right now, it is handled by a bit
field in the obd_device struct. Since the scope of the changes
needed to simplify this seem large, this will also be addressed
separately.

Test-Parameters: testlist=sanity env=ONLY=55,ONLY_REPEAT=10
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Icb2cd94a5529e79f5d3ebd0de5e0f225cf212075
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51040
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15246 ptlrpc: per-device adaptive timeout parameters

When a client is mounting multiple filesystems with different
MGSes setting global parameters at_min, at_max, etc., then the
settings from one filesystem's MGS config will also apply to RPCs
sent for the OSC, MDC, and MGC devices on the other filesystem(s).
Typically the settings of the last filesystem to mount on the client
override the earlier values, and there is no way to separate them.

Moving the parameters to be per-device values allows them to be
set independently for each set of client devices, so that the
client can interact properly with each set of servers.  This allows
e.g. different timeouts for local and remote mounts, or for flash
and HDD filesystems that have different load and performance.

Add per-device adaptive timeout parameters that can optionally
replace global parameters of the same name:

    at_min     -> *.<fsname>*.at_min
    at_max     -> *.<fsname>*.at_max
    at_history -> *.<fsname>*.at_history
    ldlm_enqeue_min -> *.<fsname>*.ldlm_enqueue_min

These parameters should always be set with fsname in the device
name, rather than pure wildcard '*' device names, or it will be
be same as the global parameters in the end (settings from one
MGS will apply to devices on other filesystems).  That is a bug
in how "lctl set_param -P" works, but will be fixed separately.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I5b04c9aa53a446fb5a78bfaff372b4f236c9eb8a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14708 ptlrpc: skip unnecessary client eviction

A server does not update last_rcvd file on connection of new clients
synchronously. If the server fails over before the last_rcvd update is
committed, recently connected client may find itself evicted
unexpectedly.

If a client has not cached any data from a server and has not
performed any modifying rpcs to the server - let the client to connect
as a new one instead of considering itself as evicted.

Test to illustrate the issue is included.

Fixes: dcc8b9c00d5 "LU-9679 ptlrpc: list_for_each improvements"
Change-Id: I0c2d9c3b67cbc69c3283422f1f581b42f7f13a1a
HPE-bug-id: LUS-7141
Signed-off-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/43834
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>

LU-12316 llog: convert llog_client defines to functions

Convert preprocessor functions into actual functions. The
original commit that added these macros (81c713adeb) did
not provide an explaination for why these were chosen to
be macros rather than functions. Hence, bias towards using
functions rather than complex macros since they appear
in stack traces, are easier to debug, and simpler to
develop with.

Signed-off-by: Ben Evans <bevans@cray.com>
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I7a73e9af57dcf5edb9d013272ff5cf85e4234146
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/34904
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-5134 utils: Add parallel option to lctl set_param

Add a "-t" option to lctl set_param to enable setting multiple matched
parameters in parallel. When called with "-t", lctl will set up a work
queue of matched file names and spawn a fixed number of threads per
CPU. Each thread will pull items off the work queue, write to the file
associated with each work item, and return when there are no more
items on the work queue.

A field called po_parallel_threads is added to struct param_opts to
indicate the number of threads set_param should run in parallel. If in
parallel, jt_lcfg_setparam initializes a work queue and passes it to
do_param_op, which adds each matched item to the work queue. Once
jt_lcfg_setparam has called do_param_op for each param-value pair, it
passes the work queue to sp_run_threads, which creates threads, each
of which call write_param to set the parameter. If not in parallel,
jt_lcfg_setparam does not pass a work queue to do_param_op, and
do_param_op directly calls write_param on each matched param.

param_display was renamed to do_param_op to more accurately reflect
what it does.

If lctl is compiled without pthread support, "lctl set_param" will
still accept the "-t" option, but it will print a warning message, and
it will set the parameters in series.

The new "-t" option to set_param was documented in the lctl usage and
in the man page.

HPE-bug-id: LUS-2592
Signed-off-by: Ryan Haasken <haasken@cray.com>
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I3f96a6f06c50d4ba2ce97050c35f46b976dfc005
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/10555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 checkpatch: Add check for %zd

CWARN() or CERROR() requires a certain format. That is
it should start with '%s:' and end with '%d' or '%ld'.
Otherwise it will throw an warning. This patch extends
checks for type '%zd'. As few return types could be of
size_t.

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ida4f41af6b8fda0b028daeea18231b2d01905c31
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52421
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-17062 lnet: Update lnet_peer_*_decref_locked usage

Move decref's to occur after last reference to prevent
use after free.

HPE-bug-id: LUS-11799
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2382ece560039383f644b6aee73a9481d6bb5673
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17040 scrub: inconsistent item

When OI does not include the fid, scrub will attempt to
fix it with zero inode number. There is
low chance that fid would be found during full inode
scan. But inode scan requires an empty inconsistent
list. With repeated EINPROGRESS replies, inconsistent list is
always not empty.

Move fid with zero inode numbers to stale list.

1 scrub fix to print real OI resurect and
skip not related
2 out_handle debug for dt_locate() fid failed
3 debug for out requests when it was interrupted

HPE-bug-id: LUS-10780
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Iad9e9cba90b4648eb0fe8fa6c99984ada60fde70
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51997
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16952 debug: use right io object to check

cp_owner is io_top(io) not an io itself,
let's fix it to make invariants happy.

HPe-bug-id: LUS-11707
Test-Parameters: trivial
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: Ib473e8616e2c339fd54cd96af7933e437e8f5869
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51676
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16952 debug: don't put extra new line at output

n already in output line.

HPe-bug-id: LUS-11707
Test-Parameters: trivial
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I30667b5fe269aa8b80d5b2101d397f34b75f8e30
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51675
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10391 lnet: migrate peer NI control to Netlink

Move peer creation and deletion to the Netlink API. This change
enables the creation of peers with large NID addresses.

Test-Parameters: trivial testlist=sanity-lnet
Change-Id: I7f2f75e73e3f39856751f65e240f2172f703d0bc
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49574
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9680 lnet: collect data about peer_ni by using Netlink

Migrate the LNet peer NI API to use the Netlink API for
the case of collecting data on peer. This change also
allows large NID support for IPv6. Since this doesn't
cover creation and deletion of peers we can't setup
large NID peers just yet.

Test-Parameters: trivial testlist=sanity-lnet
Change-Id: Iefa3f566255e768047b0f9ff21d64bc74634f284
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49516
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-10391 lnet: migrate full LNet NI information collection

Fill in the rest of the LNet NI state to report to user land. This
mostly covers the stats information of the NI and LND specific
tunables. To handle the LND specific tunables we need to reorder
the code to send an updated key table. With the additional
information I found status wasn't properly set and the nesting
for was properly set for multiple NIs per NET. This is now fixed.

Test-Parameters: trivial testlist=sanity-lnet
Fixes: 8f8f6e2f36e ("LU-10003 lnet: use Netlink to support old and new NI APIs")
Change-Id: I32b06b1ce8cb049a33f45f2310d31897ffa7dc90
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50255
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6174 lov: use standard Linux 64 divison macros

For the case of 64 bit platforms lov_do_div64() is basically the
same as div_u64_rem(). For the 32 bit platform case lustre tries
to work around the issue of the divisor being larger than 32 bits.
We can also use div64_u64_rem() as well for 32 bit platforms since
Linux supports that as well.

With the move to Linux div64 operations we need to migrate fields
related to 64 bit size fields. To pick the correct div64 function
we need to select the proper sign value. The lov layer often uses
loff_t, which is signed, for stripe sizes which doesn't make sense
since sizes should always be positive. Creating a negative size by
mistake will give an bizarre offset into a file. Avoid any potential
issues with signed vs unsigned issues by replacing the loff_t use
in this case with u64.

Test-Parameters: testlist=large-lun env=ONLY=5,REFORMAT=yes
Change-Id: Ieadc266d43d6be1d6d47ee14ba9ac0dab01e7d86
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/39343
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15671 llite: cleanup code style in xattr.c

Clean up code style in llite/xattr.c.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I511d12daefdb509adbee65aaba9dea911ebaa18a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52418
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17187 ldiskfs: use ext4_fsblk_t for block pointers

instead of ext4_lblk_t which is 32bit while actual block
pointers can be larger than 2^32.

Fixes: 0f7e6c02a9 ("LU-16843 ldiskfs: merge extent blocks")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I201cfa5cb04907eef05bc87abc5701e8aed39d62
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52633
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-14027 tests: Fix test_135 of replay-single

The test_135 ("Server failure in lock replay phase") of replay-single
has an error on the method to get the pid of a background process.

Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Fixes: 7ca495ec67 ("LU-14027 ldlm: Do not hang if recovery restarted during lock replay")
Test-Parameters: trivial
Test-Parameters: testlist=replay-single env=ONLY=135,ONLY_REPEAT=50
Test-Parameters: testlist=replay-single env=ONLY_REPEAT=20,ONLY=135,FAILURE_MODE=HARD clientcount=4 mdtcount=1 mdscount=2 osscount=2 austeroptions=-R failover=true iscsi=1 testlist=replay-single
Change-Id: I6ed41d75f4cbba796e39288bad8895ee1c24459f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/41227
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17076 nrs: wait for RCU completion

before we destroy the slab holding the objects scheduled
for a release via RCU.

    BUG: unable to handle kernel paging request at 000000046474e5c6
    Oops: 0000 [#1] SMP
    CPU: 0 PID: 9 Comm: ksoftirqd/0  4.18.0 #3
    RIP: 0010:kmem_cache_free+0xe3/0x170
    Call Trace:
     rcu_core+0x27a/0x770
     __do_softirq+0xc2/0x44d
     run_ksoftirqd+0x35/0x50
     smpboot_thread_fn+0xb8/0x160
     kthread+0x14a/0x170

Fixes: 42bf5f78ba ("LU-8130 nrs: convert NRS ORR/TRR to rhashtable")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia4b1cf6f190f17c3b85548fcb6876be72822cdd2
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52515
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-17015 gss: bump token buffer size to 16KiB

A 4 KiB large buffer is not enough to hold the GSS token under some
circumstances. So bump GSS_CTX_INIT_MAX_LEN value to 16 KiB.

Fixes: 9758129177 ("LU-17015 gss: support large kerberos token on client")
Test-Parameters: trivial kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8e72f1447593d2bf2ae537fcc920ceee20e93c09
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52475
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12896 gss: key can be unlinked when timeout expires

The key associated with a GSS context could appear to be already
unlinked when the upcall timeout expires. In this case, do not assert
but report this case with a warning message.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I714af3a1ce54648c4ba29ef13015f9291de52765
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52473
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16218 utils: add component flags "prefrd" and "prefwr"

The initial implementation of "lfs setstripe ... --comp-flags=prefer"
only allowed specifying a single "prefer" argument for a given
mirror component, which would set both the "LCME_FL_PREF_RD" and
"LCME_FL_PREF_WR" flags at the same time.

This patch adds the separated component flags "prefrd" and "prefwr"
to allow setting the individual flags on a component.

Test-Parameters: trivial testlist=sanity-flr

Change-Id: I3e413cb37fab7ab2834946536705ce61a3feeed4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52508
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17136 ldiskfs: increase max extent tree depth

this is an workaround until LU-16843 ready

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5829c10888bf32649fe7a7a72c8ee697647a89cc
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52474
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17129 tests: cleanup fileset info on nodemaps

In sanity-sec, fileset info added to nodemaps via 'set_param -P' must
be removed afterwards with 'set_param -P -d', otherwise those commands
will remain in the llogs.

Test-Parameters: trivial mdscount=2 mdtcount=4 testlist=sanity-sec env=ONLY=27
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I75bd0dc263f71c7f5d9ece028cc038eb1f2ca9a4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52408
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17109 kernel: new kernel [SLES15 SP5 5.14.21-150500.55.22.1]

This patch makes changes to support new SLES15 SP5 release
with kernel 5.14.21-150500.55.22.1 for Lustre client.

Test-Parameters: trivial clientdistro=sles15sp5 testlist=sanity
Test-Parameters: trivial clientdistro=sles15sp4 testlist=sanity

Change-Id: I278017a5c996a8cf4e3d604aa928e968ca007312
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52340
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17084 lod: fix comparision in lod_striping_load()

in if (rc > sizeof(struct lmv_foreign_md)) the latter
is unsigned and gcc treats rc (which is defined as int
and can be negative to encode an error) as unsigned.
this way -EIO becomes greater than the size of the
structure.  make sizeof() signed to avoid confusion.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ie6735578649e397ed05b6951fab941f97051305b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52265
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>

LU-16796 target: Change struct barrier_instance to use kref

This patch changes struct barrier_instance to use
kref(refcount_t) instead of atomic_t

Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I2d2312b64d873e58bbef449c0867c679feb0c31b
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52173
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8191 libcfs: convert functions to static, removed function

Static analysis shows that a number of functions
could be made static. This patch declares several
functions in libcfs static.

Remove cfs_expr_list_values_free in string.c, since
it is not used.

A header was missing in param.c, causing a number of
functions to be missing declarations.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia580881efa806bde49d532e5b2d8f5097f2294e0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51428
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16962 build: cleanup configure messages

Take advantage of LB2_MSG_LINUX_TEST_RESULT to cleanup
the remaining configure checks.

Re-order checking of OpenSSL support so checking
message and result are not split.

Test-Parameters: trivial
HPE-bug-id: LUS-11709
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I2880e2b50f4cc79106201c241fe7c078e5d8c37e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51857
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Reviewed-by: xinliang <xinliang.liu@linaro.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-15002 utils: disable meta_bg and enable packed_meta_blocks

To take advantage of the upcoming e2fsprogs changes,
disable meta_bg to allow group descriptor blocks allocate
beyond group#0. This allows the group descriptor blocks
to be packed and placed at the beginning, rather than
distributed across the entire file system with meta_bg.

Enable packed_meta_blocks to place bitmaps, inode table
and journal at the beginning of the file system.

Due to LU-16971, use 200 blocks when testing mke2fs options.

Re-enable lazy_journal_init, which got turned off accidentally.

Fix up the handling of extented options, there should be spaces
in front and after -E when building the mke2fs cmd.

Test-Parameters: trivial testlist=conf-sanity
Change-Id: If4385c1d3740bf7a47bd12f90c2c93046bc8bead
Fixes: 701cc24959 ("LU-13533 utils: ext4lazyinit should be disabled")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51723
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16966 osd: take trunc_lock for fallocate

as fallocate may need few transactions (or transaction restarted)
we have to avoid any concurrent writes/truncates on this object
until fallocate supports 'restart-from-beginning' - first stop the
transaction, then release the lock, then repeat again (like
the write path does).

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I0bf38b1886fbf24656b45fe0f87fcbad2227672a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52264
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17131 ldiskfs: el8.1 use rhel8/ext4-enc-flag.patch

Correctly specify the ext4-enc-flag.patch for el8.1

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Ice1eca4f7092de7d65e5c1e5338dba6cc6e8f4ec
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52411
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17128 build: fix lnet.service missing issue on Ubuntu 22.04

The lnet.service file was in the lustre-client-utils package
built on Ubuntu 20.04, but it was missing from the package
built on Ubuntu 22.04.

This is caused by the dpkg-buildpackage change introduced by
dpkg version 1.21.1ubuntu2.1 installed by default on Ubuntu 22.04.
To fix this issue, we need to specify build profiles explicitly
to dpkg-buildpackage via -P|--build-profiles option instead of
just setting the environment variable DEB_BUILD_PROFILES.

Test-Parameters: trivial clientdistro=ubuntu2204

Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I9975ef357f0aba722c56d27eaa9b2cfbccc9c524
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52404
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-17124 llite: Write and wait on FIEMAP_FLAG_SYNC

fiemap FIEMAP_FLAG_SYNC flag expects filemap_write_and_wait()

HPE-bug-id: LUS-11854
Fixes: c16ecc8600 ("LU-5823 clio: add cl_object_fiemap()")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I40aa8d423164cf460fda5c11093a5f7b25682a96
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52402
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17050 tests: remove sanity-sec IDENTITY_UPCALL

In sanity-sec.sh there was an IDENTITY_UPCALL variable that
conflicted with an identically-named global variable in
test-parameters.  Due to new checks by Gerrit Janitor, this
was causing any patch that ran sanity-sec.sh to log a warning.

Remove the parameter from sanity-sec.sh as it is unused.
Update code style in functions upcating identity_upcall.

Test-Parameters: trivial testlist=sanity-sec mdscount=2 mdtcount=4
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2f2ace33cf01153d16f4f25038065d33443ebbe5
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52400
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16830 lod: accurate OSTs iteration for the last speed

The first fix removed lqr->lqr_start_idx from calculation
of ost_idx for the last speed loop. It was not enough
because lqr->lqr_offset_idx is used at calculation and
also could be changed by another thread. And it leads
to the same OST index during a loop.

1694009994: lod_ost_alloc_rr()) #0 strt 2166 act 0 strp 0 ary 0 idx 0
1694010094: lod_ost_alloc_rr()) #1 strt 2167 act 0 strp 0 ary 0 idx 0
1694010197: lod_ost_alloc_rr()) rc -28

Fixes: cacdaa9251 ("LU-16830 lod: improve rr allocation")
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Idf4b528dfbf3995cd03580a4214aff9206a52378
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52393
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17000 coverity: Fix Resource Leak(1)

This patch fixes Resource leak error reported
by coverity run.

CoverityID: 397493 ("Resource Leak"): liblustreapi_mirror.c
CoverityID: 397494 ("Resource Leak"): lustre_rsync.c

Test-Parameters: trivial fstype=zfs testlist=sanity,sanityn,conf-sanity
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I10e9fd0945cf13824c25faa62b4310796b09bade
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52384
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17111 kernel: update RHEL 9.2 [5.14.0-284.30.1.el9_2]

Update RHEL 9.2 kernel to 5.14.0-284.30.1.el9_2.

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el9.2 serverdistro=el9.2 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el9.2 serverdistro=el9.2 testlist=sanity

Change-Id: Id80dbba6b4434a83cf925d6961d727941274edf4
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52358
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17010 lfsck: don't dump stack repeatedly

If there are transactions started with LFSCK in dry-run mode, don't
dump the stack repeatedly, as this can spam the console logs and
significantly hurt performance.

Test-Parameters: trivial testlist=sanity-lfsck
Fixes: 0c1ae1cb9c ("LU-13124 scrub: check for multiple linked file")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0b0d64911453dc8ab947e284656311b5d0300c1e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52356
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17110 llite: fix slab corruption with fm_extent_count=0

If userspace uses fiemap with .fm_extent_count=0, .fm_extents[0] is
not allocated. Writing on the first entry without checking the extent
count could lead to memory corruption (slab).

This patch fix also the case when osc is disable: FIEMAP_EXTENT_LAST
should be set on the extent (fe_flags) and not on the fiemap struct.

Add a regression test sanityn 71d to test fiemap with
fm_extent_count=0.
Add a regression test sanity-hsm 408 to test fiemap on release files.

Fixes: 4097196 ("LU-11848 lov: FIEMAP support for PFL and FLR file")
Test-Parameters:testlist=sanityn
Test-Parameters:testlist=sanityn env=ONLY=71d,ONLY_REPEAT=20
Test-Parameters:testlist=sanity-hsm env=ONLY=408,ONLY_REPEAT=20
Signed-off-by: Etienne AUJAMES <etienne.aujames@cea.fr>
Change-Id: Id63c6973540187e678020977f2d555dfcbf3c634
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52352
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Feng Lei <flei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17095 build: avoid modules.order nonexistence failure

The modules.order is a temporary output file generated by
kbuild while running "make" command. Sometimes, there is
a race condition that causes the file not created and makes
make command fail as follows:

cat: ...//modules.order: No such file or directory

This patch creates an empty modules.order file to avoid
the error.

Test-Parameters: trivial

Change-Id: If779a727731f18e9409c35c0cd0deddd79559d3a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52323
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17076 orr: take a ref after lookup_get_insert_fast()

We need to take a reference on all existing object returned by
nrs_orr_res_get(). Otherwise, the object could be freed before
nrs_orr_res_put().
Therefore, a ref should be taken if
rhashtable_lookup_get_insert_fast() returns an existing object.

This avoids the following use-after-free in sanityn test_77c:

ptlrpc_nrs_req_stop_nolock+0x3b/0x150
ptlrpc_server_finish_active_request+0x2b/0x140 [ptlrpc]
ptlrpc_server_handle_request+0x40e/0xc00 [ptlrpc]
ptlrpc_main+0xc8c/0x1680 [ptlrpc]
kthread+0xd1/0xe0
ret_from_fork_nospec_begin+0x21/0x21

Fixes: 42bf5f7 ("LU-8130 nrs: convert NRS ORR/TRR to rhashtable")
Test-Parameters: fstype=zfs testlist=sanityn env=ONLY=77c,ONLY_REPEAT=100
Test-Parameters: fstype=zfs testlist=sanityn env=ONLY=77c,ONLY_REPEAT=100
Test-Parameters: fstype=zfs testlist=sanityn env=ONLY=77c,ONLY_REPEAT=100
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I267287a1075ee91019d3a4492b57f272a4b0cadd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52295
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17089 mdd: fix for bi_writers ref counter in case of error

If the mdd_child_ops() called from mdd_trans_create returns error,
we dont call barrier_exit to decrement the bi_writers. Call
barrier_exit in case of error returned from mdd_child_ops().

This patch also added more CDEBUG messages in lustre/target/barrier.c.

Signed-off-by: Jeya ganesh babu J <jeyaga@amazon.com>
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia430df404b700167cb9207eb13ac938575a2030a
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52275
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17000 coverity: Fix Resource Leak(0)

This patch fixes Resource leak error reported
by coverity run.

CoverityID: 339696 ("Resource Leak"): liblustreapi_layout.c
CoverityID: 397918 ("Resource Leak"): lsnapshot.c
CoverityID: 397894 ("Resource Leak"): obd.c
CoverityID: 397851 ("Resource Leak"): lfs.c
CoverityID: 397832 ("Resource Leak"): liblusteapi.c
CoverityID: 397772 ("Resource Leak"): liblusteapi_utils.c
CoverityID: 397721 ("Resource Leak"): obd.c

Test-Parameters: trivial fstype=zfs testlist=sanity,conf-sanity,sanity-lsnapshot
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I5c0152014f987264df17fac78390a2afc12c9255
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-17087 lmv: update stale tgt statfs every 1 hour

Some tgt statfs may not be initialized upon mount due to network
issues, if the filesystem is imbalanced, these tgts won't be chosen to
create directory because their bavail and ffree are 0.

If MDT is chosen by QoS, update tgt statfs that is one hour overdue,
otherwise check update the statfs of the tgt that is chosen.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I06af8b8bd342f66cb794471df3ee0f3b127ffe05
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52270
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>

LU-16954 llite: add SB_I_CGROUPWB on super block for cgroup

Cgroup support can be enabled per super_block by setting
SB_I_CGROUPWB in ->s_iflags.
Cgroup writeback requires support from both the bdi and
filesystem.
This patch adds SB_I_CGROUPWB flag on super block for Lustre.
This is required by the subsequent patch series to support
cgroup in Lustre.

Adding this flags for Lustre super block will cause the remount
failure on Maloo testing on Unbutu 2204 v5.15 kernel due to the
duplicate filename (sysfs) for bdi device.
To avoid remount failure, we explicitly unregister the sysfs for
the @bdi.

Test-Parameters: clientdistro=ubuntu2204 testlist=sanity-sec
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I7fff4f26aa1bfdb0e5de0c4bdbff44ed74d18c2d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51955
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-13308 mdc: support additional flags for OBD_IOC_CHLG_POLL ioctl

Currently the mdc kernel code expects the flag argument for
OBD_IOC_CHLG_POLL ioctl to only be CHANGELOG_FLAG_FOLLOW. With
IPv6 we need to send a request to the kernel to present the NID
in the struct lnet_nid format since we can't just send large NIDs
to user land if we are using older tools.

With the newer user land tools we will be sending an expanded flag
which the current kernel changelog code can't handle. Rework the
code to support the new flag if we end up with the case of newer
user land tools and an older kernel. This code will also maintain
backwards compatiblity with the older user land tools.

Change-Id: I26a80d30ce2ebf2075a2a8f510ff81d6b0b8d848
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52361
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-16134 utils: adds lctl apply_yaml

Commnad set_param -F is used to parse yaml file with settings,
and makes set_param -P for it. Many type of settings are based
on a specific device and have conf_param type. When such settings
goes to set_param -P, all nodes tries to apply it and many errors
happen.
systemd-udevd[568906]: Process '/usr/sbin/lctl set_param
'osp.kjcf04-OST0003-osc-MDT0000.resend_count=43''
failed with exit code 2.

The patch adds functionality for conf_param event of YAML file,
and introduces lctl apply_yaml for both types of event.
YAML example
- {device: testfs-MDT0001, event: conf_param, index: 76, parameter:
   lod.qos_threshold_rr=100}
- { index: 17, event: set_param, device: general,
    parameter: jobid_var, value: procname_uid }

Test-Parameters: trivial
HPE-bug-id: LUS-11116
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Iec3b1f14b9ddb85ef3e110bbc4467d0d6c80c136
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48419
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Artem Blagodarenko <ablagodarenko@ddn.com>

LU-14470 dne: striped mkdir replay by client request

Once all involved MDTs of a striped mkdir were rebooted, or MDT
recovery was aborted, this mkdir will be replayed by client request.
To correctly replay such mkdir, pack directory LMV in mkdir reply,
and save it to request from reply, and MDS should use this layout to
replay mkdir.

For MDT recovery abort case, the original mkdir may be partially
executed, so mkdir replay should check below cases and don't treat
them as error:
* name entry is found on parent directory on remote MDT.
* stripe exists on remote MDT.

For backward compatibility, Add MDS_MKDIR_LMV flag to indicate a
client requires directory LMV in mkdir reply.

Updated replay-single 100c since striped mkdir can replay now.

Updated recovery-small 130 since create fetches layout now.

Added replay-single 100e.

Test-Parameters: mdscount=2 mdtcount=4 testlist=racer,racer,racer,racer,racer
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If0cc8f4aebbe55cc28786d6b4198dbb57743feb3
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47385
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13805 clio: bounce buffer for unaligned DIO

Direct I/O must normally be page aligned both in terms of
I/O size and memory alignment.  This is because the I/O
must be page aligned before being written to disk.  This
is also true for buffered I/O, but the I/O is aligned
using the page cache.

In recent versions of Lustre, direct I/O is significantly
faster than buffered I/O, due to lower overhead for page
management.  Thus it is desirable to be able to do more
I/O as direct I/O.

This patch allows unaligned direct I/O by creating a buffer
inside the kernel and aligning the I/O by copying in to
this aligned buffer.  Because the main cost of buffered I/O
is locking in the page cache rather than memcopy(), this is
still significantly faster than buffered I/O (though slower
than normal direct I/O).

This will eventually allow us to convert buffered I/O to
direct I/O when doing so would increase performance.

Here are some comparative benchmarks using IOR, all single
process.

UDIO is unaligned DIO.

io size   1M           4M          16M          64M
----------------------------------------------------------
BIO Write  | 1502 MiB/s | 1382 MiB/s | 1683 MiB/s | 1677 MiB/s
BIO Read   | 2169 MiB/s | 1902 MiB/s | 2131 MiB.s | 1955 MiB/s
DIO Write  | 1010 MiB/s | 2778 MiB/s | 5905 MiB/s | 7917 MiB/s
DIO Read   | 893 MiB/s  | 2657 MiB/s | 4724 MiB/s | 7579 MiB/s
UDIO Write | 848 MiB/s  | 1666 MiB/s | 2117 MiB/s | 2243 MiB/s
UDIO Read  | 933 MIB/s  | 2412 MiB/s | 3690 MiB/s | 5370 MiB/s

Unaligned DIO offers benefits vs buffered write and
buffered read, but is of course slower than DIO.

Notice on this node the best case DIO performance is
~8 GiB/s.  On a node with 12 GiB/s best case DIO, best case
UDIO read is 8 GiB/s and best case UDIO write is 2.5 GiB/s.

This is because UDIO read is fully parallel, UDIO write is
not.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7eeebf9a608f006c8095b95f0677adb99f19d640
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/45616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-13805 llite: unaligned direct_rw_pages

Add support for ll_direct_rw_pages to handle unaligned
IO by allowing both the first and last page to be partial
pages.

This has been broken off from the main unaligned DIO patch
to make it more reviewable.

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I055105589d5416fe6aa82fb6a087db7b8b38c8d1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49993
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>