LU-15850 lmv: always space-balance r-r directories

If the MDT free space is imbalanced, use QOS space balancing for
round-robin subdirectory creation, regardless of the depth
of the directory tree.  Otherwise, new subdirectories created
in parents with round-robin default layout may suddenly become
"sticky" on the parent MDT and upset the space balancing and
load distribution.

Add sanity/test_413h to check that round-robin dirs always balance.

Lustre-change: https://review.whamcloud.com/47578
Lustre-commit: 37c1ddc34d3a1e61c5533f48cb29fe2258ca2907

Test-Parameters: testlist=sanity env=ONLY=413h,ONLY_REPEAT=100
Fixes: 38c4c538f5 ("LU-15216 lmv: improve MDT QOS space balance")
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ia1d0b5b1a027cf14236f93ae34b5cf4929e76d23
Reviewed-on: https://review.whamcloud.com/47871
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15850 llite: pass dmv inherit depth instead of dir depth

In directory creation, once it's ancestor has default LMV, pass
the inherit depth, otherwise pass the directory depth to ROOT.

This depth will be used in QoS allocation.

Lustre-change: https://review.whamcloud.com/47577
Lustre-commit: c23c68a52a04369101db2bd3b1d3da23025fcf48

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id480f32c1718e9f62314c2dfe8905be5db94d1f2
Reviewed-on: https://review.whamcloud.com/47870
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15850 mdt: pack default LMV in open reply

Add flag MDS_OPEN_DEFAULT_LMV to indicate that default LMV should be
packed in open reply, otherwise if open fetches LOOKUP lock, client
won't know directory has default LMV, and in subdir creation default
LMV won't take effect.

Lustre-change: https://review.whamcloud.com/47576/
Lustre-commit: f6e4272fb0be5b798b7685bb40067e3f6877c8a5

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If2300ca39f406169eff9eab8f973ca1c2bfc8202
Reviewed-on: https://review.whamcloud.com/47869
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn55

New tag 2.14.0-ddn55

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I503bf7a4bc8d5f36e24563452170293fbf19a9ed

LU-16045 enc: force use of new enc xattr on new servers

When an older client uses encryption with a newer server, the client
wants to see the encryption context in security.c xattr. But
internally on server side, we force use of newer encryption.c xattr
for consistency purpose. When required, the encryption context is put
in the request to the client as usual, which interprets it as desired.

Fixes: ba50f9a977 ("LU-13717 sec: make client encryption compatible with ext4")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I667e123bdff912acc270666e8c74ebda6f0534e7
Reviewed-on: https://review.whamcloud.com/48034
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16030 pcc: enlarge PCC backend size for sanity-pcc/45

sanity-pcc/test_45 failed with the following error:
"attach more than 2 time: 0"
The reason is that the allocated loopback deivce for PCC backend
is too small.
In this patch, we enlarge it to 100MiB to avoid running out of
the space on PCC backend device during PCC-RO attach and fix the
failure of sanity-pcc/test_45.

This patch also fixes the typo error "RANDON" -> "RANDOM".

Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Ib19f01ed054cb6c9eecceabea1f1da72dea0b113
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47994
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-16010 kernel: kernel update RHEL8.6 [4.18.0-372.16.1.el8_6]

Update RHEL8.6 kernel to 4.18.0-372.16.1.el8_6.

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Change-Id: I08db577f31a1d686b88804384a05d5b418e634d5
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47992
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15548 osd-ldiskfs: hide virtual projid xattr

Add tunable enable_projid_xattr to hide the virtual
project ID xattr by default.

Lustre-change: https://review.whamcloud.com/46900
Lustre-commit: ef826db1f43b2849af26491028f4cfc25595faf6

Fixes: 665383d3a1f4 ("LU-12056 ldiskfs: add trusted.projid virtual xattr")
Change-Id: I21263d91599f9e2d5850cb9d94a8b6df90c8443c
Test-Parameters: trivial testlist=conf-sanity env=ONLY=131
Test-Parameters: testlist=sanity env=ONLY=904
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47989
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15727 lod: honor append_pool with default composite layouts

In lod_get_default_lov_striping(), correct the handling of composite
default layouts in the case where append_stripe_count is nonzero.
Align the names of the append members of struct dt_allocation_hint
with the mdd params. Remove the unused dah_mode member of struct
dt_allocation_hint.

Add sanity test_27U() to verify.

Fixes: e2ac6e1eaa ("LU-9341 lod: Add special O_APPEND striping")

Lustre-change: https://review.whamcloud.com/47014
Lustre-commit: 03963106926883cf322e085feb8caa3ea64db1d1

Change-Id: I66b426d24d6476fb483397f290229983f3da4be5
Signed-off-by: Xing Huang <hxing@ddn.com>
Reviewed-on: https://review.whamcloud.com/48027
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15993 ofd: don't leak pages if nodemap fails

ofd_commitrw() shouldn't exit w/o calling ofd_commitrw_write(),
otherwise the pages taken in ofd_preprw() are leaked.

same in mdt_obd_commitrw()

Lustre-change: https://review.whamcloud.com/47873
Lustre-commit: ee4b50278e4471b7c64e619e13699c560502b81e

Fixes: bbfdc7c167 ("LU-14739 quota: fix quota with root squash enabled")

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Icd60c7ab80c5a7b65603d7da0d2e83872dc6b97f
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/48002
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15942 utils: ofd_access_log_reader exit status

If no OSTs are mounted then the ofd module may not be leaded and hence
/dev/lustre-access-log/control may not exist. In
ofd_access_log_reader, if --exit-on-close is used then we should
handle this consistently with the case of no access logs by exiting
with status 0.

Lustre-change: https://review.whamcloud.com/47625
Lustre-commit: 9bf968db56ddbad525dffe6fc4b9fde7d55dbde3

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I91b059bee8941501f2d207d2a48d1ea5ad40ae99
Reviewed-on: https://review.whamcloud.com/48100
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15821 ldlm: Prioritize blocking callbacks

The current code places bl_ast lock callbacks at the end of
the global BL callback queue.  This is bad because it
causes urgent requests from the server to wait behind
non-urgent cleanup tasks to keep lru_size at the right
level.

This can lead to evictions if there is a large queue of
items in the global queue so the callback is not serviced
in a timely manner.

Put bl_ast callbacks on the priority queue so they do not
wait behind the background traffic.

Add some additional debug in this area.

Lustre-change: https://review.whamcloud.com/47215
Lustre-commit: 2d59294d52b696125acc464e5910c893d9aef237

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic6eb65819a4a93e9d30e807d386ca18380b30c7d
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47217
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn54

New tag 2.14.0-ddn54

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib34f8a48ccf2a90b320a509e8c5d122841f62d92

LU-15991 kernel: kernel update RHEL7.9 [3.10.0-1160.71.1.el7]

Update RHEL7.9 kernel to 3.10.0-1160.71.1.el7.

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I89215145ea8da2925e5c8c01cdf963ba8a087877
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47922
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn53

New tag 2.14.0-ddn53

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6497712245ad88e7951667416d6b67a7657cb1f8

EX-4333 sec: traces for supplementary groups handling

Add traces in the code to ease understanding of supplementary groups
handling, in particular for the INTERNAL identity_upcall.

Change-Id: I4b2298cf5c3b400e7a1436384653ce01a462a2e0
Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/47697
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15779 ofd: don't hold read lock over bulk

as this can block all operations on OST:

1) ofd_preprw_read() takes a shared object lock and initiates BULK
2) OUT needs an exclusive object lock on the same object
3) ofd_commitrw_write() starts transaction and now has to wait
   for OUT to get and release that exclusive object lock (step 2)
4) number of threads can get stuck waiting for ofd_commit_write()
   to stop it's transaction

this patch drops a shared object lock before BULK transfer.
at the moment it's not clear how such read would race with
object removal on ZFS - this should be investigated.

Lustre-change: https://review.whamcloud.com/47126
Lustre-commit: 98ba50819024b908453b62fd095647442929a61f

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I131493abd90283e9ca897f904e00c25d26e3d8d3
Reviewed-on: https://review.whamcloud.com/47824
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15653 client: able to cleanup devices manually

Using 'lctl cleanup/detach' could be needed in situations
with unclean umount. Meanwhile that doesn't work now for
LMV and also could cause panic after all

Patch restores ability to cleanup/detach client devices
manually.
- debugfs and lprocfs cleanup in lmv_precleanup() are moved
  lmv_cleanup() to be not cleared too early. This prevents
  hang on 'lctl cleanup' for LMV device
- test 172 is added in sanity. It skips device cleanup during
  normal umount, keeping device alive without client mount
  then manually cleanups/detaches them
- prevent negative lov_connections in lov_disconnect() and
  handle it gracefully
- remove obd_cleanup_client_import() in mdc_precleanup(),
  it is called already inside osc_precleanup_common()

Lustre-change: https://review.whamcloud.com/46859
Lustre-commit: 210803a2475862464045b609eee43237d74e853b

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I8a3868fabd1d805e827d04852d1614a3fe57ce35
Reviewed-on: https://review.whamcloud.com/46587
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15938 lod: prevent endless retry in recovery thread

- abort lod_sub_recovery_thread() by obd_abort_recov_mdt in
  addition to obd_abort_recovery
- handle 'short llog' situation gracefully, when remote llog
  is shorter than local copy header expects, trust remote llog
  data and consider llog processing as finished
- on other errors during remote llog read, set obd_abort_recov_mdt
  but not obd_abort_recovery in attempt to skip MDT-MDT recovery
  only and continue with client recovery while possible
- fix parsing problem with 'abort_recov' and 'abort_recov_mdt' in
  lmd_parse() causing no MDT recovery abort but client recovery
  abort always. Allow also 'abort_recovery_mdt' mount option name

The original case with endless retry is caused by such de-sync
between local llog structures and remote llog. The local llog
header says there is record with some ID, so recovery thread
is trying to get that record from remote llog. Meanwhile there
is no such record on remote server, so it reads whole llog and
return it back properly but llog processing consider that as
incomplete llog due to network issues and retry endlessly.

Lustre-change: https://review.whamcloud.com/47698
Lustre-commit: TBD (b57b8f126e0fe00e20b1a6c3164fdf902baf91c0)

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ib127fd0d1abd5289d90c7b4b3ca74ab6fc78bc71
Reviewed-on: https://review.whamcloud.com/47889
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15938 llog: llog_reader to detect more corruptions

Improve llog_reader to determine more corruptions and report
errors
 - notify if llog bitmap has bits set with no records in llog
 - compare header records count with amount of records really
   found
 - fix amount of records to output, preventing wrong output of
   NOT SET record
 - list missing records in gap if found
 - count all errors found, add prefix 'error:' in output for
   better output processing by third-party scripts
 - don't exit immediately in case of error but continue if
   possible and output all read valid data

Lustre-change: https://review.whamcloud.com/47934
Lustre-commit: 74bb524ee9b437777ecfe4e99fd032d8cac95cd0

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic47dc6bb6cbdd9db6f888a0b892254403a628912
Reviewed-on: https://review.whamcloud.com/47949
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn52

New tag 2.14.0-ddn52

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1c9f9793f83a34e40cfdd7f61617b0cd9e83fa99

LU-14893 lctl: check user for changelog_deregister

If no user is specified for "lctl changelog_deregister", usage
should be printed correctly.
Also, sanity.sh test_106e is modified to verify this fix.

Lustre-change: https://review.whamcloud.com/44432
Lustre-commit: 644cb83921451316d226d8de1c81a82abb4b333e

Fixes: a15eb4f13224e ("LU-13055 mdd: per-user changelog names and mask")
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ia7f1b18e82f6b4174b9435cd67aba5f591d43ce1
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47837
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15969 pcc: use new ->fileattr_set for PCC project

The new Ubuntu 2204 (since kernel 5.12) introduces a new inode VFS
interface ->fileattr_set() and ->fileattr_get() to set/get project
ID on a file.

In PCC, we set the project ID for PCC copies on PCC backend file
system via IOCTL FS_IOC_FSSETXATTR.
However, the new kernel removed this IOCTL code, this results in
that using this ioctl to set project ID for the PCC copy returns
-ENOTTY and sanity-pcc test_39 failed.

In this patch, we add compatibility check for this new VFS
interface about project ID, and use the new VFS interface to set
the project ID on the new kernel properly.

Test-Parameters: clientdistro=ubuntu2204 env=SANITY_PCC_EXCEPT="101a" testlist=sanity-pcc
Change-Id: I756f3d6fa97c60c416deca9e482270483ab06524
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47811
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5358 pcc: remove realpath() from lfs_pcc_state()

Before Ubuntu 20.04, realpath() executes lstat() for each
component of the path. If the file is still valid cached on PCC
device with the layout generation unchanged, the Lustre file will
be auto re-attach during the stat() call in the Lustre kernel.
This may result in misunderstanding for the operation "lfs pcc
state" on a file that has already detached but still valid cached
on PCC according to the unchanged layout generation.

This problem exposes on the newer Ubuntu 22.04 in which realpath()
executes readlink() for each component of the path instead of
lstat():
readlink("/mnt", 0x7fffd5760800, 1023)  = -1
readlink("/mnt/lustre", 0x7fffd5760800, 1023) = -1
readlink("/mnt/lustre/sanity-pcc.f15", 0x7fffd5760800, 1023) = -1

In this patch, it remove realpath() from lfs_pcc_state() to avoid
this misunderstanding behavior for the command: $LFS pcc state.
And it also fixes the test scripts sanity-pcc: test_15, test_16,
test_27, test_39.

Change-Id: I50ae46a1e952a3faaf0d7a7293579e239156d6d3
Test-Parameters: clientdistro=el8.5 env=SANITY_PCC_EXCEPT="101a" testlist=sanity-pcc
Test-Parameters: clientdistro=ubuntu2204 env=SANITY_PCC_EXCEPT="101a 39" testlist=sanity-pcc
Test-Parameters: clientdistro=ubuntu2004 env=SANITY_PCC_EXCEPT="101a" testlist=sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47632
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5497 sec: fix encrypted symlink length

Length of encrypted symlink needs to be the length of the symlink
target. To achieve this, we need to read the symlink target and
decrypt or decode it in ->getattr(). This is possible thanks to an
enhanced getattr capable of getting more stat information.

Unfortunately, configure check for kernel support of this enhanced
getattr is incomplete, making the code call the basic getattr.

Change-Id: I03bf68679a02542b3fb696bb7458ff3276423784
Fixes: ca483b3b85 ("LU-14651 llite: extend inode methods with user namespace arg")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/47863
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn51

New tag 2.14.0-ddn51

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic47ffc541b6b918ac30286c488e89159bcbf2a57

LU-15308 lod: set ldo_mirrors to NULL on error

In lod_alloc_comp_entries() if OBD_ALLOC_LARGE fails and we free
ldo_mirrors then we set it to NULL to avoid a double free later.

Lustre-change: https://review.whamcloud.com/47008
Lustre-commit: 26f6b0e952b24de2a900f6f1eeae79fea6af4787

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I27552d1eaecc06720190db9cfe89ded71b9fef53
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47850
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5462 build: build lipe-lpcc in all distro/arch

Test-Parameters: trivial
Test-Parameters: clientdistro=el8.6 clientarch=aarch64 testlist=sanity-pcc env=ONLY=210

Change-Id: Ic9a381fc2582e63863b968e1dc18bee5e181ca7f
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47834
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

EX-5477 build: link libjson-c directly to liblustreapi

To avoid adding a liblustreapi linker dependency to libjson-c,
which makes it harder to build applications using this library,
link libjson-c directly to liblustreapi fir the few functions
that it needs.

- added configure check for yaml
- added configure check for json
- added json library to the Lustre liblustreapi
- replaced hardcoded -lyaml with variable
- replaced hardcoded -ljson-c with variable
- removed redundant json-c links

Test-Parameters: trivial
Fixes: fbfd2d075534 ("EX-5176 pcc: use JSON string for trusted.pin xattr")
Change-Id: I57b8abb5170b3e3a4d18782c5611929c7c2b7072
Signed-off-by: Alex Deiter <alex.deiter@gmail.com>
Reviewed-on: https://review.whamcloud.com/47844
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

EX-5468 lipe: use absolute path in lpcc.service

Some platform needs absolute path for command in service script.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I11ff46ceaffa839540736e46d84fb22b7343ff80
Reviewed-on: https://review.whamcloud.com/47840
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15625 sec: fix set_param -P for root squash

lctl set_param -P needs to be fixed for root squash in order
to properly propagate to clients.
On server (MDT) side, the paramaters are mdt.*.*squash*.
On client side, the parameters are llite.*.*squash*.
So if the setting on MGS side is done as
set_param -P *.*.root_squash and
*.*.nosquash_nids, it works fine. But if it is done as
set_param -P mdt.*.root_squash, this setting is not applied to
client side.
The solution is to have the settings translated into
*.*.root_squash and *.*.nosquash_nids to make both sides happy.

Also, to ensure client vs. server consistency, prevent
set_param -P llite.*.root_squash.

Lustre-change: https://review.whamcloud.com/46739
Lustre-commit: 7b8449e8f9ed310376a191ec8dfa2f1cb6233377

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8984c9d9024d7a053e548ed15d1321b281d9940f
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47787
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15911 enc: null encrypted names is embedded llcrypt only

enable_filename_encryption tunable only makes sense when Lustre client
is built against embedded llcrypt. When built against in-kernel
fscrypt, this tunable is silently ignored, as fscrypt always carries
out file name encryption.

So have the enable_filename_encryption tunable only when Lustre client
is built against embedded llcrypt. Also fix sanity-sec test_54 so that
it works for in-kernel fscrypt.

Lustre-change: https://review.whamcloud.com/47520
Lustre-commit: TBD (98621cb3f33e39c458ef1c02da3b23a69d481b61)

Fixes: e68d496ada ("LU-15858 sec: reinstate null encryption for file names")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ibe52feb670a00c9f421907ecd438bcccc62856f0
Reviewed-on: https://review.whamcloud.com/47785
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15546 mdt: mdt_reint_open lookup before locking

This patch is an optimization of 33dc40d ("LU-10262 mdt:
mdt_reint_open: check EEXIST without lock").

The current behavior is to take a LCK_PR on parent to verify if the
file exist and then take a LCK_PW to create the file.

Here we do a lookup to determine the mode before tacking a lock.
This avoid to re-lock each time for create cases.

Most of the time we have:
1. lookup the child in parent directory
2. take the parent lock: file_exist ? LCK_PR : LCK_PW
3. re-lookup the child

In a race senario (create/unlink) we have:
1. lookup child in parent directory -> file exists
2. take a LCK_PR on the parent
3. re-lookup the child -> file doesn't exist
2. take a LCK_PW on the parent
4. re-lookup the child

This patch fix the "SKIP" condition for sanityn 41i/43k/45j and clear
the LRU locks cache for sanityn 43k/45j.

Lustre-change: https://review.whamcloud.com/46679
Lustre-commit: f14090e56c9d94e3cfaa6f13f357173d6d570547

LU-15907 mdt: fix the OBD_FAIL_MDS_REINT_OPEN2 race

This patch fixes the sanityn test_41i OBD_FAIL_MDS_REINT_OPEN2 race
scenario.

With the "LU-15546 mdt: mdt_reint_open lookup before locking" patch,
we have to force taking a LCK_PR for OBD_FAIL_MDS_REINT_OPEN2 to test
the full lock cycle:

- take PR parent lock
- lockup child (do not exist)
- take PW parent lock
- re-lookup
- create child

Otherwise cfs_race() timeouts because PW lock requests from the 2
threads conflict with each other.

Lustre-change: https://review.whamcloud.com/47506
Lustre-commit: TBD (a15b4e389aa6d2a30d9b79a20f0884e3725c70d7)

Fixes: 33dc40d ("LU-10262 mdt: mdt_reint_open: check EEXIST without lock")
Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: I121abd4babfb516d7a64682b054a6443d38590ef
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47810
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15521 spec: fix more bare words error with rpm 4.16

This patch fixes more bare words errors and extra tokens
warnings with rpm 4.16.

Lustre-change: https://review.whamcloud.com/47833
Lustre-commit: TBD (from 091bb5a39957b17cd3ebe77314870bbcc386eab6)

Test-Parameters: trivial

Change-Id: Ic40b5763d1cb362d5aa77b06e9a5768b2abbc708
Fixes: 597a6bf9e085 ("LU-15521 spec: fix bare words error with rpm 4.16")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47842
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15563 build: Include the new linux/blk-integrity.h

Linux v5.15-rc6-17-gfe45e630a103
  block: move integrity handling out of <linux/blkdev.h>

Include the new linux/blk-integrity.h

Since blk-integrity is only used in osd-ldiskfs we can migrate
the compatibility wrappers:
  blk_integrity_interval(),
  blk_integrity_name(),
  bdev_integrity_enabled(),
and
  bip_size()
to osd-ldiskfs/osd_internal.h

Lustre-change: https://review.whamcloud.com/47166
Lustre-commit: afe944a12091081b3bfeaa2365fb03ef272ba9ba

HPE-bug-id: LUS-10921
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Idcbef9b43b7d2a740291e3783f81c23454209a25
Reviewed-on: https://review.whamcloud.com/47841
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15451 sec: retry ro mount if read-only flag set

In case client mount fails with -EROFS because the read-only nodemap
flag is set and ro mount option is not specified, just retry ro mount
internally. This is to avoid the need for users to manually retry the
mount with ro option.

Lustre-change: https://review.whamcloud.com/47490
Lustre-commit: TBD (db0765ffda943399c9c095fbf82bcab9079feadd)

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0dedd1394eeb6804f7fdde930275f6649b935bab
Reviewed-on: https://review.whamcloud.com/47817
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15451 sec: read-only nodemap flag

Add a new 'readonly_mount' property to nodemaps. When set, we return
-EROFS from server side if the client is not mounting read-only.
So the client will have to specify the read-only mount option to be
allowed to mount.

Lustre-change: https://review.whamcloud.com/46149
Lustre-commit: TBD (a8cbf96538dbee8104180f63af037cbc5c9e08e9)

Fixes: 928714dddabb ("LU-5092 nodemap: save id maps to targets in new index file")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9931844ae46dfd5d724f592f8dfacc4a8011c7e3
Reviewed-on: https://review.whamcloud.com/47816
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn50

New tag 2.14.0-ddn50

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6239d5360245aa4cae1066c6b343803cbc59e1cd

LU-15901 mdc: Remove entry from list before freeing

mdc_changelog_cdev_init forgot to remove entries from list if
chardev allocation failed

Lustre-change: https://review.whamcloud.com/47480
Lustre-commit: 441ec2296a0938dd30cf60a70943eb0799d1d70c

Change-Id: Ic76b5320bf80c7f7f60c7682bda4bc37a0b300bd
Fixes: d0423abc1ad ("LU-12506 changelog: support large number of MDT")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47783
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11695 som: disabling xattr cache for LSOM on client

To obtain uptodate LSOM data, currently a client needs to set
llite.*.xattr_cache =0 to disable the xattr cache on client
completely. This leads that other kinds of xattr can not be cached
on the client too.
This patch introduces a heavy-weight solution to disable caching
only for LSOM xattr data ("trusted.som") on client.

Lustre-change: https://review.whamcloud.com/33711/
Lustre-commit: TBD (from 55a65276b8e62f0aa894ee0719051ed8737ab59e)

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iab5ef3030b05ac09184d01f2a3a8ed92ff1cf26b
Reviewed-on: https://review.whamcloud.com/47813
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5419 lipe: retry to get lpcc_purge stats file

Sleep and retry to get lpcc_purge stats file after sending signal
Correct statistics data algorithm.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: clientdistro=el8.5 testlist=sanity-pcc env=ONLY=210
Change-Id: I79e4b7de871f8ffd290e0c30da4780265fa7e9fa
Reviewed-on: https://review.whamcloud.com/47729
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5366 test: handle missing lpcc.conf in test_210

Handle missing /etc/lpcc.conf in sanity-pcc test_210

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial clientdistro=sles15sp3 testlist=sanity-pcc env=ONLY=210
Test-Parameters: clientdistro=el8.6 clientarch=aarch64 testlist=sanity-pcc env=ONLY=210
Change-Id: Ia19c32ac2719a52ee880a2e633495b9ce3bb9534
Reviewed-on: https://review.whamcloud.com/47728
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15855 enc: enc-unaware clients get ENOKEY if file not found

To reduce issues with applications running on clients without keys
or without fscrypt support that check for the existence of a file in
an encrypted directory, return -ENOKEY instead of -ENOENT.
For encryption-unaware clients, this is done on server side in the
mdt layer, by checking if clients have the OBD_CONNECT2_ENCRYPT
connection flag.
For clients without the key, this is done in llite when the searched
filename is not in encoded form.

Lustre-change: https://review.whamcloud.com/47349
Lustre-commit: 00898697f998c095e6dab3dd27e347cb1150ee8c

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9a3b7af3a856b7fc7222c61a308ad23168869d57
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47786
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5427 sec: fix pcc detach of encrypted file

In case of 'lfs pcc detach' of an encrypted file, its page cache
pages must be trashed as they might contain cipher text because
of the pcc file being detached.

Improve sanity-pcc test_21j to exercise this use case.

Fixes: 393697cac0 ("EX-4182 sec: support of PCC-RO for encrypted files")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ice13c5b4205c074d9b46e2175d18f8743dbe9c58
Reviewed-on: https://review.whamcloud.com/47766
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15922 sec: new connect flag for name encryption

Introduce OBD_CONNECT2_ENCRYPT_NAME connection flag for compatibility
with older versions that do not support name encryption.
When server side does not have this flag, client side is forced to
null encryption for file names. And client needs to use old xattr to
store encryption context.

Also update tests in sanity-sec to exercise name encryption only if
server side supports it.

Lustre-change: https://review.whamcloud.com/47574
Lustre-commit: TBD (ec94d5cb80bd14d90920f9295cd0dadf5cf7d499)

Fixes: efea80e8b9 ("LU-13717 sec: filename encryption - digest support")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I446a4caba8e45821d701628a14c96f03cb6c4525
Reviewed-on: https://review.whamcloud.com/47827
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn49

New tag 2.14.0-ddn49

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If9d742ace566baaa8510e3614d5a0e1625e29763

LU-15910 llite: enforce ROOT default on subdir mount

In subdirectory mount, the filesystem-wide default LMV doesn't take
effect. This fix includes the following changes:
* enforce the filesystem-wide default LMV on subdirectory mount if
  it's not set separately.
* "lfs getdirstripe -D <subdir_mount>" should print the
  filesystem-wide default LMV.

Add sanity test_413g.

Lustre-change: https://review.whamcloud.com/47518
Lustre-commit: a162e24d2da5e4bd64a79d6dcc128bd0d50e8517

LU-15910 tests: skip sanity/413g for SSK

When running sanity test_413g under review-dne-selinux-ssk-part-1
it intermittently fails.  Temporarily disable this subtest for
this config until the problem is understood and fixed.

Lustre-change: https://review.whamcloud.com/47800
Lustre-commit: TBD (from 8cf4d6ff57dfd51b18997d4b98bf04d008963d70)

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I26de9d02872f0df8918b4ef0765b6b18b84794e6
Reviewed-on: https://review.whamcloud.com/47599
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15973 build: remove AC_DEFINE(__state, state, ...)

RHEL 8.6 build failed with MLNX_OFED 5.6-2.0.9.0 as follows:

  error: 'struct task_struct' has no member named 'state';
  did you mean '__state'?
    #define __state state
                    ^~~~~

The failure was introduced by commit bb7c82f13e7a and a proper
way to resolve the original issue was in commit c04adbcd7672.
So, let's remove the improper way AC_DEFINE(__state, state, ...).

Lustre-change: https://review.whamcloud.com/47799
Lustre-commit: TBD (from 66c98e324b0182e781743cab4e938b18b6ef4255)

Test-Parameters: trivial clientdistro=el8.6 testlist=sanity
Fixes: bb7c82f13e7 ("LU-15795 kernel: RHEL 8.6 server support")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Icbc897cf5870352311262af8bd59ec24ea9d7301
Reviewed-on: https://review.whamcloud.com/47804
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn48

New tag 2.14.0-ddn48

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib83800f4af36a9f2bdd79589d019870db01ccfa4

LU-15795 kernel: RHEL 8.6 server support

This patch makes changes to support RHEL 8.6 release
with kernel 4.18.0-372.9.1.el8 for Lustre server.

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Change-Id: I86407171b40d40a05379085444eee79e2ac272b3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47523
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15583 build: Update ZFS version to 2.1.2

Update ZFS version to 2.1.2. The changes are listed in:
https://github.com/openzfs/zfs/releases/tag/zfs-2.1.2

Change-Id: If7c81a4b1fe13e29eea1c277b896223f5c06b31a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47524
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15956 gss: allow build without ssk

The GSS part of Lustre should be able to build without SSK, in case
some SSK requirements are not met at configure time.

Lustre-change: https://review.whamcloud.com/47681
Lustre-commit: TBD (from a7c16ebef33097fe7596efff2a169b18f93747ba)

Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ieacbcb5db77fcc12cc13579785e640857ce7fb02
Reviewed-on: https://review.whamcloud.com/47780
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15962 build: add in-kernel Module.symvers to symbol path

After building Lustre with in-kernel OFED, installing
ko2iblnd module hit the following errors:

ko2iblnd: disagrees about version of symbol __ib_alloc_pd
ko2iblnd: Unknown symbol __ib_alloc_pd (err -22)
ko2iblnd: disagrees about version of symbol rdma_resolve_addr
ko2iblnd: Unknown symbol rdma_resolve_addr (err -22)

Those exported symbols are contained in in-kernel Module.symvers,
which should be added to the symbol path KBUILD_EXTRA_SYMBOLS.

Lustre-change: https://review.whamcloud.com/47699
Lustre-commit: TBD (from c4ba25c4c3ddbb66ddb5a5f0e2c87140a541ec3b)

Test-Parameters: trivial

Change-Id: Ic30caa7079af00a452ea24e7e982a856874af702
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47688
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15940 build: add a required dependency for libmount

The Lustre client utilities (mount/umount) have
an optional dependency on libmount to update utab.
However, libmout has been introduced in util-linux
since 2.18 in 2010. There is no need to make the
dependency as optional.

Lustre-change: https://review.whamcloud.com/47619
Lustre-commit: TBD (from 3a7e4f9ca86ebdaace07709a49fb01a0311ec82d)

Test-Parameters: trivial clientdistro=ubuntu2004
Test-Parameters: trivial clientdistro=el8.5

Change-Id: I4b965a5ce6cb6fc5d2061a53c44ef9b709ebab49
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47721
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15652 build: On Debian detect -common kernel headers

Check for a matching /usr/src/linux-headers-<ver>-common/
and update the --with-linux argument accordingly.

Also move LC_GLIBC_SUPPORT_COPY_FILE_RANGE outside
of utils as this also breaks the dkms build on Debian
with 'static' follows 'non static' declairation of
copy_file_range.

Lustre-change: https://review.whamcloud.com/46836
Lustre-commit: TBD (from 151f807d0ea970c63e7f23bd44535eb2da1d8ca1)

HPE-bug-id: LUS-10826
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I6e4f0b27eba6c5b07cda14f064e57aa9c93ae3cc
Reviewed-on: https://review.whamcloud.com/47779
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13903 build: Move GLIBC/openssl checks to where needed.

Two config checks on glibs support:
LC_GLIBC_SUPPORT_FHANDLES
LC_GLIBC_SUPPORT_COPY_FILE_RANGE
and two on openssl support:
LC_OPENSSL_SSK
LC_OPENSSL_GETSEPOL

are currently only run when modules are being built.
The FHANDLES test is needed when building tests.
The COPY_FILE_RANGE test is needed when building
utils as are the OPENSSL checks

So move the calls to these tests to a more appropriate place, so that
  ./configure --disable-modules --disable-server
can run correctly.

Lustre-change: https://review.whamcloud.com/39653
Lustre-commit: e6d1968fbbad5971b60e6c09a02d121c31d37d59

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Id7801112cd53601b3d560119784cbd062bf9610e
Reviewed-on: https://review.whamcloud.com/47778
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15758 build: Client module DEB can't be installed on Debian

Building DEB package lustre-client-modules-*.deb depends
since commit 32c4b80192652f55bcef5786e4ec683e85234c04 on
package linux-generic. This package is however available
on Ubuntu distributions, but not e.g. on Debian distribution.
As a consequence the built package lustre-client-modules-*.deb
cannot be installed on Debian due to non-existing package
dependency.

Lustre-change: https://review.whamcloud.com/47087
Lustre-commit: d409f74274754c87cd14a7f0a19f8d46f6496eb1

Test-Parameters: trivial clientdistro=ubuntu2004
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: Ia541aaede7d779c4dd2f81f896a6be8a797780b3
Reviewed-on: https://review.whamcloud.com/47777
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15596 build: set TARBALL with m-a build helper for debs

debs built using module-assistant tool by default scan
for packages and may find a random lustre tarball

Specify the correct tarball with the TARBALL
environment variable to avoid default search picking an
incorrect tarball source.

Lustre-change: https://review.whamcloud.com/46639
Lustre-commit: 47b8b6880d93d54ca1d24effb69d8c84d8d0fb2f

HPE-bug-id: LUS-10783
Test-Parameters: trivial
Fixes: 1d97ac16d8 ("LU-14948 build: Warn about /usr/src/lustre.tar.bz2")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iaa5a31aaa81e11ee97ae2ea27811c7a4399a0efa
Reviewed-on: https://review.whamcloud.com/47776
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14948 build: Warn about /usr/src/lustre.tar.bz2

When /usr/src/lustre.tar.bz2 exists, make debs (and dkms-debs)
will fail with an error like:

  Extracting the package tarball, /usr/src/lustre.tar.bz2, ...
  ../../generic.sh: line 73: debian/rules: Permission denied
  BUILD FAILED!

Add the current git hash to the lustre tarball, as well as
attempt to remove the conflict from /usr/src.  Failing that,
give a warning to ask the user to remove the conflicting file.

Lustre-change: https://review.whamcloud.com/44677
Lustre-commit: 1d97ac16d8ade4786202c2e9005896a2ac661340

HPE-bug-id: LUS-10308
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4aaa803cb81c2ed8ffc0182bb49ea0bff5064df4
Reviewed-on: https://review.whamcloud.com/47775
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15797 build: Workaround _fix_broken_configure_for_lto

Workaround _fix_broken_configure_for_lto macro added to
configure macro in RHEL9:

redhat-rpm-config.noarch  190-1.el9.alma  @appstream

Use configure macro directly if it is not parseable

Lustre-change: https://review.whamcloud.com/47168
Lustre-commit: 9b16459fbcbd6587f1a8669490352cc389a7a410

HPE-bug-id: LUS-10918
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I53072c0ffdf82abc938322e9c408cc44f5033ab9
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47774
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15521 spec: fix bare words error with rpm 4.16

RPM 4.16 removed support for bare words in expressions
(eg a == b needs to be "a" == "b" now). The change is
backward compatible. More changes are in:
https://rpm.org/wiki/Releases/4.16.0

This patch accommodates the above change and fixes more
errors/warnings:
- E: specfile-error error: bare words are no longer supported,
     please use "...":  redhat=="redhat" || redhat=="fedora"
- E: specfile-error warning: extra tokens at the end of %else
     directive in line 140:  %else #for Suse
- W: macro-in-comment %optflags
- W: macro-in-comment %{name}

Lustre-change: https://review.whamcloud.com/46471
Lustre-commit: 597a6bf9e085150dd57eb727859b114deb8979cb

Test-Parameters: trivial

Change-Id: I725c47f62be7762a89e5919fd2865e2fb2ced407
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14195 libcfs: test for nla_strscpy

During the development of the Linux 5.10 kernel the function
nla_strlcpy() was replaced by nla_strscpy(). Handle this
change for Lustre.

Lustre-change: https://review.whamcloud.com/46876
Lustre-commit: b762319d5ad0884adfb2f79c956a798fba665566

Test-parameters: trivial
Change-Id: I47f12add619cfd88a3692f0760b8bcc35b7877d9
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47769
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5249 utils: don't complain for missing lustre.pin

Don't print an error when running "lfs pcc status" against an older
server without the "lustre.pin" xattr returns EOPNOTSUPP.

Test-Parameters: trivial testlist=sanity-pcc serverversion=2.14.0-ddn23
Fixes: f9970c0a70 ("EX-5249 utils: show pin information in 'lfs pcc state' command")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iab235b8be497386752915baeb31347cf8137eed0
Reviewed-on: https://review.whamcloud.com/47723
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14724 tests: fix sanityn/77o interop version check

Check MDS version is new enough, and not OST version.

Fixes: 345f0d8e56c4 ("LU-14724 nrs: TBF rule list broken when change rule rank")
Test-Parameters: trivial testlist=sanityn serverversion=2.14.0-ddn23
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibc468d04f92c479920d6e0a424d2603c3b940e72
Reviewed-on: https://review.whamcloud.com/47726
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>

EX-5060 tests: skip sanity-lipe in test interop

Since lipe is server-only, it doesn't have any benefit to test in
interop mode, since this just exposes failures due to new tests
failing with old servers, but provides no real benefit.

If/when a client interface for lipe_find is available then this
needs to be revisited.

Test-Parameters: trivial testlist=sanity-lipe serverversion=2.14.0-ddn23
Fixes: a4c8c816a67a ("EX-4965 lipe: use new e2fsprogs xattr API")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6def018ecf177926c2785205281852c5db251903
Reviewed-on: https://review.whamcloud.com/47724
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-15317 llite: Add D_IOTRACE fixup

Add missing changes between v2..v6 of the master patch.

Test-Parameters: trivial
Fixes: 321429093c86 ("LU-15317 llite: Add D_IOTRACE")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id105bd681aa893bd0889f4567e49e57543db57fb
Reviewed-on: https://review.whamcloud.com/47703
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn47

New tag 2.14.0-ddn47

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6a29f3195e22f3589add25569adab5f1a35ab2d8

EX-5260 pumount: test pumount with MDT0000 unmounted

Add tests (40a-40c, 41a) to verify that pumount can unmount a client
mount point with MDT0000 and OST0000 unmounted. Add a test (34a) that
pumount does not kill processes who are minding their own business on
other Lustre client mount points.

Test-Parameters: testlist=sanity-pumount,sanity-pumount,sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I02b603d1c6de9f10605d3e315d5149de72698e73
Reviewed-on: https://review.whamcloud.com/47389
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15399 llite: dont restart directIO with IOCB_NOWAIT

It should hanlde FLR mirror retry and io_uring with IOCB_NOWAIT
flag differently.

int cl_io_loop(const struct lu_env *env, struct cl_io *io)
{
...
if (result == -EAGAIN && io->ci_ndelay) {
io->ci_need_restart = 1;
result = 0;
}
...
}

ssize_t
generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter)
{
...
if (iocb->ki_flags & IOCB_NOWAIT) {
if (filemap_range_has_page(mapping, iocb->ki_pos,
   iocb->ki_pos +
   count - 1))
return -EAGAIN;
...
}

In current code, it will restart I/O engine for read when get
-EAGAIN code.
However, for io_uring direct IO with IOCB_NOWAIT, if found that
there are cache pages in the current I/O range, it should return
-EAGAIN to the upper layer immediately. Otherwise, it will stuck
in an endless loop.

This patch also adds a tool "io_uring_probe" to check whether
the kernel supports io_uring fully.
The reason adding this check is because the rhel8.5 kernel has
backported io_uring:
cat /proc/kallsyms |grep io_uring
ffffffffa8510e10 W __x64_sys_io_uring_enter
ffffffffa8510e10 W __x64_sys_io_uring_register
ffffffffa8510e10 W __x64_sys_io_uring_setup
but the io_uring syscalls return -ENOSYS.

Lustre-change: https://review.whamcloud.com/46147/
Lustre-commit: TBD (from f2aeedc5b188ea00f2e05671abb2adae5fadcf6e)

Test-Parameters: clientdistro=ubuntu2004 testlist=sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Id4374382e56e90d02349676891aa57b216b3deff
Reviewed-on: https://review.whamcloud.com/45941
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15811 llite: Rework upper/lower DIO/AIO

One of the patches for LU-13799, "Implement lower/upper aio"
(https://review.whamcloud.com/44209/) created a
complicated setup where the cl_dio_aio struct was used
both for the top level DIO or AIO and for the lower level
sub I/Os (corresponding to stripes).

This is quite complicated and hard to follow, so this
rewrites these two uses to be separate structs.  This
incidentally fixes at least one possible memory leak, but
is mostly a cleanup.

Lustre-change: https://review.whamcloud.com/47187
Lustre-commit: 1256040ca0331d7e3f43b9b3534e2db2443bf28b (TBD)

Fixes: 8eba9a2ef3 "LU-13799 Implement lower/upper aio"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ide4a2b84f48624ee97dfb57fe80d201fbb7fe8d0
Reviewed-on: https://review.whamcloud.com/47188
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15721 llite: only statfs for projid if PROJINHERIT set

If projid is set on a directory but PROJINHERIT is not, do not report
the project quota for statfs.  This matches how ext4_statfs() and
xfs_fs_statfs() behave, on which Lustre project quota is modelled.

Lustre-change: https://review.whamcloud.com/47352
Lustre-commit: 4192f28733fe62173dec4a522c6589ee741df474

Fixes: e5c8f6670f ("LU-9555 quota: df should return projid-specific values")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I27cb444c3dfabc0ec693cee6fe6f9cae6db8a77a
Reviewed-on: https://review.whamcloud.com/47586
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

LU-15195 ofd: missing OST object

as the OST-MDT resync may be not finished by the end of the recovery
it may happen new enqueue for a write op may fail due to an absent
object. Return EINPROGRESS so that the enqueue was resent until get
resynced.

to not get stuck forever in case of disappeared MDT or a double
failure, return EINPROGRESS during hard failover timeout only.

also, cleanup replay-ost-single test 12:
- eliminate a need in the hard failover
- no need in a special obd_fail_loc, just use replay_barrier
- createmany is able to create files with unique names,
  no need in special steps

Lustre-change: https://review.whamcloud.com/45459
Lustre-commit: 28769c65987cb1546918fe12d6f34b95ab9c5507

HPE-bug-id: LUS-10267
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5f16b63454c51ad8d112770c15c7e6e7f41f3c40
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47686
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9243 gss: fix GSS struct definition badness

struct lgssd_ioctl_param should not be defined in multiple places. So
move it to a new header file lgss.h that can be included from kernel
space and user space.

struct gss_header, struct gss_rep_header, struct gss_err_header and
struct gss_wire_ctx are going on the wire, so they need to be moved to
lustre_idl.h, and be wire-checked.

Lustre-change: https://review.whamcloud.com/46543
Lustre-commit: 7dfbc71350c982be20d75e7d7853038809f4e850

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I97c4a8322e6bb7627c6dff5f068931278f4567d7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47667
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5366 test: add test case for lpcc service and tool

Add test case for lpcc system service and command line tool.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: I955fd3736ce3fe222afc8a90a34c7eefbe2d834e
Reviewed-on: https://review.whamcloud.com/47644
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15832 lod: clear .do_index_ops in striping free

LDLM lock can guarantee LOD object directory striping is safe to
access, but lod_striping_free_nolock() should clear .do_index_ops,
otherwise upon some failure the directory striping is freed, while a
subsequent dt_try_as_dir() skips striping initialization, and call
.do_index_ops directly, which will cause crash.

Lustre-change: https://review.whamcloud.com/47256
Lustre-commit: 3d4ff484002bf6e29655503994c240da06f838e3

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib94a4ef2f8bf5f0d34521abff77d8be46ecbf428
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15508 gss: protect from arbitrary write to init channel

In case some arbitrary data was written to the gss init channel,
directly return -EINVAL. This protects against unsolicited
authentication requests, and avoids having a dandling entry
in the auth init cache.

Lustre-change: https://review.whamcloud.com/46461
Lustre-commit: f06807d409c82dd85c208c79c8072402e77f9bac

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iadde630012e4ded83f9609fbb3e10b2e092deb57
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-on: https://review.whamcloud.com/47669
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15902 obdclass: dt_try_as_dir() check dir exists

If an object is not directory, but dt_lookup() is called on it, it
may crash because .do_lookup is NULL for non-directory file.

Add argument to check object existence and type in dt_try_as_dir(),
and for object to create, skip this check.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I51df0cbb5a4e7abca370ee27dac678f995b76159
Reviewed-on: https://review.whamcloud.com/47622
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5248 ldiskfs: add __GFP_NOWARN to ext4_kvmalloc()

Don't dump a stack trace if kmalloc() fails, since the whole point of
these functions is retry with vmalloc() if normal kmalloc() fails.

Lustre-change: https://review.whamcloud.com/47393
Lustre-commit: 9b15917634e8eb9d5ee29b1d497b389b8f452f9f

Test-Parameters: trivial testlist=sanity-lsnapshot
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id988bd84bd9bdffcb059405f1d471ef7213ebbe5
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/47630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>

LU-15200 llite: "lfs getdirstripe -D" shows inherit layout

Once system-wide default LMV is set, "lfs getdirstripe -D subdir"
should show inherited layout from it.

Add sanity 413f.

Lustre-change: https://review.whamcloud.com/45570
Lustre-commit: 61b1fad9e3fb21edcc0e713c89a3bfaeba9c883e

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If7354cb4093c58f6d56a6a4d449fb69a9deec7cc
Reviewed-on: https://review.whamcloud.com/47642
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-15200 llite: revalidate dentry if LOOKUP lock fetched

Once ll_inode_revalidate() fetches LOOKUP lock, it should revalidate
dentry, so subsequent lookup can find it in dcache.

It should also update lli_dir_depth.

Lustre-change: https://review.whamcloud.com/45599
Lustre-commit: 92fadf9cc1d06b21b482a262ff66f435814a13f8

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I847e16d64d605b91efc93925821bc29cbea20fa2
Reviewed-on: https://review.whamcloud.com/47598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5405 lipe: fix bug divided by zero

Fix 'divided by zero' bug in lpcc.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Icb8e6bd70b1292ab70968c40a34fc6c58142a20d
Reviewed-on: https://review.whamcloud.com/47677
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5406 lipe: fix some compatibility issue in lpcc

- use absolute path of program in lpcc.service
- replace yaml.load() with yaml.safe_load()

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Iad3fa408f5b19e7370c0a5b069ad7637611118a7
Reviewed-on: https://review.whamcloud.com/47678
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn46

New tag 2.14.0-ddn46

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4590039e019187be7f010f1756b8ac649318ad8d

LU-15886 lfsck: remove unreasonable assertions

Remove unreasonable assertions in LFSCK code:
* lfsck->li_obj_dir and lfsck->li_lmv may be NULL if object wan't
  initialized successfully.
* orphan objects under ldiskfs /lost+found may not exist.
* object may not be directory in lfsck_verify_lpf()->
  lfsck_verify_linkea().
* for corner case
  (leh->leh_reccount == 0 && leh->leh_overflow_time != 0),
  LASSERT(ldata->ld_leh->leh_reccount > 0) will be triggerred in
  lfsck_namespace_linkea_clear_overflow(), remove this assertion,
  and this corner case can be handled correctly in current lfsck
  code.

Lustre-change: https://review.whamcloud.com/47447/
Lustre-commit: 0c1f5aa5e20334746b22b4e215af5153de7d6f4a (TBD)

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If114c7882a2c083e83fcfac5981eddfa526d1426
Reviewed-on: https://review.whamcloud.com/47621
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15868 lfsck: don't crash upon dir migration failure

LFSCK against directories that were migrated, but failed may crash,
it's because lost+found directory may not be initialized correctly,
and this error is skipped on purpose, add check in code that
dereference it.

lfsck_verify_lpf() may dereference NULL "child2".

lmv_name_to_stripe_index() should support stripe LMV, which is used
by LFSCK to verify name hash.

Add OBD_FAIL_OUT_EIO to simulate sub transaction failure.

Add sanity-lfsck 15d to verify LFSCK won't crash upon directory
migration failure.

Update sanity-lfsck 4 and 5 to start mds1 with OI scrub enabled, and
wait for mds1 OI scrub finish, otherwise LFSCK may fail to verify
lost+found later.

Lustre-change: https://review.whamcloud.com/47381/
Lustre-commit: 1022e485b5260e3bca0e2acafaf807dffbb02c66 (TBD)

Test-Parameters: mdscount=2 mdtcount=4 testlist=sanity-lfsck env=ONLY=15d,ONLY_REPEAT=100
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I1b1872da2b4ef8f7403effc4d1d3e298c6a0b7e6
Reviewed-on: https://review.whamcloud.com/47620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15933 libcfs: fix configure check for iov_iter member

Kernel commit v5.13-rc1-12-g8cd54c1c8480 changed iov_iter member
->type to ->iter_type, add check at configure to cover this change.

Lustre-change: https://review.whamcloud.com/47601
Lustre-commit: TBD (from f6bce444082d44d3dec9db00bce63dc92092c927)

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8da26e1f6c6242a3260b458770ec34a65aecaf7d
Reviewed-on: https://review.whamcloud.com/47603
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15887 test: add always_except()

In test-framework.sh, add a new function (always_except()) to replace
manual manipulation of $ALWAYS_EXECPT. Convert sanity.sh to use
always_except() and add a line to contrib/scripts/spelling.txt to
suggest its use.

Lustre-change: https://review.whamcloud.com/47452
Lustre-commit: c4ff4aef7eb939d536acffaac4465039f3cfa935

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1b39fe9555bab59e70db00cef73d13102668500a
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15117 ofd: don't take lock for dt_bufs_get()

osd_bufs_get() allocates the pages and can cause new transactions
as part of memory release procedure. this would break Lustre's
"start a transaction, then do locking" rule.

Lustre-change: https://review.whamcloud.com/47029
Lustre-commit: TBD (from 14db15b1dcea61b1c35e00f004124501ead86ab6)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I58a52af8e2fbbc4823aafc133893e1defedf99b7
Reviewed-on: https://review.whamcloud.com/47592
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15647 utils: 'lfs df' shouldn't loop forever

Don't loop forever in "lfs df" if there is a problem fetching
the target statfs information from the kernel and -ENODEV is
not returned for whatever reason.

Lustre-change: https://review.whamcloud.com/46813
Lustre-commit: 41703e9b5fe7061b313805384b08372d4fe82cec

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5110c7246cf53eb5c4d59424a44b73b9d23ebbe5
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47605
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13562 build: get correct kernel flavor for SLES

This patch fixes lustre.spec.in to get correct kernel flavor
for SLES when kobjdir is detected as /lib/modules/%{_kver}/build.

Test-Parameters: trivial clientdistro=sles15sp3

Change-Id: I350032af383ea8b7f48accd93e5cd11c571e6620
Fixes: d746e64fe1 ("LU-13562 build: SUSE build support for azure, cray_ari_s")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47596
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-12214 build: fix SLES build/install

Redhat and SuSe can have different library name for same devel,
lets drop a strong requrement to the library package name and
ask rpm to use an autoprovide option.

Lustre-change: https://review.whamcloud.com/39972
Lustre-commit: 253a9e3bcd30bf411c57d77e137ff734eff1edb1

Test-Parameters: trivial clientdistro=sles15sp3
HPE-bug-id: LUS-7204
Fixes: e1bf37870d LU-12214 build: fix build with gss enabled
Fixes: d746e64fe1 LU-13562 build: SUSE build support for azure
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I7e0fe83f9090e7616ab156fa75fed4821099406e
Reviewed-on: https://review.whamcloud.com/47595
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15513 lod: iterate initialized stripe

Fix an array out of bounds access.

Do not iterate over the component objects in lod_fill_mirrors() to
check non-rotational OSTs if the component is unintialized or it is
a MDT or RELEASED component.

Lustre-change: https://review.whamcloud.com/47028
Lustre-commit: TBD (from b9445c05576da11cc230b290b1ec53131a0e1c2a)

Fixes: 591a990c61 ("LU-15513 lod: skip uninit component in lod_fill_mirrors")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8530cd30fab750c47f15ee9278ec6d68871367cc
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47590
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-13363 lod: do object allocation in OST pool

Currently, the ltd->ltd_qos.lq_same_space boolean that decides
whether the LOD QOS allocator is active for an allocation or not
is tracked for the entire LOV. But when a pool is specified, this
judgement should be tracked on a per-pool basis.

sanity.sh test_116c is added to verify this patch.

Lustre-change: https://review.whamcloud.com/38136
Lustre-commit: e642e75cde0248eee30ca94aaeb81653db7f8d03

Test-Parameters: ostcount=6 testlist=sanity env=ONLY=116c
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I463d5927c7a9c9171483615d2cec629ec10dc666
Reviewed-on: https://review.whamcloud.com/47334
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15952 doc: improvement on lfs-project doc

Describe 'lfs project -C' clearly.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I1e0f70a9116265edac993b12b775bd57c8587d40
Reviewed-on: https://review.whamcloud.com/47645
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14765 test: disable sanity-flr/44c

file blocks number accounting does not include blocks occupied by
multiple mirrors, disable the test for now.

Lustre-commit: a45fe93cd8d1e941d58e0f11e21649e1956ba2c7
Lustre-change: https://review.whamcloud.com/44074

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1a5d7e0c203d5c7dc0da53ce6c020795bf17b9fc
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47647
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14583 llapi: handle symlinks in llapi_file_get_stripe()

In llapi_file_get_stripe(), if the IOC_MDC_GETFILESTRIPE ioctl handler
returns -ENOTTY or -ENODATA then try to resolve any symlinks in the
path and try again.

Lustre-change: https://review.whamcloud.com/43229
Lustre-commit: cd4caef54f63b6a0d26a90d879f378b7fe3d4862

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ic046d6ef77d8342d47336144e3066cab3a940a96
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47588
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-12022 tests: error on resync failure sanity-flr

In sanity-flr test 200, we should error if the final resync
fails.  Replace all calls to 'mirror_io resync' that does
not inject an error to  '$LFS mirror resync'.

Lustre-change: https://review.whamcloud.com/35754
Lustre-commit: 9e1e29406c3886bce0db6623be40d3edf6834117

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I9b2ec1beb7060086808b7529467bef80c8e9659f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5364 tests: add sanity-flr/38 to ALWAYS_EXCEPT

Add sanity-flr.sh test_38 to ALWAYS_EXCEPT as it is failing too often.

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia27e32d7309bc3597ee4024e465959c9dbd58b4a
Reviewed-on: https://review.whamcloud.com/47610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn45

New tag 2.14.0-ddn45

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic13969b93a939490962ecc13c950d19ba19f1a76

LU-15838 autoconf: fix use of obsolete macros

This patch fixes the following warnings when using autoconf 2.71:

configure.ac:2: warning: AC_INIT: not a literal:
                "m4_esyscmd(sh -c "./LUSTRE-VERSION-GEN | tr -d '\n'")"
configure.ac:10: warning: The macro `AC_CANONICAL_SYSTEM' is obsolete.
configure.ac:16: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
configure.ac:24: warning: The macro `AC_HELP_STRING' is obsolete.

Like m4_esyscmd, macro m4_esyscmd_s (introduced in autoconf 2.64)
expands to the result of running command in a shell. The difference
is that any trailing newlines are removed.

Since autoconf 2.50, macro 'AC_CANONICAL_TARGET' has been the new name
of 'AC_CANONICAL_SYSTEM':
AU_ALIAS([AC_CANONICAL_SYSTEM], [AC_CANONICAL_TARGET])

Since autoconf 2.58, macro 'AS_HELP_STRING' has been added to replace
'AC_HELP_STRING'.

Since libtool 2.0, new 'LT_INIT' interface has been added to replace
'AC_PROG_LIBTOOL'.

Lustre-change: https://review.whamcloud.com/47288
Lustre-commit: TBD (from a94a730df48089e3ffbf853af78a2444dc5a3042)

Change-Id: I3c06c21460d7a2cf643fe825e72a26a5416609cf
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>