git://git.whamcloud.com - fs/lustre-release.git/log

LU-15901 mdc: Remove entry from list before freeing

mdc_changelog_cdev_init forgot to remove entries from list if
chardev allocation failed

Lustre-change: https://review.whamcloud.com/47480
Lustre-commit: 441ec2296a0938dd30cf60a70943eb0799d1d70c

Change-Id: Ic76b5320bf80c7f7f60c7682bda4bc37a0b300bd
Fixes: d0423abc1ad ("LU-12506 changelog: support large number of MDT")
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47783
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11695 som: disabling xattr cache for LSOM on client

To obtain uptodate LSOM data, currently a client needs to set
llite.*.xattr_cache =0 to disable the xattr cache on client
completely. This leads that other kinds of xattr can not be cached
on the client too.
This patch introduces a heavy-weight solution to disable caching
only for LSOM xattr data ("trusted.som") on client.

Lustre-change: https://review.whamcloud.com/33711/
Lustre-commit: TBD (from 55a65276b8e62f0aa894ee0719051ed8737ab59e)

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Iab5ef3030b05ac09184d01f2a3a8ed92ff1cf26b
Reviewed-on: https://review.whamcloud.com/47813
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5419 lipe: retry to get lpcc_purge stats file

Sleep and retry to get lpcc_purge stats file after sending signal
Correct statistics data algorithm.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: clientdistro=el8.5 testlist=sanity-pcc env=ONLY=210
Change-Id: I79e4b7de871f8ffd290e0c30da4780265fa7e9fa
Reviewed-on: https://review.whamcloud.com/47729
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5366 test: handle missing lpcc.conf in test_210

Handle missing /etc/lpcc.conf in sanity-pcc test_210

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial clientdistro=sles15sp3 testlist=sanity-pcc env=ONLY=210
Test-Parameters: clientdistro=el8.6 clientarch=aarch64 testlist=sanity-pcc env=ONLY=210
Change-Id: Ia19c32ac2719a52ee880a2e633495b9ce3bb9534
Reviewed-on: https://review.whamcloud.com/47728
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15855 enc: enc-unaware clients get ENOKEY if file not found

To reduce issues with applications running on clients without keys
or without fscrypt support that check for the existence of a file in
an encrypted directory, return -ENOKEY instead of -ENOENT.
For encryption-unaware clients, this is done on server side in the
mdt layer, by checking if clients have the OBD_CONNECT2_ENCRYPT
connection flag.
For clients without the key, this is done in llite when the searched
filename is not in encoded form.

Lustre-change: https://review.whamcloud.com/47349
Lustre-commit: 00898697f998c095e6dab3dd27e347cb1150ee8c

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9a3b7af3a856b7fc7222c61a308ad23168869d57
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47786
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5427 sec: fix pcc detach of encrypted file

In case of 'lfs pcc detach' of an encrypted file, its page cache
pages must be trashed as they might contain cipher text because
of the pcc file being detached.

Improve sanity-pcc test_21j to exercise this use case.

Fixes: 393697cac0 ("EX-4182 sec: support of PCC-RO for encrypted files")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ice13c5b4205c074d9b46e2175d18f8743dbe9c58
Reviewed-on: https://review.whamcloud.com/47766
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15922 sec: new connect flag for name encryption

Introduce OBD_CONNECT2_ENCRYPT_NAME connection flag for compatibility
with older versions that do not support name encryption.
When server side does not have this flag, client side is forced to
null encryption for file names. And client needs to use old xattr to
store encryption context.

Also update tests in sanity-sec to exercise name encryption only if
server side supports it.

Lustre-change: https://review.whamcloud.com/47574
Lustre-commit: TBD (ec94d5cb80bd14d90920f9295cd0dadf5cf7d499)

Fixes: efea80e8b9 ("LU-13717 sec: filename encryption - digest support")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I446a4caba8e45821d701628a14c96f03cb6c4525
Reviewed-on: https://review.whamcloud.com/47827
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn49

New tag 2.14.0-ddn49

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If9d742ace566baaa8510e3614d5a0e1625e29763

LU-15910 llite: enforce ROOT default on subdir mount

In subdirectory mount, the filesystem-wide default LMV doesn't take
effect. This fix includes the following changes:
* enforce the filesystem-wide default LMV on subdirectory mount if
  it's not set separately.
* "lfs getdirstripe -D <subdir_mount>" should print the
  filesystem-wide default LMV.

Add sanity test_413g.

Lustre-change: https://review.whamcloud.com/47518
Lustre-commit: a162e24d2da5e4bd64a79d6dcc128bd0d50e8517

LU-15910 tests: skip sanity/413g for SSK

When running sanity test_413g under review-dne-selinux-ssk-part-1
it intermittently fails.  Temporarily disable this subtest for
this config until the problem is understood and fixed.

Lustre-change: https://review.whamcloud.com/47800
Lustre-commit: TBD (from 8cf4d6ff57dfd51b18997d4b98bf04d008963d70)

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I26de9d02872f0df8918b4ef0765b6b18b84794e6
Reviewed-on: https://review.whamcloud.com/47599
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15973 build: remove AC_DEFINE(__state, state, ...)

RHEL 8.6 build failed with MLNX_OFED 5.6-2.0.9.0 as follows:

  error: 'struct task_struct' has no member named 'state';
  did you mean '__state'?
    #define __state state
                    ^~~~~

The failure was introduced by commit bb7c82f13e7a and a proper
way to resolve the original issue was in commit c04adbcd7672.
So, let's remove the improper way AC_DEFINE(__state, state, ...).

Lustre-change: https://review.whamcloud.com/47799
Lustre-commit: TBD (from 66c98e324b0182e781743cab4e938b18b6ef4255)

Test-Parameters: trivial clientdistro=el8.6 testlist=sanity
Fixes: bb7c82f13e7 ("LU-15795 kernel: RHEL 8.6 server support")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: Icbc897cf5870352311262af8bd59ec24ea9d7301
Reviewed-on: https://review.whamcloud.com/47804
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn48

New tag 2.14.0-ddn48

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib83800f4af36a9f2bdd79589d019870db01ccfa4

LU-15795 kernel: RHEL 8.6 server support

This patch makes changes to support RHEL 8.6 release
with kernel 4.18.0-372.9.1.el8 for Lustre server.

Test-Parameters: trivial fstype=ldiskfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Test-Parameters: trivial fstype=zfs \
clientdistro=el8.6 serverdistro=el8.6 testlist=sanity

Change-Id: I86407171b40d40a05379085444eee79e2ac272b3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47523
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15583 build: Update ZFS version to 2.1.2

Update ZFS version to 2.1.2. The changes are listed in:
https://github.com/openzfs/zfs/releases/tag/zfs-2.1.2

Change-Id: If7c81a4b1fe13e29eea1c277b896223f5c06b31a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47524
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15956 gss: allow build without ssk

The GSS part of Lustre should be able to build without SSK, in case
some SSK requirements are not met at configure time.

Lustre-change: https://review.whamcloud.com/47681
Lustre-commit: TBD (from a7c16ebef33097fe7596efff2a169b18f93747ba)

Test-Parameters: trivial
Test-Parameters: testgroup=review-dne-selinux-ssk-part-1
Test-Parameters: testgroup=review-dne-selinux-ssk-part-2
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ieacbcb5db77fcc12cc13579785e640857ce7fb02
Reviewed-on: https://review.whamcloud.com/47780
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15962 build: add in-kernel Module.symvers to symbol path

After building Lustre with in-kernel OFED, installing
ko2iblnd module hit the following errors:

ko2iblnd: disagrees about version of symbol __ib_alloc_pd
ko2iblnd: Unknown symbol __ib_alloc_pd (err -22)
ko2iblnd: disagrees about version of symbol rdma_resolve_addr
ko2iblnd: Unknown symbol rdma_resolve_addr (err -22)

Those exported symbols are contained in in-kernel Module.symvers,
which should be added to the symbol path KBUILD_EXTRA_SYMBOLS.

Lustre-change: https://review.whamcloud.com/47699
Lustre-commit: TBD (from c4ba25c4c3ddbb66ddb5a5f0e2c87140a541ec3b)

Test-Parameters: trivial

Change-Id: Ic30caa7079af00a452ea24e7e982a856874af702
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47688
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15940 build: add a required dependency for libmount

The Lustre client utilities (mount/umount) have
an optional dependency on libmount to update utab.
However, libmout has been introduced in util-linux
since 2.18 in 2010. There is no need to make the
dependency as optional.

Lustre-change: https://review.whamcloud.com/47619
Lustre-commit: TBD (from 3a7e4f9ca86ebdaace07709a49fb01a0311ec82d)

Test-Parameters: trivial clientdistro=ubuntu2004
Test-Parameters: trivial clientdistro=el8.5

Change-Id: I4b965a5ce6cb6fc5d2061a53c44ef9b709ebab49
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47721
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15652 build: On Debian detect -common kernel headers

Check for a matching /usr/src/linux-headers-<ver>-common/
and update the --with-linux argument accordingly.

Also move LC_GLIBC_SUPPORT_COPY_FILE_RANGE outside
of utils as this also breaks the dkms build on Debian
with 'static' follows 'non static' declairation of
copy_file_range.

Lustre-change: https://review.whamcloud.com/46836
Lustre-commit: TBD (from 151f807d0ea970c63e7f23bd44535eb2da1d8ca1)

HPE-bug-id: LUS-10826
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I6e4f0b27eba6c5b07cda14f064e57aa9c93ae3cc
Reviewed-on: https://review.whamcloud.com/47779
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-13903 build: Move GLIBC/openssl checks to where needed.

Two config checks on glibs support:
LC_GLIBC_SUPPORT_FHANDLES
LC_GLIBC_SUPPORT_COPY_FILE_RANGE
and two on openssl support:
LC_OPENSSL_SSK
LC_OPENSSL_GETSEPOL

are currently only run when modules are being built.
The FHANDLES test is needed when building tests.
The COPY_FILE_RANGE test is needed when building
utils as are the OPENSSL checks

So move the calls to these tests to a more appropriate place, so that
./configure --disable-modules --disable-server
can run correctly.

Lustre-change: https://review.whamcloud.com/39653
Lustre-commit: e6d1968fbbad5971b60e6c09a02d121c31d37d59

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Id7801112cd53601b3d560119784cbd062bf9610e
Reviewed-on: https://review.whamcloud.com/47778
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15758 build: Client module DEB can't be installed on Debian

Building DEB package lustre-client-modules-*.deb depends
since commit 32c4b80192652f55bcef5786e4ec683e85234c04 on
package linux-generic. This package is however available
on Ubuntu distributions, but not e.g. on Debian distribution.
As a consequence the built package lustre-client-modules-*.deb
cannot be installed on Debian due to non-existing package
dependency.

Lustre-change: https://review.whamcloud.com/47087
Lustre-commit: d409f74274754c87cd14a7f0a19f8d46f6496eb1

Test-Parameters: trivial clientdistro=ubuntu2004
Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: Ia541aaede7d779c4dd2f81f896a6be8a797780b3
Reviewed-on: https://review.whamcloud.com/47777
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15596 build: set TARBALL with m-a build helper for debs

debs built using module-assistant tool by default scan
for packages and may find a random lustre tarball

Specify the correct tarball with the TARBALL
environment variable to avoid default search picking an
incorrect tarball source.

Lustre-change: https://review.whamcloud.com/46639
Lustre-commit: 47b8b6880d93d54ca1d24effb69d8c84d8d0fb2f

HPE-bug-id: LUS-10783
Test-Parameters: trivial
Fixes: 1d97ac16d8 ("LU-14948 build: Warn about /usr/src/lustre.tar.bz2")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iaa5a31aaa81e11ee97ae2ea27811c7a4399a0efa
Reviewed-on: https://review.whamcloud.com/47776
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14948 build: Warn about /usr/src/lustre.tar.bz2

When /usr/src/lustre.tar.bz2 exists, make debs (and dkms-debs)
will fail with an error like:

  Extracting the package tarball, /usr/src/lustre.tar.bz2, ...
  ../../generic.sh: line 73: debian/rules: Permission denied
  BUILD FAILED!

Add the current git hash to the lustre tarball, as well as
attempt to remove the conflict from /usr/src.  Failing that,
give a warning to ask the user to remove the conflicting file.

Lustre-change: https://review.whamcloud.com/44677
Lustre-commit: 1d97ac16d8ade4786202c2e9005896a2ac661340

HPE-bug-id: LUS-10308
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I4aaa803cb81c2ed8ffc0182bb49ea0bff5064df4
Reviewed-on: https://review.whamcloud.com/47775
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15797 build: Workaround _fix_broken_configure_for_lto

Workaround _fix_broken_configure_for_lto macro added to
configure macro in RHEL9:

redhat-rpm-config.noarch 190-1.el9.alma @appstream

Use configure macro directly if it is not parseable

Lustre-change: https://review.whamcloud.com/47168
Lustre-commit: 9b16459fbcbd6587f1a8669490352cc389a7a410

HPE-bug-id: LUS-10918
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I53072c0ffdf82abc938322e9c408cc44f5033ab9
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47774
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15521 spec: fix bare words error with rpm 4.16

RPM 4.16 removed support for bare words in expressions
(eg a == b needs to be "a" == "b" now). The change is
backward compatible. More changes are in:
https://rpm.org/wiki/Releases/4.16.0

This patch accommodates the above change and fixes more
errors/warnings:
- E: specfile-error error: bare words are no longer supported,
please use "...": redhat=="redhat" || redhat=="fedora"
- E: specfile-error warning: extra tokens at the end of %else
directive in line 140: %else #for Suse
- W: macro-in-comment %optflags
- W: macro-in-comment %{name}

Lustre-change: https://review.whamcloud.com/46471
Lustre-commit: 597a6bf9e085150dd57eb727859b114deb8979cb

Test-Parameters: trivial

Change-Id: I725c47f62be7762a89e5919fd2865e2fb2ced407
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47772
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14195 libcfs: test for nla_strscpy

During the development of the Linux 5.10 kernel the function
nla_strlcpy() was replaced by nla_strscpy(). Handle this
change for Lustre.

Lustre-change: https://review.whamcloud.com/46876
Lustre-commit: b762319d5ad0884adfb2f79c956a798fba665566

Test-parameters: trivial
Change-Id: I47f12add619cfd88a3692f0760b8bcc35b7877d9
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47769
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5249 utils: don't complain for missing lustre.pin

Don't print an error when running "lfs pcc status" against an older
server without the "lustre.pin" xattr returns EOPNOTSUPP.

Test-Parameters: trivial testlist=sanity-pcc serverversion=2.14.0-ddn23
Fixes: f9970c0a70 ("EX-5249 utils: show pin information in 'lfs pcc state' command")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iab235b8be497386752915baeb31347cf8137eed0
Reviewed-on: https://review.whamcloud.com/47723
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14724 tests: fix sanityn/77o interop version check

Check MDS version is new enough, and not OST version.

Fixes: 345f0d8e56c4 ("LU-14724 nrs: TBF rule list broken when change rule rank")
Test-Parameters: trivial testlist=sanityn serverversion=2.14.0-ddn23
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibc468d04f92c479920d6e0a424d2603c3b940e72
Reviewed-on: https://review.whamcloud.com/47726
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>

EX-5060 tests: skip sanity-lipe in test interop

Since lipe is server-only, it doesn't have any benefit to test in
interop mode, since this just exposes failures due to new tests
failing with old servers, but provides no real benefit.

If/when a client interface for lipe_find is available then this
needs to be revisited.

Test-Parameters: trivial testlist=sanity-lipe serverversion=2.14.0-ddn23
Fixes: a4c8c816a67a ("EX-4965 lipe: use new e2fsprogs xattr API")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6def018ecf177926c2785205281852c5db251903
Reviewed-on: https://review.whamcloud.com/47724
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>

LU-15317 llite: Add D_IOTRACE fixup

Add missing changes between v2..v6 of the master patch.

Test-Parameters: trivial
Fixes: 321429093c86 ("LU-15317 llite: Add D_IOTRACE")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id105bd681aa893bd0889f4567e49e57543db57fb
Reviewed-on: https://review.whamcloud.com/47703
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn47

New tag 2.14.0-ddn47

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6a29f3195e22f3589add25569adab5f1a35ab2d8

EX-5260 pumount: test pumount with MDT0000 unmounted

Add tests (40a-40c, 41a) to verify that pumount can unmount a client
mount point with MDT0000 and OST0000 unmounted. Add a test (34a) that
pumount does not kill processes who are minding their own business on
other Lustre client mount points.

Test-Parameters: testlist=sanity-pumount,sanity-pumount,sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I02b603d1c6de9f10605d3e315d5149de72698e73
Reviewed-on: https://review.whamcloud.com/47389
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15399 llite: dont restart directIO with IOCB_NOWAIT

It should hanlde FLR mirror retry and io_uring with IOCB_NOWAIT
flag differently.

int cl_io_loop(const struct lu_env *env, struct cl_io *io)
{
...
if (result == -EAGAIN && io->ci_ndelay) {
io->ci_need_restart = 1;
result = 0;
}
...
}

ssize_t
generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter)
{
...
if (iocb->ki_flags & IOCB_NOWAIT) {
if (filemap_range_has_page(mapping, iocb->ki_pos,
iocb->ki_pos +
count - 1))
return -EAGAIN;
...
}

In current code, it will restart I/O engine for read when get
-EAGAIN code.
However, for io_uring direct IO with IOCB_NOWAIT, if found that
there are cache pages in the current I/O range, it should return
-EAGAIN to the upper layer immediately. Otherwise, it will stuck
in an endless loop.

This patch also adds a tool "io_uring_probe" to check whether
the kernel supports io_uring fully.
The reason adding this check is because the rhel8.5 kernel has
backported io_uring:
cat /proc/kallsyms |grep io_uring
ffffffffa8510e10 W __x64_sys_io_uring_enter
ffffffffa8510e10 W __x64_sys_io_uring_register
ffffffffa8510e10 W __x64_sys_io_uring_setup
but the io_uring syscalls return -ENOSYS.

Lustre-change: https://review.whamcloud.com/46147/
Lustre-commit: TBD (from f2aeedc5b188ea00f2e05671abb2adae5fadcf6e)

Test-Parameters: clientdistro=ubuntu2004 testlist=sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Id4374382e56e90d02349676891aa57b216b3deff
Reviewed-on: https://review.whamcloud.com/45941
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15811 llite: Rework upper/lower DIO/AIO

One of the patches for LU-13799, "Implement lower/upper aio"
(https://review.whamcloud.com/44209/) created a
complicated setup where the cl_dio_aio struct was used
both for the top level DIO or AIO and for the lower level
sub I/Os (corresponding to stripes).

This is quite complicated and hard to follow, so this
rewrites these two uses to be separate structs. This
incidentally fixes at least one possible memory leak, but
is mostly a cleanup.

Lustre-change: https://review.whamcloud.com/47187
Lustre-commit: 1256040ca0331d7e3f43b9b3534e2db2443bf28b (TBD)

Fixes: 8eba9a2ef3 "LU-13799 Implement lower/upper aio"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ide4a2b84f48624ee97dfb57fe80d201fbb7fe8d0
Reviewed-on: https://review.whamcloud.com/47188
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15721 llite: only statfs for projid if PROJINHERIT set

If projid is set on a directory but PROJINHERIT is not, do not report
the project quota for statfs. This matches how ext4_statfs() and
xfs_fs_statfs() behave, on which Lustre project quota is modelled.

Lustre-change: https://review.whamcloud.com/47352
Lustre-commit: 4192f28733fe62173dec4a522c6589ee741df474

Fixes: e5c8f6670f ("LU-9555 quota: df should return projid-specific values")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I27cb444c3dfabc0ec693cee6fe6f9cae6db8a77a
Reviewed-on: https://review.whamcloud.com/47586
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

LU-15195 ofd: missing OST object

as the OST-MDT resync may be not finished by the end of the recovery
it may happen new enqueue for a write op may fail due to an absent
object. Return EINPROGRESS so that the enqueue was resent until get
resynced.

to not get stuck forever in case of disappeared MDT or a double
failure, return EINPROGRESS during hard failover timeout only.

also, cleanup replay-ost-single test 12:
- eliminate a need in the hard failover
- no need in a special obd_fail_loc, just use replay_barrier
- createmany is able to create files with unique names,
no need in special steps

Lustre-change: https://review.whamcloud.com/45459
Lustre-commit: 28769c65987cb1546918fe12d6f34b95ab9c5507

HPE-bug-id: LUS-10267
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5f16b63454c51ad8d112770c15c7e6e7f41f3c40
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47686
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9243 gss: fix GSS struct definition badness

struct lgssd_ioctl_param should not be defined in multiple places. So
move it to a new header file lgss.h that can be included from kernel
space and user space.

struct gss_header, struct gss_rep_header, struct gss_err_header and
struct gss_wire_ctx are going on the wire, so they need to be moved to
lustre_idl.h, and be wire-checked.

Lustre-change: https://review.whamcloud.com/46543
Lustre-commit: 7dfbc71350c982be20d75e7d7853038809f4e850

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I97c4a8322e6bb7627c6dff5f068931278f4567d7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47667
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5366 test: add test case for lpcc service and tool

Add test case for lpcc system service and command line tool.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: I955fd3736ce3fe222afc8a90a34c7eefbe2d834e
Reviewed-on: https://review.whamcloud.com/47644
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15832 lod: clear .do_index_ops in striping free

LDLM lock can guarantee LOD object directory striping is safe to
access, but lod_striping_free_nolock() should clear .do_index_ops,
otherwise upon some failure the directory striping is freed, while a
subsequent dt_try_as_dir() skips striping initialization, and call
.do_index_ops directly, which will cause crash.

Lustre-change: https://review.whamcloud.com/47256
Lustre-commit: 3d4ff484002bf6e29655503994c240da06f838e3

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib94a4ef2f8bf5f0d34521abff77d8be46ecbf428
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15508 gss: protect from arbitrary write to init channel

In case some arbitrary data was written to the gss init channel,
directly return -EINVAL. This protects against unsolicited
authentication requests, and avoids having a dandling entry
in the auth init cache.

Lustre-change: https://review.whamcloud.com/46461
Lustre-commit: f06807d409c82dd85c208c79c8072402e77f9bac

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iadde630012e4ded83f9609fbb3e10b2e092deb57
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-on: https://review.whamcloud.com/47669
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15902 obdclass: dt_try_as_dir() check dir exists

If an object is not directory, but dt_lookup() is called on it, it
may crash because .do_lookup is NULL for non-directory file.

Add argument to check object existence and type in dt_try_as_dir(),
and for object to create, skip this check.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I51df0cbb5a4e7abca370ee27dac678f995b76159
Reviewed-on: https://review.whamcloud.com/47622
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5248 ldiskfs: add __GFP_NOWARN to ext4_kvmalloc()

Don't dump a stack trace if kmalloc() fails, since the whole point of
these functions is retry with vmalloc() if normal kmalloc() fails.

Lustre-change: https://review.whamcloud.com/47393
Lustre-commit: 9b15917634e8eb9d5ee29b1d497b389b8f452f9f

Test-Parameters: trivial testlist=sanity-lsnapshot
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id988bd84bd9bdffcb059405f1d471ef7213ebbe5
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/47630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>

LU-15200 llite: "lfs getdirstripe -D" shows inherit layout

Once system-wide default LMV is set, "lfs getdirstripe -D subdir"
should show inherited layout from it.

Add sanity 413f.

Lustre-change: https://review.whamcloud.com/45570
Lustre-commit: 61b1fad9e3fb21edcc0e713c89a3bfaeba9c883e

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If7354cb4093c58f6d56a6a4d449fb69a9deec7cc
Reviewed-on: https://review.whamcloud.com/47642
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-15200 llite: revalidate dentry if LOOKUP lock fetched

Once ll_inode_revalidate() fetches LOOKUP lock, it should revalidate
dentry, so subsequent lookup can find it in dcache.

It should also update lli_dir_depth.

Lustre-change: https://review.whamcloud.com/45599
Lustre-commit: 92fadf9cc1d06b21b482a262ff66f435814a13f8

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I847e16d64d605b91efc93925821bc29cbea20fa2
Reviewed-on: https://review.whamcloud.com/47598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5405 lipe: fix bug divided by zero

Fix 'divided by zero' bug in lpcc.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Icb8e6bd70b1292ab70968c40a34fc6c58142a20d
Reviewed-on: https://review.whamcloud.com/47677
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5406 lipe: fix some compatibility issue in lpcc

- use absolute path of program in lpcc.service
- replace yaml.load() with yaml.safe_load()

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Iad3fa408f5b19e7370c0a5b069ad7637611118a7
Reviewed-on: https://review.whamcloud.com/47678
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn46

New tag 2.14.0-ddn46

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4590039e019187be7f010f1756b8ac649318ad8d

LU-15886 lfsck: remove unreasonable assertions

Remove unreasonable assertions in LFSCK code:
* lfsck->li_obj_dir and lfsck->li_lmv may be NULL if object wan't
  initialized successfully.
* orphan objects under ldiskfs /lost+found may not exist.
* object may not be directory in lfsck_verify_lpf()->
  lfsck_verify_linkea().
* for corner case
  (leh->leh_reccount == 0 && leh->leh_overflow_time != 0),
  LASSERT(ldata->ld_leh->leh_reccount > 0) will be triggerred in
  lfsck_namespace_linkea_clear_overflow(), remove this assertion,
  and this corner case can be handled correctly in current lfsck
  code.

Lustre-change: https://review.whamcloud.com/47447/
Lustre-commit: 0c1f5aa5e20334746b22b4e215af5153de7d6f4a (TBD)

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If114c7882a2c083e83fcfac5981eddfa526d1426
Reviewed-on: https://review.whamcloud.com/47621
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15868 lfsck: don't crash upon dir migration failure

LFSCK against directories that were migrated, but failed may crash,
it's because lost+found directory may not be initialized correctly,
and this error is skipped on purpose, add check in code that
dereference it.

lfsck_verify_lpf() may dereference NULL "child2".

lmv_name_to_stripe_index() should support stripe LMV, which is used
by LFSCK to verify name hash.

Add OBD_FAIL_OUT_EIO to simulate sub transaction failure.

Add sanity-lfsck 15d to verify LFSCK won't crash upon directory
migration failure.

Update sanity-lfsck 4 and 5 to start mds1 with OI scrub enabled, and
wait for mds1 OI scrub finish, otherwise LFSCK may fail to verify
lost+found later.

Lustre-change: https://review.whamcloud.com/47381/
Lustre-commit: 1022e485b5260e3bca0e2acafaf807dffbb02c66 (TBD)

Test-Parameters: mdscount=2 mdtcount=4 testlist=sanity-lfsck env=ONLY=15d,ONLY_REPEAT=100
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I1b1872da2b4ef8f7403effc4d1d3e298c6a0b7e6
Reviewed-on: https://review.whamcloud.com/47620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15933 libcfs: fix configure check for iov_iter member

Kernel commit v5.13-rc1-12-g8cd54c1c8480 changed iov_iter member
->type to ->iter_type, add check at configure to cover this change.

Lustre-change: https://review.whamcloud.com/47601
Lustre-commit: TBD (from f6bce444082d44d3dec9db00bce63dc92092c927)

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8da26e1f6c6242a3260b458770ec34a65aecaf7d
Reviewed-on: https://review.whamcloud.com/47603
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15887 test: add always_except()

In test-framework.sh, add a new function (always_except()) to replace
manual manipulation of $ALWAYS_EXECPT. Convert sanity.sh to use
always_except() and add a line to contrib/scripts/spelling.txt to
suggest its use.

Lustre-change: https://review.whamcloud.com/47452
Lustre-commit: c4ff4aef7eb939d536acffaac4465039f3cfa935

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1b39fe9555bab59e70db00cef73d13102668500a
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15117 ofd: don't take lock for dt_bufs_get()

osd_bufs_get() allocates the pages and can cause new transactions
as part of memory release procedure. this would break Lustre's
"start a transaction, then do locking" rule.

Lustre-change: https://review.whamcloud.com/47029
Lustre-commit: TBD (from 14db15b1dcea61b1c35e00f004124501ead86ab6)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I58a52af8e2fbbc4823aafc133893e1defedf99b7
Reviewed-on: https://review.whamcloud.com/47592
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15647 utils: 'lfs df' shouldn't loop forever

Don't loop forever in "lfs df" if there is a problem fetching
the target statfs information from the kernel and -ENODEV is
not returned for whatever reason.

Lustre-change: https://review.whamcloud.com/46813
Lustre-commit: 41703e9b5fe7061b313805384b08372d4fe82cec

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5110c7246cf53eb5c4d59424a44b73b9d23ebbe5
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47605
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13562 build: get correct kernel flavor for SLES

This patch fixes lustre.spec.in to get correct kernel flavor
for SLES when kobjdir is detected as /lib/modules/%{_kver}/build.

Test-Parameters: trivial clientdistro=sles15sp3

Change-Id: I350032af383ea8b7f48accd93e5cd11c571e6620
Fixes: d746e64fe1 ("LU-13562 build: SUSE build support for azure, cray_ari_s")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47596
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-12214 build: fix SLES build/install

Redhat and SuSe can have different library name for same devel,
lets drop a strong requrement to the library package name and
ask rpm to use an autoprovide option.

Lustre-change: https://review.whamcloud.com/39972
Lustre-commit: 253a9e3bcd30bf411c57d77e137ff734eff1edb1

Test-Parameters: trivial clientdistro=sles15sp3
HPE-bug-id: LUS-7204
Fixes: e1bf37870d LU-12214 build: fix build with gss enabled
Fixes: d746e64fe1 LU-13562 build: SUSE build support for azure
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I7e0fe83f9090e7616ab156fa75fed4821099406e
Reviewed-on: https://review.whamcloud.com/47595
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15513 lod: iterate initialized stripe

Fix an array out of bounds access.

Do not iterate over the component objects in lod_fill_mirrors() to
check non-rotational OSTs if the component is unintialized or it is
a MDT or RELEASED component.

Lustre-change: https://review.whamcloud.com/47028
Lustre-commit: TBD (from b9445c05576da11cc230b290b1ec53131a0e1c2a)

Fixes: 591a990c61 ("LU-15513 lod: skip uninit component in lod_fill_mirrors")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8530cd30fab750c47f15ee9278ec6d68871367cc
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47590
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-13363 lod: do object allocation in OST pool

Currently, the ltd->ltd_qos.lq_same_space boolean that decides
whether the LOD QOS allocator is active for an allocation or not
is tracked for the entire LOV. But when a pool is specified, this
judgement should be tracked on a per-pool basis.

sanity.sh test_116c is added to verify this patch.

Lustre-change: https://review.whamcloud.com/38136
Lustre-commit: e642e75cde0248eee30ca94aaeb81653db7f8d03

Test-Parameters: ostcount=6 testlist=sanity env=ONLY=116c
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I463d5927c7a9c9171483615d2cec629ec10dc666
Reviewed-on: https://review.whamcloud.com/47334
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15952 doc: improvement on lfs-project doc

Describe 'lfs project -C' clearly.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I1e0f70a9116265edac993b12b775bd57c8587d40
Reviewed-on: https://review.whamcloud.com/47645
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14765 test: disable sanity-flr/44c

file blocks number accounting does not include blocks occupied by
multiple mirrors, disable the test for now.

Lustre-commit: a45fe93cd8d1e941d58e0f11e21649e1956ba2c7
Lustre-change: https://review.whamcloud.com/44074

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1a5d7e0c203d5c7dc0da53ce6c020795bf17b9fc
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47647
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14583 llapi: handle symlinks in llapi_file_get_stripe()

In llapi_file_get_stripe(), if the IOC_MDC_GETFILESTRIPE ioctl handler
returns -ENOTTY or -ENODATA then try to resolve any symlinks in the
path and try again.

Lustre-change: https://review.whamcloud.com/43229
Lustre-commit: cd4caef54f63b6a0d26a90d879f378b7fe3d4862

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ic046d6ef77d8342d47336144e3066cab3a940a96
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47588
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-12022 tests: error on resync failure sanity-flr

In sanity-flr test 200, we should error if the final resync
fails. Replace all calls to 'mirror_io resync' that does
not inject an error to '$LFS mirror resync'.

Lustre-change: https://review.whamcloud.com/35754
Lustre-commit: 9e1e29406c3886bce0db6623be40d3edf6834117

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I9b2ec1beb7060086808b7529467bef80c8e9659f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5364 tests: add sanity-flr/38 to ALWAYS_EXCEPT

Add sanity-flr.sh test_38 to ALWAYS_EXCEPT as it is failing too often.

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia27e32d7309bc3597ee4024e465959c9dbd58b4a
Reviewed-on: https://review.whamcloud.com/47610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn45

New tag 2.14.0-ddn45

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic13969b93a939490962ecc13c950d19ba19f1a76

LU-15838 autoconf: fix use of obsolete macros

This patch fixes the following warnings when using autoconf 2.71:

configure.ac:2: warning: AC_INIT: not a literal:
"m4_esyscmd(sh -c "./LUSTRE-VERSION-GEN | tr -d '\n'")"
configure.ac:10: warning: The macro `AC_CANONICAL_SYSTEM' is obsolete.
configure.ac:16: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
configure.ac:24: warning: The macro `AC_HELP_STRING' is obsolete.

Like m4_esyscmd, macro m4_esyscmd_s (introduced in autoconf 2.64)
expands to the result of running command in a shell. The difference
is that any trailing newlines are removed.

Since autoconf 2.50, macro 'AC_CANONICAL_TARGET' has been the new name
of 'AC_CANONICAL_SYSTEM':
AU_ALIAS([AC_CANONICAL_SYSTEM], [AC_CANONICAL_TARGET])

Since autoconf 2.58, macro 'AS_HELP_STRING' has been added to replace
'AC_HELP_STRING'.

Since libtool 2.0, new 'LT_INIT' interface has been added to replace
'AC_PROG_LIBTOOL'.

Lustre-change: https://review.whamcloud.com/47288
Lustre-commit: TBD (from a94a730df48089e3ffbf853af78a2444dc5a3042)

Change-Id: I3c06c21460d7a2cf643fe825e72a26a5416609cf
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5347 lipe: update lpcc to python3

Update the lpcc utility to use python 3.

Test-Parameters: trivial clientdistro=ubuntu2204 env=SANITY_PCC_EXCEPT="15 16 27 39 101a 201" testlist=sanity-pcc
Test-Parameters: trivial clientdistro=ubuntu2004 env=SANITY_PCC_EXCEPT="101a" testlist=sanity-pcc
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I690ef4362cbdd0118419ab37a15127619a860178
Reviewed-on: https://review.whamcloud.com/47553
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5360 lipe: lpcc handles unmounted lustre fs

If lustre fs is unmounted, lpcc daemon crashes while serving
'lpcc status' command. Fix it.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-lipe
Change-Id: I5304745aed001fbe26b77ed3cebfaabfddfce11f
Reviewed-on: https://review.whamcloud.com/47572
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

EX-5045 lipe: use LDADD

Use foo_LDADD rather than foo_LDFLAGS to avoid future errors with
strict linkers. Remove AC_SEARCH_LIBS() uses for ext2fs and com_err,
since we already use PKG_CHECK_MODULES() to check for ext2fs
libraries, and not all lipe utilities need -lext2fs.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1de960fcec5a8770984d9b1437423a854948eb84
Reviewed-on: https://review.whamcloud.com/47443
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15900 hsm: don't return error on state change during mount

HSM coordinator is started in stopped state always, but
mount may have hsm_control parameters 'disabled'. Such
parameter cause wrong state change so mount would fail
with error.

Treat parameter change from 'stopping/stopped' to 'disabled'
as not critical error, keep state unchanged and report
no error back to a caller.

Lustre-change: https://review.whamcloud.com/47473
Lustre-commit: TBD (from c3fc6fef304d052da011c5840134af197eaefd94)

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I9d1366423391971b9511c46b6aed39d21ebf637c
Reviewed-on: https://review.whamcloud.com/47474
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15484 tests: Increase timeout to wait for child

Sanity-pcc 101a is failing because the remote 'sleep 600'
process is not always started in time, meaning the NS
doesn't exist.

Increase the wait time for this process to start.

Lustre-change: https://review.whamcloud.com/46764
Lustre-commit: TBD (20f794e7e8b736a944c9e1e87970e7232abd6a19)

Fixes: f3be560031 "LU-15170 llite: Switch pcc to lookup_one_len"

Test-Parameters: trivial
Test-Parameters: testlist=sanity-pcc env="ONLY=101a,ONLY_REPEAT=50"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I57175ec2f0e948045cf88d4a13a1e56cc34f8407
Reviewed-on: https://review.whamcloud.com/47557
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14785 build: changelog updates should not dirty version

When building lustre debs the final version should not
include 'dirty' due to an update of the changelog

Lustre-change: https://review.whamcloud.com/44069
Lustre-commit: a91839eb57103e6fbf5a3f6bacfcc0c471620057

HPE-bug-id: LUS-10152
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I59f4da4b3006302e3598cfa56a0364b052f885ef
Reviewed-on: https://review.whamcloud.com/47535
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5347 build: update libssl3

In Ubuntu 22.04 libssl1.1 has been superceded by libssl3

Test-Parameters: trivial clientdistro=ubuntu2004

Change-Id: Ic2504c3a40d5c756c2da7b81edb1501fb6b44712
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47556
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15918 build: update debhelper compatibility level to 11

This patch updates debhelper compatibility level to 11 (which is
currently the recommended level) to resolve the following build
warnings while running "make debs":

Compatibility levels before 10 are deprecated (level 9 in use)

Test-Parameters: trivial clientdistro=ubuntu2004 \
testlist=sanity

Change-Id: Ie5c63f6a3eb29e9aeace1f008d8d492e7587adf2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47551
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15573 build: remove mpi, CC deps from config cache

Also drop mpi and CC dependencies compiler type from the
initial values in the configure cache as these may be
changed during the setup in the rpm spec or debian build rules

Lustre-change: https://review.whamcloud.com/46562
Lustre-commit: TBD (from 497cea4815a3e08f3ffd88ecba0c129a9e9ca5b8)

Test-Parameters: trivial
Fixes: a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I986c2ae3653deae08b9da8d64d0b3c02fdc8fa2b
Reviewed-on: https://review.whamcloud.com/47536
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15908 build: fix debian rules

A comment in debian/rules file breaks a commands-line continuation,
causing exported env vars to be dropped.

Lustre-change: https://review.whamcloud.com/47512
Lustre-commit: TBD (from d1d5b35306bc164afe444164a40fdd3940a2abcd)

Fixes: a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
Test-Parameters: trivial
Signed-off-by: Ake Sandgren <ake.sandgren@hpc2n.umu.se>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0894199301c489b1961e043651c74d7c6c0089d2
Reviewed-on: https://review.whamcloud.com/47537
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15796 build: require python2 for lustre-iokit deb

Package python is not available on Ubuntu 22.04.
We can just require python2 in the control files.

Lustre-change: https://review.whamcloud.com/47164
Lustre-commit: TBD (from d46e59a541a4a2db73ccae95707f08d5d9c40ccd)

Test-Parameters: trivial clientdistro=ubuntu2004 \
testlist=obdfilter-survey

Change-Id: Id37c9babba31c575855238592cbac4ce5c9ee79a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47538
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15420 build: fixes to support building on Ubuntu 22.04 LTS

Lustre uses the glibc stdarg.h instead of the kernel's version which
causes the following build issue.

lustre/include/lu_object.h:35,
/usr/lib/gcc/x86_64-linux-gnu/11/include/stdarg.h:52: note: this is the
location of the previous definition
#define va_copy(d,s) __builtin_va_copy(d,s)

The solution is to use the kernels version of stdarg.h

The second build issue :
update_trans.c:1608:30: error: ‘struct task_struct’ has no member named
‘state’; did you mean ‘__state’?

is due Linux commit 2f064a59a11ff9bc22e52e9678bc601404c7cb34
(sched: Change task_struct::state). The state field was
changed and the barrier macros READ_ONCE()/WRITE_ONCE()
are used to access it now which is the proper thing to do.
Since the check in update_trans.c is equivalent to testing
if the kernel thread is not running, since TASK_RUNNING == 0,
we can just change the code to use task_is_running(). The
task_is_running() was introduced in 5.13.

Lustre-change: https://review.whamcloud.com/47133
Lustre-commit: TBD (from 6b096f55bad9bd169b2aa01237764cf97b58ee56)

Test-Parameters: trivial
Change-Id: Ib5985b187c3013fbc513e9962a5f27bed4996f5b
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47540
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10632 tests: recovery-small test_26 idle_timeout

In recovery-small test_26() use "lfs df" instead of plain "df"
since statfs may be fetched from the MDS cache and will not
ensure that the client->OST connections are currently active.

Also, check a few entries further back in the OSC state log for an
EVICTED message, in case the client idle disconnects from the server
again while checking all of the imports.

Lustre-change: https://review.whamcloud.com/42006
Lustre-commit: b4391fcdaf392a50bd1419342eca3b730c077ed2

Test-Parameters: trivial testlist=recovery-small env=ONLY=26a,ONLY_REPEAT=100
Fixes: 5a6ceb664f07 ("LU-7236 ptlrpc: idle connections can disconnect")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8c370cb75f4e06258ef3c032630fc20354a15dcc
Reviewed-on: https://review.whamcloud.com/45296
Tested-by: jenkins <devops@whamcloud.com>

LU-15220 lipe: avoid gcc-11 -Werror=stringop-truncation warning

This patch fixes -Werror=stringop-truncation warnings
in lipe codes.

Test-Parameters: trivial testlist=sanity-lipe

Change-Id: Ie007083b1fe94d81e3772d7c0947457d2c3042ac
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47510
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn44

New tag 2.14.0-ddn44

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia5ec26041bac7cc62fcef79fd7c8ff2d5ca170da

LU-15896 gss: ignore OpenSSLv3 deprecated API

Lustre GSS code makes use of some OpenSSL API that has been
deprecated in v3, namely all the functions in the DH_* family.
Those would need to be replaced with their EVP_PKEY_* counterparts
but in the meantime we just use the OPENSSL_SUPPRESS_DEPRECATED
define to continue using the deprecated API.

Lustre-change: https://review.whamcloud.com/47485
Lustre-commit: TBD (from 05d5b73ebb253c0e3ed5a7954b751ebbdb2d61bf)

Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I413a2af1299123b29ed2043188da0348ce9d0a5e
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47496
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15776 tgt: fix transaction handling in tgt_brw_write()

Hotfix to prevent possible data loss during WRITE replay.
Since commit f0f92773ee18 from LU-14187 the obd_commitrw()
may restart write transaction in OFD and MDT. That causes
transaction number to be assigned multiple times if such
restart happens. Without flag tti_mult_trans the first
transaction number is stored only so later one could remain
not applied causing data loss after recovery.

Patch sets tti_mult_trans for tgt_brw_write() so the latest
transaction number will be used as request transno.

Lustre-change: https://review.whamcloud.com/47371
Lustre-commit: TBD (from ccce4f23c773d9b9cc98cb15ed31861803167bb9)

Fixes: f0f92773ee ("LU-14187 osd-ldiskfs: fix locking in write commit")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I364b478591942be5562c3e98ee6e6aa487f3e0c5
Reviewed-on: https://review.whamcloud.com/47428
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15840 lov: return st_blocks=1 for HSM released files

The MDT will return st_blocks=1 for a HSM released file.
In the call ->coo_attr_get in LOV layer, the client should also
return st_blocks=1 for a HSM released file.

Otherwise, the client may get 0 block count. It is very easy to
reproduce this problem via the following commands for a archived
file:
# $LFS hsm_restore $file
# $LFS hsm_release $file
# $LFS hsm_release $file
After release a file twice, the reported block count via stat()
call will become 0.

Lustre-change: https://review.whamcloud.com/47291
Lustre-commit: TBD (from 974320c61ee7d165073bb2810795575261996dc7)

Change-Id: Id1841147e40a7df0ca615e887f324cff8e613f11
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47290
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5249 utils: show pin information in 'lfs pcc state' command

If xattr lustre.pin exists, show its content in 'lfs pcc state'
command.

Change-Id: I867ea3b246fc3b7d10e166d754c081c8afc462d5
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/47287
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15841 lod: iterate component to collect avoid array

In newly created file, the mirror information hasn't been established
as LOD is trying allocate OST for its components, so we need to
iterate component instead of mirror to collect the avoid guidance
information.

Lustre-change: https://review.whamcloud.com/47293
Lustre-commit: TBD (from d04b04056e45edc95117f0761359ddc4637d3f37)

Test-Parameters: testlist=sanity-flr env=ONLY=47,ONLY_REPEAT=40
Fixes: fabf3fe7 ("LU-9007 lod: improve obj alloc for FLR file")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I6bbe5f6b6dfea06c5213b77b7ebb6a5d28aa0d17
Reviewed-on: https://review.whamcloud.com/47295
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5176 tests: re-enable sanity test_400a

Re-enable test_400a after it was disabled due to json-c dependency
during build test. This library is now installed on client nodes.

Test-Parameters: trivial testlist=sanity env=ONLY=400 clientdistro=sles15sp3
Test-Parameters: testlist=sanity env=ONLY=400 clientdistro=ubuntu2004
Test-Parameters: testlist=sanity env=ONLY=400 clientdistro=el8.5
Fixes: fbfd2d0755 ("EX-5176 pcc: use JSON string for trusted.pin xattr")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3daca219cedb447127f9402a79e6124afb3ebbe5
Reviewed-on: https://review.whamcloud.com/47363
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>

LU-15848 ldiskfs: escape encrypted file names

When a Lustre MDT is mounted as ldiskfs, the names of the encrypted
files have to be escaped to avoid breaking the shell.
On CentOS 7, the LDISKFS_ENCRYPT_FL flag does not exist. So we add it,
and when the target is mounted as ldiskfs (LDISKFS_MOUNT_DIRDATA flag
not present) we critical-encode encrypted file names before
presentation. And conversely, we critical-decode names upon lookup.
On CentOS 8, the LDISKFS_ENCRYPT_FL flag exists. The fscrypt functions
from kernel 4.18 are also wired up, but they all refer to -EOPNOTSUPP
or equivalent, so they cannot be used to present usable names. So when
the target is mounted as ldiskfs (LDISKFS_MOUNT_DIRDATA flag not
present) we proceed to critical-encoding of encrypted file names
before presentation, and to critical-decoding upon lookup.

Lustre-change: https://review.whamcloud.com/47309
Lustre-commit: fa7c81e81442c8d1ed0ce1c457769f2e144ff2c3

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaca467eaa233be8142356efa822962953754c2ce
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47482
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15706 llog: deal with "SKIP" pool llog records correctly

If the requested "start" llog records by one ioctl are just the
useful ones "protected" by one marker, they will don't know their
cfg_flags(e.g. "SKIP" or not), and then will be mis-labeled in
class_config_yaml_output().
To fix this issue, this patch does the following changes:
- In kernel space, remember the marker cfg_flags for 10 records
  earlier than the requested "start" in function llog_print_cb(),
  so that the "start" can be output with its correct cfg_flags.
- In user space, since the pool_new/add record will be marked as
  "SKIP" if its corresponding pool_destroy/remove record is logged
  later in mgs_pool_cmd(), these "SKIP" records won't be printed
  by function jt_llog_print_iter(), so lpd_ost_num doesn't need to
  be decreased in callback function llog_search_pool_cb() as well,
  otherwise, it will cause "lctl pool_destroy" error.

Lustre-change: https://review.whamcloud.com/46951/

Test-Parameters: standalonemgs=true testlist=ost-pools
Test-Parameters: standalonemgs=true testlist=conf-sanity env=ONLY=123,HONOR_EXCEPT=y
Fixes: 2a5b50d20717 ("LU-15142 lctl: fixes for set_param -P and llog_print")
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ie45127ac8b80a75eaeb7158559c690da52eef103
Reviewed-on: https://review.whamcloud.com/47463
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15800 ofd: take a read lock for fallocate

there is no need to take an write (exclusive) object's
lock for fallocate - we just need to serialize fallocate
vs destroy, all internal structures should be protected
by OSD and disk filesystem like the write path does.

Lustre-change: https://review.whamcloud.com/47268
Lustre-commit: TBD (from 64e269c800533c4ce15235b21597038c29f0040a)

Fixes: cdaaa87f6b ("LU-14214 ofd: fix locking in ofd_object_fallocate()")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I65986745865ee329c5257a7efca5e79403830608
Reviewed-on: https://review.whamcloud.com/47477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14541 llite: Check vmpage in releasepage

We cannot release a page if the vmpage reference count is
>1, otherwise we will detach a vmpage from Lustre when the
page is still referenced in the VM.

This creates a situation where page discard for lock
cancellation will not find the page, so we can get stale
data reads.

This re-introduces the LU-12587 issue where direct I/O on
a client falls back to buffered I/O if there are pages in
cache, since it cannot flush them. This is annoying but
not a huge problem.

This allows us to safely re-enable fast reads.

Lustre-change: https://review.whamcloud.com/47262/
Lustre-commit: c524079f4f59a39b99467d9868ee4aafdcf033e9

Fixes: e59f0c9a245f ("LU-12587 llite: don't check vmpage refcount in ll_releasepage()")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3aa1cd7330f5e7d1ba2ddb0c12779aa22f3d70b7
Reviewed-on: https://review.whamcloud.com/47273
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15875 kernel: kernel update RHEL7.9 [3.10.0-1160.66.1.el7]

Update RHEL7.9 kernel to 3.10.0-1160.66.1.el7.

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I9e8ab33edd6cacbbf895399962027827a1befd5b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47423
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-15793 kernel: kernel update RHEL8.5 [4.18.0-348.23.1.el8_5]

Update RHEL8.5 kernel to 4.18.0-348.23.1.el8_5.

Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.5 serverdistro=el8.5 testlist=sanity
Test-Parameters: trivial fstype=zfs clientdistro=el8.5 serverdistro=el8.5 testlist=sanity
Change-Id: I9724e304c2d32bb0ac2dcf4138438c79d4abca0f
Signed-off-by: Jian Yu <yujian@whamcloud.com>

LU-15795 kernel: new kernel [RHEL 8.6 4.18.0-372.9.1.el8]

This patch makes changes to support new RHEL 8.6 release
for Lustre client.

Test-Parameters: trivial clientdistro=el8.6

Change-Id: Id738259ed94104c3a3c7bb5c1b853cfabad49405
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47304
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-14541 llite: add rw_seq_cst_vs_drop_caches

Add a reproducer (rw_seq_cst_vs_drop_caches) for the read/write vs
drop_caches sequnetial consistency violation described in
LU-14541. Add an always excepted test (sanityn test_16f) to run
rw_seq_cst_vs_drop_caches.

Lustre-change: https://review.whamcloud.com/47243/
Lustre-commit: 8b9185d500b1e3ccdf66c84c1cbba82a0e392e69

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I557ae7386b38214110a4d85ba0515e95fed7a11e
Reviewed-on: https://review.whamcloud.com/47272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5295 lipe: allow for missing changelog files

In lipe/scripts/stratagem-hp-config.sh, suppress errors from a missing
.chlg file.

Fixes: af041fccc1 ("EX-4973 lipe: remove changelog user registration from config")
Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ic00a9bb591de22036936eecd76d08ec9649e4a9e
Reviewed-on: https://review.whamcloud.com/47429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15830 mdt: mkdir to lookup target name

distributed mkdir should lookup the target name to
avoid rollback as much as possible as the latter is
very expensive due to llog re-initialization.

Lustre-change: https://review.whamcloud.com/47226
Lustre-commit: 6bac96ed3b4a6bf514e4bd4c9c27c8ad89d627d1

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If28760e0afb804dca11e1e7501e0a53ff9067ca1
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47324
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5260 pumount: use O_PATH for mount point

In pumount, use O_PATH when opening the mount point to try to avoid
RPCs from permission or atomic_open.

Test-Parameters: trivial testlist=sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ibd66579d9b5ecef724334c01d90367af4bd93179
Reviewed-on: https://review.whamcloud.com/47323
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-3838 test: sanity-pumount races

In sanity-pumount, add functions wait_for_open(), wait_for_exec(), and
wait_for_mmap() to wait for the forked PID to exec, open, or mmap the
file on the Lustre FS. Use these to prevent fork exec check races.

Test-Parameters: trivial testlist=sanity-pumount,sanity-pumount,sanity-pumount,sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I2a89989521cc4e61d35799c1d4fdb8ce37ae12d9
Reviewed-on: https://review.whamcloud.com/47344
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-3763 test: disable sanity-pcc test_44

Disable sanity-pcc test_44 until the failures are sorted out.

Test-Parameters: trivial testlist=sanity-pcc
Change-Id: I8f5ab5025e8d049e1488b3f48708057ac7523600
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47336
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-15827 osd: respect filldir buffer limits

In osd_ldiskfs_filldir() ensure that the encoded name also fits in the
buffer. In osd_ldiskfs_filldir() and obj_name2lu_name() remove a
superfluous and potentially incorrect check on names that appear to be
FIDs.

Lustre-change: https://review.whamcloud.com/47224
Lustre-commit: 5879647624c834d4a579f425a0fc8d408235e275

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1540f058801b002474ceac1206317344cebb1084
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/47325
Tested-by: jenkins <devops@whamcloud.com>

LU-14541 llite: reenable fast_read by default

Reenable fast_read by default.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I9985beb88d13d21ee538c632fc288470a58b0257
Reviewed-on: https://review.whamcloud.com/47299
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>