git://git.whamcloud.com - fs/lustre-release.git/log

EX-5260 pumount: test pumount with MDT0000 unmounted

Add tests (40a-40c, 41a) to verify that pumount can unmount a client
mount point with MDT0000 and OST0000 unmounted. Add a test (34a) that
pumount does not kill processes who are minding their own business on
other Lustre client mount points.

Test-Parameters: testlist=sanity-pumount,sanity-pumount,sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I02b603d1c6de9f10605d3e315d5149de72698e73
Reviewed-on: https://review.whamcloud.com/47389
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15399 llite: dont restart directIO with IOCB_NOWAIT

It should hanlde FLR mirror retry and io_uring with IOCB_NOWAIT
flag differently.

int cl_io_loop(const struct lu_env *env, struct cl_io *io)
{
...
if (result == -EAGAIN && io->ci_ndelay) {
io->ci_need_restart = 1;
result = 0;
}
...
}

ssize_t
generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter)
{
...
if (iocb->ki_flags & IOCB_NOWAIT) {
if (filemap_range_has_page(mapping, iocb->ki_pos,
iocb->ki_pos +
count - 1))
return -EAGAIN;
...
}

In current code, it will restart I/O engine for read when get
-EAGAIN code.
However, for io_uring direct IO with IOCB_NOWAIT, if found that
there are cache pages in the current I/O range, it should return
-EAGAIN to the upper layer immediately. Otherwise, it will stuck
in an endless loop.

This patch also adds a tool "io_uring_probe" to check whether
the kernel supports io_uring fully.
The reason adding this check is because the rhel8.5 kernel has
backported io_uring:
cat /proc/kallsyms |grep io_uring
ffffffffa8510e10 W __x64_sys_io_uring_enter
ffffffffa8510e10 W __x64_sys_io_uring_register
ffffffffa8510e10 W __x64_sys_io_uring_setup
but the io_uring syscalls return -ENOSYS.

Lustre-change: https://review.whamcloud.com/46147/
Lustre-commit: TBD (from f2aeedc5b188ea00f2e05671abb2adae5fadcf6e)

Test-Parameters: clientdistro=ubuntu2004 testlist=sanity-pcc
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Id4374382e56e90d02349676891aa57b216b3deff
Reviewed-on: https://review.whamcloud.com/45941
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15811 llite: Rework upper/lower DIO/AIO

One of the patches for LU-13799, "Implement lower/upper aio"
(https://review.whamcloud.com/44209/) created a
complicated setup where the cl_dio_aio struct was used
both for the top level DIO or AIO and for the lower level
sub I/Os (corresponding to stripes).

This is quite complicated and hard to follow, so this
rewrites these two uses to be separate structs. This
incidentally fixes at least one possible memory leak, but
is mostly a cleanup.

Lustre-change: https://review.whamcloud.com/47187
Lustre-commit: 1256040ca0331d7e3f43b9b3534e2db2443bf28b (TBD)

Fixes: 8eba9a2ef3 "LU-13799 Implement lower/upper aio"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ide4a2b84f48624ee97dfb57fe80d201fbb7fe8d0
Reviewed-on: https://review.whamcloud.com/47188
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15721 llite: only statfs for projid if PROJINHERIT set

If projid is set on a directory but PROJINHERIT is not, do not report
the project quota for statfs. This matches how ext4_statfs() and
xfs_fs_statfs() behave, on which Lustre project quota is modelled.

Lustre-change: https://review.whamcloud.com/47352
Lustre-commit: 4192f28733fe62173dec4a522c6589ee741df474

Fixes: e5c8f6670f ("LU-9555 quota: df should return projid-specific values")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I27cb444c3dfabc0ec693cee6fe6f9cae6db8a77a
Reviewed-on: https://review.whamcloud.com/47586
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>

LU-15195 ofd: missing OST object

as the OST-MDT resync may be not finished by the end of the recovery
it may happen new enqueue for a write op may fail due to an absent
object. Return EINPROGRESS so that the enqueue was resent until get
resynced.

to not get stuck forever in case of disappeared MDT or a double
failure, return EINPROGRESS during hard failover timeout only.

also, cleanup replay-ost-single test 12:
- eliminate a need in the hard failover
- no need in a special obd_fail_loc, just use replay_barrier
- createmany is able to create files with unique names,
no need in special steps

Lustre-change: https://review.whamcloud.com/45459
Lustre-commit: 28769c65987cb1546918fe12d6f34b95ab9c5507

HPE-bug-id: LUS-10267
Signed-off-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I5f16b63454c51ad8d112770c15c7e6e7f41f3c40
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47686
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-9243 gss: fix GSS struct definition badness

struct lgssd_ioctl_param should not be defined in multiple places. So
move it to a new header file lgss.h that can be included from kernel
space and user space.

struct gss_header, struct gss_rep_header, struct gss_err_header and
struct gss_wire_ctx are going on the wire, so they need to be moved to
lustre_idl.h, and be wire-checked.

Lustre-change: https://review.whamcloud.com/46543
Lustre-commit: 7dfbc71350c982be20d75e7d7853038809f4e850

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I97c4a8322e6bb7627c6dff5f068931278f4567d7
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47667
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5366 test: add test case for lpcc service and tool

Add test case for lpcc system service and command line tool.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: I955fd3736ce3fe222afc8a90a34c7eefbe2d834e
Reviewed-on: https://review.whamcloud.com/47644
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15832 lod: clear .do_index_ops in striping free

LDLM lock can guarantee LOD object directory striping is safe to
access, but lod_striping_free_nolock() should clear .do_index_ops,
otherwise upon some failure the directory striping is freed, while a
subsequent dt_try_as_dir() skips striping initialization, and call
.do_index_ops directly, which will cause crash.

Lustre-change: https://review.whamcloud.com/47256
Lustre-commit: 3d4ff484002bf6e29655503994c240da06f838e3

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ib94a4ef2f8bf5f0d34521abff77d8be46ecbf428
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47555
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15508 gss: protect from arbitrary write to init channel

In case some arbitrary data was written to the gss init channel,
directly return -EINVAL. This protects against unsolicited
authentication requests, and avoids having a dandling entry
in the auth init cache.

Lustre-change: https://review.whamcloud.com/46461
Lustre-commit: f06807d409c82dd85c208c79c8072402e77f9bac

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iadde630012e4ded83f9609fbb3e10b2e092deb57
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-on: https://review.whamcloud.com/47669
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15902 obdclass: dt_try_as_dir() check dir exists

If an object is not directory, but dt_lookup() is called on it, it
may crash because .do_lookup is NULL for non-directory file.

Add argument to check object existence and type in dt_try_as_dir(),
and for object to create, skip this check.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I51df0cbb5a4e7abca370ee27dac678f995b76159
Reviewed-on: https://review.whamcloud.com/47622
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5248 ldiskfs: add __GFP_NOWARN to ext4_kvmalloc()

Don't dump a stack trace if kmalloc() fails, since the whole point of
these functions is retry with vmalloc() if normal kmalloc() fails.

Lustre-change: https://review.whamcloud.com/47393
Lustre-commit: 9b15917634e8eb9d5ee29b1d497b389b8f452f9f

Test-Parameters: trivial testlist=sanity-lsnapshot
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id988bd84bd9bdffcb059405f1d471ef7213ebbe5
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/47630
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>

LU-15200 llite: "lfs getdirstripe -D" shows inherit layout

Once system-wide default LMV is set, "lfs getdirstripe -D subdir"
should show inherited layout from it.

Add sanity 413f.

Lustre-change: https://review.whamcloud.com/45570
Lustre-commit: 61b1fad9e3fb21edcc0e713c89a3bfaeba9c883e

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If7354cb4093c58f6d56a6a4d449fb69a9deec7cc
Reviewed-on: https://review.whamcloud.com/47642
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-15200 llite: revalidate dentry if LOOKUP lock fetched

Once ll_inode_revalidate() fetches LOOKUP lock, it should revalidate
dentry, so subsequent lookup can find it in dcache.

It should also update lli_dir_depth.

Lustre-change: https://review.whamcloud.com/45599
Lustre-commit: 92fadf9cc1d06b21b482a262ff66f435814a13f8

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I847e16d64d605b91efc93925821bc29cbea20fa2
Reviewed-on: https://review.whamcloud.com/47598
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5405 lipe: fix bug divided by zero

Fix 'divided by zero' bug in lpcc.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Icb8e6bd70b1292ab70968c40a34fc6c58142a20d
Reviewed-on: https://review.whamcloud.com/47677
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5406 lipe: fix some compatibility issue in lpcc

- use absolute path of program in lpcc.service
- replace yaml.load() with yaml.safe_load()

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Change-Id: Iad3fa408f5b19e7370c0a5b069ad7637611118a7
Reviewed-on: https://review.whamcloud.com/47678
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn46

New tag 2.14.0-ddn46

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I4590039e019187be7f010f1756b8ac649318ad8d

LU-15886 lfsck: remove unreasonable assertions

Remove unreasonable assertions in LFSCK code:
* lfsck->li_obj_dir and lfsck->li_lmv may be NULL if object wan't
  initialized successfully.
* orphan objects under ldiskfs /lost+found may not exist.
* object may not be directory in lfsck_verify_lpf()->
  lfsck_verify_linkea().
* for corner case
  (leh->leh_reccount == 0 && leh->leh_overflow_time != 0),
  LASSERT(ldata->ld_leh->leh_reccount > 0) will be triggerred in
  lfsck_namespace_linkea_clear_overflow(), remove this assertion,
  and this corner case can be handled correctly in current lfsck
  code.

Lustre-change: https://review.whamcloud.com/47447/
Lustre-commit: 0c1f5aa5e20334746b22b4e215af5153de7d6f4a (TBD)

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: If114c7882a2c083e83fcfac5981eddfa526d1426
Reviewed-on: https://review.whamcloud.com/47621
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15868 lfsck: don't crash upon dir migration failure

LFSCK against directories that were migrated, but failed may crash,
it's because lost+found directory may not be initialized correctly,
and this error is skipped on purpose, add check in code that
dereference it.

lfsck_verify_lpf() may dereference NULL "child2".

lmv_name_to_stripe_index() should support stripe LMV, which is used
by LFSCK to verify name hash.

Add OBD_FAIL_OUT_EIO to simulate sub transaction failure.

Add sanity-lfsck 15d to verify LFSCK won't crash upon directory
migration failure.

Update sanity-lfsck 4 and 5 to start mds1 with OI scrub enabled, and
wait for mds1 OI scrub finish, otherwise LFSCK may fail to verify
lost+found later.

Lustre-change: https://review.whamcloud.com/47381/
Lustre-commit: 1022e485b5260e3bca0e2acafaf807dffbb02c66 (TBD)

Test-Parameters: mdscount=2 mdtcount=4 testlist=sanity-lfsck env=ONLY=15d,ONLY_REPEAT=100
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I1b1872da2b4ef8f7403effc4d1d3e298c6a0b7e6
Reviewed-on: https://review.whamcloud.com/47620
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15933 libcfs: fix configure check for iov_iter member

Kernel commit v5.13-rc1-12-g8cd54c1c8480 changed iov_iter member
->type to ->iter_type, add check at configure to cover this change.

Lustre-change: https://review.whamcloud.com/47601
Lustre-commit: TBD (from f6bce444082d44d3dec9db00bce63dc92092c927)

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8da26e1f6c6242a3260b458770ec34a65aecaf7d
Reviewed-on: https://review.whamcloud.com/47603
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15887 test: add always_except()

In test-framework.sh, add a new function (always_except()) to replace
manual manipulation of $ALWAYS_EXECPT. Convert sanity.sh to use
always_except() and add a line to contrib/scripts/spelling.txt to
suggest its use.

Lustre-change: https://review.whamcloud.com/47452
Lustre-commit: c4ff4aef7eb939d536acffaac4465039f3cfa935

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1b39fe9555bab59e70db00cef73d13102668500a
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47616
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15117 ofd: don't take lock for dt_bufs_get()

osd_bufs_get() allocates the pages and can cause new transactions
as part of memory release procedure. this would break Lustre's
"start a transaction, then do locking" rule.

Lustre-change: https://review.whamcloud.com/47029
Lustre-commit: TBD (from 14db15b1dcea61b1c35e00f004124501ead86ab6)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I58a52af8e2fbbc4823aafc133893e1defedf99b7
Reviewed-on: https://review.whamcloud.com/47592
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15647 utils: 'lfs df' shouldn't loop forever

Don't loop forever in "lfs df" if there is a problem fetching
the target statfs information from the kernel and -ENODEV is
not returned for whatever reason.

Lustre-change: https://review.whamcloud.com/46813
Lustre-commit: 41703e9b5fe7061b313805384b08372d4fe82cec

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5110c7246cf53eb5c4d59424a44b73b9d23ebbe5
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47605
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-13562 build: get correct kernel flavor for SLES

This patch fixes lustre.spec.in to get correct kernel flavor
for SLES when kobjdir is detected as /lib/modules/%{_kver}/build.

Test-Parameters: trivial clientdistro=sles15sp3

Change-Id: I350032af383ea8b7f48accd93e5cd11c571e6620
Fixes: d746e64fe1 ("LU-13562 build: SUSE build support for azure, cray_ari_s")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47596
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-12214 build: fix SLES build/install

Redhat and SuSe can have different library name for same devel,
lets drop a strong requrement to the library package name and
ask rpm to use an autoprovide option.

Lustre-change: https://review.whamcloud.com/39972
Lustre-commit: 253a9e3bcd30bf411c57d77e137ff734eff1edb1

Test-Parameters: trivial clientdistro=sles15sp3
HPE-bug-id: LUS-7204
Fixes: e1bf37870d LU-12214 build: fix build with gss enabled
Fixes: d746e64fe1 LU-13562 build: SUSE build support for azure
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Change-Id: I7e0fe83f9090e7616ab156fa75fed4821099406e
Reviewed-on: https://review.whamcloud.com/47595
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15513 lod: iterate initialized stripe

Fix an array out of bounds access.

Do not iterate over the component objects in lod_fill_mirrors() to
check non-rotational OSTs if the component is unintialized or it is
a MDT or RELEASED component.

Lustre-change: https://review.whamcloud.com/47028
Lustre-commit: TBD (from b9445c05576da11cc230b290b1ec53131a0e1c2a)

Fixes: 591a990c61 ("LU-15513 lod: skip uninit component in lod_fill_mirrors")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I8530cd30fab750c47f15ee9278ec6d68871367cc
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47590
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-13363 lod: do object allocation in OST pool

Currently, the ltd->ltd_qos.lq_same_space boolean that decides
whether the LOD QOS allocator is active for an allocation or not
is tracked for the entire LOV. But when a pool is specified, this
judgement should be tracked on a per-pool basis.

sanity.sh test_116c is added to verify this patch.

Lustre-change: https://review.whamcloud.com/38136
Lustre-commit: e642e75cde0248eee30ca94aaeb81653db7f8d03

Test-Parameters: ostcount=6 testlist=sanity env=ONLY=116c
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: I463d5927c7a9c9171483615d2cec629ec10dc666
Reviewed-on: https://review.whamcloud.com/47334
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15952 doc: improvement on lfs-project doc

Describe 'lfs project -C' clearly.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Change-Id: I1e0f70a9116265edac993b12b775bd57c8587d40
Reviewed-on: https://review.whamcloud.com/47645
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14765 test: disable sanity-flr/44c

file blocks number accounting does not include blocks occupied by
multiple mirrors, disable the test for now.

Lustre-commit: a45fe93cd8d1e941d58e0f11e21649e1956ba2c7
Lustre-change: https://review.whamcloud.com/44074

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I1a5d7e0c203d5c7dc0da53ce6c020795bf17b9fc
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47647
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14583 llapi: handle symlinks in llapi_file_get_stripe()

In llapi_file_get_stripe(), if the IOC_MDC_GETFILESTRIPE ioctl handler
returns -ENOTTY or -ENODATA then try to resolve any symlinks in the
path and try again.

Lustre-change: https://review.whamcloud.com/43229
Lustre-commit: cd4caef54f63b6a0d26a90d879f378b7fe3d4862

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ic046d6ef77d8342d47336144e3066cab3a940a96
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47588
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-12022 tests: error on resync failure sanity-flr

In sanity-flr test 200, we should error if the final resync
fails. Replace all calls to 'mirror_io resync' that does
not inject an error to '$LFS mirror resync'.

Lustre-change: https://review.whamcloud.com/35754
Lustre-commit: 9e1e29406c3886bce0db6623be40d3edf6834117

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I9b2ec1beb7060086808b7529467bef80c8e9659f
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47617
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5364 tests: add sanity-flr/38 to ALWAYS_EXCEPT

Add sanity-flr.sh test_38 to ALWAYS_EXCEPT as it is failing too often.

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia27e32d7309bc3597ee4024e465959c9dbd58b4a
Reviewed-on: https://review.whamcloud.com/47610
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn45

New tag 2.14.0-ddn45

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ic13969b93a939490962ecc13c950d19ba19f1a76

LU-15838 autoconf: fix use of obsolete macros

This patch fixes the following warnings when using autoconf 2.71:

configure.ac:2: warning: AC_INIT: not a literal:
"m4_esyscmd(sh -c "./LUSTRE-VERSION-GEN | tr -d '\n'")"
configure.ac:10: warning: The macro `AC_CANONICAL_SYSTEM' is obsolete.
configure.ac:16: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
configure.ac:24: warning: The macro `AC_HELP_STRING' is obsolete.

Like m4_esyscmd, macro m4_esyscmd_s (introduced in autoconf 2.64)
expands to the result of running command in a shell. The difference
is that any trailing newlines are removed.

Since autoconf 2.50, macro 'AC_CANONICAL_TARGET' has been the new name
of 'AC_CANONICAL_SYSTEM':
AU_ALIAS([AC_CANONICAL_SYSTEM], [AC_CANONICAL_TARGET])

Since autoconf 2.58, macro 'AS_HELP_STRING' has been added to replace
'AC_HELP_STRING'.

Since libtool 2.0, new 'LT_INIT' interface has been added to replace
'AC_PROG_LIBTOOL'.

Lustre-change: https://review.whamcloud.com/47288
Lustre-commit: TBD (from a94a730df48089e3ffbf853af78a2444dc5a3042)

Change-Id: I3c06c21460d7a2cf643fe825e72a26a5416609cf
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47539
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5347 lipe: update lpcc to python3

Update the lpcc utility to use python 3.

Test-Parameters: trivial clientdistro=ubuntu2204 env=SANITY_PCC_EXCEPT="15 16 27 39 101a 201" testlist=sanity-pcc
Test-Parameters: trivial clientdistro=ubuntu2004 env=SANITY_PCC_EXCEPT="101a" testlist=sanity-pcc
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Signed-off-by: Lei Feng <flei@whamcloud.com>
Change-Id: I690ef4362cbdd0118419ab37a15127619a860178
Reviewed-on: https://review.whamcloud.com/47553
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5360 lipe: lpcc handles unmounted lustre fs

If lustre fs is unmounted, lpcc daemon crashes while serving
'lpcc status' command. Fix it.

Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-lipe
Change-Id: I5304745aed001fbe26b77ed3cebfaabfddfce11f
Reviewed-on: https://review.whamcloud.com/47572
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

EX-5045 lipe: use LDADD

Use foo_LDADD rather than foo_LDFLAGS to avoid future errors with
strict linkers. Remove AC_SEARCH_LIBS() uses for ext2fs and com_err,
since we already use PKG_CHECK_MODULES() to check for ext2fs
libraries, and not all lipe utilities need -lext2fs.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1de960fcec5a8770984d9b1437423a854948eb84
Reviewed-on: https://review.whamcloud.com/47443
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15900 hsm: don't return error on state change during mount

HSM coordinator is started in stopped state always, but
mount may have hsm_control parameters 'disabled'. Such
parameter cause wrong state change so mount would fail
with error.

Treat parameter change from 'stopping/stopped' to 'disabled'
as not critical error, keep state unchanged and report
no error back to a caller.

Lustre-change: https://review.whamcloud.com/47473
Lustre-commit: TBD (from c3fc6fef304d052da011c5840134af197eaefd94)

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I9d1366423391971b9511c46b6aed39d21ebf637c
Reviewed-on: https://review.whamcloud.com/47474
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15484 tests: Increase timeout to wait for child

Sanity-pcc 101a is failing because the remote 'sleep 600'
process is not always started in time, meaning the NS
doesn't exist.

Increase the wait time for this process to start.

Lustre-change: https://review.whamcloud.com/46764
Lustre-commit: TBD (20f794e7e8b736a944c9e1e87970e7232abd6a19)

Fixes: f3be560031 "LU-15170 llite: Switch pcc to lookup_one_len"

Test-Parameters: trivial
Test-Parameters: testlist=sanity-pcc env="ONLY=101a,ONLY_REPEAT=50"
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I57175ec2f0e948045cf88d4a13a1e56cc34f8407
Reviewed-on: https://review.whamcloud.com/47557
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14785 build: changelog updates should not dirty version

When building lustre debs the final version should not
include 'dirty' due to an update of the changelog

Lustre-change: https://review.whamcloud.com/44069
Lustre-commit: a91839eb57103e6fbf5a3f6bacfcc0c471620057

HPE-bug-id: LUS-10152
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I59f4da4b3006302e3598cfa56a0364b052f885ef
Reviewed-on: https://review.whamcloud.com/47535
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5347 build: update libssl3

In Ubuntu 22.04 libssl1.1 has been superceded by libssl3

Test-Parameters: trivial clientdistro=ubuntu2004

Change-Id: Ic2504c3a40d5c756c2da7b81edb1501fb6b44712
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47556
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15918 build: update debhelper compatibility level to 11

This patch updates debhelper compatibility level to 11 (which is
currently the recommended level) to resolve the following build
warnings while running "make debs":

Compatibility levels before 10 are deprecated (level 9 in use)

Test-Parameters: trivial clientdistro=ubuntu2004 \
testlist=sanity

Change-Id: Ie5c63f6a3eb29e9aeace1f008d8d492e7587adf2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47551
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15573 build: remove mpi, CC deps from config cache

Also drop mpi and CC dependencies compiler type from the
initial values in the configure cache as these may be
changed during the setup in the rpm spec or debian build rules

Lustre-change: https://review.whamcloud.com/46562
Lustre-commit: TBD (from 497cea4815a3e08f3ffd88ecba0c129a9e9ca5b8)

Test-Parameters: trivial
Fixes: a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I986c2ae3653deae08b9da8d64d0b3c02fdc8fa2b
Reviewed-on: https://review.whamcloud.com/47536
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15908 build: fix debian rules

A comment in debian/rules file breaks a commands-line continuation,
causing exported env vars to be dropped.

Lustre-change: https://review.whamcloud.com/47512
Lustre-commit: TBD (from d1d5b35306bc164afe444164a40fdd3940a2abcd)

Fixes: a5084c2f2e ("LU-14937 build: re-use config cache in 'make rpms/debs'")
Test-Parameters: trivial
Signed-off-by: Ake Sandgren <ake.sandgren@hpc2n.umu.se>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0894199301c489b1961e043651c74d7c6c0089d2
Reviewed-on: https://review.whamcloud.com/47537
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15796 build: require python2 for lustre-iokit deb

Package python is not available on Ubuntu 22.04.
We can just require python2 in the control files.

Lustre-change: https://review.whamcloud.com/47164
Lustre-commit: TBD (from d46e59a541a4a2db73ccae95707f08d5d9c40ccd)

Test-Parameters: trivial clientdistro=ubuntu2004 \
testlist=obdfilter-survey

Change-Id: Id37c9babba31c575855238592cbac4ce5c9ee79a
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47538
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15420 build: fixes to support building on Ubuntu 22.04 LTS

Lustre uses the glibc stdarg.h instead of the kernel's version which
causes the following build issue.

lustre/include/lu_object.h:35,
/usr/lib/gcc/x86_64-linux-gnu/11/include/stdarg.h:52: note: this is the
location of the previous definition
#define va_copy(d,s) __builtin_va_copy(d,s)

The solution is to use the kernels version of stdarg.h

The second build issue :
update_trans.c:1608:30: error: ‘struct task_struct’ has no member named
‘state’; did you mean ‘__state’?

is due Linux commit 2f064a59a11ff9bc22e52e9678bc601404c7cb34
(sched: Change task_struct::state). The state field was
changed and the barrier macros READ_ONCE()/WRITE_ONCE()
are used to access it now which is the proper thing to do.
Since the check in update_trans.c is equivalent to testing
if the kernel thread is not running, since TASK_RUNNING == 0,
we can just change the code to use task_is_running(). The
task_is_running() was introduced in 5.13.

Lustre-change: https://review.whamcloud.com/47133
Lustre-commit: TBD (from 6b096f55bad9bd169b2aa01237764cf97b58ee56)

Test-Parameters: trivial
Change-Id: Ib5985b187c3013fbc513e9962a5f27bed4996f5b
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47540
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10632 tests: recovery-small test_26 idle_timeout

In recovery-small test_26() use "lfs df" instead of plain "df"
since statfs may be fetched from the MDS cache and will not
ensure that the client->OST connections are currently active.

Also, check a few entries further back in the OSC state log for an
EVICTED message, in case the client idle disconnects from the server
again while checking all of the imports.

Lustre-change: https://review.whamcloud.com/42006
Lustre-commit: b4391fcdaf392a50bd1419342eca3b730c077ed2

Test-Parameters: trivial testlist=recovery-small env=ONLY=26a,ONLY_REPEAT=100
Fixes: 5a6ceb664f07 ("LU-7236 ptlrpc: idle connections can disconnect")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8c370cb75f4e06258ef3c032630fc20354a15dcc
Reviewed-on: https://review.whamcloud.com/45296
Tested-by: jenkins <devops@whamcloud.com>

LU-15220 lipe: avoid gcc-11 -Werror=stringop-truncation warning

This patch fixes -Werror=stringop-truncation warnings
in lipe codes.

Test-Parameters: trivial testlist=sanity-lipe

Change-Id: Ie007083b1fe94d81e3772d7c0947457d2c3042ac
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47510
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn44

New tag 2.14.0-ddn44

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia5ec26041bac7cc62fcef79fd7c8ff2d5ca170da

LU-15896 gss: ignore OpenSSLv3 deprecated API

Lustre GSS code makes use of some OpenSSL API that has been
deprecated in v3, namely all the functions in the DH_* family.
Those would need to be replaced with their EVP_PKEY_* counterparts
but in the meantime we just use the OPENSSL_SUPPRESS_DEPRECATED
define to continue using the deprecated API.

Lustre-change: https://review.whamcloud.com/47485
Lustre-commit: TBD (from 05d5b73ebb253c0e3ed5a7954b751ebbdb2d61bf)

Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I413a2af1299123b29ed2043188da0348ce9d0a5e
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47496
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15776 tgt: fix transaction handling in tgt_brw_write()

Hotfix to prevent possible data loss during WRITE replay.
Since commit f0f92773ee18 from LU-14187 the obd_commitrw()
may restart write transaction in OFD and MDT. That causes
transaction number to be assigned multiple times if such
restart happens. Without flag tti_mult_trans the first
transaction number is stored only so later one could remain
not applied causing data loss after recovery.

Patch sets tti_mult_trans for tgt_brw_write() so the latest
transaction number will be used as request transno.

Lustre-change: https://review.whamcloud.com/47371
Lustre-commit: TBD (from ccce4f23c773d9b9cc98cb15ed31861803167bb9)

Fixes: f0f92773ee ("LU-14187 osd-ldiskfs: fix locking in write commit")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I364b478591942be5562c3e98ee6e6aa487f3e0c5
Reviewed-on: https://review.whamcloud.com/47428
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15840 lov: return st_blocks=1 for HSM released files

The MDT will return st_blocks=1 for a HSM released file.
In the call ->coo_attr_get in LOV layer, the client should also
return st_blocks=1 for a HSM released file.

Otherwise, the client may get 0 block count. It is very easy to
reproduce this problem via the following commands for a archived
file:
# $LFS hsm_restore $file
# $LFS hsm_release $file
# $LFS hsm_release $file
After release a file twice, the reported block count via stat()
call will become 0.

Lustre-change: https://review.whamcloud.com/47291
Lustre-commit: TBD (from 974320c61ee7d165073bb2810795575261996dc7)

Change-Id: Id1841147e40a7df0ca615e887f324cff8e613f11
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47290
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5249 utils: show pin information in 'lfs pcc state' command

If xattr lustre.pin exists, show its content in 'lfs pcc state'
command.

Change-Id: I867ea3b246fc3b7d10e166d754c081c8afc462d5
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/47287
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15841 lod: iterate component to collect avoid array

In newly created file, the mirror information hasn't been established
as LOD is trying allocate OST for its components, so we need to
iterate component instead of mirror to collect the avoid guidance
information.

Lustre-change: https://review.whamcloud.com/47293
Lustre-commit: TBD (from d04b04056e45edc95117f0761359ddc4637d3f37)

Test-Parameters: testlist=sanity-flr env=ONLY=47,ONLY_REPEAT=40
Fixes: fabf3fe7 ("LU-9007 lod: improve obj alloc for FLR file")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I6bbe5f6b6dfea06c5213b77b7ebb6a5d28aa0d17
Reviewed-on: https://review.whamcloud.com/47295
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5176 tests: re-enable sanity test_400a

Re-enable test_400a after it was disabled due to json-c dependency
during build test. This library is now installed on client nodes.

Test-Parameters: trivial testlist=sanity env=ONLY=400 clientdistro=sles15sp3
Test-Parameters: testlist=sanity env=ONLY=400 clientdistro=ubuntu2004
Test-Parameters: testlist=sanity env=ONLY=400 clientdistro=el8.5
Fixes: fbfd2d0755 ("EX-5176 pcc: use JSON string for trusted.pin xattr")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3daca219cedb447127f9402a79e6124afb3ebbe5
Reviewed-on: https://review.whamcloud.com/47363
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>

LU-15848 ldiskfs: escape encrypted file names

When a Lustre MDT is mounted as ldiskfs, the names of the encrypted
files have to be escaped to avoid breaking the shell.
On CentOS 7, the LDISKFS_ENCRYPT_FL flag does not exist. So we add it,
and when the target is mounted as ldiskfs (LDISKFS_MOUNT_DIRDATA flag
not present) we critical-encode encrypted file names before
presentation. And conversely, we critical-decode names upon lookup.
On CentOS 8, the LDISKFS_ENCRYPT_FL flag exists. The fscrypt functions
from kernel 4.18 are also wired up, but they all refer to -EOPNOTSUPP
or equivalent, so they cannot be used to present usable names. So when
the target is mounted as ldiskfs (LDISKFS_MOUNT_DIRDATA flag not
present) we proceed to critical-encoding of encrypted file names
before presentation, and to critical-decoding upon lookup.

Lustre-change: https://review.whamcloud.com/47309
Lustre-commit: fa7c81e81442c8d1ed0ce1c457769f2e144ff2c3

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Iaca467eaa233be8142356efa822962953754c2ce
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47482
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-15706 llog: deal with "SKIP" pool llog records correctly

If the requested "start" llog records by one ioctl are just the
useful ones "protected" by one marker, they will don't know their
cfg_flags(e.g. "SKIP" or not), and then will be mis-labeled in
class_config_yaml_output().
To fix this issue, this patch does the following changes:
- In kernel space, remember the marker cfg_flags for 10 records
  earlier than the requested "start" in function llog_print_cb(),
  so that the "start" can be output with its correct cfg_flags.
- In user space, since the pool_new/add record will be marked as
  "SKIP" if its corresponding pool_destroy/remove record is logged
  later in mgs_pool_cmd(), these "SKIP" records won't be printed
  by function jt_llog_print_iter(), so lpd_ost_num doesn't need to
  be decreased in callback function llog_search_pool_cb() as well,
  otherwise, it will cause "lctl pool_destroy" error.

Lustre-change: https://review.whamcloud.com/46951/

Test-Parameters: standalonemgs=true testlist=ost-pools
Test-Parameters: standalonemgs=true testlist=conf-sanity env=ONLY=123,HONOR_EXCEPT=y
Fixes: 2a5b50d20717 ("LU-15142 lctl: fixes for set_param -P and llog_print")
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Change-Id: Ie45127ac8b80a75eaeb7158559c690da52eef103
Reviewed-on: https://review.whamcloud.com/47463
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15800 ofd: take a read lock for fallocate

there is no need to take an write (exclusive) object's
lock for fallocate - we just need to serialize fallocate
vs destroy, all internal structures should be protected
by OSD and disk filesystem like the write path does.

Lustre-change: https://review.whamcloud.com/47268
Lustre-commit: TBD (from 64e269c800533c4ce15235b21597038c29f0040a)

Fixes: cdaaa87f6b ("LU-14214 ofd: fix locking in ofd_object_fallocate()")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I65986745865ee329c5257a7efca5e79403830608
Reviewed-on: https://review.whamcloud.com/47477
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14541 llite: Check vmpage in releasepage

We cannot release a page if the vmpage reference count is
>1, otherwise we will detach a vmpage from Lustre when the
page is still referenced in the VM.

This creates a situation where page discard for lock
cancellation will not find the page, so we can get stale
data reads.

This re-introduces the LU-12587 issue where direct I/O on
a client falls back to buffered I/O if there are pages in
cache, since it cannot flush them. This is annoying but
not a huge problem.

This allows us to safely re-enable fast reads.

Lustre-change: https://review.whamcloud.com/47262/
Lustre-commit: c524079f4f59a39b99467d9868ee4aafdcf033e9

Fixes: e59f0c9a245f ("LU-12587 llite: don't check vmpage refcount in ll_releasepage()")
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3aa1cd7330f5e7d1ba2ddb0c12779aa22f3d70b7
Reviewed-on: https://review.whamcloud.com/47273
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15875 kernel: kernel update RHEL7.9 [3.10.0-1160.66.1.el7]

Update RHEL7.9 kernel to 3.10.0-1160.66.1.el7.

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I9e8ab33edd6cacbbf895399962027827a1befd5b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47423
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-15793 kernel: kernel update RHEL8.5 [4.18.0-348.23.1.el8_5]

Update RHEL8.5 kernel to 4.18.0-348.23.1.el8_5.

Test-Parameters: trivial fstype=ldiskfs clientdistro=el8.5 serverdistro=el8.5 testlist=sanity
Test-Parameters: trivial fstype=zfs clientdistro=el8.5 serverdistro=el8.5 testlist=sanity
Change-Id: I9724e304c2d32bb0ac2dcf4138438c79d4abca0f
Signed-off-by: Jian Yu <yujian@whamcloud.com>

LU-15795 kernel: new kernel [RHEL 8.6 4.18.0-372.9.1.el8]

This patch makes changes to support new RHEL 8.6 release
for Lustre client.

Test-Parameters: trivial clientdistro=el8.6

Change-Id: Id738259ed94104c3a3c7bb5c1b853cfabad49405
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47304
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Colin Faber <cfaber@ddn.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-14541 llite: add rw_seq_cst_vs_drop_caches

Add a reproducer (rw_seq_cst_vs_drop_caches) for the read/write vs
drop_caches sequnetial consistency violation described in
LU-14541. Add an always excepted test (sanityn test_16f) to run
rw_seq_cst_vs_drop_caches.

Lustre-change: https://review.whamcloud.com/47243/
Lustre-commit: 8b9185d500b1e3ccdf66c84c1cbba82a0e392e69

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I557ae7386b38214110a4d85ba0515e95fed7a11e
Reviewed-on: https://review.whamcloud.com/47272
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5295 lipe: allow for missing changelog files

In lipe/scripts/stratagem-hp-config.sh, suppress errors from a missing
.chlg file.

Fixes: af041fccc1 ("EX-4973 lipe: remove changelog user registration from config")
Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ic00a9bb591de22036936eecd76d08ec9649e4a9e
Reviewed-on: https://review.whamcloud.com/47429
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15830 mdt: mkdir to lookup target name

distributed mkdir should lookup the target name to
avoid rollback as much as possible as the latter is
very expensive due to llog re-initialization.

Lustre-change: https://review.whamcloud.com/47226
Lustre-commit: 6bac96ed3b4a6bf514e4bd4c9c27c8ad89d627d1

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If28760e0afb804dca11e1e7501e0a53ff9067ca1
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47324
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5260 pumount: use O_PATH for mount point

In pumount, use O_PATH when opening the mount point to try to avoid
RPCs from permission or atomic_open.

Test-Parameters: trivial testlist=sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ibd66579d9b5ecef724334c01d90367af4bd93179
Reviewed-on: https://review.whamcloud.com/47323
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-3838 test: sanity-pumount races

In sanity-pumount, add functions wait_for_open(), wait_for_exec(), and
wait_for_mmap() to wait for the forked PID to exec, open, or mmap the
file on the Lustre FS. Use these to prevent fork exec check races.

Test-Parameters: trivial testlist=sanity-pumount,sanity-pumount,sanity-pumount,sanity-pumount clientextra_install_params="--packages pumount"
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I2a89989521cc4e61d35799c1d4fdb8ce37ae12d9
Reviewed-on: https://review.whamcloud.com/47344
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-3763 test: disable sanity-pcc test_44

Disable sanity-pcc test_44 until the failures are sorted out.

Test-Parameters: trivial testlist=sanity-pcc
Change-Id: I8f5ab5025e8d049e1488b3f48708057ac7523600
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/47336
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

LU-15827 osd: respect filldir buffer limits

In osd_ldiskfs_filldir() ensure that the encoded name also fits in the
buffer. In osd_ldiskfs_filldir() and obj_name2lu_name() remove a
superfluous and potentially incorrect check on names that appear to be
FIDs.

Lustre-change: https://review.whamcloud.com/47224
Lustre-commit: 5879647624c834d4a579f425a0fc8d408235e275

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1540f058801b002474ceac1206317344cebb1084
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/47325
Tested-by: jenkins <devops@whamcloud.com>

LU-14541 llite: reenable fast_read by default

Reenable fast_read by default.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I9985beb88d13d21ee538c632fc288470a58b0257
Reviewed-on: https://review.whamcloud.com/47299
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-3442 test: disable hot-pools test_15

Disable hot-pools test_15 until the failures are sorted out.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I787f08161846ec0dbb29524aa60d6eec40007d9c

RM-620 build: New tag 2.14.0-ddn43

New tag 2.14.0-ddn43

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibe83ff8c478cd7678297cc11a0e205b5ee7b892f

LU-15846 tests: don't use comma-separated debug flags

To avoid test interop issues between b_es6_0 clients and b_es5_2
servers, don't use comma-separated debug flags in sanity-quota.sh
quota_init() and quota_fini().

Lustre-change: https://review.whamcloud.com/47308
Lustre-commit: TBD (from 12282314e2567973b1a7cb4d6806e987bf1a6755)

Test-Parameters: trivial testlist=sanity-quota.sh serverversion=2.12.6-ddn66
Fixes: 6b6fde1026 ("LU-13055 libcfs: allow comma-separated masks")
Fixes: 78be823f33 ("LU-15218 quota: delete unused quota ID")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifca39054d14292bca8bcff9b8e03ae58fd5cc3a8
Reviewed-on: https://review.whamcloud.com/47307
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15285 mdt: fix same-dir racing rename deadlock

With LU-12125 lifting the BFL for same directory rename,
a deadlock possibility opens up since we lock source and target
of rename in the source-target order, if there are two renames
racing to rename arguments in reverse order:
mv a b &
mv b a

a lock inversion happens and a deadlock has been observed.

To avert this - instill additional order requirement:
lower PDO hash value is to be locked ahead of the higher one.

Lustre-change: https://review.whamcloud.com/45676
Lustre-commit: 82ec537d8b4cc9261828f4efe6b03d8d33f38432

Fixes: d76cc65d5d68 ("LU-12125 mds: allow parallel regular file rename")
Fixes: b50bb830f92e ("LU-3538 dne: Commit-on-Sharing for DNE")
Fixes: 9f1711f3d7d1 ("LU-12081 mdt: rename shouldn't PDO lock if parent is remote")
Change-Id: I88dd3aebb394ea40e97e6029d6dcc161116f982e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47278
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-4973 lipe: remove changelog user registration from config

In stratagem-hp-config.sh, remove the changelog user registration
code. If no chnagelog user is configured then lamigo will register a
new named changelog user with the required mask.

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I457f4539451e93bf7980fc7fb807f67e67abf247
Reviewed-on: https://review.whamcloud.com/47045
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

EX-5176 utils: fix package requirements for json-c

SLES15.3 uses "libjson-c3" for json-c RPM vs. "json-c" for RHEL.
Ubuntu 20.04 uses "libjson-c4" and 22.04 uses "libjson-c5", but
the right one is depended on by "libjson-c-dev".

Test-Parameters: trivial testlist=sanity clientdistro=sles15sp3
Test-Parameters: testlist=sanity clientdistro=ubuntu2004
Fixes: fbfd2d075534 ("EX-5176 pcc: use JSON string for trusted.pin xattr")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibb01cfb7dadda83572be68961cf98450536980cf
Reviewed-on: https://review.whamcloud.com/47267
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5235 lipe: Fix lipe-server build

To compile against liblustreapi, we now also need json-c.

Signed-off-by: Nathaniel Clark <nclark@whamcloud.com>
Change-Id: I5e5323c556d1d3bdd4fd6d7f9bc59e33baf59310
Reviewed-on: https://review.whamcloud.com/47263
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15025 quota: stale edquot after clearing limits

When hard and soft limit set to 0, lqe enforced flag is also
set to false. As qmt_adjust_qunit does not handle not enforced
lqes, edquot set to the pool continues to be true and a user
gets -EDQUOT even if all pool limits are cleared. This was ok
for global pool lqe as since it turned off, zero limits are
sent to OSTs causing OSTs to release all granted space and
avoid EDQUOT. Fix this for PQ - set edquot and qunit to zero,
since appropriate lqe becomes "not enforced".

Lustre-change: https://review.whamcloud.com/45000
Lustre-commit: 6ed5d6ed3bd0a705c6ae8c37a0b7e8906ebc6079

HPE-bug-id: LUS-10146
Change-Id: I1e08929bae7e1b37b1e8cbbc44859a786b5fb090
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-on: https://review.whamcloud.com/46657
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15787 sec: document enc-unaware clients on enc files

Document the behavior of encryption-unaware clients when they access
encrypted files.

Lustre-change: https://review.whamcloud.com/47182
Lustre-commit: 751a8114ef3afe9abe7692b3974b070db6a705a2

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0354e3051e10aa0542baeb8e34c6201d47e65710
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47228
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15803 sec: correctly handle page lock in ll_io_zero_page

In ll_io_zero_page(), we need to make sure we have locked the page,
and it is up-to-date, before zeroing. So modify ll_io_read_page()
behavior to not disown the clpage for our use case. It avoids being
exposed to concurrent modifications.

Lustre-change: https://review.whamcloud.com/47170
Lustre-commit: edcd05e5ac035dd1daf263105df33d7cfc6dbf6e

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I58e4cf80374a798c9c4302364cf2fb39da9033bb
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47229
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-14802 nodemap: return proper error code

In nodemap_add_range_helper() it was always returning -ENOMEM when
there was an error inserting a new range into the existing nodemap.

    nodemap_add_range_helper()) cannot insert nodemap range: rc = -17
    mgs_iocontrol_nodemap()) MGS: OBD_IOC_NODEMAP command: rc = -12

This was confusing because the error returned by range_insert() was
typically -EEXIST (i.e. the entry being inserted already was in the
nodemap).  Do not print an error to the console in this common case.

Return the actual error to the caller so that this is more clear
to the end user.  Have l_ioctl() always set errno on error, in
addition to returning the error, since many callers depend on this.

Lustre-change: https://review.whamcloud.com/45626
Lustre-commit: 7d6cddff24de2f79e539eae462554dc21e674511

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2c80a11dfdf9e6e1c9a8235b8f74f5bcea68c08e
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-on: https://review.whamcloud.com/47247
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-15787 sec: block enc unaware clients on enc files

Prevent encryption unaware clients from manipulating encrypted files
and directories. Those can be old clients, or clients built without
encryption support (intentionally or because they run on an old
kernel).
In the mdt layer, check that clients have the OBD_CONNECT2_ENCRYPT
connection flag, and if not, block access if they try to manipulate
a file or directory that has the LUSTRE_ENCRYPT_FL flag.
The forbidden operations from encryption unaware clients are:
- open
- create
- link
- rename
- migrate
Improve sanity-sec test_54 to test this use case.

Lustre-change: https://review.whamcloud.com/47156
Lustre-commit: a31db2ec062ccc995527d37f0330edbca9d486a9

Test-Parameters: testlist=sanity-sec mdscount=2 mdtcount=4 osscount=1 ostcount=8 clientcount=2 serverdistro=el7.9
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ief0639e49c0a8e1a1a0cb19cb13c006edfdff6c4
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47227
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn42

New tag 2.14.0-ddn42

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ifcaf23d84abf5802ab305e242d6ec3b3936d837e

LU-14790 tests: Check NI status when link is downed

Add test to check that NI status is set to down when the
ni_fatal_error_on flag is set (i.e. when a link is down).

Lustre-change: https://review.whamcloud.com/44073
Lustre-commit: 412970c586ea189d442620316c5ed6f1fa43cc0e

HPE-bug-id: LUS-10167
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: If98a899b0ee8dd9637c08774109668ad06244c60
Reviewed-on: https://review.whamcloud.com/46846
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14790 lnet: Reflect ni_fatal in NI status

If the ni_fatal_error_on flag is set on an NI then that NI should be
considered down.

Lustre-change: https://review.whamcloud.com/44072
Lustre-commit: d77e95cc6d4e947be60757a861c656d8b30d2972

HPE-bug-id: LUS-10167
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: I201bda7e06da1fb1cc23db70ce0cfa3118635d0f
Reviewed-on: https://review.whamcloud.com/46845
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

EX-5176 pcc: use JSON string for trusted.pin xattr

Change the format of trusted.pin xattr to JSON.
For example:
trusted.pin=[{"hsm":2}]
trusted.pin=[{"hsm":2},{"hsm":3},{"pool":"ddn_ssd"}]

Change-Id: I38aeb4ad960cf8324e77ac467f5e5cd742f9ec86
Signed-off-by: Lei Feng <flei@whamcloud.com>
Test-Parameters: trivial testlist=sanity-pcc
Reviewed-on: https://review.whamcloud.com/47161
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15593 tests: fix version checks for backported patches

Check for 2.14.0.x versions instead of 2.14.5x versions:
   conf-sanity.sh: test_123ag
   sanity.sh: test_27S, test_64i, test_160q, test_413e
   sanity-flr.sh: test_44e
   sanity-sec.sh: test_60

Test-Parameters: trivial testlist=conf-sanity env=ONLY=123
Test-Parameters: testlist=sanity-flr,sanity-sec
Fixes: 32f0bbe2646a ("LU-15176 sec: allow subdir mount of encrypted dir")
Fixes: d053ce30ca2f ("LU-15593 mdt: Add option to disable use of SOM")
Fixes: a35ae4ec46ec ("LU-15142 lctl: fixes for set_param -P and llog_print")
Fixes: d35aa94e79de ("LU-14448 lod: verify LOV before set/inherit")
Fixes: 276182ae9b95 ("LU-13055 changelog: use default mask if server has no mask")
Fixes: 930d4fe4952f ("LU-9704 grant: ignore grant info on read resend")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibd0e01ff5ac3b4a810b3eb5b8052e8a727ca416d
Reviewed-on: https://review.whamcloud.com/47206
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-14889 lproc: Add server checksum_type

This patch adds 'checksum_type' lproc entries under server:
1. obdfilter.$FSNAME-OST${count}.checksum_type
2. mdt.$FSNAME-MDT${count}.checksum_type

Test-case: sanity/77o added

Lustre-change: https://review.whamcloud.com/44755
Lustre-commit: c18d5d892b62ccee3cc9ec20a2278f17bb2d2f2c

Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I12a26f5c8c8f2f93d57f377626b1753fc13ffbb3
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/47142
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

EX-5199 sec: reinstate null encryption for file names

Reinstate null encryption for file names by adding a new llite
parameter named 'enable_filename_encryption', set to 0 by default.
When this parameter is 0, new empty directories configured as
encrypted ignore the filenames_encryption_mode and use
LLCRYPT_MODE_NULL instead, which is a no-op. This LLCRYPT_MODE_NULL
mode is inherited for all subdirectories and files.
When this parameter is 1, new empty directories configured as
encrypted use the normal encryption mode.

To set this parameter globally for all clients, do on the MGS:
mgs# lctl set_param -P llite.*.enable_filename_encryption=0

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9d726ba26cc91a51690d59a81efe3eb98ee2995c
Reviewed-on: https://review.whamcloud.com/47184
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15810 sec: fix enc dir migration

Now that the encryption context is stored in an xattr named
"encryption.c" instead of "security.c", we need to fetch this xattr
explicitly in case of encrypted directory migration. Indeed, there is
no xattr handler in ldiskfs for this "encryption." xattr type, so it
is not returned when listing all xattrs to migrate.

Lustre-change: https://review.whamcloud.com/47201
Lustre-commit: TBD (540f9208a9f6edcf6fdd1711e86fc182dfb92298)

Fixes: 4231fab66e ("LU-13717 sec: make client encryption compatible with ext4")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I628f9b253e86343db0b71f6a5b1ad2c5728ca38d
Reviewed-on: https://review.whamcloud.com/47207
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-12373 utils: fix llapi_pcc_del_internal() flags

The pch_flags value used by llapi_pcc_del_internal() was declared
as a "bool" but is used to pass a __u32 flags value from the caller.
This coincidentally works in the case where "flags = 1" (which is
the only value currently in use), but would break in confusing ways
in the future.

Change pch_flags to be "enum lu_pcc_cleanup_flags" to make it clear
what possible flag values are being used in this interface.

Fixes: 871da4898405 ("LU-12373 pcc: uncache the pcc copies when remove a PCC backend")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I41a008856af396376303fb156e2a92bce53ebbe5
Reviewed-on: https://review.whamcloud.com/47123
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Feng, Lei <flei@whamcloud.com>

LU-15761 obdclass: fix locking in llog_cat_refresh()

the patch fixes two problems:
1) pairing up_write() should be used with cathandle
2) llog_read_header() manipulates loghandle's internal
   structures (header, last_idx, etc) which are supposed
   to stay consistent from another user's point of view
   (like llog_add_rec())

Lustre-change: https://review.whamcloud.com/47185
Lustre-commit: 69cdcbe3cf732124b3b5be1a1f235aa1b78f6c85

Fixes: 71f409c9b31b ("LU-11418 llog: refresh remote llog upon -ESTALE")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib86e10a925b541d02c22d74e6ddbc4368345ac11
Reviewed-on: https://review.whamcloud.com/47190
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15815 llite: disable fast_read and workaround

Revert the fast_read stale data workaround from LU-14541 and disable
fast_read by default. The workaround causes applications to receive
spurious SIGBUGs when reclaim is concurrent with mmap page fault
handlers. We disable fast read to avoid the stale data issue entirely.

This reverts commit f2a16793fa4316fc9ccdc46bcfe54f6b8d1e442b and
re-exposes us to the consistency issues described in LU-14541.

Lustre-change: https://review.whamcloud.com/47204

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I716b1fd6ab22242b9267b8883f0371a360aaecef
Reviewed-on: https://review.whamcloud.com/47205
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

RM-620 build: New tag 2.14.0-ddn41 and kernel ddn17

Increase kernel build version to -ddn17.

New tag 2.14.0-ddn41

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3ebbe53c8a8d3c02d2831b8ac6d3068c0b3ebbe5

RM-620 build: New tag 2.14.0-ddn40

New tag 2.14.0-ddn40

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3a069f3c8a8d3c02d2831b8ac6d3068c0be50a64

DDN-1956 tests: exercise laudit/laudit-report

Add sanity-laudit.sh to exercise laudit and laudit-report command
line tools.

Test-Parameters: trivial
Test-Parameters: mdtcount=2 testlist=sanity-laudit serverextra_install_params="--packages lipe-client" facet=mds1
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I97fefecab6442d9031de6a7fcd86d07a517386c2
Reviewed-on: https://review.whamcloud.com/46907
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14156 utils: mirror split to check for last in-sync early

currently this check to prevent last in-sync component is done
once the file is open with O_RDWR which interrupts on-going
resync/extend process. instead we can do this check early once
the layout is fetched (after the first open with O_RDONLY).

Lustre-change: https://review.whamcloud.com/40782
Lustre-commit: TBD (from 9996c22e346c219df5b165eb5ceab09873303bc2)

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Iee08d23008b44d2a7b2127358116a95ace40b7dd
Reviewed-on: https://review.whamcloud.com/47058
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15733 kernel: kernel update RHEL7.9 [3.10.0-1160.62.1.el7]

Update RHEL7.9 kernel to 3.10.0-1160.62.1.el7.

Test-Parameters: trivial clientdistro=el7.9 serverdistro=el7.9

Change-Id: I063ebf0434dceb023847f49f37d6217f5c719b58
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/47063
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-15536 utils: add lfs somsync utility

This patch adds lfs somsync utility to synchronize
SOM xattr(s) for given FILE(s) or FID(s).

lfs somsync FILE ...
lfs somsync --by-fid MOUNT FID ...

Test-Parameters: trivial env=ONLY=807 testlist=sanity

Change-Id: Ie9ee39625d56ec026c89dcc0f27025904ca354e3
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/46942
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-14110 obdclass: Protect cl_env_percpu[]

cl_env_percpu is not protected against multi client mounts on the
same node: "keys_fill" could be called with the same cl_env_percpu
context by several mount processes (race on lu_context.lc_value).

This patch add a mutex for cl_env_percpu to proctect contexts
"refill".

Lustre-change: https://review.whamcloud.com/40565
Lustre-commit: 881551fbb7335694b89a877072bcda0aeaf8705c

Signed-off-by: Etienne AUJAMES <eaujames@ddn.com>
Change-Id: Icfd6f3715899fa4ac5279e932f462e7cf29d98bd
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/47073
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>