LU-13132 osd: osd-zfs to cache dbufs for llog objects

LU-13132 osd: osd-zfs to cache dbufs for llog objects

working set for llog objects is tiny and very predictable. osd-zfs
can cache couple dbufs (first block storing the header and last
block for new records).

for sanity/60a (llog test) it gives 5939307 hits and 5776 misses
while average osd_write() goes down from 1.09 usec to 0.27 usec,
total time for sanity/60a: before - 153s, after - 101s.

this approach can be used in few other cases like last_rcvd.

Lustre-change: https://review.whamcloud.com/37222
Lustre-commit: 11a89c5ec16685fda91dd7c052b72012833c2f88

Was-Change-Id: Icc0126658894085d33ef79ae41ac6c1ed4140f4c

LU-16479 utils: Add option to manage degraded ZFS OST

Add new Lustre specific ZFS dataset user property to
control/manage degraded ZFS OSTs, also modify the existing
lustre/scripts/statechange-lustre.sh zedlet accordingly.
Extend the same to mkfs.lustre utility to add this property
by default when creating a new Lustre ZFS server.

Lustre-change: https://review.whamcloud.com/49660
Lustre-commit: a2de6af65d21bff0d9357c30e6eb4ba049ff2059

Was-Change-Id: I7032538f507c9ad20d5b109b54e3c3bab8138458

HPE-bug-id: LUS-11447
Signed-off-by: Akash B <akash-b@hpe.com>
LU-14918 osd: don't declare similar zfs writes twice

in some cases (like overstriping) the same operations can be
declared multiple times (new llog records) and this lead to
huge number of credits and performance degradation. we can
avoid this checking for duplicate declarations.
notice each declare operation results in a allocation in ZFS.

the example for an overstriped file (2000 stripes over 4 OSTs),
declare ops			before	after
create:				2001	2
unlink:				10001	10

creation of 1K-stripe files (over 4 OSTs) is 2.5% faster.
removal of 1K-stripe files is 44% faster.

single-stripe file creation/removal does not degrade.

Lustre-change: https://review.whamcloud.com/49701
Lustre-commit: c1936c9d294d53ff39741e1b07ffc74f51fcddb6

Was-Change-Id: I5d9e6d3a1574ccd7bf97fd3a67ab4fff0b6a352c

LU-14918 osd: don't declare similar ldiskfs writes twice

in some cases (like overstriping) the same operations can be
declared multiple times (new llog records) and this lead to
huge number of credits and performance degradation. we can
avoid this checking for duplicate declarations.
As every declaration would need an allocation, limit the scope
of this checks to transaction likely to be large.

% of "large" transaction in sanity-benchmark, depending on threshold:

  creates < 5 && writes < 5:
  0.58% (mds1) and  2.97% (mds2)

  create < 7 & writes < 7:
  0.58% and 2.4%

  create < 9 & writes  < 9:
  0.6% and 1.85%

  create < 10 & write2 < 10:
  0.0004% and 0.000001%

thus 10 creates or writes is selected as a threshold to enable this
logic.

Lustre-change: https://review.whamcloud.com/45765
Lustre-commit: 9e6225b2e7385cbb7be0474df01075fafc4966d5

Was-Change-Id: I7c893fe3b95646b4b813b999bc832659dfcf03ad

LU-15642 obdclass: use consistent stats units

Use consistent stats units, since some were "usec" and others "usecs".
Most stats already use LPROCFS_TYPE_* to encode type stats type, so
use this to provide units for those stats, and only explicitly provide
strings for the few stats that don't match the commonly-used units.
This also reduces the number of repeat static strings in the modules.

Lustre-change: https://review.whamcloud.com/46833
Lustre-commit: b515c6ec2ab84598c77c65eb78f1afd5e67b1ede

Was-Change-Id: I25f31478f238072ddbf9a3918cd43bb08c3ebbe5

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
LU-16150 zfs: Fix ZFS(2.1.99-1) build error on CentOS (3.10)

ZFS: (2.1.99-1)
Lustre: 27723374a38 LU-16073 utils: double snapshot_mount fix
CentOS: 3.10.0-1160.15.2.el7.x86_64

This patch fixes build failures seens as below for the
above configuration:

First:
make[4]: Entering directory `/root/lustre01/lustre-release/lustre/utils'
gcc  -rdynamic -shared -export-dynamic -pthread \
-L/root/zfs/zfs_git_lustre_build/zfs//lib/libzfs/.libs/
-L/root/zfs/zfs_git_lustre_build/zfs//lib/libnvpair/.libs/
-L/root/zfs/zfs_git_lustre_build/zfs//lib/libzpool/.libs/ -o
mount_osd_zfs.so \
`ar -t libmount_utils_zfs.a` \
-ldl   -lzfs -lnvpair -lzpool
/usr/bin/ld: cannot find -lzfs
/usr/bin/ld: cannot find -lnvpair
/usr/bin/ld: cannot find -lzpool
collect2: error: ld returned 1 exit status

Lustre-change: https://review.whamcloud.com/48536
Lustre-commit: 448963c9a33dbf0e0988ceeb407027f2488e7f42

Was-Change-Id: I32f270c7912379f7dce940e0aa2bceee5e49ad79

Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
LU-15611 osd-zfs: Cleanup while mount failed

Need clean up in error out path in osd-zfs.

Lustre-change: https://review.whamcloud.com/46678
Lustre-commit: 9b973ad37f66a10eb7db1ced6865708497ecc02b

Was-Change-Id: I47d9ee9483acb8e1d60c77e8cfc481902a1535ac

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Test-Parameters: optional fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el9.5 serverdistro=el8.10 testgroup=full-dne-zfs-part-1
Test-Parameters: optional fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el9.5 serverdistro=el8.10 testgroup=full-dne-zfs-part-2
Test-Parameters: optional fstype=zfs mdtcount=4 mdscount=2 \
  clientdistro=el9.5 serverdistro=el8.10 testgroup=full-dne-zfs-part-3

Change-Id: Icc0126658894085d33ef79ae41ac6c1ed4140f4c
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/59123
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>