LU-14776 zfs: fix Ubuntu 20 HWE build issues With newer Ubuntu systems using ZFS dkms have the following build errors: In file included from zfs/2.0.2/source/include/sys/arc.h:32, from lustre/osd-zfs/osd_internal.h:50, from lustre/osd-zfs/osd_handler.c:51: zfs/2.0.2/source/include/sys/zfs_context.h:45:10: fatal error: sys/types.h: No such file or directory 45 | #include <sys/types.h> | ^~~~~~~~~~~~~ compilation terminated. This is due to layout of the tree containing the needed headers. Include those paths in build system. Test-Parameters: trivial Change-Id: I453830c4111ad88ec655d3d7d0ee51627331cb0b Signed-off-by: James Simmons <jsimmons@infradead.org> Reviewed-on: https://review.whamcloud.com/44749 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Jian Yu <yujian@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12745 build: Account for optional SPL for ZFS 0.8+ With ZFS 0.8.0 and later, SPL is not longer present. Some zfs packages provide vestigial spl package contents, but zfs-dkms does not. This makes testing SPL directories optional depending on version of ZFS, this also accounts for the new location of the spl include directory under the zfs include directory. Test-Parameters: trivial Signed-off-by: Nathaniel Clark <nclark@whamcloud.com> Change-Id: I8afcff079f25543a3c86df0c404146a859b226aa Reviewed-on: https://review.whamcloud.com/36161 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-7585 zfs: OI scrub for ZFS The ZFS backend OI scrub is used for verifying OI mappings consistency. ZFS has some mechanism to maintion the data integrity, but there is still possible data corruption, especially consider the data migration from other backend, such as ldiskfs, or server side data backup and restore. The OI scrub can check OI mappings consistency and rebuild them when needed. The ZFS backend OI scrub shares the same control interface as ldiskfs backend. It can be triggered manually via the lctl command: lctl lfsck_start -M $device -t scrub It also can be triggered automatically when inconsistency detected if you do not disable 'auto_scrub' that can be controlled via: lct set_param -n osd-zfs.*.auto_scrub_interval=xxx You can check the OI scrub status similar as you do for ldiskfs backend: lctl get_param -n osd-zfs.*.oi_scrub Test-Parameters: envdefinitions=SLOW=yes testlist=sanity-scrub mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4 Signed-off-by: Fan Yong <fan.yong@intel.com> Change-Id: I59ae3142ecd7b27f48b14f2a2d1d110d9c8296e3 Reviewed-on: https://review.whamcloud.com/30909 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4821 osd: cleanups in osd-zfs many small changes to get rid of udmu wrappers. Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Change-Id: Ic8746345da1e6695149bacf066be10bf284aecdf Reviewed-on: http://review.whamcloud.com/9721 Tested-by: Jenkins Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Mike Pershin <mike.pershin@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3117 build: zfs-0.6.1 kmod+dkms compatibility With the release of zfs-0.6.1 the default install location of the zfs kmod headers and objects has changed. The kmod headers which are common for a zfs version are now installed under /usr/src/zfs-<version>/ path. The objects, which are kernel specific, are installed under /usr/src/zfs-<version>/<kernel>/. This was done just prior to the official 0.6.1 release because this scheme satisfies the packaging requirements of the major distributions. Making the change now means we shouldn't need to change it again. To accomidate this change the lustre-build-zfs.m4 has been updated in the following ways: * The new zfs header and object paths were added to the list of default search paths. The DKMS build paths were also added to allow compilation against zfs-kmod or zfs-dkms packages. * Support for building the spl and zfs code recursively as part of the Lustre build process was removed. * The lustre-osd-zfs packages 'Requires' line was changed to require zfs-kmod. Either the zfs-kmod or zfs-dkms packages can be used to satisfy this requirement. * Fix incorrect usage of @ZFS_OBJ@ in osd-zfs/Makefile.in, the include directory us under @ZFS@ with the headers. These happens to be the same location before so it never caused issues. * EXTRA_LIBZFS_INCLUDE renamed ZFS_LIBZFS_INCLUDE, this was done for consistency. * Failing to build ldiskfs should not automatically disable all server support. The zfs osd may still be buildable. * General m4 cleanup and simplification of lustre-build-zfs.m4. * Ensure new zfs/spl build correctly with lbuild. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Change-Id: Ib686211c4f9ace39a41053ce8a20112d1121def9 Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com> Reviewed-on: http://review.whamcloud.com/5960 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1930 build: Back end file system fixes. Currently Lustre for ZFS requires the zfs development rpm for its userland support to be installed on the build machine so we can create the lustre zfs utilities. What this patch does is allow a user to be able to build against the zfs/spl source drops as well as the rpms. A work around is provided so we can point the lustre build system to were a user can temporary install the zfs user land headers. Signed-off-by: James Simmons <uja.ornl@gmail.com> Change-Id: I8e5586ed22956a9dd4799826a442b8f5a895d872 Reviewed-on: http://review.whamcloud.com/3980 Reviewed-by: Brian J. Murrell <brian.murrell@intel.com> Tested-by: Hudson Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1182 osd-zfs: add accounting support Create QSD instance in ZFS OSD so that space accounting data are exported via procfs. Signed-off-by: Johann Lombardi <johann@whamcloud.com> Change-Id: Ie44a8032d437beadcd0cf337f6271247538946c4 Reviewed-on: http://review.whamcloud.com/3201 Tested-by: Hudson Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
LU-1305 osd: Makefiles for osd-zfs Makefiles ... Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Change-Id: I1fdf9eecf98306b0cfe0bbc0611df3a977f48b10 Reviewed-on: http://review.whamcloud.com/2972 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Fan Yong <yong.fan@whamcloud.com>