Whamcloud - gitweb
EX-6206 lustre: add lgzip kernel module
authorSebastien Buisson <sbuisson@ddn.com>
Fri, 23 Jun 2023 11:58:39 +0000 (13:58 +0200)
committerAndreas Dilger <adilger@whamcloud.com>
Sat, 1 Jul 2023 10:34:56 +0000 (10:34 +0000)
commitd91815046c83b3a29a76cf6bdd972655bda7bbc4
tree7347b6d5d9eccc09c4f0cd57aee87aba5696f979
parent47209c7689f9994025a47516e2dd2c9796282c9d
EX-6206 lustre: add lgzip kernel module

lgzip kernel module implements compression according to the
deflate/zlib algorithm, through the kernel Crypto API.
It provides 2 cipher drivers under the generic name 'deflate':
* deflate-lustre-generic of type compression
* deflate-lustre-scomp of type scomp

Note the 'deflate' name is identical to the in-kernel module, but
lgzip registers it with a slightly higher .cra_priority = 110, so that
it is preferred over the in-kernel module.
Our 'deflate' is also different in that it accepts a compression level
as explained below.

lgzip kernel module sources are copied from linux v6.2-rc5 and renamed
to gzip.c to avoid name collisions. It implements the Crypto API
interface, and rely on the deflate/zlib kernel library for compression
implementation. It has been modified to grok a compression level, as
read from the top 4 bits of the crypto_tfm flags, and pass it to the
underlying library.
The deflate/zlib library sources are also copied from linux v6.2-rc5
and built statically. Headers have also been copied from linux
v6.2-rc5 for consistency, and source files modified to include the
copied headers instead of the system headers.
All aforementioned sources are located in the lustre/gzip directory.
The lgzip module is always built with Lustre.

This patch enhances the test kernel module kcompr.ko to exercise the
compression level of the provided 'deflate' module.
It also tries to manually load the lgzip kernel module when a file
requires compression with the 'delfate' alg. This is a "one-shot" try
that allows us to prefer our module that has level support, but
continues to at least compress/decompress files even if our own module
is not available.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I2b0386457bc781d91816172dea6b52ce3dd273f4
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51422
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
29 files changed:
debian/dkms.conf.in
lustre/Makefile.in
lustre/autoMakefile.am
lustre/autoconf/lustre-core.m4
lustre/gzip/Makefile.in [new file with mode: 0644]
lustre/gzip/autoMakefile.am [new file with mode: 0644]
lustre/gzip/deflate.c [new file with mode: 0644]
lustre/gzip/deftree.c [new file with mode: 0644]
lustre/gzip/defutil.h [new file with mode: 0644]
lustre/gzip/gzip.c [new file with mode: 0644]
lustre/gzip/inffast.c [new file with mode: 0644]
lustre/gzip/inffast.h [new file with mode: 0644]
lustre/gzip/inffixed.h [new file with mode: 0644]
lustre/gzip/inflate.c [new file with mode: 0644]
lustre/gzip/inflate.h [new file with mode: 0644]
lustre/gzip/inftrees.c [new file with mode: 0644]
lustre/gzip/inftrees.h [new file with mode: 0644]
lustre/gzip/infutil.h [new file with mode: 0644]
lustre/gzip/scompress.h [new file with mode: 0644]
lustre/gzip/zconf.h [new file with mode: 0644]
lustre/gzip/zlib.h [new file with mode: 0644]
lustre/gzip/zutil.h [new file with mode: 0644]
lustre/include/lustre_crypto.h
lustre/lz4/llz4.c
lustre/lz4/llz4hc.c
lustre/osc/osc_compress.c
lustre/scripts/dkms.mkconf
lustre/scripts/lustre_rmmod
lustre/tests/kernel/kcompr.c