Whamcloud - gitweb
EX-6127 osc: osc brw request compression
authorArtem Blagodarenko <ablagodarenko@ddn.com>
Wed, 30 Nov 2022 14:54:57 +0000 (14:54 +0000)
committerAndreas Dilger <adilger@whamcloud.com>
Fri, 23 Jun 2023 23:35:28 +0000 (23:35 +0000)
commitf43b9ce9afaee73ae2537cbe32994850fe2964b6
tree1ea66fd7f6210d08a7c32dbd9beb005b3caff0f5
parent0c65a5b787e4a0e85d93857d327f6795d827e395
EX-6127 osc: osc brw request compression

This patch adds client-side compression/decompression.

The client-side data compression project (CSDC) reduces storage and network
utilization by leveraging the more plentiful memory and CPU resources on the
local client. Data is sent compressed over the network, saved directly to
storage on the server side, and decompressed back on the client side.
Uncompressed data is kept in client page cache, all while being functionally
transparent to the end user and application.

As an example, a test file is compressed and decompressed.
The resulting file is compared with the original one.

The test case shows 2.5x compression ratio:
356K /mnt/lustre/d460.sanity/sanity.sh
884K /tmp/cmp-46ofie/decompressed_sanity.sh

Compression should read whole chunk even if offset
and size differ.

Let's modify readahead to force reading data from the
offset and size multiple to the chunk size.

Test-Parameters: testlist=sanity env=ONLY=460
Signed-off-by: Artem Blagodarenko <ablagodarenko@ddn.com>
Change-Id: I9b41ab815db3df9ad7bdea5fca4c093cbda8814b
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/49511
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
20 files changed:
lustre/include/cl_object.h
lustre/include/lustre_osc.h
lustre/include/obd.h
lustre/include/obd_support.h
lustre/include/uapi/linux/lustre/lustre_idl.h
lustre/include/uapi/linux/lustre/lustre_user.h
lustre/llite/dir.c
lustre/llite/file.c
lustre/llite/rw.c
lustre/lov/lov_io.c
lustre/lov/lov_object.c
lustre/osc/Makefile.in
lustre/osc/osc_compress.c [new file with mode: 0644]
lustre/osc/osc_compress.h [new file with mode: 0644]
lustre/osc/osc_internal.h
lustre/osc/osc_request.c
lustre/ptlrpc/wiretest.c
lustre/tests/sanity.sh
lustre/utils/wirecheck.c
lustre/utils/wiretest.c