Whamcloud - gitweb
LU-18813 osd-wbcfs: MemFS-based OSD with writeback support 39/58439/33
authorQian Yingjin <qian@ddn.com>
Mon, 17 Mar 2025 15:41:17 +0000 (23:41 +0800)
committerOleg Drokin <green@whamcloud.com>
Wed, 14 May 2025 03:54:41 +0000 (03:54 +0000)
commit25813cf8ba532ca1c690be6f9659c09a70722536
tree0f9351ad75416b7a1f82c1660978f454f2b0d9a7
parent38c013c04e42ff654750c2fa951a35090616cd30
LU-18813 osd-wbcfs: MemFS-based OSD with writeback support

Implement a memory filesystem based OSD with writeback support for
Lustre.
It borrows lots of design from memory-based file systems such as
tmpfs/ramfs. The data is first written into the memory-based file
system (called MemFS for short). And then, the data can be flushed
to the persistent storage in a delayed write-back manner.

This patch implemented the basic functionality to store data in
MemFS. It can reuse lots of VFS codes in Linux kernel such as:
- Page caching for data;
- dcache for dentry management and lookup;
- icache for inode management and lookup;
- Writeback mechanism in Linux kernel

Test-Parameters: testlist=sanity fstype=wbcfs mdscount=1 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs mdscount=1 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs mdscount=4 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs mdscount=4 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs combinedmdsmgs=false standalonemgs=true mdscount=1 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs combinedmdsmgs=false standalonemgs=true mdscount=1 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs combinedmdsmgs=false standalonemgs=true mdscount=4 mdtcount=1 osscount=4 ostcount=1
Test-Parameters: testlist=sanity fstype=wbcfs combinedmdsmgs=false standalonemgs=true mdscount=4 mdtcount=1 osscount=4 ostcount=1
Signed-off-by: Yingjin Qian <qian@ddn.com>
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ia07c1d95b7ad3f7f5e817a8de69d0a4ab6995ffa
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/58439
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
31 files changed:
MAINTAINERS
config/lustre-build.m4
config/lustre-core.m4
lustre.spec.in
lustre/Makefile.in
lustre/autoMakefile.am
lustre/include/obd.h
lustre/include/uapi/linux/lustre/lustre_disk.h
lustre/mgs/lproc_mgs.c
lustre/osd-wbcfs/Makefile.in [new file with mode: 0644]
lustre/osd-wbcfs/TODO [new file with mode: 0644]
lustre/osd-wbcfs/autoMakefile.am [new file with mode: 0644]
lustre/osd-wbcfs/index.h [new file with mode: 0644]
lustre/osd-wbcfs/osd_dirent.c [new file with mode: 0644]
lustre/osd-wbcfs/osd_handler.c [new file with mode: 0644]
lustre/osd-wbcfs/osd_hash.c [new file with mode: 0644]
lustre/osd-wbcfs/osd_index_hash.c [new file with mode: 0644]
lustre/osd-wbcfs/osd_internal.h [new file with mode: 0644]
lustre/osd-wbcfs/osd_io.c [new file with mode: 0644]
lustre/osd-wbcfs/osd_object.c [new file with mode: 0644]
lustre/osd-wbcfs/wbcfs.c [new file with mode: 0644]
lustre/osd-wbcfs/wbcfs.h [new file with mode: 0644]
lustre/tests/sanity.sh
lustre/tests/test-framework.sh
lustre/utils/Makefile.am
lustre/utils/libmount_utils_wbcfs.c [new file with mode: 0644]
lustre/utils/mount_lustre.c
lustre/utils/mount_utils.c
lustre/utils/mount_utils.h
rpm/kmp-lustre-osd-wbcfs.files [new file with mode: 0644]
rpm/kmp-lustre-osd-wbcfs.preamble [new file with mode: 0644]