Whamcloud - gitweb
LU-10092 llite: Add persistent cache on client 63/32963/38
authorLi Xi <lixi@ddn.com>
Tue, 27 Jun 2017 12:18:14 +0000 (20:18 +0800)
committerOleg Drokin <green@whamcloud.com>
Thu, 13 Jun 2019 04:29:12 +0000 (04:29 +0000)
commitf172b116885753d0f316549a2fb9d451e9b4bd2e
tree7b73b056da1191778af8cd59cc185a6386bc9dab
parent225e7b8c70fb68bc3aa3a6d88c5e9bda322c9cc9
LU-10092 llite: Add persistent cache on client

PCC is a new framework which provides a group of local cache
on Lustre client side. No global namespace will be provided
by PCC. Each client uses its own local storage as a cache for
itself. Local file system is used to manage the data on local
caches. Cached I/O is directed to local filesystem while
normal I/O is directed to OSTs.

PCC uses HSM for data synchronization. It uses HSM copytool
to restore file from local caches to Lustre OSTs. Each PCC
has a copytool instance running with unique archive number.
Any remote access from another Lustre client would trigger
the data synchronization. If a client with PCC goes offline,
the cached data becomes inaccessible for other client
temporarilly. And after the PCC client reboots and the copytool
restarts, the data will be accessible again.

ToDo:
1) Make PCC exclusive with HSM.
2) Strong size consistence for PCC cached file among clients.
3) Support to cache partial content of a file.

Change-Id: I188ed36c48aae223380739f607cc6caf2f789298
Test-Parameters: clientcount=3 testlist=sanity-pcc,sanity-pcc,sanity-pcc
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/32963
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
54 files changed:
lustre/autoconf/lustre-core.m4
lustre/doc/Makefile.am
lustre/doc/lctl-pcc.8 [new file with mode: 0644]
lustre/doc/lctl.8
lustre/doc/lfs-pcc.1 [new file with mode: 0644]
lustre/doc/lfs.1
lustre/doc/llapi_pcc_attach.3 [new file with mode: 0644]
lustre/doc/llapi_pcc_detach_fid.3 [new file with mode: 0644]
lustre/doc/llapi_pcc_detach_fid_fd.3 [new file with mode: 0644]
lustre/doc/llapi_pcc_detach_fid_str.3 [new file with mode: 0644]
lustre/doc/llapi_pcc_detach_file.3 [new file with mode: 0644]
lustre/doc/llapi_pcc_state_get.3 [new file with mode: 0644]
lustre/doc/llapi_pcc_state_get_fd.3 [new file with mode: 0644]
lustre/doc/llapi_pccdev_get.3 [new file with mode: 0644]
lustre/doc/llapi_pccdev_set.3 [new file with mode: 0644]
lustre/include/lustre/lustreapi.h
lustre/include/lustre_compat.h
lustre/include/md_object.h
lustre/include/obd.h
lustre/include/uapi/linux/lustre/lustre_idl.h
lustre/include/uapi/linux/lustre/lustre_user.h
lustre/llite/Makefile.in
lustre/llite/dir.c
lustre/llite/file.c
lustre/llite/llite_internal.h
lustre/llite/llite_lib.c
lustre/llite/llite_mmap.c
lustre/llite/lproc_llite.c
lustre/llite/namei.c
lustre/llite/pcc.c [new file with mode: 0644]
lustre/llite/pcc.h [new file with mode: 0644]
lustre/llite/super25.c
lustre/llite/xattr.c
lustre/llite/xattr26.c
lustre/lmv/lmv_intent.c
lustre/lmv/lmv_obd.c
lustre/mdc/mdc_lib.c
lustre/mdd/mdd_dir.c
lustre/mdt/mdt_lib.c
lustre/mdt/mdt_open.c
lustre/tests/Makefile.am
lustre/tests/replay-vbr.sh
lustre/tests/sanity-hsm.sh
lustre/tests/sanity-pcc.sh [new file with mode: 0644]
lustre/tests/sanity-quota.sh
lustre/tests/test-framework.sh
lustre/tests/test-groups/regression
lustre/utils/Makefile.am
lustre/utils/lctl.c
lustre/utils/lfs.c
lustre/utils/lhsmtool_posix.c
lustre/utils/liblustreapi_pcc.c [new file with mode: 0644]
lustre/utils/obd.c
lustre/utils/obdctl.h