Whamcloud - gitweb
EX-7585 pcc: parallel data copy for attach
This patch parallelize the data copying work for pcc attach
by using multiple threads in the ll_fid_path_copy helper.
Nvidia provided performance numbers for this from their
environment. This was with 4 MiB I/O size, they reported
speed was similar but *slightly* lower at larger block
sizes. This is probably an EXT4 limitation since Lustre
speed scales with those larger sizes. (As PCC attach is a
copy from Lustre to EXT4.)
This is for attaching a single 2 TiB file, they also
reported no performance regression for datasets with many
small files.
threads: 1 2 4 8
speed: 4 GiB/s 7.8 GiB/s 14.1 GiB/s 15.2 GiB/s
Performance improved only very slightly past 8 threads,
and 4 threads is clearly the sweet spot for performance.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iffbb3892cfb5b2e71afe15d03f9aec9c84975092
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51171
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>