From: Nathan Rutman <nathan.rutman@hpe.com>
Date: Wed, 22 Feb 2023 22:34:09 +0000 (-0800)
Subject: LU-16587 utils: give lfs migrate a larger buffer
X-Git-Url: https://git.whamcloud.com/gitweb?a=commitdiff_plain;h=04526d8c500af1031a42cfedc4422bd6aa8c65ec;p=fs%2Flustre-release.git

LU-16587 utils: give lfs migrate a larger buffer

lfs migrate is slow because it mostly uses a small 1MB buffer. Bigify.

[root@kjlmo4n00 16G]# time lfs migrate -S 1M -p flash 16G.1
real	0m25.341s
[root@kjlmo4n00 16G]# time /root/tools/lfs_nzr migrate -S 1M -p flash 16G.1
real	0m6.526s

Lustre-change: https://review.whamcloud.com/50118
Lustre-commit: 23224e03dc30c89dd449de5a7fe99b0bd3aca495

Signed-off-by: Nathan Rutman <nathan.rutman@hpe.com>
Change-Id: I850ca475fcd0efe2d71d26e4d1544f462c60252a
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/ex/lustre-release/+/51352
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
---

diff --git a/lustre/utils/lfs.c b/lustre/utils/lfs.c
index 1dc7351..bccb772 100644
--- a/lustre/utils/lfs.c
+++ b/lustre/utils/lfs.c
@@ -1005,7 +1005,7 @@ static int migrate_copy_data(int fd_src, int fd_dst, int (*check_file)(int),
 			     long stats_interval_sec, off_t file_size_bytes)
 {
 	struct llapi_layout *layout;
-	size_t buf_size = 4 * 1024 * 1024;
+	size_t buf_size = 64 * 1024 * 1024;
 	void *buf = NULL;
 	off_t pos = 0;
 	off_t data_end = 0;
@@ -1023,8 +1023,14 @@ static int migrate_copy_data(int fd_src, int fd_dst, int (*check_file)(int),
 		uint64_t stripe_size;
 
 		rc = llapi_layout_stripe_size_get(layout, &stripe_size);
-		if (rc == 0)
-			buf_size = stripe_size;
+		if (rc == 0) {
+			/* We like big bufs */
+			if (stripe_size > buf_size)
+				buf_size = stripe_size;
+			else
+				/* Trim to stripe_size multiple */
+				buf_size -= buf_size % stripe_size;
+		}
 
 		llapi_layout_free(layout);
 	}