From 36629c15fbd0e2d88e813f4d8af392b71da8147d Mon Sep 17 00:00:00 2001 From: Patrick Farrell Date: Thu, 25 Apr 2024 22:27:26 -0400 Subject: [PATCH] LU-17779 lnet: use copy_page() When copying one page to another in kernel memory, the kernel has an optimized copy_page which can be used instead of memcpy(). This is relevant in the lnet loopback subsystem, which copies from the kiov from the client to that on the server. (Using the same page is nasty for a lot of reasons, so copying is best.) So we can check for the full page to full page copy and use that. On my little tiny VM system, this improves maximum write performance (with the fake write fail_loc enabled) by about 20%, from 4.4 GiB/s to 5.7 GiB/s. We should also eventually be able to add a fake copy to the fake read/fake write fail loc, but that's a bit tricky, so will be left out of this patch. Signed-off-by: Patrick Farrell Change-Id: Iea89447ed03bd4646544883b588873700f6e09a4 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54923 Tested-by: jenkins Tested-by: Maloo Reviewed-by: Oleg Drokin Reviewed-by: Andreas Dilger Reviewed-by: Chris Horn --- lnet/lnet/lib-move.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lnet/lnet/lib-move.c b/lnet/lnet/lib-move.c index 63c987b..add5a27 100644 --- a/lnet/lnet/lib-move.c +++ b/lnet/lnet/lib-move.c @@ -14,6 +14,7 @@ #define DEBUG_SUBSYSTEM S_LNET #include +#include #include #include @@ -404,7 +405,11 @@ lnet_copy_kiov2kiov(unsigned int ndiov, struct bio_vec *diov, * However in practice at least one of the kiovs will be mapped * kernel pages and the map/unmap will be NOOPs */ - memcpy (daddr, saddr, this_nob); + if (this_nob == PAGE_SIZE && !diov->bv_offset && !doffset && + !siov->bv_offset && !soffset) + copy_page(daddr, saddr); + else + memcpy (daddr, saddr, this_nob); nob -= this_nob; if (diov->bv_len > doffset + this_nob) { -- 1.8.3.1