Whamcloud - gitweb
LU-13799 llite: Move free user pages 43/39443/29
authorPatrick Farrell <pfarrell@whamcloud.com>
Fri, 30 Jul 2021 16:13:45 +0000 (12:13 -0400)
committerOleg Drokin <green@whamcloud.com>
Wed, 26 Jan 2022 05:14:15 +0000 (05:14 +0000)
commit7f9b8465bc1125e51aa29cdc27db9a9d6fdc0b89
tree717f941f73bddb06a4ffc939e8dfaf1778b5b861
parent6e59408f1a1c0679e8bfce0c3d9e14137a45abe1
LU-13799 llite: Move free user pages

It is incorrect to release our reference on the user pages
before we're done with them - We need to keep it until the
i/o is complete, otherwise we access them after releasing
our reference.  This has not caused any known bugs so far,
but it's still wrong.

So only drop these references when we free the aio struct,
which is only freed once i/o is complete.

Also rename free_user_pages to release_user_pages, because
it does not free them - it just releases our reference.

This also helps performance by moving free_user_pages to
the daemon threads.  This is a 5-10% boost.

This patch reduces i/o time in ms/GiB by:
Write: 18 ms/GiB
Read: 19 ms/GiB

Totals:
Write: 180 ms/GiB
Read: 178 ms/GiB

mpirun -np 1  $IOR -w -r -t 64M -b 64G -o ./iorfile --posix.odirect

With previous patches in series:
write     5183 MiB/s
read      5201 MiB/s

Plus this patch:
write        5702 MiB/s
read         5756 MiB/s

Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ibfe808611bbe6743a1b5fe3aa6a8d42691256d22
Reviewed-on: https://review.whamcloud.com/39443
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
lustre/include/cl_object.h
lustre/llite/rw26.c
lustre/obdclass/cl_io.c