Whamcloud - gitweb
LU-13134 obdclass: use slab allocation for cl_page
Currently we use kmalloc() for cl_page allocation, this is because
cl_page size could be different with different objects.
For most cases (except obdecho object) we have 408 bytes for cl_page.
kmalloc() uses 512 bytes slab for it. So for 4KB PAGE_SIZE, we could
allocate 8 cl_page in a single page.
If we use private slab cache, it can fit 10 cl_page for 4KB PAGE_SIZE,
and even more struct cl_page can fit if we reduce the struct size.
For 4GB of pages the cl_page memory usage will shrink from 512M to 408M.
With clients already having 100GB of memory, it will save us 2.5GB
memory usage for cl_page, and some systems have as much as 24TB today!
This patch uses a static array with N (currently 16) entries storing
the { size, slab pointer }, with only 2-3 entries in current usage.
If the static array is not large enough, then fall back to kmalloc().
Benchmark numbers:
4KB Random Read
fio -iodepth=128 -direct=1 -size=2g -runtime=60 -numjobs=256
-group_reporting -directory=/ai400/out -create_serialize=0
-filename_format='f.$jobnum.$filenum'
master+ master+
LU-4198 LU-4198+
LU-13134 delta
QD=1, numjobs=1 4518 4538 +0.44%
QD=128, numjobs=256 2177K 2207k +1.38%
Change-Id: I565b09616b22706f93d4c0fdc0df396d06cd51cc
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37225
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>