Whamcloud - gitweb
LU-12345 ldiskfs: optimize nodelalloc mode 38/37538/5
authorArtem Blagodarenko <artem.blagodarenko@seagate.com>
Tue, 28 May 2019 16:51:21 +0000 (19:51 +0300)
committerOleg Drokin <green@whamcloud.com>
Tue, 14 Apr 2020 17:55:45 +0000 (17:55 +0000)
commitf91552d85cd086b37f78524113cadf799c045220
tree9458fe176f5eaa3a88b6c6a340978789b612c9f7
parentcccd5c79c6697906c8409e867984f73d0ae8474d
LU-12345 ldiskfs: optimize nodelalloc mode

We found performance regression when using bigalloc with "nodelalloc"
(1MB cluster size):

1. mke2fs -C 1048576 -O ^has_journal,bigalloc /dev/sda
2. mount -o nodelalloc /dev/sda /test/
3. time dd if=/dev/zero of=/test/io bs=1048576 count=1024

The "dd" will cost about 2 seconds to finish, but if we mke2fs without
"bigalloc", "dd" will only cost less than 1 second.

The reason is: when using ext4 with "nodelalloc", it will call
ext4_find_delalloc_cluster() nearly everytime it call
ext4_ext_map_blocks(), and ext4_find_delalloc_range() will also scan
all pages in cluster because no buffer is "delayed".  A cluster has
256 pages (1MB cluster), so it will scan 256 * 256k pags when creating
a 1G file. That severely hurts the performance.

Therefore, we return immediately from ext4_find_delalloc_range() in
nodelalloc mode, since by definition there can't be any delalloc
pages.

The same optimization also added for ldiskfs_find_delayed_extent()
function that improve performance dromaticaly.

Here is results of testing on two node system.
Without the patch:
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.00    0.00   56.30    0.06    0.00   43.63

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sds               0.00     0.00    0.00 1174.00     0.00     4.59
8.00     0.84    0.71    0.00    0.71   0.01   1.20

With patch:
08/29/2018 01:13:22 AM
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
0.00    0.00    4.13   30.37    0.00   65.50

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s      wMB/s
avgrq-sz avgqu-sz   await r_await w_await  svctm %util
sds               0.00     0.00    0.00 54117.82     0.00     211.43
8.00   152.59    2.82    0.00    2.82   0.02 99.01

Lustre-change: https://review.whamcloud.com/34982
Lustre-commit: af48ae8bff289b2bc083a888efeafa3c48df91e2

Cray-bug-id: LUS-5835
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Change-Id: Ie33410d4481778ee4f76a054ab8cfc11cc19a0ed
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37538
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
16 files changed:
ldiskfs/kernel_patches/patches/rhel7/ext4-optimize-ext4_find_delalloc_range-in-nodelalloc.patch [new file with mode: 0644]
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.2.series
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.3.series
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.4.series
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.5.series
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.6.series
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.7.series
ldiskfs/kernel_patches/series/ldiskfs-3.10-rhel7.series
ldiskfs/kernel_patches/series/ldiskfs-3.12-sles12.series
ldiskfs/kernel_patches/series/ldiskfs-3.12-sles12sp1.series
ldiskfs/kernel_patches/series/ldiskfs-4.4-sles12sp2.series
ldiskfs/kernel_patches/series/ldiskfs-4.4-sles12sp3.series
ldiskfs/kernel_patches/series/ldiskfs-4.4.0-45-ubuntu14+16.series
ldiskfs/kernel_patches/series/ldiskfs-4.4.0-49-ubuntu14+16.series
ldiskfs/kernel_patches/series/ldiskfs-4.4.0-62-ubuntu14+16.series
ldiskfs/kernel_patches/series/ldiskfs-4.4.0-73-ubuntu14+16.series