Whamcloud - gitweb
LU-14361 statahead: add tunable for fname pattern detection 92/51592/39
authorQian Yingjin <qian@ddn.com>
Thu, 6 Jul 2023 13:21:40 +0000 (09:21 -0400)
committerOleg Drokin <green@whamcloud.com>
Mon, 2 Dec 2024 05:41:19 +0000 (05:41 +0000)
commitf4bf36d8f4957b62e4edab58b971829dccedd0c0
tree172f450e1e76d5e81aa68132eb13ffaf19fae294
parent5803284ac3a5d477df9afffe48ff35f08d67da1a
LU-14361 statahead: add tunable for fname pattern detection

This patch adds two tunable parameters for the detection of the
fname pattern statahead:
- llite.*.statahead_fname_predict_hit: when the naming of stat()
  files under a directory follows a certain name rule roughly more
  than this value, the directory is considered to meet the first
  requirement for statahead. For an example, file naming rule is
  mdtest.$rank.$i, the suffix of the stat() dentry name is number
  and do stat() for dentries with name ending with number more
  then this parameter.
- llite.*.statahead_fname_match_hit: After meet the first
  requirement, then the naming of stat() files under a directory
  continuously satisfies a certain name rule strictly more than
  this value, it will start a statahead thread to do attribute
  prefetching under the directory.

This patch also fixes the following panic:
IP: _atomic_dec_and_lock+0xc/0x70
->ll_sax_put [lustre]
->ll_statahead_thread [lustre]
->kthread

The reason is that the @lli_sax is set with NULL by wrong
statahead context (sax) put.

This patch also fixes the possible deadlock between hardlink and
batch stat-ahead operations.
Fix the test failure on lustre-rsync-test/test_6.

It must wait all inuse inodes for statahead to release during
umount.
Otherwsie, it may cause the following panic:
BUG: Dentry 0000000033ca4f3e{i=280001b840002c7,n=l4}  still in use
(1) [unmount of lustre lustre]
RIP: 0010:umount_check.cold.52+0x2f/0x3b
d_walk+0xe7/0x290
do_one_tree+0x20/0x40
shrink_dcache_for_umount+0x28/0x90
generic_shutdown_super+0x1a/0x110
kill_anon_super+0x14/0x30
deactivate_locked_super+0x34/0x70
cleanup_mnt+0x3b/0x70

RIP: 0010:ll_prep_md_op_data+0x73/0x870 [lustre]
sa_prep_data+0xde/0x350 [lustre]
sa_statahead+0x3b9/0xd20 [lustre]
ll_statahead_thread+0x1507/0x21f0 [lustre]
kthread+0x134/0x150

Test-Parameters: clientdistro=el8.10 testlist=sanity
Test-Parameters: clientdistro=el8.10 testlist=sanity
Test-Parameters: clientdistro=el8.10 testlist=sanity
Test-Parameters: clientdistro=el8.10 testlist=sanity
Test-Parameters: clientdistro=el8.10 testlist=sanity
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I42d9478e796918d9f2498ab64cf7c20b61334144
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51592
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
lustre/llite/llite_internal.h
lustre/llite/llite_lib.c
lustre/llite/lproc_llite.c
lustre/llite/statahead.c
lustre/mdt/mdt_handler.c