Whamcloud - gitweb
LU-18446 ptlrpc: lower CPUs latency during client I/O 39/57039/12
authorBruno Faccini <bfaccini@nvidia.com>
Fri, 15 Nov 2024 09:24:08 +0000 (10:24 +0100)
committerOleg Drokin <green@whamcloud.com>
Thu, 6 Mar 2025 08:04:50 +0000 (08:04 +0000)
commit54a64ea818d936cb52c263fd34bb359972ae9fb1
treefda039eb7ca4cbf075945e5aeac911365855726d
parent2da8542e7069af71566a5d36d53fdc840a63228a
LU-18446 ptlrpc: lower CPUs latency during client I/O

Some CPUs with power-management can suffer with high
latency to exit from idle state.
This can have a strong impact on Lustre client perfs.
Use PM-QoS framework to guarantee usage of low-latency
power management mode, for CPUs/Cores known to be
involved to handle RPC replies for Lustre I/Os
completion.

Added PM-QoS configure checks:

PM-QoS framework is present since Kernel v3.2.
DEV_PM_QOS_RESUME_LATENCY was on DEV_PM_QOS_LATENCY before v3.15.

to handle all these cases for older kernels compatibility.

Add 4 tuneables :
  _ 'enable_pmqos' to enable/disable using PM-QoS to
    bump CPUs latency
  _ 'pmqos_latency_max_usec' to allow modifying the max
    latency value to be used
  _ 'pmqos_default_duration_usec' to allow modifying
    the timeout value to unset low latency
  _ 'pmqos_use_stats_for_duration to enable/disable
    using the per-target stats to set low latency timeout

Here is a table summarising the single node fio (randread)
performance :
NJOBS Target perf Original perf perf with patch
1           2.5              1.05            2.56
2           5.24             2.14            5.26
4           10.8             4.36            10.5
8           21.3             8.68            20.9
16          40               16.9            40
32          65.4             32.2            64.1
64          84               56.8            83.4
128         90.8             79.6            89.9
192         91.7             85.2            91.5
256         91.9             87.4            91.8
320         91.8             89.7            91.9

Signed-off-by: Bruno Faccini <bfaccini@nvidia.com>
Change-Id: I784a699f355da413db5029c6c7584ce3ee4ba9e1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/57039
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
config/lustre-core.m4
lustre/include/lustre_net.h
lustre/obdclass/lprocfs_status.c
lustre/ptlrpc/connection.c
lustre/ptlrpc/lproc_ptlrpc.c
lustre/ptlrpc/niobuf.c
lustre/ptlrpc/ptlrpc_internal.h
lustre/ptlrpc/ptlrpc_module.c