From 3faa31e59c8d07dd7bea5433ea6782f1e6bbab52 Mon Sep 17 00:00:00 2001 From: Andreas Dilger Date: Thu, 18 Jan 2024 02:49:48 -0700 Subject: [PATCH] LU-17441 mdc: use MDS_IO_PORTAL for rename Some workloads like Apache Spark are very rename intensive, and there here may be many concurrent renames that need the BFL lock (more than the number of MDS_REQUEST_PORTAL service threads), they will block these threads until each is able to get the rename lock, and prevent other MDS_REINT RPCs from being processed. Since the MDS_IO_PORTAL is often unused (only needed for DoM files), and has existed since 2.11.0, it seems possible to move the rename RPCs to be serviced by the MDS_IO_PORTAL threads to avoid contention on the primary MDS service threads. Also, it will avoid blocking normal file open, setattr, statfs, and other common operations if the BFL lock is contended. Even with DoM files they may have read-on-open handling and only DoM writes would be blocked by the uncommon rename. Test-Parameters: testlist=sanity serverversion=2.15 \ env=SANITY_EXCEPT="56x 56xa 56xc 65p 70a 119h 119i 123g 123h 123i 398d 398o" Signed-off-by: Andreas Dilger Change-Id: I623a27de1482778f3c9fc6bb5bbcf917611dc75b Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53725 Reviewed-by: Lai Siyao Reviewed-by: Mikhail Pershin Reviewed-by: Oleg Drokin Tested-by: Maloo Tested-by: jenkins --- lustre/mdc/mdc_reint.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lustre/mdc/mdc_reint.c b/lustre/mdc/mdc_reint.c index 017f619..6619e55 100644 --- a/lustre/mdc/mdc_reint.c +++ b/lustre/mdc/mdc_reint.c @@ -524,6 +524,14 @@ int mdc_rename(struct obd_export *exp, struct md_op_data *op_data, sptlrpc_sepol_put(sepol); + /* LU-17441: avoid blocking MDS_REQUEST_PORTAL for renames with BFL */ +#if LUSTRE_VERSION_CODE < OBD_OCD_VERSION(2, 20, 53, 0) + /* MDS_IO_PORTAL available since v2_10_53_0-33-g2bcc5ad0ed */ + if ((exp_connect_flags(exp) & + (OBD_CONNECT_GRANT | OBD_CONNECT_SRVLOCK)) == + (OBD_CONNECT_GRANT | OBD_CONNECT_SRVLOCK)) +#endif + req->rq_request_portal = MDS_IO_PORTAL; req_capsule_set_size(&req->rq_pill, &RMF_MDT_MD, RCL_SERVER, obd->u.cli.cl_default_mds_easize); ptlrpc_request_set_replen(req); -- 1.8.3.1