From: Patrick Farrell Date: Sun, 21 Jul 2019 17:06:37 +0000 (-0400) Subject: LU-12569 o2iblnd: Make credits hiw connection aware X-Git-Tag: 2.12.3-RC1~34 X-Git-Url: https://git.whamcloud.com/gitweb?a=commitdiff_plain;h=90ba471e367754ea6ddb9a95060591f46b95b0b6;p=fs%2Flustre-release.git LU-12569 o2iblnd: Make credits hiw connection aware The IBLND_CREDITS_HIGHWATER mark check currently looks only at the global peer credits tunable, ignoring the connection specific queue depth when determining the threshold at which to send a NOOP message to return credits. This is incorrect because while connection queue depth defaults to the same as peer credits, it can be less than that global value for specific connections. So we must check for this case when setting the threshold. Lustre-change: https://review.whamcloud.com/35578 Lustre-commit: 1b87e8f61781e48c31b4da647214d66addf2b90c Test-Parameter: nettype=o2ib Signed-off-by: Patrick Farrell Change-Id: Ie028ae11cdbd0f75a38b265b7ab5830f92f08d90 Reviewed-by: Chris Horn Reviewed-by: Amir Shehata Reviewed-by: James Simmons Signed-off-by: Minh Diep Reviewed-on: https://review.whamcloud.com/36254 Tested-by: jenkins Tested-by: Maloo Reviewed-by: Oleg Drokin --- diff --git a/lnet/klnds/o2iblnd/o2iblnd.h b/lnet/klnds/o2iblnd/o2iblnd.h index 7bd04d3..88ff410 100644 --- a/lnet/klnds/o2iblnd/o2iblnd.h +++ b/lnet/klnds/o2iblnd/o2iblnd.h @@ -115,9 +115,9 @@ extern struct kib_tunables kiblnd_tunables; #define IBLND_CREDITS_MAX ((typeof(((struct kib_msg *) 0)->ibm_credits)) - 1) /* Max # of peer_ni credits */ /* when eagerly to return credits */ -#define IBLND_CREDITS_HIGHWATER(t, v) ((v) == IBLND_MSG_VERSION_1 ? \ +#define IBLND_CREDITS_HIGHWATER(t, conn) ((conn->ibc_version) == IBLND_MSG_VERSION_1 ? \ IBLND_CREDIT_HIGHWATER_V1 : \ - t->lnd_peercredits_hiw) + min(t->lnd_peercredits_hiw, (__u32)conn->ibc_queue_depth - 1)) #ifdef HAVE_RDMA_CREATE_ID_5ARG # define kiblnd_rdma_create_id(cb, dev, ps, qpt) rdma_create_id(current->nsproxy->net_ns, \ @@ -932,7 +932,7 @@ kiblnd_need_noop(struct kib_conn *conn) tunables = &ni->ni_lnd_tunables.lnd_tun_u.lnd_o2ib; if (conn->ibc_outstanding_credits < - IBLND_CREDITS_HIGHWATER(tunables, conn->ibc_version) && + IBLND_CREDITS_HIGHWATER(tunables, conn) && !kiblnd_send_keepalive(conn)) return 0; /* No need to send NOOP */