From c8e74c395d5634dbb0d9d8a86605bb36ab2b8233 Mon Sep 17 00:00:00 2001 From: Chris Horn Date: Tue, 30 Nov 2021 10:57:34 -0600 Subject: [PATCH] LU-15275 lnet: Skip router discovery on send path When the router checker is enabled, routes are regularly marked as out of date w.r.t. discovery. This can cause upper level messages to be delayed while the router undergoes discovery. We can avoid delaying messages by relying on the router checker to initiate discovery of routers. If we happen to send a message to a router before it has been discovered then the worst case scenario is that the route is actually down or we end up utilizing a subset of a multi-rail router's interfaces. Both situations can be remedied by utilizing the check_routers_before_use parameter. Change the logic in lnet_handle_find_routed_path() so that we only initiate discovery if the alive_router_check_interval is <= 0 (i.e. router checker pings are disabled). Test-Parameters: trivial testlist=sanity-lnet Signed-off-by: Chris Horn Change-Id: If0332c21f6157117598b7b908fe17f2d2690fc1d Reviewed-on: https://review.whamcloud.com/45684 Tested-by: jenkins Tested-by: Maloo Reviewed-by: Alexey Lyashkov Reviewed-by: Andriy Skulysh Reviewed-by: Oleg Drokin --- lnet/lnet/lib-move.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/lnet/lnet/lib-move.c b/lnet/lnet/lib-move.c index 9551627..73e9e25 100644 --- a/lnet/lnet/lib-move.c +++ b/lnet/lnet/lib-move.c @@ -2306,13 +2306,23 @@ use_lpn: } /* - * Discover this gateway if it hasn't already been discovered. - * This means we might delay the message until discovery has - * completed + * If the router checker is not active then discover the gateway here. + * This ensures we are able to take advantage of multi-rail routing, but + * if the router checker is active then we do not unecessarily delay + * messages while the gateway is being checked by the dedicated monitor + * thread. + * + * NB: We're only checking the alive_router_check_interval here, rather + * than calling lnet_router_checker_active(), because the other + * conditions that are checked by that function are either + * irrelevant (the_lnet.ln_routing) or must be true (list of routers + * is not empty) */ - rc = lnet_initiate_peer_discovery(gwni, sd->sd_msg, sd->sd_cpt); - if (rc) - return rc; + if (alive_router_check_interval <= 0) { + rc = lnet_initiate_peer_discovery(gwni, sd->sd_msg, sd->sd_cpt); + if (rc) + return rc; + } if (!sd->sd_best_ni) { lpn = gwni->lpni_peer_net; -- 1.8.3.1