LU-15275 lnet: Skip router discovery on send path

When the router checker is enabled, routes are regularly marked as out
of date w.r.t. discovery. This can cause upper level messages to be
delayed while the router undergoes discovery. We can avoid delaying
messages by relying on the router checker to initiate discovery of
routers. If we happen to send a message to a router before it has
been discovered then the worst case scenario is that the route is
actually down or we end up utilizing a subset of a multi-rail router's
interfaces. Both situations can be remedied by utilizing the
check_routers_before_use parameter.

Change the logic in lnet_handle_find_routed_path() so that we only
initiate discovery if the alive_router_check_interval is <= 0 (i.e.
router checker pings are disabled).

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Chris Horn <chris.horn@hpe.com>
Change-Id: If0332c21f6157117598b7b908fe17f2d2690fc1d
Reviewed-on: https://review.whamcloud.com/45684
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

diff --git a/lnet/lnet/lib-move.c b/lnet/lnet/lib-move.c
index 9551627..73e9e25 100644 (file)
--- a/lnet/lnet/lib-move.c
+++ b/lnet/lnet/lib-move.c
@@ -2306,13 +2306,23 @@ use_lpn:
        }
 
        /*
-        * Discover this gateway if it hasn't already been discovered.
-        * This means we might delay the message until discovery has
-        * completed
+        * If the router checker is not active then discover the gateway here.
+        * This ensures we are able to take advantage of multi-rail routing, but
+        * if the router checker is active then we do not unecessarily delay
+        * messages while the gateway is being checked by the dedicated monitor
+        * thread.
+        *
+        * NB: We're only checking the alive_router_check_interval here, rather
+        * than calling lnet_router_checker_active(), because the other
+        * conditions that are checked by that function are either
+        * irrelevant (the_lnet.ln_routing) or must be true (list of routers
+        * is not empty)
         */
-       rc = lnet_initiate_peer_discovery(gwni, sd->sd_msg, sd->sd_cpt);
-       if (rc)
-               return rc;
+       if (alive_router_check_interval <= 0) {
+               rc = lnet_initiate_peer_discovery(gwni, sd->sd_msg, sd->sd_cpt);
+               if (rc)
+                       return rc;
+       }
 
        if (!sd->sd_best_ni) {
                lpn = gwni->lpni_peer_net;