LU-9971 lnet: use after free in lnet_discover_peer_locked()

When the lnet_net_lock is unlocked, the peer attached to an
lnet_peer_ni (found via lnet_peer_ni::lpni_peer_net->lpn_peer)
can change, and the old peer deallocated. If we are really
unlucky, then all the churn could give us a new, different,
peer at the same address in memory.

Change the reference counting on the lnet_peer lp so that it
is guaranteed to be alive when we relock the lnet_net_lock for
the cpt. When the reference count is dropped lp may go away if
it was unlinked, but the new peer is guaranteed to have a
different address, so we can still correctly determine whether
the peer changed and discovery should be redone.

Signed-off-by: Olaf Weber <olaf.weber@hpe.com>
Change-Id: Ia44dce20074b27ec0e77d7c1908c6a44ec73d326
Reviewed-on: https://review.whamcloud.com/28944
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

diff --git a/lnet/lnet/peer.c b/lnet/lnet/peer.c
index 2ae8956..01fcad3 100644 (file)
--- a/lnet/lnet/peer.c
+++ b/lnet/lnet/peer.c
@@ -2162,6 +2162,8 @@ again:
         * zombie if we race with DLC, so we must check for that.
         */
        for (;;) {
+               /* Keep lp alive when the lnet_net_lock is unlocked */
+               lnet_peer_addref_locked(lp);
                prepare_to_wait(&lp->lp_dc_waitq, &wait, TASK_INTERRUPTIBLE);
                if (signal_pending(current))
                        break;
@@ -2174,15 +2176,14 @@ again:
                lnet_peer_queue_for_discovery(lp);
 
                /*
-                * if caller requested a non-blocking operation then
-                * return immediately. Once discovery is complete then the
-                * peer ref will be decremented and any pending messages
-                * that were stopped due to discovery will be transmitted.
+                * If caller requested a non-blocking operation then
+                * return immediately. Once discovery is complete any
+                * pending messages that were stopped due to discovery
+                * will be transmitted.
                 */
                if (!block)
                        break;
 
-               lnet_peer_addref_locked(lp);
                lnet_net_unlock(LNET_LOCK_EX);
                schedule();
                finish_wait(&lp->lp_dc_waitq, &wait);
@@ -2205,11 +2206,13 @@ again:
 
        lnet_net_unlock(LNET_LOCK_EX);
        lnet_net_lock(cpt);
-
+       lnet_peer_decref_locked(lp);
        /*
-        * If the peer has changed after we've discovered the older peer,
-        * then we need to discovery the new peer to make sure the
-        * interface information is up to date
+        * The peer may have changed, so re-check and rediscover if that turns
+        * out to have been the case. The reference count on lp ensured that
+        * even if it was unlinked from lpni the memory could not be recycled.
+        * Thus the check below is sufficient to determine whether the peer
+        * changed. If the peer changed, then lp must not be dereferenced.
         */
        if (lp != lpni->lpni_peer_net->lpn_peer)
                goto again;