LU-9119 lnet: loopback NID in lnet_select_pathway()

LU-9119 lnet: loopback NID in lnet_select_pathway()

In lnet_select_pathway() sending to the loopback NID is handled
as a special case, because there are no credits involved. (The
loopback NID doesn't use credits, and therefore does not have
any credits. If a message goes through the credit-managing code
it therefore ends up waiting indefinitely for credits to become
available.)

The check whether we're sending over the loopback NID must be
done after we've completed choosing the NI to send over. In its
present location it only handles the case where the loopback
NID was explicitly passed in as the source NID.

(Lustre does not exercise this code path during normal operation,
the bug was encountered while testing code for the peer discovery
feature.)

Test-Parameters: trivial
Signed-off-by: Olaf Weber <olaf@sgi.com>
Change-Id: Ifa25abf508214ae363a2f1bb04ffeab1891a2564
Reviewed-on: https://review.whamcloud.com/26692
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>