LU-11476 lnet: set the health status correctly

LU-11476 lnet: set the health status correctly

There are cases where the health status wasn't set properly.
Most notably in the tx_done we need to deal with a specific
set of errno: ENETDOWN, EHOSTUNREACH, ENETUNREACH, ECONNREFUSED,
ECONNRESET. In all those cases we can try and resend to other
available peer NIs.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ie8f0275582d434bda5e394fccc2a4d88dd538c69
Reviewed-on: https://review.whamcloud.com/33307
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>