LU-9120 lnet: health error simulation

LU-9120 lnet: health error simulation

Modified the error simulation code to simulate health errors for
testing purposes. The specific error can be set. If multiple
errors are configured then one at random is chosen from the set.

EX:
lctl net_drop_add -s *@tcp -d *@tcp -m GET -i 1 -e local_interrupt

The -e can be repeated multiple times to specify different
errors to simulate. The available set are
	local_interrupt
	local_dropped
	local_aborted
	local_no_route
	local_error
	local_timeout
	remote_error
	remote_dropped
	remote_timeout
	network_timeout
	random

a -n, "--random", has been added to randomize error generation for
drop rules. This will rely an interval value provided via -i. This
will generate a random number no bigger than interval. If the number
is smaller than half of the interval then the rule isn't matched,
otherwise it is.

The purpose of this is because drop matching can happen multiple
times in the path of sending the message, and using time based
or rate will not result in even error generation across the
multiple calls.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: If070e29f68c3de10100a9d5eaa49d10cdb76a59a
Reviewed-on: https://review.whamcloud.com/32951
Tested-by: Jenkins
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Tested-by: Maloo <hpdd-maloo@intel.com>