LU-5577 ptlrpc: fix comparison between signed and unsigned

LU-5577 ptlrpc: fix comparison between signed and unsigned

Change return type and size argiments of lustre_msg_hdr_size(),
lustre_msg_buf{len,count}() and req_capsule_*_size() to __u32.
Change type of req_format->rf_idx and req_format->rf_fields.nr
to size_t. Also return zero for incorrect message magic instead
of -EINVAL. This will be more robust because of few of them after
LASSERTF(0, "...") and will not be returned. In the rest places
it return zero size instead of huge number after implicit
unsigned conversion.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Iae8b83b4285a1ea2fa010f9de2792426c1772b2c
Reviewed-on: http://review.whamcloud.com/12475
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>