LU-9943 lnd: correct WR fast reg accounting

Ensure that enough WRs are allocated for the fast reg
case which needs two additional WRs per transfer:
the first for memory window registration and the second for
memory window invalidation.

Failure to allocate these causes the following problem:
mlx5_warn:mlx5_0:begin_wqe:4085(pid 9590): work queue overflow

Test-Parameters: trivial
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Icf98b6bbb3d98fb29794173da84412070f13541b
Reviewed-on: https://review.whamcloud.com/30311
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

diff --git a/lnet/klnds/o2iblnd/o2iblnd.c b/lnet/klnds/o2iblnd/o2iblnd.c
index fb0222a..603cca2 100644 (file)
--- a/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/lnet/klnds/o2iblnd/o2iblnd.c
@@ -735,6 +735,11 @@ static unsigned int kiblnd_send_wrs(struct kib_conn *conn)
         * And ibc_max_frags for the transfer WRs
         */
        unsigned int ret = 1 + conn->ibc_max_frags;
+       __u32 dev_caps = conn->ibc_hdev->ibh_dev->ibd_dev_caps;
+
+       /* FastReg needs two extra WRs for map and invalidate */
+       if (dev_caps & IBLND_DEV_CAPS_FASTREG_ENABLED)
+               ret += 2;
 
        /* account for a maximum of ibc_queue_depth in-flight transfers */
        ret *= conn->ibc_queue_depth;