LU-14733 o2iblnd: Move racy NULL assignment

kiblnd_fmr_pool_unmap() can race map and subsequent processing
because of this flaw in unmap:

	if (frd) {
		frd->frd_valid = false;
		spin_lock(&fps->fps_lock);
		list_add_tail(&frd->frd_list, &fpo->fast_reg.fpo_pool_list);
		spin_unlock(&fps->fps_lock);
		fmr->fmr_frd = NULL;
	}

The fmr can be pulled off the list in kiblnd_fmr_pool_unmap() on
another CPU an fmr_frd could be in a state of flux and
potentially be seen incorrectly later on as the kib_tx is processed.

Fix my moving the fmr_frd assignment to before the fmr is added to the
list.

Lustre-change: https://review.whamcloud.com/44189
Lustre-commit: 023113fb8946f3565529e7327fdcd90ab9db3ba3
Test-Parameters: fortestonly testgroup=review-dne-zfs-part-1

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
Change-Id: Ibddf132a363ecfe9db3cc06287cec873c021d2fb
Signed-off-by: Gian-Carlo DeFazio <defazio1@llnl.gov>
Reviewed-on: https://review.whamcloud.com/44295
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

diff --git a/lnet/klnds/o2iblnd/o2iblnd.c b/lnet/klnds/o2iblnd/o2iblnd.c
index 18ac5fa..7785fc9 100644 (file)
--- a/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/lnet/klnds/o2iblnd/o2iblnd.c
@@ -1766,10 +1766,10 @@ kiblnd_fmr_pool_unmap(struct kib_fmr *fmr, int status)
 
                if (frd) {
                        frd->frd_valid = false;
+                       fmr->fmr_frd = NULL;
                        spin_lock(&fps->fps_lock);
                        list_add_tail(&frd->frd_list, &fpo->fast_reg.fpo_pool_list);
                        spin_unlock(&fps->fps_lock);
-                       fmr->fmr_frd = NULL;
                }
        }
        fmr->fmr_pool = NULL;