LU-12374 lustre: push rcu_barrier() before destroying slab

From rcubarrier.txt:

"
We could try placing a synchronize_rcu() in the module-exit code path,
but this is not sufficient. Although synchronize_rcu() does wait for a
grace period to elapse, it does not wait for the callbacks to complete.

One might be tempted to try several back-to-back synchronize_rcu()
calls, but this is still not guaranteed to work. If there is a very
heavy RCU-callback load, then some of the callbacks might be deferred
in order to allow other processing to proceed. Such deferral is required
in realtime kernels in order to avoid excessive scheduling latencies.

We instead need the rcu_barrier() primitive. This primitive is similar
to synchronize_rcu(), but instead of waiting solely for a grace
period to elapse, it also waits for all outstanding RCU callbacks to
complete. Pseudo-code using rcu_barrier() is as follows:

   1. Prevent any new RCU callbacks from being posted.
   2. Execute rcu_barrier().
   3. Allow the module to be unloaded.
"

So use synchronize_rcu() in ldlm_exit() is not safe enough, and we might
still hit use-after-free problem, also we missed rcu_barrier() when destory
inode cache, this is simiar idea what current local filesystem does.

Change-Id: I76c7dfe7b6472d377fe1b60b0891c61ac8a0fbfc
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35030
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

diff --git a/lustre/ldlm/ldlm_lockd.c b/lustre/ldlm/ldlm_lockd.c
index fe9668b..18221a7 100644 (file)
--- a/lustre/ldlm/ldlm_lockd.c
+++ b/lustre/ldlm/ldlm_lockd.c
@@ -3376,10 +3376,10 @@ void ldlm_exit(void)
        kmem_cache_destroy(ldlm_resource_slab);
        /*
         * ldlm_lock_put() use RCU to call ldlm_lock_free, so need call
-        * synchronize_rcu() to wait a grace period elapsed, so that
-        * ldlm_lock_free() get a chance to be called.
+        * rcu_barrier() to wait all outstanding RCU callbacks to complete,
+        * so that ldlm_lock_free() get a chance to be called.
         */
-       synchronize_rcu();
+       rcu_barrier();
        kmem_cache_destroy(ldlm_lock_slab);
        kmem_cache_destroy(ldlm_interval_slab);
        kmem_cache_destroy(ldlm_interval_tree_slab);

diff --git a/lustre/llite/super25.c b/lustre/llite/super25.c
index bd38776..5158d0a 100644 (file)
--- a/lustre/llite/super25.c
+++ b/lustre/llite/super25.c
@@ -177,6 +177,13 @@ static void __exit lustre_exit(void)
        cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
        vvp_global_fini();
 
+#ifdef HAVE_INODE_I_RCU
+       /*
+        * Make sure all delayed rcu free inodes are flushed before we
+        * destroy cache.
+        */
+       rcu_barrier();
+#endif
        kmem_cache_destroy(ll_inode_cachep);
        kmem_cache_destroy(ll_file_data_slab);
 }