From 1f7613968c800f99ed074f17cd7ba1086847d2db Mon Sep 17 00:00:00 2001 From: Wang Shilong Date: Sat, 1 Jun 2019 19:22:11 +0800 Subject: [PATCH] LU-12374 lustre: push rcu_barrier() before destroying slab From rcubarrier.txt: " We could try placing a synchronize_rcu() in the module-exit code path, but this is not sufficient. Although synchronize_rcu() does wait for a grace period to elapse, it does not wait for the callbacks to complete. One might be tempted to try several back-to-back synchronize_rcu() calls, but this is still not guaranteed to work. If there is a very heavy RCU-callback load, then some of the callbacks might be deferred in order to allow other processing to proceed. Such deferral is required in realtime kernels in order to avoid excessive scheduling latencies. We instead need the rcu_barrier() primitive. This primitive is similar to synchronize_rcu(), but instead of waiting solely for a grace period to elapse, it also waits for all outstanding RCU callbacks to complete. Pseudo-code using rcu_barrier() is as follows: 1. Prevent any new RCU callbacks from being posted. 2. Execute rcu_barrier(). 3. Allow the module to be unloaded. " So use synchronize_rcu() in ldlm_exit() is not safe enough, and we might still hit use-after-free problem, also we missed rcu_barrier() when destory inode cache, this is simiar idea what current local filesystem does. Change-Id: I76c7dfe7b6472d377fe1b60b0891c61ac8a0fbfc Signed-off-by: Wang Shilong Reviewed-on: https://review.whamcloud.com/35030 Tested-by: Jenkins Reviewed-by: Andreas Dilger Tested-by: Maloo Reviewed-by: Gu Zheng Reviewed-by: Li Xi Reviewed-by: Oleg Drokin --- lustre/ldlm/ldlm_lockd.c | 6 +++--- lustre/llite/super25.c | 7 +++++++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/lustre/ldlm/ldlm_lockd.c b/lustre/ldlm/ldlm_lockd.c index fe9668b..18221a7 100644 --- a/lustre/ldlm/ldlm_lockd.c +++ b/lustre/ldlm/ldlm_lockd.c @@ -3376,10 +3376,10 @@ void ldlm_exit(void) kmem_cache_destroy(ldlm_resource_slab); /* * ldlm_lock_put() use RCU to call ldlm_lock_free, so need call - * synchronize_rcu() to wait a grace period elapsed, so that - * ldlm_lock_free() get a chance to be called. + * rcu_barrier() to wait all outstanding RCU callbacks to complete, + * so that ldlm_lock_free() get a chance to be called. */ - synchronize_rcu(); + rcu_barrier(); kmem_cache_destroy(ldlm_lock_slab); kmem_cache_destroy(ldlm_interval_slab); kmem_cache_destroy(ldlm_interval_tree_slab); diff --git a/lustre/llite/super25.c b/lustre/llite/super25.c index bd38776..5158d0a 100644 --- a/lustre/llite/super25.c +++ b/lustre/llite/super25.c @@ -177,6 +177,13 @@ static void __exit lustre_exit(void) cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck); vvp_global_fini(); +#ifdef HAVE_INODE_I_RCU + /* + * Make sure all delayed rcu free inodes are flushed before we + * destroy cache. + */ + rcu_barrier(); +#endif kmem_cache_destroy(ll_inode_cachep); kmem_cache_destroy(ll_file_data_slab); } -- 1.8.3.1