Whamcloud - gitweb
LU-56 ptlrpc: partitioned ptlrpc service
authorLiang Zhen <liang@whamcloud.com>
Sun, 17 Jun 2012 02:57:03 +0000 (10:57 +0800)
committerAndreas Dilger <adilger@whamcloud.com>
Wed, 4 Jul 2012 00:19:42 +0000 (20:19 -0400)
commitd800fc41a1abdaf7aaf6c0e3e7ddcdec489985a8
tree92f083554773453d12418ac7fef80690251aeeee
parent79c740588c809754338cb496cf9cf2f2afbff631
LU-56 ptlrpc: partitioned ptlrpc service

Current ptlrpc service only have a per-service instance, all service
threads share locks and request queue of this instance, this causes
many perforamnce issues like heavy lock contention and data/threads
migration between CPUs/NUMA nodes.

This patch created per-partition data for ptlrpc service, each service
have locks/request-queues on each partition, also, service will have
CPT (CPU partition) affinity threads on each partition, threads
bond on a CPT will only access data on local partition, this feature
should decrease lock contention and data/thread migration and improve
server side performance.

Another change is we use cfs_hash to replace big array fo_iobuf_pool
in obdfilter, filter_iobuf can be found by from the hash by thread ID.
The reason we made this change is because we removed absolute limit
of ptlrpc threads number, which means we have no idea how big the
fo_iobuf_pool should be. Also, even we have obsolute limit of threads
number, it's still dangerous to use a pre-allocated array because
it's difficult to guarantee thread ID to be contiguous if we want
to shrink threads in the future.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I5f8dce7bcf389f9f076f5ce2d4685a03f910260b
Reviewed-on: http://review.whamcloud.com/3133
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
16 files changed:
libcfs/include/libcfs/libcfs_string.h
lnet/lnet/config.c
lustre/include/lustre_net.h
lustre/include/obd.h
lustre/ldlm/ldlm_lockd.c
lustre/mdt/mdt_handler.c
lustre/mgs/mgs_handler.c
lustre/obdfilter/filter.c
lustre/obdfilter/filter_internal.h
lustre/obdfilter/filter_io_26.c
lustre/ost/ost_handler.c
lustre/ost/ost_internal.h
lustre/ptlrpc/events.c
lustre/ptlrpc/lproc_ptlrpc.c
lustre/ptlrpc/niobuf.c
lustre/ptlrpc/service.c