From 79ac2c2c2903e1e98625a577749ef23b59542d62 Mon Sep 17 00:00:00 2001 From: Linda Bebernes Date: Thu, 17 Oct 2013 14:38:39 -0700 Subject: [PATCH] LUDOC-50 LNET: Documented small and tiny router buffers in Chapter 25. Added more details about buffer pools to Section 25.3.5 "Router Buffers". Signed-off-by: Linda Bebernes Change-Id: I6592771e8a83939e2d4e61228c26de24f7273e39 Reviewed-on: http://review.whamcloud.com/7997 Tested-by: Hudson Reviewed-by: Richard Henwood --- LustreTuning.xml | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/LustreTuning.xml b/LustreTuning.xml index dab9254..8fbacb0 100644 --- a/LustreTuning.xml +++ b/LustreTuning.xml @@ -154,14 +154,28 @@ ksocklnd credits=256 applies 256 credits to TCP connections. Applying 256 credits to IB connections can be achieved with: ko2iblnd credits=256 - From Lustre 2.3 and beyond, it is possible that LNet may revalidate the NI Credits and the administrator's request do not persist. + In Lustre 2.3 and beyond, LNET may revalidate the NI Credits so that the administrator's + request does not persist.
<indexterm><primary>tuning</primary><secondary>router buffers</secondary></indexterm>Router Buffers - Router buffers are shared by all CPU partitions. For a machine with a large number of CPTs, the router buffer number may need to be specified manually for best performance. A low number of router buffers risks starving the CPU Partitions of resources. - The default setting for router buffers will typically perform well. LNet automatically sets a default value to reduce the likelihood of resource starvation - An administrator may modify router buffers using the large_router_buffers parameter. For example: + When a node is set up as an LNET router, three pools of buffers are allocated: tiny, + small and large. These pools are allocated per CPU partition and are used to buffer messages + that arrive at the router to be forwarded to the next hop. The three different buffer sizes + accommodate different size messages. If a message arrives that can fit in a tiny buffer then + a tiny buffer is used, if a message doesn’t fit in a tiny buffer, but fits in a small + buffer, then a small buffer is used. Finally if a message doesn’t fit in either a tiny + buffer or a small buffer, a large buffer is used. + Router buffers are shared by all CPU partitions. For a machine with a large number of CPTs, + the router buffer number may need to be specified manually for best performance. A low + number of router buffers risks starving the CPU partitions of resources. + The default setting for router buffers typically results in acceptable performance. LNET + automatically sets a default value to reduce the likelihood of resource starvation. The size + of a router buffer can be modified as shown in the example below. In this example, the size + of the large buffer is modified using the large_router_buffers + parameter. lnet large_router_buffers=8192 - From Lustre 2.3 and beyond, it is possible that LNet may revalidate the router buffer setting and the administrator's request do not persist. + In Lustre 2.3 and beyond, LNET may revalidate the NI Credits so that the administrator's + request does not persist.
<indexterm><primary>tuning</primary><secondary>portal round-robin</secondary></indexterm>Portal Round-Robin Portal round-robin defines the policy LNet applies to deliver events and messages to the upper layers. The upper layers are ptlrpc service or LNet selftest. -- 1.8.3.1