From 62b9bc3ecc0d3ef432bd8d8920bda901b3bee345 Mon Sep 17 00:00:00 2001 From: Joseph Gmitter Date: Thu, 8 Dec 2016 12:54:59 -0500 Subject: [PATCH] LUDOC-356 rpc: Add documentation for large bulk IO Add basic documentation for large bulk IO (16MB RPC). Signed-off-by: Joseph Gmitter Change-Id: Iaba6782316b8bf17142dda4b023f394dcb4a78d5 Reviewed-on: https://review.whamcloud.com/24245 Tested-by: Jenkins Reviewed-by: Andreas Dilger Reviewed-by: Jinshan Xiong --- LustreTuning.xml | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/LustreTuning.xml b/LustreTuning.xml index 6e6ebb0..e98574a 100644 --- a/LustreTuning.xml +++ b/LustreTuning.xml @@ -1493,6 +1493,56 @@ $ lctl set_param ost.OSS.ost_io.nrs_tbf_rule="hp stop loginnode" +
+ + <indexterm> + <primary>tuning</primary> + <secondary>Large Bulk IO</secondary> + </indexterm> + Large Bulk IO (16MB RPC) + +
Overview + Beginning with Lustre 2.9, Lustre is extended to support RPCs up + to 16MB in size. By enabling a larger RPC size, fewer RPCs will be + required to transfer the same amount of data between clients and + servers. With a larger RPC size, the OST can submit more data to the + underlying disks at once, therefore it can produce larger disk I/Os + to fully utilize the increasing bandwidth of disks. + At client connecting time, clients will negotiate with + servers for the RPC size it is going to use. + A new parameter, brw_size, is introduced on + the OST to tell the client the preferred IO size. All clients that + talk to this target should never send an RPC greater than this size. + +
+
Usage + In order to enable a larger RPC size, + brw_size must be changed to an IO size value up to + 16MB. To temporarily change brw_size, the + following command should be run on the OSS: + oss# lctl set_param obdfilter.fsname-OST*.brw_size=16 + To persistently change brw_size, one of the following + commands should be run on the OSS: + oss# lctl set_param -P obdfilter.fsname-OST*.brw_size=16 + oss# lctl conf_param fsname-OST*.obdfilter.brw_size=16 + When a client connects to an OST target, it will fetch + brw_size from the target and pick the maximum value + of brw_size and its local setting for + max_pages_per_rpc as the actual RPC size. + Therefore, the max_pages_per_rpc on the client side + would have to be set to 16M, or 4096 if the PAGESIZE is 4KB, to enable + a 16MB RPC. To temporarily make the change, the following command + should be run on the client to set + max_pages_per_rpc: + client$ lctl set_param osc.fsname-OST*.max_pages_per_rpc=16M + To persistently make this change, the following command should + be run: + client$ lctl conf_param fsname-OST*.osc.max_pages_per_rpc=16M + The brw_size of an OST can be + changed on the fly. However, clients have to be remounted to + renegotiate the new RPC size. +
+
<indexterm> -- 1.8.3.1