2 .TH lnetctl 1 "2014 Sep 12" Lustre "configuration utilities"
8 \fBlnetctl\fR \fB<cmd> <subcmd> [optional parameters]\fR
11 \fBlnetctl\fR is used to configure LNet parameters allowing various configuration
12 and debugging features to be accessed\.
15 \fBlnetctl\fR can be invoked in interactive mode by issuing lnetctl command\.
16 After that, commands are issued as below\.
19 To get a complete listing of available commands, type \fBhelp\fR at the
20 lnetctl prompt\. To get basic help on the meaning and syntax of a command
21 type \fB<command>\fR \-\-help\.
24 For non\-interactive use the \fBlnetctl\fR utility can be invoked from the
25 command line as follows:
28 lnetctl \fIcommand\fR \fIsubcommand\fR [optional parameters]
30 .SS "LNet Initialization:"
33 \fBlnetctl lnet\fR configure [\-\-all]
34 Brings up the LNet Network Interface (NI) system\. If the \-\-all option is
35 provided it will load network interfaces defined in the modprobe files\.
36 Otherwise, it doesn\'t initialize any networks by default\.
39 \fBlnetctl lnet\fR unconfigure
40 Brings down the LNet Network Interface (NI) system including any configured
43 .SS "Network Configuration"
47 Configures a network interface either given the network name and physical
48 interface device name, or given the ip2net parameter\. Other parameters
52 \-\-net: net name (e.g. tcp0)
55 \-\-if: physical interface (e.g. eth0)
58 \-\-ip2net: specify networks based on IP address patterns
61 \-\-peer\-timeout: time to wait before declaring a peer dead (in seconds).
62 Default value for o2iblnd and socklnd is 180 seconds.
65 \-\-peer\-credits: define the max number of in\-flight messages per peer.
68 \-\-peer\-buffer\-credits: the max number of routed in\-flight messages
72 \-\-credits: The total number of in\-flight messages over a network interface.
75 \-\-cpt: The CPU partitions on which the created network interface is bound to.
76 Refer to the Lustre Manual Section "Binding Network Interface Against CPU
77 Partitions" for more details. For example to bind a Network Interface to
78 CPU partitions 0 and 1, you would specify this parameter as \-\-cpt [0,
86 Delete a network interface given the network name\.
89 \-\-net: net name (e.g. tcp0)
95 \fBlnetctl net\fR show
96 Show all currently configured network interfaces if no parameters given or filter
97 on the network name\. More details can be shown by specifying the \-\-verbose
101 \-\-net: net name (e.g. tcp0) to filter on
104 \-\-verbose: display detailed output per network
107 .SS "Peer Configuration"
109 \fBlnetctl peer\fR add
110 Configure an LNET peer with at least one supplied NID\. By default, peers are marked as multi-rail capable\. If prim_nid is not specified, the first NID in this list is assumed to be the primary NID for the peer.
114 \-\-nid: one or more peer NIDs to add to the peer\.
118 \-\-prim_nid: Primary NID of the peer\.
121 \-\-non_mr: create this peer as not Multi-Rail capable\.
126 \fBlnetctl peer\fR del
127 Delete a peer NID. The primary NID must be specified. If the removed NID is the primary NID, the peer entry will be deleted.
131 \-\-nid: one or more peer NIDs to remove from the peer\.
135 \-\-prim_nid: Primary NID of the peer\.
140 \fBlnetctl peer\fR show
141 Show configured peers. By default, lists all peers and associated NIDs.
145 \-\-nid: list of primary nids to filter on
149 \-\-verbose: Include extended statistics, including credits and counters.
154 .SS "Route Configuration"
157 \fBlnetctl route\fR add
161 \-\-net: net name (e.g. tcp0)
164 \-\-gateway: gateway nid (e.g. 10\.1\.1\.2@tcp)
167 \-\-hop: number to final destination (1 < hops < 255)
170 \-\-priority: priority of route (0 \- highest prio)
176 \fBlnetctl route\fR del
177 Delete a route specified via the network and gateway\.
180 \-\-net: net name (e.g. tcp0)
183 \-\-gateway: gateway nid (e.g. 10\.1\.1\.2@tcp)
189 \fBlnetctl route\fR show
190 Show all currently configured routes if no parameters given, or filter on
191 given parameters\. More details can be shown by specifying the \-\-verbose
195 \-\-net: net name (e.g. tcp0) to filter on
198 \-\-gateway: gateway nid (e.g. 10\.1\.1\.2@tcp) to filter on
201 \-\-hop: number to final destination (1 < hops < 255) to filter on
204 \-\-priority: priority of route (0 \- highest prio to filter on)
207 \-\-verbose: display detailed output per route
212 .SS "Routing Information"
215 \fBlnetctl routing\fR show
216 Show router buffers values as well as show the status of routing (IE: whether
217 the node is set to be a router)
220 Individual values can be set using the \fBlnetctl set\fR command\.
223 \fBlnetctl set\fR tiny_buffers \fIvalue\fR
224 Set the number of tiny buffers in the system\. This is the total number of tiny
225 buffers for all CPU partitions\.
228 \fBlnetctl set\fR small_buffers \fIvalue\fR
229 Set the number of small buffers in the system\. This is the total number of
230 small buffers for all CPU partitions\.
233 \fBlnetctl set\fR large_buffers \fIvalue\fR
234 Set the number of large buffers in the system\. This is the total number of
235 large buffers for all CPU partitions\.
238 \fBlnetctl set\fR routing \fI[0, 1]\fR
239 0 value indicates to disable routing\. 1 value indicates to enable routing\.
240 When routing is disabled the values of the buffers that might have been changed
241 are not remembered, and the next time routing is enabled the default buffer
244 .SS "Import and Export YAML Configuration Files"
245 LNet configuration can be represented in YAML format\. A YAML configuration
246 file can be passed to the lnetctl utility via the \fBimport\fR command\. The
247 lnetctl utility will attempt to configure all elements defined in the YAML
251 Similarly the \fBexport\fR command can be used to dump all supported LNet
252 configuration to stdout\. The output can be redirected to a file\.
255 \fBlnetctl import\fR \fIFILE\fR:
259 \fBlnetctl import\fR < \fIFILE\fR
260 \fBimport\fR command uses the specified YAML configuration file to configure
261 LNet parameters defined within\. The import command by default adds the LNet
262 parameters defined in the YAML file, but this default behavior can be
263 overwritten by specifying the desired behavior\.
266 \-\-add: add configuration
269 \-\-del: delete configuration
272 \-\-show: show configuration
275 \-\-help: display this help
278 \fBlnetctl export\fR \fIFILE\fR:
282 \fBlnetctl export\fR > \fIFILE\fR
283 \fBexport\fR command dumps the supported LNet configuration in YAML format to
284 stdout, which can be redirected to a normal file\. The output of the
285 \fBimport\fR command can be used as input to the \fBexport\fR command\.
288 \-\-help: display this help
290 .SS "LNet Statistics"
297 \-> Number of messages allocated
300 \-> Maximum number of messages allocated
303 \-> Number of errors encountered
306 \-> Number of messages sent
309 \-> Number of messages received
312 \-> Number of messages routed
315 \-> Total size in bytes of messages sent
318 \-> Total size in bytes of messages received
321 \-> Total size in bytes of messages routed
324 \-> Total size in bytes of messages dropped
331 .SS "Initializing LNet after load"
334 lnetctl lnet configure
337 lnetctl lnet configure \-\-all
341 .SS "Shutting down LNet"
344 lnetctl lnet unconfigure
351 lnetctl net add \-\-net tcp0 \-\-if eth0
354 lnetctl net add \-\-ip2net "tcp0(eth0) 192\.168\.0\.[2,4]; tcp0 192\.168\.0\.*;
355 o2ib0 132\.6\.[1\-3]\.[2\-8/2]"
362 lnetctl net del \-\-net tcp0
369 lnetctl net show \-\-verbose:
391 peer_buffer_credits: 0
397 \- nid: 192\.168\.205\.130@tcp1
421 peer_buffer_credits: 0
431 lnetctl route add \-\-net tcp0 \-\-gateway 10\.10\.10\.1@tcp1 \-\-hop 1
439 lnetctl route del \-\-net tcp0 \-\-gateway 10\.10\.10\.1@tcp1
446 lnetctl route show \-\-verbose
457 gateway: 192\.168\.205\.131@tcp1
463 priority: 0 state: down
528 .SS "Setting variables"
531 lnetctl set tiny_buffers 2048
534 lnetctl set small_buffers 16384
537 lnetctl set large_buffers 256
540 lnetctl set routing 1
544 .SS "Importing YAML files for configuring"
547 lnetctl import lnet\.conf
550 lnetctl import < lnet\.conf
554 .SS "Exporting LNet Configuration"
557 lnetctl export lnet\.conf
560 lnetctl export > lnet\.conf
564 .SS "Showing LNet Stats"
609 .SS "Showing peer information"
620 \- primary nid: 10\.148\.0\.8@o2ib
629 \- nid: 10\.148\.0\.8@o2ib
635 \- primary nid: 10\.148\.0\.20@o2ib
644 \- nid: 10\.148\.0\.20@o2ib
650 \- nid: 10\.148\.0\.25@o2ib