From 7d1f65c497503390e592ee71f95ac310f0054a16 Mon Sep 17 00:00:00 2001 From: Joseph Gmitter Date: Wed, 7 Dec 2016 10:52:02 -0500 Subject: [PATCH] LUDOC-324 lnet: Change LNET to LNet across the entire manual Change the abbreviation for Lustre Networking from LNET to LNet across the manual. LNet was chosen as the proper abbreviation as all caps typically indicates an acronym. Signed-off-by: Joseph Gmitter Change-Id: I2677184789314316a75dc4bc66fb66a4c34c5d28 Reviewed-on: https://review.whamcloud.com/24198 Tested-by: Jenkins --- BenchmarkingTests.xml | 2 +- ConfigurationFilesModuleParameters.xml | 26 +++--- ConfiguringLNET.xml | 146 ++++++++++++++++----------------- ConfiguringLustre.xml | 12 +-- Glossary.xml | 16 ++-- InstallOverview.xml | 8 +- InstallingLustre.xml | 8 +- LNETSelfTest.xml | 48 +++++------ LNetConfigurationApi.xml | 70 ++++++++-------- LustreDebugging.xml | 6 +- LustreMaintenance.xml | 4 +- LustreNodemap.xml | 2 +- LustreOperations.xml | 4 +- LustreProc.xml | 16 ++-- LustreTroubleshooting.xml | 4 +- LustreTuning.xml | 42 +++++----- ManagingLNET.xml | 62 +++++++------- ManagingSecurity.xml | 6 +- SettingUpLustreSystem.xml | 12 +-- SystemConfigurationUtilities.xml | 40 ++++----- UnderstandingLustre.xml | 10 +-- UnderstandingLustreNetworking.xml | 54 ++++++------ 22 files changed, 299 insertions(+), 299 deletions(-) diff --git a/BenchmarkingTests.xml b/BenchmarkingTests.xml index 7369fdf..ba90041 100644 --- a/BenchmarkingTests.xml +++ b/BenchmarkingTests.xml @@ -68,7 +68,7 @@ Password-free remote access to nodes in the system (provided by ssh or rsh). - LNET self-test completed to test that Lustre networking has been properly installed + LNet self-test completed to test that Lustre networking has been properly installed and configured. See . diff --git a/ConfigurationFilesModuleParameters.xml b/ConfigurationFilesModuleParameters.xml index 67d112d..e526884 100644 --- a/ConfigurationFilesModuleParameters.xml +++ b/ConfigurationFilesModuleParameters.xml @@ -12,16 +12,16 @@
<indexterm><primary>configuring</primary></indexterm> - <indexterm><primary>LNET</primary><see>configuring</see></indexterm> + <indexterm><primary>LNet</primary><see>configuring</see></indexterm> Introduction - LNET network hardware and routing are now configured via module parameters. Parameters should be specified in the /etc/modprobe.d/lustre.conffile, for example: + LNet network hardware and routing are now configured via module parameters. Parameters should be specified in the /etc/modprobe.d/lustre.conffile, for example: options lnet networks=tcp0(eth2) The above option specifies that this node should use the TCP protocol on the eth2 network interface. - Module parameters are read when the module is first loaded. Type-specific LND modules (for instance, ksocklnd) are loaded automatically by the LNET module when LNET starts (typically upon modprobe ptlrpc). - LNET configuration parameters can be viewed under /sys/module/lnet/parameters/, and LND-specific parameters under the name of the corresponding LND, for example /sys/module/ksocklnd/parameters/ for the socklnd (TCP) LND. - For the following parameters, default option settings are shown in parenthesis. Changes to parameters marked with a W affect running systems. Unmarked parameters can only be set when LNET loads for the first time. Changes to parameters marked with Wc only have effect when connections are established (existing connections are not affected by these changes.) + Module parameters are read when the module is first loaded. Type-specific LND modules (for instance, ksocklnd) are loaded automatically by the LNet module when LNet starts (typically upon modprobe ptlrpc). + LNet configuration parameters can be viewed under /sys/module/lnet/parameters/, and LND-specific parameters under the name of the corresponding LND, for example /sys/module/ksocklnd/parameters/ for the socklnd (TCP) LND. + For the following parameters, default option settings are shown in parenthesis. Changes to parameters marked with a W affect running systems. Unmarked parameters can only be set when LNet loads for the first time. Changes to parameters marked with Wc only have effect when connections are established (existing connections are not affected by these changes.)
@@ -39,7 +39,7 @@ <para>A separate <literal>lustre.conf</literal> file makes distributing the configuration much easier.</para> </listitem> <listitem> - <para>If you set <literal>config_on_load=1</literal>, LNET starts at + <para>If you set <literal>config_on_load=1</literal>, LNet starts at <literal>modprobe</literal> time rather than waiting for the Lustre file system to start. This ensures routers start working at module load time.</para> </listitem> @@ -48,13 +48,13 @@ # lctl> net down</screen> <itemizedlist> <listitem> - <para>Remember the <literal>lctl ping {nid}</literal> command - it is a handy way to check your LNET configuration.</para> + <para>Remember the <literal>lctl ping {nid}</literal> command - it is a handy way to check your LNet configuration.</para> </listitem> </itemizedlist> <section remap="h3"> - <title><indexterm><primary>configuring</primary><secondary>LNET options</secondary></indexterm> -LNET Options - This section describes LNET options. + <indexterm><primary>configuring</primary><secondary>LNet options</secondary></indexterm> +LNet Options + This section describes LNet options.
<indexterm><primary>configuring</primary><secondary>network topology</secondary></indexterm> Network Topology @@ -107,7 +107,7 @@ Network Topology Lustre file system uses any IP addresses aliased to the loopback (by default). When in doubt, explicitly specify networks. - ip2nets ("") is a string that lists globally-available networks, each with a set of IP address ranges. LNET determines the locally-available networks from this list by matching the IP address ranges with the local IPs of a node. The purpose of this option is to be able to use the same modules.conf file across a variety of nodes on different networks. The string has the following syntax. + ip2nets ("") is a string that lists globally-available networks, each with a set of IP address ranges. LNet determines the locally-available networks from this list by matching the IP address ranges with the local IPs of a node. The purpose of this option is to be able to use the same modules.conf file across a variety of nodes on different networks. The string has the following syntax. <ip2nets> :== <net-match> [ <comment> ] { <net-sep> <net-match> } <net-match> :== [ <w> ] <net-spec> <w> <ip-range> { <w> <ip-range> } [ <w> ] @@ -184,7 +184,7 @@ routes ("") <indexterm><primary>configuring</primary><secondary>network</secondary><tertiary>forwarding</tertiary></indexterm> forwarding ("") This is a string that can be set either to "enabled" or "disabled" for explicit control of whether this node should act as a router, forwarding communications between all local networks. - A standalone router can be started by simply starting LNET ('modprobe ptlrpc') with appropriate network topology options. + A standalone router can be started by simply starting LNet ('modprobe ptlrpc') with appropriate network topology options. @@ -205,7 +205,7 @@ forwarding ("") acceptor - The acceptor is a TCP/IP service that some LNDs use to establish communications. If a local network requires it and it has not been disabled, the acceptor listens on a single port for connection requests that it redirects to the appropriate local network. The acceptor is part of the LNET module and configured by the following options: + The acceptor is a TCP/IP service that some LNDs use to establish communications. If a local network requires it and it has not been disabled, the acceptor listens on a single port for connection requests that it redirects to the appropriate local network. The acceptor is part of the LNet module and configured by the following options: secure - Accept connections only from reserved TCP ports (below 1023). diff --git a/ConfiguringLNET.xml b/ConfiguringLNET.xml index 653f44e..fa461e1 100755 --- a/ConfiguringLNET.xml +++ b/ConfiguringLNET.xml @@ -1,7 +1,7 @@ - Configuring Lustre Networking (LNET) - This chapter describes how to configure Lustre Networking (LNET). It includes the following sections: + Configuring Lustre Networking (LNet) + This chapter describes how to configure Lustre Networking (LNet). It includes the following sections: @@ -37,28 +37,28 @@ - Configuring LNET is optional. - LNET will use the first TCP/IP interface it discovers on a + Configuring LNet is optional. + LNet will use the first TCP/IP interface it discovers on a system (eth0) if it's loaded using the lctl network up. If this network configuration is - sufficient, you do not need to configure LNET. LNet configuration is + sufficient, you do not need to configure LNet. LNet configuration is required if you are using Infiniband or multiple Ethernet interfaces. The lnetctl utility can be used - to initialize LNET without bringing up any network interfaces. This - gives flexibility to the user to add interfaces after LNET has been + to initialize LNet without bringing up any network interfaces. This + gives flexibility to the user to add interfaces after LNet has been loaded. DLC also introduces a C-API to enable - configuring LNET programatically. See
<indexterm> - <primary>LNET</primary> - <secondary>Configuring LNET</secondary> - </indexterm>Configuring LNET via <literal>lnetctl</literal> + LNet + Configuring LNet + Configuring LNet via lnetctl The lnetctl utility can be used to initialize - and configure the LNET kernel module after it has been loaded via + and configure the LNet kernel module after it has been loaded via modprobe. In general the lnetctl format is as follows: lnetctl cmd subcmd [options] @@ -66,7 +66,7 @@ - Configuring/unconfiguring LNET + Configuring/unconfiguring LNet Adding/removing/showing Networks @@ -84,11 +84,11 @@
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> - </indexterm>Configuring LNET - After LNET has been loaded via modprobe, - lnetctl utility can be used to configure LNET + Configuring LNet + After LNet has been loaded via modprobe, + lnetctl utility can be used to configure LNet without bringing up networks which are specified in the module parameters. It can also be used to configure network interfaces specified in the module prameters by providing the @@ -96,15 +96,15 @@ lnetctl lnet configure [--all] # --all: load NI configuration from module parameters The lnetctl utility can also be used to - unconfigure LNET. + unconfigure LNet. lnetctl lnet unconfigure
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Adding, Deleting and Showing networks - Networks can be added and deleted after the LNET kernel module + Networks can be added and deleted after the LNet kernel module is loaded. lnetctl net add: add a network --net: net name (ex tcp0) @@ -161,16 +161,16 @@ net:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Adding, Deleting and Showing routes - A set of routes can be added to identify how LNET messages are + A set of routes can be added to identify how LNet messages are to be routed. lnetctl route add: add a route - --net: net name (ex tcp0) LNET message is destined to. + --net: net name (ex tcp0) LNet message is destined to. The can not be a local network. --gateway: gateway node nid (ex 10.1.1.2@tcp) to route - all LNET messaged destined for the identified + all LNet messaged destined for the identified network --hop: number of hops to final destination (1 < hops < 255) @@ -223,10 +223,10 @@ route:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Enabling and Disabling Routing - When an LNET node is configured as a router it will route LNet + When an LNet node is configured as a router it will route LNet messages not destined to itself. This feature can be enabled or disabled as follows. lnetctl set routing [0 | 1] @@ -235,7 +235,7 @@ route:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Showing routing information When routing is enabled on a node, the tiny, small and large @@ -269,7 +269,7 @@ routing:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Configuring Routing Buffers The routing buffers values configured specify the number of @@ -308,7 +308,7 @@ set large_buffers: set large routing buffers
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Importing YAML Configuration File Configuration can be described in YAML format and can be fed @@ -337,26 +337,26 @@ lnetctl import --show < FILE.yaml
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> </indexterm>Exporting Configuration in YAML format lnetctl utility provides the 'export' - command to dump current LNET configuration in YAML format + command to dump current LNet configuration in YAML format lnetctl export FILE.yaml lnetctl export > FILE.yaml
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cli</secondary> - </indexterm>Showing LNET Traffic Statistics - lnetctl utility can dump the LNET traffic + Showing LNet Traffic Statistics + lnetctl utility can dump the LNet traffic statistiscs as follows lnetctl stats show
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>yaml syntax</secondary> </indexterm>YAML Syntax The lnetctl utility can take in a YAML file @@ -378,7 +378,7 @@ lnetctl export > FILE.yaml operation.
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>network yaml syntax</secondary> </indexterm>Network Configuration @@ -400,7 +400,7 @@ lnetctl export > FILE.yaml
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>buffer yaml syntax</secondary> </indexterm>Enable Routing and Adjust Router Buffer Configuration @@ -414,7 +414,7 @@ lnetctl export > FILE.yaml
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>statistics yaml syntax</secondary> </indexterm>Show Statistics @@ -424,7 +424,7 @@ lnetctl export > FILE.yaml
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>router yaml syntax</secondary> </indexterm>Route Configuration @@ -439,14 +439,14 @@ lnetctl export > FILE.yaml
- <indexterm><primary>LNET</primary></indexterm> + <title><indexterm><primary>LNet</primary></indexterm> - Overview of LNET Module Parameters - LNET kernel module (lnet) parameters specify how LNet is to be + Overview of LNet Module Parameters + LNet kernel module (lnet) parameters specify how LNet is to be configured to work with Lustre, including which NICs will be configured to work with Lustre and the routing to be used with Lustre. - Parameters for LNET can be specified in the + Parameters for LNet can be specified in the /etc/modprobe.d/lustre.conf file. In some cases the parameters may have been stored in /etc/modprobe.conf, but this has been deprecated @@ -466,7 +466,7 @@ lnetctl export > FILE.yaml ip2nets - Lists globally-available - networks, each with a range of IP addresses. LNET then identifies + networks, each with a range of IP addresses. LNet then identifies locally-available networks through address list-matching lookup. @@ -483,8 +483,8 @@ lnetctl export > FILE.yaml Lustre nodes to detect router health status, avoid routers that appear dead, and reuse those that restore service after failures. See for more details. - For a complete reference to the LNET module parameters, see - LNET + For a complete reference to the LNet module parameters, see + LNet Options. We recommend that you use 'dotted-quad' notation for @@ -492,7 +492,7 @@ lnetctl export > FILE.yaml logs and debug configurations with multiple interfaces.
- <indexterm><primary>LNET</primary><secondary>using + <title><indexterm><primary>LNet</primary><secondary>using NID</secondary></indexterm>Using a Lustre Network Identifier (NID) to Identify a Node A Lustre network identifier (NID) is used to uniquely identify @@ -517,11 +517,11 @@ lnetctl export > FILE.yaml
- <indexterm><primary>LNET</primary><secondary>module parameters</secondary></indexterm>Setting the LNet Module networks Parameter + <indexterm><primary>LNet</primary><secondary>module parameters</secondary></indexterm>Setting the LNet Module networks Parameter If a node has more than one network interface, you'll typically want to dedicate a specific interface to Lustre. You can do this by including an entry in the lustre.conf file - on the node that sets the LNET module networks + on the node that sets the LNet module networks parameter: options lnet networks=comma-separated list of networks @@ -543,7 +543,7 @@ lnetctl export > FILE.yaml another interface, even if multiple interfaces are available on the same node. - LNET lines in lustre.conf are only used by + LNet lines in lustre.conf are only used by the local node to determine what to call its interfaces. They are not used for routing decisions. @@ -600,7 +600,7 @@ lnetctl export > FILE.yaml (lo0) interface. Lustre does not ignore IP addresses aliased to the loopback. If you alias IP addresses to the loopback interface, you must specify all Lustre networks using - the LNET networks parameter. + the LNet networks parameter. If the server has multiple interfaces on the same subnet, @@ -613,7 +613,7 @@ lnetctl export > FILE.yaml
- <indexterm><primary>LNET</primary><secondary>ip2nets</secondary></indexterm>Setting the LNet Module ip2nets Parameter + <indexterm><primary>LNet</primary><secondary>ip2nets</secondary></indexterm>Setting the LNet Module ip2nets Parameter The ip2nets option is typically used when a single, universal lustre.conf file is run on all servers and clients. Each node identifies the locally available @@ -622,7 +622,7 @@ lnetctl export > FILE.yaml Note that the IP address patterns listed in the ip2nets option are only used to identify the networks that an individual node should instantiate. - They are not used by LNET for any other + They are not used by LNet for any other communications purpose. For the example below, the nodes in the network have these IP addresses: @@ -654,12 +654,12 @@ lnetctl export > FILE.yaml options lnet 'ip2nets="tcp0(eth0) 192.168.0.[2,4]; \ tcp0 192.168.0.*; o2ib0 132.6.[1-3].[2-8/2]"' Each entry in ip2nets is referred to as a 'rule'. - The order of LNET entries is important when configuring servers. + The order of LNet entries is important when configuring servers. If a server node can be reached using more than one network, the first network specified in lustre.conf will be used. Because svr1 and svr2 - match the first rule, LNET uses eth0 for + match the first rule, LNet uses eth0 for tcp0 on those machines. (Although svr1 and svr2 also match the second rule, the first matching rule for a particular network is @@ -670,15 +670,15 @@ tcp0 192.168.0.*; o2ib0 132.6.[1-3].[2-8/2]"' network.
- <indexterm><primary>LNET</primary><secondary>routes</secondary></indexterm>Setting + <title><indexterm><primary>LNet</primary><secondary>routes</secondary></indexterm>Setting the LNet Module routes Parameter - The LNET module routes parameter is used to identify routers in + The LNet module routes parameter is used to identify routers in a Lustre configuration. These parameters are set in modprobe.conf on each Lustre node. Routes are typically set to connect to segregated subnetworks or to cross connect two different types of networks such as tcp and o2ib - The LNET routes parameter specifies a colon-separated list of + The LNet routes parameter specifies a colon-separated list of router definitions. Each route is defined as a network number, followed by a list of routers: routes=net_type router_NID(s) @@ -688,14 +688,14 @@ tcp0 192.168.0.*; o2ib0 132.6.[1-3].[2-8/2]"' options lnet 'ip2nets="tcp0 192.168.0.*; \ o2ib0(ib0) 132.6.1.[1-128]"' 'routes="tcp0 132.6.1.[1-8]@o2ib0; \ o2ib0 192.16.8.0.[1-8]@tcp0"' - All LNET routers that bridge two networks are equivalent. They + All LNet routers that bridge two networks are equivalent. They are not configured as primary or secondary, and the load is balanced across all available routers. - The number of LNET routers is not limited. Enough routers should + The number of LNet routers is not limited. Enough routers should be used to handle the required file serving bandwidth plus a 25 percent margin for headroom.
- <indexterm><primary>LNET</primary><secondary>routing + <title><indexterm><primary>LNet</primary><secondary>routing example</secondary></indexterm>Routing Example On the clients, place the following entry in the lustre.conf file @@ -710,15 +710,15 @@ lctl network configure
- <indexterm><primary>LNET</primary><secondary>testing</secondary></indexterm>Testing + <title><indexterm><primary>LNet</primary><secondary>testing</secondary></indexterm>Testing the LNet Configuration After configuring Lustre Networking, it is highly recommended - that you test your LNET configuration using the LNet Self-Test + that you test your LNet configuration using the LNet Self-Test provided with the Lustre software. For more information about using - LNET Self-Test, see . + LNet Self-Test, see .
- <indexterm><primary>LNET</primary><secondary>route + <title><indexterm><primary>LNet</primary><secondary>route checker</secondary></indexterm>Configuring the Router Checker In a Lustre configuration in which different types of networks, such as a TCP/IP network and an Infiniband network, are connected by @@ -726,7 +726,7 @@ lctl network configure routed configuration to monitor the status of the routers. In a multi-hop routing configuration, router checkers can be configured on routers to monitor the health of their next-hop routers. - A router checker is configured by setting lnet parameters in + A router checker is configured by setting LNet parameters in lustre.conf by including an entry in this form: options lnet @@ -795,14 +795,14 @@ lctl network configure sent-packets counter for that router will have a value of 100.
- <indexterm><primary>LNET</primary><secondary>best - practice</secondary></indexterm>Best Practices for LNET + <title><indexterm><primary>LNet</primary><secondary>best + practice</secondary></indexterm>Best Practices for LNet Options For the networks, ip2nets, and routes options, follow these best practices to avoid configuration errors.
- <indexterm><primary>LNET</primary><secondary>escaping commas + <title><indexterm><primary>LNet</primary><secondary>escaping commas with quotes</secondary></indexterm>Escaping commas with quotes Depending on the Linux distribution, commas may need to be @@ -815,17 +815,17 @@ lctl network configure the following may indicate an issue related to added quotes: lnet: Unknown parameter 'networks' A 'Refusing connection - no matching - NID' message generally points to an error in the LNET + NID' message generally points to an error in the LNet module configuration.
- <indexterm><primary>LNET</primary><secondary>comments</secondary></indexterm>Including + <title><indexterm><primary>LNet</primary><secondary>comments</secondary></indexterm>Including comments Place the semicolon terminating a comment - immediately after the comment. LNET silently ignores + immediately after the comment. LNet silently ignores everything between the # character at the beginning of the comment and the next semicolon. - In this incorrect example, LNET silently + In this incorrect example, LNet silently ignores pt11 192.168.0.[92,96], resulting in these nodes not being properly initialized. No error message is generated. diff --git a/ConfiguringLustre.xml b/ConfiguringLustre.xml index 7ff40d1..c40a5dd 100644 --- a/ConfiguringLustre.xml +++ b/ConfiguringLustre.xml @@ -68,17 +68,17 @@ xml:id="configuringlustre"> Set lnet - module parameters to specify how Lustre Networking (LNET) is - to be configured to work with a Lustre file system and test the LNET - configuration.LNET will, by default, use the first TCP/IP + module parameters to specify how Lustre Networking (LNet) is + to be configured to work with a Lustre file system and test the LNet + configuration.LNet will, by default, use the first TCP/IP interface it discovers on a system. If this network configuration is - sufficient, you do not need to configure LNET. LNET configuration is + sufficient, you do not need to configure LNet. LNet configuration is required if you are using InfiniBand or multiple Ethernet interfaces. - For information about configuring LNET, see - . For information about testing LNET, see + For information about configuring LNet, see + . For information about testing LNet, see . diff --git a/Glossary.xml b/Glossary.xml index 172f7d7..ad1e829 100644 --- a/Glossary.xml +++ b/Glossary.xml @@ -335,16 +335,16 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"> LND - Lustre network driver. A code module that enables LNET support + Lustre network driver. A code module that enables LNet support over particular transports, such as TCP and various kinds of InfiniBand networks. - LNET + LNet Lustre networking. A message passing network protocol capable of - running and routing through various physical layers. LNET forms the + running and routing through various physical layers. LNet forms the underpinning of LNETrpc. @@ -410,7 +410,7 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"> MDC Metadata client. A Lustre client component that sends metadata - requests via RPC over LNET to the metadata target (MDT). + requests via RPC over LNet to the metadata target (MDT). @@ -479,7 +479,7 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"> NIO API - A subset of the LNET RPC module that implements a library for + A subset of the LNet RPC module that implements a library for sending large network requests, moving buffers with RDMA. @@ -638,7 +638,7 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"> Portal - A service address on an LNET NID that binds requests to a + A service address on an LNet NID that binds requests to a specific request service, such as an MDS, MGS, OSS, or LDLM. Services may listen on multiple portals to ensure that high priority messages are not queued behind many slow requests on another portal. @@ -647,7 +647,7 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"> PTLRPC - An RPC protocol layered on LNET. This protocol deals with + An RPC protocol layered on LNet. This protocol deals with stateful servers and has exactly-once semantics and built in support for recovery. @@ -723,7 +723,7 @@ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"> Routing - LNET routing between different networks and LNDs. + LNet routing between different networks and LNDs. diff --git a/InstallOverview.xml b/InstallOverview.xml index 1c0fa51..223daa5 100644 --- a/InstallOverview.xml +++ b/InstallOverview.xml @@ -33,10 +33,10 @@ (Optional) - Configure Lustre networking (LNET). - See - Describes how to configure LNET if the default - configuration is not sufficient. By default, LNET will use the first TCP/IP interface it - discovers on a system. LNET configuration is required if you are using InfiniBand or + Configure Lustre Networking (LNet). + See - Describes how to configure LNet if the default + configuration is not sufficient. By default, LNet will use the first TCP/IP interface it + discovers on a system. LNet configuration is required if you are using InfiniBand or multiple Ethernet interfaces. diff --git a/InstallingLustre.xml b/InstallingLustre.xml index 05d4207..973dbc7 100644 --- a/InstallingLustre.xml +++ b/InstallingLustre.xml @@ -276,9 +276,9 @@ xml:id="installinglustre"> - Lustre LNET network driver (LND) + Lustre LNet network driver (LND) . The Lustre LNDs provided with the Lustre software are - listed in the table below. For more information about Lustre LNET, + listed in the table below. For more information about Lustre LNet, see . @@ -579,9 +579,9 @@ rpm -qa|egrep "lustre|wc"|sort - To configure LNET, go to + To configure LNet, go to . If default settings will be used for - LNET, go to + LNet, go to .
diff --git a/LNETSelfTest.xml b/LNETSelfTest.xml index c717ca3..bf53166 100644 --- a/LNETSelfTest.xml +++ b/LNETSelfTest.xml @@ -1,7 +1,7 @@ - Testing Lustre Network Performance (LNET Self-Test) - This chapter describes the LNET - self-test, which is used by site administrators to confirm that Lustre networking (LNET) has + Testing Lustre Network Performance (LNet Self-Test) + This chapter describes the LNet + self-test, which is used by site administrators to confirm that Lustre Networking (LNet) has been properly installed and configured, and that underlying network software and hardware are performing according to expectations. The chapter includes: @@ -16,9 +16,9 @@
- <indexterm><primary>LNET</primary><secondary>self-test</secondary></indexterm> -LNET Self-Test Overview - LNET self-test is a kernel module that runs over LNET and the Lustre network drivers (LNDs). It is designed to: + <indexterm><primary>LNet</primary><secondary>self-test</secondary></indexterm> +LNet Self-Test Overview + LNet self-test is a kernel module that runs over LNet and the Lustre network drivers (LNDs). It is designed to: Test the connection ability of the Lustre network @@ -30,21 +30,21 @@ LNET Self-Test Overview Test performance of the Lustre network - After you have obtained performance results for your Lustre network, refer to for information about parameters that can be used to tune LNET for optimum performance. + After you have obtained performance results for your Lustre network, refer to for information about parameters that can be used to tune LNet for optimum performance. - Apart from the performance impact, LNET self-test is invisible to the Lustre file + Apart from the performance impact, LNet self-test is invisible to the Lustre file system. - An LNET self-test cluster includes two types of nodes: + An LNet self-test cluster includes two types of nodes: - Console node - A node used to control and monitor an LNET self-test cluster. The console node serves as the user interface of the LNET self-test system and can be any node in the test cluster. All self-test commands are entered from the console node. From the console node, a user can control and monitor the status of the entire LNET self-test cluster (session). The console node is exclusive in that a user cannot control two different sessions from one console node. + Console node - A node used to control and monitor an LNet self-test cluster. The console node serves as the user interface of the LNet self-test system and can be any node in the test cluster. All self-test commands are entered from the console node. From the console node, a user can control and monitor the status of the entire LNet self-test cluster (session). The console node is exclusive in that a user cannot control two different sessions from one console node. Test nodes - The nodes on which the tests are run. Test nodes are controlled by the user from the console node; the user does not need to log into them directly. - LNET self-test has two user utilities: + LNet self-test has two user utilities: @@ -54,7 +54,7 @@ LNET Self-Test Overview lstclient - - The userspace LNET self-test program (run on a test node). The lstclient utility is linked with userspace LNDs and LNET. This utility is not needed if only kernel space LNET and LNDs are used. + - The userspace LNet self-test program (run on a test node). The lstclient utility is linked with userspace LNDs and LNet. This utility is not needed if only kernel space LNet and LNDs are used. @@ -62,7 +62,7 @@ LNET Self-Test Overview
Prerequisites - To run LNET self-test, these modules must be loaded on both console nodes and test nodes: + To run LNet self-test, these modules must be loaded on both console nodes and test nodes: libcfs @@ -79,15 +79,15 @@ LNET Self-Test Overview To load the required modules, run: modprobe lnet_selftest - This command recursively loads the modules on which LNET self-test depends. + This command recursively loads the modules on which LNet self-test depends. While the console node and test nodes require all the prerequisite modules to be loaded, userspace test nodes do not require these modules.
- Using LNET Self-Test - This section describes how to create and run an LNET self-test. The examples shown are for a test that simulates the traffic pattern of a set of Lustre servers on a TCP network accessed by Lustre clients on an InfiniBand network connected via LNET routers. In this example, half the clients are reading and half the clients are writing. + Using LNet Self-Test + This section describes how to create and run an LNet self-test. The examples shown are for a test that simulates the traffic pattern of a set of Lustre servers on a TCP network accessed by Lustre clients on an InfiniBand network connected via LNet routers. In this example, half the clients are reading and half the clients are writing.
Creating a Session A session is a set of processes that run on a test node. Only one session can be run at a time on a test node to ensure that the session has exclusive use of the node. The console node is used to create, change or destroy a session (new_session, end_session, show_session). For more about session parameters, see . @@ -98,7 +98,7 @@ lst new_session read_write
Setting Up Groups - A group is a named collection of nodes. Any number of groups can exist in a single LNET self-test session. Group membership is not restricted in that a test node can be included in any number of groups. + A group is a named collection of nodes. Any number of groups can exist in a single LNet self-test session. Group membership is not restricted in that a test node can be included in any number of groups. Each node in a group has a rank, determined by the order in which it was added to the group. The rank is used to establish test traffic patterns. A user can only control nodes in his/her session. To allocate nodes to the session, the user needs to add nodes to a group (of the session). All nodes in a group can be referenced by the group name. A node can be allocated to multiple groups of a session. In the following example, three groups are established on a console node: @@ -108,7 +108,7 @@ lst add_group writers 192.168.1.[2-254/2]@o2ib These three groups include: - Nodes that will function as 'servers' to be accessed by 'clients' during the LNET self-test session + Nodes that will function as 'servers' to be accessed by 'clients' during the LNet self-test session Nodes that will function as 'clients' that will simulate reading data from the 'servers' @@ -153,7 +153,7 @@ lst add_test --batch bulk_rw --from writers --to servers \
Sample Script - This sample LNET self-test script simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an InfiniBand network (connected via LNET routers). In this example, half the clients are reading and half the clients are writing. + This sample LNet self-test script simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an InfiniBand network (connected via LNet routers). In this example, half the clients are reading and half the clients are writing. Run this script on the console node: #!/bin/bash export LST_SESSION=$$ @@ -178,8 +178,8 @@ lst end_session
- LNET Self-Test Command Reference - The LNET self-test (lst) utility is used to issue LNET self-test commands. The lst utility takes a number of command line arguments. The first argument is the command name and subsequent arguments are command-specific. + LNet Self-Test Command Reference + The LNet self-test (lst) utility is used to issue LNet self-test commands. The lst utility takes a number of command line arguments. The first argument is the command name and subsequent arguments are command-specific.
Session Commands This section describes lst session commands. @@ -288,7 +288,7 @@ lst end_session NIDs - A string that may be expanded to include one or more LNET NIDs. + A string that may be expanded to include one or more LNet NIDs. @@ -538,7 +538,7 @@ Total 1 node --server_mode - When included, forces LNET to behave as a server, such as starting an acceptor if the underlying NID needs it or using privileged ports. Only root is allowed to use the --server_mode option. + When included, forces LNet to behave as a server, such as starting an acceptor if the underlying NID needs it or using privileged ports. Only root is allowed to use the --server_mode option. @@ -1085,7 +1085,7 @@ $ lst stat clients where servers is the name of a test group created by lst add_group Specifying a NID range (NIDs) causes statistics to be gathered for selected nodes. For example: $ lst stat 192.168.0.[1-100/2]@tcp - Only LNET performance statistics are available. By default, all statistics + Only LNet performance statistics are available. By default, all statistics information is displayed. Users can specify additional information with these options. show_error [--session] [group|NIDs]... Lists the number of failed RPCs on test nodes. diff --git a/LNetConfigurationApi.xml b/LNetConfigurationApi.xml index 1151974..cb15da8 100755 --- a/LNetConfigurationApi.xml +++ b/LNetConfigurationApi.xml @@ -1,10 +1,10 @@ LNet Configuration C-API - This section describes the LNET Configuration C-API library. This - API allows the developer to programatically configure LNET. It provides - APIs to add, delete and show LNET configuration items listed below. The + This section describes the LNet Configuration C-API library. This + API allows the developer to programatically configure LNet. It provides + APIs to add, delete and show LNet configuration items listed below. The API utilizes IOCTL to communicate with the kernel. Changes take effect - immediately and do not require restarting LNET. API calls are + immediately and do not require restarting LNet. API calls are synchronous @@ -27,13 +27,13 @@
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>capi general information</secondary> </indexterm>General API Information
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>capi return code</secondary> </indexterm>API Return Code LUSTRE_CFG_RC_NO_ERR 0 @@ -45,7 +45,7 @@ LUSTRE_CFG_RC_GENERIC_ERR -5
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>capi input params</secondary> </indexterm>API Common Input Parameters All APIs take as input a sequence number. This is a number @@ -60,13 +60,13 @@ LUSTRE_CFG_RC_GENERIC_ERR -5
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>capi output params</secondary> </indexterm>API Common Output Parameters
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>cyaml</secondary> </indexterm>Internal YAML Representation (cYAML) Once a YAML block is parsed it needs to be stored @@ -111,7 +111,7 @@ typedef struct cYAML {
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>error block</secondary> </indexterm>Error Block All APIs return a cYAML error block. This error block has @@ -132,7 +132,7 @@ add:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>show block</secondary> </indexterm>Show Block All Show APIs return a cYAML show block. This show block @@ -155,20 +155,20 @@ add:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>show block</secondary> - </indexterm>The LNET Configuration C-API + The LNet Configuration C-API
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_config_ni_system</secondary> </indexterm>Configuring LNet /* * lustre_lnet_config_ni_system - * Initialize/Uninitialize the lnet NI system. + * Initialize/Uninitialize the LNet NI system. * * up - whether to init or uninit the system * load_ni_from_mod - load NI from mod params. @@ -195,7 +195,7 @@ int lustre_lnet_config_ni_system(bool up, bool load_ni_from_mod,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_enable_routing</secondary> </indexterm>Enabling and Disabling Routing @@ -225,14 +225,14 @@ extern int lustre_lnet_enable_routing(int enable, messages not destined to this node are dropped. Enabling Routing on an already enabled node, or vice versa - In both these cases the LNET Kernel module ignores this request. + In both these cases the LNet Kernel module ignores this request. Return Value -ENOMEM: if there is no memory to allocate buffer pools 0: if success
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_config_route</secondary> </indexterm>Adding Routes @@ -253,7 +253,7 @@ extern int lustre_lnet_config_route(char *nw, char *gw, IOCTL to Kernel: IOC_LIBCFS_ADD_ROUTE Description: - The LNET Kernel module adds this route to the list of + The LNet Kernel module adds this route to the list of existing routes, if one doesn't already exist. If hop parameter is not specified (IE: -1) then the hop count is set to 1. If the priority parameter is not specified (IE: -1) then the priority is @@ -269,7 +269,7 @@ extern int lustre_lnet_config_route(char *nw, char *gw,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_del_route</secondary> </indexterm>Deleting Routes @@ -286,7 +286,7 @@ extern int lustre_lnet_del_route(char *nw, char *gw, IOCTL to Kernel: IOC_LIBCFS_DEL_ROUTE Description: - LNET will remove the route which matches the network and gateway passed in. If + LNet will remove the route which matches the network and gateway passed in. If no route matches, then the operation fails with an appropriate error number. Return Value -ENOENT: if the entry being deleted doesn't exist @@ -294,7 +294,7 @@ extern int lustre_lnet_del_route(char *nw, char *gw,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_show_route</secondary> </indexterm>Showing Routes @@ -343,7 +343,7 @@ extern int lustre_lnet_show_route(char *nw, char *gw,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_config_net</secondary> </indexterm>Adding a Network Interface @@ -379,7 +379,7 @@ extern int lustre_lnet_config_net(char *net, peer timeout, peer credits, peer buffer credits and credits. The CPU affinity of the network interface being added can also be specified. These parameters become - network specific under Dynamic LNET Configuration (DLC), as + network specific under Dynamic LNet Configuration (DLC), as opposed to being per LND as it was previously. If an already existing network is added the request is ignored. Return Value @@ -389,7 +389,7 @@ extern int lustre_lnet_config_net(char *net,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_del_net</secondary> </indexterm>Deleting a Network Interface @@ -416,7 +416,7 @@ extern int lustre_lnet_del_net(char *nw,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_show_net</secondary> </indexterm>Showing Network Interfaces @@ -465,7 +465,7 @@ extern int lustre_lnet_show_net(char *nw, int detail,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_config_buf</secondary> </indexterm>Adjusting Router Buffer Pools @@ -508,7 +508,7 @@ extern int lustre_lnet_config_buf(int tiny,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_show_buf</secondary> </indexterm>Showing Routing information @@ -574,14 +574,14 @@ extern int lustre_lnet_show_routing(int seq_no, struct cYAML **show_rc,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_lnet_show stats</secondary> - </indexterm>Showing LNET Traffic Statistics + Showing LNet Traffic Statistics /* * lustre_lnet_show_stats - * Shows internal LNET statistics. This is useful to display the - * current LNET activity, such as number of messages route, etc + * Shows internal LNet statistics. This is useful to display the + * current LNet activity, such as number of messages route, etc * * seq_no - sequence number of the command * show_rc - YAML structure of the resultant show @@ -592,8 +592,8 @@ extern int lustre_lnet_show_stats(int seq_no, cYAML **show_rc, IOCTL to Kernel: IOC_LIBCFS_GET_LNET_STATS Description: - This API returns a cYAML block describing the LNET traffic - statistics. Statistics are continuously incremented by LNET while + This API returns a cYAML block describing the LNet traffic + statistics. Statistics are continuously incremented by LNet while it's alive. This API retuns the statistics at the time of the API call. The statistics include the following @@ -652,7 +652,7 @@ extern int lustre_lnet_show_stats(int seq_no, cYAML **show_rc,
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>lustre_yaml</secondary> </indexterm>Adding/Deleting/Showing Parameters through a YAML Block diff --git a/LustreDebugging.xml b/LustreDebugging.xml index fcd030d..abeef9c 100644 --- a/LustreDebugging.xml +++ b/LustreDebugging.xml @@ -127,7 +127,7 @@ Diagnostic and Debugging Tools packet inspection tool that allows debugging of information that was sent between the various Lustre nodes. This tool is built on top of tcpdump and can read packet dumps generated by - it. There are plug-ins available to dissassemble the LNET and + it. There are plug-ins available to dissassemble the LNet and Lustre protocols. They are located within the Lustre git repository under lustre/contrib/wireshark/. Installation @@ -839,7 +839,7 @@ lctl> debug_kernel [filename] - Behaves similarly to CERROR(), but prints error messages for LNET if D_NETERR is set in the debug mask. This is appropriate for serious networking errors. Messages printed to the console are rate-limited. + Behaves similarly to CERROR(), but prints error messages for LNet if D_NETERR is set in the debug mask. This is appropriate for serious networking errors. Messages printed to the console are rate-limited. @@ -988,7 +988,7 @@ lctl> debug_kernel [filename]
Accessing the <literal>ptlrpc</literal> Request History Each service maintains a request history, which can be useful for first occurrence troubleshooting. - ptlrpc is an RPC protocol layered on LNET that deals with stateful servers and has semantics and built-in support for recovery. + ptlrpc is an RPC protocol layered on LNet that deals with stateful servers and has semantics and built-in support for recovery. The ptlrpc request history works as follows: diff --git a/LustreMaintenance.xml b/LustreMaintenance.xml index 4ce9c39..3ccd54b 100644 --- a/LustreMaintenance.xml +++ b/LustreMaintenance.xml @@ -242,7 +242,7 @@ Changing a Server NID To change a server NID: - Update the LNET configuration in the /etc/modprobe.conf file so the list of server NIDs is correct. Use lctl list_nids to view the list of server NIDS. + Update the LNet configuration in the /etc/modprobe.conf file so the list of server NIDs is correct. Use lctl list_nids to view the list of server NIDS. The lctl list_nids command indicates which network(s) are configured to work with the Lustre file system. @@ -614,7 +614,7 @@ Determining Which Machine is Serving an OST In the course of administering a Lustre file system, you may need to determine which machine is serving a specific OST. It is not as simple as identifying the machine’s IP address, as IP is only one of several networking protocols that the Lustre software uses and, - as such, LNET does not use IP addresses as node identifiers, but NIDs instead. To identify the + as such, LNet does not use IP addresses as node identifiers, but NIDs instead. To identify the NID that is serving a specific OST, run one of the following commands on a client (you do not need to be a root user): client$ lctl get_param osc.fsname-OSTnumber*.ost_conn_uuid diff --git a/LustreNodemap.xml b/LustreNodemap.xml index e9e1b8c..f5ce501 100644 --- a/LustreNodemap.xml +++ b/LustreNodemap.xml @@ -77,7 +77,7 @@ is described using a dash to separate the range, for example, 192.168.20.[0-255]@tcp. - The range must be contiguous. The full LNET definition for a + The range must be contiguous. The full LNet definition for a nidlist is as follows: diff --git a/LustreOperations.xml b/LustreOperations.xml index 6e705b5..64ddbb4 100644 --- a/LustreOperations.xml +++ b/LustreOperations.xml @@ -643,7 +643,7 @@ lctl set_param -P -d
Listing Parameters - To list Lustre or LNET parameters that are available to set, use + To list Lustre or LNet parameters that are available to set, use the lctl list_param command. For example: @@ -711,7 +711,7 @@ osc.myth-OST0004-osc-ffff8800376bdc00.cur_grant_bytes=33808384 --mgsnode= or --servicenode=). To display the NIDs of all servers in networks configured to work - with the Lustre file system, run (while LNET is running): + with the Lustre file system, run (while LNet is running): lctl list_nids diff --git a/LustreProc.xml b/LustreProc.xml index ca1c8d7..02dd0c8 100644 --- a/LustreProc.xml +++ b/LustreProc.xml @@ -1821,9 +1821,9 @@ req_timeout 6 samples [sec] 1 10 15 105 messages or enable printing of D_NETERROR messages to the console using:lctl set_param printk=+neterror Congested routers can be a source of spurious LND timeouts. To avoid this - situation, increase the number of LNET router buffers to reduce back-pressure and/or + situation, increase the number of LNet router buffers to reduce back-pressure and/or increase LND timeouts on all nodes on all connected networks. Also consider increasing - the total number of LNET router nodes in the system so that the aggregate router + the total number of LNet router nodes in the system so that the aggregate router bandwidth matches the aggregate server bandwidth. @@ -1912,12 +1912,12 @@ req_timeout 6 samples [sec] 1 10 15 105
<indexterm> <primary>proc</primary> - <secondary>LNET</secondary> + <secondary>LNet</secondary> </indexterm><indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>proc</secondary> - </indexterm>Monitoring LNET - LNET information is located in /proc/sys/lnet in these files: + Monitoring LNet + LNet information is located in /proc/sys/lnet in these files: peers - Shows all NIDs known to this node and provides information on the queue state. @@ -2032,7 +2032,7 @@ nid refs state max rtr min tx min queue Credits are initialized to allow a certain number of operations (in the example - above the table, eight as shown in the max column. LNET keeps track + above the table, eight as shown in the max column. LNet keeps track of the minimum number of credits ever seen over time showing the peak congestion that has occurred during the time monitored. Fewer available credits indicates a more congested resource. @@ -2047,7 +2047,7 @@ nid refs state max rtr min tx min queue credits (rtr/tx) that is less than max indicates operations are in progress. If the ratio rtr/tx is greater than max, operations are blocking. - LNET also limits concurrent sends and number of router buffers allocated to a single + LNet also limits concurrent sends and number of router buffers allocated to a single peer so that no peer can occupy all these resources. diff --git a/LustreTroubleshooting.xml b/LustreTroubleshooting.xml index 749647a..d349dd7 100644 --- a/LustreTroubleshooting.xml +++ b/LustreTroubleshooting.xml @@ -521,7 +521,7 @@ tail -30 /tmp/objects.{diskname} use error and reject to start the operation. This is caused by a portmap service (often NFS locking) that starts before the Lustre file system and binds to the default port 988. You must have port 988 open from firewall or IP tables for incoming connections on the - client, OSS, and MDS nodes. LNET will create three outgoing connections on available, + client, OSS, and MDS nodes. LNet will create three outgoing connections on available, reserved ports to each client-server pair, starting with 1023, 1022 and 1021. Unfortunately, you cannot set sunprc to avoid port 988. If you receive this error, do the following: @@ -530,7 +530,7 @@ tail -30 /tmp/objects.{diskname} Use a port other than 988 for the Lustre file system. This is configured in - /etc/modprobe.d/lustre.conf as an option to the LNET module. For + /etc/modprobe.d/lustre.conf as an option to the LNet module. For example: options lnet accept_port=988 diff --git a/LustreTuning.xml b/LustreTuning.xml index a73f870..6e6ebb0 100644 --- a/LustreTuning.xml +++ b/LustreTuning.xml @@ -220,14 +220,14 @@ options mdt mds_num_cpts=[0]
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>tuning</secondary> </indexterm> <indexterm> <primary>tuning</primary> - <secondary>LNET</secondary> - </indexterm>Tuning LNET Parameters - This section describes LNET tunables, the use of which may be + LNet + Tuning LNet Parameters + This section describes LNet tunables, the use of which may be necessary on some systems to improve performance. To test the performance of your Lustre network, see . @@ -277,7 +277,7 @@ options ksocklnd enable_irq_affinity=0 Lustre software release 2.3 and beyond provide enhanced network interface control. The enhancement means that an administrator can bind an interface to one or more CPU partitions. Bindings are specified as - options to the LNET modules. For more information on specifying module + options to the LNet modules. For more information on specifying module options, see For example, @@ -298,11 +298,11 @@ options ksocklnd enable_irq_affinity=0 Network interface (NI) credits are shared across all CPU partitions (CPT). For example, if a machine has four CPTs and the number of NI credits is 512, then each partition has 128 credits. If a large number of - CPTs exist on the system, LNET checks and validates the NI credits for + CPTs exist on the system, LNet checks and validates the NI credits for each CPT to ensure each CPT has a workable number of credits. For example, if a machine has 16 CPTs and the number of NI credits is 256, then each partition only has 16 credits. 16 NI credits is low and could - negatively impact performance. As a result, LNET automatically adjusts + negatively impact performance. As a result, LNet automatically adjusts the credits to 8* peer_credits( peer_credits is 8 by default), so each partition has 64 @@ -310,7 +310,7 @@ options ksocklnd enable_irq_affinity=0 Increasing the number of credits/ peer_credits can improve the performance of high - latency networks (at the cost of consuming more memory) by enabling LNET + latency networks (at the cost of consuming more memory) by enabling LNet to send more inflight messages to a specific network/peer and keep the pipeline saturated. An administrator can modify the NI credit count using @@ -325,7 +325,7 @@ ksocklnd credits=256 ko2iblnd credits=256 - In Lustre software release 2.3 and beyond, LNET may revalidate + In Lustre software release 2.3 and beyond, LNet may revalidate the NI credits, so the administrator's request may not persist.
@@ -335,7 +335,7 @@ ko2iblnd credits=256 tuning router buffers Router Buffers - When a node is set up as an LNET router, three pools of buffers are + When a node is set up as an LNet router, three pools of buffers are allocated: tiny, small and large. These pools are allocated per CPU partition and are used to buffer messages that arrive at the router to be forwarded to the next hop. The three different buffer sizes accommodate @@ -367,7 +367,7 @@ ko2iblnd credits=256 The default setting for router buffers typically results in - acceptable performance. LNET automatically sets a default value to reduce + acceptable performance. LNet automatically sets a default value to reduce the likelihood of resource starvation. The size of a router buffer can be modified as shown in the example below. In this example, the size of the large buffer is modified using the @@ -376,7 +376,7 @@ ko2iblnd credits=256 lnet large_router_buffers=8192 - In Lustre software release 2.3 and beyond, LNET may revalidate + In Lustre software release 2.3 and beyond, LNet may revalidate the router buffer setting, so the administrator's request may not persist. @@ -387,15 +387,15 @@ lnet large_router_buffers=8192 tuning portal round-robin Portal Round-Robin - Portal round-robin defines the policy LNET applies to deliver + Portal round-robin defines the policy LNet applies to deliver events and messages to the upper layers. The upper layers are PLRPC - service or LNET selftest. - If portal round-robin is disabled, LNET will deliver messages to + service or LNet selftest. + If portal round-robin is disabled, LNet will deliver messages to CPTs based on a hash of the source NID. Hence, all messages from a specific peer will be handled by the same CPT. This can reduce data traffic between CPUs. However, for some workloads, this behavior may result in poorly balancing loads across the CPU. - If portal round-robin is enabled, LNET will round-robin incoming + If portal round-robin is enabled, LNet will round-robin incoming events across all CPTs. This may balance load better across the CPU but can incur a cross CPU overhead. The current policy can be changed by an administrator with @@ -435,7 +435,7 @@ lnet large_router_buffers=8192
- LNET Peer Health + LNet Peer Health Two options are available to help determine peer health: @@ -445,7 +445,7 @@ lnet large_router_buffers=8192 peer_timeout is set to 180sec, an aliveness query is sent to the peer every 180 seconds. This feature only takes effect if the node is - configured as an LNET router. + configured as an LNet router. In a routed environment, the peer_timeout feature should always be on (set to a value in seconds) on routers. If the router checker has been enabled, @@ -478,7 +478,7 @@ lnet large_router_buffers=8192 all the routers corresponding to the NIDs identified in the routes parameter setting on the node to determine the status of each router interface. The default setting is 1. (For more information about the - LNET routes parameter, see + LNet routes parameter, see A router is considered down if any of its NIDs are down. For @@ -844,7 +844,7 @@ ost.OSS.ost_io.nrs_policies="trr reg" first in, first out (FIFO) policy First In, First Out (FIFO) policy The first in, first out (FIFO) policy handles RPCs in a service in - the same order as they arrive from the LNET layer, so no special + the same order as they arrive from the LNet layer, so no special processing takes place to modify the RPC handling stream. FIFO is the default policy for all types of RPCs on all PTLRPC services, and is always enabled irrespective of the state of other policies, so that it @@ -1209,7 +1209,7 @@ $ lctl set_param x.x.x.nrs_tbf_rule= The format of ' nidlist' argument is the same as the - format when configuring LNET route. The ' + format when configuring LNet route. The ' rate' argument is the RPC rate of the rule, means the upper limit number of requests per second. Following commands are valid. Please note that a newly started diff --git a/ManagingLNET.xml b/ManagingLNET.xml index a9843b6..d0ba235 100644 --- a/ManagingLNET.xml +++ b/ManagingLNET.xml @@ -1,6 +1,6 @@ - Managing Lustre Networking (LNET) - This chapter describes some tools for managing Lustre networking (LNET) and includes the + Managing Lustre Networking (LNet) + This chapter describes some tools for managing Lustre networking (LNet) and includes the following sections: @@ -20,15 +20,15 @@
- <indexterm><primary>LNET</primary><secondary>management</secondary></indexterm> + <title><indexterm><primary>LNet</primary><secondary>management</secondary></indexterm> Updating the Health Status of a Peer or Router There are two mechanisms to update the health status of a peer or a router: - LNET can actively check health status of all routers and mark them as dead or alive automatically. By default, this is off. To enable it set auto_down and if desired check_routers_before_use. This initial check may cause a pause equal to router_ping_timeout at system startup, if there are dead routers in the system. + LNet can actively check health status of all routers and mark them as dead or alive automatically. By default, this is off. To enable it set auto_down and if desired check_routers_before_use. This initial check may cause a pause equal to router_ping_timeout at system startup, if there are dead routers in the system. - When there is a communication error, all LNDs notify LNET that the peer (not necessarily a router) is down. This mechanism is always on, and there is no parameter to turn it off. However, if you set the LNET module parameter auto_down to 0, LNET ignores all such peer-down notifications. + When there is a communication error, all LNDs notify LNet that the peer (not necessarily a router) is down. This mechanism is always on, and there is no parameter to turn it off. However, if you set the LNet module parameter auto_down to 0, LNet ignores all such peer-down notifications. Several key differences in both mechanisms: @@ -45,13 +45,13 @@
- <indexterm><primary>LNET</primary><secondary>starting/stopping</secondary></indexterm>Starting and Stopping LNET - The Lustre software automatically starts and stops LNET, but it can also be manually + <indexterm><primary>LNet</primary><secondary>starting/stopping</secondary></indexterm>Starting and Stopping LNet + The Lustre software automatically starts and stops LNet, but it can also be manually started in a standalone manner. This is particularly useful to verify that your networking setup is working correctly before you attempt to start the Lustre file system.
- Starting LNET - To start LNET, run: + Starting LNet + To start LNet, run: $ modprobe lnet $ lctl network up To see the list of local NIDs, run: @@ -72,18 +72,18 @@ $ lctl network up
- Stopping LNET - Before the LNET modules can be removed, LNET references must be removed. In general, + Stopping LNet + Before the LNet modules can be removed, LNet references must be removed. In general, these references are removed automatically when the Lustre file system is shut down, but for - standalone routers, an explicit step is needed to stop LNET. Run: + standalone routers, an explicit step is needed to stop LNet. Run: lctl network unconfigure Attempting to remove Lustre modules prior to stopping the network may result in a - crash or an LNET hang. If this occurs, the node must be rebooted (in most cases). Make + crash or an LNet hang. If this occurs, the node must be rebooted (in most cases). Make sure that the Lustre network and Lustre file system are stopped prior to unloading the modules. Be extremely careful using rmmod -f. - To unconfigure the LNET network, run: + To unconfigure the LNet network, run: modprobe -r lnd_and_lnet_modules @@ -93,16 +93,16 @@ To remove all Lustre modules, run:
- <indexterm><primary>LNET</primary><secondary>multi-rail configuration</secondary></indexterm>Multi-Rail Configurations with LNET + <indexterm><primary>LNet</primary><secondary>multi-rail configuration</secondary></indexterm>Multi-Rail Configurations with LNet To aggregate bandwidth across both rails of a dual-rail IB cluster (o2iblnd) Multi-rail configurations are only supported by o2iblnd; other IB LNDs do not support multiple interfaces. - using LNET, consider these points: + using LNet, consider these points: - LNET can work with multiple rails, however, it does not load balance across them. The actual rail used for any communication is determined by the peer NID. + LNet can work with multiple rails, however, it does not load balance across them. The actual rail used for any communication is determined by the peer NID. - Multi-rail LNET configurations do not provide an additional level of network fault + Multi-rail LNet configurations do not provide an additional level of network fault tolerance. The configurations described below are for bandwidth aggregation only. @@ -115,7 +115,7 @@ To remove all Lustre modules, run: Fewest hops (to minimize routing), and - Appears first in the "networks" or "ip2nets" LNET configuration strings + Appears first in the "networks" or "ip2nets" LNet configuration strings @@ -123,15 +123,15 @@ To remove all Lustre modules, run:
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>InfiniBand load balancing</secondary> </indexterm>Load Balancing with an InfiniBand<superscript>*</superscript> Network A Lustre file system contains OSSs with two InfiniBand HCAs. Lustre clients have only one InfiniBand HCA using OFED-based Infiniband ''o2ib'' drivers. Load - balancing between the HCAs on the OSS is accomplished through LNET. + balancing between the HCAs on the OSS is accomplished through LNet.
- <indexterm><primary>LNET</primary><secondary>lustre.conf</secondary></indexterm>Setting Up <literal>lustre.conf</literal> for Load Balancing - To configure LNET for load balancing on clients and servers: + <indexterm><primary>LNet</primary><secondary>lustre.conf</secondary></indexterm>Setting Up <literal>lustre.conf</literal> for Load Balancing + To configure LNet for load balancing on clients and servers: Set the lustre.conf options. @@ -234,12 +234,12 @@ ents"
- <indexterm><primary>LNET</primary></indexterm>Dynamically Configuring LNET Routes + <indexterm><primary>LNet</primary></indexterm>Dynamically Configuring LNet Routes Two scripts are provided: lustre/scripts/lustre_routes_config and lustre/scripts/lustre_routes_conversion. - lustre_routes_config sets or cleans up LNET routes from the specified config file. /etc/sysconfig/lustre_routes.conf file can be used to automatically configure routes on LNET startup. + lustre_routes_config sets or cleans up LNet routes from the specified config file. /etc/sysconfig/lustre_routes.conf file can be used to automatically configure routes on LNet startup. lustre_routes_conversion converts a legacy routes configuration file to the new syntax, which is parsed by lustre_routes_config.
- <indexterm><primary>LNET</primary></indexterm><literal>lustre_routes_config</literal> + <indexterm><primary>LNet</primary></indexterm><literal>lustre_routes_config</literal> lustre_routes_config usage is as follows lustre_routes_config [--setup|--cleanup|--dry-run|--verbose] config_file --setup: configure routes listed in config_file @@ -248,10 +248,10 @@ ents" --verbose: echo commands before they are executed The format of the file which is passed into the script is as follows: network: { gateway: gateway@exit_network [hop: hop] [priority: priority] } - An LNET router is identified when its local NID appears within the list of routes. However, this can not be achieved by the use of this script, since the script only adds extra routes after the router is identified. To ensure that a router is identified correctly, make sure to add its local NID in the routes parameter in the modprobe lustre configuration file. See . + An LNet router is identified when its local NID appears within the list of routes. However, this can not be achieved by the use of this script, since the script only adds extra routes after the router is identified. To ensure that a router is identified correctly, make sure to add its local NID in the routes parameter in the modprobe lustre configuration file. See .
- <indexterm><primary>LNET</primary></indexterm><literal>lustre_routes_conversion</literal> + <indexterm><primary>LNet</primary></indexterm><literal>lustre_routes_conversion</literal> lustre_routes_conversion usage is as follows: lustre_routes_conversion legacy_file new_file lustre_routes_conversion takes as a first parameter a file with routes configured as follows: @@ -261,12 +261,12 @@ ents" and appends each converted entry to the output file passed in as the second parameter to the script.
- <indexterm><primary>LNET</primary></indexterm><literal>Route Configuration Examples</literal> - Below is an example of a legacy LNET route configuration. A legacy configuration file can have multiple entries. + <indexterm><primary>LNet</primary></indexterm><literal>Route Configuration Examples</literal> + Below is an example of a legacy LNet route configuration. A legacy configuration file can have multiple entries. tcp1 10.1.1.2@tcp0:1; tcp2 10.1.1.3@tcp0:2; tcp3 10.1.1.4@tcp0; - Below is an example of the converted LNET route configuration. The following would be the result of the lustre_routes_conversion script, when run on the above legacy entries. + Below is an example of the converted LNet route configuration. The following would be the result of the lustre_routes_conversion script, when run on the above legacy entries. tcp1: { gateway: 10.1.1.2@tcp0 priority: 1 } tcp2: { gateway: 10.1.1.2@tcp0 priority: 2 } tcp1: { gateway: 10.1.1.4@tcp0 } diff --git a/ManagingSecurity.xml b/ManagingSecurity.xml index b1dcccc..6035c7f 100644 --- a/ManagingSecurity.xml +++ b/ManagingSecurity.xml @@ -117,7 +117,7 @@ other::--- The root_squash parameter specifies the UID and GID with which the root user accesses the Lustre file system. - The nosquash_nids parameter specifies the set of clients to which root squash does not apply. LNET NID range syntax is used for this parameter (see the NID range syntax rules described in ). For example: + The nosquash_nids parameter specifies the set of clients to which root squash does not apply. LNet NID range syntax is used for this parameter (see the NID range syntax rules described in ). For example: nosquash_nids=172.16.245.[0-255/2]@tcp @@ -179,10 +179,10 @@ lctl get_param mdt.*.nosquash_nids mkfs.lustre and tunefs.lustre do not perform parameter syntax checking. If the root squash parameters are incorrect, they are ignored on mount and the default values are used instead. - Root squash parameters are parsed with rigorous syntax checking. The root_squash parameter should be specified as <decnum>:<decnum>. The nosquash_nids parameter should follow LNET NID range list syntax. + Root squash parameters are parsed with rigorous syntax checking. The root_squash parameter should be specified as <decnum>:<decnum>. The nosquash_nids parameter should follow LNet NID range list syntax. - LNET NID range syntax: + LNet NID range syntax: <nidlist> :== <nidrange> [ ' ' <nidrange> ] <nidrange> :== <addrrange> '@' <net> <addrrange> :== '*' | diff --git a/SettingUpLustreSystem.xml b/SettingUpLustreSystem.xml index 875d861..db611db 100644 --- a/SettingUpLustreSystem.xml +++ b/SettingUpLustreSystem.xml @@ -903,7 +903,7 @@ Lustre networks and routing are configured and managed by specifying parameters to the - Lustre networking (lnet) module in + Lustre Networking (lnet) module in /etc/modprobe.d/lustre.conf. To prepare to configure Lustre networking, complete the following steps: @@ -921,18 +921,18 @@ If routing is needed, identify the nodes to be used to route traffic between networks. If you are using multiple network types, then you will need a router. Any node with - appropriate interfaces can route Lustre networking (LNET) traffic between different + appropriate interfaces can route Lustre networking (LNet) traffic between different network hardware types or topologies --the node may be a server, a client, or a standalone - router. LNET can route messages between different network types (such as + router. LNet can route messages between different network types (such as TCP-to-InfiniBand) or across different topologies (such as bridging two InfiniBand or TCP/IP networks). Routing will be configured in . Identify the network interfaces to include - in or exclude from LNET. - If not explicitly specified, LNET uses either the first available + in or exclude from LNet. + If not explicitly specified, LNet uses either the first available interface or a pre-defined default for a given network type. Interfaces - that LNET should not use (such as an administrative network or + that LNet should not use (such as an administrative network or IP-over-IB), can be excluded. Network interfaces to be used or excluded will be specified using the lnet kernel module parameters networks and diff --git a/SystemConfigurationUtilities.xml b/SystemConfigurationUtilities.xml index d748e46..40915cc 100644 --- a/SystemConfigurationUtilities.xml +++ b/SystemConfigurationUtilities.xml @@ -261,7 +261,7 @@ $ lctl conf_param testfs.llite.max_read_ahead_mb=16 network up|down|tcp|elan|myrinet - Starts or stops LNET, or selects a network type for other lctl LNET commands. + Starts or stops LNet, or selects a network type for other lctl LNet commands. @@ -269,7 +269,7 @@ $ lctl conf_param testfs.llite.max_read_ahead_mb=16 list_nids - Prints all NIDs on the local node. LNET must be running. + Prints all NIDs on the local node. LNet must be running. @@ -285,7 +285,7 @@ $ lctl conf_param testfs.llite.max_read_ahead_mb=16 ping nid - Checks LNET connectivity via an LNET ping. This uses the fabric appropriate to the specified NID. + Checks LNet connectivity via an LNet ping. This uses the fabric appropriate to the specified NID. @@ -398,7 +398,7 @@ $ lctl conf_param testfs.llite.max_read_ahead_mb=16 list_param [-F|-R] parameter [parameter ...] - Lists the Lustre or LNET parameter name. + Lists the Lustre or LNet parameter name.   @@ -429,7 +429,7 @@ $ lctl conf_param testfs.llite.max_read_ahead_mb=16 get_param [-n|-N|-F] parameter [parameter ...] - Gets the value of a Lustre or LNET parameter from the specified path. + Gets the value of a Lustre or LNet parameter from the specified path. @@ -470,7 +470,7 @@ $ lctl conf_param testfs.llite.max_read_ahead_mb=16 set_param [-n] parameter=value - Sets the value of a Lustre or LNET parameter from the specified path. + Sets the value of a Lustre or LNet parameter from the specified path. @@ -1256,34 +1256,34 @@ lshowmount
<indexterm><primary>lst</primary></indexterm> lst - The lst utility starts LNET self-test. + The lst utility starts LNet self-test.
Synopsis lst
Description - LNET self-test helps site administrators confirm that Lustre Networking (LNET) has been properly installed and configured. The self-test also confirms that LNET and the network software and hardware underlying it are performing as expected. - Each LNET self-test runs in the context of a session. A node can be associated with only one session at a time, to ensure that the session has exclusive use of the nodes on which it is running. A session is create, controlled and monitored from a single node; this is referred to as the self-test console. + LNet self-test helps site administrators confirm that Lustre Networking (LNet) has been properly installed and configured. The self-test also confirms that LNet and the network software and hardware underlying it are performing as expected. + Each LNet self-test runs in the context of a session. A node can be associated with only one session at a time, to ensure that the session has exclusive use of the nodes on which it is running. A session is create, controlled and monitored from a single node; this is referred to as the self-test console. Any node may act as the self-test console. Nodes are named and allocated to a self-test session in groups. This allows all nodes in a group to be referenced by a single name. Test configurations are built by describing and running test batches. A test batch is a named collection of tests, with each test composed of a number of individual point-to-point tests running in parallel. These individual point-to-point tests are instantiated according to the test type, source group, target group and distribution specified when the test is added to the test batch.
Modules - To run LNET self-test, load these modules: libcfs, lnet, lnet_selftest and any one of the klnds (ksocklnd, ko2iblnd...). To load all necessary modules, run modprobe lnet_selftest, which recursively loads the modules on which lnet_selftest depends. - There are two types of nodes for LNET self-test: the console node and test nodes. Both node types require all previously-specified modules to be loaded. (The userspace test node does not require these modules). + To run LNet self-test, load these modules: libcfs, lnet, lnet_selftest and any one of the klnds (ksocklnd, ko2iblnd...). To load all necessary modules, run modprobe lnet_selftest, which recursively loads the modules on which lnet_selftest depends. + There are two types of nodes for LNet self-test: the console node and test nodes. Both node types require all previously-specified modules to be loaded. (The userspace test node does not require these modules). Test nodes can be in either kernel or in userspace. A console user can invite a kernel test node to join the test session by running lst add_group NID, but the user cannot actively add a userspace test node to the test session. However, the console user can passively accept a test node to the test session while the test node runs lst client to connect to the console.
Utilities - LNET self-test includes two user utilities, lst and lstclient. + LNet self-test includes two user utilities, lst and lstclient. lst is the user interface for the self-test console (run on the console node). It provides a list of commands to control the entire test system, such as create session, create test groups, etc. - lstclient is the userspace self-test program which is linked with userspace LNDs and LNET. A user can invoke lstclient to join a self-test session: + lstclient is the userspace self-test program which is linked with userspace LNDs and LNet. A user can invoke lstclient to join a self-test session: lstclient -sesid CONSOLE_NID group NAME
Example Script - This is a sample LNET self-test script which simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an IB network (connected via LNET routers), with half the clients reading and half the clients writing. + This is a sample LNet self-test script which simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an IB network (connected via LNet routers), with half the clients reading and half the clients writing. #!/bin/bash export LST_SESSION=$$ lst new_session read/write @@ -1306,7 +1306,7 @@ lst end_session
<indexterm><primary>lustre_rmmod.sh</primary></indexterm> lustre_rmmod.sh - The lustre_rmmod.sh utility removes all Lustre and LNET modules (assuming no Lustre services are running). It is located in /usr/bin. + The lustre_rmmod.sh utility removes all Lustre and LNet modules (assuming no Lustre services are running). It is located in /usr/bin. The lustre_rmmod.sh utility does not work if Lustre modules are being used or if you have manually run the lctl network up command. @@ -2196,7 +2196,7 @@ routerstat
Description - The routerstat utility displays LNET router statistics. If no interval is specified, then statistics are sampled and printed only one time. Otherwise, statistics are sampled and printed at the specified interval (in seconds). + The routerstat utility displays LNet router statistics. If no interval is specified, then statistics are sampled and printed only one time. Otherwise, statistics are sampled and printed at the specified interval (in seconds).
Output @@ -2221,7 +2221,7 @@ routerstat M - Number of messages currently being processed by LNET (The maximum number of messages ever processed by LNET concurrently) + Number of messages currently being processed by LNet (The maximum number of messages ever processed by LNet concurrently) @@ -2229,7 +2229,7 @@ routerstat E - Number of LNET errors + Number of LNet errors @@ -2288,7 +2288,7 @@ routerstat M - Number of messages currently being processed by LNET (The maximum number of messages ever processed by LNET concurrently) + Number of messages currently being processed by LNet (The maximum number of messages ever processed by LNet concurrently) @@ -2296,7 +2296,7 @@ routerstat E - Number of LNET errors per second + Number of LNet errors per second diff --git a/UnderstandingLustre.xml b/UnderstandingLustre.xml index 390f711..238677f 100644 --- a/UnderstandingLustre.xml +++ b/UnderstandingLustre.xml @@ -463,7 +463,7 @@ xml:id="understandinglustre"> Lustre Components An installation of the Lustre software includes a management server (MGS) and one or more Lustre file systems interconnected with Lustre - networking (LNET). + networking (LNet). A basic configuration of Lustre file system components is shown in .
@@ -656,12 +656,12 @@ xml:id="understandinglustre"> <indexterm> <primary>Lustre</primary> - <secondary>LNET</secondary> - </indexterm>Lustre Networking (LNET) - Lustre Networking (LNET) is a custom networking API that provides + LNet + Lustre Networking (LNet) + Lustre Networking (LNet) is a custom networking API that provides the communication infrastructure that handles metadata and file I/O data for the Lustre file system servers and clients. For more information - about LNET, see + about LNet, see .
diff --git a/UnderstandingLustreNetworking.xml b/UnderstandingLustreNetworking.xml index f85bd6c..4dd940c 100644 --- a/UnderstandingLustreNetworking.xml +++ b/UnderstandingLustreNetworking.xml @@ -1,7 +1,7 @@ - Understanding Lustre Networking (LNET) - This chapter introduces Lustre networking (LNET). It includes the following sections: + Understanding Lustre Networking (LNet) + This chapter introduces Lustre networking (LNet). It includes the following sections: @@ -24,15 +24,15 @@
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> </indexterm><indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>understanding</secondary> - </indexterm> Introducing LNET + Introducing LNet In a cluster using one or more Lustre file systems, the network communication infrastructure required by the Lustre file system is implemented using the Lustre networking - (LNET) feature. - LNET supports many commonly-used network types, such as InfiniBand and IP networks, and + (LNet) feature. + LNet supports many commonly-used network types, such as InfiniBand and IP networks, and allows simultaneous availability across multiple network types with routing between them. Remote direct memory access (RDMA) is permitted when supported by underlying networks using the appropriate Lustre network driver (LND). High availability and recovery features enable @@ -41,15 +41,15 @@ example ksocklnd is the driver which implements the TCP Socket LND that supports TCP networks. LNDs are loaded into the driver stack, with one LND for each network type in use. - For information about configuring LNET, see . - For information about administering LNET, see . + For information about configuring LNet, see . + For information about administering LNet, see .
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>features</secondary> - </indexterm>Key Features of LNET - Key features of LNET include: + Key Features of LNet + Key features of LNet include: RDMA, when supported by underlying networks @@ -67,7 +67,7 @@ Routing among disparate networks - LNET permits end-to-end read/write throughput at or near peak bandwidth rates on a variety + LNet permits end-to-end read/write throughput at or near peak bandwidth rates on a variety of network interconnects.
@@ -76,31 +76,31 @@ Networks Lustre Networks A Lustre network is comprised of clients and servers running the Lustre software. It need - not be confined to one LNET subnet but can span several networks provided routing is possible - between the networks. In a similar manner, a single network can have multiple LNET subnets. - The Lustre networking stack is comprised of two layers, the LNET code module and the LND. - The LNET layer operates above the LND layer in a manner similar to the way the network layer - operates above the data link layer. LNET layer is connectionless, asynchronous and does not + not be confined to one LNet subnet but can span several networks provided routing is possible + between the networks. In a similar manner, a single network can have multiple LNet subnets. + The Lustre networking stack is comprised of two layers, the LNet code module and the LND. + The LNet layer operates above the LND layer in a manner similar to the way the network layer + operates above the data link layer. LNet layer is connectionless, asynchronous and does not verify that data has been transmitted while the LND layer is connection oriented and typically does verify data transmission. - LNETs are uniquely identified by a label comprised of a string corresponding to an LND and - a number, such as tcp0, o2ib0, or o2ib1, that uniquely identifies each LNET. Each node on an - LNET has at least one network identifier (NID). A NID is a combination of the address of the - network interface and the LNET label in the - form:address@LNET_label. + LNets are uniquely identified by a label comprised of a string corresponding to an LND and + a number, such as tcp0, o2ib0, or o2ib1, that uniquely identifies each LNet. Each node on an + LNet has at least one network identifier (NID). A NID is a combination of the address of the + network interface and the LNet label in the + form:address@LNet_label. Examples: 192.168.1.2@tcp0 10.13.24.90@o2ib1 In certain circumstances it might be desirable for Lustre file system traffic to pass - between multiple LNETs. This is possible using LNET routing. It is important to realize that - LNET routing is not the same as network routing. For more details about LNET routing, see + between multiple LNets. This is possible using LNet routing. It is important to realize that + LNet routing is not the same as network routing. For more details about LNet routing, see
<indexterm> - <primary>LNET</primary> + <primary>LNet</primary> <secondary>supported networks</secondary> </indexterm>Supported Network Types - The LNET code module includes LNDs to support many network types including: + The LNet code module includes LNDs to support many network types including: InfiniBand: OpenFabrics OFED (o2ib) -- 1.8.3.1