<?xml version="1.0" encoding="UTF-8"?>
-<chapter version="5.0" xml:lang="en-US" xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink">
+<chapter version="5.0" xml:lang="en-US" xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" xml:id='configurationfilesmodulesparamters'>
<info>
- <title>Configuration Files and Module Parameters</title>
+ <title xml:id='configurationfilesmodulesparamters.title'>Configuration Files and Module Parameters</title>
</info>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293284" xreflabel=""/>This section describes configuration files and module parameters and includes the following sections:</para>
<itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438293_pgfId-1293288" xreflabel=""/><link xl:href="ConfigurationFilesModuleParameters.html#50438293_15350">Introduction</link></para>
+ <para><xref linkend="dbdoclet.50438293_15350"/></para>
</listitem>
+
<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438293_pgfId-1293292" xreflabel=""/><link xl:href="ConfigurationFilesModuleParameters.html#50438293_78010">Module Options</link></para>
- </listitem>
-<listitem>
- <para> </para>
+ <para><xref linkend="dbdoclet.50438293_78010"/></para>
</listitem>
+
</itemizedlist>
- <section remap="h2">
- <title><anchor xml:id="dbdoclet.50438293_pgfId-1293294" xreflabel=""/></title>
- <section remap="h2">
- <title>35.1 <anchor xml:id="dbdoclet.50438293_15350" xreflabel=""/>Introduction</title>
+ <section xml:id="dbdoclet.50438293_15350">
+ <title>35.1 Introduction</title>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293295" xreflabel=""/>LNET network hardware and routing are now configured via module parameters. Parameters should be specified in the /etc/modprobe.conf file, for example:</para>
<screen><anchor xml:id="dbdoclet.50438293_pgfId-1293296" xreflabel=""/>alias lustre llite
<anchor xml:id="dbdoclet.50438293_pgfId-1293297" xreflabel=""/>options lnet networks=tcp0,elan0
</screen>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293309" xreflabel=""/>For the following parameters, default option settings are shown in parenthesis. Changes to parameters marked with a W affect running systems. (Unmarked parameters can only be set when LNET loads for the first time.) Changes to parameters marked with Wc only have effect when connections are established (existing connections are not affected by these changes.)</para>
</section>
- <section remap="h2">
- <title>35.2
<anchor xml:id="dbdoclet.50438293_78010" xreflabel=""/>Module <anchor xml:id="dbdoclet.50438293_marker-1293311" xreflabel=""/>Options</title>
+ <section xml:id="dbdoclet.50438293_78010">
+ <title>35.2 Module <anchor xml:id="dbdoclet.50438293_marker-1293311" xreflabel=""/>Options</title>
<itemizedlist><listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293313" xreflabel=""/> With routed or other multi-network configurations, use ip2nets rather than networks, so all nodes can use the same configuration.</para>
</listitem>
+
<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438293_pgfId-1293314" xreflabel=""/> For a routed network, use the same “routes†configuration everywhere. Nodes specified as routers automatically enable forwarding and any routes that are not relevant to a particular node are ignored. Keep a common configuration to guarantee that all nodes have consistent routing tables.</para>
- </listitem>
-<listitem>
- <para> </para>
+ <para><anchor xml:id="dbdoclet.50438293_pgfId-1293314" xreflabel=""/> For a routed network, use the same 'routes†configuration everywhere. Nodes specified as routers automatically enable forwarding and any routes that are not relevant to a particular node are ignored. Keep a common configuration to guarantee that all nodes have consistent routing tables.</para>
</listitem>
+
<listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293315" xreflabel=""/> A separate modprobe.conf.lnet included from modprobe.conf makes distributing the configuration much easier.</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
<listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293316" xreflabel=""/> If you set config_on_load=1, LNET starts at modprobe time rather than waiting for Lustre to start. This ensures routers start working at module load time.</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
</itemizedlist>
<screen><anchor xml:id="dbdoclet.50438293_pgfId-1293317" xreflabel=""/># lctl
<anchor xml:id="dbdoclet.50438293_pgfId-1293318" xreflabel=""/># lctl> net down
<itemizedlist><listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293319" xreflabel=""/> Remember the lctl ping {nid} command - it is a handy way to check your LNET configuration.</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
</itemizedlist>
<section remap="h3">
<title><anchor xml:id="dbdoclet.50438293_pgfId-1293321" xreflabel=""/>35.2.1 <anchor xml:id="dbdoclet.50438293_94707" xreflabel=""/>LNET <anchor xml:id="dbdoclet.50438293_marker-1293320" xreflabel=""/>Options</title>
</tbody>
</tgroup>
</informaltable>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438293_pgfId-1293363" xreflabel=""/>Lustre ignores the loopback interface (lo0), but Lustre use any IP addresses aliased to the loopback (by default). When in doubt, explicitly specify networks.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
+ <note><para>Lustre ignores the loopback interface (lo0), but Lustre use any IP addresses aliased to the loopback (by default). When in doubt, explicitly specify networks.</para></note>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293364" xreflabel=""/><emphasis role="bold">ip2nets</emphasis> ("") is a string that lists globally-available networks, each with a set of IP address ranges. LNET determines the locally-available networks from this list by matching the IP address ranges with the local IPs of a node. The purpose of this option is to be able to use the same modules.conf file across a variety of nodes on different networks. The string has the following syntax.</para>
<screen><anchor xml:id="dbdoclet.50438293_pgfId-1293365" xreflabel=""/><ip2nets> :== <net-match> [ <comment> ] { <net-sep> <net-match> }
<anchor xml:id="dbdoclet.50438293_pgfId-1293366" xreflabel=""/><net-match> :== [ <w> ] <net-spec> <w> <ip-range> { <w> <ip-range> }
<itemizedlist><listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293388" xreflabel=""/> Two TCP subnets</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
<listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293389" xreflabel=""/> One Elan subnet</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
<listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293390" xreflabel=""/> One machine set up as a router, with both TCP and Elan interfaces</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
<listitem>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293391" xreflabel=""/> IP over Elan configured, but only IP will be used to label the nodes.</para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
+
</itemizedlist>
<screen><anchor xml:id="dbdoclet.50438293_pgfId-1293392" xreflabel=""/>options lnet ip2nets=â€tcp 198.129.135.* 192.128.88.98; \
elan 198.128.88.98 198.129.135.3; \
- routes=â€tcp 1022@elan # Elan NID of router; \
- elan 198.128.88.98@tcp # TCP NID of router “
+ routes='cp 1022@elan # Elan NID of router; \
+ elan 198.128.88.98@tcp # TCP NID of router '
</screen>
</section>
<section remap="h4">
<title><anchor xml:id="dbdoclet.50438293_pgfId-1293393" xreflabel=""/>35.2.1.2 networks ("tcp")</title>
- <para><anchor xml:id="dbdoclet.50438293_pgfId-1293394" xreflabel=""/>This is an alternative to "ip2nets" which can be used to specify the networks to be instantiated explicitly. The syntax is a simple comma separated list of <net-spec>s (see above). The default is only used if neither “ip2nets†nor “networks†is specified.</para>
+ <para><anchor xml:id="dbdoclet.50438293_pgfId-1293394" xreflabel=""/>This is an alternative to "ip2nets" which can be used to specify the networks to be instantiated explicitly. The syntax is a simple comma separated list of <net-spec>s (see above). The default is only used if neither 'ip2nets†nor 'networks†is specified.</para>
</section>
<section remap="h4">
- <title><anchor xml:id="dbdoclet.50438293_pgfId-1293395" xreflabel=""/>35.2.1.3 routes (“â€)</title>
+ <title><anchor xml:id="dbdoclet.50438293_pgfId-1293395" xreflabel=""/>35.2.1.3 routes ("")</title>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293396" xreflabel=""/>This is a string that lists networks and the NIDs of routers that forward to them.</para>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293397" xreflabel=""/>It has the following syntax (<w> is one or more whitespace characters):</para>
<screen><anchor xml:id="dbdoclet.50438293_pgfId-1293398" xreflabel=""/><routes> :== <route>{ ; <route> }
<anchor xml:id="dbdoclet.50438293_pgfId-1293399" xreflabel=""/><route> :== [<net>[<w><hopcount>]<w><nid>{<w><nid>}
</screen>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293400" xreflabel=""/>So a node on the network tcp1 that needs to go through a router to get to the Elan network:</para>
- <screen><anchor xml:id="dbdoclet.50438293_pgfId-1293401" xreflabel=""/>options lnet networks=tcp1 routes="elan 1 192.168.2.2@tcp1â€
+ <screen><anchor xml:id="dbdoclet.50438293_pgfId-1293401" xreflabel=""/>options lnet networks=tcp1 routes="elan 1 192.168.2.2@tcpA"
</screen>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293402" xreflabel=""/>The hopcount is used to help choose the best path between multiply-routed configurations.</para>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293403" xreflabel=""/>A simple but powerful expansion syntax is provided, both for target networks and router NIDs as follows.</para>
<section remap="h4">
<title><anchor xml:id="dbdoclet.50438293_pgfId-1293413" xreflabel=""/>35.2.1.4 forwarding ("")</title>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293414" xreflabel=""/>This is a string that can be set either to "enabled" or "disabled" for explicit control of whether this node should act as a router, forwarding communications between all local networks.</para>
- <para><anchor xml:id="dbdoclet.50438293_pgfId-1293415" xreflabel=""/>A standalone router can be started by simply starting LNET (“modprobe ptlrpcâ€) with appropriate network topology options.</para>
+ <para><anchor xml:id="dbdoclet.50438293_pgfId-1293415" xreflabel=""/>A standalone router can be started by simply starting LNET ('modprobe ptlrpcâ€) with appropriate network topology options.</para>
<informaltable frame="all">
<tgroup cols="2">
<colspec colname="c1" colwidth="50*"/>
<title><anchor xml:id="dbdoclet.50438293_pgfId-1293449" xreflabel=""/>35.2.2 SOCKLND <anchor xml:id="dbdoclet.50438293_marker-1293448" xreflabel=""/>Kernel TCP/IP LND</title>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293450" xreflabel=""/>The SOCKLND kernel TCP/IP LND (socklnd) is connection-based and uses the acceptor to establish communications via sockets with its peers.</para>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293451" xreflabel=""/>It supports multiple instances and load balances dynamically over multiple interfaces. If no interfaces are specified by the ip2nets or networks module parameter, all non-loopback IP interfaces are used. The address-within-network is determined by the address of the first IP interface an instance of the socklnd encounters.</para>
- <para><anchor xml:id="dbdoclet.50438293_pgfId-1293452" xreflabel=""/>Consider a node on the “edge†of an InfiniBand network, with a low-bandwidth management Ethernet (eth0), IP over IB configured (ipoib0), and a pair of GigE NICs (eth1,eth2) providing off-cluster connectivity. This node should be configured with "networks=<emphasis role="bold">vib</emphasis>,tcp(eth1,eth2)†to ensure that the socklnd ignores the management Ethernet and IPoIB.</para>
+ <para><anchor xml:id="dbdoclet.50438293_pgfId-1293452" xreflabel=""/>Consider a node on the 'edge†of an InfiniBand network, with a low-bandwidth management Ethernet (eth0), IP over IB configured (ipoib0), and a pair of GigE NICs (eth1,eth2) providing off-cluster connectivity. This node should be configured with "networks=<emphasis role="bold">vib</emphasis>,tcp(eth1,eth2)†to ensure that the socklnd ignores the management Ethernet and IPoIB.</para>
<informaltable frame="all">
<tgroup cols="2">
<colspec colname="c1" colwidth="50*"/>
<para><anchor xml:id="dbdoclet.50438293_pgfId-1293923" xreflabel=""/>polling (0) determines whether this host will poll or block for MX request completions. A value of 0 blocks and any positive value will poll that many times before blocking. Since polling increases CPU usage, we suggest that you set this to zero (0) on the client and experiment with different values for servers.</para>
</section>
</section>
- </section>
</chapter>
git://git.whamcloud.com / doc / manual.git / commitdiff