LUDOC-157 lnet: Add priority parameter to routes for LNet

Change LU-2934 added a new LNet route parameter called priority.
This change is to document the route priority parameter in the manual.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I4cff3a9caf5ce3d1ef6285f4663f8cb31dba9cfd
Reviewed-on: http://review.whamcloud.com/7428
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Tested-by: Hudson

diff --git a/ConfigurationFilesModuleParameters.xml b/ConfigurationFilesModuleParameters.xml
index c50db0e..ce0d9f3 100644 (file)
--- a/ConfigurationFilesModuleParameters.xml
+++ b/ConfigurationFilesModuleParameters.xml
@@ -160,10 +160,11 @@ routes (&quot;&quot;)</title>
         <para>This is a string that lists networks and the NIDs of routers that forward to them.</para>
         <para>It has the following syntax (<literal>&lt;w&gt;</literal> is one or more whitespace characters):</para>
         <screen>&lt;routes&gt; :== &lt;route&gt;{ ; &lt;route&gt; }
-&lt;route&gt; :== [&lt;net&gt;[&lt;w&gt;&lt;hopcount&gt;]&lt;w&gt;&lt;nid&gt;{&lt;w&gt;&lt;nid&gt;}</screen>
+&lt;route&gt; :== [&lt;net&gt;[&lt;w&gt;&lt;hopcount&gt;]&lt;w&gt;&lt;nid&gt;[:&lt;priority&gt;]{&lt;w&gt;&lt;nid&gt;[:&lt;priority&gt;]}</screen>
+        <para>Note: the priority parameter was added in release 2.5.</para>
         <para>So a node on the network <literal>tcp1</literal> that needs to go through a router to get to the Elan network:</para>
         <screen>options lnet networks=tcp1 routes=&quot;elan 1 192.168.2.2@tcpA&quot;</screen>
-        <para>The hopcount is used to help choose the best path between multiply-routed configurations.</para>
+        <para>The hopcount and priority numbers are used to help choose the best path between multiply-routed configurations.</para>
         <para>A simple but powerful expansion syntax is provided, both for target networks and router NIDs as follows.</para>
         <screen>&lt;expansion&gt; :== &quot;[&quot; &lt;entry&gt; { &quot;,&quot; &lt;entry&gt; } &quot;]&quot;
 &lt;entry&gt; :== &lt;numeric range&gt; | &lt;non-numeric item&gt;
@@ -171,7 +172,8 @@ routes (&quot;&quot;)</title>
         <para>The expansion is a list enclosed in square brackets. Numeric items in the list may be a single number, a contiguous range of numbers, or a strided range of numbers. For example, <literal>routes=&quot;elan 192.168.1.[22-24]@tcp&quot;</literal> says that network <literal>elan0</literal> is adjacent (hopcount defaults to 1); and is accessible via 3 routers on the <literal>tcp0</literal> network (<literal>192.168.1.22@tcp</literal>, <literal>192.168.1.23@tcp</literal> and <literal>192.168.1.24@tcp</literal>).</para>
         <para><literal>routes=&quot;[tcp,vib] 2 [8-14/2]@elan&quot;</literal> says that 2 networks (<literal>tcp0</literal> and <literal>vib0</literal>) are accessible through 4 routers (<literal>8@elan</literal>, <literal>10@elan</literal>, <literal>12@elan</literal> and <literal>14@elan</literal>). The hopcount of 2 means that traffic to both these networks will be traversed 2 routers - first one of the routers specified in this entry, then one more.</para>
         <para>Duplicate entries, entries that route to a local network, and entries that specify routers on a non-local network are ignored.</para>
-        <para>Equivalent entries are resolved in favor of the route with the shorter hopcount. The hopcount, if omitted, defaults to 1 (the remote network is adjacent).</para>
+        <para>Prior to release 2.5, a conflict between equivalent entries was resolved in favor of the route with the shorter hopcount. The hopcount, if omitted, defaults to 1 (the remote network is adjacent)..</para>
+        <para condition='125'>Since 2.5, equivalent entries are resolved in favor of the route with the lowest priority number or shorter hopcount if the priorities are equal. The priority, if omitted, defaults to 0.  The hopcount, if omitted, defaults to 1 (the remote network is adjacent).</para>
         <para>It is an error to specify routes to the same destination with routers on different local networks.</para>
         <para>If the target network string contains no expansions, then the hopcount defaults to 1 and may be omitted (that is, the remote network is adjacent). In practice, this is true for most multi-network configurations. It is an error to specify an inconsistent hop count for a given target network. This is why an explicit hopcount is required if the target network string specifies more than one network.</para>
       </section>

diff --git a/ConfiguringLNET.xml b/ConfiguringLNET.xml
index cabd7ed..c7f57a1 100644 (file)
--- a/ConfiguringLNET.xml
+++ b/ConfiguringLNET.xml
@@ -90,7 +90,8 @@
     <screen>options lnet networks=tcp0(eth1)</screen>
     <para>Depending on the network design, it may be necessary to specify explicit interfaces. To explicitly specify that interface <literal>eth2</literal> be used for network <literal>tcp0</literal> and <literal>eth3</literal> be used for <literal>tcp1</literal> , use this entry:</para>
     <screen>options lnet networks=tcp0(eth2),tcp1(eth3)</screen>
-    <para>When more than one interface is available during the network setup, Lustre chooses the best route based on the hop count. Once the network connection is established, Lustre expects the network to stay connected. In a Lustre network, connections do not fail over to another interface, even if multiple interfaces are available on the same node.</para>
+    <para>When more than one interface is available during the network setup, Lustre chooses the best route based on the hopcount. Once the network connection is established, Lustre expects the network to stay connected. In a Lustre network, connections do not fail over to another interface, even if multiple interfaces are available on the same node.</para>
+    <para condition='l25'>Since version 2.5 of the Lustre software, the route priority is also used to decide which interface to use. The priority has precendence over hopcount so the route with the lower priority number will be selected regardless of what the hopcounts are.</para>
     <note>
       <para>LNET lines in <literal>lustre.conf</literal> are only used by the local node to determine what to call its interfaces. They are not used for routing decisions.</para>
     </note>

diff --git a/ManagingLNET.xml b/ManagingLNET.xml
index a6f3516..4d4d380 100644 (file)
--- a/ManagingLNET.xml
+++ b/ManagingLNET.xml
@@ -99,6 +99,9 @@ To remove all Lustre modules, run:</para>
       <listitem>
         <para>A Lustre node always uses the same local NID to communicate with a given peer NID. The criteria used to determine the local NID are:</para>
         <itemizedlist>
+          <listitem>
+            <para condition='125'>Lowest route priority number (lower number, higher priority).</para>
+          </listitem>
           <listitem>
             <para>Fewest hops (to minimize routing), and</para>
           </listitem>