+ <section condition='l2B' xml:id="lnet_config.dynamic_discovery">
+ <title><indexterm>
+ <primary>LNet</primary>
+ <secondary>cli</secondary>
+ <tertiary>dynamic discovery</tertiary>
+ </indexterm>Dynamic Peer Discovery</title>
+ <section xml:id="lnet_config.dynamic_discovery.overview">
+ <title>Overview</title>
+ <para>Dynamic Discovery (DD) is a feature that allows nodes to
+ dynamically discover a peer's interfaces without having to explicitly
+ configure them. This is very useful for Multi-Rail (MR)
+ configurations. In large clusters, there could be hundreds of nodes
+ and having to configure MR peers on each node becomes error prone.
+ Dynamic Discovery is enabled by default and uses a new protocol based
+ on LNet pings to discover the interfaces of the remote peers on first
+ message.</para>
+ </section>
+ <section xml:id="lnet_config.dynamic_discovery.protocol">
+ <title>Protocol</title>
+ <para>When LNet on a node is requested to send a message to a peer it
+ first attempts to ping the peer. The reply to the ping contains the
+ peer's NIDs as well as a feature bit outlining what the peer supports.
+ Dynamic Discovery adds a Multi-Rail feature bit. If the peer is
+ Multi-Rail capable, it sets the MR bit in the ping reply. When the
+ node receives the reply it checks the MR bit, and if it is set it then
+ pushes its own list of NIDs to the peer using a new PUT message,
+ referred to as a "push ping". After this brief protocol, both the peer
+ and the node will have each other's list of interfaces. The MR
+ algorithm can then proceed to use the list of interfaces of the
+ corresponding peer.</para>
+ <para>If the peer is not MR capable, it will not set the MR feature
+ bit in the ping reply. The node will understand that the peer is
+ not MR capable and will only use the interface provided by upper
+ layers for sending messages.</para>
+ </section>
+ <section xml:id="lnet_config.dynamic_discovery.userspace_config">
+ <title>Dynamic Discovery and User-space Configuration</title>
+ <para>It is possible to configure the peer manually while Dynamic
+ Discovery is running. Manual peer configuration always takes precedence
+ over Dynamic Discovery. If there is a discrepancy between the manual
+ configuration and the dynamically discovered information, a warning is
+ printed.</para>
+ </section>
+ <section xml:id="lnet_config.dynamic_discovery.config">
+ <title>Configuration</title>
+ <para>Dynamic Discovery is very light on the configuration side. It can
+ only be turned on or turned off. To turn the feature on or off, the
+ following command is used:</para>
+ <screen>lnetctl set discovery [0 | 1]</screen>
+ <para>To check the current <literal>discovery</literal> setting, the
+ <literal>lnetctl global show</literal> command can be used as shown in
+ <xref linkend="lnet_config.show_global_settings"/>.</para>
+ </section>
+ <section xml:id="lnet_config.dynamic_discovery.ondemand">
+ <title>Initiating Dynamic Discovery on Demand</title>
+ <para>It is possible to initiate the Dynamic Discovery protocol on demand
+ without having to wait for a message to be sent to the peer. This can
+ be done with the following command:</para>
+ <screen>lnetctl discover <peer_nid> [<peer_nid> ...]</screen>
+ </section>
+ </section>