<para>LNET kernel module (lnet) parameters specify how LNET is to be configured to work with Lustre, including which NICs will be configured to work with Lustre and the routing to be used with Lustre.</para>
<para>Parameters for LNET can be specified in the <literal>/etc/modprobe.d/lustre.conf</literal> file. In some cases the parameters may have been stored in <literal>/etc/modprobe.conf</literal>, but this has been deprecated since before RHEL5 and SLES10, and having a separate <literal>/etc/modprobe.d/lustre.conf</literal> file simplifies administration and distribution of the Lustre networking configuration. This file contains one or more entries with the syntax:</para>
<screen>options lnet <replaceable>parameter</replaceable>=<replaceable>value</replaceable></screen>
- <para>To specify the network interfaces that are to be used for Lustre, set either the networks parameter or the <literal>ip2nets</literal> parameter (only one of these parameters can be used at a time):</para>
+ <para>To specify the network interfaces that are to be used for Lustre, set either the <literal>networks</literal> parameter or the <literal>ip2nets</literal> parameter (only one of these parameters can be used at a time):</para>
<itemizedlist>
<listitem>
<para><literal>networks</literal> - Specifies the networks to be used.</para>
<para>To set up routing between networks, use:</para>
<itemizedlist>
<listitem>
- <para>routes - Lists networks and the NIDs of routers that forward to them.</para>
+ <para><literal>routes</literal> - Lists networks and the NIDs of routers that forward to them.</para>
</listitem>
</itemizedlist>
<para>See <xref linkend="dbdoclet.50438216_71227"/> for more details.</para>
<title>
<indexterm><primary>Lustre</primary><secondary>configuring</secondary></indexterm>
Configuring a Simple Lustre File System</title>
- <para>A Lustre system can be set up in a variety of configurations by using the administrative utilities provided with Lustre. The procedure below shows how to to configure a simple Lustre file system consisting of a combined MGS/MDS, one OSS with two OSTs, and a client. For an overview of the entire Lustre installation procedure, see <xref linkend="installoverview"/>.</para>
+ <para>A Lustre system can be set up in a variety of configurations by using the administrative utilities provided with Lustre. The procedure below shows how to configure a simple Lustre file system consisting of a combined MGS/MDS, one OSS with two OSTs, and a client. For an overview of the entire Lustre installation procedure, see <xref linkend="installoverview"/>.</para>
<para>This configuration procedure assumes you have completed the following:</para>
<itemizedlist>
<listitem>
<para>Mount the combined MGS/MDT file system on the block device. On the MDS node, run:</para>
<screen>mount -t lustre <replaceable>/dev/block_device</replaceable> <replaceable>/mount_point</replaceable></screen>
<note>
- <para>If you have created and MGS and an MDT on separate block devices, mount them both.</para>
+ <para>If you have created an MGS and an MDT on separate block devices, mount them both.</para>
</note>
</listitem>
<listitem xml:id="dbdoclet.50438267_pgfId-1290915">
<para>When you create an OST, you are formatting a <literal>ldiskfs</literal> file system on a block storage device like you would with any local file system.</para>
<para>You can have as many OSTs per OSS as the hardware or drivers allow. For more information about storage and memory requirements for a Lustre file system, see <xref linkend="settinguplustresystem"/>.</para>
<para>You can only configure one OST per block device. You should create an OST that uses the raw block device and does not use partitioning.</para>
- <para>You should specify the OST index number at format time in order to simplify the translating the OST number in error messages or file striping to the OSS node and block device later on.</para>
+ <para>You should specify the OST index number at format time in order to simplify translating the OST number in error messages or file striping to the OSS node and block device later on.</para>
<para>If you are using block devices that are accessible from multiple OSS nodes, ensure that you mount the OSTs from only one OSS node at at time. It is strongly recommended that multiple-mount protection be enabled for such devices to prevent serious data corruption. For more information about multiple-mount protection, see <xref linkend="managingfailover"/>.</para>
<note>
<para>Lustre currently supports block devices up to 128 TB on RHEL 5/6 (up to 8 TB on other distributions). If the device size is only slightly larger that 16 TB, it is recommended that you limit the file system size to 16 TB at format time. We recommend that you not place DOS partitions on top of RAID 5/6 block devices due to negative impacts on performance, but instead format the whole disk for the filesystem.</para>
<section remap="h4">
<title><indexterm><primary>installing</primary><secondary>tools and utilities</secondary></indexterm>
Lustre-Specific Tools and Utilities</title>
- <para>Several third-party utilities are must be installed on servers:</para>
+ <para>Several third-party utilities must be installed on servers:</para>
<itemizedlist>
<listitem>
<para><emphasis role="bold">e2fsprogs</emphasis> : Lustre requires a recent version of e2fsprogs that understands extents and large xattr. Use <literal>e2fsprogs-1.41.90.wc4</literal> or later, available at:</para>
<section xml:id="imperativerecovery">
<title><indexterm><primary>imperative recovery</primary></indexterm>Imperative Recovery</title>
<para>Imperative Recovery (IR) was first introduced in Lustre 2.2.0</para>
- <para>Large-scale lustre implementations have historically experienced problems recovering in a timely manner after a server failure. This is due to the way that clients detect the server failure and how the servers perform their recovery. Many of the processes are driven by the RPC timeout, which must be scaled with system size to prevent false diagnosis of server death. The purpose of imperative recovery is to reduce the recovery window by actively informing clients of server failure. The resulting reduction in the recovery window will minimize target downtime and therefore increase overall system availability. Imperative Recovery does not remove previous recovery mechanisms, and client timeout-based recovery actions can occur in a cluster when IR is enabled as each client can still independently disconnect and reconnect from a target. In case of a mix of IR and non-IR clients connecting to an OST or MDT, the server cannot reduce its recovery timeout window, because it cannot be sure that all clients have been notified of the server restart in a timely manner. Even in such mixed environments the time to complete recovery may be reduced, since IR-enabled clients will still be notified reconnect to the server promptly and allow recovery to complete as soon as the last the non-IR client detects the server failure.</para>
+ <para>Large-scale lustre implementations have historically experienced problems recovering in a timely manner after a server failure. This is due to the way that clients detect the server failure and how the servers perform their recovery. Many of the processes are driven by the RPC timeout, which must be scaled with system size to prevent false diagnosis of server death. The purpose of imperative recovery is to reduce the recovery window by actively informing clients of server failure. The resulting reduction in the recovery window will minimize target downtime and therefore increase overall system availability. Imperative Recovery does not remove previous recovery mechanisms, and client timeout-based recovery actions can occur in a cluster when IR is enabled as each client can still independently disconnect and reconnect from a target. In case of a mix of IR and non-IR clients connecting to an OST or MDT, the server cannot reduce its recovery timeout window, because it cannot be sure that all clients have been notified of the server restart in a timely manner. Even in such mixed environments the time to complete recovery may be reduced, since IR-enabled clients will still be notified to reconnect to the server promptly and allow recovery to complete as soon as the last non-IR client detects the server failure.</para>
<section remap="h3">
<title><indexterm><primary>imperative recovery</primary><secondary>MGS role</secondary></indexterm>MGS role</title>
<para>The MGS now holds additional information about Lustre targets, in the form of a Target Status Table. Whenever a target registers with the MGS, there is a corresponding entry in this table identifying the target. This entry includes NID information, and state/version information for the target. When a client mounts the filesystem, it caches a locked copy of this table, in the form of a Lustre configuration log. When a target restart occurs, the MGS revokes the client lock, forcing all clients to reload the table. Any new targets will have an updated version number, the client detects this and reconnects to the restarted target. Since successful IR notification of server restart depends on all clients being registered with the MGS, and there is no other node to notify clients in case of MGS restart, the MGS will disable IR for a period when it first starts. This interval is configurable, as shown in <xref linkend="imperativerecoveryparameters"/></para>
<para>Which server node it was communicating with, and so on.</para>
</listitem>
</itemizedlist>
- <para>Lustre logs are dumped to <literal>/proc/sys/lnet/debug_pat</literal>h.</para>
+ <para>Lustre logs are dumped to <literal>/proc/sys/lnet/debug_path</literal>.</para>
<para>Collect the first group of messages related to a problem, and any messages that precede "LBUG" or "assertion failure" errors. Messages that mention server nodes (OST or MDS) are specific to that server; you must collect similar messages from the relevant server console logs.</para>
<para>Another Lustre debug log holds information for Lustre action for a short period of time which, in turn, depends on the processes on the node to use Lustre. Use the following command to extract debug logs on each of the nodes, run</para>
<screen>$ lctl dk <replaceable>filename</replaceable></screen>
git://git.whamcloud.com / doc / manual.git / commitdiff