-<listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1290869" xreflabel=""/><link xl:href="ConfiguringLustre.html#50438267_76752">Additional Configuration Options</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-</itemizedlist>
- <section remap="h2">
- <title><anchor xml:id="dbdoclet.50438267_pgfId-1290874" xreflabel=""/></title>
- <section remap="h2">
- <title>10.1 <anchor xml:id="dbdoclet.50438267_50692" xreflabel=""/>Configuring a Simple Lustre File System</title>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1290876" xreflabel=""/>A Lustre system can be set up in a variety of configurations by using the administrative utilities provided with Lustre. The procedure below shows how to to configure a simple Lustre file system consisting of a combined MGS/MDS, one OSS with two OSTs, and a client. For an overview of the entire Lustre installation procedure, see <link xl:href="InstallOverview.html#50438192_70895">Chapter 4</link>: <link xl:href="InstallOverview.html#50438192_66186">Installation Overview</link>.</para>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1291868" xreflabel=""/>This configuration procedure assumes you have completed the following:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1293115" xreflabel=""/><emphasis>Set up and configured your hardware</emphasis> . For more information about hardware requirements, see <link xl:href="SettingUpLustreSystem.html#50438256_38751">Chapter 5</link>: <link xl:href="SettingUpLustreSystem.html#50438256_66186">Setting Up a Lustre File System</link>.</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1293121" xreflabel=""/><emphasis>Downloaded and installed the Lustre software.</emphasis> For more information about preparing for and installing the Lustre software, see <link xl:href="InstallingLustre.html#50438261_81829">Chapter 8</link>: <link xl:href="InstallingLustre.html#50438261_62973">Installing the Lustre Software</link>.</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-</itemizedlist>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1293288" xreflabel=""/>The following optional steps should also be completed, if needed, before the Lustre software is configured:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1293289" xreflabel=""/><emphasis>Set up a hardware or software RAID on block devices to be used as OSTs or MDTs.</emphasis> For information about setting up RAID, see the documentation for your RAID controller or <link xl:href="ConfiguringStorage.html#50438208_36899">Chapter 6</link><emphasis>:</emphasis><link xl:href="ConfiguringStorage.html#50438208_70525">Configuring Storage on a Lustre File System</link>.</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1293297" xreflabel=""/><emphasis>Set up network interface bonding on Ethernet interfaces.</emphasis> For information about setting up network interface bonding, see <link xl:href="SettingUpBonding.html#50438258_72593">Chapter 7</link>: <link xl:href="SettingUpBonding.html#50438258_99571">Setting Up Network Interface Bonding</link>.</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1294595" xreflabel=""/><emphasis>Set</emphasis>lnet<emphasis>module parameters to specify how Lustre Networking (LNET) is to be configured to work with Lustre and test the LNET configuration.</emphasis> LNET will, by default, use the first TCP/IP interface it discovers on a system. If this network configuration is sufficient, you do not need to configure LNET. LNET configuration is required if you are using Infiniband or multiple Ethernet interfaces.</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-</itemizedlist>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1294840" xreflabel=""/>For information about configuring LNET, see <link xl:href="ConfiguringLNET.html#50438216_64580">Chapter 9</link>: <link xl:href="ConfiguringLNET.html#50438216_29256">Configuring Lustre Networking (LNET)</link>. For information about testing LNET, see <link xl:href="LNETSelfTest.html#50438223_71556">Chapter 23</link><emphasis>:</emphasis><link xl:href="LNETSelfTest.html#50438223_21832">Testing Lustre Network Performance (LNET Self-Test)</link>.</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438267_pgfId-1294602" xreflabel=""/><emphasis>Run the benchmark script sgpdd_survey to determine baseline performance of your hardware.</emphasis> Benchmarking your hardware will simplify debugging performance issues that are unrelated to Lustre and ensure you are getting the best possible performance with your installation. For information about running sgpdd_survey, see <link xl:href="BenchmarkingTests.html#50438212_51053">Testing I/O Performance of Raw Hardware (sgpdd_survey)</link>.</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-</itemizedlist>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
+ </itemizedlist>
+ <section xml:id="dbdoclet.50438267_50692">
+ <title>
+ <indexterm>
+ <primary>Lustre</primary>
+ <secondary>configuring</secondary>
+ </indexterm>Configuring a Simple Lustre File System</title>
+ <para>A Lustre file system can be set up in a variety of configurations by
+ using the administrative utilities provided with the Lustre software. The
+ procedure below shows how to configure a simple Lustre file system
+ consisting of a combined MGS/MDS, one OSS with two OSTs, and a client. For
+ an overview of the entire Lustre installation procedure, see
+ <xref linkend="installoverview" />.</para>
+ <para>This configuration procedure assumes you have completed the
+ following:</para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <emphasis>
+ <emphasis role="bold">Set up and configured your hardware</emphasis>
+ </emphasis>. For more information about hardware requirements, see
+ <xref linkend="settinguplustresystem" />.</para>
+ </listitem>
+ <listitem>
+ <para>
+ <emphasis role="bold">Downloaded and installed the Lustre
+ software.</emphasis>For more information about preparing for and
+ installing the Lustre software, see
+ <xref linkend="installinglustre" />.</para>
+ </listitem>
+ </itemizedlist>
+ <para>The following optional steps should also be completed, if needed,
+ before the Lustre software is configured:</para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <emphasis>Set up a hardware or software RAID on block devices to be
+ used as OSTs or MDTs.</emphasis>For information about setting up RAID,
+ see the documentation for your RAID controller or
+ <xref linkend="configuringstorage" />.</para>
+ </listitem>
+ <listitem>
+ <para>
+ <emphasis>Set up network interface bonding on Ethernet
+ interfaces.</emphasis>For information about setting up network
+ interface bonding, see
+ <xref linkend="settingupbonding" />.</para>
+ </listitem>
+ <listitem>
+ <para>
+ <emphasis>Set</emphasis> lnet
+ <emphasis>module parameters to specify how Lustre Networking (LNet) is
+ to be configured to work with a Lustre file system and test the LNet
+ configuration.</emphasis>LNet will, by default, use the first TCP/IP
+ interface it discovers on a system. If this network configuration is
+ sufficient, you do not need to configure LNet. LNet configuration is
+ required if you are using InfiniBand or multiple Ethernet
+ interfaces.</para>
+ </listitem>
+ </itemizedlist>
+ <para>For information about configuring LNet, see
+ <xref linkend="configuringlnet" />. For information about testing LNet, see
+
+ <xref linkend="lnetselftest" />.</para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <emphasis>Run the benchmark script
+ <literal>sgpdd-survey</literal> to determine baseline performance of
+ your hardware.</emphasis>Benchmarking your hardware will simplify
+ debugging performance issues that are unrelated to the Lustre software
+ and ensure you are getting the best possible performance with your
+ installation. For information about running
+ <literal>sgpdd-survey</literal>, see
+ <xref linkend="benchmarkingtests" />.</para>
+ </listitem>
+ </itemizedlist>
+ <note>
+ <para>The
+ <literal>sgpdd-survey</literal> script overwrites the device being tested
+ so it must be run before the OSTs are configured.</para>
+ </note>
+ <para>To configure a simple Lustre file system, complete these
+ steps:</para>
+ <orderedlist>
+ <listitem>
+ <para>Create a combined MGS/MDT file system on a block device. On the
+ MDS node, run:</para>
+ <screen>
+mkfs.lustre --fsname=
+<replaceable>fsname</replaceable> --mgs --mdt --index=0
+<replaceable>/dev/block_device</replaceable>
+</screen>
+ <para>The default file system name (
+ <literal>fsname</literal>) is
+ <literal>lustre</literal>.</para>
+ <note>
+ <para>If you plan to create multiple file systems, the MGS should be
+ created separately on its own dedicated block device, by
+ running:</para>
+ <screen>
+mkfs.lustre --fsname=
+<replaceable>fsname</replaceable> --mgs
+<replaceable>/dev/block_device</replaceable>
+</screen>
+ <para>See
+ <xref linkend="dbdoclet.50438194_88063" />for more details.</para>
+ </note>
+ </listitem>
+ <listitem xml:id="dbdoclet.addmdtindex">
+ <para>Optional for Lustre software release 2.4 and later.
+ Add in additional MDTs.</para>
+ <screen>
+mkfs.lustre --fsname=
+<replaceable>fsname</replaceable> --mgsnode=
+<replaceable>nid</replaceable> --mdt --index=1
+<replaceable>/dev/block_device</replaceable>
+</screen>
+ <note>
+ <para>Up to 4095 additional MDTs can be added.</para>
+ </note>
+ </listitem>
+ <listitem>
+ <para>Mount the combined MGS/MDT file system on the block device. On
+ the MDS node, run:</para>
+ <screen>
+mount -t lustre
+<replaceable>/dev/block_device</replaceable>
+<replaceable>/mount_point</replaceable>
+</screen>
+ <note>
+ <para>If you have created an MGS and an MDT on separate block
+ devices, mount them both.</para>
+ </note>
+ </listitem>
+ <listitem xml:id="dbdoclet.format_ost">
+ <para>Create the OST. On the OSS node, run:</para>
+ <screen>
+mkfs.lustre --fsname=
+<replaceable>fsname</replaceable> --mgsnode=
+<replaceable>MGS_NID</replaceable> --ost --index=
+<replaceable>OST_index</replaceable>
+<replaceable>/dev/block_device</replaceable>
+</screen>
+ <para>When you create an OST, you are formatting a
+ <literal>ldiskfs</literal> or
+ <literal>ZFS</literal> file system on a block storage device like you
+ would with any local file system.</para>
+ <para>You can have as many OSTs per OSS as the hardware or drivers
+ allow. For more information about storage and memory requirements for a
+ Lustre file system, see
+ <xref linkend="settinguplustresystem" />.</para>
+ <para>You can only configure one OST per block device. You should
+ create an OST that uses the raw block device and does not use
+ partitioning.</para>
+ <para>You should specify the OST index number at format time in order
+ to simplify translating the OST number in error messages or file
+ striping to the OSS node and block device later on.</para>
+ <para>If you are using block devices that are accessible from multiple
+ OSS nodes, ensure that you mount the OSTs from only one OSS node at at
+ time. It is strongly recommended that multiple-mount protection be
+ enabled for such devices to prevent serious data corruption. For more
+ information about multiple-mount protection, see
+ <xref linkend="managingfailover" />.</para>
+ <note>
+ <para>The Lustre software currently supports block devices up to 128
+ TB on Red Hat Enterprise Linux 5 and 6 (up to 8 TB on other
+ distributions). If the device size is only slightly larger that 16
+ TB, it is recommended that you limit the file system size to 16 TB at
+ format time. We recommend that you not place DOS partitions on top of
+ RAID 5/6 block devices due to negative impacts on performance, but
+ instead format the whole disk for the file system.</para>
+ </note>
+ </listitem>
+ <listitem xml:id="dbdoclet.mount_ost">
+ <para>Mount the OST. On the OSS node where the OST was created,
+ run:</para>
+ <screen>
+mount -t lustre
+<replaceable>/dev/block_device</replaceable>
+<replaceable>/mount_point</replaceable>
+</screen>
+ <note>
+ <para>To create additional OSTs, repeat Step
+ <xref linkend="dbdoclet.format_ost" />and Step
+ <xref linkend="dbdoclet.mount_ost" />, specifying the
+ next higher OST index number.</para>
+ </note>
+ </listitem>
+ <listitem xml:id="dbdoclet.mount_on_client">
+ <para>Mount the Lustre file system on the client. On the client node,
+ run:</para>
+ <screen>
+mount -t lustre
+<replaceable>MGS_node</replaceable>:/
+<replaceable>fsname</replaceable>
+<replaceable>/mount_point</replaceable>
+</screen>
+ <note>
+ <para>To mount the filesystem on additional clients, repeat Step
+ <xref linkend="dbdoclet.mount_on_client" />.</para>
+ </note>
+ <note>
+ <para>If you have a problem mounting the file system, check the
+ syslogs on the client and all the servers for errors and also check
+ the network settings. A common issue with newly-installed systems is
+ that
+ <literal>hosts.deny</literal> or firewall rules may prevent
+ connections on port 988.</para>
+ </note>
+ </listitem>
+ <listitem>
+ <para>Verify that the file system started and is working correctly. Do
+ this by running
+ <literal>lfs df</literal>,
+ <literal>dd</literal> and
+ <literal>ls</literal> commands on the client node.</para>
+ </listitem>
+ <listitem>
+ <para>
+ <emphasis>(Optional)</emphasis>Run benchmarking tools to validate the
+ performance of hardware and software layers in the cluster. Available
+ tools include:</para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>obdfilter-survey</literal>- Characterizes the storage
+ performance of a Lustre file system. For details, see
+ <xref linkend="dbdoclet.50438212_26516" />.</para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>ost-survey</literal>- Performs I/O against OSTs to detect
+ anomalies between otherwise identical disk subsystems. For details,
+ see
+ <xref linkend="dbdoclet.50438212_85136" />.</para>
+ </listitem>
+ </itemizedlist>
+ </listitem>
+ </orderedlist>
+ <section remap="h3">
+ <title>
+ <indexterm>
+ <primary>Lustre</primary>
+ <secondary>configuring</secondary>
+ <tertiary>simple example</tertiary>
+ </indexterm>Simple Lustre Configuration Example</title>
+ <para>To see the steps to complete for a simple Lustre file system
+ configuration, follow this example in which a combined MGS/MDT and two
+ OSTs are created to form a file system called
+ <literal>temp</literal>. Three block devices are used, one for the
+ combined MGS/MDS node and one for each OSS node. Common parameters used
+ in the example are listed below, along with individual node
+ parameters.</para>
+ <informaltable frame="all">
+ <tgroup cols="4">
+ <colspec colname="c1" colwidth="2*" />
+ <colspec colname="c2" colwidth="25*" />
+ <colspec colname="c3" colwidth="25*" />
+ <colspec colname="c4" colwidth="25*" />
+ <thead>