ConfiguringLustre.xml

   1 <?xml version="1.0" encoding="UTF-8"?>
   2 <article version="5.0" xml:lang="en-US" xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink">
   3   <info>
   4     <title>Configuring Lustre</title>
   5   </info>
   6   <informaltable frame="none">
   7     <tgroup cols="2">
   8       <colspec colname="c1" colwidth="50*"/>
   9       <colspec colname="c2" colwidth="50*"/>
  10
  11
  12       <tbody>
  13         <row>
  14           <entry align="left"><para>Lustre 2.0 Operations Manual</para></entry>
  15           <entry align="right" valign="top"><para><link xl:href="index.html"><inlinemediaobject><imageobject role="html">
  16                     <imagedata contentdepth="26" contentwidth="30" fileref="./shared/toc01.gif" scalefit="1"/>
  17                   </imageobject>
  18 <imageobject role="fo">
  19                     <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/toc01.gif" scalefit="1" width="100%"/>
  20                   </imageobject>
  21 </inlinemediaobject></link><link xl:href="ConfiguringLNET.html"><inlinemediaobject><imageobject role="html">
  22                     <imagedata contentdepth="26" contentwidth="30" fileref="./shared/prev01.gif" scalefit="1"/>
  23                   </imageobject>
  24 <imageobject role="fo">
  25                     <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/prev01.gif" scalefit="1" width="100%"/>
  26                   </imageobject>
  27 </inlinemediaobject></link><link xl:href="ConfiguringFailover.html"><inlinemediaobject><imageobject role="html">
  28                     <imagedata contentdepth="26" contentwidth="30" fileref="./shared/next01.gif" scalefit="1"/>
  29                   </imageobject>
  30 <imageobject role="fo">
  31                     <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/next01.gif" scalefit="1" width="100%"/>
  32                   </imageobject>
  33 </inlinemediaobject></link><link xl:href="ix.html"><inlinemediaobject><imageobject role="html">
  34                     <imagedata contentdepth="26" contentwidth="30" fileref="./shared/index01.gif" scalefit="1"/>
  35                   </imageobject>
  36 <imageobject role="fo">
  37                     <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/index01.gif" scalefit="1" width="100%"/>
  38                   </imageobject>
  39 </inlinemediaobject></link></para></entry>
  40         </row>
  41       </tbody>
  42     </tgroup>
  43   </informaltable>
  44   <para><link xl:href=""/></para>
  45   <informaltable frame="none">
  46     <tgroup cols="1">
  47       <colspec colname="c1" colwidth="100*"/>
  48
  49       <tbody>
  50         <row>
  51           <entry align="right"><para><anchor xml:id="dbdoclet.50438267_pgfId-874" xreflabel=""/>C H A P T E R  10<anchor xml:id="dbdoclet.50438267_88428" xreflabel=""/></para></entry>
  52         </row>
  53       </tbody>
  54     </tgroup>
  55   </informaltable>
  56   <informaltable frame="none">
  57     <tgroup cols="1">
  58       <colspec colname="c1" colwidth="100*"/>
  59
  60       <tbody>
  61         <row>
  62           <entry align="right"><para><anchor xml:id="dbdoclet.50438267_pgfId-1290860" xreflabel=""/><anchor xml:id="dbdoclet.50438267_66186" xreflabel=""/><anchor xml:id="dbdoclet.50438267_42640" xreflabel=""/>Configuring Lustre</para></entry>
  63         </row>
  64       </tbody>
  65     </tgroup>
  66   </informaltable>
  67   <para><anchor xml:id="dbdoclet.50438267_pgfId-1290861" xreflabel=""/>This chapter shows how to configure a simple Lustre system comprised of a combined MGS/MDT, an OST and a client. It includes:</para>
  68   <itemizedlist><listitem>
  69       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290865" xreflabel=""/><link xl:href="ConfiguringLustre.html#50438267_50692">Configuring a Simple Lustre File System</link></para>
  70     </listitem>
  71 <listitem>
  72       <para> </para>
  73     </listitem>
  74 <listitem>
  75       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290869" xreflabel=""/><link xl:href="ConfiguringLustre.html#50438267_76752">Additional Configuration Options</link></para>
  76     </listitem>
  77 <listitem>
  78       <para> </para>
  79     </listitem>
  80 </itemizedlist>
  81   <section remap="h2">
  82     <title><anchor xml:id="dbdoclet.50438267_pgfId-1290874" xreflabel=""/></title>
  83     <section remap="h2">
  84       <title>10.1 <anchor xml:id="dbdoclet.50438267_50692" xreflabel=""/>Configuring a Simple Lustre File System</title>
  85       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290876" xreflabel=""/>A Lustre system can be set up in a variety of configurations by using the administrative utilities provided with Lustre. The procedure below shows how to to configure a simple Lustre file system consisting of a combined MGS/MDS, one OSS with two OSTs, and a client. For an overview of the entire Lustre installation procedure, see <link xl:href="InstallOverview.html#50438192_70895">Chapter 4</link>: <link xl:href="InstallOverview.html#50438192_66186">Installation Overview</link>.</para>
  86       <para><anchor xml:id="dbdoclet.50438267_pgfId-1291868" xreflabel=""/>This configuration procedure assumes you have completed the following:</para>
  87       <itemizedlist><listitem>
  88           <para><anchor xml:id="dbdoclet.50438267_pgfId-1293115" xreflabel=""/><emphasis>Set up and configured your hardware</emphasis> . For more information about hardware requirements, see <link xl:href="SettingUpLustreSystem.html#50438256_38751">Chapter 5</link>: <link xl:href="SettingUpLustreSystem.html#50438256_66186">Setting Up a Lustre File System</link>.</para>
  89         </listitem>
  90 <listitem>
  91           <para> </para>
  92         </listitem>
  93 <listitem>
  94           <para><anchor xml:id="dbdoclet.50438267_pgfId-1293121" xreflabel=""/><emphasis>Downloaded and installed the Lustre software.</emphasis>  For more information about preparing for and installing the Lustre software, see <link xl:href="InstallingLustre.html#50438261_81829">Chapter 8</link>: <link xl:href="InstallingLustre.html#50438261_62973">Installing the Lustre Software</link>.</para>
  95         </listitem>
  96 <listitem>
  97           <para> </para>
  98         </listitem>
  99 </itemizedlist>
 100       <para><anchor xml:id="dbdoclet.50438267_pgfId-1293288" xreflabel=""/>The following optional steps should also be completed, if needed, before the Lustre software is configured:</para>
 101       <itemizedlist><listitem>
 102           <para><anchor xml:id="dbdoclet.50438267_pgfId-1293289" xreflabel=""/><emphasis>Set up a hardware or software RAID on block devices to be used as OSTs or MDTs.</emphasis>  For information about setting up RAID, see the documentation for your RAID controller or <link xl:href="ConfiguringStorage.html#50438208_36899">Chapter 6</link><emphasis>:</emphasis><link xl:href="ConfiguringStorage.html#50438208_70525">Configuring Storage on a Lustre File System</link>.</para>
 103         </listitem>
 104 <listitem>
 105           <para> </para>
 106         </listitem>
 107 <listitem>
 108           <para><anchor xml:id="dbdoclet.50438267_pgfId-1293297" xreflabel=""/><emphasis>Set up network interface bonding on Ethernet interfaces.</emphasis>  For information about setting up network interface bonding, see <link xl:href="SettingUpBonding.html#50438258_72593">Chapter 7</link>: <link xl:href="SettingUpBonding.html#50438258_99571">Setting Up Network Interface Bonding</link>.</para>
 109         </listitem>
 110 <listitem>
 111           <para> </para>
 112         </listitem>
 113 <listitem>
 114           <para><anchor xml:id="dbdoclet.50438267_pgfId-1294595" xreflabel=""/><emphasis>Set</emphasis>lnet<emphasis>module parameters to specify how Lustre Networking (LNET) is to be configured to work with Lustre and test the LNET configuration.</emphasis>  LNET will, by default, use the first TCP/IP interface it discovers on a system. If this network configuration is sufficient, you do not need to configure LNET. LNET configuration is required if you are using Infiniband or multiple Ethernet interfaces.</para>
 115         </listitem>
 116 <listitem>
 117           <para> </para>
 118         </listitem>
 119 </itemizedlist>
 120       <para><anchor xml:id="dbdoclet.50438267_pgfId-1294840" xreflabel=""/>For information about configuring LNET, see <link xl:href="ConfiguringLNET.html#50438216_64580">Chapter 9</link>: <link xl:href="ConfiguringLNET.html#50438216_29256">Configuring Lustre Networking (LNET)</link>. For information about testing LNET, see <link xl:href="LNETSelfTest.html#50438223_71556">Chapter 23</link><emphasis>:</emphasis><link xl:href="LNETSelfTest.html#50438223_21832">Testing Lustre Network Performance (LNET Self-Test)</link>.</para>
 121       <itemizedlist><listitem>
 122           <para><anchor xml:id="dbdoclet.50438267_pgfId-1294602" xreflabel=""/><emphasis>Run the benchmark script sgpdd_survey to determine baseline performance of your hardware.</emphasis>  Benchmarking your hardware will simplify debugging performance issues that are unrelated to Lustre and ensure you are getting the best possible performance with your installation. For information about running sgpdd_survey, see <link xl:href="BenchmarkingTests.html#50438212_51053">Testing I/O Performance of Raw Hardware (sgpdd_survey)</link>.</para>
 123         </listitem>
 124 <listitem>
 125           <para> </para>
 126         </listitem>
 127 </itemizedlist>
 128       <informaltable frame="none">
 129         <tgroup cols="1">
 130           <colspec colname="c1" colwidth="100*"/>
 131           <tbody>
 132             <row>
 133               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1294603" xreflabel=""/>The sgpdd_survey script overwrites the device being tested so it must be run before the OSTs are configured.</para></entry>
 134             </row>
 135           </tbody>
 136         </tgroup>
 137       </informaltable>
 138       <para><anchor xml:id="dbdoclet.50438267_pgfId-1291941" xreflabel=""/>To configure a simple Lustre file system, complete these steps:</para>
 139       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290894" xreflabel=""/> 1. <emphasis role="bold">Create</emphasis> a combined MGS/MDT file system on a block device. On the MDS node, run:</para>
 140       <screen><anchor xml:id="dbdoclet.50438267_pgfId-1290905" xreflabel=""/>mkfs.lustre --fsname=&lt;<emphasis>fsname</emphasis>&gt; --mgs --mdt &lt;<emphasis>block device name</emphasis>&gt;
 141 </screen>
 142       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290906" xreflabel=""/>The default file system name (<emphasis>fsname)</emphasis> is lustre.</para>
 143       <informaltable frame="none">
 144         <tgroup cols="1">
 145           <colspec colname="c1" colwidth="100*"/>
 146           <tbody>
 147             <row>
 148               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1293505" xreflabel=""/>If you plan to generate multiple file systems, the MGS should be created separately on its own dedicated block device, by running:</para><para> mkfs.lustre --fsname=&lt;<emphasis>fsname</emphasis>&gt; --mgs &lt;<emphasis>block device name</emphasis>&gt;</para></entry>
 149             </row>
 150           </tbody>
 151         </tgroup>
 152       </informaltable>
 153        <para><anchor xml:id="dbdoclet.50438267_pgfId-1290909" xreflabel=""/> 2. <anchor xml:id="dbdoclet.50438267_21878" xreflabel=""/>Mount the combined MGS/MDT file system on the block device. On the MDS node, run:</para>
 154       <screen><anchor xml:id="dbdoclet.50438267_pgfId-1290910" xreflabel=""/>mount -t lustre &lt;<emphasis>block device name</emphasis>&gt; &lt;<emphasis>mount point</emphasis>&gt;
 155 </screen>
 156       <informaltable frame="none">
 157         <tgroup cols="1">
 158           <colspec colname="c1" colwidth="100*"/>
 159           <tbody>
 160             <row>
 161               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1293170" xreflabel=""/>If you have created and MGS and an MDT on separate block devices, mount them both.</para></entry>
 162             </row>
 163           </tbody>
 164         </tgroup>
 165       </informaltable>
 166       <para><anchor xml:id="dbdoclet.50438267_pgfId-1294976" xreflabel=""/> </para>
 167       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290915" xreflabel=""/> 3. <anchor xml:id="dbdoclet.50438267_81555" xreflabel=""/>Create the OST. On the OSS node, run:</para>
 168       <screen><anchor xml:id="dbdoclet.50438267_pgfId-1290916" xreflabel=""/>mkfs.lustre --ost --fsname=&lt;<emphasis>fsname</emphasis>&gt; --mgsnode=&lt;<emphasis>NID</emphasis>&gt; &lt;<emphasis>block device name</emphasis>&gt;
 169 </screen>
 170       <para><anchor xml:id="dbdoclet.50438267_pgfId-1293526" xreflabel=""/>When you create an OST, you are formatting a ldiskfs file system on a block storage device like you would with any local file system.</para>
 171       <para><anchor xml:id="dbdoclet.50438267_pgfId-1291922" xreflabel=""/>You can have as many OSTs per OSS as the hardware or drivers allow. For more information about storage and memory requirements for a Lustre file system, see <link xl:href="SettingUpLustreSystem.html#50438256_38751">Chapter 5</link>: <link xl:href="SettingUpLustreSystem.html#50438256_66186">Setting Up a Lustre File System</link>.</para>
 172       <para><anchor xml:id="dbdoclet.50438267_pgfId-1290918" xreflabel=""/>You can only configure one OST per block device. You should create an OST that uses the raw block device and does not use partitioning.</para>
 173       <para><anchor xml:id="dbdoclet.50438267_pgfId-1293567" xreflabel=""/>If you are using block devices that are accessible from multiple OSS nodes, ensure that you mount the OSTs from only one OSS node at at time. It is strongly recommended that multiple-mount protection be enabled for such devices to prevent serious data corruption. For more information about multiple-mount protection, see <link xl:href="ManagingFailover.html#50438213_13563">Lustre Failover and Multiple-Mount Protection</link>.</para>
 174       <informaltable frame="none">
 175         <tgroup cols="1">
 176           <colspec colname="c1" colwidth="100*"/>
 177           <tbody>
 178             <row>
 179               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1294220" xreflabel=""/>Lustre currently supports block devices up to 16 TB on OEL 5/RHEL 5 (up to 8 TB on other distributions). If the device size is only slightly larger that 16 TB, it is recommended that you limit the file system size to 16 TB at format time. If the size is significantly larger than 16 TB, you should reconfigure the storage into devices smaller than 16 TB. We recommend that you not place partitions on top of RAID 5/6 block devices due to negative impacts on performance.</para></entry>
 180             </row>
 181           </tbody>
 182         </tgroup>
 183       </informaltable>
 184       <para><anchor xml:id="dbdoclet.50438267_pgfId-1293955" xreflabel=""/> 4. <anchor xml:id="dbdoclet.50438267_24062" xreflabel=""/>Mount the OST. On the OSS node where the OST was created, run:</para>
 185       <screen><anchor xml:id="dbdoclet.50438267_pgfId-1290925" xreflabel=""/>mount -t lustre <emphasis>&lt;block device name&gt; &lt;mount point&gt;</emphasis></screen>
 186       <informaltable frame="none">
 187         <tgroup cols="1">
 188           <colspec colname="c1" colwidth="100*"/>
 189           <tbody>
 190             <row>
 191               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1290932" xreflabel=""/>To create additional OSTs, repeat <link xl:href="ConfiguringLustre.html#50438267_81555">Step 3</link> and <link xl:href="ConfiguringLustre.html#50438267_24062">Step 4</link>.</para></entry>
 192             </row>
 193           </tbody>
 194         </tgroup>
 195       </informaltable>
 196        <para><anchor xml:id="dbdoclet.50438267_pgfId-1290934" xreflabel=""/> 5. <anchor xml:id="dbdoclet.50438267_43365" xreflabel=""/>Mount the Lustre file system on the client. On the client node, run:</para>
 197       <screen><anchor xml:id="dbdoclet.50438267_pgfId-1290935" xreflabel=""/>mount -t lustre &lt;<emphasis>MGS node</emphasis>&gt;:/&lt;<emphasis>fsname</emphasis>&gt; &lt;<emphasis>mount point</emphasis>&gt;
 198 </screen>
 199       <informaltable frame="none">
 200         <tgroup cols="1">
 201           <colspec colname="c1" colwidth="100*"/>
 202           <tbody>
 203             <row>
 204               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1290939" xreflabel=""/>To create additional clients, repeat <link xl:href="ConfiguringLustre.html#50438267_43365">Step 5</link>.</para></entry>
 205             </row>
 206           </tbody>
 207         </tgroup>
 208       </informaltable>
 209        <para><anchor xml:id="dbdoclet.50438267_pgfId-1290940" xreflabel=""/> 6. Verify that the file system started and is working correctly. Do this by running the lfs df, dd and ls commands on the client node.</para>
 210       <informaltable frame="none">
 211         <tgroup cols="1">
 212           <colspec colname="c1" colwidth="100*"/>
 213           <tbody>
 214             <row>
 215               <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1290954" xreflabel=""/>If you have a problem mounting the file system, check the syslogs on the client and all the servers for errors and also check the network settings. A common issue with newly-installed systems is that hosts.deny or firewall rules may prevent connections on port 988.</para></entry>
 216             </row>
 217           </tbody>
 218         </tgroup>
 219       </informaltable>
 220        <para><anchor xml:id="dbdoclet.50438267_pgfId-1291806" xreflabel=""/> 7. <emphasis role="bold">(Optional) Run benchmarking to</emphasis>ols to validate the performance of hardware and software layers in the cluster. Available tools include:</para>
 221       <itemizedlist><listitem>
 222           <para><anchor xml:id="dbdoclet.50438267_pgfId-1294743" xreflabel=""/>obdfilter_survey  - Characterizes the storage performance of a Lustre file system. For details, see <emphasis><link xl:href="BenchmarkingTests.html#50438212_40624">Testing OST Performance (obdfilter_survey)</link></emphasis>.</para>
 223         </listitem>
 224 <listitem>
 225           <para> </para>
 226         </listitem>
 227 <listitem>
 228           <para><anchor xml:id="dbdoclet.50438267_pgfId-1294744" xreflabel=""/>ost_survey  - Performs I/O against OSTs to detect anomalies between otherwise identical disk subsystems. For details, see <emphasis><link xl:href="BenchmarkingTests.html#50438212_85136">Testing OST I/O Performance (ost_survey)</link></emphasis>.</para>
 229         </listitem>
 230 <listitem>
 231           <para> </para>
 232         </listitem>
 233 </itemizedlist>
 234       <section remap="h3">
 235         <title><anchor xml:id="dbdoclet.50438267_pgfId-1290956" xreflabel=""/>10.1.1 Simple Lustre <anchor xml:id="dbdoclet.50438267_marker-1290955" xreflabel=""/>Configuration Example</title>
 236         <para><anchor xml:id="dbdoclet.50438267_pgfId-1290959" xreflabel=""/>To see the steps in a simple Lustre configuration, follow this example in which a combined MGS/MDT and two OSTs are created. Three block devices are used, one for the combined MGS/MDS node and one for each OSS node. Common parameters used in the example are listed below, along with individual node parameters.</para>
 237         <informaltable frame="all">
 238           <tgroup cols="4">
 239             <colspec colname="c1" colwidth="25*"/>
 240             <colspec colname="c2" colwidth="25*"/>
 241             <colspec colname="c3" colwidth="25*"/>
 242             <colspec colname="c4" colwidth="25*"/>
 243             <thead>
 244               <row>
 245                 <entry nameend="c2" namest="c1"><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1290962" xreflabel=""/>Common Parameters</emphasis></para></entry>
 246                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1290966" xreflabel=""/>Value</emphasis></para></entry>
 247                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1290968" xreflabel=""/>Description</emphasis></para></entry>
 248               </row>
 249             </thead>
 250             <tbody>
 251               <row>
 252                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290970" xreflabel=""/> </para></entry>
 253                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290972" xreflabel=""/><emphasis role="bold">MGS node</emphasis></para></entry>
 254                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290974" xreflabel=""/>10.2.0.1@tcp0</para></entry>
 255                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290976" xreflabel=""/>Node for the combined MGS/MDS</para></entry>
 256               </row>
 257               <row>
 258                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290978" xreflabel=""/> </para></entry>
 259                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290980" xreflabel=""/><emphasis role="bold">file system</emphasis></para></entry>
 260                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290982" xreflabel=""/>temp</para></entry>
 261                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290984" xreflabel=""/>Name of the Lustre file system</para></entry>
 262               </row>
 263               <row>
 264                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290986" xreflabel=""/> </para></entry>
 265                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290988" xreflabel=""/><emphasis role="bold">network type</emphasis></para></entry>
 266                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290990" xreflabel=""/>TCP/IP</para></entry>
 267                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1290992" xreflabel=""/>Network type used for Lustre file system temp</para></entry>
 268               </row>
 269             </tbody>
 270           </tgroup>
 271         </informaltable>
 272           <informaltable frame="all">
 273           <tgroup cols="4">
 274             <colspec colname="c1" colwidth="25*"/>
 275             <colspec colname="c2" colwidth="25*"/>
 276             <colspec colname="c3" colwidth="25*"/>
 277             <colspec colname="c4" colwidth="25*"/>
 278             <thead>
 279               <row>
 280                 <entry nameend="c2" namest="c1"><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1290995" xreflabel=""/>Node Parameters</emphasis></para></entry>
 281                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1290999" xreflabel=""/>Value</emphasis></para></entry>
 282                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1291001" xreflabel=""/>Description</emphasis></para></entry>
 283               </row>
 284             </thead>
 285             <tbody>
 286               <row>
 287                 <entry nameend="c4" namest="c1"><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291003" xreflabel=""/>MGS/MDS node</para></entry>
 288               </row>
 289               <row>
 290                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291011" xreflabel=""/> </para></entry>
 291                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291013" xreflabel=""/><emphasis role="bold">MGS/MDS node</emphasis></para></entry>
 292                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291015" xreflabel=""/>mdt1</para></entry>
 293                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291017" xreflabel=""/>MDS in Lustre file system temp</para></entry>
 294               </row>
 295               <row>
 296                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291019" xreflabel=""/> </para></entry>
 297                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291021" xreflabel=""/><emphasis role="bold">block device</emphasis></para></entry>
 298                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291023" xreflabel=""/>/dev/sdb</para></entry>
 299                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291025" xreflabel=""/>Block device for the combined MGS/MDS node</para></entry>
 300               </row>
 301               <row>
 302                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291027" xreflabel=""/> </para></entry>
 303                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291029" xreflabel=""/><emphasis role="bold">mount point</emphasis></para></entry>
 304                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291031" xreflabel=""/>/mnt/mdt</para></entry>
 305                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291033" xreflabel=""/>Mount point for the mdt1 block device (/dev/sdb) on the MGS/MDS node</para></entry>
 306               </row>
 307               <row>
 308                 <entry nameend="c4" namest="c1"><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291035" xreflabel=""/>First OSS node</para></entry>
 309               </row>
 310               <row>
 311                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291043" xreflabel=""/> </para></entry>
 312                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291045" xreflabel=""/><emphasis role="bold">OSS node</emphasis></para></entry>
 313                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291047" xreflabel=""/>oss1</para></entry>
 314                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291049" xreflabel=""/>First OSS node in Lustre file system temp</para></entry>
 315               </row>
 316               <row>
 317                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291051" xreflabel=""/> </para></entry>
 318                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291053" xreflabel=""/><emphasis role="bold">OST</emphasis></para></entry>
 319                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291055" xreflabel=""/>ost1</para></entry>
 320                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291057" xreflabel=""/>First OST in Lustre file system temp</para></entry>
 321               </row>
 322               <row>
 323                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291059" xreflabel=""/> </para></entry>
 324                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291061" xreflabel=""/><emphasis role="bold">block device</emphasis></para></entry>
 325                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291063" xreflabel=""/>/dev/sdc</para></entry>
 326                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291065" xreflabel=""/>Block device for the first OSS node (oss1)</para></entry>
 327               </row>
 328               <row>
 329                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291067" xreflabel=""/> </para></entry>
 330                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291069" xreflabel=""/><emphasis role="bold">mount point</emphasis></para></entry>
 331                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291071" xreflabel=""/>/mnt/ost1</para></entry>
 332                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291073" xreflabel=""/>Mount point for the ost1 block device (/dev/sdc) on the oss1 node</para></entry>
 333               </row>
 334               <row>
 335                 <entry nameend="c4" namest="c1"><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291075" xreflabel=""/>Second OSS node</para></entry>
 336               </row>
 337               <row>
 338                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291083" xreflabel=""/> </para></entry>
 339                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291085" xreflabel=""/><emphasis role="bold">OSS node</emphasis></para></entry>
 340                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291087" xreflabel=""/>oss2</para></entry>
 341                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291089" xreflabel=""/>Second OSS node in Lustre file system temp</para></entry>
 342               </row>
 343               <row>
 344                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291091" xreflabel=""/> </para></entry>
 345                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291093" xreflabel=""/><emphasis role="bold">OST</emphasis></para></entry>
 346                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291095" xreflabel=""/>ost2</para></entry>
 347                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291097" xreflabel=""/>Second OST in Lustre file system temp</para></entry>
 348               </row>
 349               <row>
 350                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291099" xreflabel=""/> </para></entry>
 351                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291101" xreflabel=""/><emphasis role="bold">block device</emphasis></para></entry>
 352                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291103" xreflabel=""/>/dev/sdd</para></entry>
 353                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291105" xreflabel=""/>Block device for the second OSS node (oss2)</para></entry>
 354               </row>
 355               <row>
 356                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291107" xreflabel=""/> </para></entry>
 357                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291109" xreflabel=""/><emphasis role="bold">mount point</emphasis></para></entry>
 358                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291111" xreflabel=""/>/mnt/ost2</para></entry>
 359                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291113" xreflabel=""/>Mount point for the ost2 block device (/dev/sdd) on the oss2 node</para></entry>
 360               </row>
 361               <row>
 362                 <entry nameend="c4" namest="c1"><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291115" xreflabel=""/>Client node</para></entry>
 363               </row>
 364               <row>
 365                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291123" xreflabel=""/> </para></entry>
 366                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291125" xreflabel=""/><emphasis role="bold">client node</emphasis></para></entry>
 367                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291127" xreflabel=""/>client1</para></entry>
 368                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291129" xreflabel=""/>Client in Lustre file system temp</para></entry>
 369               </row>
 370               <row>
 371                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291131" xreflabel=""/> </para></entry>
 372                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291133" xreflabel=""/><emphasis role="bold">mount point</emphasis></para></entry>
 373                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291135" xreflabel=""/>/lustre</para></entry>
 374                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1291137" xreflabel=""/>Mount point for Lustre file system temp on the client1 node</para></entry>
 375               </row>
 376             </tbody>
 377           </tgroup>
 378         </informaltable>
 379          <informaltable frame="none">
 380           <tgroup cols="1">
 381             <colspec colname="c1" colwidth="100*"/>
 382             <tbody>
 383               <row>
 384                 <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1291432" xreflabel=""/>We recommend that you use â€œdotted-quadâ€ notation for IP addresses rather than host names to make it easier to read debug logs and debug configurations with multiple interfaces.</para></entry>
 385               </row>
 386             </tbody>
 387           </tgroup>
 388         </informaltable>
 389         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291430" xreflabel=""/>For this example, complete the steps below:</para>
 390         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291140" xreflabel=""/> 1. Create a combined MGS/MDT file system on the block device. On the MDS node, run:</para>
 391         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291141" xreflabel=""/>[root@mds /]# mkfs.lustre --fsname=temp --mgs --mdt /dev/sdb
 392 </screen>
 393         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291142" xreflabel=""/>This command generates this output:</para>
 394         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291143" xreflabel=""/>    Permanent disk data:
 395 <anchor xml:id="dbdoclet.50438267_pgfId-1291144" xreflabel=""/>Target:            temp-MDTffff
 396 <anchor xml:id="dbdoclet.50438267_pgfId-1291145" xreflabel=""/>Index:             unassigned
 397 <anchor xml:id="dbdoclet.50438267_pgfId-1291146" xreflabel=""/>Lustre FS: temp
 398 <anchor xml:id="dbdoclet.50438267_pgfId-1291147" xreflabel=""/>Mount type:        ldiskfs
 399 <anchor xml:id="dbdoclet.50438267_pgfId-1291148" xreflabel=""/>Flags:             0x75
 400 <anchor xml:id="dbdoclet.50438267_pgfId-1291149" xreflabel=""/>   (MDT MGS needs_index first_time update )
 401 <anchor xml:id="dbdoclet.50438267_pgfId-1291150" xreflabel=""/>Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
 402 <anchor xml:id="dbdoclet.50438267_pgfId-1291151" xreflabel=""/>Parameters: mdt.group_upcall=/usr/sbin/l_getgroups
 403 <anchor xml:id="dbdoclet.50438267_pgfId-1291152" xreflabel=""/>
 404 <anchor xml:id="dbdoclet.50438267_pgfId-1291153" xreflabel=""/>checking for existing Lustre data: not found
 405 <anchor xml:id="dbdoclet.50438267_pgfId-1291154" xreflabel=""/>device size = 16MB
 406 <anchor xml:id="dbdoclet.50438267_pgfId-1291155" xreflabel=""/>2 6 18
 407 <anchor xml:id="dbdoclet.50438267_pgfId-1291156" xreflabel=""/>formatting backing filesystem ldiskfs on /dev/sdb
 408 <anchor xml:id="dbdoclet.50438267_pgfId-1291157" xreflabel=""/>   target name             temp-MDTffff
 409 <anchor xml:id="dbdoclet.50438267_pgfId-1291158" xreflabel=""/>   4k blocks               0
 410 <anchor xml:id="dbdoclet.50438267_pgfId-1291159" xreflabel=""/>   options                 -i 4096 -I 512 -q -O dir_index,uninit_groups -F
 411 <anchor xml:id="dbdoclet.50438267_pgfId-1291160" xreflabel=""/>mkfs_cmd = mkfs.ext2 -j -b 4096 -L temp-MDTffff  -i 4096 -I 512 -q -O
 412 <anchor xml:id="dbdoclet.50438267_pgfId-1291161" xreflabel=""/>dir_index,uninit_groups -F /dev/sdb
 413 <anchor xml:id="dbdoclet.50438267_pgfId-1291162" xreflabel=""/>Writing CONFIGS/mountdata
 414 </screen>
 415         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291163" xreflabel=""/> 2. Mount the combined MGS/MDT file system on the block device. On the MDS node, run:</para>
 416         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291164" xreflabel=""/>[root@mds /]# mount -t lustre /dev/sdb /mnt/mdt
 417 </screen>
 418         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291165" xreflabel=""/>This command generates this output:</para>
 419         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291166" xreflabel=""/>Lustre: temp-MDT0000: new disk, initializing
 420 <anchor xml:id="dbdoclet.50438267_pgfId-1291167" xreflabel=""/>Lustre: 3009:0:(lproc_mds.c:262:lprocfs_wr_group_upcall()) temp-MDT0000: gr\
 421 oup upcall set to /usr/sbin/l_getgroups
 422 <anchor xml:id="dbdoclet.50438267_pgfId-1291168" xreflabel=""/>Lustre: temp-MDT0000.mdt: set parameter group_upcall=/usr/sbin/l_getgroups
 423 <anchor xml:id="dbdoclet.50438267_pgfId-1291169" xreflabel=""/>Lustre: Server temp-MDT0000 on device /dev/sdb has started
 424 </screen>
 425         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291170" xreflabel=""/> 3. Create and mount ost1.</para>
 426         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291171" xreflabel=""/>In this example, the OSTs (ost1 and ost2) are being created on different OSSs (oss1 and oss2 respectively).</para>
 427         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291172" xreflabel=""/>a. Create ost1. On oss1 node, run:</para>
 428         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291173" xreflabel=""/>[root@oss1 /]# mkfs.lustre --ost --fsname=temp --mgsnode=10.2.0.1@tcp0 /dev\
 429 /sdc
 430 </screen>
 431         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291174" xreflabel=""/>The command generates this output:</para>
 432         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291175" xreflabel=""/>    Permanent disk data:
 433 <anchor xml:id="dbdoclet.50438267_pgfId-1291176" xreflabel=""/>Target:            temp-OSTffff
 434 <anchor xml:id="dbdoclet.50438267_pgfId-1291177" xreflabel=""/>Index:             unassigned
 435 <anchor xml:id="dbdoclet.50438267_pgfId-1291178" xreflabel=""/>Lustre FS: temp
 436 <anchor xml:id="dbdoclet.50438267_pgfId-1291179" xreflabel=""/>Mount type:        ldiskfs
 437 <anchor xml:id="dbdoclet.50438267_pgfId-1291180" xreflabel=""/>Flags:             0x72
 438 <anchor xml:id="dbdoclet.50438267_pgfId-1291181" xreflabel=""/>(OST needs_index first_time update)
 439 <anchor xml:id="dbdoclet.50438267_pgfId-1291182" xreflabel=""/>Persistent mount opts: errors=remount-ro,extents,mballoc
 440 <anchor xml:id="dbdoclet.50438267_pgfId-1291183" xreflabel=""/>Parameters: mgsnode=10.2.0.1@tcp
 441 <anchor xml:id="dbdoclet.50438267_pgfId-1291184" xreflabel=""/>
 442 <anchor xml:id="dbdoclet.50438267_pgfId-1291185" xreflabel=""/>checking for existing Lustre data: not found
 443 <anchor xml:id="dbdoclet.50438267_pgfId-1291186" xreflabel=""/>device size = 16MB
 444 <anchor xml:id="dbdoclet.50438267_pgfId-1291187" xreflabel=""/>2 6 18
 445 <anchor xml:id="dbdoclet.50438267_pgfId-1291188" xreflabel=""/>formatting backing filesystem ldiskfs on /dev/sdc
 446 <anchor xml:id="dbdoclet.50438267_pgfId-1291189" xreflabel=""/>   target name             temp-OSTffff
 447 <anchor xml:id="dbdoclet.50438267_pgfId-1291190" xreflabel=""/>   4k blocks               0
 448 <anchor xml:id="dbdoclet.50438267_pgfId-1291191" xreflabel=""/>   options                 -I 256 -q -O dir_index,uninit_groups -F
 449 <anchor xml:id="dbdoclet.50438267_pgfId-1291192" xreflabel=""/>mkfs_cmd = mkfs.ext2 -j -b 4096 -L temp-OSTffff  -I 256 -q -O
 450 <anchor xml:id="dbdoclet.50438267_pgfId-1291193" xreflabel=""/>dir_index,uninit_groups -F /dev/sdc
 451 <anchor xml:id="dbdoclet.50438267_pgfId-1291194" xreflabel=""/>Writing CONFIGS/mountdata
 452 </screen>
 453         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293613" xreflabel=""/>b. Mount ost1 on the OSS on which it was created. On oss1 node, run:</para>
 454         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1293614" xreflabel=""/>root@oss1 /] mount -t lustre /dev/sdc /mnt/ost1
 455 </screen>
 456         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293615" xreflabel=""/>The command generates this output:</para>
 457         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1293616" xreflabel=""/>LDISKFS-fs: file extents enabled
 458 <anchor xml:id="dbdoclet.50438267_pgfId-1293617" xreflabel=""/>LDISKFS-fs: mballoc enabled
 459 <anchor xml:id="dbdoclet.50438267_pgfId-1293618" xreflabel=""/>Lustre: temp-OST0000: new disk, initializing
 460 <anchor xml:id="dbdoclet.50438267_pgfId-1293619" xreflabel=""/>Lustre: Server temp-OST0000 on device /dev/sdb has started
 461 </screen>
 462         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293620" xreflabel=""/>Shortly afterwards, this output appears:</para>
 463         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1293621" xreflabel=""/>Lustre: temp-OST0000: received MDS connection from 10.2.0.1@tcp0
 464 <anchor xml:id="dbdoclet.50438267_pgfId-1293643" xreflabel=""/>Lustre: MDS temp-MDT0000: temp-OST0000_UUID now active, resetting orphans
 465 </screen>
 466         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293656" xreflabel=""/> 4. Create and mount ost2.</para>
 467         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293645" xreflabel=""/>a. Create ost2. On oss2 node, run:</para>
 468         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291196" xreflabel=""/>[root@oss2 /]# mkfs.lustre --ost --fsname=temp --mgsnode=10.2.0.1@tcp0 /dev\
 469 /sdd
 470 </screen>
 471         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291197" xreflabel=""/>The command generates this output:</para>
 472         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291198" xreflabel=""/>    Permanent disk data:
 473 <anchor xml:id="dbdoclet.50438267_pgfId-1291199" xreflabel=""/>Target:            temp-OSTffff
 474 <anchor xml:id="dbdoclet.50438267_pgfId-1291200" xreflabel=""/>Index:             unassigned
 475 <anchor xml:id="dbdoclet.50438267_pgfId-1291201" xreflabel=""/>Lustre FS: temp
 476 <anchor xml:id="dbdoclet.50438267_pgfId-1291202" xreflabel=""/>Mount type:        ldiskfs
 477 <anchor xml:id="dbdoclet.50438267_pgfId-1291203" xreflabel=""/>Flags:             0x72
 478 <anchor xml:id="dbdoclet.50438267_pgfId-1291204" xreflabel=""/>(OST needs_index first_time update)
 479 <anchor xml:id="dbdoclet.50438267_pgfId-1291205" xreflabel=""/>Persistent mount opts: errors=remount-ro,extents,mballoc
 480 <anchor xml:id="dbdoclet.50438267_pgfId-1291206" xreflabel=""/>Parameters: mgsnode=10.2.0.1@tcp
 481 <anchor xml:id="dbdoclet.50438267_pgfId-1291209" xreflabel=""/>
 482 <anchor xml:id="dbdoclet.50438267_pgfId-1291211" xreflabel=""/>checking for existing Lustre data: not found
 483 <anchor xml:id="dbdoclet.50438267_pgfId-1291212" xreflabel=""/>device size = 16MB
 484 <anchor xml:id="dbdoclet.50438267_pgfId-1291213" xreflabel=""/>2 6 18
 485 <anchor xml:id="dbdoclet.50438267_pgfId-1291214" xreflabel=""/>formatting backing filesystem ldiskfs on /dev/sdd
 486 <anchor xml:id="dbdoclet.50438267_pgfId-1291215" xreflabel=""/>   target name             temp-OSTffff
 487 <anchor xml:id="dbdoclet.50438267_pgfId-1291216" xreflabel=""/>   4k blocks               0
 488 <anchor xml:id="dbdoclet.50438267_pgfId-1291217" xreflabel=""/>   options                 -I 256 -q -O dir_index,uninit_groups -F
 489 <anchor xml:id="dbdoclet.50438267_pgfId-1291218" xreflabel=""/>mkfs_cmd = mkfs.ext2 -j -b 4096 -L temp-OSTffff  -I 256 -q -O
 490 <anchor xml:id="dbdoclet.50438267_pgfId-1291219" xreflabel=""/>dir_index,uninit_groups -F /dev/sdc
 491 <anchor xml:id="dbdoclet.50438267_pgfId-1291220" xreflabel=""/>Writing CONFIGS/mountdata
 492 </screen>
 493         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291233" xreflabel=""/>b. Mount ost2 on the OSS on which it was created. On oss2 node, run:</para>
 494         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291234" xreflabel=""/>root@oss2 /] mount -t lustre /dev/sdd /mnt/ost2
 495 </screen>
 496         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291235" xreflabel=""/>The command generates this output:</para>
 497         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291236" xreflabel=""/>LDISKFS-fs: file extents enabled
 498 <anchor xml:id="dbdoclet.50438267_pgfId-1291237" xreflabel=""/>LDISKFS-fs: mballoc enabled
 499 <anchor xml:id="dbdoclet.50438267_pgfId-1291238" xreflabel=""/>Lustre: temp-OST0000: new disk, initializing
 500 <anchor xml:id="dbdoclet.50438267_pgfId-1291239" xreflabel=""/>Lustre: Server temp-OST0000 on device /dev/sdb has started
 501 </screen>
 502         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291240" xreflabel=""/>Shortly afterwards, this output appears:</para>
 503         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291241" xreflabel=""/>Lustre: temp-OST0000: received MDS connection from 10.2.0.1@tcp0
 504 <anchor xml:id="dbdoclet.50438267_pgfId-1291242" xreflabel=""/>Lustre: MDS temp-MDT0000: temp-OST0000_UUID now active, resetting orphans
 505 </screen>
 506         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291243" xreflabel=""/> 5. Mount the Lustre file system on the client. On the client node, run:</para>
 507         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291244" xreflabel=""/>root@client1 /] mount -t lustre 10.2.0.1@tcp0:/temp /lustre
 508 </screen>
 509         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291245" xreflabel=""/>This command generates this output:</para>
 510         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291246" xreflabel=""/>Lustre: Client temp-client has started
 511 </screen>
 512         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291247" xreflabel=""/> 6. Verify that the file system started and is working by running the df, dd and ls commands on the client node.</para>
 513         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291248" xreflabel=""/>a. Run the lfsdf -h command:</para>
 514         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291249" xreflabel=""/>[root@client1 /] lfs df -h
 515 </screen>
 516         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293698" xreflabel=""/>The lfsdf-h command lists space usage per OST and the MDT in human-readable format. This command generates output similar to this:</para>
 517         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1294042" xreflabel=""/>UUID                        bytes           Used            Available      \
 518  Use%    Mounted on
 519 <anchor xml:id="dbdoclet.50438267_pgfId-1294043" xreflabel=""/>temp-MDT0000_UUID  8.0G            400.0M          7.6G            0%      \
 520 /lustre[MDT:0]
 521 <anchor xml:id="dbdoclet.50438267_pgfId-1294044" xreflabel=""/>temp-OST0000_UUID  800.0G          400.0M          799.6G          0%      \
 522 /lustre[OST:0]
 523 <anchor xml:id="dbdoclet.50438267_pgfId-1294045" xreflabel=""/>temp-OST0001_UUID  800.0G          400.0M          799.6G          0%      \
 524 /lustre[OST:1]
 525 <anchor xml:id="dbdoclet.50438267_pgfId-1294046" xreflabel=""/>filesystem summary:        1.6T            800.0M          1.6T            \
 526 0%      /lustre
 527 <anchor xml:id="dbdoclet.50438267_pgfId-1294039" xreflabel=""/>
 528 </screen>
 529         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293715" xreflabel=""/>b. Run the lfsdf-ih command.</para>
 530         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1293716" xreflabel=""/>[root@client1 /] lfs df -ih
 531 </screen>
 532         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293717" xreflabel=""/>The lfsdf-ih command lists inode usage per OST and the MDT. This command generates output similar to this:</para>
 533         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1294079" xreflabel=""/>UUID                        Inodes       IUsed      IFree   IUse%   Mounted\
 534  on
 535 <anchor xml:id="dbdoclet.50438267_pgfId-1294080" xreflabel=""/>temp-MDT0000_UUID  2.5M         32         2.5M    0%      /lustre[MDT:0]
 536 <anchor xml:id="dbdoclet.50438267_pgfId-1294081" xreflabel=""/>temp-OST0000_UUID  5.5M         54         5.5M    0%      /lustre[OST:0]
 537 <anchor xml:id="dbdoclet.50438267_pgfId-1294082" xreflabel=""/>temp-OST0001_UUID  5.5M         54         5.5M    0%      /lustre[OST:1]
 538 <anchor xml:id="dbdoclet.50438267_pgfId-1294083" xreflabel=""/>filesystem summary:        2.5M         32         2.5M    0%      /lustre
 539 <anchor xml:id="dbdoclet.50438267_pgfId-1294084" xreflabel=""/>
 540 </screen>
 541         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291257" xreflabel=""/>c. Run the dd command:</para>
 542         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291258" xreflabel=""/>[root@client1 /] cd /lustre
 543 <anchor xml:id="dbdoclet.50438267_pgfId-1291259" xreflabel=""/>[root@client1 /lustre] dd if=/dev/zero of=/lustre/zero.dat bs=4M count=2
 544 </screen>
 545         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293740" xreflabel=""/>The dd command verifies write functionality by creating a file containing all zeros (0s). In this command, an 8 MB file is created. This command generates output similar to this:</para>
 546         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291261" xreflabel=""/>2+0 records in
 547 <anchor xml:id="dbdoclet.50438267_pgfId-1291262" xreflabel=""/>2+0 records out
 548 <anchor xml:id="dbdoclet.50438267_pgfId-1291263" xreflabel=""/>8388608 bytes (8.4 MB) copied, 0.159628 seconds, 52.6 MB/s
 549 </screen>
 550         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291264" xreflabel=""/>d. Run the ls command:</para>
 551         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291265" xreflabel=""/>[root@client1 /lustre] ls -lsah
 552 </screen>
 553         <para><anchor xml:id="dbdoclet.50438267_pgfId-1291266" xreflabel=""/>The ls-lsah command lists files and directories in the current working directory. This command generates output similar to this:</para>
 554         <screen><anchor xml:id="dbdoclet.50438267_pgfId-1291267" xreflabel=""/>total 8.0M
 555 <anchor xml:id="dbdoclet.50438267_pgfId-1291268" xreflabel=""/>4.0K drwxr-xr-x  2 root root 4.0K Oct 16 15:27 .
 556 <anchor xml:id="dbdoclet.50438267_pgfId-1292416" xreflabel=""/>8.0K drwxr-xr-x 25 root root 4.0K Oct 16 15:27 ..
 557 <anchor xml:id="dbdoclet.50438267_pgfId-1292417" xreflabel=""/>8.0M -rw-r--r--  1 root root 8.0M Oct 16 15:27 zero.dat
 558 <anchor xml:id="dbdoclet.50438267_pgfId-1292423" xreflabel=""/>
 559 </screen>
 560         <para><anchor xml:id="dbdoclet.50438267_pgfId-1293200" xreflabel=""/>Once the Lustre file system is configured, it is ready for use.</para>
 561       </section>
 562     </section>
 563     <section remap="h2">
 564       <title>10.2 <anchor xml:id="dbdoclet.50438267_76752" xreflabel=""/>Additional Configuration Options</title>
 565       <para><anchor xml:id="dbdoclet.50438267_pgfId-1292458" xreflabel=""/>This section describes how to scale the Lustre file system or make configuration changes using the Lustre configuration utilities.</para>
 566       <section remap="h3">
 567         <title><anchor xml:id="dbdoclet.50438267_pgfId-1292441" xreflabel=""/>10.2.1 Scaling the <anchor xml:id="dbdoclet.50438267_marker-1292440" xreflabel=""/>Lustre File System</title>
 568         <para><anchor xml:id="dbdoclet.50438267_pgfId-1292442" xreflabel=""/>A Lustre file system can be scaled by adding OSTs or clients. For instructions on creating additional OSTs repeat <link xl:href="ConfiguringLustre.html#50438267_81555">Step 3</link> and <link xl:href="ConfiguringLustre.html#50438267_24062">Step 4</link> above. For mounting additional clients, repeat <link xl:href="ConfiguringLustre.html#50438267_43365">Step 5</link> for each client.</para>
 569       </section>
 570       <section remap="h3">
 571         <title><anchor xml:id="dbdoclet.50438267_pgfId-1292798" xreflabel=""/>10.2.2 <anchor xml:id="dbdoclet.50438267_50212" xreflabel=""/>Changing Striping Defaults</title>
 572         <para><anchor xml:id="dbdoclet.50438267_pgfId-1292906" xreflabel=""/>The default settings for the file layout stripe pattern are shown in <link xl:href="ConfiguringLustre.html#50438267_70881">TABLE 10-1</link>.</para>
 573         <table frame="none">
 574           <title><anchor xml:id="dbdoclet.50438267_pgfId-1292871" xreflabel=""/> TABLE 10-1 <anchor xml:id="dbdoclet.50438267_70881" xreflabel=""/></title>
 575           <tgroup cols="3">
 576             <colspec colname="c1" colwidth="33*"/>
 577             <colspec colname="c2" colwidth="33*"/>
 578             <colspec colname="c3" colwidth="33*"/>
 579             <tbody>
 580               <row>
 581                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1292877" xreflabel=""/>File Layout Parameter</emphasis></para></entry>
 582                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1292879" xreflabel=""/>Default</emphasis></para></entry>
 583                 <entry><para><emphasis role="bold"><anchor xml:id="dbdoclet.50438267_pgfId-1292881" xreflabel=""/>Description</emphasis></para></entry>
 584               </row>
 585               <row>
 586                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292883" xreflabel=""/>stripe_size</para></entry>
 587                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292885" xreflabel=""/>1 MB</para></entry>
 588                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292887" xreflabel=""/>Amount of data to write to one OST before moving to the next OST.</para></entry>
 589               </row>
 590               <row>
 591                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292889" xreflabel=""/>stripe_count</para></entry>
 592                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292891" xreflabel=""/>1</para></entry>
 593                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292893" xreflabel=""/>The number of OSTs to use for a single file.</para></entry>
 594               </row>
 595               <row>
 596                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292895" xreflabel=""/>start_ost</para></entry>
 597                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292897" xreflabel=""/>-1</para></entry>
 598                 <entry><para> <anchor xml:id="dbdoclet.50438267_pgfId-1292899" xreflabel=""/>The first OST where objects are created for each file. The default -1 allows the MDS to choose the starting index based on available space and load balancing. <emphasis>Itâ€™s strongly recommended not to change the default for this parameter to a value other than -1.</emphasis></para></entry>
 599               </row>
 600             </tbody>
 601           </tgroup>
 602         </table>
 603         <para><anchor xml:id="dbdoclet.50438267_pgfId-1292907" xreflabel=""/>Use the lfs setstripe command described in <link xl:href="ManagingStripingFreeSpace.html#50438209_78664">Setting the File Layout/Striping Configuration (lfs setstripe)</link> to change the file layout configuration.</para>
 604       </section>
 605       <section remap="h3">
 606         <title><anchor xml:id="dbdoclet.50438267_pgfId-1292908" xreflabel=""/>10.2.3 Using the Lustre Configuration Utilities</title>
 607         <para><anchor xml:id="dbdoclet.50438267_pgfId-1292436" xreflabel=""/>If additional configuration is necessary, several configuration utilities are available:</para>
 608         <itemizedlist><listitem>
 609             <para><anchor xml:id="dbdoclet.50438267_pgfId-1292490" xreflabel=""/>mkfs.lustre  - Use to format a disk for a Lustre service.</para>
 610           </listitem>
 611 <listitem>
 612             <para> </para>
 613           </listitem>
 614 <listitem>
 615             <para><anchor xml:id="dbdoclet.50438267_pgfId-1292492" xreflabel=""/>tunefs.lustre  - Use to modify configuration information on a Lustre target disk.</para>
 616           </listitem>
 617 <listitem>
 618             <para> </para>
 619           </listitem>
 620 <listitem>
 621             <para><anchor xml:id="dbdoclet.50438267_pgfId-1292494" xreflabel=""/>lctl  - Use to directly control Lustre via an ioctl interface, allowing various configuration, maintenance and debugging features to be accessed.</para>
 622           </listitem>
 623 <listitem>
 624             <para> </para>
 625           </listitem>
 626 <listitem>
 627             <para><anchor xml:id="dbdoclet.50438267_pgfId-1291315" xreflabel=""/>mount.lustre  - Use to start a Lustre client or target service.</para>
 628           </listitem>
 629 <listitem>
 630             <para> </para>
 631           </listitem>
 632 </itemizedlist>
 633         <para><anchor xml:id="dbdoclet.50438267_pgfId-1295269" xreflabel=""/>For examples using these utilities, see the topic <emphasis><link xl:href="http://wiki.lustre.org/index.php/Lustre_System_Configuration_Utilities"/></emphasis><link xl:href="SystemConfigurationUtilities.html#50438219_66186">Chapter 36</link><emphasis>:</emphasis><link xl:href="SystemConfigurationUtilities.html#50438219_66186">System Configuration Utilities</link> on the Lustre wiki.</para>
 634         <para><anchor xml:id="dbdoclet.50438267_pgfId-1295265" xreflabel=""/>The lfs utility is usful for configuring and querying a variety of options related to files. For more information, see <link xl:href="UserUtilities_HTML.html#50438206_94597">lfs</link>.</para>
 635         <informaltable frame="none">
 636           <tgroup cols="1">
 637             <colspec colname="c1" colwidth="100*"/>
 638             <tbody>
 639               <row>
 640                 <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438267_pgfId-1291457" xreflabel=""/>Some sample scripts are included in the directory where Lustre is installed. If you have installed the Lustre source code, the scripts are located in the lustre/tests sub-directory. These scripts enable quick setup of some simple standard Lustre configurations.</para></entry>
 641               </row>
 642             </tbody>
 643           </tgroup>
 644         </informaltable>
 645         <para><anchor xml:id="dbdoclet.50438267_pgfId-1292699" xreflabel=""/> </para>
 646         <!--
 647 Begin SiteCatalyst code version: G.5.
 648 -->
 649         <!--
 650 End SiteCatalyst code version: G.5.
 651 -->
 652           <informaltable frame="none">
 653           <tgroup cols="3">
 654             <colspec colname="c1" colwidth="33*"/>
 655             <colspec colname="c2" colwidth="33*"/>
 656             <colspec colname="c3" colwidth="33*"/>
 657
 658
 659
 660             <tbody>
 661               <row>
 662                 <entry align="left"><para>Lustre 2.0 Operations Manual</para></entry>
 663                 <entry align="right"><para>821-2076-10</para></entry>
 664                 <entry align="right" valign="top"><para><link xl:href="index.html"><inlinemediaobject><imageobject role="html">
 665                           <imagedata contentdepth="26" contentwidth="30" fileref="./shared/toc01.gif" scalefit="1"/>
 666                         </imageobject>
 667 <imageobject role="fo">
 668                           <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/toc01.gif" scalefit="1" width="100%"/>
 669                         </imageobject>
 670 </inlinemediaobject></link><link xl:href="ConfiguringLNET.html"><inlinemediaobject><imageobject role="html">
 671                           <imagedata contentdepth="26" contentwidth="30" fileref="./shared/prev01.gif" scalefit="1"/>
 672                         </imageobject>
 673 <imageobject role="fo">
 674                           <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/prev01.gif" scalefit="1" width="100%"/>
 675                         </imageobject>
 676 </inlinemediaobject></link><link xl:href="ConfiguringFailover.html"><inlinemediaobject><imageobject role="html">
 677                           <imagedata contentdepth="26" contentwidth="30" fileref="./shared/next01.gif" scalefit="1"/>
 678                         </imageobject>
 679 <imageobject role="fo">
 680                           <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/next01.gif" scalefit="1" width="100%"/>
 681                         </imageobject>
 682 </inlinemediaobject></link><link xl:href="ix.html"><inlinemediaobject><imageobject role="html">
 683                           <imagedata contentdepth="26" contentwidth="30" fileref="./shared/index01.gif" scalefit="1"/>
 684                         </imageobject>
 685 <imageobject role="fo">
 686                           <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/index01.gif" scalefit="1" width="100%"/>
 687                         </imageobject>
 688 </inlinemediaobject></link></para></entry>
 689               </row>
 690             </tbody>
 691           </tgroup>
 692         </informaltable>
 693         <para><link xl:href=""/></para>
 694         <para><link xl:href="copyright.html">Copyright</link> © 2011, Oracle and/or its affiliates. All rights reserved.</para>
 695       </section>
 696     </section>
 697   </section>
 698 </article>