LUDOC-331 setup: improve ldiskfs formatting options

Improve the description of the ldiskfs formatting options.
Add limit for the maximum number of objects per ldiskfs OST.
Improve related cross-reference section names.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I96eac773362327d303153d3409dbc740723ebbe5
Reviewed-on: http://review.whamcloud.com/21262
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

diff --git a/ConfiguringStorage.xml b/ConfiguringStorage.xml
index 16af1bb..00e7bee 100644 (file)
--- a/ConfiguringStorage.xml
+++ b/ConfiguringStorage.xml
@@ -19,7 +19,7 @@
     </listitem>
     <listitem>
       <para>
-            <xref linkend="dbdoclet.50438208_51921"/>
+            <xref linkend="dbdoclet.ldiskfs_raid_opts"/>
         </para>
     </listitem>
     <listitem>
@@ -66,17 +66,31 @@
     <para>If writeback cache is enabled, a file system check is required after the array loses power. Data may also be lost because of this.</para>
     <para>Therefore, we recommend against the use of writeback cache when data integrity is critical. You should carefully consider whether the benefits of using writeback cache outweigh the risks.</para>
   </section>
-  <section xml:id="dbdoclet.50438208_51921">
-    <title><indexterm><primary>storage</primary><secondary>configuring</secondary><tertiary>RAID options</tertiary></indexterm>Formatting Options for RAID Devices</title>
-    <para>When formatting a file system on a RAID device, it is beneficial to ensure that I/O
-      requests are aligned with the underlying RAID geometry. This ensures that Lustre RPCs do not
-      generate unnecessary disk operations which may reduce performance dramatically. Use the
-        <literal>--mkfsoptions</literal> parameter to specify additional parameters when formatting
-      the OST or MDT.</para>
-    <para>For RAID 5, RAID 6, or RAID 1+0 storage, specifying the following option to the <literal>--mkfsoptions</literal> parameter option improves the layout of the file system metadata, ensuring that no single disk contains all of the allocation bitmaps:</para>
+  <section xml:id="dbdoclet.ldiskfs_raid_opts">
+    <title>
+      <indexterm>
+        <primary>storage</primary>
+        <secondary>configuring</secondary>
+       <tertiary>RAID options</tertiary>
+      </indexterm>Formatting Options for ldiskfs RAID Devices</title>
+    <para>When formatting an ldiskfs file system on a RAID device, it can be
+    beneficial to ensure that I/O requests are aligned with the underlying
+    RAID geometry. This ensures that Lustre RPCs do not generate unnecessary
+    disk operations which may reduce performance dramatically. Use the
+    <literal>--mkfsoptions</literal> parameter to specify additional parameters
+    when formatting the OST or MDT.</para>
+    <para>For RAID 5, RAID 6, or RAID 1+0 storage, specifying the following
+    option to the <literal>--mkfsoptions</literal> parameter option improves
+    the layout of the file system metadata, ensuring that no single disk
+    contains all of the allocation bitmaps:</para>
     <screen>-E stride = <replaceable>chunk_blocks</replaceable> </screen>
-    <para>The <literal><replaceable>chunk_blocks</replaceable></literal> variable is in units of 4096-byte blocks and represents the amount of contiguous data written to a single disk before moving to the next disk. This is alternately referred to as the RAID stripe size. This is applicable to both MDT and OST file systems.</para>
-    <para>For more information on how to override the defaults while formatting MDT or OST file systems, see <xref linkend="dbdoclet.50438256_84701"/>.</para>
+    <para>The <literal><replaceable>chunk_blocks</replaceable></literal>
+    variable is in units of 4096-byte blocks and represents the amount of
+    contiguous data written to a single disk before moving to the next disk.
+    This is alternately referred to as the RAID stripe size. This is
+    applicable to both MDT and OST file systems.</para>
+    <para>For more information on how to override the defaults while formatting
+    MDT or OST file systems, see <xref linkend="dbdoclet.ldiskfs_mkfs_opts"/>.</para>
     <section remap="h3">
       <title><indexterm><primary>storage</primary><secondary>configuring</secondary><tertiary>for mkfs</tertiary></indexterm>Computing file system parameters for mkfs</title>
       <para>For best results, use RAID 5 with 5 or 9 disks or RAID 6 with 6 or 10 disks, each on a different controller. The stripe width is the optimal minimum I/O size. Ideally, the RAID configuration should allow 1 MB Lustre RPCs to fit evenly on a single RAID stripe without an expensive read-modify-write cycle. Use this formula to determine the
@@ -132,7 +146,7 @@
         <listitem>
           <para>Create the OST.</para>
           <para>In this example, <literal>/dev/sdc</literal> is the RAID 6 device to be used as the OST, run:</para>
-          <screen>[oss#] mkfs.lustre --mgsnode=mds@osib --ost --index=0 \
+          <screen>[oss#] mkfs.lustre --ost ... \
 --mkfsoptions=&quot;-J device=/dev/sdb1&quot; /dev/sdc</screen>
         </listitem>
         <listitem>

diff --git a/SettingUpLustreSystem.xml b/SettingUpLustreSystem.xml
index c473783..ac425aa 100644 (file)
--- a/SettingUpLustreSystem.xml
+++ b/SettingUpLustreSystem.xml
@@ -17,7 +17,7 @@
     </listitem>
     <listitem>
       <para>
-          <xref linkend="dbdoclet.50438256_84701"/>
+          <xref linkend="dbdoclet.ldiskfs_mkfs_opts"/>
       </para>
     </listitem>
     <listitem>
@@ -276,14 +276,22 @@
       See <xref linkend="dbdoclet.ldiskfs_ost_mkfs"/> for more details.</para>
     </section>
   </section>
-  <section xml:id="dbdoclet.50438256_84701">
-      <title>
-          <indexterm><primary>file system</primary><secondary>formatting options</secondary></indexterm>
-          <indexterm><primary>setup</primary><secondary>file system</secondary></indexterm>
-          Setting File System Formatting Options</title>
-    <para>By default, the <literal>mkfs.lustre</literal> utility applies these options to the Lustre
-      backing file system used to store data and metadata in order to enhance Lustre file system
-      performance and scalability. These options include:</para>
+  <section xml:id="dbdoclet.ldiskfs_mkfs_opts">
+    <title>
+      <indexterm>
+        <primary>ldiskfs</primary>
+       <secondary>formatting options</secondary>
+      </indexterm>
+      <indexterm>
+        <primary>setup</primary>
+       <secondary>ldiskfs</secondary>
+      </indexterm>
+      Setting ldiskfs File System Formatting Options
+    </title>
+    <para>By default, the <literal>mkfs.lustre</literal> utility applies these
+    options to the Lustre backing file system used to store data and metadata
+    in order to enhance Lustre file system performance and scalability. These
+    options include:</para>
         <itemizedlist>
             <listitem>
               <para><literal>flex_bg</literal> - When the flag is set to enable this
@@ -359,7 +367,7 @@
           <primary>inodes</primary>
           <secondary>OST</secondary>
         </indexterm>Setting Formatting Options for an ldiskfs OST</title>
-      <para>When formatting an OST file system, it is normally advantageous
+      <para>When formatting an OST file system, it can be beneficial
       to take local file system usage into account. When doing so, try to
       reduce the number of inodes on each OST, while keeping enough margin
       for potential variations in future usage. This helps reduce the format
@@ -369,8 +377,8 @@
       used for OSTs of various sizes when they are formatted.</para>
       <para>
         <table frame="all">
-          <title xml:id="settinguplustresystem.tab1">Inode Ratios Used for Newly Formatted
-            OSTs</title>
+          <title xml:id="settinguplustresystem.tab1">Default Inode Ratios
+         Used for Newly Formatted OSTs</title>
           <tgroup cols="3">
             <colspec colname="c1" colwidth="3*"/>
             <colspec colname="c2" colwidth="2*"/>
@@ -381,7 +389,7 @@
                   <para><emphasis role="bold">LUN/OST size</emphasis></para>
                 </entry>
                 <entry>
-                  <para><emphasis role="bold">Inode ratio</emphasis></para>
+                  <para><emphasis role="bold">Default Inode ratio</emphasis></para>
                 </entry>
                 <entry>
                   <para><emphasis role="bold">Total inodes</emphasis></para>
@@ -441,9 +449,22 @@
       may result in far too many inodes for the average file size. In this
       case, performance can be improved by increasing the number of
       <emphasis role="italic">bytes-per-inode</emphasis>.  To set the inode
-      ratio, use the <literal>-i</literal> argument to
-      <literal>mkfs.lustre</literal> to specify the
-      <emphasis role="italic">bytes-per-inode</emphasis> value.</para>
+      ratio, use the <literal>--mkfsoptions="-i <replaceable>bytes-per-inode</replaceable>"</literal>
+      argument to <literal>mkfs.lustre</literal> to specify the expected
+      average (mean) size of OST objects.  For example, to create an OST
+      with an expected average object size of 8MB run:
+      <screen>[oss#] mkfs.lustre --ost --mkfsoptions=&quot;-i $((8192 * 1024))&quot; ...</screen>
+      </para>
+      <note>
+        <para>OSTs formatted with ldiskfs are limited to a maximum of
+       320 million to 1 billion objects.  Specifying a very small
+       bytes-per-inode ratio for a large OST that causes this limit to be
+       exceeded can cause either premature out-of-space errors and prevent
+       the full OST space from being used, or will waste space and slow down
+       e2fsck more than necessary.  The default inode ratios are chosen to
+       ensure that the total number of inodes remain below this limit.
+       </para>
+      </note>
       <note>
         <para>File system check time on OSTs is affected by a number of
        variables in addition to the number of inodes, including the size of
@@ -454,7 +475,7 @@
        if substantial errors are detected and need to be required.</para>
       </note>
       <para>For more details about formatting MDT and OST file systems,
-      see <xref linkend="dbdoclet.50438208_51921"/>.</para>
+      see <xref linkend="dbdoclet.ldiskfs_raid_opts"/>.</para>
     </section>
     <section remap="h3">
       <title><indexterm>