LUDOC-504 nodemap: servers must be in a trusted+admin group

[doc/manual.git] / LustreMaintenance.xml

diff --git a/LustreMaintenance.xml b/LustreMaintenance.xml
index 271d1b3..ac0a3bf 100644 (file)
--- a/LustreMaintenance.xml
+++ b/LustreMaintenance.xml
@@ -1,4 +1,7 @@
-<?xml version='1.0' encoding='UTF-8'?><chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="lustremaintenance">
+<?xml version='1.0' encoding='UTF-8'?>
+<chapter xmlns="http://docbook.org/ns/docbook"
+ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
+ xml:id="lustremaintenance">

   <title xml:id="lustremaintenance.title">Lustre Maintenance</title>
   <para>Once you have the Lustre file system up and running, you can use the procedures in this section to perform these basic Lustre maintenance tasks:</para>
   <itemizedlist>
   <title xml:id="lustremaintenance.title">Lustre Maintenance</title>
   <para>Once you have the Lustre file system up and running, you can use the procedures in this section to perform these basic Lustre maintenance tasks:</para>
   <itemizedlist>


@@ -62,6 +65,9 @@
     <listitem>
       <para><xref linkend="lustremaint.setMDTReadonly"/></para>
     </listitem>
     <listitem>
       <para><xref linkend="lustremaint.setMDTReadonly"/></para>
     </listitem>

+    <listitem>
+      <para><xref linkend="lustremaint.tunefallocate"/></para>
+    </listitem>

   </itemizedlist>
   <section xml:id="lustremaint.inactiveOST">
       <title>
   </itemizedlist>
   <section xml:id="lustremaint.inactiveOST">
       <title>


@@ -230,15 +236,18 @@ mgs# lctl --device MGS llog_print <replaceable>fsname</replaceable>-OST0000
     <section xml:id="lustremaint.changingservernid">
       <title><indexterm><primary>maintenance</primary><secondary>changing a NID</secondary></indexterm>
 Changing a Server NID</title>
     <section xml:id="lustremaint.changingservernid">
       <title><indexterm><primary>maintenance</primary><secondary>changing a NID</secondary></indexterm>
 Changing a Server NID</title>

-      <para>In Lustre software release 2.3 or earlier, the <literal>tunefs.lustre
-        --writeconf</literal> command is used to rewrite all of the configuration files.</para>
-      <para condition="l24">If you need to change the NID on the MDT or OST, a new
-        <literal>replace_nids</literal> command was added in Lustre software release 2.4 to simplify
-      this process. The <literal>replace_nids</literal> command differs from <literal>tunefs.lustre
-        --writeconf</literal> in that it does not erase the entire configuration log, precluding the
-      need the need to execute the <literal>writeconf</literal> command on all servers and
-      re-specify all permanent parameter settings. However, the <literal>writeconf</literal> command
-      can still be used if desired.</para>
+      <para>In order to totally rewrite the Lustre configuration, the
+      <literal>tunefs.lustre --writeconf</literal> command is used to
+        rewrite all of the configuration files.</para>
+      <para>If you need to change only the NID of the MDT or OST, the
+        <literal>replace_nids</literal> command can simplify this process.
+       The <literal>replace_nids</literal> command differs from
+       <literal>tunefs.lustre --writeconf</literal> in that it does not
+       erase the entire configuration log, precluding the need the need to
+       execute the <literal>writeconf</literal> command on all servers and
+        re-specify all permanent parameter settings. However, the
+       <literal>writeconf</literal> command can still be used if desired.
+      </para>

       <para>Change a server NID in these situations:</para>
       <itemizedlist>
         <listitem>
       <para>Change a server NID in these situations:</para>
       <itemizedlist>
         <listitem>


@@ -344,7 +353,7 @@ Changing a Server NID</title>
             </listitem>
           </orderedlist>
        </section>
             </listitem>
           </orderedlist>
        </section>

-    <section xml:id="lustremaint.adding_new_mdt" condition='l24'>
+    <section xml:id="lustremaint.adding_new_mdt">

       <title><indexterm>
         <primary>maintenance</primary>
         <secondary>adding an MDT</secondary>
       <title><indexterm>
         <primary>maintenance</primary>
         <secondary>adding an MDT</secondary>


@@ -356,7 +365,7 @@ Changing a Server NID</title>
         user or application workloads from other users of the filesystem. It
         is possible to have multiple remote sub-directories reference the
         same MDT.  However, the root directory will always be located on
         user or application workloads from other users of the filesystem. It
         is possible to have multiple remote sub-directories reference the
         same MDT.  However, the root directory will always be located on

-        MDT0. To add a new MDT into the file system:</para>
+        MDT0000. To add a new MDT into the file system:</para>

       <orderedlist>
         <listitem>
           <para>Discover the maximum MDT index. Each MDT must have unique index.</para>
       <orderedlist>
         <listitem>
           <para>Discover the maximum MDT index. Each MDT must have unique index.</para>


@@ -409,7 +418,7 @@ Adding a New OST to a Lustre File System</title>
         <listitem>
           <para> Add a new OST by using <literal>mkfs.lustre</literal> as when
          the filesystem was first formatted, see
         <listitem>
           <para> Add a new OST by using <literal>mkfs.lustre</literal> as when
          the filesystem was first formatted, see

-         <xref linkend="dbdoclet.format_ost" /> for details.  Each new OST
+         <xref linkend="format_ost" /> for details.  Each new OST

          must have a unique index number, use <literal>lctl dl</literal> to
          see a list of all OSTs. For example, to add a new OST at index 12
          to the <literal>testfs</literal> filesystem run following commands
          must have a unique index number, use <literal>lctl dl</literal> to
          see a list of all OSTs. For example, to add a new OST at index 12
          to the <literal>testfs</literal> filesystem run following commands


@@ -437,7 +446,7 @@ oss# mount -t lustre /dev/sda /mnt/testfs/ost12</screen>
          system on <literal>OST0004</literal> that are larger than 4GB in
          size to other OSTs, enter:</para>
           <screen>client# lfs find /test --ost test-OST0004 -size +4G | lfs_migrate -y</screen>
          system on <literal>OST0004</literal> that are larger than 4GB in
          size to other OSTs, enter:</para>
           <screen>client# lfs find /test --ost test-OST0004 -size +4G | lfs_migrate -y</screen>

-          <para>See <xref linkend="dbdoclet.lfs_migrate"/> for details.</para>
+          <para>See <xref linkend="lfs_migrate"/> for details.</para>

         </listitem>
       </orderedlist>
     </section>
         </listitem>
       </orderedlist>
     </section>


@@ -467,14 +476,14 @@ Removing and Restoring MDTs and OSTs</title>
           <para>A hard drive has failed and a RAID resync/rebuild is underway,
           though the OST can also be marked <emphasis>degraded</emphasis> by
           the RAID system to avoid allocating new files on the slow OST which
           <para>A hard drive has failed and a RAID resync/rebuild is underway,
           though the OST can also be marked <emphasis>degraded</emphasis> by
           the RAID system to avoid allocating new files on the slow OST which

-          can reduce performance, see <xref linkend='dbdoclet.degraded_ost' />
+          can reduce performance, see <xref linkend='degraded_ost' />

           for more details.
           </para>
         </listitem>
         <listitem>
           <para>OST is nearing its space capacity, though the MDS will already
           try to avoid allocating new files on overly-full OSTs if possible,
           for more details.
           </para>
         </listitem>
         <listitem>
           <para>OST is nearing its space capacity, though the MDS will already
           try to avoid allocating new files on overly-full OSTs if possible,

-          see <xref linkend='dbdoclet.balancing_free_space' /> for details.
+          see <xref linkend='balancing_free_space' /> for details.

           </para>
         </listitem>
         <listitem>
           </para>
         </listitem>
         <listitem>


@@ -483,7 +492,7 @@ Removing and Restoring MDTs and OSTs</title>
           desire to continue using the filesystem before it is repaired.</para>
         </listitem>
       </itemizedlist>
           desire to continue using the filesystem before it is repaired.</para>
         </listitem>
       </itemizedlist>

-      <section condition="l24" xml:id="lustremaint.rmremotedir">
+      <section xml:id="lustremaint.rmremotedir">

       <title><indexterm><primary>maintenance</primary><secondary>removing an MDT</secondary></indexterm>Removing an MDT from the File System</title>
         <para>If the MDT is permanently inaccessible,
         <literal>lfs rm_entry {directory}</literal> can be used to delete the
       <title><indexterm><primary>maintenance</primary><secondary>removing an MDT</secondary></indexterm>Removing an MDT from the File System</title>
         <para>If the MDT is permanently inaccessible,
         <literal>lfs rm_entry {directory}</literal> can be used to delete the


@@ -505,7 +514,7 @@ client$ lfs getstripe --mdt-index /mnt/lustre/local_dir0
         <para>The <literal>lfs getstripe --mdt-index</literal> command
         returns the index of the MDT that is serving the given directory.</para>
       </section>
         <para>The <literal>lfs getstripe --mdt-index</literal> command
         returns the index of the MDT that is serving the given directory.</para>
       </section>

-      <section xml:id="lustremaint.inactivemdt" condition='l24'>
+      <section xml:id="lustremaint.inactivemdt">

       <title>
           <indexterm><primary>maintenance</primary></indexterm>
           <indexterm><primary>maintenance</primary><secondary>inactive MDTs</secondary></indexterm>Working with Inactive MDTs</title>
       <title>
           <indexterm><primary>maintenance</primary></indexterm>
           <indexterm><primary>maintenance</primary><secondary>inactive MDTs</secondary></indexterm>Working with Inactive MDTs</title>


@@ -616,14 +625,45 @@ client$ lfs getstripe --mdt-index /mnt/lustre/local_dir0
             </listitem>
             <listitem>
               <para>If there is not expected to be a replacement for this OST in
             </listitem>
             <listitem>
               <para>If there is not expected to be a replacement for this OST in

-              the near future, permanently deactivate it on all clients and
-             the MDS by running the following command on the MGS:
-              <screen>mgs# lctl conf_param <replaceable>ost_name</replaceable>.osc.active=0</screen></para>
+                the near future, permanently deactivate it on all clients and
+               the MDS by running the following command on the MGS:
+                <screen>mgs# lctl conf_param <replaceable>ost_name</replaceable>.osc.active=0</screen></para>

               <note><para>A deactivated OST still appears in the file system
               <note><para>A deactivated OST still appears in the file system

-                configuration, though a replacement OST can be created using the
+                configuration, though a replacement OST can be created that
+                re-uses the same OST index with the

                 <literal>mkfs.lustre --replace</literal> option, see
                 <xref linkend="lustremaint.restore_ost"/>.
               </para></note>
                 <literal>mkfs.lustre --replace</literal> option, see
                 <xref linkend="lustremaint.restore_ost"/>.
               </para></note>

+              <para>To totally remove the OST from the filesystem configuration,
+                the OST configuration records should be found in the startup
+                logs by running the command
+                "<literal>lctl --device MGS llog_print <replaceable>fsname</replaceable>-client</literal>"
+                on the MGS (and also
+                "<literal>... <replaceable>$fsname</replaceable>-MDT<replaceable>xxxx</replaceable></literal>"
+                for all the MDTs) to list all <literal>attach</literal>,
+                <literal>setup</literal>, <literal>add_osc</literal>,
+                <literal>add_pool</literal>, and other records related to the
+                removed OST(s).  Once the <literal>index</literal> value is
+                known for each configuration record, the command
+                "<literal>lctl --device MGS llog_cancel <replaceable>llog_name</replaceable> -i <replaceable>index</replaceable> </literal>"
+                will drop that record from the configuration log
+                <replaceable>llog_name</replaceable> for each of the
+                <literal><replaceable>fsname</replaceable>-client</literal> and
+                <literal><replaceable>fsname</replaceable>-MDTxxxx</literal>
+                configuration logs so that new mounts will no longer process it.
+                If a whole OSS is being removed, the<literal>add_uuid</literal>
+                records for the OSS should similarly be canceled.
+                <screen>
+mgs# lctl --device MGS llog_print testfs-client | egrep "192.168.10.99@tcp|OST0003"
+- { index: 135, event: add_uuid, nid: 192.168.10.99@tcp(0x20000c0a80a63), node: 192.168.10.99@tcp }
+- { index: 136, event: attach, device: testfs-OST0003-osc, type: osc, UUID: testfs-clilov_UUID }
+- { index: 137, event: setup, device: testfs-OST0003-osc, UUID: testfs-OST0003_UUID, node: 192.168.10.99@tcp }
+- { index: 138, event: add_osc, device: testfs-clilov, ost: testfs-OST0003_UUID, index: 3, gen: 1 }
+mgs# lctl --device MGS llog_cancel testfs-client -i 138
+mgs# lctl --device MGS llog_cancel testfs-client -i 137
+mgs# lctl --device MGS llog_cancel testfs-client -i 136
+                </screen>
+              </para>

             </listitem>
           </orderedlist>
         </listitem>
             </listitem>
           </orderedlist>
         </listitem>


@@ -676,7 +716,7 @@ oss# mount -t ldiskfs <replaceable>/dev/ost_device</replaceable> /mnt/ost</scree
         </indexterm> Restoring OST Configuration Files</title>
       <para>If the original OST is still available, it is best to follow the
         OST backup and restore procedure given in either
         </indexterm> Restoring OST Configuration Files</title>
       <para>If the original OST is still available, it is best to follow the
         OST backup and restore procedure given in either

-        <xref linkend="dbdoclet.backup_device"/>, or
+        <xref linkend="backup_device"/>, or

         <xref linkend="backup_fs_level"/> and
         <xref linkend="backup_fs_level.restore"/>.</para>
       <para>To replace an OST that was removed from service due to corruption
         <xref linkend="backup_fs_level"/> and
         <xref linkend="backup_fs_level.restore"/>.</para>
       <para>To replace an OST that was removed from service due to corruption


@@ -718,7 +758,7 @@ oss# mount -t ldiskfs <replaceable>/dev/new_ost_dev</replaceable> <replaceable>/
         <listitem>
           <para>Recreate the OST configuration files, if unavailable. </para>
           <para>Follow the procedure in
         <listitem>
           <para>Recreate the OST configuration files, if unavailable. </para>
           <para>Follow the procedure in

-            <xref linkend="dbdoclet.repair_ost_lastid"/> to recreate the LAST_ID
+            <xref linkend="repair_ost_lastid"/> to recreate the LAST_ID

             file for this OST index. The <literal>last_rcvd</literal> file
             will be recreated when the OST is first mounted using the default
             parameters, which are normally correct for all file systems. The
             file for this OST index. The <literal>last_rcvd</literal> file
             will be recreated when the OST is first mounted using the default
             parameters, which are normally correct for all file systems. The


@@ -869,4 +909,26 @@ touch: cannot touch ‘test_file’: Read-only file system
 mds# lctl set_param mdt.fs-MDT0000.readonly=0
 mdt.fs-MDT0000.readonly=0</screen>
     </section>
 mds# lctl set_param mdt.fs-MDT0000.readonly=0
 mdt.fs-MDT0000.readonly=0</screen>
     </section>

+    <section xml:id="lustremaint.tunefallocate" condition="l2E">
+      <title><indexterm><primary>maintenance</primary>
+        <secondary>Tune fallocate</secondary></indexterm>
+        Tune Fallocate for ldiskfs</title>
+     <para>This section shows how to tune/enable/disable fallocate for
+     ldiskfs OSTs.</para>
+     <para>The default <literal>mode=0</literal> is the standard
+     "allocate unwritten extents" behavior used by ext4.  This is by far the
+     fastest for space allocation, but requires the unwritten extents to be
+     split and/or zeroed when they are overwritten.</para>
+     <para> The OST fallocate <literal>mode=1</literal> can also be set to use
+     "zeroed extents", which may be handled by "WRITE SAME", "TRIM zeroes data",
+     or other low-level functionality in the underlying block device.</para>
+     <para><literal>mode=-1</literal> completely disables fallocate.</para>
+     <para>Example: To completely disable fallocate</para>
+     <screen>lctl set_param osd-ldiskfs.*.fallocate_zero_blocks=-1</screen>
+     <para>Example: To enable fallocate to use 'zeroed extents'</para>
+     <screen>lctl set_param osd-ldiskfs.*.fallocate_zero_blocks=1</screen>
+     </section>

 </chapter>
 </chapter>

+<!--
+  vim:expandtab:shiftwidth=2:tabstop=8:
+  -->