</note>
</listitem>
<listitem xml:id="dbdoclet.addmdtindex">
- <para>Optional for Lustre software release 2.4 and later. Add in
- additional MDTs.</para>
+ <para>Optional for Lustre software release 2.4 and later.
+ Add in additional MDTs.</para>
<screen>
mkfs.lustre --fsname=
<replaceable>fsname</replaceable> --mgsnode=
new subdirectories at the time they are created.</para>
</glossdef>
</glossentry>
- <glossentry xml:id="DNE">
+ <glossentry xml:id="DNE" condition='l24'>
<glossterm>Distributed namespace (DNE)</glossterm>
<glossdef>
- <para>A collection of metadata targets implementing a single file
- system namespace.</para>
+ <para>A collection of metadata targets serving a single file
+ system namespace. Prior to DNE, Lustre file systems were limited to a
+ single metadata target for the entire name space. Without the ability
+ to distribute metadata load over multiple targets, Lustre file system
+ performance is limited. Lustre was enhanced with DNE functionality in
+ two development phases. After completing the first phase of development
+ in Lustre software version 2.4, <emphasis>Remote Directories</emphasis>
+ allowed the metadata for sub-directories to be serviced by an
+ independent MDT(s). After completing the second phase of development in
+ Lustre software version 2.8, <emphasis>Striped Directories</emphasis>
+ allowed files in a single directory to be serviced by multiple MDTs.
+ </para>
</glossdef>
</glossentry>
</glossdiv>
server restarts.</para>
</glossdef>
</glossentry>
+ <glossentry xml:id="remotedirectories" condition='l24'>
+ <glossterm>Remote directory</glossterm>
+ <glossdef>
+ <para>A remote directory describes a feature of
+ Lustre where metadata for files in a given directory may be
+ stored on a different MDT than the metadata for the parent
+ directory. Remote directories only became possible with the
+ advent of DNE Phase 1, which arrived in Lustre version
+ 2.4. Remote directories are available to system
+ administrators who wish to provide individual metadata
+ targets for individual workloads.</para>
+ </glossdef>
+ </glossentry>
<glossentry xml:id="replay">
<glossterm>Replay request</glossterm>
<glossdef>
</glossdiv>
<glossdiv>
<title>S</title>
- <glossentry xml:id="stride">
+ <glossentry xml:id="stripe">
<glossterm>Stripe</glossterm>
<glossdef>
<para>A contiguous, logical extent of a Lustre file written to a single
of a file visible to user applications.</para>
</glossdef>
</glossentry>
+ <glossentry xml:id="stripeddirectory" condition='l28'>
+ <glossterm>Striped Directory</glossterm>
+ <glossdef>
+ <para>A striped directory is a feature of Lustre
+ software where metadata for files in a given directory are
+ distributed evenly over multiple MDTs. Striped directories
+ are only available in Lustre software version 2.8 or later.
+ An administrator can use a striped directory to increase
+ metadata performance by distributing the metadata requests
+ in a single directory over two or more MDTs.</para>
+ </glossdef>
+ </glossentry>
<glossentry xml:id="stridesize">
<glossterm>Stripe size</glossterm>
<glossdef>
file.</para>
</glossdef>
</glossentry>
- <glossentry xml:id="stripingmetadata">
- <glossterm>Striping metadata</glossterm>
- <glossdef>
- <para>The extended attribute associated with a file that describes how
- its data is distributed over storage objects. See also
- <emphasis role="italic">Default stripe pattern</emphasis>.</para>
- </glossdef>
- </glossentry>
</glossdiv>
<glossdiv>
<title>T</title>
<title xml:id="lustreoperations.title">Lustre Operations</title>
<para>Once you have the Lustre file system up and running, you can use the
procedures in this section to perform these basic Lustre administration
- tasks:</para>
- <itemizedlist>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_42877" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_24122" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_84876" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_69255" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_57420" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_54138" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_88063" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.lfsmkdir" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_88980" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_41817" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_70905" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_16954" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_69998" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438194_30872" />
- </para>
- </listitem>
- </itemizedlist>
+ tasks.</para>
<section xml:id="dbdoclet.50438194_42877">
<title>
<indexterm>
unique MDTs. An administrator can allocate a sub-directory to a given MDT
using the command:</para>
<screen>
-client# lfs mkdir –i
-<replaceable>mdt_index</replaceable>
+client# lfs mkdir –i
+<replaceable>mdt_index</replaceable>
<replaceable>/mount_point/remote_dir</replaceable>
-
</screen>
- <para>This command will allocate the sub-directory
- <literal>remote_dir</literal> onto the MDT of index
+ <para>This command will allocate the sub-directory
+ <literal>remote_dir</literal> onto the MDT of index
<literal>mdtindex</literal>. For more information on adding additional MDTs
and
- <literal>mdtindex</literal> see
+ <literal>mdtindex</literal> see
<xref linkend='dbdoclet.addmdtindex' />.</para>
<warning>
<para>An administrator can allocate remote sub-directories to separate
will leave the namespace below it inaccessible. For this reason, by
default it is only possible to create remote sub-directories off MDT0. To
relax this restriction and enable remote sub-directories off any MDT, an
- administrator must issue the command
- <literal>lctl set_param mdd.*.enable_remote_dir=1</literal>.</para>
+ administrator must issue the command
+ <literal>lctl set_param mdt.*.enable_remote_dir=1</literal>.</para>
</warning>
+ <para condition='l28'>With Lustre software version 2.8, a new
+ tunable is available to allow users with a specific group ID to create
+ and delete remote and striped directories. This tunable is
+ <literal>enable_remote_dir_gid</literal>. For example, setting this
+ parameter to the 'wheel' or 'admin' group ID allows users with that GID
+ to create and delete remote and striped directories. Setting this
+ parameter to <literal>-1</literal> on MDT0 to permanently allow any
+ non-root users create and delete remote and striped directories. For
+ example:
+ <screen>lctl set_param -P mdt.*.enable_remote_dir_gid=-1</screen>
+ </para>
+ </section>
+ <section xml:id="dbdoclet.lfsmkdirdne2" condition='l28'>
+ <title>
+ <indexterm>
+ <primary>operations</primary>
+ <secondary>striped directory</secondary>
+ </indexterm>
+ <indexterm>
+ <primary>operations</primary>
+ <secondary>mkdir</secondary>
+ </indexterm>
+ <indexterm>
+ <primary>operations</primary>
+ <secondary>setdirstripe</secondary>
+ </indexterm>
+ <indexterm>
+ <primary>striping</primary>
+ <secondary>metadata</secondary>
+ </indexterm>Creating a directory striped across multiple MDTs</title>
+ <para>Lustre 2.8 enables individual files in a given directory to
+ record their metadata on separate MDTs (a <emphasis>striped
+ directory</emphasis>). The result of this is that metadata requests for
+ files in a striped directory are serviced by multiple MDTs and metadata
+ service load is distributed over all the MDTs that service a given
+ directory. By distributing metadata service load over multiple MDTs,
+ performance can be improved beyond the limit of single MDT
+ performance. Prior to the development of this feature all files in a
+ directory must record their metadata on a single MDT.</para>
+ <para>This command to stripe a directory over
+ <replaceable>mdt_count</replaceable> MDTs is:
+ </para>
+ <screen>
+client# lfs setdirstripe -c
+<replaceable>mdt_count</replaceable>
+<replaceable>/mount_point/new_directory</replaceable>
+</screen>
</section>
<section xml:id="dbdoclet.50438194_88980">
<title>
xml:id="managingfilesystemio">
<title xml:id="managingfilesystemio.title">Managing the File System and
I/O</title>
- <para>This chapter describes file striping and I/O options, and includes the
- following sections:</para>
- <itemizedlist>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_17536" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_75549" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_11204" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_80295" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_61024" />
- </para>
- </listitem>
- </itemizedlist>
<section xml:id="dbdoclet.50438211_17536">
<title>
<indexterm>
<secondary>migrating data</secondary>
</indexterm>
<indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>
+ <indexterm>
<primary>maintenance</primary>
<secondary>full OSTs</secondary>
</indexterm>Migrating Data within a File System</title>
+
+ <para condition='l28'>Lustre software version 2.8 includes a
+ feature to migrate metadata between MDTs. This migration can only be
+ performed on whole directories. To migrate the contents of
+ <literal>/lustre/testremote</literal> from the current MDT to
+ MDT index 0, the sequence of commands is as follows:</para>
+ <screen>$ cd /lustre
+lfs getdirstripe -M ./testremote <lineannotation>which MDT is dir on?</lineannotation>
+1
+$ for i in $(seq 3); do touch ./testremote/${i}.txt; done <lineannotation>create test files</lineannotation>
+$ for i in $(seq 3); do lfs getstripe -M ./testremote/${i}.txt; done <lineannotation>check files are on MDT 1</lineannotation>
+1
+1
+1
+$ lfs migrate -m 0 ./testremote <lineannotation>migrate testremote to MDT 0</lineannotation>
+$ lfs getdirstripe -M ./testremote <lineannotation>which MDT is dir on now?</lineannotation>
+0
+$ for i in $(seq 3); do lfs getstripe -M ./testremote/${i}.txt; done <lineannotation>check files are on MDT 0 too</lineannotation>
+0
+0
+0</screen>
+ <para>For more information, see <literal>man lfs</literal></para>
+ <warning><para>Currently, only whole directories can be migrated
+ between MDTs. During migration each file receives a new identifier
+ (FID). As a consequence, the file receives a new inode number. File
+ system tools (for example, backup and archiving tools) may behave
+ incorrectly with files that are unchanged except for a new inode number.
+ </para></warning>
<para>As stripes cannot be moved within the file system, data must be
migrated manually by copying and renaming the file, removing the original
file, and renaming the new file with the original file name. The simplest
- way to do this is to use the
- <literal>lfs_migrate</literal> command (see
+ way to do this is to use the
+ <literal>lfs_migrate</literal> command (see
<xref linkend="dbdoclet.50438206_42260" />). However, the steps for
migrating a file by hand are also shown here for reference.</para>
<orderedlist>
<listitem>
<para>Identify the file(s) to be moved.</para>
- <para>In the example below, output from the
- <literal>getstripe</literal> command indicates that the file
+ <para>In the example below, output from the
+ <literal>getstripe</literal> command indicates that the file
<literal>test_2</literal> is located entirely on OST2:</para>
<screen>
client# lfs getstripe /mnt/lustre/test_2
<listitem>
<para>For MDT failover, two MDSs can be configured to serve the same
MDT. Only one MDS node can serve an MDT at a time.</para>
- <para condition="l24">Lustresoftware release 2.4 allows multiple MDTs.
+ <para condition="l24">Lustre software release 2.4 allows multiple MDTs.
By placing two or more MDT partitions on storage shared by two MDSs,
one MDS can fail and the remaining MDS can begin serving the unserved
MDT. This is described as an active/active failover pair.</para>
machines share storage for two or more MDTs. After the failure of one
MDS, the remaining MDS begins serving the MDT(s) of the failed
MDS.</para>
+ <para condition="l28">Since Lustre software release 2.8,
+ multiple MDTs can be employed to share the inode records for files
+ contained in a single directory. A directory for which inode records
+ are distributed across multiple MDTs is known as a <emphasis>striped
+ directory</emphasis>. In the case of a Lustre filesystem the inode
+ records maybe also be referred to as the 'metadata' portion of the
+ file record.</para>
</listitem>
<listitem>
<para>
git://git.whamcloud.com / doc / manual.git / commitdiff