<para>See <xref linkend="dbdoclet.50438194_88063"/> for more details.</para>
</note>
</listitem>
+ <listitem xml:id="dbdoclet.addmdtindex" condition='l24'>
+ <para>Optional for Lustre 2.4 and later. Add in additional MDTs.</para>
+ <screen>mkfs.lustre --fsname=<<emphasis>fsname</emphasis>> --mgsnode=<nid> --mdt --index=1 <<emphasis>block device name</emphasis>></screen>
+ <note><para>Up to 4095 additional MDTs can be added.</para></note>
+ </listitem>
<listitem>
<para>Mount the combined MGS/MDT file system on the block device. On the MDS node, run:</para>
<screen>mount -t lustre <<emphasis>block device name</emphasis>> <<emphasis>mount point</emphasis>></screen>
<para>Metadata Target. A metadata device made available through the Lustre meta-data network protocol.</para>
</glossdef>
</glossentry>
+ <glossentry xml:id="mdt0" condition='l24'>
+ <glossterm>MDT0
+ </glossterm>
+ <glossdef>
+ <para>The metadata target for the file system root. Since Lustre 2.4, multiple metadata targets are possible in the same filesystem. MDT0 is the root of the filesystem which must be available for the filesystem to be accessible.</para>
+ </glossdef>
+ </glossentry>
<glossentry xml:id="metadatawriteback">
<glossterm>Metadata Write-back Cache
</glossterm>
<para><xref linkend="dbdoclet.50438199_31353"/></para>
</listitem>
<listitem>
+ <para><xref linkend="dbdoclet.addingamdt"/></para>
+ </listitem>
+ <listitem>
<para><xref linkend="dbdoclet.50438199_22527"/></para>
</listitem>
<listitem>
<para><xref linkend="dbdoclet.50438199_14978"/></para>
</listitem>
<listitem>
+ <para><xref linkend="dbdoclet.rmremotedir"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="dbdoclet.inactivemdt"/></para>
+ </listitem>
+ <listitem>
<para><xref linkend="dbdoclet.50438199_77819"/></para>
</listitem>
<listitem>
</orderedlist>
<para>After the <literal>writeconf</literal> command is run, the configuration logs are re-generated as servers restart, and server NIDs in the updated <literal>list_nids</literal> file are used.</para>
</section>
+ <section xml:id="dbdoclet.addingamdt" condition='l24'>
+ <title><indexterm><primary>maintenance</primary><secondary>adding an MDT</secondary></indexterm>Adding a new MDT to a Lustre file system</title>
+ <para>Additional MDTs can be added to serve one or more remote sub-directories within the filesystem. It is possible to have multiple remote sub-directories reference the same MDT. However, the root directory will always be located on MDT0. To add a new MDT into the file system:</para>
+ <orderedlist>
+ <listitem>
+ <para>Discover the maximum MDT index. Each MDTs must have unique index.</para>
+ <screen>
+client$ lctl dl | grep mdc
+36 UP mdc lustre-MDT0000-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
+37 UP mdc lustre-MDT0001-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
+38 UP mdc lustre-MDT0002-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
+39 UP mdc lustre-MDT0003-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
+ </screen>
+ </listitem>
+ <listitem>
+ <para>Add the new block device as a new MDT at the next available index. In this example, the next available index is 4.</para>
+ <screen>
+mkfs.lustre --reformat --fsname=<filesystemname> --mdt --mgsnode=<mgsnode> --index 4 <blockdevice>
+ </screen>
+ </listitem>
+ <listitem>
+ <para>Mount the MDTs.</para>
+ <screen>
+mount –t lustre <blockdevice> /mnt/mdt4
+ </screen>
+ </listitem>
+ </orderedlist>
+ </section>
<section xml:id="dbdoclet.50438199_22527">
<title><indexterm><primary>maintenance</primary><secondary>adding a OST</secondary></indexterm>
Adding a New OST to a Lustre File System</title>
<para>OST is nearing its space capacity</para>
</listitem>
</itemizedlist>
+ <section condition="l24" xml:id="dbdoclet.rmremotedir">
+ <title><indexterm><primary>maintenance</primary><secondary>removing a MDT</secondary></indexterm>Removing a MDT from the File System</title>
+ <para>If the MDT is permanently inaccessible, <literal>lfs rmdir {directory}</literal> can be used to delete the directory entry. A normal <literal>rmdir</literal> will report an IO error due to the remote MDT being inactive. After the remote directory has been removed, the administrator should mark the MDT as permanently inactive with:</para>
+<screen>lctl conf_param {MDT name}.mdc.active=0
+</screen>
+<para>A user can identify the location of a remote sub-directory using the <literal>lfs</literal> utility. For example:</para>
+<screen>client$ lfs getstripe -M /mnt/lustre/remote_dir1
+1
+client$ mkdir /mnt/lustre/local_dir0
+client$ lfs getstripe -M /mnt/lustre/local_dir0
+0
+</screen>
+ <para>The <literal>getstripe -M</literal> parameters return the index of the MDT that is serving the given directory.</para>
+ </section>
+ <section xml:id="dbdoclet.inactivemdt" condition='l24'>
+ <title>
+ <indexterm><primary>maintenance</primary></indexterm>
+ <indexterm><primary>maintenance</primary><secondary>inactive MDTs</secondary></indexterm>Working with Inactive MDTs</title>
+ <para>Files located on or below an inactive MDT are inaccessible until the MDT is activated again. Clients accessing an inactive MDT will receive an EIO error.</para></section>
<section remap="h3">
<title><indexterm><primary>maintenance</primary><secondary>removing a OST</secondary></indexterm>
Removing an OST from the File System</title>
</listitem>
<listitem>
<para>Temporarily deactivate the OSC on the MDT. On the MDT, run: </para>
- <screen>$ mdt> lctl --device >devno< deactivate</screen>
+ <screen>$ mdt> lctl --device <devno> deactivate</screen>
<para>For example, based on the command output in Step 1, to deactivate device 13 (the MDT’s OSC for <literal>OST-0000</literal>), the command would be:
</para>
<screen>$ mdt> lctl --device 13 deactivate</screen>
<para>This setting is only temporary and will be reset if the clients or MDS are rebooted. It needs to be run on all clients.</para>
</note></para>
</listitem>
- <listitem>
- <para>To permanently disable the deactivated OST, enter: <screen>[mgs]# lctl conf_param {OST name}.osc.active=0</screen></para>
- </listitem>
</orderedlist>
- <para>If there is not expected to be a replacement for this OST in the near future, permanently deactivate the OST on all clients and the MDS: </para>
+ <para>If there is not expected to be a replacement for this OST in the near future, permanently deactivate the OST on all clients and the MDS: <screen>[mgs]# lctl conf_param {OST name}.osc.active=0</screen></para>
<note>
<para>A removed OST still appears in the file system; do not create a new OST with the same name.</para>
</note>
<para><xref linkend="dbdoclet.50438194_88063"/></para>
</listitem>
<listitem>
+ <para><xref linkend="dbdoclet.lfsmkdir"/></para>
+ </listitem>
+ <listitem>
<para><xref linkend="dbdoclet.50438194_88980"/></para>
</listitem>
<listitem>
<para>To mount a client on file system bar at mount point <literal>/mnt/bar</literal>, run:</para>
<screen>mount -t lustre mgsnode@tcp0:/bar /mnt/bar</screen>
</section>
+ <section xml:id="dbdoclet.lfsmkdir" condition='l24'>
+ <title><indexterm><primary>operations</primary><secondary>remote directory</secondary></indexterm>Creating a sub-directory on a given MDT</title>
+ <para>Lustre 2.4 enables individual sub-directories to be serviced by unique MDTs. An administrator can allocate a sub-directory to a given MDT using the command:</para>
+ <screen>lfs mkdir –i <mdtindex> <remote_dir>
+ </screen>
+ <para>This command will allocate the sub-directory <literal>remote_dir</literal> onto the MDT of index <literal>mdtindex</literal>. For more information on adding additional MDTs and <literal>mdtindex</literal> see <xref linkend='dbdoclet.addmdtindex'/>.</para>
+ <warning><para>An administrator can allocate remote sub-directories to separate MDTs. Creating remote sub-directories in parent directories not hosted on MDT0 is not recommended. This is because the failure of the parent MDT will leave the namespace below it inaccessible. For this reason, by default it is only possible to create remote sub-directories off MDT0. To relax this restriction and enable remote sub-directories off any MDT, an administrator must issue the command <literal>lctl set_param mdd.*.enable_remote_dir=1</literal>.</para></warning>
+ </section>
<section xml:id="dbdoclet.50438194_88980">
<title><indexterm><primary>operations</primary><secondary>parameters</secondary></indexterm>Setting and Retrieving Lustre Parameters</title>
<para>Several options are available for setting parameters in Lustre:</para>
</section>
<section xml:id="dbdoclet.50438194_69998">
<title><indexterm><primary>operations</primary><secondary>replacing an OST or MDS</secondary></indexterm>Replacing an Existing OST or MDT</title>
- <para>To copy the contents of an existing OST to a new OST (or an old
- MDT to a new MDT), follow the process for either OST/MDT backups in
+ <para>To copy the contents of an existing OST to a new OST (or an old MDT to a new MDT), follow the process for either OST/MDT backups in
<xref linkend='dbdoclet.50438207_71633'/> or
- <xref linkend='dbdoclet.50438207_21638'/>.</para>
+ <xref linkend='dbdoclet.50438207_21638'/>. For more information on removing a MDT, see <xref linkend='dbdoclet.rmremotedir'/>.</para>
</section>
<section xml:id="dbdoclet.50438194_30872">
<title><indexterm><primary>operations</primary><secondary>identifying OSTs</secondary></indexterm>Identifying To Which Lustre File an OST Object Belongs</title>
<para>When <xref linkend="imperativerecovery"/> is enabled, clients are notified of an MDS restart (either the backup or a restored primary). Clients always may detect an MDS failure either by timeouts of in-flight requests or idle-time ping messages. In either case the clients then connect to the new backup MDS and use the Metadata Replay protocol. Metadata Replay is responsible for ensuring that the backup MDS re-acquires state resulting from transactions whose effects were made visible to clients, but which were not committed to the disk.</para>
<para>The reconnection to a new (or restarted) MDS is managed by the file system configuration loaded by the client when the file system is first mounted. If a failover MDS has been configured (using the <literal>--failnode=</literal> option to <literal>mkfs.lustre</literal> or <literal>tunefs.lustre</literal>), the client tries to reconnect to both the primary and backup MDS until one of them responds that the failed MDT is again available. At that point, the client begins recovery. For more information, see <xref linkend="metadatereplay"/>.</para>
<para>Transaction numbers are used to ensure that operations are replayed in the order they were originally performed, so that they are guaranteed to succeed and present the same filesystem state as before the failure. In addition, clients inform the new server of their existing lock state (including locks that have not yet been granted). All metadata and lock replay must complete before new, non-recovery operations are permitted. In addition, only clients that were connected at the time of MDS failure are permitted to reconnect during the recovery window, to avoid the introduction of state changes that might conflict with what is being replayed by previously-connected clients.</para>
+ <para condition='l24'>Lustre 2.4 introduces multiple metadata targets. If multiple metadata targets are in use, active-active failover is possible. See <xref linkend='dbdoclet.mdtactiveactive'/> for more information.</para>
</section>
<section remap="h3">
<title><indexterm><primary>recovery</primary><secondary>OST failure</secondary></indexterm>OST Failure (Failover)</title>
<note><para>Default values for the thread counts are automatically selected. The values are chosen to best exploit the number of CPUs present in the system and to provide best overall performance for typical workloads.</para></note>
</section>
</section>
- <section xml:id="dbdoclet.mdsbinding">
+ <section xml:id="dbdoclet.mdsbinding" condition='l24'>
<title><indexterm><primary>tuning</primary><secondary>MDS binding</secondary></indexterm>Binding MDS Service Thread to CPU Partitions</title>
<para>With the introduction of Node Affinity (<xref linkend="nodeaffdef"/>) in Lustre 2.3, MDS threads can be bound to particular CPU Partitions (CPTs). Default values for bindings are selected automatically to provide good overall performance for a given CPU count. However, an administrator can deviate from these setting if they choose.</para>
<itemizedlist>
<para>Typical output is:</para>
<screen>osc.lustre-OST0002-osc.ost_conn_uuid=192.168.20.1@tcp</screen>
</section>
+ <section condition='l24'>
+ <title><indexterm><primary>striping</primary><secondary>remote directories</secondary></indexterm>Locating the MDT for a remote directory</title>
+ <para>Lustre 2.4 can be configured with multiple MDTs in the same filesystem. Each sub-directory can have a different MDT. To identify which MDT a given subdirectory is located on, pass the <literal>getstripe -M</literal> parameters to <literal>lfs</literal>. An example of this command is provided in the section <xref linkend='dbdoclet.rmremotedir'/>.</para>
+ </section>
</section>
<section xml:id="dbdoclet.50438209_10424">
<title><indexterm><primary>space</primary><secondary>free space</secondary></indexterm>Managing Free Space</title>
<para>For maximum performance, the MDT should be configured as RAID1 with an internal journal and two disks from different controllers.</para>
<para>If you need a larger MDT, create multiple RAID1 devices from pairs of disks, and then make a RAID0 array of the RAID1 devices. This ensures maximum reliability because multiple disk failures only have a small chance of hitting both disks in the same RAID1 device.</para>
<para>Doing the opposite (RAID1 of a pair of RAID0 devices) has a 50% chance that even two disk failures can cause the loss of the whole MDT device. The first failure disables an entire half of the mirror and the second failure has a 50% chance of disabling the remaining mirror.</para>
+ <para condition='l24'>If multiple MDTs are going to be present in the system, each MDT should be specified for the anticipated usage and load.</para>
+ <warning condition='l24'><para>MDT0 contains the root of the Lustre filesystem. If MDT0 is unavailable for any reason, the filesystem cannot be used.</para></warning>
+ <note condition='l24'><para>Additional MDTs can be dedicated to sub-directories off the root filesystem provided by MDT0. Subsequent directories may also be configured to have their own MDT. If an MDT serving a subdirectory becomes unavailable this subdirectory and all directories beneath it will also become unavailable. Configuring multiple levels of MDTs is an experimental feature for Lustre 2.4.</para></note>
</section>
<section remap="h3">
<title><indexterm><primary>setup</primary><secondary>OST</secondary></indexterm>OST Storage Hardware Considerations</title>
</entry>
<entry>
<para> 1</para>
+ <para condition='l24'>4096</para>
</entry>
<entry>
- <para>Maximum of 1 MDT per file system, but a single MDS can host multiple MDTs, each one for a separate file system.</para>
+ <para>Lustre 2.3 and earlier allows a maximum of 1 MDT per file system, but a single MDS can host multiple MDTs, each one for a separate file system.</para>
+ <para condition='l24'>Lustre 2.4 and later requires one MDT for the root. Upto 4095 additional MDTs can be added to the filesystem and attached into the namespace with remote directories.</para>
</entry>
</row>
<row>
</entry>
<entry>
<para> 4 billion</para>
+ <para condition='l24'>4096 * 4 billion</para>
</entry>
<entry>
- <para>The ldiskfs file system imposes an upper limit of 4 billion inodes. By default, the MDS file system is formatted with 4 KB of space per inode, meaning 512 million inodes per file system of 2 TB.</para>
+ <para>The ldiskfs file system imposes an upper limit of 4 billion inodes. By default, the MDS file system is formatted with 2KB of space per inode, meaning 1 billion inodes per file system of 2 TB.</para>
<para>This can be increased initially, at the time of MDS file system creation. For more information, see <xref linkend="settinguplustresystem"/>.</para>
+ <para condition='l24'>Each additional MDT can hold up to 4 billion additional files, depending on available inodes and the distribution directories and files in the filesystem.</para>
</entry>
</row>
<row>
<para><emphasis role="bold">Active/active</emphasis> pair - In this configuration, both nodes are active, each providing a subset of resources. In case of a failure, the second node takes over resources from the failed node.</para>
</listitem>
</itemizedlist>
- <para>Typically, Lustre MDSs are configured as an active/passive pair, while OSSs are deployed in an active/active configuration that provides redundancy without extra overhead. Often the standby MDS is the active MDS for another Lustre file system or the MGS, so no nodes are idle in the cluster.</para>
+ <para>Before Lustre 2.4 MDSs are configured as an active/passive pair, while OSSs are deployed in an active/active configuration that provides redundancy without extra overhead. Often the standby MDS is the active MDS for another Lustre file system or the MGS, so no nodes are idle in the cluster.</para>
+ <para condition='l24'>Lustre 2.4 introduces metadata targets for individual sub-directories. Active-active failover configurations are available for MDSs that serve MDTs on shared storage.</para>
</section>
</section>
<section xml:id="dbdoclet.50540653_97944">
<itemizedlist>
<listitem>
<para>For MDT failover, two MDSs are configured to serve the same MDT. Only one MDS node can serve an MDT at a time.</para>
+ <para condition='l24'>Lustre 2.4 allows multiple MDTs. By placing two or more MDT partitions on storage shared by two MDSs, one MDS can fail and the remaining MDS can begin serving the unserved MDT. This is described as an active/active failover pair.</para>
</listitem>
<listitem>
<para>For OST failover, multiple OSS nodes are configured to be able to serve the same OST. However, only one OSS node can serve the OST at a time. An OST can be moved between OSS nodes that have access to the same storage device using <literal>umount/mount</literal> commands.</para>
<para>In an environment with multiple file systems, the MDSs can be configured in a quasi active/active configuration, with each MDS managing metadata for a subset of the Lustre file system.</para>
</note>
<figure>
- <title xml:id="understandingfailover.fig.configmdt"> Lustre failover configuration for an MDT </title>
+ <title xml:id="understandingfailover.fig.configmdt"> Lustre failover configuration for a active/passive MDT</title>
<mediaobject>
<imageobject>
<imagedata fileref="./figures/MDT_Failover.png"/>
</mediaobject>
</figure>
</section>
+ <section xml:id='dbdoclet.mdtactiveactive' condition='l24'>
+ <title><indexterm><primary>failover</primary><secondary>MDT</secondary></indexterm>MDT Failover Configuration (Active/Active)</title>
+ <para>Multiple MDTs became available with the advent of Lustre 2.4. MDTs can be setup as an active/active failover configuration. A failover cluster is built from two MDSs as shown in <xref linkend="understandingfailover.fig.configmdts"/>.</para>
+ <figure>
+ <title xml:id="understandingfailover.fig.configmdts"> Lustre failover configuration for a active/active MDTs </title>
+ <mediaobject>
+ <imageobject>
+ <imagedata scalefit="1" width="50%" fileref="./figures/MDTs_Failover.svg"/>
+ </imageobject>
+ <textobject>
+ <phrase>Lustre failover configuration for two MDTs</phrase>
+ </textobject>
+ </mediaobject>
+ </figure>
+ </section>
<section remap="h3">
<title><indexterm><primary>failover</primary><secondary>OST</secondary></indexterm>OST Failover Configuration (Active/Active)</title>
<para>OSTs are usually configured in a load-balanced, active/active failover configuration. A failover cluster is built from two OSSs as shown in <xref linkend="understandingfailover.fig.configost"/>.</para>
<para> 4 billion files</para>
<para> <emphasis>MDS count:</emphasis></para>
<para> 1 primary + 1 backup</para>
+ <para condition='l24'>Since Lustre 2.4: up to 4096 MDSs and up to 4096 MDTs.</para>
</entry>
<entry>
<para> <emphasis>Single MDS:</emphasis></para>
<para><emphasis role="bold">High-performance heterogeneous networking:</emphasis> Lustre supports a variety of high performance, low latency networks and permits Remote Direct Memory Access (RDMA) for Infiniband (OFED) and other advanced networks for fast and efficient network transport. Multiple RDMA networks can be bridged using Lustre routing for maximum performance. Lustre also provides integrated network diagnostics.</para>
</listitem>
<listitem>
- <para><emphasis role="bold">High-availability:</emphasis> Lustre offers active/active failover using shared storage partitions for OSS targets (OSTs) and active/passive failover using a shared storage partition for the MDS target (MDT). This allows application transparent recovery. Lustre can work with a variety of high availability (HA) managers to allow automated failover and has no single point of failure (NSPF). Multiple mount protection (MMP) provides integrated protection from errors in highly-available systems that would otherwise cause file system corruption.</para>
+ <para><emphasis role="bold">High-availability:</emphasis> Lustre offers active/active failover using shared storage partitions for OSS targets (OSTs). Lustre 2.3 and earlier offers active/passive failover using a shared storage partition for the MDS target (MDT).</para><para condition='l24'>With Lustre 2.4 or later servers and clients it is possible to configure active/active failover of multiple MDTs</para><para>This allows application transparent recovery. Lustre can work with a variety of high availability (HA) managers to allow automated failover and has no single point of failure (NSPF). Multiple mount protection (MMP) provides integrated protection from errors in highly-available systems that would otherwise cause file system corruption.</para>
</listitem>
<listitem>
<para><emphasis role="bold">Security:</emphasis> By default TCP connections are only allowed from privileged ports. Unix group membership is verified on the MDS.</para>
<para><emphasis role="bold">Metadata Server (MDS)</emphasis> - The MDS makes metadata stored in one or more MDTs available to Lustre clients. Each MDS manages the names and directories in the Lustre file system(s) and provides network request handling for one or more local MDTs.</para>
</listitem>
<listitem>
- <para><emphasis role="bold">Metadata Target (MDT</emphasis> ) - The MDT stores metadata (such as filenames, directories, permissions and file layout) on storage attached to an MDS. Each file system has one MDT. An MDT on a shared storage target can be available to multiple MDSs, although only one can access it at a time. If an active MDS fails, a standby MDS can serve the MDT and make it available to clients. This is referred to as MDS failover.</para>
+ <para><emphasis role="bold">Metadata Target (MDT</emphasis> ) - For Lustre 2.3 and earlier, each filesystem has one MDT. The MDT stores metadata (such as filenames, directories, permissions and file layout) on storage attached to an MDS. Each file system has one MDT. An MDT on a shared storage target can be available to multiple MDSs, although only one can access it at a time. If an active MDS fails, a standby MDS can serve the MDT and make it available to clients. This is referred to as MDS failover.</para>
+ <para condition='l24'>Since Lustre 2.4, multiple MDTs are supported. Each filesystem has at least one MDT. An MDT on shared storage target can be available via multiple MDSs, although only one MDS can export the MDT to the clients at one time. Two MDS machines share storage for two or more MDTs. After the failure of one MDS, the remaining MDS begins serving the MDT(s) of the failed MDS.</para>
</listitem>
<listitem>
<para><emphasis role="bold">Object Storage Servers (OSS)</emphasis> : The OSS provides file I/O service, and network request handling for one or more local OSTs. Typically, an OSS serves between 2 and 8 OSTs, up to 16 TB each. A typical configuration is an MDT on a dedicated node, two or more OSTs on each OSS node, and a client on each of a large number of compute nodes.</para>
<listitem>
<para><xref linkend="dbdoclet.50438205_51369"/>Upgrading Lustre 1.8 to 2.x</para>
</listitem>
+ <listitem>
+ <para><xref linkend="dbdoclet.upgradetodne"/>Upgrading to multiple metadata targets</para>
+ </listitem>
</itemizedlist>
<section xml:id="dbdoclet.50438205_82079">
<title><indexterm><primary>Lustre</primary><secondary>upgrading</secondary><see>upgrading</see></indexterm>
<note>
<para>Lustre 2.x servers are compatible with clients 1.8.6 and later, though it is strongly recommended that the clients are upgraded to the latest version of Lustre 1.8 available. If you are planning a heterogeneous environment (mixed 1.8 and 2.x servers), make sure that version 1.8.6 or later is installed on the client nodes that are not upgraded to 2.x.</para>
</note>
+ <warning condition='l24'><para>Lustre 2.4 allows remote sub-directories to be hosted on separate MDTs. Clients prior to 2.4 can only see the namespace hosted by MDT0, and will return an IO error if a directory on a remote MDT is accessed.</para></warning>
</section>
<section xml:id="dbdoclet.50438205_51369">
<title><indexterm><primary>upgrading</primary><secondary>1.8 to 2.x</secondary></indexterm>Upgrading Lustre 1.8 to 2.x</title>
<para>If you have a problem upgrading Lustre, use the <link xl:href="https://groups.google.com/a/whamcloud.com/group/wc-discuss/">wc-discuss</link> mailing list, or file a ticket at the <link xl:href="https://bugs.whamcloud.com">Intel Lustre</link> bug tracker.</para>
</section>
</section>
+ <section xml:id='dbdoclet.upgradetodne' condition='l24'>
+ <title><indexterm><primary>upgrading</primary><secondary>multiple metadata targets</secondary></indexterm>Upgrading to multiple metadata targets</title>
+ <para>Lustre 2.4 allows separate metadata servers to serve separate sub directories. To upgrade a filesystem to Lustre 2.4 that support multiple metadata servers:</para>
+ <orderedlist>
+ <listitem>
+ <para>Stop MGT/MDT/OST and upgrade to 2.4</para>
+ </listitem>
+ <listitem>
+ <para>Format new MDT according to <xref linkend="dbdoclet.addingamdt"/>.</para>
+ </listitem>
+ <listitem>
+ <para>Mount all of the targets according to <xref linkend="dbdoclet.addingamdt"/>.</para>
+ </listitem>
+ <listitem>
+ <para>After recovery is completed clients will be connected MDT0.</para>
+ <note><para>Clients prior to 2.4 will only be have the namespace provided by MDT0 visible and will return an IO error if a directory hosted on a remote MDT is accessed.</para></note>
+ </listitem>
+ </orderedlist>
+ </section>
</chapter>
lfs getstripe [--obd|-O <uuid>] [--quiet|-q] [--verbose|-v]
[--count|-c] [--index|-i | --offset|-o]
[--size|-s] [--pool|-p] [--directory|-d]
- [--recursive|-r] [--raw|-R] <dirname|filename> ...
+ [--recursive|-r] [--raw|-R] [-M] <dirname|filename> ...
lfs setstripe [--size|-s stripe_size] [--count|-c stripe_cnt]
[--index|-i|--offset|-o start_ost_index]
[--pool|-p <pool>]
<para>Lists striping information for a given filename or directory. By default, the stripe count, stripe size and offset are returned.</para>
<para>If you only want specific striping information, then the options of <literal>--count</literal>,<literal>--size</literal>,<literal>--index</literal> or <literal>--offset</literal> plus various combinations of these options can be used to retrieve specific information.</para>
<para>If the <literal>--raw</literal> option is specified, the stripe information is printed without substituting the filesystem's default values for unspecified fields. If the striping EA is not set, 0, 0, and -1 will be printed for the stripe count, size, and offset respectively.</para>
+ <para condition='l24'>The <literal>-M</literal> prints the index of the MDT for a given directory. See <xref linkend='dbdoclet.rmremotedir'/>.</para>
</entry>
</row>
<row>
git://git.whamcloud.com / doc / manual.git / commitdiff