<?xml version='1.0' encoding='utf-8'?>
<chapter xmlns="http://docbook.org/ns/docbook"
-xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
-xml:id="upgradinglustre">
+ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
+ xml:id="upgradinglustre">
<title xml:id="upgradinglustre.title">Upgrading a Lustre File System</title>
<para>This chapter describes interoperability between Lustre software
- releases. It also provides procedures for upgrading from Lustre software
- release 1.8 to Lustre software release 2.x , from a Lustre software release
- 2.x to a more recent Lustre software release 2.x (major release upgrade), and
- from a a Lustre software release 2.x.y to a more recent Lustre software
- release 2.x.y (minor release upgrade). It includes the following
- sections:</para>
+ releases. It also provides procedures for upgrading from older Lustre 2.x
+ software releases to a more recent 2.y Lustre release a (major release
+ upgrade), and from a Lustre software release 2.x.y to a more recent
+ Lustre software release 2.x.z (minor release upgrade). It includes the
+ following sections:</para>
<itemizedlist>
<listitem>
<para>
<listitem>
<para>All servers must be be upgraded to a Linux kernel supported by
the Lustre software. See the Lustre Release Notes for your Lustre
- version for a list of tested Linux distributions.</para>
+ version for a list of tested Linux distributions.</para>
</listitem>
<listitem>
<para>Clients to be upgraded must be running a compatible Linux
<secondary>multiple MDSs</secondary>
</indexterm>
<indexterm>
- <primary>large_xattr</primary>
- <secondary>ea_inode</secondary>
- </indexterm>
- <indexterm>
- <primary>wide striping</primary>
+ <primary>ea_inode</primary>
<secondary>large_xattr</secondary>
- <tertiary>ea_inode</tertiary>
</indexterm>Upgrading to Lustre Software Release 2.x (Major
Release)</title>
<para>The procedure for upgrading from a Lustre software release 2.x to a
- more recent 2.x release of the Lustre software is described in this
- section.</para>
- <note>
- <para>This procedure can also be used to upgrade Lustre software release
- 1.8.6-wc1 or later to any Lustre software release 2.x. To upgrade other
- versions of Lustre software release 1.8.x, contact your support
- provider.</para>
- </note>
- <note>
- <para>In Lustre software release 2.2, a feature has been added for
- ldiskfs-based MDTs that allows striping a single file across up to 2000
- OSTs. By default, this "wide striping" feature is disabled. It is
- activated by setting the <literal>ea_inode</literal> option on the MDT
- using either <literal>mkfs.lustre</literal> or <literal>tune2fs</literal>.
- For example after upgrading an existing file system to Lustre software
- release 2.2 or later, wide striping can be enabled by running the
- following command on the MDT device before mounting it:
- <screen>tune2fs -O large_xattr</screen>
- Once the wide striping feature is enabled and in use on the MDT, it is
- not possible to directly downgrade the MDT file system to an earlier
- version of the Lustre software that does not support wide striping. To
- disable wide striping:
- <orderedlist>
- <listitem>
- <para>Delete all wide-striped files, <emphasis>OR</emphasis>
- use <literal>lfs_migrate -c 160</literal> (or fewer stripes)
- to migrate the files to use fewer OSTs. This does not affect the
- total number of OSTs that the whole filesystem can access.</para>
- </listitem>
- <listitem>
- <para>Unmount the MDT.</para>
- </listitem>
- <listitem>
- <para>Run the following command to turn off the
- <literal>large_xattr</literal> option:
- <screen>tune2fs -O ^large_xattr</screen></para>
- </listitem>
- </orderedlist>Using either
- <literal>mkfs.lustre</literal> or
- <literal>tune2fs</literal> with
- <literal>large_xattr</literal> or
- <literal>ea_inode</literal> option reseults in
- <literal>ea_inode</literal> in the file system feature list.</para>
- </note>
- <note>
- <para>To generate a list of all files with more than 160 stripes use
- <literal>lfs find</literal> with the
- <literal>--stripe-count</literal> option:
- <screen>lfs find ${mountpoint} --stripe-count=+160</screen></para>
- </note>
- <note condition="l24">
- <para>In Lustre software release 2.4, a new feature allows using multiple
- MDTs, which can each serve one or more remote sub-directories in the file
- system. The
- <literal>root</literal> directory is always located on MDT0.</para>
- <para>Note that clients running a release prior to the Lustre software
- release 2.4 can only see the namespace hosted by MDT0 and will return an
- IO error if an attempt is made to access a directory on another
- MDT.</para>
- </note>
- <para>To upgrade a Lustre software release 2.x to a more recent major
- release, complete these steps:</para>
+ more recent 2.y major release of the Lustre software is described in this
+ section. To upgrade an existing 2.x installation to a more recent major
+ release, complete the following steps:</para>
<orderedlist>
<listitem>
<para>Create a complete, restorable file system backup.</para>
(tested) Linux distribution and reboot.</para>
</listitem>
<listitem>
- <para>Upgrade the Linux operating system on all clients to Red Hat
- Enterprise Linux 6 or other compatible (tested) distribution and
- reboot.</para>
+ <para>Upgrade the Linux operating system on all clients to a
+ compatible (tested) distribution and reboot.</para>
</listitem>
<listitem>
<para>Download the Lustre server RPMs for your platform from the
<link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
- Lustre Releases</link>repository. See
+ Lustre Releases</link> repository. See
<xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="table_cnh_5m3_gk" />for a list of required packages.</para>
+ linkend="table_cnh_5m3_gk" /> for a list of required packages.</para>
</listitem>
<listitem>
<para>Install the Lustre server packages on all Lustre servers (MGS,
<listitem>
<para>Download the Lustre client RPMs for your platform from the
<link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
- Lustre Releases</link>repository. See
+ Lustre Releases</link> repository. See
<xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="table_j3r_ym3_gk" />for a list of required packages.</para>
+ linkend="table_j3r_ym3_gk" /> for a list of required packages.</para>
<note>
<para>The version of the kernel running on a Lustre client must be
the same as the version of the
</orderedlist>
</listitem>
<listitem>
- <para>(Optional) For upgrades to Lustre software release 2.2 or higher,
- to enable wide striping on an existing MDT, run the following command
- on the MDT:
- <screen>tune2fs -O ea_inode /dev/<replaceable>mdtdev</replaceable></screen>
- </para>
- <para>For more information about wide striping, see
- <xref xmlns:xlink="http://www.w3.org/1999/xlink" linkend="wide_striping" />.</para>
- </listitem>
- <listitem>
- <para>(Optional) For upgrades to Lustre software release 2.4 or higher,
- to format an additional MDT, complete these steps:
+ <para>The DNE feature allows using multiple MDTs within a single
+ filesystem namespace, and each MDT can each serve one or more remote
+ sub-directories in the file system. The <literal>root</literal>
+ directory is always located on MDT0.</para>
+ <para>Note that clients running a release prior to the Lustre software
+ release 2.4 can only see the namespace hosted by MDT0 and will return an
+ IO error if an attempt is made to access a directory on another
+ MDT.</para>
+ <para>(Optional) To format an additional MDT, complete these steps:
<orderedlist numeration="loweralpha">
<listitem>
<para>Determine the index used for the first MDT (each MDT must
<para>In this example, the next available index is 1.</para>
</listitem>
<listitem>
- <para>Add the new block device as a new MDT at the next available
- index by entering (on one line):
+ <para>Format the new block device as a new MDT at the next
+ available MDT index by entering (on one line):
<screen>mds# mkfs.lustre --reformat --fsname=<replaceable>filesystem_name</replaceable> --mdt \
- --mgsnode=<replaceable>mgsnode</replaceable> --index <replaceable>1</replaceable>
+ --mgsnode=<replaceable>mgsnode</replaceable> --index <replaceable>new_mdt_index</replaceable>
<replaceable>/dev/mdt1_device</replaceable></screen></para>
</listitem>
</orderedlist></para>
</listitem>
<listitem>
- <para>(Optional) If you are upgrading to Lustre software release 2.3 or
- higher from Lustre software release 2.2 or earlier and want to enable
- the quota feature, complete these steps:
- <orderedlist numeration="loweralpha">
- <listitem>
- <para>Before setting up the file system, enter on both the MDS and
- OSTs:
- <screen>tunefs.lustre --quota</screen></para>
- </listitem>
- <listitem>
- <para>(Optional) If you are upgrading before Lustre software release
+ <para>(Optional) If you are upgrading from a release before Lustre
2.10, to enable the project quota feature enter the following on every
- ldiskfs backend target:
+ ldiskfs backend target while unmounted:
<screen>tune2fs –O project /dev/<replaceable>dev</replaceable></screen>
</para>
<note><para>Enabling the <literal>project</literal> feature will prevent
the filesystem from being used by older versions of ldiskfs, so it
- should only be enabled if the project quota feature is required and/or
- after it is known that the upgraded release does not need to be
- downgraded.</para></note>
+ should only be enabled if the project quota feature is required
+ and/or after it is known that the upgraded release does not need
+ to be downgraded.</para></note>
</listitem>
- <listitem>
-
<para>When setting up the file system, enter:
- <screen>conf_param $FSNAME.quota.mdt=$QUOTA_TYPE
+ <listitem>
+ <para>When setting up the file system, enter:
+ <screen>conf_param $FSNAME.quota.mdt=$QUOTA_TYPE
conf_param $FSNAME.quota.ost=$QUOTA_TYPE</screen></para>
- </listitem>
- </orderedlist></para>
</listitem>
<listitem>
- <para>(Optional) If you are upgrading from Lustre software release 1.8,
- you must manually enable the FID-in-dirent feature. On the MDS, enter:
- <screen>tune2fs –O dirdata /dev/<replaceable>mdtdev</replaceable></screen></para>
- <warning>
- <para>This step is not reversible. Do not complete this step until
- you are sure you will not be downgrading the Lustre software.</para>
- </warning>
- <para condition='l24'>This step only enables FID-in-dirent for newly
- created files. If you are upgrading to Lustre software release 2.4,
- you can use namespace LFSCK to enable FID-in-dirent for the existing
- files. For the case of upgrading from Lustre software release 1.8, it is
- important to note that if you do NOT enable <literal>dirdata</literal> via
- the <literal>tune2fs</literal> command above, the namespace LFSCK will NOT
- generate FID-in-dirent for the existing files. For more information about
- FID-in-dirent and related functionalities in LFSCK, see
- <xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="understandinglustre.storageio" />.</para>
+ <para condition="l2D">(Optional) If upgrading an ldiskfs MDT formatted
+ prior to Lustre 2.13, the "wide striping" feature that allows files
+ to have more than 160 stripes and store other large xattrs was not
+ enabled by default. This feature can be enabled on existing MDTs
+ by running the following command on all MDT devices:
+ <screen>mds# tune2fs -O ea_inode /dev/<replaceable>mdtdev</replaceable></screen>
+ </para>
+ <para>For more information about wide striping, see
+ <xref xmlns:xlink="https://www.w3.org/1999/xlink"
+ linkend="wide_striping" />.</para>
</listitem>
<listitem>
<para>Start the Lustre file system by starting the components in the
</listitem>
</orderedlist>
</listitem>
+ <listitem>
+ <para condition='l27'>(Optional) If you are upgrading from a release before Lustre
+ 2.7, to enable OST FIDs to also store the OST index (to improve
+ reliability of LFSCK and debug messages), <emphasis>after</emphasis>
+ the OSTs are mounted run once on each OSS:
+ <screen>oss# lctl set_param osd-ldiskfs.*.osd_index_in_idif=1</screen>
+ </para>
+ <note><para>Enabling the <literal>index_in_idif</literal> feature will
+ prevent the OST from being used by older versions of Lustre, so it
+ should only be enabled once it is known there is no need for the
+ OST to be downgraded to an earlier release.</para></note>
+ </listitem>
+ <listitem>
+ <para>If a new MDT was added to the filesystem, the new MDT must be
+ attached into the namespace by creating one or more
+ <emphasis>new</emphasis> DNE subdirectories with the
+ <literal>lfs mkdir</literal> command that use the new MDT:
+<screen>
+client# lfs mkdir -i <replaceable>new_mdt_index /testfs/new_dir</replaceable>
+</screen>
+ </para>
+ <para condition='l28'>In Lustre 2.8 and later, it is possible to
+ split a new directory across multiple MDTs by creating it with
+ multiple stripes:
+<screen>
+client# lfs mkdir -c 2 <replaceable>/testfs/new_striped_dir</replaceable>
+</screen>
+ </para>
+ <para condition='l2D'>In Lustre 2.13 and later, it is possible to set
+ the default striping on <emphasis>existing</emphasis> directories
+ so that new remote subdirectories are created on less-full MDTs:
+<screen>
+client# lfs setdirstripe -c 1 -i -1 <replaceable>/testfs/some_dir</replaceable>
+</screen>
+ </para>
+ </listitem>
</orderedlist>
<note>
<para>The mounting order described in the steps above must be followed
is MGT, OSTs, MDT(s), clients.</para>
</note>
<para>If you have a problem upgrading a Lustre file system, see
- <xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="dbdoclet.50438198_30989" />for some ways to get help.</para>
+ <xref xmlns:xlink="http://www.w3.org/1999/xlink"
+ linkend="dbdoclet.reporting_lustre_problem"/>for ways to get help.</para>
</section>
<section xml:id="Upgrading_2.x.x">
<title>
<listitem>
<para>Download the Lustre server RPMs for your platform from the
<link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
- Lustre Releases</link>repository. See
+ Lustre Releases</link> repository. See
<xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="table_cnh_5m3_gk" />for a list of required packages.</para>
+ linkend="table_cnh_5m3_gk" /> for a list of required packages.</para>
</listitem>
<listitem>
<para>For a rolling upgrade, complete any procedures required to keep
<listitem>
<para>Download the Lustre client RPMs for your platform from the
<link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
- Lustre Releases</link>repository. See
+ Lustre Releases</link> repository. See
<xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="table_j3r_ym3_gk" />for a list of required packages.</para>
+ linkend="table_j3r_ym3_gk" /> for a list of required packages.</para>
</listitem>
<listitem>
<para>Install the Lustre client packages on each of the Lustre clients
</orderedlist>
<para>If you have a problem upgrading a Lustre file system, see
<xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="dbdoclet.50438198_30989" />for some suggestions for how to get
- help.</para>
+ linkend="dbdoclet.reporting_lustre_problem" />for some suggestions for
+ how to get help.</para>
</section>
</chapter>
+<!--
+ vim:expandtab:shiftwidth=2:tabstop=8:
+ -->
git://git.whamcloud.com / doc / manual.git / blobdiff