1 <?xml version='1.0' encoding='UTF-8'?><chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="upgradinglustre">
2 <title xml:id="upgradinglustre.title">Upgrading a Lustre* File System</title>
3 <para>This chapter describes interoperability between Lustre* software releases. It also provides
4 procedures for upgrading from a Lustre 1.8 release to a Lustre 2.x release, from a Lustre 2.x
5 release to a more recent Lustre 2.x release (major release upgrade), and from a Lustre 2.X.y
6 release to 2.X.y (minor release upgrade). It includes the following sections:</para>
9 <para><xref linkend="dbdoclet.50438205_82079"/></para>
12 <para><xref xmlns:xlink="http://www.w3.org/1999/xlink" linkend="Upgrading_2.x"/></para>
15 <para><xref xmlns:xlink="http://www.w3.org/1999/xlink" linkend="Upgrading_2.x.x"/></para>
18 <section xml:id="dbdoclet.50438205_82079">
20 <primary>Lustre</primary>
21 <secondary>upgrading</secondary>
23 </indexterm><indexterm>
24 <primary>upgrading</primary>
25 </indexterm>Release Interoperability and Upgrade Requirements</title>
26 <para><emphasis role="italic"><emphasis role="bold">Lustre 2.x release (major)
27 upgrade:</emphasis></emphasis><itemizedlist>
29 <para>All servers must be upgraded at the same time, while some or all clients may be
33 <para>All servers must be be upgraded to Red Hat* Enterprise Linux* 6 (RHEL 6) or other
34 compatible Linux distribution. See the Linux Test Matrix at <xref
35 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="LustreTestMatrixTable"/> for a
36 list of tested Linux distros.</para>
39 <para>Clients to be upgraded to the Lustre 2.4 release or higher must be running RHEL 6 or
40 other compatible Linux distribution. See the Linux Test Matrix at <xref
41 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="LustreTestMatrixTable"/> for a
42 list of tested Linux distros.</para>
44 </itemizedlist></para>
45 <para><emphasis role="italic"><emphasis role="bold">Lustre 2.X.y release (minor)
46 upgrade:</emphasis></emphasis></para>
49 <para>All servers must be upgraded at the same time, while some or all clients may be
53 <para>Rolling upgrades are supported for minor releases allowing individual servers and
54 clients to be upgraded without stopping the Lustre file system.</para>
58 <section xml:id="Upgrading_2.x">
60 <primary>upgrading</primary>
61 <secondary>major release (2.x to 2.x)</secondary>
62 </indexterm><indexterm>
63 <primary>wide striping</primary>
64 </indexterm><indexterm>
65 <primary>MDT</primary>
66 <secondary>multiple MDSx</secondary>
67 </indexterm>Upgrading to Lustre Release 2.x (Major Release)</title>
68 <para> The procedure for upgrading a Lustre release 2.x file system to a more recent 2.x release
69 of the Lustre software is described in this section. </para>
71 <para>This procedure can also be used to upgrade Lustre release 1.8.6-wc1 or later to any
72 Lustre release 2.x. To upgrade other versions of 1.8.x, contact your support
76 <para condition="l22">In Lustre release 2.2, a feature has been added that allows striping
77 across up to 2000 OSTs. By default, this "wide striping" feature is disabled. It is
78 activated by setting the <literal>large-xattr</literal> option on the MDT using either
79 <literal>mkfs.lustre</literal> or <literal>tune2fs</literal>. For example after upgrading
80 an existing file system to Lustre release 2.2 or later, wide striping can be enabled by
81 running the following command on the MDT device before mounting
82 it:<screen>tune2fs -O large_xattr</screen>Once the wide striping feature is enabled and in
83 use on the MDT, it is not possible to directly downgrade the MDT file system to an earlier
84 version of the Lustre software that does not support wide striping. The only way to disable
85 wide striping is to delete <emphasis role="underline">all files with an extended attribute
86 (<literal>xattr</literal></emphasis>), unmount the MDT, and run the following command to
87 turn off the <literal>large-xattr</literal>
88 option:<screen>tune2fs -O ^large_xattr</screen></para>
90 <note condition="l24">
91 <para>In Lustre release 2.4, a new feature allows using multiple MDTs, which can each serve
92 one or more remote sub-directories in the file system. The <literal>root</literal> directory
93 is always located on MDT0. </para>
94 <para>Note that clients running a release prior to the Lustre 2.4 release can only see the
95 namespace hosted by MDT0 and will return an IO error if an attempt is made to access a
96 directory on another MDT.</para>
98 <para>To upgrade a Lustre release 2.x file system to a more recent major release, complete these
102 <para>Create a complete, restorable file system backup. </para>
104 <para>Before installing the Lustre software, back up ALL data. The Lustre software
105 contains kernel modifications that interact with storage devices and may introduce
106 security issues and data loss if not installed, configured, or administered properly. If
107 a full backup of the file system is not practical, a device-level backup of the MDT file
108 system is recommended. See <xref linkend="backupandrestore"/> for a procedure.</para>
112 <para>Shut down the file system by unmounting all clients and servers in the order shown
113 below (unmounting a block device causes the Lustre software to be shut down on that
115 <orderedlist numeration="loweralpha">
117 <para>Unmount the clients. On each client node, run:</para>
118 <screen>umount -a -t lustre</screen>
121 <para>Unmount the MDT. On the MDS node, run:</para>
122 <screen>umount -a -t lustre</screen>
125 <para>Unmount all the OSTs. On each OSS node, run:</para>
126 <screen>umount -a -t lustre</screen>
131 <para>Upgrade the Linux operating system on all servers to RHEL 6 or other compatible
132 (tested) distribution and reboot. See the Linux Test Matrix at <xref
133 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="LustreTestMatrixTable"/>.</para>
136 <para>Upgrade the Linux operating system on all clients to RHEL 6 or other compatible
137 (tested) distribution and reboot. See the Linux Test Matrix at <xref
138 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="LustreTestMatrixTable"/>.</para>
141 <para>Download the Lustre server RPMs for your platform from the <link
142 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
143 repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
144 linkend="table_cnh_5m3_gk"/> for a list of required packages.</para>
147 <para>Install the Lustre server packages on all Lustre servers (MGS, MDSs, and OSSs).</para>
148 <orderedlist numeration="loweralpha">
150 <para>Log onto a Lustre server as the <literal>root</literal> user</para>
153 <para>Use the <literal>yum</literal> command to install the packages:</para>
155 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
159 <para>Verify the packages are installed correctly:</para>
161 <screen>rpm -qa|egrep "lustre|wc"|sort</screen>
165 <para>Repeat these steps on each Lustre server.</para>
170 <para>Download the Lustre client RPMs for your platform from the <link
171 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
172 repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
173 linkend="table_j3r_ym3_gk"/> for a list of required packages.</para>
175 <para>The version of the kernel running on a Lustre client must be the same as the version
176 of the <literal>lustre-client-modules-</literal><replaceable>ver</replaceable> package
177 being installed. If not, a compatible kernel must be installed on the client before the
178 Lustre client packages are installed.</para>
182 <para>Install the Lustre client packages on each of the Lustre clients to be
184 <orderedlist numeration="loweralpha">
186 <para>Log onto a Lustre client as the <literal>root</literal> user.</para>
189 <para>Use the <literal>yum</literal> command to install the packages:</para>
191 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
195 <para>Verify the packages were installed correctly:</para>
197 <screen># rpm -qa|egrep "lustre|kernel"|sort</screen>
201 <para>Repeat these steps on each Lustre client.</para>
206 <para condition="l22">(Optional) For upgrades to Lustre release 2.2 or higher, to enable
207 wide striping on an existing MDT, run the following command on the MDT
208 :<screen>mdt# tune2fs -O large_xattr <replaceable>device</replaceable></screen></para>
211 <para condition="l24">(Optional) For upgrades to Lustre release 2.4 or higher, to format an
212 additional MDT, complete these steps:<orderedlist numeration="loweralpha">
214 <para>Determine the index used for the first MDT (each MDT must have unique index).
215 Enter:<screen>client$ lctl dl | grep mdc
216 36 UP mdc lustre-MDT0000-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5</screen></para>
217 <para>In this example, the next available index is 1.</para>
220 <para>Add the new block device as a new MDT at the next available index by entering
222 line):<screen>mds# mkfs.lustre --reformat --fsname=<replaceable>filesystem_name</replaceable> --mdt \
223 --mgsnode=<replaceable>mgsnode</replaceable> --index <replaceable>1</replaceable> <replaceable>/dev/mdt1_device</replaceable>
226 </orderedlist></para>
229 <para>(<?oxy_comment_start author="lbeberne" timestamp="20130815T140026-0700" comment="Have James test"?>Optional<?oxy_comment_end?>)
230 If you are upgrading to Lustre release 2.3, or a release previous to Lustre release 2.3,
231 and want to enable the quota feature, complete these steps: <orderedlist
232 numeration="loweralpha">
234 <para>Before setting up the file system, enter on both the MDS and
235 OSTs:<screen>tunefs.lustre --quota</screen></para>
238 <para>When setting up the file system,
239 enter:<screen>conf_param $FSNAME.quota.mdt=$QUOTA_TYPE
240 conf_param $FSNAME.quota.ost=$QUOTA_TYPE</screen></para>
242 </orderedlist></para>
245 <para>Start the Lustre file system by starting the components in the order shown in the
246 following steps:</para>
247 <orderedlist numeration="loweralpha">
249 <para>Mount the MGT. On the MGS, run<screen>mgs# mount -a -t lustre</screen></para>
252 <para>Mount the MDT(s). On each MDT, run:<screen>mds# mount -a -t lustre</screen></para>
255 <para>Mount all the OSTs. On each OSS node, run:</para>
256 <screen>oss# mount -a -t lustre</screen>
258 <para>This command assumes that all the OSTs are listed in the
259 <literal>/etc/fstab</literal> file. OSTs that are not listed in the
260 <literal>/etc/fstab</literal> file, must be mounted individually by running the
261 mount command:</para>
262 <screen> mount -t lustre <replaceable>/dev/block_device</replaceable> <replaceable>/mount_point</replaceable> </screen>
266 <para>Mount the file system on the clients. On each client node, run:</para>
267 <screen>client# mount -a -t lustre</screen>
273 <para>The mounting order described in the steps above must be followed for the intial mount
274 and registration of a Lustre file system after an upgrade. For a normal start of a Lustre
275 file system, the mounting order is MGT, OSTs, MDT(s), clients.</para>
277 <para>If you have a problem upgrading a Lustre file system, see <xref
278 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438198_30989"/> for some ways
281 <section xml:id="Upgrading_2.x.x">
283 <primary>upgrading</primary>
284 <secondary>2.X.y to 2.X.y (minor release)</secondary>
285 </indexterm><indexterm/>Upgrading to a Lustre Release 2.X.y (Minor Release)</title>
286 <para>Rolling upgrades are supported for upgrading from any Lustre release 2.X.y to a more
287 recent 2.X.y release. This allows the Lustre file system to continue to run while individual
288 servers (or their failover partners) and clients are upgraded one at a time. The procedure for
289 upgrading a Lustre release 2.X.y file system to a more recent minor release is described in
291 <para>To upgrade Lustre release 2.X.y to a more recent minor release, complete these
295 <para>Create a complete, restorable file system backup. </para>
297 <para>Before installing the Lustre software, back up ALL data. The Lustre software
298 contains kernel modifications that interact with storage devices and may introduce
299 security issues and data loss if not installed, configured, or administered properly. If
300 a full backup of the file system is not practical, a device-level backup of the MDT file
301 system is recommended. See <xref linkend="backupandrestore"/> for a procedure.</para>
305 <para>Download the Lustre server RPMs for your platform from the <link
306 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
307 repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
308 linkend="table_cnh_5m3_gk"/> for a list of required packages.</para>
311 <para>For a rolling upgrade, complete any procedures required to keep the Lustre file system
312 running while the server to be upgraded is offline, such as failing over a primary server
313 to its secondary partner. </para>
316 <para>Unmount the Lustre server to be upgraded (MGS, MDS, or OSS)</para>
319 <para>Install the Lustre server packages on the Lustre server.</para>
320 <orderedlist numeration="loweralpha">
322 <para>Log onto the Lustre server as the <literal>root</literal> user</para>
325 <para>Use the <literal>yum</literal> command to install the packages:</para>
327 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
331 <para>Verify the packages are installed correctly:</para>
333 <screen>rpm -qa|egrep "lustre|wc"|sort</screen>
337 <para>Mount the Lustre server to restart the Lustre software on the
338 server:<screen>server# mount -a -t lustre</screen></para>
341 <para>Repeat these steps on each Lustre server.</para>
346 <para>Download the Lustre client RPMs for your platform from the <link
347 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
348 repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
349 linkend="table_j3r_ym3_gk"/> for a list of required packages.</para>
352 <para>Install the Lustre client packages on each of the Lustre clients to be
354 <orderedlist numeration="loweralpha">
356 <para>Log onto a Lustre client as the <literal>root</literal> user.</para>
359 <para>Use the <literal>yum</literal> command to install the packages:</para>
361 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
365 <para>Verify the packages were installed correctly:</para>
367 <screen># rpm -qa|egrep "lustre|kernel"|sort</screen>
371 <para>Mount the Lustre client to restart the Lustre software on the
372 client:<screen>client# mount -a -t lustre</screen></para>
375 <para>Repeat these steps on each Lustre client.</para>
380 <para>If you have a problem upgrading a Lustre file system, see <xref
381 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438198_30989"/> for some
382 suggestions for how to get help.</para>