1 <?xml version='1.0' encoding='utf-8'?>
2 <chapter xmlns="http://docbook.org/ns/docbook"
3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
4 xml:id="upgradinglustre">
5 <title xml:id="upgradinglustre.title">Upgrading a Lustre File System</title>
6 <para>This chapter describes interoperability between Lustre software
7 releases. It also provides procedures for upgrading from older Lustre 2.x
8 software releases to a more recent 2.y Lustre release a (major release
9 upgrade), and from a Lustre software release 2.x.y to a more recent
10 Lustre software release 2.x.z (minor release upgrade). It includes the
11 following sections:</para>
15 <xref linkend="dbdoclet.50438205_82079" />
20 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
21 linkend="Upgrading_2.x" />
26 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
27 linkend="Upgrading_2.x.x" />
31 <section xml:id="dbdoclet.50438205_82079">
34 <primary>Lustre</primary>
35 <secondary>upgrading</secondary>
39 <primary>upgrading</primary>
40 </indexterm>Release Interoperability and Upgrade Requirements</title>
42 <emphasis role="italic">
43 <emphasis role="bold">Lustre software release 2.x (major)
48 <para>All servers must be upgraded at the same time, while some or
49 all clients may be upgraded independently of the servers.</para>
52 <para>All servers must be be upgraded to a Linux kernel supported by
53 the Lustre software. See the Lustre Release Notes for your Lustre
54 version for a list of tested Linux distributions.</para>
57 <para>Clients to be upgraded must be running a compatible Linux
58 distribution as described in the Release Notes.</para>
63 <emphasis role="italic">
64 <emphasis role="bold">Lustre software release 2.x.y release (minor)
70 <para>All servers must be upgraded at the same time, while some or all
71 clients may be upgraded.</para>
74 <para>Rolling upgrades are supported for minor releases allowing
75 individual servers and clients to be upgraded without stopping the
76 Lustre file system.</para>
80 <section xml:id="Upgrading_2.x">
83 <primary>upgrading</primary>
84 <secondary>major release (2.x to 2.x)</secondary>
87 <primary>wide striping</primary>
90 <primary>MDT</primary>
91 <secondary>multiple MDSs</secondary>
94 <primary>ea_inode</primary>
95 <secondary>large_xattr</secondary>
96 </indexterm>Upgrading to Lustre Software Release 2.x (Major
98 <para>The procedure for upgrading from a Lustre software release 2.x to a
99 more recent 2.y major release of the Lustre software is described in this
100 section. To upgrade an existing 2.x installation to a more recent major
101 release, complete the following steps:</para>
104 <para>Create a complete, restorable file system backup.</para>
106 <para>Before installing the Lustre software, back up ALL data. The
107 Lustre software contains kernel modifications that interact with
108 storage devices and may introduce security issues and data loss if
109 not installed, configured, or administered properly. If a full backup
110 of the file system is not practical, a device-level backup of the MDT
111 file system is recommended. See
112 <xref linkend="backupandrestore" /> for a procedure.</para>
116 <para>Shut down the entire filesystem by following
117 <xref linkend="dbdoclet.shutdownLustre"/></para>
120 <para>Upgrade the Linux operating system on all servers to a compatible
121 (tested) Linux distribution and reboot.</para>
124 <para>Upgrade the Linux operating system on all clients to a
125 compatible (tested) distribution and reboot.</para>
128 <para>Download the Lustre server RPMs for your platform from the
129 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
130 Lustre Releases</link> repository. See
131 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
132 linkend="table.installed_server_pkg" />for a list of required packages.
136 <para>Install the Lustre server packages on all Lustre servers (MGS,
137 MDSs, and OSSs).</para>
138 <orderedlist numeration="loweralpha">
140 <para>Log onto a Lustre server as the
141 <literal>root</literal> user</para>
145 <literal>yum</literal> command to install the packages:</para>
147 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
151 <para>Verify the packages are installed correctly:</para>
153 <screen>rpm -qa|egrep "lustre|wc"</screen>
157 <para>Repeat these steps on each Lustre server.</para>
162 <para>Download the Lustre client RPMs for your platform from the
163 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
164 Lustre Releases</link> repository. See
165 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
166 linkend="table.installed_client_pkg" /> for a list of required packages.
169 <para>The version of the kernel running on a Lustre client must be
170 the same as the version of the
171 <literal>lustre-client-modules-</literal>
172 <replaceable>ver</replaceable>package being installed. If not, a
173 compatible kernel must be installed on the client before the Lustre
174 client packages are installed.</para>
178 <para>Install the Lustre client packages on each of the Lustre clients
179 to be upgraded.</para>
180 <orderedlist numeration="loweralpha">
182 <para>Log onto a Lustre client as the
183 <literal>root</literal> user.</para>
187 <literal>yum</literal> command to install the packages:</para>
189 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
193 <para>Verify the packages were installed correctly:</para>
195 <screen># rpm -qa|egrep "lustre|kernel"</screen>
199 <para>Repeat these steps on each Lustre client.</para>
204 <para>The DNE feature allows using multiple MDTs within a single
205 filesystem namespace, and each MDT can each serve one or more remote
206 sub-directories in the file system. The <literal>root</literal>
207 directory is always located on MDT0.</para>
208 <para>Note that clients running a release prior to the Lustre software
209 release 2.4 can only see the namespace hosted by MDT0 and will return an
210 IO error if an attempt is made to access a directory on another
212 <para>(Optional) To format an additional MDT, complete these steps:
213 <orderedlist numeration="loweralpha">
215 <para>Determine the index used for the first MDT (each MDT must
216 have unique index). Enter:
217 <screen>client$ lctl dl | grep mdc
218 36 UP mdc lustre-MDT0000-mdc-ffff88004edf3c00
219 4c8be054-144f-9359-b063-8477566eb84e 5</screen></para>
220 <para>In this example, the next available index is 1.</para>
223 <para>Format the new block device as a new MDT at the next
224 available MDT index by entering (on one line):
225 <screen>mds# mkfs.lustre --reformat --fsname=<replaceable>filesystem_name</replaceable> --mdt \
226 --mgsnode=<replaceable>mgsnode</replaceable> --index <replaceable>new_mdt_index</replaceable>
227 <replaceable>/dev/mdt1_device</replaceable></screen></para>
229 </orderedlist></para>
232 <para>(Optional) If you are upgrading from a release before Lustre
233 2.10, to enable the project quota feature enter the following on every
234 ldiskfs backend target while unmounted:
235 <screen>tune2fs –O project /dev/<replaceable>dev</replaceable></screen>
237 <note><para>Enabling the <literal>project</literal> feature will prevent
238 the filesystem from being used by older versions of ldiskfs, so it
239 should only be enabled if the project quota feature is required
240 and/or after it is known that the upgraded release does not need
241 to be downgraded.</para></note>
244 <para>When setting up the file system, enter:
245 <screen>conf_param $FSNAME.quota.mdt=$QUOTA_TYPE
246 conf_param $FSNAME.quota.ost=$QUOTA_TYPE</screen></para>
249 <para condition="l2D">(Optional) If upgrading an ldiskfs MDT formatted
250 prior to Lustre 2.13, the "wide striping" feature that allows files
251 to have more than 160 stripes and store other large xattrs was not
252 enabled by default. This feature can be enabled on existing MDTs
253 by running the following command on all MDT devices:
254 <screen>mds# tune2fs -O ea_inode /dev/<replaceable>mdtdev</replaceable></screen>
256 <para>For more information about wide striping, see
257 <xref xmlns:xlink="https://www.w3.org/1999/xlink"
258 linkend="wide_striping" />.</para>
261 <para>Start the Lustre file system by starting the components in the
262 order shown in the following steps:</para>
263 <orderedlist numeration="loweralpha">
265 <para>Mount the MGT. On the MGS, run
266 <screen>mgs# mount -a -t lustre</screen></para>
269 <para>Mount the MDT(s). On each MDT, run:
270 <screen>mds# mount -a -t lustre</screen></para>
273 <para>Mount all the OSTs. On each OSS node, run:</para>
274 <screen>oss# mount -a -t lustre</screen>
276 <para>This command assumes that all the OSTs are listed in the
277 <literal>/etc/fstab</literal> file. OSTs that are not listed in
279 <literal>/etc/fstab</literal> file, must be mounted individually
280 by running the mount command:</para>
281 <screen>mount -t lustre <replaceable>/dev/block_device</replaceable><replaceable>/mount_point</replaceable></screen>
285 <para>Mount the file system on the clients. On each client node,
287 <screen>client# mount -a -t lustre</screen>
292 <para condition='l27'>(Optional) If you are upgrading from a release before Lustre
293 2.7, to enable OST FIDs to also store the OST index (to improve
294 reliability of LFSCK and debug messages), <emphasis>after</emphasis>
295 the OSTs are mounted run once on each OSS:
296 <screen>oss# lctl set_param osd-ldiskfs.*.osd_index_in_idif=1</screen>
298 <note><para>Enabling the <literal>index_in_idif</literal> feature will
299 prevent the OST from being used by older versions of Lustre, so it
300 should only be enabled once it is known there is no need for the
301 OST to be downgraded to an earlier release.</para></note>
304 <para>If a new MDT was added to the filesystem, the new MDT must be
305 attached into the namespace by creating one or more
306 <emphasis>new</emphasis> DNE subdirectories with the
307 <literal>lfs mkdir</literal> command that use the new MDT:
309 client# lfs mkdir -i <replaceable>new_mdt_index /testfs/new_dir</replaceable>
312 <para condition='l28'>In Lustre 2.8 and later, it is possible to
313 split a new directory across multiple MDTs by creating it with
316 client# lfs mkdir -c 2 <replaceable>/testfs/new_striped_dir</replaceable>
319 <para condition='l2D'>In Lustre 2.13 and later, it is possible to set
320 the default striping on <emphasis>existing</emphasis> directories
321 so that new remote subdirectories are created on less-full MDTs:
323 client# lfs setdirstripe -c 1 -i -1 <replaceable>/testfs/some_dir</replaceable>
329 <para>The mounting order described in the steps above must be followed
330 for the initial mount and registration of a Lustre file system after an
331 upgrade. For a normal start of a Lustre file system, the mounting order
332 is MGT, OSTs, MDT(s), clients.</para>
334 <para>If you have a problem upgrading a Lustre file system, see
335 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
336 linkend="dbdoclet.reporting_lustre_problem"/>for ways to get help.</para>
338 <section xml:id="Upgrading_2.x.x">
341 <primary>upgrading</primary>
342 <secondary>2.X.y to 2.X.y (minor release)</secondary>
343 </indexterm>Upgrading to Lustre Software Release 2.x.y (Minor
345 <para>Rolling upgrades are supported for upgrading from any Lustre software
346 release 2.x.y to a more recent Lustre software release 2.X.y. This allows
347 the Lustre file system to continue to run while individual servers (or
348 their failover partners) and clients are upgraded one at a time. The
349 procedure for upgrading a Lustre software release 2.x.y to a more recent
350 minor release is described in this section.</para>
351 <para>To upgrade Lustre software release 2.x.y to a more recent minor
352 release, complete these steps:</para>
355 <para>Create a complete, restorable file system backup.</para>
357 <para>Before installing the Lustre software, back up ALL data. The
358 Lustre software contains kernel modifications that interact with
359 storage devices and may introduce security issues and data loss if
360 not installed, configured, or administered properly. If a full backup
361 of the file system is not practical, a device-level backup of the MDT
362 file system is recommended. See
363 <xref linkend="backupandrestore" /> for a procedure.</para>
367 <para>Download the Lustre server RPMs for your platform from the
368 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
369 Lustre Releases</link> repository. See
370 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
371 linkend="table.installed_server_pkg" /> for a list of required packages.
375 <para>For a rolling upgrade, complete any procedures required to keep
376 the Lustre file system running while the server to be upgraded is
377 offline, such as failing over a primary server to its secondary
381 <para>Unmount the Lustre server to be upgraded (MGS, MDS, or
385 <para>Install the Lustre server packages on the Lustre server.</para>
386 <orderedlist numeration="loweralpha">
388 <para>Log onto the Lustre server as the
389 <literal>root</literal> user</para>
393 <literal>yum</literal> command to install the packages:</para>
395 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
399 <para>Verify the packages are installed correctly:</para>
401 <screen>rpm -qa|egrep "lustre|wc"</screen>
405 <para>Mount the Lustre server to restart the Lustre software on the
407 <screen>server# mount -a -t lustre</screen></para>
410 <para>Repeat these steps on each Lustre server.</para>
415 <para>Download the Lustre client RPMs for your platform from the
416 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
417 Lustre Releases</link> repository. See
418 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
419 linkend="table.installed_client_pkg" /> for a list of required packages.
423 <para>Install the Lustre client packages on each of the Lustre clients
424 to be upgraded.</para>
425 <orderedlist numeration="loweralpha">
427 <para>Log onto a Lustre client as the
428 <literal>root</literal> user.</para>
432 <literal>yum</literal> command to install the packages:</para>
434 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
438 <para>Verify the packages were installed correctly:</para>
440 <screen># rpm -qa|egrep "lustre|kernel"</screen>
444 <para>Mount the Lustre client to restart the Lustre software on the
446 <screen>client# mount -a -t lustre</screen></para>
449 <para>Repeat these steps on each Lustre client.</para>
454 <para>If you have a problem upgrading a Lustre file system, see
455 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
456 linkend="dbdoclet.reporting_lustre_problem" />for some suggestions for
457 how to get help.</para>
461 vim:expandtab:shiftwidth=2:tabstop=8: