1 <?xml version='1.0' encoding='utf-8'?>
2 <chapter xmlns="http://docbook.org/ns/docbook"
3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
4 xml:id="upgradinglustre">
5 <title xml:id="upgradinglustre.title">Upgrading a Lustre File System</title>
6 <para>This chapter describes interoperability between Lustre software
7 releases. It also provides procedures for upgrading from Lustre software
8 release 1.8 to Lustre software release 2.x , from a Lustre software release
9 2.x to a more recent Lustre software release 2.x (major release upgrade), and
10 from a a Lustre software release 2.x.y to a more recent Lustre software
11 release 2.x.y (minor release upgrade). It includes the following
16 <xref linkend="dbdoclet.50438205_82079" />
21 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
22 linkend="Upgrading_2.x" />
27 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
28 linkend="Upgrading_2.x.x" />
32 <section xml:id="dbdoclet.50438205_82079">
35 <primary>Lustre</primary>
36 <secondary>upgrading</secondary>
40 <primary>upgrading</primary>
41 </indexterm>Release Interoperability and Upgrade Requirements</title>
43 <emphasis role="italic">
44 <emphasis role="bold">Lustre software release 2.x (major)
49 <para>All servers must be upgraded at the same time, while some or
50 all clients may be upgraded independently of the servers.</para>
53 <para>All servers must be be upgraded to a Linux kernel supported by
54 the Lustre software. See the Lustre Release Notes for your Lustre
55 version for a list of tested Linux distributions.</para>
58 <para>Clients to be upgraded must be running a compatible Linux
59 distribution as described in the Release Notes.</para>
64 <emphasis role="italic">
65 <emphasis role="bold">Lustre software release 2.x.y release (minor)
71 <para>All servers must be upgraded at the same time, while some or all
72 clients may be upgraded.</para>
75 <para>Rolling upgrades are supported for minor releases allowing
76 individual servers and clients to be upgraded without stopping the
77 Lustre file system.</para>
81 <section xml:id="Upgrading_2.x">
84 <primary>upgrading</primary>
85 <secondary>major release (2.x to 2.x)</secondary>
88 <primary>wide striping</primary>
91 <primary>MDT</primary>
92 <secondary>multiple MDSs</secondary>
95 <primary>ea_inode</primary>
96 <secondary>large_xattr</secondary>
99 <primary>wide striping</primary>
100 <secondary>ea_inode</secondary>
101 <tertiary>large_xattr</tertiary>
102 </indexterm>Upgrading to Lustre Software Release 2.x (Major
104 <para>The procedure for upgrading from a Lustre software release 2.x to a
105 more recent 2.y major release of the Lustre software is described in this
106 section. To upgrade an existing 2.x installation to a more recent major
107 release, complete the following steps:</para>
110 <para>Create a complete, restorable file system backup.</para>
112 <para>Before installing the Lustre software, back up ALL data. The
113 Lustre software contains kernel modifications that interact with
114 storage devices and may introduce security issues and data loss if
115 not installed, configured, or administered properly. If a full backup
116 of the file system is not practical, a device-level backup of the MDT
117 file system is recommended. See
118 <xref linkend="backupandrestore" /> for a procedure.</para>
122 <para>Shut down the entire filesystem by following
123 <xref linkend="dbdoclet.shutdownLustre"/></para>
126 <para>Upgrade the Linux operating system on all servers to a compatible
127 (tested) Linux distribution and reboot.</para>
130 <para>Upgrade the Linux operating system on all clients to Red Hat
131 Enterprise Linux 6 or other compatible (tested) distribution and
135 <para>Download the Lustre server RPMs for your platform from the
136 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
137 Lustre Releases</link> repository. See
138 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
139 linkend="table_cnh_5m3_gk" /> for a list of required packages.</para>
142 <para>Install the Lustre server packages on all Lustre servers (MGS,
143 MDSs, and OSSs).</para>
144 <orderedlist numeration="loweralpha">
146 <para>Log onto a Lustre server as the
147 <literal>root</literal> user</para>
151 <literal>yum</literal> command to install the packages:</para>
153 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
157 <para>Verify the packages are installed correctly:</para>
159 <screen>rpm -qa|egrep "lustre|wc"</screen>
163 <para>Repeat these steps on each Lustre server.</para>
168 <para>Download the Lustre client RPMs for your platform from the
169 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
170 Lustre Releases</link> repository. See
171 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
172 linkend="table_j3r_ym3_gk" /> for a list of required packages.</para>
174 <para>The version of the kernel running on a Lustre client must be
175 the same as the version of the
176 <literal>lustre-client-modules-</literal>
177 <replaceable>ver</replaceable>package being installed. If not, a
178 compatible kernel must be installed on the client before the Lustre
179 client packages are installed.</para>
183 <para>Install the Lustre client packages on each of the Lustre clients
184 to be upgraded.</para>
185 <orderedlist numeration="loweralpha">
187 <para>Log onto a Lustre client as the
188 <literal>root</literal> user.</para>
192 <literal>yum</literal> command to install the packages:</para>
194 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
198 <para>Verify the packages were installed correctly:</para>
200 <screen># rpm -qa|egrep "lustre|kernel"</screen>
204 <para>Repeat these steps on each Lustre client.</para>
209 <para condition="l2D">Lustre allows striping a single file across up to
210 2000 OSTs. Before Lustre 2.13, the "wide striping" feature that
211 allowed creating files with more than 160 stripes was not enabled by
212 default. From the 2.13 release onward, the <literal>ea_inode</literal>
213 feature is enabled for newly-formatted MDTs. The feature can also
214 be enabled by the <literal>tune2fs</literal> command on existing MDTs:
215 <screen>mds# tune2fs -O ea_inode /dev/<replaceable>mdtdev</replaceable></screen>
217 <para>For more information about wide striping, see
218 <xref xmlns:xlink="http://www.w3.org/1999/xlink" linkend="wide_striping" />.</para>
221 <para>(Optional) To format an additional MDT, complete these steps:
222 <orderedlist numeration="loweralpha">
224 <para>Determine the index used for the first MDT (each MDT must
225 have unique index). Enter:
226 <screen>client$ lctl dl | grep mdc
227 36 UP mdc lustre-MDT0000-mdc-ffff88004edf3c00
228 4c8be054-144f-9359-b063-8477566eb84e 5</screen></para>
229 <para>In this example, the next available index is 1.</para>
232 <para>Add the new block device as a new MDT at the next available
233 index by entering (on one line):
234 <screen>mds# mkfs.lustre --reformat --fsname=<replaceable>filesystem_name</replaceable> --mdt \
235 --mgsnode=<replaceable>mgsnode</replaceable> --index <replaceable>1</replaceable>
236 <replaceable>/dev/mdt1_device</replaceable></screen></para>
238 </orderedlist></para>
241 <para>(Optional) If you are upgrading before Lustre software release
242 2.10, to enable the project quota feature enter the following on every
243 ldiskfs backend target:
244 <screen>tune2fs –O project /dev/<replaceable>dev</replaceable></screen>
246 <note><para>Enabling the <literal>project</literal> feature will prevent
247 the filesystem from being used by older versions of ldiskfs, so it
248 should only be enabled if the project quota feature is required and/or
249 after it is known that the upgraded release does not need to be
250 downgraded.</para></note>
251 <para>When setting up the file system, enter:
252 <screen>conf_param $FSNAME.quota.mdt=$QUOTA_TYPE
253 conf_param $FSNAME.quota.ost=$QUOTA_TYPE</screen></para>
256 <para>Start the Lustre file system by starting the components in the
257 order shown in the following steps:</para>
258 <orderedlist numeration="loweralpha">
260 <para>Mount the MGT. On the MGS, run
261 <screen>mgs# mount -a -t lustre</screen></para>
264 <para>Mount the MDT(s). On each MDT, run:
265 <screen>mds# mount -a -t lustre</screen></para>
268 <para>Mount all the OSTs. On each OSS node, run:</para>
269 <screen>oss# mount -a -t lustre</screen>
271 <para>This command assumes that all the OSTs are listed in the
272 <literal>/etc/fstab</literal> file. OSTs that are not listed in
274 <literal>/etc/fstab</literal> file, must be mounted individually
275 by running the mount command:</para>
276 <screen>mount -t lustre <replaceable>/dev/block_device</replaceable><replaceable>/mount_point</replaceable></screen>
280 <para>Mount the file system on the clients. On each client node,
282 <screen>client# mount -a -t lustre</screen>
288 <para>The mounting order described in the steps above must be followed
289 for the initial mount and registration of a Lustre file system after an
290 upgrade. For a normal start of a Lustre file system, the mounting order
291 is MGT, OSTs, MDT(s), clients.</para>
293 <para>If you have a problem upgrading a Lustre file system, see
294 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
295 linkend="dbdoclet.reporting_lustre_problem"/>for ways to get help.</para>
297 <section xml:id="Upgrading_2.x.x">
300 <primary>upgrading</primary>
301 <secondary>2.X.y to 2.X.y (minor release)</secondary>
302 </indexterm>Upgrading to Lustre Software Release 2.x.y (Minor
304 <para>Rolling upgrades are supported for upgrading from any Lustre software
305 release 2.x.y to a more recent Lustre software release 2.X.y. This allows
306 the Lustre file system to continue to run while individual servers (or
307 their failover partners) and clients are upgraded one at a time. The
308 procedure for upgrading a Lustre software release 2.x.y to a more recent
309 minor release is described in this section.</para>
310 <para>To upgrade Lustre software release 2.x.y to a more recent minor
311 release, complete these steps:</para>
314 <para>Create a complete, restorable file system backup.</para>
316 <para>Before installing the Lustre software, back up ALL data. The
317 Lustre software contains kernel modifications that interact with
318 storage devices and may introduce security issues and data loss if
319 not installed, configured, or administered properly. If a full backup
320 of the file system is not practical, a device-level backup of the MDT
321 file system is recommended. See
322 <xref linkend="backupandrestore" /> for a procedure.</para>
326 <para>Download the Lustre server RPMs for your platform from the
327 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
328 Lustre Releases</link> repository. See
329 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
330 linkend="table_cnh_5m3_gk" /> for a list of required packages.</para>
333 <para>For a rolling upgrade, complete any procedures required to keep
334 the Lustre file system running while the server to be upgraded is
335 offline, such as failing over a primary server to its secondary
339 <para>Unmount the Lustre server to be upgraded (MGS, MDS, or
343 <para>Install the Lustre server packages on the Lustre server.</para>
344 <orderedlist numeration="loweralpha">
346 <para>Log onto the Lustre server as the
347 <literal>root</literal> user</para>
351 <literal>yum</literal> command to install the packages:</para>
353 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
357 <para>Verify the packages are installed correctly:</para>
359 <screen>rpm -qa|egrep "lustre|wc"</screen>
363 <para>Mount the Lustre server to restart the Lustre software on the
365 <screen>server# mount -a -t lustre</screen></para>
368 <para>Repeat these steps on each Lustre server.</para>
373 <para>Download the Lustre client RPMs for your platform from the
374 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
375 Lustre Releases</link> repository. See
376 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
377 linkend="table_j3r_ym3_gk" /> for a list of required packages.</para>
380 <para>Install the Lustre client packages on each of the Lustre clients
381 to be upgraded.</para>
382 <orderedlist numeration="loweralpha">
384 <para>Log onto a Lustre client as the
385 <literal>root</literal> user.</para>
389 <literal>yum</literal> command to install the packages:</para>
391 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ... </screen>
395 <para>Verify the packages were installed correctly:</para>
397 <screen># rpm -qa|egrep "lustre|kernel"</screen>
401 <para>Mount the Lustre client to restart the Lustre software on the
403 <screen>client# mount -a -t lustre</screen></para>
406 <para>Repeat these steps on each Lustre client.</para>
411 <para>If you have a problem upgrading a Lustre file system, see
412 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
413 linkend="dbdoclet.reporting_lustre_problem" />for some suggestions for
414 how to get help.</para>
418 vim:expandtab:shiftwidth=2:tabstop=8: