1 <?xml version='1.0' encoding='utf-8'?>
2 <chapter xmlns="http://docbook.org/ns/docbook"
3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
4 xml:id="installinglustre">
5 <title xml:id="installinglustre.title">Installing the Lustre Software</title>
6 <para>This chapter describes how to install the Lustre software from RPM
7 packages. It includes:</para>
11 <xref linkend="dbdoclet.50438261_99193" />
16 <xref linkend="dbdoclet.50438261_41900" />
20 <para>For hardware and system requirements and hardware configuration
22 <xref linkend="settinguplustresystem" />.</para>
23 <section xml:id="dbdoclet.50438261_99193">
26 <primary>installing</primary>
27 <secondary>preparation</secondary>
28 </indexterm>Preparing to Install the Lustre Software</title>
29 <para>You can install the Lustre software from downloaded packages (RPMs)
30 or directly from the source code. This chapter describes how to install the
31 Lustre RPM packages. Instructions to install from source code are beyond
32 the scope of this document, and can be found elsewhere online.</para>
33 <para>The Lustre RPM packages are tested on current versions of Linux
34 enterprise distributions at the time they are created. See the release
35 notes for each version for specific details.</para>
36 <section xml:id="section_rqs_tjw_3k">
37 <title>Software Requirements</title>
38 <para>To install the Lustre software from RPMs, the following are
43 <emphasis role="bold">
44 <emphasis role="italic">Lustre server packages</emphasis>
45 </emphasis>. The required packages for Lustre 2.9 EL7 servers are
46 listed in the table below, where
47 <replaceable>ver</replaceable> refers to the Lustre release and
48 kernel version (e.g., 2.9.0-1.el7) and
49 <replaceable>arch</replaceable> refers to the processor architecture
50 (e.g., x86_64). These packages are available in the
51 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
52 Lustre Releases</link> repository, and may differ depending on
53 your distro and version.</para>
55 <table frame="all" xml:id="table_cnh_5m3_gk">
56 <title>Packages Installed on Lustre Servers</title>
58 <colspec colname="c1" colnum="1" colwidth="1.18*" />
59 <colspec colname="c2" colnum="2" colwidth="1*" />
62 <entry>Package Name</entry>
63 <entry>Description</entry>
69 <code>kernel-<replaceable>ver</replaceable>_lustre.<replaceable>arch</replaceable></code>
71 <entry>Linux kernel with Lustre software patches (often
72 referred to as "patched kernel")</entry>
76 <code>lustre-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
78 <entry>Lustre software command line tools</entry>
82 <code>kmod-lustre-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
84 <entry>Lustre-patched kernel modules</entry>
88 <code>kmod-lustre-osd-ldiskfs-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
90 <entry>Lustre back-end file system tools for ldiskfs-based
96 <code>lustre-osd-ldiskfs-mount-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
98 <entry>Helper library for <literal>mount.lustre</literal>
99 and <literal>mkfs.lustre</literal> for ldiskfs-based servers.
104 <code>kmod-lustre-osd-zfs-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
106 <entry>Lustre back-end file system tools for ZFS. This is
108 <literal>lustre-osd-ldiskfs</literal> (kmod-spl and
109 kmod-zfs available separately).</entry>
113 <code>lustre-osd-zfs-mount-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
115 <entry>Helper library for <literal>mount.lustre</literal>
116 and <literal>mkfs.lustre</literal> for ZFS-based servers
117 (zfs utilities available separately).
122 <code>e2fsprogs</code>
124 <entry>Utilities to maintain Lustre ldiskfs back-end file
130 <code>lustre-tests-<replaceable>ver</replaceable>_lustre.<replaceable>arch</replaceable></code>
132 <entry>Lustre I/O Kit benchmarking tools
133 <emphasis role="italic">(Included in Lustre software as of
134 release 2.2)</emphasis></entry>
143 <emphasis role="bold">
144 <emphasis role="italic">Lustre client packages</emphasis>
145 </emphasis>. The required packages for Lustre 2.9 EL7 clients are
146 listed in the table below, where
147 <replaceable>ver</replaceable> refers to the Linux distribution (e.g.,
148 3.6.18-348.1.1.el5). These packages are available in the
149 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
150 Lustre Releases</link> repository.</para>
152 <table frame="all" xml:id="table_j3r_ym3_gk">
153 <title>Packages Installed on Lustre Clients</title>
155 <colspec colname="c1" colnum="1" colwidth="1*" />
156 <colspec colname="c2" colnum="2" colwidth="1.02*" />
159 <entry>Package Name</entry>
160 <entry>Description</entry>
166 <code>kmod-lustre-client-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
168 <entry>Patchless kernel modules for client</entry>
172 <code>lustre-client-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
174 <entry>Client command line tools</entry>
178 <code>lustre-client-dkms-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
180 <entry>Alternate client RPM to kmod-lustre-client with
181 Dynamic Kernel Module Support (DKMS) installation. This
182 avoids the need to install a new RPM for each kernel update,
183 but requires a full build environment on the client.</entry>
190 <para>The version of the kernel running on a Lustre client must be
191 the same as the version of the
192 <code>kmod-lustre-client-<replaceable>ver</replaceable></code>
193 package being installed, unless the DKMS package is installed.
194 If the kernel running on the client is not compatible, a kernel
195 that is compatible must be installed on the client before the
196 Lustre file system software is used.</para>
201 <emphasis role="bold">
202 <emphasis role="italic">Lustre LNet network driver (LND)</emphasis>
203 </emphasis>. The Lustre LNDs provided with the Lustre software are
204 listed in the table below. For more information about Lustre LNet,
206 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
207 linkend="understandinglustrenetworking" />.</para>
208 <table frame="all" xml:id="table_bkx_dn3_gk">
209 <title>Network Types Supported by Lustre LNDs</title>
211 <colspec colname="c1" colnum="1" colwidth="1*" />
212 <colspec colname="newCol2" colnum="2" colwidth="1.95*" />
215 <entry>Supported Network Types</entry>
222 <entry>Any network carrying TCP traffic, including GigE,
223 10GigE, and IPoIB</entry>
226 <entry>InfiniBand network</entry>
227 <entry>OpenFabrics OFED (o2ib)</entry>
231 <entry>Gemini (Cray)</entry>
237 </itemizedlist></para>
240 <para>The InfiniBand and TCP Lustre LNDs are routinely tested during
241 release cycles. The other LNDs are maintained by their respective
247 <emphasis role="bold">
248 <emphasis role="italic">High availability software</emphasis>
249 </emphasis>. If needed, install third party high-availability
250 software. For more information, see
251 <xref linkend="dbdoclet.50438188_92688" />.</para>
255 <emphasis role="bold">
256 <emphasis role="italic">Optional packages.</emphasis>
257 </emphasis>Optional packages provided in the
258 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
259 Lustre Releases</link>repository may include the following
260 (depending on the operating system and platform):
264 <code>kernel-debuginfo</code>,
265 <code>kernel-debuginfo-common</code>,
266 <code>lustre-debuginfo</code>,
267 <code>lustre-osd-ldiskfs-debuginfo</code>- Versions of required
268 packages with debugging symbols and other debugging options
269 enabled for use in troubleshooting.</para>
273 <code>kernel-devel</code>, - Portions of the kernel tree needed
274 to compile third party modules, such as network drivers.</para>
278 <code>kernel-firmware</code>- Standard Red Hat Enterprise Linux
279 distribution that has been recompiled to work with the Lustre
284 <code>kernel-headers</code>- Header files installed under
285 /user/include and used when compiling user-space,
286 kernel-related code.</para>
290 <code>lustre-source</code>- Lustre software source code.</para>
294 <emphasis role="italic">(Recommended)</emphasis>
296 <code>perf-debuginfo</code>,
297 <code>python-perf</code>,
298 <code>python-perf-debuginfo</code>- Linux performance analysis
299 tools that have been compiled to match the Lustre kernel
302 </itemizedlist></para>
307 <section xml:id="section_rh2_d4w_gk">
308 <title>Environmental Requirements</title>
309 <para>Before installing the Lustre software, make sure the following
310 environmental requirements are met.
314 <emphasis role="italic">(Required)</emphasis>
315 <emphasis role="italic">
316 <emphasis role="bold">Disable Security-Enhanced Linux
317 <superscript>*</superscript>(SELinux) on all Lustre servers
319 </emphasis>The Lustre software does not support SELinux. Therefore,
320 the SELinux system extension must be disabled on all Lustre nodes.
321 Also, make sure other security extensions (such as the Novell
323 <superscript>*</superscript>security system) and network packet
324 filtering tools (such as iptables) do not interfere with the Lustre
329 <emphasis role="italic">(Required)</emphasis>
330 <emphasis role="italic">
331 <emphasis role="bold">Use the same user IDs (UID) and group IDs
332 (GID) on all clients.</emphasis>
333 </emphasis>If use of supplemental groups is required, see
334 <xref linkend="dbdoclet.50438291_32926" />for information about
335 supplementary user and group cache upcall (
336 <code>identity_upcall</code>).</para>
340 <emphasis role="italic">(Recommended)</emphasis>
341 <emphasis role="italic">
342 <emphasis role="bold">Provide remote shell access to
344 </emphasis>It is recommended that all cluster nodes have remote shell
345 client access to facilitate the use of Lustre configuration and
346 monitoring scripts. Parallel Distributed SHell (pdsh) is preferable,
347 although Secure SHell (SSH) is acceptable.</para>
351 <emphasis role="italic">(Recommended)</emphasis>
352 <emphasis role="italic">
353 <emphasis role="bold">Ensure client clocks are
354 synchronized.</emphasis>
355 </emphasis>The Lustre file system uses client clocks for timestamps.
356 If clocks are out of sync between clients, files will appear with
357 different time stamps when accessed by different clients. Drifting
358 clocks can also cause problems by, for example, making it difficult
359 to debug multi-node issues or correlate logs, which depend on
360 timestamps. We recommend that you use Network Time Protocol (NTP) to
361 keep client and server clocks in sync with each other. For more
362 information about NTP, see:
363 <link xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
365 </itemizedlist></para>
368 <section xml:id="dbdoclet.50438261_41900">
369 <title>Lustre Software Installation Procedure</title>
371 <para>Before installing the Lustre software, back up ALL data. The Lustre
372 software contains kernel modifications that interact with storage devices
373 and may introduce security issues and data loss if not installed,
374 configured, or administered properly.</para>
376 <para>To install the Lustre software from RPMs, complete the steps
380 <para>Verify that all Lustre installation requirements have been
384 <para>For hardware requirements, see
385 <xref linkend="settinguplustresystem" />.</para>
388 <para>For software and environmental requirements, see the section
389 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
390 linkend="dbdoclet.50438261_99193" />above.</para>
396 <literal>e2fsprogs</literal> RPMs for your platform from the
397 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
398 Lustre Releases</link>repository.</para>
401 <para>Download the Lustre server RPMs for your platform from the
402 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
403 Lustre Releases</link>repository. See
404 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
405 linkend="table_cnh_5m3_gk" />for a list of required packages.</para>
407 <listitem xml:id="dbdoclet.50438261_88296">
408 <para>Install the Lustre server and
409 <literal>e2fsprogs</literal> packages on all Lustre servers (MGS, MDSs,
411 <orderedlist numeration="loweralpha">
413 <para>Log onto a Lustre server as the
414 <literal>root</literal> user</para>
418 <literal>yum</literal> command to install the packages:</para>
421 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
426 <para>Verify the packages are installed correctly:</para>
429 rpm -qa|egrep "lustre|wc"|sort
434 <para>Reboot the server.</para>
437 <para>Repeat these steps on each Lustre server.</para>
439 </orderedlist></para>
442 <para>Download the Lustre client RPMs for your platform from the
443 <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
444 Lustre Releases</link>repository. See
445 <xref xmlns:xlink="http://www.w3.org/1999/xlink"
446 linkend="table_j3r_ym3_gk" />for a list of required packages.</para>
449 <para>Install the Lustre client packages on all Lustre clients.
451 <para>The version of the kernel running on a Lustre client must be
452 the same as the version of the
453 <literal>lustre-client-modules-</literal>
454 <replaceable>ver</replaceable> package being installed. If not, a
455 compatible kernel must be installed on the client before the Lustre
456 client packages are installed.</para>
458 <orderedlist numeration="loweralpha">
460 <para>Log onto a Lustre client as the root user.</para>
464 <literal>yum</literal> command to install the packages:</para>
467 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
472 <para>Verify the packages were installed correctly:</para>
475 # rpm -qa|egrep "lustre|kernel"|sort
480 <para>Reboot the client.</para>
483 <para>Repeat these steps on each Lustre client.</para>
488 <para>To configure LNet, go to
489 <xref linkend="configuringlnet" />. If default settings will be used for
491 <xref linkend="configuringlustre" />.</para>