1 <?xml version='1.0' encoding='UTF-8'?>
2 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0"
3 xml:lang="en-US" xml:id="installinglustre">
4 <title xml:id="installinglustre.title">Installing the Lustre* Software</title>
5 <para>This chapter describes how to install the Lustre* software from RPM packages. It
10 <xref linkend="dbdoclet.50438261_99193"/>
15 <xref linkend="dbdoclet.50438261_41900"/>
19 <para>For hardware and system requirements and hardware configuration information, see <xref
20 linkend="settinguplustresystem"/>.</para>
21 <section xml:id="dbdoclet.50438261_99193">
24 <primary>installing</primary>
25 <secondary>preparation</secondary>
26 </indexterm> Preparing to Install the Lustre Software</title>
27 <para>You can install the Lustre software from downloaded packages (RPMs) or directly from the
28 source code. This chapter describes how to install the Lustre RPM packages. For information
29 about installing from source code, see <xref linkend="installinglustrefromsourcecode"
31 <para>The Lustre RPM packages have been tested on the Linux distributions listed in the table
34 <table frame="all" xml:id="LustreTestMatrixTable">
35 <title>Lustre Test Matrix</title>
37 <colspec colname="c1" colnum="1" colwidth="1*"/>
38 <colspec colname="c2" colnum="2" colwidth="1.1*"/>
39 <colspec colname="c3" colnum="3" colwidth="1.46*"/>
42 <entry>Lustre Version</entry>
43 <entry>Servers Tested<superscript>1</superscript></entry>
44 <entry>Clients Tested</entry>
53 <para>RHEL 5, CentOS 5</para>
56 <para>RHEL 5, CentOS 5, SLES 11 SP0</para>
64 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6</para>
67 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
75 <para>RHEL 6, CentOS 6</para>
78 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
86 <para>RHEL 6.3, CentOS 6.3</para>
89 <para>RHEL 6.3, CentOS 6.3, RHEL 5.8, CentOS 5.8, SLES 11 SP1</para>
95 <para>RHEL 6.4, CentOS 6.4</para>
98 <para>RHEL 6.4, CentOS 6.4, SLES 11 SP2, FC18</para>
105 <para><superscript>1</superscript>Red Hat* Enterprise Edition*, CentOS Enterprise Linux
106 Distribution, SUSE Linux Enterprise Server, Fedora* F18 Linux kernel.</para>
107 <section xml:id="section_rqs_tjw_3k">
108 <title>Software Requirements</title>
109 <para>To install the Lustre software from RPMs, the following are required:<itemizedlist>
111 <para><emphasis role="bold"><emphasis role="italic">Lustre server
112 packages</emphasis></emphasis>. The required packages for Lustre servers are listed
113 in the table below, where <replaceable>ver</replaceable> refers to the Linux* kernel
114 distribution (e.g., 2.6.32-358.6.2.el6) and <replaceable>arch</replaceable> refers to
115 the processor architecture (e.g., x86_64). These packages are available in the <link
116 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
117 Releases</link> repository.</para>
119 <table frame="all" xml:id="table_cnh_5m3_gk">
120 <title>Packages Installed on Lustre Servers</title>
122 <colspec colname="c1" colnum="1" colwidth="1.18*"/>
123 <colspec colname="c2" colnum="2" colwidth="1*"/>
126 <entry>Package Name*</entry>
127 <entry>Description</entry>
132 <entry><code>kernel-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
133 <entry>Linux kernel with Lustre patches (often referred to as "patched
137 <entry><code>lustre-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
138 <entry>Lustre command line tools</entry>
141 <entry><code>lustre-modules-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
142 <entry>Lustre-patched kernel modules</entry>
145 <entry><code>lustre-ldiskfs-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
146 <entry>Lustre back-end file system tools</entry>
149 <entry><code>e2fsprogs</code></entry>
150 <entry>Utility to maintain Lustre back-end file system</entry>
153 <entry><code>lustre-tests-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
154 <entry>Lustre I/O Kit benchmarking tools<emphasis role="italic"> (Included in
155 Lustre software as of Version 2.2)</emphasis></entry>
163 <para><emphasis role="bold"><emphasis role="italic">Lustre client
164 packages</emphasis></emphasis>. The required packages for Lustre clients are listed
165 in the table below, where <replaceable>ver</replaceable> refers to the Linux
166 distribution (e.g., 2.6.18-348.1.1.el5). These packages are available in the <link
167 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
168 Releases</link> repository.</para>
170 <table frame="all" xml:id="table_j3r_ym3_gk">
171 <title>Packages Installed in Lustre Clients</title>
173 <colspec colname="c1" colnum="1" colwidth="1*"/>
174 <colspec colname="c2" colnum="2" colwidth="1.02*"/>
177 <entry>Package Name</entry>
178 <entry>Description</entry>
183 <entry><code>lustre-client-modules-</code><replaceable>ver</replaceable></entry>
184 <entry>Patchless kernel modules for client</entry>
187 <entry><code>lustre-client-</code><replaceable>ver</replaceable><code>_lustre</code></entry>
188 <entry>Client command line tools</entry>
191 <entry><code>lustre-client-tests-</code><replaceable>ver</replaceable></entry>
192 <entry>Lustre I/O Kit <emphasis role="italic">(Included in Lustre software as
193 of Version 2.2)</emphasis>
201 <para>The version of the kernel running on a Lustre client must be the same as the
202 version of the <code>lustre-client-modules-</code><replaceable>ver</replaceable>
203 package being installed. If the kernel running on the client is not compatible, a
204 kernel that is compatible must be installed on the client before the Lustre file
205 system software is installed.</para>
209 <para><emphasis role="bold"><emphasis role="italic">Lustre LNET network driver
210 (LND)</emphasis></emphasis>. The Lustre LNDs provided with the Lustre software are
211 listed in the table below. For more information about Lustre LNET, see <xref
212 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="understandinglustrenetworking"
214 <table frame="all" xml:id="table_bkx_dn3_gk">
215 <title>Network Types Supported by Lustre LNDs</title>
217 <colspec colname="c1" colnum="1" colwidth="1*"/>
218 <colspec colname="newCol2" colnum="2" colwidth="1.95*"/>
221 <entry>Supported Network Types</entry>
228 <entry>Any network carrying TCP traffic, including GigE, 10GigE, and
232 <entry>InfiniBand* network</entry>
233 <entry>OpenFabrics OFED (o2ib)</entry>
237 <entry>Gemini (Cray)</entry>
240 <entry>Seastar</entry>
245 <entry>Myrinet* network</entry>
249 <entry>RapidArray* interconnect</entry>
253 <entry>Quadrics</entry>
259 </itemizedlist></para>
262 <para>The InfiniBand* and TCP Lustre LNDs are routinely tested during release cycles. The
263 other LNDs are maintained by their respective owners</para>
267 <para><emphasis role="bold"><emphasis role="italic"> High availability
268 software</emphasis></emphasis>. If needed, install third party high-availability
269 software. For more information, see <xref linkend="dbdoclet.50438188_92688"/>.</para>
272 <para><emphasis role="bold"><emphasis role="italic">Optional
273 packages.</emphasis></emphasis> Optional packages provided in the <link
274 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
275 Releases</link> repository may include the following (depending on the operating
276 system and platform):<itemizedlist>
278 <para><code>kernel-debuginfo</code>, <code>kernel-debuginfo-common</code>,
279 <code>lustre-debuginfo</code>, <code>lustre-ldiskfs-debuginfo</code> -
280 Versions of required packages with debugging symbols and other debugging options
281 enabled for use in troubleshooting.</para>
284 <para><code>kernel-devel</code>, - Portions of the kernel tree needed to compile
285 third party modules, such as network drivers.</para>
288 <para><code>kernel-firmware</code> - Standard Red Hat Enterprise Linux package
289 that has been recompiled to work with the Lustre kernel.</para>
292 <para><code>kernel-headers</code> - Header files installed under /user/include and
293 used when compiling user-space, kernel-related code.</para>
296 <para><code>lustre-source</code> - Source code for Lustre.</para>
299 <para><emphasis role="italic">(Recommended) </emphasis><code>perf</code>,
300 <code>perf-debuginfo</code>, <code>python-perf</code>,
301 <code>python-perf-debuginfo</code> - Linux performance analysis tools that
302 have been compiled to match the Lustre kernel version.</para>
304 </itemizedlist></para>
309 <section xml:id="section_rh2_d4w_gk">
310 <title>Environmental Requirements</title>
311 <para>Before installing the Lustre software, make sure the following environmental
312 requirements are met.<itemizedlist>
314 <para><emphasis role="italic">(Required)</emphasis>
315 <emphasis role="italic"><emphasis role="bold">Disable Security-Enhanced Linux
316 (SELinux) on all Lustre servers and clients.</emphasis></emphasis> The Lustre
317 software does not support SELinux. Therefore, the SELinux system extension must be
318 disabled on all Lustre nodes. Also, make sure other security extensions (such as the
319 Novell AppArmor* security system) and network packet filtering tools (such as
320 iptables) do not interfere with the Lustre software.</para>
323 <para><emphasis role="italic">(Required) </emphasis><emphasis role="italic"><emphasis
324 role="bold">Use the same user IDs (UID) and group IDs (GID) on all
325 clients.</emphasis></emphasis> If use of supplemental groups is required, see
326 <xref linkend="dbdoclet.50438291_32926"/> for information about supplementary user
327 and group cache upcall (<code>identity_upcall</code>).</para>
330 <para><emphasis role="italic">(Recommended) </emphasis><emphasis role="italic"><emphasis
331 role="bold">Provide remote shell access to clients.</emphasis></emphasis> It is
332 recommended that all cluster nodes have remote shell client access to facilitate the
333 use of Lustre configuration and monitoring scripts. Parallel Distributed SHell (pdsh)
334 is preferable, although Secure SHell (SSH) is acceptable.</para>
337 <para><emphasis role="italic">(Recommended) </emphasis><emphasis role="italic"><emphasis
338 role="bold">Ensure client clocks are synchronized.</emphasis></emphasis> The
339 Lustre file system uses client clocks for timestamps. If clocks are out of sync
340 between clients, files will appear with different time stamps when accessed by
341 different clients. Drifting clocks can also cause problems by, for example, making it
342 difficult to debug multi-node issues or correlate logs, which depend on timestamps. We
343 recommend that you use Network Time Protocol (NTP) to keep client and server clocks in
344 sync with each other. For more information about NTP, see: <link
345 xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
347 </itemizedlist></para>
350 <section xml:id="dbdoclet.50438261_41900">
351 <title>Lustre Installation Procedure</title>
353 <para>Before installing the Lustre software, back up ALL data. The Lustre software contains
354 kernel modifications that interact with storage devices and may introduce security issues
355 and data loss if not installed, configured, or administered properly.</para>
357 <para>To install the Lustre software from RPMs, complete the steps below.</para>
360 <para>Verify that all Lustre installation requirements have been met. </para>
363 <para>For hardware requirements, see <xref linkend="settinguplustresystem"/>.</para>
366 <para>For software and environmental requirements, see the section <xref
367 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438261_99193"
373 <para>Download the Lustre server RPMs for your platform from the <link
374 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
375 repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
376 linkend="table_cnh_5m3_gk"/> for a list of required packages.</para>
378 <listitem xml:id="dbdoclet.50438261_88296">
379 <para>Install the Lustre server packages on all Lustre servers (MGS, MDSs, and
380 OSSs).<orderedlist numeration="loweralpha">
382 <para>Log onto a Lustre server as the <literal>root</literal> user</para>
385 <para>Use the <literal>yum</literal> command to install the packages:</para>
387 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
391 <para>Verify the packages are installed correctly:</para>
393 <screen>rpm -qa|egrep "lustre|wc"|sort</screen>
397 <para>Reboot the server.</para>
400 <para>Repeat these steps on each Lustre server.</para>
402 </orderedlist></para>
405 <para>Download the Lustre client RPMs for your platform from the <link
406 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
407 repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
408 linkend="table_j3r_ym3_gk"/> for a list of required packages.</para>
411 <para>Install the Lustre client packages on all Lustre clients.<note>
412 <para>The version of the kernel running on a Lustre client must be the same as the
413 version of the <literal>lustre-client-modules-</literal><replaceable>ver</replaceable>
414 package being installed. If not, a compatible kernel must be installed on the client
415 before the Lustre client packages are installed.</para>
417 <orderedlist numeration="loweralpha">
419 <para>Log onto a Lustre client as the root user.</para>
422 <para>Use the <literal>yum</literal> command to install the packages:</para>
424 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
428 <para>Verify the packages were installed correctly:</para>
430 <screen># rpm -qa|egrep "lustre|kernel"|sort</screen>
434 <para>Reboot the client.</para>
437 <para>Repeat these steps on each Lustre client.</para>
442 <para>To configure LNET, go to <xref linkend="configuringlnet"/>. If default settings will be
443 used for LNET, go to <xref linkend="configuringlustre"/>.</para>