InstallingLustre.xml

   1 <?xml version='1.0' encoding='UTF-8'?>
   2 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0"
   3   xml:lang="en-US" xml:id="installinglustre">
   4   <title xml:id="installinglustre.title">Installing the Lustre Software</title>
   5   <para>This chapter describes how to install the Lustre software from RPM packages. It
   6     includes:</para>
   7   <itemizedlist>
   8     <listitem>
   9       <para>
  10         <xref linkend="dbdoclet.50438261_99193"/>
  11       </para>
  12     </listitem>
  13     <listitem>
  14       <para>
  15         <xref linkend="dbdoclet.50438261_41900"/>
  16       </para>
  17     </listitem>
  18   </itemizedlist>
  19   <para>For hardware and system requirements and hardware configuration information, see <xref
  20       linkend="settinguplustresystem"/>.</para>
  21   <section xml:id="dbdoclet.50438261_99193">
  22     <title>
  23       <indexterm>
  24         <primary>installing</primary>
  25         <secondary>preparation</secondary>
  26       </indexterm> Preparing to Install the Lustre Software</title>
  27     <para>You can install the Lustre software from downloaded packages (RPMs) or directly from the
  28       source code. This chapter describes how to install the Lustre RPM packages. For information
  29       about installing from source code, see <xref linkend="installinglustrefromsourcecode"
  30       />.</para>
  31     <para>The Lustre RPM packages have been tested on the Linux distributions listed in the table
  32       below.</para>
  33     <para>
  34       <table frame="all" xml:id="LustreTestMatrixTable">
  35         <title>Lustre Test Matrix</title>
  36         <tgroup cols="3">
  37           <colspec colname="c1" colnum="1" colwidth="1*"/>
  38           <colspec colname="c2" colnum="2" colwidth="1.1*"/>
  39           <colspec colname="c3" colnum="3" colwidth="1.46*"/>
  40           <thead>
  41             <row>
  42               <entry>Lustre Version</entry>
  43               <entry>Servers Tested<superscript>1</superscript></entry>
  44               <entry>Clients Tested</entry>
  45             </row>
  46           </thead>
  47           <tbody>
  48             <row>
  49               <entry>
  50                 <para>2.0</para>
  51               </entry>
  52               <entry>
  53                 <para>RHEL 5, CentOS 5</para>
  54               </entry>
  55               <entry>
  56                 <para>RHEL 5, CentOS 5, SLES 11 SP0</para>
  57               </entry>
  58             </row>
  59             <row>
  60               <entry>
  61                 <para>2.1.x</para>
  62               </entry>
  63               <entry>
  64                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6</para>
  65               </entry>
  66               <entry>
  67                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
  68               </entry>
  69             </row>
  70             <row>
  71               <entry>
  72                 <para>2.2</para>
  73               </entry>
  74               <entry>
  75                 <para>RHEL 6, CentOS 6</para>
  76               </entry>
  77               <entry>
  78                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
  79               </entry>
  80             </row>
  81             <row>
  82               <entry>
  83                 <para>2.3</para>
  84               </entry>
  85               <entry>
  86                 <para>RHEL 6.3, CentOS 6.3</para>
  87               </entry>
  88               <entry>
  89                 <para>RHEL 6.3, CentOS 6.3, RHEL 5.8, CentOS 5.8, SLES 11 SP1</para>
  90               </entry>
  91             </row>
  92             <row>
  93               <entry>2.4.x</entry>
  94               <entry>
  95                 <para>RHEL 6.4, CentOS 6.4</para>
  96               </entry>
  97               <entry>
  98                 <para>RHEL 6.4, CentOS 6.4, SLES 11 SP2, FC18</para>
  99               </entry>
 100             </row>
 101           </tbody>
 102         </tgroup>
 103       </table>
 104     </para>
 105     <para><superscript>1</superscript>Red Hat Enterprise Edition (RHEL), CentOS Enterprise Linux
 106       Distribution (CentOS), SUSE Linux Enterprise Server (SLES), Fedora<superscript>*</superscript>
 107       F18 Linux kernel (FC18).</para>
 108     <section xml:id="section_rqs_tjw_3k">
 109       <title>Software Requirements</title>
 110       <para>To install the Lustre software from RPMs, the following are required:<itemizedlist>
 111           <listitem>
 112             <para><emphasis role="bold"><emphasis role="italic">Lustre server
 113                 packages</emphasis></emphasis>. The required packages for Lustre servers are listed
 114               in the table below, where <replaceable>ver</replaceable> refers to the Linux kernel
 115               distribution (e.g., 2.6.32-358.6.2.el6) and <replaceable>arch</replaceable> refers to
 116               the processor architecture (e.g., x86_64). These packages are available in the <link
 117                 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
 118                 Releases</link> repository.</para>
 119             <para>
 120               <table frame="all" xml:id="table_cnh_5m3_gk">
 121                 <title>Packages Installed on Lustre Servers</title>
 122                 <tgroup cols="2">
 123                   <colspec colname="c1" colnum="1" colwidth="1.18*"/>
 124                   <colspec colname="c2" colnum="2" colwidth="1*"/>
 125                   <thead>
 126                     <row>
 127                       <entry>Package Name</entry>
 128                       <entry>Description</entry>
 129                     </row>
 130                   </thead>
 131                   <tbody>
 132                     <row>
 133                       <entry><code>kernel-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 134                       <entry>Linux kernel with Lustre patches (often referred to as "patched
 135                         kernel")</entry>
 136                     </row>
 137                     <row>
 138                       <entry><code>lustre-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 139                       <entry>Lustre command line tools</entry>
 140                     </row>
 141                     <row>
 142                       <entry><code>lustre-modules-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 143                       <entry>Lustre-patched kernel modules</entry>
 144                     </row>
 145                     <row>
 146                       <entry><code>lustre-ldiskfs-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 147                       <entry>Lustre back-end file system tools</entry>
 148                     </row>
 149                     <row>
 150                       <entry><code>e2fsprogs</code></entry>
 151                       <entry>Utility to maintain Lustre back-end file system</entry>
 152                     </row>
 153                     <row>
 154                       <entry><code>lustre-tests-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 155                       <entry>Lustre I/O Kit benchmarking tools<emphasis role="italic"> (Included in
 156                           Lustre software as of Version 2.2)</emphasis></entry>
 157                     </row>
 158                   </tbody>
 159                 </tgroup>
 160               </table>
 161             </para>
 162           </listitem>
 163           <listitem>
 164             <para><emphasis role="bold"><emphasis role="italic">Lustre client
 165                 packages</emphasis></emphasis>. The required packages for Lustre clients are listed
 166               in the table below, where <replaceable>ver</replaceable> refers to the Linux
 167               distribution (e.g., 2.6.18-348.1.1.el5). These packages are available in the <link
 168                 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
 169                 Releases</link> repository.</para>
 170             <para>
 171               <table frame="all" xml:id="table_j3r_ym3_gk">
 172                 <title>Packages Installed in Lustre Clients</title>
 173                 <tgroup cols="2">
 174                   <colspec colname="c1" colnum="1" colwidth="1*"/>
 175                   <colspec colname="c2" colnum="2" colwidth="1.02*"/>
 176                   <thead>
 177                     <row>
 178                       <entry>Package Name</entry>
 179                       <entry>Description</entry>
 180                     </row>
 181                   </thead>
 182                   <tbody>
 183                     <row>
 184                       <entry><code>lustre-client-modules-</code><replaceable>ver</replaceable></entry>
 185                       <entry>Patchless kernel modules for client</entry>
 186                     </row>
 187                     <row>
 188                       <entry><code>lustre-client-</code><replaceable>ver</replaceable><code>_lustre</code></entry>
 189                       <entry>Client command line tools</entry>
 190                     </row>
 191                     <row>
 192                       <entry><code>lustre-client-tests-</code><replaceable>ver</replaceable></entry>
 193                       <entry>Lustre I/O Kit <emphasis role="italic">(Included in Lustre software as
 194                           of Version 2.2)</emphasis>
 195                       </entry>
 196                     </row>
 197                   </tbody>
 198                 </tgroup>
 199               </table>
 200             </para>
 201             <note>
 202               <para>The version of the kernel running on a Lustre client must be the same as the
 203                 version of the <code>lustre-client-modules-</code><replaceable>ver</replaceable>
 204                 package being installed. If the kernel running on the client is not compatible, a
 205                 kernel that is compatible must be installed on the client before the Lustre file
 206                 system software is installed.</para>
 207             </note>
 208           </listitem>
 209           <listitem>
 210             <para><emphasis role="bold"><emphasis role="italic">Lustre LNET network driver
 211                   (LND)</emphasis></emphasis>. The Lustre LNDs provided with the Lustre software are
 212               listed in the table below. For more information about Lustre LNET, see <xref
 213                 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="understandinglustrenetworking"
 214               />.</para>
 215             <table frame="all" xml:id="table_bkx_dn3_gk">
 216               <title>Network Types Supported by Lustre LNDs</title>
 217               <tgroup cols="2">
 218                 <colspec colname="c1" colnum="1" colwidth="1*"/>
 219                 <colspec colname="newCol2" colnum="2" colwidth="1.95*"/>
 220                 <thead>
 221                   <row>
 222                     <entry>Supported Network Types</entry>
 223                     <entry>Notes</entry>
 224                   </row>
 225                 </thead>
 226                 <tbody>
 227                   <row>
 228                     <entry>TCP</entry>
 229                     <entry>Any network carrying TCP traffic, including GigE, 10GigE, and
 230                       IPoIB</entry>
 231                   </row>
 232                   <row>
 233                     <entry>InfiniBand network</entry>
 234                     <entry>OpenFabrics OFED (o2ib)</entry>
 235                   </row>
 236                   <row>
 237                     <entry>gni</entry>
 238                     <entry>Gemini (Cray)</entry>
 239                   </row>
 240                   <row>
 241                     <entry>Seastar</entry>
 242                     <entry>Cray</entry>
 243                   </row>
 244                   <row>
 245                     <entry>MX</entry>
 246                     <entry>Myrinet network</entry>
 247                   </row>
 248                   <row>
 249                     <entry>ra</entry>
 250                     <entry>RapidArray<superscript>*</superscript> interconnect</entry>
 251                   </row>
 252                   <row>
 253                     <entry>Elan</entry>
 254                     <entry>Quadrics</entry>
 255                   </row>
 256                 </tbody>
 257               </tgroup>
 258             </table>
 259           </listitem>
 260         </itemizedlist></para>
 261       <para>
 262         <note>
 263           <para>The InfiniBand and TCP Lustre LNDs are routinely tested during release cycles. The
 264             other LNDs are maintained by their respective owners</para>
 265         </note>
 266         <itemizedlist>
 267           <listitem>
 268             <para><emphasis role="bold"><emphasis role="italic"> High availability
 269                   software</emphasis></emphasis>. If needed, install third party high-availability
 270               software. For more information, see <xref linkend="dbdoclet.50438188_92688"/>.</para>
 271           </listitem>
 272           <listitem>
 273             <para><emphasis role="bold"><emphasis role="italic">Optional
 274                 packages.</emphasis></emphasis> Optional packages  provided in the <link
 275                 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
 276                 Releases</link> repository may include the following (depending on the operating
 277               system and platform):<itemizedlist>
 278                 <listitem>
 279                   <para><code>kernel-debuginfo</code>, <code>kernel-debuginfo-common</code>,
 280                       <code>lustre-debuginfo</code>, <code>lustre-ldiskfs-debuginfo</code> -
 281                     Versions of required packages with debugging symbols and other debugging options
 282                     enabled for use in troubleshooting.</para>
 283                 </listitem>
 284                 <listitem>
 285                   <para><code>kernel-devel</code>,  - Portions of the kernel tree needed to compile
 286                     third party modules, such as network drivers.</para>
 287                 </listitem>
 288                 <listitem>
 289                   <para><code>kernel-firmware</code> - Standard Red Hat Enterprise Linux
 290                     distribution that has been recompiled to work with the Lustre kernel.</para>
 291                 </listitem>
 292                 <listitem>
 293                   <para><code>kernel-headers</code> - Header files installed under /user/include and
 294                     used when compiling user-space, kernel-related code.</para>
 295                 </listitem>
 296                 <listitem>
 297                   <para><code>lustre-source</code> - Source code for Lustre.</para>
 298                 </listitem>
 299                 <listitem>
 300                   <para><emphasis role="italic">(Recommended) </emphasis><code>perf</code>,
 301                       <code>perf-debuginfo</code>, <code>python-perf</code>,
 302                       <code>python-perf-debuginfo</code> -  Linux performance analysis tools that
 303                     have been compiled to match the Lustre kernel version.</para>
 304                 </listitem>
 305               </itemizedlist></para>
 306           </listitem>
 307         </itemizedlist>
 308       </para>
 309     </section>
 310     <section xml:id="section_rh2_d4w_gk">
 311       <title>Environmental Requirements</title>
 312       <para>Before installing the Lustre software, make sure the following environmental
 313         requirements are met.<itemizedlist>
 314           <listitem>
 315             <para><emphasis role="italic">(Required)</emphasis>
 316               <emphasis role="italic"><emphasis role="bold">Disable Security-Enhanced
 317                     Linux<superscript>*</superscript> (SELinux) on all Lustre servers and
 318                   clients.</emphasis></emphasis> The Lustre software does not support SELinux.
 319               Therefore, the SELinux system extension must be disabled on all Lustre nodes. Also,
 320               make sure other security extensions (such as the Novell
 321                 AppArmor<superscript>*</superscript> security system) and network packet filtering
 322               tools (such as iptables) do not interfere with the Lustre software.</para>
 323           </listitem>
 324           <listitem>
 325             <para><emphasis role="italic">(Required) </emphasis><emphasis role="italic"><emphasis
 326                   role="bold">Use the same user IDs (UID) and group IDs (GID) on all
 327                   clients.</emphasis></emphasis> If use of supplemental groups is required, see
 328                 <xref linkend="dbdoclet.50438291_32926"/> for information about supplementary user
 329               and group cache upcall (<code>identity_upcall</code>).</para>
 330           </listitem>
 331           <listitem>
 332             <para><emphasis role="italic">(Recommended) </emphasis><emphasis role="italic"><emphasis
 333                   role="bold">Provide remote shell access to clients.</emphasis></emphasis> It is
 334               recommended that all cluster nodes have remote shell client access to facilitate the
 335               use of Lustre configuration and monitoring scripts. Parallel Distributed SHell (pdsh)
 336               is preferable, although Secure SHell (SSH) is acceptable.</para>
 337           </listitem>
 338           <listitem>
 339             <para><emphasis role="italic">(Recommended) </emphasis><emphasis role="italic"><emphasis
 340                   role="bold">Ensure client clocks are synchronized.</emphasis></emphasis> The
 341               Lustre file system uses client clocks for timestamps. If clocks are out of sync
 342               between clients, files will appear with different time stamps when accessed by
 343               different clients. Drifting clocks can also cause problems by, for example, making it
 344               difficult to debug multi-node issues or correlate logs, which depend on timestamps. We
 345               recommend that you use Network Time Protocol (NTP) to keep client and server clocks in
 346               sync with each other. For more information about NTP, see: <link
 347                 xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
 348           </listitem>
 349         </itemizedlist></para>
 350     </section>
 351   </section>
 352   <section xml:id="dbdoclet.50438261_41900">
 353     <title>Lustre Installation Procedure</title>
 354     <caution>
 355       <para>Before installing the Lustre software, back up ALL data. The Lustre software contains
 356         kernel modifications that interact with storage devices and may introduce security issues
 357         and data loss if not installed, configured, or administered properly.</para>
 358     </caution>
 359     <para>To install the Lustre software from RPMs, complete the steps below.</para>
 360     <orderedlist>
 361       <listitem>
 362         <para>Verify that all Lustre installation requirements have been met. </para>
 363         <itemizedlist>
 364           <listitem>
 365             <para>For hardware requirements, see <xref linkend="settinguplustresystem"/>.</para>
 366           </listitem>
 367           <listitem>
 368             <para>For software and environmental requirements, see the section <xref
 369                 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438261_99193"
 370               />above.</para>
 371           </listitem>
 372         </itemizedlist>
 373       </listitem>
 374       <listitem>
 375                 <para>Download the <literal>e2fsprogs</literal> RPMs for your platform from the <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
 376           repository.</para>
 377       </listitem>
 378       <listitem>
 379         <para>Download the Lustre server RPMs for your platform from the <link
 380             xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
 381           repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 382             linkend="table_cnh_5m3_gk"/> for a list of required packages.</para>
 383       </listitem>
 384       <listitem xml:id="dbdoclet.50438261_88296">
 385         <para>Install the Lustre server and <literal>e2fsprogs</literal> packages on all Lustre servers (MGS, MDSs, and
 386             OSSs).<orderedlist numeration="loweralpha">
 387             <listitem>
 388               <para>Log onto a Lustre server as the <literal>root</literal> user</para>
 389             </listitem>
 390             <listitem>
 391               <para>Use the <literal>yum</literal> command to install the packages:</para>
 392               <para>
 393                 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
 394               </para>
 395             </listitem>
 396             <listitem>
 397               <para>Verify the packages are installed correctly:</para>
 398               <para>
 399                 <screen>rpm -qa|egrep "lustre|wc"|sort</screen>
 400               </para>
 401             </listitem>
 402             <listitem>
 403               <para>Reboot the server.</para>
 404             </listitem>
 405             <listitem>
 406               <para>Repeat these steps on each Lustre server.</para>
 407             </listitem>
 408           </orderedlist></para>
 409       </listitem>
 410       <listitem>
 411         <para>Download the Lustre client RPMs for your platform from the <link
 412             xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
 413           repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 414             linkend="table_j3r_ym3_gk"/> for a list of required packages.</para>
 415       </listitem>
 416       <listitem>
 417         <para>Install the Lustre client packages on all Lustre clients.<note>
 418             <para>The version of the kernel running on a Lustre client must be the same as the
 419               version of the <literal>lustre-client-modules-</literal><replaceable>ver</replaceable>
 420               package being installed. If not, a compatible kernel must be installed on the client
 421               before the Lustre client packages are installed.</para>
 422           </note></para>
 423         <orderedlist numeration="loweralpha">
 424           <listitem>
 425             <para>Log onto a Lustre client as the root user.</para>
 426           </listitem>
 427           <listitem>
 428             <para>Use the <literal>yum</literal> command to install the packages:</para>
 429             <para>
 430               <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
 431             </para>
 432           </listitem>
 433           <listitem>
 434             <para>Verify the packages were installed correctly:</para>
 435             <para>
 436               <screen># rpm -qa|egrep "lustre|kernel"|sort</screen>
 437             </para>
 438           </listitem>
 439           <listitem>
 440             <para>Reboot the client.</para>
 441           </listitem>
 442           <listitem>
 443             <para>Repeat these steps on each Lustre client.</para>
 444           </listitem>
 445         </orderedlist>
 446       </listitem>
 447     </orderedlist>
 448     <para>To configure LNET, go  to <xref linkend="configuringlnet"/>. If default settings will be
 449       used for LNET, go to <xref linkend="configuringlustre"/>.</para>
 450   </section>
 451 </chapter>