InstallingLustre.xml

   1 <?xml version='1.0' encoding='utf-8'?>
   2 <chapter xmlns="http://docbook.org/ns/docbook"
   3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
   4 xml:id="installinglustre">
   5   <title xml:id="installinglustre.title">Installing the Lustre Software</title>
   6   <para>This chapter describes how to install the Lustre software from RPM
   7   packages. It includes:</para>
   8   <itemizedlist>
   9     <listitem>
  10       <para>
  11         <xref linkend="dbdoclet.50438261_99193" />
  12       </para>
  13     </listitem>
  14     <listitem>
  15       <para>
  16         <xref linkend="dbdoclet.50438261_41900" />
  17       </para>
  18     </listitem>
  19   </itemizedlist>
  20   <para>For hardware and system requirements and hardware configuration
  21   information, see
  22   <xref linkend="settinguplustresystem" />.</para>
  23   <section xml:id="dbdoclet.50438261_99193">
  24     <title>
  25     <indexterm>
  26       <primary>installing</primary>
  27       <secondary>preparation</secondary>
  28     </indexterm>Preparing to Install the Lustre Software</title>
  29     <para>You can install the Lustre software from downloaded packages (RPMs)
  30     or directly from the source code. This chapter describes how to install the
  31     Lustre RPM packages. For information about installing from source code, see
  32
  33     <xref linkend="installinglustrefromsourcecode" />.</para>
  34     <para>The Lustre RPM packages have been tested on the Linux distributions
  35     listed in the table below.</para>
  36     <para>
  37       <table frame="all" xml:id="LustreTestMatrixTable">
  38         <title>Lustre Test Matrix</title>
  39         <tgroup cols="3">
  40           <colspec colname="c1" colnum="1" colwidth="1*" />
  41           <colspec colname="c2" colnum="2" colwidth="1.1*" />
  42           <colspec colname="c3" colnum="3" colwidth="1.46*" />
  43           <thead>
  44             <row>
  45               <entry>Lustre Release</entry>
  46               <entry>Servers Tested
  47               <superscript>1</superscript></entry>
  48               <entry>Clients Tested</entry>
  49             </row>
  50           </thead>
  51           <tbody>
  52             <row>
  53               <entry>
  54                 <para>2.0</para>
  55               </entry>
  56               <entry>
  57                 <para>RHEL 5, CentOS 5</para>
  58               </entry>
  59               <entry>
  60                 <para>RHEL 5, CentOS 5, SLES 11 SP0</para>
  61               </entry>
  62             </row>
  63             <row>
  64               <entry>
  65                 <para>2.1.x</para>
  66               </entry>
  67               <entry>
  68                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6</para>
  69               </entry>
  70               <entry>
  71                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
  72               </entry>
  73             </row>
  74             <row>
  75               <entry>
  76                 <para>2.2</para>
  77               </entry>
  78               <entry>
  79                 <para>RHEL 6, CentOS 6</para>
  80               </entry>
  81               <entry>
  82                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
  83               </entry>
  84             </row>
  85             <row>
  86               <entry>
  87                 <para>2.3</para>
  88               </entry>
  89               <entry>
  90                 <para>RHEL 6.3, CentOS 6.3</para>
  91               </entry>
  92               <entry>
  93                 <para>RHEL 6.3, CentOS 6.3, RHEL 5.8, CentOS 5.8, SLES 11
  94                 SP1</para>
  95               </entry>
  96             </row>
  97             <row>
  98               <entry>2.4.x</entry>
  99               <entry>
 100                 <para>RHEL 6.4, CentOS 6.4</para>
 101               </entry>
 102               <entry>
 103                 <para>RHEL 6.4, CentOS 6.4, SLES 11 SP2, FC18</para>
 104               </entry>
 105             </row>
 106           </tbody>
 107         </tgroup>
 108       </table>
 109     </para>
 110     <para>
 111     <superscript>1</superscript>Red Hat Enterprise Edition (RHEL), CentOS
 112     Enterprise Linux Distribution (CentOS), SUSE Linux Enterprise Server
 113     (SLES), Fedora
 114     <superscript>*</superscript>F18 Linux kernel (FC18).</para>
 115     <section xml:id="section_rqs_tjw_3k">
 116       <title>Software Requirements</title>
 117       <para>To install the Lustre software from RPMs, the following are
 118       required:
 119       <itemizedlist>
 120         <listitem>
 121           <para>
 122           <emphasis role="bold">
 123             <emphasis role="italic">Lustre server packages</emphasis>
 124           </emphasis>. The required packages for Lustre servers are listed in
 125           the table below, where
 126           <replaceable>ver</replaceable>refers to the Linux kernel distribution
 127           (e.g., 2.6.32-358.6.2.el6) and
 128           <replaceable>arch</replaceable>refers to the processor architecture
 129           (e.g., x86_64). These packages are available in the
 130           <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">
 131           Lustre Releases</link>repository.</para>
 132           <para>
 133             <table frame="all" xml:id="table_cnh_5m3_gk">
 134               <title>Packages Installed on Lustre Servers</title>
 135               <tgroup cols="2">
 136                 <colspec colname="c1" colnum="1" colwidth="1.18*" />
 137                 <colspec colname="c2" colnum="2" colwidth="1*" />
 138                 <thead>
 139                   <row>
 140                     <entry>Package Name</entry>
 141                     <entry>Description</entry>
 142                   </row>
 143                 </thead>
 144                 <tbody>
 145                   <row>
 146                     <entry>
 147                       <code>kernel-</code>
 148                       <replaceable>ver</replaceable>
 149                       <code>_lustre.</code>
 150                       <replaceable>arch</replaceable>
 151                     </entry>
 152                     <entry>Linux kernel with Lustre software patches (often
 153                     referred to as "patched kernel")</entry>
 154                   </row>
 155                   <row>
 156                     <entry>
 157                       <code>lustre-</code>
 158                       <replaceable>ver</replaceable>
 159                       <code>_lustre.</code>
 160                       <replaceable>arch</replaceable>
 161                     </entry>
 162                     <entry>Lustre software command line tools</entry>
 163                   </row>
 164                   <row>
 165                     <entry>
 166                       <code>lustre-modules-</code>
 167                       <replaceable>ver</replaceable>
 168                       <code>_lustre.</code>
 169                       <replaceable>arch</replaceable>
 170                     </entry>
 171                     <entry>Lustre-patched kernel modules</entry>
 172                   </row>
 173                   <row>
 174                     <entry>
 175                       <code>lustre-osd-ldiskfs-</code>
 176                       <replaceable>ver</replaceable>
 177                       <code>_lustre.</code>
 178                       <replaceable>arch</replaceable>
 179                     </entry>
 180                     <entry>Lustre back-end file system tools</entry>
 181                   </row>
 182                   <row>
 183                     <entry>
 184                       <code>lustre-zfs-ldiskfs-</code>
 185                       <replaceable>ver</replaceable>
 186                       <code>_lustre.</code>
 187                       <replaceable>arch</replaceable>
 188                     </entry>
 189                     <entry>Lustre back-end file system tools for ZFS. This is
 190                     an alternative to
 191                     <literal>lustre-osd-ldiskfs</literal></entry>
 192                   </row>
 193                   <row>
 194                     <entry>
 195                       <code>e2fsprogs</code>
 196                     </entry>
 197                     <entry>Utility to maintain Lustre back-end file
 198                     system</entry>
 199                   </row>
 200                   <row>
 201                     <entry>
 202                       <code>lustre-tests-</code>
 203                       <replaceable>ver</replaceable>
 204                       <code>_lustre.</code>
 205                       <replaceable>arch</replaceable>
 206                     </entry>
 207                     <entry>Lustre I/O Kit benchmarking tools
 208                     <emphasis role="italic">(Included in Lustre software as of
 209                     release 2.2)</emphasis></entry>
 210                   </row>
 211                 </tbody>
 212               </tgroup>
 213             </table>
 214           </para>
 215         </listitem>
 216         <listitem>
 217           <para>
 218           <emphasis role="bold">
 219             <emphasis role="italic">Lustre client packages</emphasis>
 220           </emphasis>. The required packages for Lustre clients are listed in
 221           the table below, where
 222           <replaceable>ver</replaceable>refers to the Linux distribution (e.g.,
 223           2.6.18-348.1.1.el5). These packages are available in the
 224           <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">
 225           Lustre Releases</link>repository.</para>
 226           <para>
 227             <table frame="all" xml:id="table_j3r_ym3_gk">
 228               <title>Packages Installed on Lustre Clients</title>
 229               <tgroup cols="2">
 230                 <colspec colname="c1" colnum="1" colwidth="1*" />
 231                 <colspec colname="c2" colnum="2" colwidth="1.02*" />
 232                 <thead>
 233                   <row>
 234                     <entry>Package Name</entry>
 235                     <entry>Description</entry>
 236                   </row>
 237                 </thead>
 238                 <tbody>
 239                   <row>
 240                     <entry>
 241                       <code>lustre-client-modules-</code>
 242                       <replaceable>ver</replaceable>
 243                     </entry>
 244                     <entry>Patchless kernel modules for client</entry>
 245                   </row>
 246                   <row>
 247                     <entry>
 248                       <code>lustre-client-</code>
 249                       <replaceable>ver</replaceable>
 250                       <code>_lustre</code>
 251                     </entry>
 252                     <entry>Client command line tools</entry>
 253                   </row>
 254                   <row>
 255                     <entry>
 256                       <code>lustre-client-tests-</code>
 257                       <replaceable>ver</replaceable>
 258                     </entry>
 259                     <entry>Lustre I/O Kit
 260                     <emphasis role="italic">(Included in Lustre software as of
 261                     release 2.2)</emphasis></entry>
 262                   </row>
 263                 </tbody>
 264               </tgroup>
 265             </table>
 266           </para>
 267           <note>
 268             <para>The version of the kernel running on a Lustre client must be
 269             the same as the version of the
 270             <code>lustre-client-modules-</code>
 271             <replaceable>ver</replaceable>package being installed. If the
 272             kernel running on the client is not compatible, a kernel that is
 273             compatible must be installed on the client before the Lustre file
 274             system software is installed.</para>
 275           </note>
 276         </listitem>
 277         <listitem>
 278           <para>
 279           <emphasis role="bold">
 280             <emphasis role="italic">Lustre LNET network driver (LND)</emphasis>
 281           </emphasis>. The Lustre LNDs provided with the Lustre software are
 282           listed in the table below. For more information about Lustre LNET,
 283           see
 284           <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 285           linkend="understandinglustrenetworking" />.</para>
 286           <table frame="all" xml:id="table_bkx_dn3_gk">
 287             <title>Network Types Supported by Lustre LNDs</title>
 288             <tgroup cols="2">
 289               <colspec colname="c1" colnum="1" colwidth="1*" />
 290               <colspec colname="newCol2" colnum="2" colwidth="1.95*" />
 291               <thead>
 292                 <row>
 293                   <entry>Supported Network Types</entry>
 294                   <entry>Notes</entry>
 295                 </row>
 296               </thead>
 297               <tbody>
 298                 <row>
 299                   <entry>TCP</entry>
 300                   <entry>Any network carrying TCP traffic, including GigE,
 301                   10GigE, and IPoIB</entry>
 302                 </row>
 303                 <row>
 304                   <entry>InfiniBand network</entry>
 305                   <entry>OpenFabrics OFED (o2ib)</entry>
 306                 </row>
 307                 <row>
 308                   <entry>gni</entry>
 309                   <entry>Gemini (Cray)</entry>
 310                 </row>
 311                 <row>
 312                   <entry>Seastar</entry>
 313                   <entry>Cray</entry>
 314                 </row>
 315                 <row>
 316                   <entry>MX</entry>
 317                   <entry>Myrinet network</entry>
 318                 </row>
 319                 <row>
 320                   <entry>ra</entry>
 321                   <entry>RapidArray
 322                   <superscript>*</superscript>interconnect</entry>
 323                 </row>
 324                 <row>
 325                   <entry>Elan</entry>
 326                   <entry>Quadrics</entry>
 327                 </row>
 328               </tbody>
 329             </tgroup>
 330           </table>
 331         </listitem>
 332       </itemizedlist></para>
 333       <para>
 334         <note>
 335           <para>The InfiniBand and TCP Lustre LNDs are routinely tested during
 336           release cycles. The other LNDs are maintained by their respective
 337           owners</para>
 338         </note>
 339         <itemizedlist>
 340           <listitem>
 341             <para>
 342             <emphasis role="bold">
 343               <emphasis role="italic">High availability software</emphasis>
 344             </emphasis>. If needed, install third party high-availability
 345             software. For more information, see
 346             <xref linkend="dbdoclet.50438188_92688" />.</para>
 347           </listitem>
 348           <listitem>
 349             <para>
 350             <emphasis role="bold">
 351               <emphasis role="italic">Optional packages.</emphasis>
 352             </emphasis>Optional packages provided in the
 353             <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">
 354             Lustre Releases</link>repository may include the following
 355             (depending on the operating system and platform):
 356             <itemizedlist>
 357               <listitem>
 358                 <para>
 359                 <code>kernel-debuginfo</code>,
 360                 <code>kernel-debuginfo-common</code>,
 361                 <code>lustre-debuginfo</code>,
 362                 <code>lustre-osd-ldiskfs-debuginfo</code>- Versions of required
 363                 packages with debugging symbols and other debugging options
 364                 enabled for use in troubleshooting.</para>
 365               </listitem>
 366               <listitem>
 367                 <para>
 368                 <code>kernel-devel</code>, - Portions of the kernel tree needed
 369                 to compile third party modules, such as network drivers.</para>
 370               </listitem>
 371               <listitem>
 372                 <para>
 373                 <code>kernel-firmware</code>- Standard Red Hat Enterprise Linux
 374                 distribution that has been recompiled to work with the Lustre
 375                 kernel.</para>
 376               </listitem>
 377               <listitem>
 378                 <para>
 379                 <code>kernel-headers</code>- Header files installed under
 380                 /user/include and used when compiling user-space,
 381                 kernel-related code.</para>
 382               </listitem>
 383               <listitem>
 384                 <para>
 385                 <code>lustre-source</code>- Lustre software source code.</para>
 386               </listitem>
 387               <listitem>
 388                 <para>
 389                 <emphasis role="italic">(Recommended)</emphasis>
 390                 <code>perf</code>,
 391                 <code>perf-debuginfo</code>,
 392                 <code>python-perf</code>,
 393                 <code>python-perf-debuginfo</code>- Linux performance analysis
 394                 tools that have been compiled to match the Lustre kernel
 395                 version.</para>
 396               </listitem>
 397             </itemizedlist></para>
 398           </listitem>
 399         </itemizedlist>
 400       </para>
 401     </section>
 402     <section xml:id="section_rh2_d4w_gk">
 403       <title>Environmental Requirements</title>
 404       <para>Before installing the Lustre software, make sure the following
 405       environmental requirements are met.
 406       <itemizedlist>
 407         <listitem>
 408           <para>
 409           <emphasis role="italic">(Required)</emphasis>
 410           <emphasis role="italic">
 411             <emphasis role="bold">Disable Security-Enhanced Linux
 412             <superscript>*</superscript>(SELinux) on all Lustre servers and
 413             clients.</emphasis>
 414           </emphasis>The Lustre software does not support SELinux. Therefore,
 415           the SELinux system extension must be disabled on all Lustre nodes.
 416           Also, make sure other security extensions (such as the Novell
 417           AppArmor
 418           <superscript>*</superscript>security system) and network packet
 419           filtering tools (such as iptables) do not interfere with the Lustre
 420           software.</para>
 421         </listitem>
 422         <listitem>
 423           <para>
 424           <emphasis role="italic">(Required)</emphasis>
 425           <emphasis role="italic">
 426             <emphasis role="bold">Use the same user IDs (UID) and group IDs
 427             (GID) on all clients.</emphasis>
 428           </emphasis>If use of supplemental groups is required, see
 429           <xref linkend="dbdoclet.50438291_32926" />for information about
 430           supplementary user and group cache upcall (
 431           <code>identity_upcall</code>).</para>
 432         </listitem>
 433         <listitem>
 434           <para>
 435           <emphasis role="italic">(Recommended)</emphasis>
 436           <emphasis role="italic">
 437             <emphasis role="bold">Provide remote shell access to
 438             clients.</emphasis>
 439           </emphasis>It is recommended that all cluster nodes have remote shell
 440           client access to facilitate the use of Lustre configuration and
 441           monitoring scripts. Parallel Distributed SHell (pdsh) is preferable,
 442           although Secure SHell (SSH) is acceptable.</para>
 443         </listitem>
 444         <listitem>
 445           <para>
 446           <emphasis role="italic">(Recommended)</emphasis>
 447           <emphasis role="italic">
 448             <emphasis role="bold">Ensure client clocks are
 449             synchronized.</emphasis>
 450           </emphasis>The Lustre file system uses client clocks for timestamps.
 451           If clocks are out of sync between clients, files will appear with
 452           different time stamps when accessed by different clients. Drifting
 453           clocks can also cause problems by, for example, making it difficult
 454           to debug multi-node issues or correlate logs, which depend on
 455           timestamps. We recommend that you use Network Time Protocol (NTP) to
 456           keep client and server clocks in sync with each other. For more
 457           information about NTP, see:
 458           <link xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
 459         </listitem>
 460       </itemizedlist></para>
 461     </section>
 462   </section>
 463   <section xml:id="dbdoclet.50438261_41900">
 464     <title>Lustre Software Installation Procedure</title>
 465     <caution>
 466       <para>Before installing the Lustre software, back up ALL data. The Lustre
 467       software contains kernel modifications that interact with storage devices
 468       and may introduce security issues and data loss if not installed,
 469       configured, or administered properly.</para>
 470     </caution>
 471     <para>To install the Lustre software from RPMs, complete the steps
 472     below.</para>
 473     <orderedlist>
 474       <listitem>
 475         <para>Verify that all Lustre installation requirements have been
 476         met.</para>
 477         <itemizedlist>
 478           <listitem>
 479             <para>For hardware requirements, see
 480             <xref linkend="settinguplustresystem" />.</para>
 481           </listitem>
 482           <listitem>
 483             <para>For software and environmental requirements, see the section
 484             <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 485             linkend="dbdoclet.50438261_99193" />above.</para>
 486           </listitem>
 487         </itemizedlist>
 488       </listitem>
 489       <listitem>
 490         <para>Download the
 491         <literal>e2fsprogs</literal> RPMs for your platform from the
 492         <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">
 493         Lustre Releases</link>repository.</para>
 494       </listitem>
 495       <listitem>
 496         <para>Download the Lustre server RPMs for your platform from the
 497         <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">
 498         Lustre Releases</link>repository. See
 499         <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 500         linkend="table_cnh_5m3_gk" />for a list of required packages.</para>
 501       </listitem>
 502       <listitem xml:id="dbdoclet.50438261_88296">
 503         <para>Install the Lustre server and
 504         <literal>e2fsprogs</literal> packages on all Lustre servers (MGS, MDSs,
 505         and OSSs).
 506         <orderedlist numeration="loweralpha">
 507           <listitem>
 508             <para>Log onto a Lustre server as the
 509             <literal>root</literal> user</para>
 510           </listitem>
 511           <listitem>
 512             <para>Use the
 513             <literal>yum</literal> command to install the packages:</para>
 514             <para>
 515               <screen>
 516 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
 517 </screen>
 518             </para>
 519           </listitem>
 520           <listitem>
 521             <para>Verify the packages are installed correctly:</para>
 522             <para>
 523               <screen>
 524 rpm -qa|egrep "lustre|wc"|sort
 525 </screen>
 526             </para>
 527           </listitem>
 528           <listitem>
 529             <para>Reboot the server.</para>
 530           </listitem>
 531           <listitem>
 532             <para>Repeat these steps on each Lustre server.</para>
 533           </listitem>
 534         </orderedlist></para>
 535       </listitem>
 536       <listitem>
 537         <para>Download the Lustre client RPMs for your platform from the
 538         <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">
 539         Lustre Releases</link>repository. See
 540         <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 541         linkend="table_j3r_ym3_gk" />for a list of required packages.</para>
 542       </listitem>
 543       <listitem>
 544         <para>Install the Lustre client packages on all Lustre clients.
 545         <note>
 546           <para>The version of the kernel running on a Lustre client must be
 547           the same as the version of the
 548           <literal>lustre-client-modules-</literal>
 549           <replaceable>ver</replaceable>package being installed. If not, a
 550           compatible kernel must be installed on the client before the Lustre
 551           client packages are installed.</para>
 552         </note></para>
 553         <orderedlist numeration="loweralpha">
 554           <listitem>
 555             <para>Log onto a Lustre client as the root user.</para>
 556           </listitem>
 557           <listitem>
 558             <para>Use the
 559             <literal>yum</literal> command to install the packages:</para>
 560             <para>
 561               <screen>
 562 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
 563 </screen>
 564             </para>
 565           </listitem>
 566           <listitem>
 567             <para>Verify the packages were installed correctly:</para>
 568             <para>
 569               <screen>
 570 # rpm -qa|egrep "lustre|kernel"|sort
 571 </screen>
 572             </para>
 573           </listitem>
 574           <listitem>
 575             <para>Reboot the client.</para>
 576           </listitem>
 577           <listitem>
 578             <para>Repeat these steps on each Lustre client.</para>
 579           </listitem>
 580         </orderedlist>
 581       </listitem>
 582     </orderedlist>
 583     <para>To configure LNET, go to
 584     <xref linkend="configuringlnet" />. If default settings will be used for
 585     LNET, go to
 586     <xref linkend="configuringlustre" />.</para>
 587   </section>
 588 </chapter>