InstallingLustre.xml

   1 <?xml version='1.0' encoding='UTF-8'?>
   2 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0"
   3   xml:lang="en-US" xml:id="installinglustre">
   4   <title xml:id="installinglustre.title">Installing the Lustre* Software</title>
   5   <para>This chapter describes how to install the Lustre* software from RPM packages. It
   6     includes:</para>
   7   <itemizedlist>
   8     <listitem>
   9       <para>
  10         <xref linkend="dbdoclet.50438261_99193"/>
  11       </para>
  12     </listitem>
  13     <listitem>
  14       <para>
  15         <xref linkend="dbdoclet.50438261_41900"/>
  16       </para>
  17     </listitem>
  18   </itemizedlist>
  19   <para>For hardware and system requirements and hardware configuration information, see <xref
  20       linkend="settinguplustresystem"/>.</para>
  21   <section xml:id="dbdoclet.50438261_99193">
  22     <title>
  23       <indexterm>
  24         <primary>installing</primary>
  25         <secondary>preparation</secondary>
  26       </indexterm> Preparing to Install the Lustre Software</title>
  27     <para>You can install the Lustre software from downloaded packages (RPMs) or directly from the
  28       source code. This chapter describes how to install the Lustre RPM packages. For information
  29       about installing from source code, see <xref linkend="installinglustrefromsourcecode"
  30       />.</para>
  31     <para>The Lustre RPM packages have been tested on the Linux distributions listed in the table
  32       below.</para>
  33     <para>
  34       <table frame="all" xml:id="LustreTestMatrixTable">
  35         <title>Lustre Test Matrix</title>
  36         <tgroup cols="3">
  37           <colspec colname="c1" colnum="1" colwidth="1*"/>
  38           <colspec colname="c2" colnum="2" colwidth="1.1*"/>
  39           <colspec colname="c3" colnum="3" colwidth="1.46*"/>
  40           <thead>
  41             <row>
  42               <entry>Lustre Version</entry>
  43               <entry>Servers Tested<superscript>1</superscript></entry>
  44               <entry>Clients Tested</entry>
  45             </row>
  46           </thead>
  47           <tbody>
  48             <row>
  49               <entry>
  50                 <para>2.0</para>
  51               </entry>
  52               <entry>
  53                 <para>RHEL 5, CentOS 5</para>
  54               </entry>
  55               <entry>
  56                 <para>RHEL 5, CentOS 5, SLES 11 SP0</para>
  57               </entry>
  58             </row>
  59             <row>
  60               <entry>
  61                 <para>2.1.x</para>
  62               </entry>
  63               <entry>
  64                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6</para>
  65               </entry>
  66               <entry>
  67                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
  68               </entry>
  69             </row>
  70             <row>
  71               <entry>
  72                 <para>2.2</para>
  73               </entry>
  74               <entry>
  75                 <para>RHEL 6, CentOS 6</para>
  76               </entry>
  77               <entry>
  78                 <para>RHEL 5, CentOS 5, RHEL 6, CentOS 6, SLES 11 SP1</para>
  79               </entry>
  80             </row>
  81             <row>
  82               <entry>
  83                 <para>2.3</para>
  84               </entry>
  85               <entry>
  86                 <para>RHEL 6.3, CentOS 6.3</para>
  87               </entry>
  88               <entry>
  89                 <para>RHEL 6.3, CentOS 6.3, RHEL 5.8, CentOS 5.8, SLES 11 SP1</para>
  90               </entry>
  91             </row>
  92             <row>
  93               <entry>2.4.x</entry>
  94               <entry>
  95                 <para>RHEL 6.4, CentOS 6.4</para>
  96               </entry>
  97               <entry>
  98                 <para>RHEL 6.4, CentOS 6.4, SLES 11 SP2, FC18</para>
  99               </entry>
 100             </row>
 101           </tbody>
 102         </tgroup>
 103       </table>
 104     </para>
 105     <para><superscript>1</superscript>Red Hat* Enterprise Edition*, CentOS Enterprise Linux
 106       Distribution, SUSE Linux Enterprise Server, Fedora* F18 Linux kernel.</para>
 107     <section xml:id="section_rqs_tjw_3k">
 108       <title>Software Requirements</title>
 109       <para>To install the Lustre software from RPMs, the following are required:<itemizedlist>
 110           <listitem>
 111             <para><emphasis role="bold"><emphasis role="italic">Lustre server
 112                 packages</emphasis></emphasis>. The required packages for Lustre servers are listed
 113               in the table below, where <replaceable>ver</replaceable> refers to the Linux* kernel
 114               distribution (e.g., 2.6.32-358.6.2.el6) and <replaceable>arch</replaceable> refers to
 115               the processor architecture (e.g., x86_64). These packages are available in the <link
 116                 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
 117                 Releases</link> repository.</para>
 118             <para>
 119               <table frame="all" xml:id="table_cnh_5m3_gk">
 120                 <title>Packages Installed on Lustre Servers</title>
 121                 <tgroup cols="2">
 122                   <colspec colname="c1" colnum="1" colwidth="1.18*"/>
 123                   <colspec colname="c2" colnum="2" colwidth="1*"/>
 124                   <thead>
 125                     <row>
 126                       <entry>Package Name*</entry>
 127                       <entry>Description</entry>
 128                     </row>
 129                   </thead>
 130                   <tbody>
 131                     <row>
 132                       <entry><code>kernel-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 133                       <entry>Linux kernel with Lustre patches (often referred to as "patched
 134                         kernel")</entry>
 135                     </row>
 136                     <row>
 137                       <entry><code>lustre-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 138                       <entry>Lustre command line tools</entry>
 139                     </row>
 140                     <row>
 141                       <entry><code>lustre-modules-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 142                       <entry>Lustre-patched kernel modules</entry>
 143                     </row>
 144                     <row>
 145                       <entry><code>lustre-ldiskfs-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 146                       <entry>Lustre back-end file system tools</entry>
 147                     </row>
 148                     <row>
 149                       <entry><code>e2fsprogs</code></entry>
 150                       <entry>Utility to maintain Lustre back-end file system</entry>
 151                     </row>
 152                     <row>
 153                       <entry><code>lustre-tests-</code><replaceable>ver</replaceable><code>_lustre.</code><replaceable>arch</replaceable></entry>
 154                       <entry>Lustre I/O Kit benchmarking tools<emphasis role="italic"> (Included in
 155                           Lustre software as of Version 2.2)</emphasis></entry>
 156                     </row>
 157                   </tbody>
 158                 </tgroup>
 159               </table>
 160             </para>
 161           </listitem>
 162           <listitem>
 163             <para><emphasis role="bold"><emphasis role="italic">Lustre client
 164                 packages</emphasis></emphasis>. The required packages for Lustre clients are listed
 165               in the table below, where <replaceable>ver</replaceable> refers to the Linux
 166               distribution (e.g., 2.6.18-348.1.1.el5). These packages are available in the <link
 167                 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
 168                 Releases</link> repository.</para>
 169             <para>
 170               <table frame="all" xml:id="table_j3r_ym3_gk">
 171                 <title>Packages Installed in Lustre Clients</title>
 172                 <tgroup cols="2">
 173                   <colspec colname="c1" colnum="1" colwidth="1*"/>
 174                   <colspec colname="c2" colnum="2" colwidth="1.02*"/>
 175                   <thead>
 176                     <row>
 177                       <entry>Package Name</entry>
 178                       <entry>Description</entry>
 179                     </row>
 180                   </thead>
 181                   <tbody>
 182                     <row>
 183                       <entry><code>lustre-client-modules-</code><replaceable>ver</replaceable></entry>
 184                       <entry>Patchless kernel modules for client</entry>
 185                     </row>
 186                     <row>
 187                       <entry><code>lustre-client-</code><replaceable>ver</replaceable><code>_lustre</code></entry>
 188                       <entry>Client command line tools</entry>
 189                     </row>
 190                     <row>
 191                       <entry><code>lustre-client-tests-</code><replaceable>ver</replaceable></entry>
 192                       <entry>Lustre I/O Kit <emphasis role="italic">(Included in Lustre software as
 193                           of Version 2.2)</emphasis>
 194                       </entry>
 195                     </row>
 196                   </tbody>
 197                 </tgroup>
 198               </table>
 199             </para>
 200             <note>
 201               <para>The version of the kernel running on a Lustre client must be the same as the
 202                 version of the <code>lustre-client-modules-</code><replaceable>ver</replaceable>
 203                 package being installed. If the kernel running on the client is not compatible, a
 204                 kernel that is compatible must be installed on the client before the Lustre file
 205                 system software is installed.</para>
 206             </note>
 207           </listitem>
 208           <listitem>
 209             <para><emphasis role="bold"><emphasis role="italic">Lustre LNET network driver
 210                   (LND)</emphasis></emphasis>. The Lustre LNDs provided with the Lustre software are
 211               listed in the table below. For more information about Lustre LNET, see <xref
 212                 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="understandinglustrenetworking"
 213               />.</para>
 214             <table frame="all" xml:id="table_bkx_dn3_gk">
 215               <title>Network Types Supported by Lustre LNDs</title>
 216               <tgroup cols="2">
 217                 <colspec colname="c1" colnum="1" colwidth="1*"/>
 218                 <colspec colname="newCol2" colnum="2" colwidth="1.95*"/>
 219                 <thead>
 220                   <row>
 221                     <entry>Supported Network Types</entry>
 222                     <entry>Notes</entry>
 223                   </row>
 224                 </thead>
 225                 <tbody>
 226                   <row>
 227                     <entry>TCP</entry>
 228                     <entry>Any network carrying TCP traffic, including GigE, 10GigE, and
 229                       IPoIB</entry>
 230                   </row>
 231                   <row>
 232                     <entry>InfiniBand* network</entry>
 233                     <entry>OpenFabrics OFED (o2ib)</entry>
 234                   </row>
 235                   <row>
 236                     <entry>gni</entry>
 237                     <entry>Gemini (Cray)</entry>
 238                   </row>
 239                   <row>
 240                     <entry>Seastar</entry>
 241                     <entry>Cray</entry>
 242                   </row>
 243                   <row>
 244                     <entry>MX</entry>
 245                     <entry>Myrinet* network</entry>
 246                   </row>
 247                   <row>
 248                     <entry>ra</entry>
 249                     <entry>RapidArray* interconnect</entry>
 250                   </row>
 251                   <row>
 252                     <entry>Elan</entry>
 253                     <entry>Quadrics</entry>
 254                   </row>
 255                 </tbody>
 256               </tgroup>
 257             </table>
 258           </listitem>
 259         </itemizedlist></para>
 260       <para>
 261         <note>
 262           <para>The InfiniBand* and TCP Lustre LNDs are routinely tested during release cycles. The
 263             other LNDs are maintained by their respective owners</para>
 264         </note>
 265         <itemizedlist>
 266           <listitem>
 267             <para><emphasis role="bold"><emphasis role="italic"> High availability
 268                   software</emphasis></emphasis>. If needed, install third party high-availability
 269               software. For more information, see <xref linkend="dbdoclet.50438188_92688"/>.</para>
 270           </listitem>
 271           <listitem>
 272             <para><emphasis role="bold"><emphasis role="italic">Optional
 273                 packages.</emphasis></emphasis> Optional packages  provided in the <link
 274                 xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
 275                 Releases</link> repository may include the following (depending on the operating
 276               system and platform):<itemizedlist>
 277                 <listitem>
 278                   <para><code>kernel-debuginfo</code>, <code>kernel-debuginfo-common</code>,
 279                       <code>lustre-debuginfo</code>, <code>lustre-ldiskfs-debuginfo</code> -
 280                     Versions of required packages with debugging symbols and other debugging options
 281                     enabled for use in troubleshooting.</para>
 282                 </listitem>
 283                 <listitem>
 284                   <para><code>kernel-devel</code>,  - Portions of the kernel tree needed to compile
 285                     third party modules, such as network drivers.</para>
 286                 </listitem>
 287                 <listitem>
 288                   <para><code>kernel-firmware</code> - Standard Red Hat Enterprise Linux package
 289                     that has been recompiled to work with the Lustre kernel.</para>
 290                 </listitem>
 291                 <listitem>
 292                   <para><code>kernel-headers</code> - Header files installed under /user/include and
 293                     used when compiling user-space, kernel-related code.</para>
 294                 </listitem>
 295                 <listitem>
 296                   <para><code>lustre-source</code> - Source code for Lustre.</para>
 297                 </listitem>
 298                 <listitem>
 299                   <para><emphasis role="italic">(Recommended) </emphasis><code>perf</code>,
 300                       <code>perf-debuginfo</code>, <code>python-perf</code>,
 301                       <code>python-perf-debuginfo</code> -  Linux performance analysis tools that
 302                     have been compiled to match the Lustre kernel version.</para>
 303                 </listitem>
 304               </itemizedlist></para>
 305           </listitem>
 306         </itemizedlist>
 307       </para>
 308     </section>
 309     <section xml:id="section_rh2_d4w_gk">
 310       <title>Environmental Requirements</title>
 311       <para>Before installing the Lustre software, make sure the following environmental
 312         requirements are met.<itemizedlist>
 313           <listitem>
 314             <para><emphasis role="italic">(Required)</emphasis>
 315               <emphasis role="italic"><emphasis role="bold">Disable Security-Enhanced Linux
 316                   (SELinux) on all Lustre servers and clients.</emphasis></emphasis> The Lustre
 317               software does not support SELinux. Therefore, the SELinux system extension must be
 318               disabled on all Lustre nodes. Also, make sure other security extensions (such as the
 319               Novell AppArmor* security system) and network packet filtering tools (such as
 320               iptables) do not interfere with the Lustre software.</para>
 321           </listitem>
 322           <listitem>
 323             <para><emphasis role="italic">(Required) </emphasis><emphasis role="italic"><emphasis
 324                   role="bold">Use the same user IDs (UID) and group IDs (GID) on all
 325                   clients.</emphasis></emphasis> If use of supplemental groups is required, see
 326                 <xref linkend="dbdoclet.50438291_32926"/> for information about supplementary user
 327               and group cache upcall (<code>identity_upcall</code>).</para>
 328           </listitem>
 329           <listitem>
 330             <para><emphasis role="italic">(Recommended) </emphasis><emphasis role="italic"><emphasis
 331                   role="bold">Provide remote shell access to clients.</emphasis></emphasis> It is
 332               recommended that all cluster nodes have remote shell client access to facilitate the
 333               use of Lustre configuration and monitoring scripts. Parallel Distributed SHell (pdsh)
 334               is preferable, although Secure SHell (SSH) is acceptable.</para>
 335           </listitem>
 336           <listitem>
 337             <para><emphasis role="italic">(Recommended) </emphasis><emphasis role="italic"><emphasis
 338                   role="bold">Ensure client clocks are synchronized.</emphasis></emphasis> The
 339               Lustre file system uses client clocks for timestamps. If clocks are out of sync
 340               between clients, files will appear with different time stamps when accessed by
 341               different clients. Drifting clocks can also cause problems by, for example, making it
 342               difficult to debug multi-node issues or correlate logs, which depend on timestamps. We
 343               recommend that you use Network Time Protocol (NTP) to keep client and server clocks in
 344               sync with each other. For more information about NTP, see: <link
 345                 xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
 346           </listitem>
 347         </itemizedlist></para>
 348     </section>
 349   </section>
 350   <section xml:id="dbdoclet.50438261_41900">
 351     <title>Lustre Installation Procedure</title>
 352     <caution>
 353       <para>Before installing the Lustre software, back up ALL data. The Lustre software contains
 354         kernel modifications that interact with storage devices and may introduce security issues
 355         and data loss if not installed, configured, or administered properly.</para>
 356     </caution>
 357     <para>To install the Lustre software from RPMs, complete the steps below.</para>
 358     <orderedlist>
 359       <listitem>
 360         <para>Verify that all Lustre installation requirements have been met. </para>
 361         <itemizedlist>
 362           <listitem>
 363             <para>For hardware requirements, see <xref linkend="settinguplustresystem"/>.</para>
 364           </listitem>
 365           <listitem>
 366             <para>For software and environmental requirements, see the section <xref
 367                 xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438261_99193"
 368               />above.</para>
 369           </listitem>
 370         </itemizedlist>
 371       </listitem>
 372       <listitem>
 373                 <para>Download the <literal>e2fsprogs</literal> RPMs for your platform from the <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
 374           repository.</para>
 375       </listitem>
 376       <listitem>
 377         <para>Download the Lustre server RPMs for your platform from the <link
 378             xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
 379           repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 380             linkend="table_cnh_5m3_gk"/> for a list of required packages.</para>
 381       </listitem>
 382       <listitem xml:id="dbdoclet.50438261_88296">
 383         <para>Install the Lustre server and <literal>e2fsprogs</literal> packages on all Lustre servers (MGS, MDSs, and
 384             OSSs).<orderedlist numeration="loweralpha">
 385             <listitem>
 386               <para>Log onto a Lustre server as the <literal>root</literal> user</para>
 387             </listitem>
 388             <listitem>
 389               <para>Use the <literal>yum</literal> command to install the packages:</para>
 390               <para>
 391                 <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
 392               </para>
 393             </listitem>
 394             <listitem>
 395               <para>Verify the packages are installed correctly:</para>
 396               <para>
 397                 <screen>rpm -qa|egrep "lustre|wc"|sort</screen>
 398               </para>
 399             </listitem>
 400             <listitem>
 401               <para>Reboot the server.</para>
 402             </listitem>
 403             <listitem>
 404               <para>Repeat these steps on each Lustre server.</para>
 405             </listitem>
 406           </orderedlist></para>
 407       </listitem>
 408       <listitem>
 409         <para>Download the Lustre client RPMs for your platform from the <link
 410             xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
 411           repository. See <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 412             linkend="table_j3r_ym3_gk"/> for a list of required packages.</para>
 413       </listitem>
 414       <listitem>
 415         <para>Install the Lustre client packages on all Lustre clients.<note>
 416             <para>The version of the kernel running on a Lustre client must be the same as the
 417               version of the <literal>lustre-client-modules-</literal><replaceable>ver</replaceable>
 418               package being installed. If not, a compatible kernel must be installed on the client
 419               before the Lustre client packages are installed.</para>
 420           </note></para>
 421         <orderedlist numeration="loweralpha">
 422           <listitem>
 423             <para>Log onto a Lustre client as the root user.</para>
 424           </listitem>
 425           <listitem>
 426             <para>Use the <literal>yum</literal> command to install the packages:</para>
 427             <para>
 428               <screen># yum --nogpgcheck install pkg1.rpm pkg2.rpm ...</screen>
 429             </para>
 430           </listitem>
 431           <listitem>
 432             <para>Verify the packages were installed correctly:</para>
 433             <para>
 434               <screen># rpm -qa|egrep "lustre|kernel"|sort</screen>
 435             </para>
 436           </listitem>
 437           <listitem>
 438             <para>Reboot the client.</para>
 439           </listitem>
 440           <listitem>
 441             <para>Repeat these steps on each Lustre client.</para>
 442           </listitem>
 443         </orderedlist>
 444       </listitem>
 445     </orderedlist>
 446     <para>To configure LNET, go  to <xref linkend="configuringlnet"/>. If default settings will be
 447       used for LNET, go to <xref linkend="configuringlustre"/>.</para>
 448   </section>
 449 </chapter>