InstallingLustre.xml

   1 <?xml version='1.0' encoding='utf-8'?>
   2 <chapter xmlns="http://docbook.org/ns/docbook"
   3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
   4 xml:id="installinglustre">
   5   <title xml:id="installinglustre.title">Installing the Lustre Software</title>
   6   <para>This chapter describes how to install the Lustre software from RPM
   7   packages. It includes:</para>
   8   <itemizedlist>
   9     <listitem>
  10       <para>
  11         <xref linkend="dbdoclet.50438261_99193" />
  12       </para>
  13     </listitem>
  14     <listitem>
  15       <para>
  16         <xref linkend="dbdoclet.50438261_41900" />
  17       </para>
  18     </listitem>
  19   </itemizedlist>
  20   <para>For hardware and system requirements and hardware configuration
  21   information, see
  22   <xref linkend="settinguplustresystem" />.</para>
  23   <section xml:id="dbdoclet.50438261_99193">
  24     <title>
  25     <indexterm>
  26       <primary>installing</primary>
  27       <secondary>preparation</secondary>
  28     </indexterm>Preparing to Install the Lustre Software</title>
  29     <para>You can install the Lustre software from downloaded packages (RPMs)
  30     or directly from the source code. This chapter describes how to install the
  31     Lustre RPM packages. Instructions to install from source code are beyond
  32     the scope of this document, and can be found elsewhere online.</para>
  33     <para>The Lustre RPM packages are tested on current versions of Linux
  34     enterprise distributions at the time they are created.  See the release
  35     notes for each version for specific details.</para>
  36     <section xml:id="section_rqs_tjw_3k">
  37       <title>Software Requirements</title>
  38       <para>To install the Lustre software from RPMs, the following are
  39       required:
  40       <itemizedlist>
  41         <listitem>
  42           <para>
  43           <emphasis role="bold">
  44             <emphasis role="italic">Lustre server packages</emphasis>
  45           </emphasis>. The required packages for Lustre 2.9 EL7 servers are
  46           listed in the table below, where
  47           <replaceable>ver</replaceable> refers to the Lustre release and
  48           kernel version (e.g., 2.9.0-1.el7) and
  49           <replaceable>arch</replaceable> refers to the processor architecture
  50           (e.g., x86_64). These packages are available in the
  51           <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
  52           Lustre Releases</link> repository, and may differ depending on
  53           your distro and version.</para>
  54           <para>
  55             <table frame="all" xml:id="table_cnh_5m3_gk">
  56               <title>Packages Installed on Lustre Servers</title>
  57               <tgroup cols="2">
  58                 <colspec colname="c1" colnum="1" colwidth="1.18*" />
  59                 <colspec colname="c2" colnum="2" colwidth="1*" />
  60                 <thead>
  61                   <row>
  62                     <entry>Package Name</entry>
  63                     <entry>Description</entry>
  64                   </row>
  65                 </thead>
  66                 <tbody>
  67                   <row>
  68                     <entry>
  69                       <code>kernel-<replaceable>ver</replaceable>_lustre.<replaceable>arch</replaceable></code>
  70                     </entry>
  71                     <entry>Linux kernel with Lustre software patches (often
  72                     referred to as "patched kernel")</entry>
  73                   </row>
  74                   <row>
  75                     <entry>
  76                       <code>lustre-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  77                     </entry>
  78                     <entry>Lustre software command line tools</entry>
  79                   </row>
  80                   <row>
  81                     <entry>
  82                       <code>kmod-lustre-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  83                     </entry>
  84                     <entry>Lustre-patched kernel modules</entry>
  85                   </row>
  86                   <row>
  87                     <entry>
  88                       <code>kmod-lustre-osd-ldiskfs-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  89                     </entry>
  90                     <entry>Lustre back-end file system tools for ldiskfs-based
  91                     servers.
  92                     </entry>
  93                   </row>
  94                   <row>
  95                     <entry>
  96                       <code>lustre-osd-ldiskfs-mount-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  97                     </entry>
  98                     <entry>Helper library for <literal>mount.lustre</literal>
  99                     and <literal>mkfs.lustre</literal> for ldiskfs-based servers.
 100                     </entry>
 101                   </row>
 102                   <row>
 103                     <entry>
 104                       <code>kmod-lustre-osd-zfs-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 105                     </entry>
 106                     <entry>Lustre back-end file system tools for ZFS. This is
 107                     an alternative to
 108                     <literal>lustre-osd-ldiskfs</literal> (kmod-spl and
 109                     kmod-zfs available separately).</entry>
 110                   </row>
 111                   <row>
 112                     <entry>
 113                       <code>lustre-osd-zfs-mount-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 114                     </entry>
 115                     <entry>Helper library for <literal>mount.lustre</literal>
 116                     and <literal>mkfs.lustre</literal> for ZFS-based servers
 117                     (zfs utilities available separately).
 118                     </entry>
 119                   </row>
 120                   <row>
 121                     <entry>
 122                       <code>e2fsprogs</code>
 123                     </entry>
 124                     <entry>Utilities to maintain Lustre ldiskfs back-end file
 125                     system(s)
 126                     </entry>
 127                   </row>
 128                   <row>
 129                     <entry>
 130                       <code>lustre-tests-<replaceable>ver</replaceable>_lustre.<replaceable>arch</replaceable></code>
 131                     </entry>
 132                     <entry>Lustre I/O Kit benchmarking tools
 133                     <emphasis role="italic">(Included in Lustre software as of
 134                     release 2.2)</emphasis></entry>
 135                   </row>
 136                 </tbody>
 137               </tgroup>
 138             </table>
 139           </para>
 140         </listitem>
 141         <listitem>
 142           <para>
 143           <emphasis role="bold">
 144             <emphasis role="italic">Lustre client packages</emphasis>
 145           </emphasis>. The required packages for Lustre 2.9 EL7 clients are
 146           listed in the table below, where
 147           <replaceable>ver</replaceable> refers to the Linux distribution (e.g.,
 148           3.6.18-348.1.1.el5). These packages are available in the
 149           <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 150           Lustre Releases</link> repository.</para>
 151           <para>
 152             <table frame="all" xml:id="table_j3r_ym3_gk">
 153               <title>Packages Installed on Lustre Clients</title>
 154               <tgroup cols="2">
 155                 <colspec colname="c1" colnum="1" colwidth="1*" />
 156                 <colspec colname="c2" colnum="2" colwidth="1.02*" />
 157                 <thead>
 158                   <row>
 159                     <entry>Package Name</entry>
 160                     <entry>Description</entry>
 161                   </row>
 162                 </thead>
 163                 <tbody>
 164                   <row>
 165                     <entry>
 166                       <code>kmod-lustre-client-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 167                     </entry>
 168                     <entry>Patchless kernel modules for client</entry>
 169                   </row>
 170                   <row>
 171                     <entry>
 172                       <code>lustre-client-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 173                     </entry>
 174                     <entry>Client command line tools</entry>
 175                   </row>
 176                   <row>
 177                     <entry>
 178                       <code>lustre-client-dkms-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 179                     </entry>
 180                     <entry>Alternate client RPM to kmod-lustre-client with
 181                     Dynamic Kernel Module Support (DKMS) installation. This
 182                     avoids the need to install a new RPM for each kernel update,
 183                     but requires a full build environment on the client.</entry>
 184                   </row>
 185                 </tbody>
 186               </tgroup>
 187             </table>
 188           </para>
 189           <note>
 190             <para>The version of the kernel running on a Lustre client must be
 191             the same as the version of the
 192             <code>kmod-lustre-client-<replaceable>ver</replaceable></code>
 193             package being installed, unless the DKMS package is installed.
 194             If the kernel running on the client is not compatible, a kernel
 195             that is compatible must be installed on the client before the
 196             Lustre file system software is used.</para>
 197           </note>
 198         </listitem>
 199         <listitem>
 200           <para>
 201           <emphasis role="bold">
 202             <emphasis role="italic">Lustre LNet network driver (LND)</emphasis>
 203           </emphasis>. The Lustre LNDs provided with the Lustre software are
 204           listed in the table below. For more information about Lustre LNet,
 205           see
 206           <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 207           linkend="understandinglustrenetworking" />.</para>
 208           <table frame="all" xml:id="table_bkx_dn3_gk">
 209             <title>Network Types Supported by Lustre LNDs</title>
 210             <tgroup cols="2">
 211               <colspec colname="c1" colnum="1" colwidth="1*" />
 212               <colspec colname="newCol2" colnum="2" colwidth="1.95*" />
 213               <thead>
 214                 <row>
 215                   <entry>Supported Network Types</entry>
 216                   <entry>Notes</entry>
 217                 </row>
 218               </thead>
 219               <tbody>
 220                 <row>
 221                   <entry>TCP</entry>
 222                   <entry>Any network carrying TCP traffic, including GigE,
 223                   10GigE, and IPoIB</entry>
 224                 </row>
 225                 <row>
 226                   <entry>InfiniBand network</entry>
 227                   <entry>OpenFabrics OFED (o2ib)</entry>
 228                 </row>
 229                 <row>
 230                   <entry>gni</entry>
 231                   <entry>Gemini (Cray)</entry>
 232                 </row>
 233               </tbody>
 234             </tgroup>
 235           </table>
 236         </listitem>
 237       </itemizedlist></para>
 238       <para>
 239         <note>
 240           <para>The InfiniBand and TCP Lustre LNDs are routinely tested during
 241           release cycles. The other LNDs are maintained by their respective
 242           owners</para>
 243         </note>
 244         <itemizedlist>
 245           <listitem>
 246             <para>
 247             <emphasis role="bold">
 248               <emphasis role="italic">High availability software</emphasis>
 249             </emphasis>. If needed, install third party high-availability
 250             software. For more information, see
 251             <xref linkend="dbdoclet.50438188_92688" />.</para>
 252           </listitem>
 253           <listitem>
 254             <para>
 255             <emphasis role="bold">
 256               <emphasis role="italic">Optional packages.</emphasis>
 257             </emphasis>Optional packages provided in the
 258             <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 259             Lustre Releases</link>repository may include the following
 260             (depending on the operating system and platform):
 261             <itemizedlist>
 262               <listitem>
 263                 <para>
 264                 <code>kernel-debuginfo</code>,
 265                 <code>kernel-debuginfo-common</code>,
 266                 <code>lustre-debuginfo</code>,
 267                 <code>lustre-osd-ldiskfs-debuginfo</code>- Versions of required
 268                 packages with debugging symbols and other debugging options
 269                 enabled for use in troubleshooting.</para>
 270               </listitem>
 271               <listitem>
 272                 <para>
 273                 <code>kernel-devel</code>, - Portions of the kernel tree needed
 274                 to compile third party modules, such as network drivers.</para>
 275               </listitem>
 276               <listitem>
 277                 <para>
 278                 <code>kernel-firmware</code>- Standard Red Hat Enterprise Linux
 279                 distribution that has been recompiled to work with the Lustre
 280                 kernel.</para>
 281               </listitem>
 282               <listitem>
 283                 <para>
 284                 <code>kernel-headers</code>- Header files installed under
 285                 /user/include and used when compiling user-space,
 286                 kernel-related code.</para>
 287               </listitem>
 288               <listitem>
 289                 <para>
 290                 <code>lustre-source</code>- Lustre software source code.</para>
 291               </listitem>
 292               <listitem>
 293                 <para>
 294                 <emphasis role="italic">(Recommended)</emphasis>
 295                 <code>perf</code>,
 296                 <code>perf-debuginfo</code>,
 297                 <code>python-perf</code>,
 298                 <code>python-perf-debuginfo</code>- Linux performance analysis
 299                 tools that have been compiled to match the Lustre kernel
 300                 version.</para>
 301               </listitem>
 302             </itemizedlist></para>
 303           </listitem>
 304         </itemizedlist>
 305       </para>
 306     </section>
 307     <section xml:id="section_rh2_d4w_gk">
 308       <title>Environmental Requirements</title>
 309       <para>Before installing the Lustre software, make sure the following
 310       environmental requirements are met.
 311       <itemizedlist>
 312         <listitem>
 313           <para>
 314           <emphasis role="italic">(Required)</emphasis>
 315           <emphasis role="italic">
 316             <emphasis role="bold">Disable Security-Enhanced Linux
 317             <superscript>*</superscript>(SELinux) on all Lustre servers
 318             </emphasis>
 319           </emphasis>The Lustre software does not support SELinux. Therefore,
 320           the SELinux system extension must be disabled on all Lustre nodes.
 321           Also, make sure other security extensions (such as the Novell
 322           AppArmor
 323           <superscript>*</superscript>security system) and network packet
 324           filtering tools (such as iptables) do not interfere with the Lustre
 325           software.</para>
 326         </listitem>
 327         <listitem>
 328           <para>
 329           <emphasis role="italic">(Required)</emphasis>
 330           <emphasis role="italic">
 331             <emphasis role="bold">Use the same user IDs (UID) and group IDs
 332             (GID) on all clients.</emphasis>
 333           </emphasis>If use of supplemental groups is required, see
 334           <xref linkend="dbdoclet.identity_upcall" /> for information about
 335           supplementary user and group cache upcall (<code>identity_upcall</code>).</para>
 336         </listitem>
 337         <listitem>
 338           <para>
 339           <emphasis role="italic">(Recommended)</emphasis>
 340           <emphasis role="italic">
 341             <emphasis role="bold">Provide remote shell access to
 342             clients.</emphasis>
 343           </emphasis>It is recommended that all cluster nodes have remote shell
 344           client access to facilitate the use of Lustre configuration and
 345           monitoring scripts. Parallel Distributed SHell (pdsh) is preferable,
 346           although Secure SHell (SSH) is acceptable.</para>
 347         </listitem>
 348         <listitem>
 349           <para>
 350           <emphasis role="italic">(Recommended)</emphasis>
 351           <emphasis role="italic">
 352             <emphasis role="bold">Ensure client clocks are
 353             synchronized.</emphasis>
 354           </emphasis>The Lustre file system uses client clocks for timestamps.
 355           If clocks are out of sync between clients, files will appear with
 356           different time stamps when accessed by different clients. Drifting
 357           clocks can also cause problems by, for example, making it difficult
 358           to debug multi-node issues or correlate logs, which depend on
 359           timestamps. We recommend that you use Network Time Protocol (NTP) to
 360           keep client and server clocks in sync with each other. For more
 361           information about NTP, see:
 362           <link xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
 363         </listitem>
 364       </itemizedlist></para>
 365     </section>
 366   </section>
 367   <section xml:id="dbdoclet.50438261_41900">
 368     <title>Lustre Software Installation Procedure</title>
 369     <caution>
 370       <para>Before installing the Lustre software, back up ALL data. The Lustre
 371       software contains kernel modifications that interact with storage devices
 372       and may introduce security issues and data loss if not installed,
 373       configured, or administered properly.</para>
 374     </caution>
 375     <para>To install the Lustre software from RPMs, complete the steps
 376     below.</para>
 377     <orderedlist>
 378       <listitem>
 379         <para>Verify that all Lustre installation requirements have been
 380         met.</para>
 381         <itemizedlist>
 382           <listitem>
 383             <para>For hardware requirements, see
 384             <xref linkend="settinguplustresystem" />.</para>
 385           </listitem>
 386           <listitem>
 387             <para>For software and environmental requirements, see the section
 388             <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 389             linkend="dbdoclet.50438261_99193" />above.</para>
 390           </listitem>
 391         </itemizedlist>
 392       </listitem>
 393       <listitem>
 394         <para>Download the
 395         <literal>e2fsprogs</literal> RPMs for your platform from the
 396         <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 397         Lustre Releases</link>repository.</para>
 398       </listitem>
 399       <listitem>
 400         <para>Download the Lustre server RPMs for your platform from the
 401         <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 402         Lustre Releases</link>repository. See
 403         <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 404         linkend="table_cnh_5m3_gk" />for a list of required packages.</para>
 405       </listitem>
 406       <listitem xml:id="dbdoclet.50438261_88296">
 407         <para>Install the Lustre server and
 408         <literal>e2fsprogs</literal> packages on all Lustre servers (MGS, MDSs,
 409         and OSSs).
 410         <orderedlist numeration="loweralpha">
 411           <listitem>
 412             <para>Log onto a Lustre server as the
 413             <literal>root</literal> user</para>
 414           </listitem>
 415           <listitem>
 416             <para>Use the
 417             <literal>yum</literal> command to install the packages:</para>
 418             <para>
 419               <screen>
 420 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
 421 </screen>
 422             </para>
 423           </listitem>
 424           <listitem>
 425             <para>Verify the packages are installed correctly:</para>
 426             <para>
 427               <screen>
 428 rpm -qa|egrep "lustre|wc"|sort
 429 </screen>
 430             </para>
 431           </listitem>
 432           <listitem>
 433             <para>Reboot the server.</para>
 434           </listitem>
 435           <listitem>
 436             <para>Repeat these steps on each Lustre server.</para>
 437           </listitem>
 438         </orderedlist></para>
 439       </listitem>
 440       <listitem>
 441         <para>Download the Lustre client RPMs for your platform from the
 442         <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 443         Lustre Releases</link>repository. See
 444         <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 445         linkend="table_j3r_ym3_gk" />for a list of required packages.</para>
 446       </listitem>
 447       <listitem>
 448         <para>Install the Lustre client packages on all Lustre clients.
 449         <note>
 450           <para>The version of the kernel running on a Lustre client must be
 451           the same as the version of the
 452           <literal>lustre-client-modules-</literal>
 453           <replaceable>ver</replaceable> package being installed. If not, a
 454           compatible kernel must be installed on the client before the Lustre
 455           client packages are installed.</para>
 456         </note></para>
 457         <orderedlist numeration="loweralpha">
 458           <listitem>
 459             <para>Log onto a Lustre client as the root user.</para>
 460           </listitem>
 461           <listitem>
 462             <para>Use the
 463             <literal>yum</literal> command to install the packages:</para>
 464             <para>
 465               <screen>
 466 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
 467 </screen>
 468             </para>
 469           </listitem>
 470           <listitem>
 471             <para>Verify the packages were installed correctly:</para>
 472             <para>
 473               <screen>
 474 # rpm -qa|egrep "lustre|kernel"|sort
 475 </screen>
 476             </para>
 477           </listitem>
 478           <listitem>
 479             <para>Reboot the client.</para>
 480           </listitem>
 481           <listitem>
 482             <para>Repeat these steps on each Lustre client.</para>
 483           </listitem>
 484         </orderedlist>
 485       </listitem>
 486     </orderedlist>
 487     <para>To configure LNet, go to
 488     <xref linkend="configuringlnet" />. If default settings will be used for
 489     LNet, go to
 490     <xref linkend="configuringlustre" />.</para>
 491   </section>
 492 </chapter>