InstallingLustre.xml

   1 <?xml version='1.0' encoding='utf-8'?>
   2 <chapter xmlns="http://docbook.org/ns/docbook"
   3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
   4 xml:id="installinglustre">
   5   <title xml:id="installinglustre.title">Installing the Lustre Software</title>
   6   <para>This chapter describes how to install the Lustre software from RPM
   7   packages. It includes:</para>
   8   <itemizedlist>
   9     <listitem>
  10       <para>
  11         <xref linkend="dbdoclet.50438261_99193" />
  12       </para>
  13     </listitem>
  14     <listitem>
  15       <para>
  16         <xref linkend="dbdoclet.50438261_41900" />
  17       </para>
  18     </listitem>
  19   </itemizedlist>
  20   <para>For hardware and system requirements and hardware configuration
  21   information, see
  22   <xref linkend="settinguplustresystem" />.</para>
  23   <section xml:id="dbdoclet.50438261_99193">
  24     <title>
  25     <indexterm>
  26       <primary>installing</primary>
  27       <secondary>preparation</secondary>
  28     </indexterm>Preparing to Install the Lustre Software</title>
  29     <para>You can install the Lustre software from downloaded packages (RPMs)
  30     or directly from the source code. This chapter describes how to install the
  31     Lustre RPM packages. Instructions to install from source code are beyond
  32     the scope of this document, and can be found elsewhere online.</para>
  33     <para>The Lustre RPM packages are tested on current versions of Linux
  34     enterprise distributions at the time they are created.  See the release
  35     notes for each version for specific details.</para>
  36     <section xml:id="section_rqs_tjw_3k">
  37       <title>Software Requirements</title>
  38       <para>To install the Lustre software from RPMs, the following are
  39       required:
  40       <itemizedlist>
  41         <listitem>
  42           <para>
  43           <emphasis role="bold">
  44             <emphasis role="italic">Lustre server packages</emphasis>
  45           </emphasis>. The required packages for Lustre 2.9 EL7 servers are
  46           listed in the table below, where
  47           <replaceable>ver</replaceable> refers to the Lustre release and
  48           kernel version (e.g., 2.9.0-1.el7) and
  49           <replaceable>arch</replaceable> refers to the processor architecture
  50           (e.g., x86_64). These packages are available in the
  51           <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
  52           Lustre Releases</link> repository, and may differ depending on
  53           your distro and version.</para>
  54           <para>
  55             <table frame="all" xml:id="table_cnh_5m3_gk">
  56               <title>Packages Installed on Lustre Servers</title>
  57               <tgroup cols="2">
  58                 <colspec colname="c1" colnum="1" colwidth="1.18*" />
  59                 <colspec colname="c2" colnum="2" colwidth="1*" />
  60                 <thead>
  61                   <row>
  62                     <entry>Package Name</entry>
  63                     <entry>Description</entry>
  64                   </row>
  65                 </thead>
  66                 <tbody>
  67                   <row>
  68                     <entry>
  69                       <code>kernel-<replaceable>ver</replaceable>_lustre.<replaceable>arch</replaceable></code>
  70                     </entry>
  71                     <entry>Linux kernel with Lustre software patches (often
  72                     referred to as "patched kernel")</entry>
  73                   </row>
  74                   <row>
  75                     <entry>
  76                       <code>lustre-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  77                     </entry>
  78                     <entry>Lustre software command line tools</entry>
  79                   </row>
  80                   <row>
  81                     <entry>
  82                       <code>kmod-lustre-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  83                     </entry>
  84                     <entry>Lustre-patched kernel modules</entry>
  85                   </row>
  86                   <row>
  87                     <entry>
  88                       <code>kmod-lustre-osd-ldiskfs-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  89                     </entry>
  90                     <entry>Lustre back-end file system tools for ldiskfs-based
  91                     servers.
  92                     </entry>
  93                   </row>
  94                   <row>
  95                     <entry>
  96                       <code>lustre-osd-ldiskfs-mount-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
  97                     </entry>
  98                     <entry>Helper library for <literal>mount.lustre</literal>
  99                     and <literal>mkfs.lustre</literal> for ldiskfs-based servers.
 100                     </entry>
 101                   </row>
 102                   <row>
 103                     <entry>
 104                       <code>kmod-lustre-osd-zfs-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 105                     </entry>
 106                     <entry>Lustre back-end file system tools for ZFS. This is
 107                     an alternative to
 108                     <literal>lustre-osd-ldiskfs</literal> (kmod-spl and
 109                     kmod-zfs available separately).</entry>
 110                   </row>
 111                   <row>
 112                     <entry>
 113                       <code>lustre-osd-zfs-mount-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 114                     </entry>
 115                     <entry>Helper library for <literal>mount.lustre</literal>
 116                     and <literal>mkfs.lustre</literal> for ZFS-based servers
 117                     (zfs utilities available separately).
 118                     </entry>
 119                   </row>
 120                   <row>
 121                     <entry>
 122                       <code>e2fsprogs</code>
 123                     </entry>
 124                     <entry>Utilities to maintain Lustre ldiskfs back-end file
 125                     system(s)
 126                     </entry>
 127                   </row>
 128                   <row>
 129                     <entry>
 130                       <code>lustre-tests-<replaceable>ver</replaceable>_lustre.<replaceable>arch</replaceable></code>
 131                     </entry>
 132                     <entry>Lustre I/O Kit benchmarking tools
 133                     <emphasis role="italic">(Included in Lustre software as of
 134                     release 2.2)</emphasis></entry>
 135                   </row>
 136                 </tbody>
 137               </tgroup>
 138             </table>
 139           </para>
 140         </listitem>
 141         <listitem>
 142           <para>
 143           <emphasis role="bold">
 144             <emphasis role="italic">Lustre client packages</emphasis>
 145           </emphasis>. The required packages for Lustre 2.9 EL7 clients are
 146           listed in the table below, where
 147           <replaceable>ver</replaceable> refers to the Linux distribution (e.g.,
 148           3.6.18-348.1.1.el5). These packages are available in the
 149           <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 150           Lustre Releases</link> repository.</para>
 151           <para>
 152             <table frame="all" xml:id="table_j3r_ym3_gk">
 153               <title>Packages Installed on Lustre Clients</title>
 154               <tgroup cols="2">
 155                 <colspec colname="c1" colnum="1" colwidth="1*" />
 156                 <colspec colname="c2" colnum="2" colwidth="1.02*" />
 157                 <thead>
 158                   <row>
 159                     <entry>Package Name</entry>
 160                     <entry>Description</entry>
 161                   </row>
 162                 </thead>
 163                 <tbody>
 164                   <row>
 165                     <entry>
 166                       <code>kmod-lustre-client-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 167                     </entry>
 168                     <entry>Patchless kernel modules for client</entry>
 169                   </row>
 170                   <row>
 171                     <entry>
 172                       <code>lustre-client-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 173                     </entry>
 174                     <entry>Client command line tools</entry>
 175                   </row>
 176                   <row>
 177                     <entry>
 178                       <code>lustre-client-dkms-<replaceable>ver</replaceable>.<replaceable>arch</replaceable></code>
 179                     </entry>
 180                     <entry>Alternate client RPM to kmod-lustre-client with
 181                     Dynamic Kernel Module Support (DKMS) installation. This
 182                     avoids the need to install a new RPM for each kernel update,
 183                     but requires a full build environment on the client.</entry>
 184                   </row>
 185                 </tbody>
 186               </tgroup>
 187             </table>
 188           </para>
 189           <note>
 190             <para>The version of the kernel running on a Lustre client must be
 191             the same as the version of the
 192             <code>kmod-lustre-client-<replaceable>ver</replaceable></code>
 193             package being installed, unless the DKMS package is installed.
 194             If the kernel running on the client is not compatible, a kernel
 195             that is compatible must be installed on the client before the
 196             Lustre file system software is used.</para>
 197           </note>
 198         </listitem>
 199         <listitem>
 200           <para>
 201           <emphasis role="bold">
 202             <emphasis role="italic">Lustre LNet network driver (LND)</emphasis>
 203           </emphasis>. The Lustre LNDs provided with the Lustre software are
 204           listed in the table below. For more information about Lustre LNet,
 205           see
 206           <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 207           linkend="understandinglustrenetworking" />.</para>
 208           <table frame="all" xml:id="table_bkx_dn3_gk">
 209             <title>Network Types Supported by Lustre LNDs</title>
 210             <tgroup cols="2">
 211               <colspec colname="c1" colnum="1" colwidth="1*" />
 212               <colspec colname="newCol2" colnum="2" colwidth="1.95*" />
 213               <thead>
 214                 <row>
 215                   <entry>Supported Network Types</entry>
 216                   <entry>Notes</entry>
 217                 </row>
 218               </thead>
 219               <tbody>
 220                 <row>
 221                   <entry>TCP</entry>
 222                   <entry>Any network carrying TCP traffic, including GigE,
 223                   10GigE, and IPoIB</entry>
 224                 </row>
 225                 <row>
 226                   <entry>InfiniBand network</entry>
 227                   <entry>OpenFabrics OFED (o2ib)</entry>
 228                 </row>
 229                 <row>
 230                   <entry>gni</entry>
 231                   <entry>Gemini (Cray)</entry>
 232                 </row>
 233               </tbody>
 234             </tgroup>
 235           </table>
 236         </listitem>
 237       </itemizedlist></para>
 238       <para>
 239         <note>
 240           <para>The InfiniBand and TCP Lustre LNDs are routinely tested during
 241           release cycles. The other LNDs are maintained by their respective
 242           owners</para>
 243         </note>
 244         <itemizedlist>
 245           <listitem>
 246             <para>
 247             <emphasis role="bold">
 248               <emphasis role="italic">High availability software</emphasis>
 249             </emphasis>. If needed, install third party high-availability
 250             software. For more information, see
 251             <xref linkend="dbdoclet.50438188_92688" />.</para>
 252           </listitem>
 253           <listitem>
 254             <para>
 255             <emphasis role="bold">
 256               <emphasis role="italic">Optional packages.</emphasis>
 257             </emphasis>Optional packages provided in the
 258             <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 259             Lustre Releases</link>repository may include the following
 260             (depending on the operating system and platform):
 261             <itemizedlist>
 262               <listitem>
 263                 <para>
 264                 <code>kernel-debuginfo</code>,
 265                 <code>kernel-debuginfo-common</code>,
 266                 <code>lustre-debuginfo</code>,
 267                 <code>lustre-osd-ldiskfs-debuginfo</code>- Versions of required
 268                 packages with debugging symbols and other debugging options
 269                 enabled for use in troubleshooting.</para>
 270               </listitem>
 271               <listitem>
 272                 <para>
 273                 <code>kernel-devel</code>, - Portions of the kernel tree needed
 274                 to compile third party modules, such as network drivers.</para>
 275               </listitem>
 276               <listitem>
 277                 <para>
 278                 <code>kernel-firmware</code>- Standard Red Hat Enterprise Linux
 279                 distribution that has been recompiled to work with the Lustre
 280                 kernel.</para>
 281               </listitem>
 282               <listitem>
 283                 <para>
 284                 <code>kernel-headers</code>- Header files installed under
 285                 /user/include and used when compiling user-space,
 286                 kernel-related code.</para>
 287               </listitem>
 288               <listitem>
 289                 <para>
 290                 <code>lustre-source</code>- Lustre software source code.</para>
 291               </listitem>
 292               <listitem>
 293                 <para>
 294                 <emphasis role="italic">(Recommended)</emphasis>
 295                 <code>perf</code>,
 296                 <code>perf-debuginfo</code>,
 297                 <code>python-perf</code>,
 298                 <code>python-perf-debuginfo</code>- Linux performance analysis
 299                 tools that have been compiled to match the Lustre kernel
 300                 version.</para>
 301               </listitem>
 302             </itemizedlist></para>
 303           </listitem>
 304         </itemizedlist>
 305       </para>
 306     </section>
 307     <section xml:id="section_rh2_d4w_gk">
 308       <title>Environmental Requirements</title>
 309       <para>Before installing the Lustre software, make sure the following
 310       environmental requirements are met.
 311       <itemizedlist>
 312         <listitem>
 313           <para>
 314           <emphasis role="italic">(Required)</emphasis>
 315           <emphasis role="italic">
 316             <emphasis role="bold">Disable Security-Enhanced Linux
 317             <superscript>*</superscript>(SELinux) on all Lustre servers
 318             </emphasis>
 319           </emphasis>The Lustre software does not support SELinux. Therefore,
 320           the SELinux system extension must be disabled on all Lustre nodes.
 321           Also, make sure other security extensions (such as the Novell
 322           AppArmor
 323           <superscript>*</superscript>security system) and network packet
 324           filtering tools (such as iptables) do not interfere with the Lustre
 325           software.</para>
 326         </listitem>
 327         <listitem>
 328           <para>
 329           <emphasis role="italic">(Required)</emphasis>
 330           <emphasis role="italic">
 331             <emphasis role="bold">Use the same user IDs (UID) and group IDs
 332             (GID) on all clients.</emphasis>
 333           </emphasis>If use of supplemental groups is required, see
 334           <xref linkend="dbdoclet.50438291_32926" />for information about
 335           supplementary user and group cache upcall (
 336           <code>identity_upcall</code>).</para>
 337         </listitem>
 338         <listitem>
 339           <para>
 340           <emphasis role="italic">(Recommended)</emphasis>
 341           <emphasis role="italic">
 342             <emphasis role="bold">Provide remote shell access to
 343             clients.</emphasis>
 344           </emphasis>It is recommended that all cluster nodes have remote shell
 345           client access to facilitate the use of Lustre configuration and
 346           monitoring scripts. Parallel Distributed SHell (pdsh) is preferable,
 347           although Secure SHell (SSH) is acceptable.</para>
 348         </listitem>
 349         <listitem>
 350           <para>
 351           <emphasis role="italic">(Recommended)</emphasis>
 352           <emphasis role="italic">
 353             <emphasis role="bold">Ensure client clocks are
 354             synchronized.</emphasis>
 355           </emphasis>The Lustre file system uses client clocks for timestamps.
 356           If clocks are out of sync between clients, files will appear with
 357           different time stamps when accessed by different clients. Drifting
 358           clocks can also cause problems by, for example, making it difficult
 359           to debug multi-node issues or correlate logs, which depend on
 360           timestamps. We recommend that you use Network Time Protocol (NTP) to
 361           keep client and server clocks in sync with each other. For more
 362           information about NTP, see:
 363           <link xl:href="http://www.ntp.org/">http://www.ntp.org</link>.</para>
 364         </listitem>
 365       </itemizedlist></para>
 366     </section>
 367   </section>
 368   <section xml:id="dbdoclet.50438261_41900">
 369     <title>Lustre Software Installation Procedure</title>
 370     <caution>
 371       <para>Before installing the Lustre software, back up ALL data. The Lustre
 372       software contains kernel modifications that interact with storage devices
 373       and may introduce security issues and data loss if not installed,
 374       configured, or administered properly.</para>
 375     </caution>
 376     <para>To install the Lustre software from RPMs, complete the steps
 377     below.</para>
 378     <orderedlist>
 379       <listitem>
 380         <para>Verify that all Lustre installation requirements have been
 381         met.</para>
 382         <itemizedlist>
 383           <listitem>
 384             <para>For hardware requirements, see
 385             <xref linkend="settinguplustresystem" />.</para>
 386           </listitem>
 387           <listitem>
 388             <para>For software and environmental requirements, see the section
 389             <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 390             linkend="dbdoclet.50438261_99193" />above.</para>
 391           </listitem>
 392         </itemizedlist>
 393       </listitem>
 394       <listitem>
 395         <para>Download the
 396         <literal>e2fsprogs</literal> RPMs for your platform from the
 397         <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 398         Lustre Releases</link>repository.</para>
 399       </listitem>
 400       <listitem>
 401         <para>Download the Lustre server RPMs for your platform from the
 402         <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 403         Lustre Releases</link>repository. See
 404         <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 405         linkend="table_cnh_5m3_gk" />for a list of required packages.</para>
 406       </listitem>
 407       <listitem xml:id="dbdoclet.50438261_88296">
 408         <para>Install the Lustre server and
 409         <literal>e2fsprogs</literal> packages on all Lustre servers (MGS, MDSs,
 410         and OSSs).
 411         <orderedlist numeration="loweralpha">
 412           <listitem>
 413             <para>Log onto a Lustre server as the
 414             <literal>root</literal> user</para>
 415           </listitem>
 416           <listitem>
 417             <para>Use the
 418             <literal>yum</literal> command to install the packages:</para>
 419             <para>
 420               <screen>
 421 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
 422 </screen>
 423             </para>
 424           </listitem>
 425           <listitem>
 426             <para>Verify the packages are installed correctly:</para>
 427             <para>
 428               <screen>
 429 rpm -qa|egrep "lustre|wc"|sort
 430 </screen>
 431             </para>
 432           </listitem>
 433           <listitem>
 434             <para>Reboot the server.</para>
 435           </listitem>
 436           <listitem>
 437             <para>Repeat these steps on each Lustre server.</para>
 438           </listitem>
 439         </orderedlist></para>
 440       </listitem>
 441       <listitem>
 442         <para>Download the Lustre client RPMs for your platform from the
 443         <link xl:href="https://wiki.whamcloud.com/display/PUB/Lustre+Releases">
 444         Lustre Releases</link>repository. See
 445         <xref xmlns:xlink="http://www.w3.org/1999/xlink"
 446         linkend="table_j3r_ym3_gk" />for a list of required packages.</para>
 447       </listitem>
 448       <listitem>
 449         <para>Install the Lustre client packages on all Lustre clients.
 450         <note>
 451           <para>The version of the kernel running on a Lustre client must be
 452           the same as the version of the
 453           <literal>lustre-client-modules-</literal>
 454           <replaceable>ver</replaceable> package being installed. If not, a
 455           compatible kernel must be installed on the client before the Lustre
 456           client packages are installed.</para>
 457         </note></para>
 458         <orderedlist numeration="loweralpha">
 459           <listitem>
 460             <para>Log onto a Lustre client as the root user.</para>
 461           </listitem>
 462           <listitem>
 463             <para>Use the
 464             <literal>yum</literal> command to install the packages:</para>
 465             <para>
 466               <screen>
 467 # yum --nogpgcheck install pkg1.rpm pkg2.rpm ...
 468 </screen>
 469             </para>
 470           </listitem>
 471           <listitem>
 472             <para>Verify the packages were installed correctly:</para>
 473             <para>
 474               <screen>
 475 # rpm -qa|egrep "lustre|kernel"|sort
 476 </screen>
 477             </para>
 478           </listitem>
 479           <listitem>
 480             <para>Reboot the client.</para>
 481           </listitem>
 482           <listitem>
 483             <para>Repeat these steps on each Lustre client.</para>
 484           </listitem>
 485         </orderedlist>
 486       </listitem>
 487     </orderedlist>
 488     <para>To configure LNet, go to
 489     <xref linkend="configuringlnet" />. If default settings will be used for
 490     LNet, go to
 491     <xref linkend="configuringlustre" />.</para>
 492   </section>
 493 </chapter>