-<?xml version="1.0" encoding="UTF-8"?>
-<article version="5.0" xml:lang="en-US" xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink">
- <info>
- <title>Installing Lustre from Source Code</title>
- </info>
- <informaltable frame="none">
- <tgroup cols="2">
- <colspec colname="c1" colwidth="50*"/>
- <colspec colname="c2" colwidth="50*"/>
-
-
- <tbody>
- <row>
- <entry align="left"><para>Lustre 2.0 Operations Manual</para></entry>
- <entry align="right" valign="top"><para><link xl:href="index.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/toc01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/toc01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="VI_Reference.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/prev01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/prev01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="LustreRecovery.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/next01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/next01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="ix.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/index01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/index01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link></para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><link xl:href=""/></para>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
-
- <tbody>
- <row>
- <entry align="right"><para><anchor xml:id="dbdoclet.50438210_pgfId-874" xreflabel=""/>C H A P T E R 29<anchor xml:id="dbdoclet.50438210_75891" xreflabel=""/></para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
-
- <tbody>
- <row>
- <entry align="right"><para><anchor xml:id="dbdoclet.50438210_pgfId-5529" xreflabel=""/><anchor xml:id="dbdoclet.50438210_67391" xreflabel=""/>Installing Lustre from Source Code</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1304664" xreflabel=""/>If you need to build a customized Lustre server kernel or are using a Linux kernel that has not been tested with the version of Lustre you are installing, you may need to build and install Lustre from source code. This chapter describes:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305083" xreflabel=""/><link xl:href="InstallingLustrefrSourceCode.html#50438210_69313">Overview and Prerequisites</link></para>
+<?xml version='1.0' encoding='UTF-8'?><chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="installinglustrefromsourcecode">
+ <title xml:id="installinglustrefromsourcecode.title">Installing a Lustre File System from Source
+ Code</title>
+ <para>This chapter describes how to create a customized Lustre server kernel from source code.
+ Sections included are:</para>
+ <itemizedlist>
+ <listitem>
+ <para><xref linkend="dbdoclet.50438210_69313"/></para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para><xref linkend="dbdoclet.50438210_65411"/></para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305088" xreflabel=""/><link xl:href="InstallingLustrefrSourceCode.html#50438210_65411">Patching the Kernel</link></para>
+ <listitem>
+ <para><xref linkend="dbdoclet.50438210_47529"/></para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para><xref linkend="section_g4d_yby_lk"/></para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305093" xreflabel=""/><link xl:href="InstallingLustrefrSourceCode.html#50438210_47529">Creating and Installing the Lustre Packages</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305098" xreflabel=""/><link xl:href="InstallingLustrefrSourceCode.html#50438210_27248">Installing Lustre with a Third-Party Network Stack</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-</itemizedlist>
- <section remap="h2">
- <title><anchor xml:id="dbdoclet.50438210_pgfId-1297212" xreflabel=""/></title>
- <section remap="h2">
- <title>29.1 <emphasis role="bold">Overview and Prerequisites</emphasis><anchor xml:id="dbdoclet.50438210_69313" xreflabel=""/></title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305142" xreflabel=""/>Lustre can be installed from either pre-built binary packages (RPMs) or freely-available source code. Installing from the package release is recommended unless you need to customize the Lustre server kernel or will be using an Linux kernel that has not been tested with Lustre. For a list of supported Linux distributions and architectures, see the topic <link xl:href="http://wiki.lustre.org/index.php/Lustre_2.0">Lustre_2.0</link> on the Lustre wiki. The procedure for installing Lustre from RPMs is describe in <link xl:href="InstallingLustre.html#50438261_81829">Chapter 8</link>: <link xl:href="InstallingLustre.html#50438261_62973">Installing the Lustre Software</link>.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297213" xreflabel=""/>To install Lustre from source code, the following are required:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297214" xreflabel=""/> Linux kernel patched with Lustre-specific patches</para>
+ </itemizedlist>
+ <note>
+ <para>It is recommended that you install from prebuild RPMs of the Lustre software unless you
+ need to customize the Lustre server kernel or will be using an Linux kernel that has not been
+ tested with the Lustre software. Instructions on installing pre-built RPMs are in <xref linkend='installinglustre.title'/></para>
+ <para>For a list of supported Linux distributions and architectures, see <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="LustreTestMatrixTable"/>. Prebuild RPMs
+ are available in the <link xl:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre Releases</link>
+ repository. For information about installing Lustre RPMs, see <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="installinglustre"/>.</para>
+ </note>
+ <section xml:id="dbdoclet.50438210_69313">
+ <title><indexterm>
+ <primary>installing</primary>
+ <secondary>from source</secondary>
+ <tertiary>prerequisites</tertiary>
+ </indexterm>Overview and Prerequisites</title>
+ <para>To install the Lustre software from source code, the following are required:</para>
+ <itemizedlist>
+ <listitem>
+ <para>A x86_64 machine with a fresh installation of a Linux operating system.</para>
+ <note><para>This section of the Manual only describes the specific procedure against building on Red Hat
+ Enterprise Linux 6 and derivatives. For a complete set of supported platforms please
+ visit <xref xmlns:xlink="http://www.w3.org/1999/xlink" linkend="LustreTestMatrixTable"
+ />.The server kernel should be one of the supported kernel versions but it is not
+ necessary to run a vendor distribution to build the Lustre software.</para></note>
+ </listitem>
+ <listitem>
+ <para>Access to the Lustre software git repository at <link xl:href="http://git.hpdd.intel.com/">http://git.hpdd.intel.com/</link>.</para>
+ </listitem>
+ <listitem>
+ <para>(Recommended) Access to a recent version of <link xmlns:xlink="http://www.w3.org/1999/xlink"
+ xlink:href="http://fedoraproject.org/wiki/EPEL">EPEL</link> containing the
+ <literal>quilt</literal> utility used for managing a series of patches. </para>
+ <note><para>The use of <literal>quilt</literal> is optional for building the Lustre software, and is
+ detected at configure time if it is installed. Otherwise the <literal>patch</literal>
+ utility is used to apply <literal>ldiskfs</literal> patches to the kernel source. Quilt
+ is useful if you intend to modify the Lustre kernel patches.</para></note>
+ </listitem>
+ <listitem>
+ <para><emphasis role="italic">(Recommended)</emphasis> At least 1 GB memory on the machine
+ used for the build.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role="italic">(Recommended)</emphasis> At least 20 GB hard disk space on the
+ machine used for the build. </para>
+ </listitem>
+ <listitem>
+ <para>Security-Enhanced Linux (SELinux) <emphasis role="italic">disabled</emphasis> on all
+ Lustre servers and clients. The Lustre software does not support SELinux.</para>
+ </listitem>
+ </itemizedlist>
+ <para>The installation procedure includes several steps:</para>
+ <itemizedlist>
+ <listitem>
+ <para> Patching the core kernel.</para>
+ </listitem>
+ <listitem>
+ <para> Building Lustre RPMs.</para>
+ </listitem>
+ <listitem>
+ <para> Installing and testing the Lustre file system.</para>
+ </listitem>
+ </itemizedlist>
+ <para>These steps are described in the following sections.</para>
+ </section>
+ <section xml:id="dbdoclet.50438210_65411">
+ <title><indexterm><primary>installing</primary><secondary>from source</secondary><tertiary>patching the kernel</tertiary></indexterm>Patching the Kernel</title>
+ <para>This section first describes how to prepare your machine to serve as a development
+ environment, including build tools, the Lustre source, and the Linux kernel source. It then
+ describes how to apply Lustre patches to the Linux kernel.</para>
+ <note>
+ <para>A patched Linux kernel is <emphasis>NOT</emphasis> required in order to build the
+Lustre client code. The steps below for building the Linux kernel are only
+required if a new kernel is needed for a Lustre server.</para>
+ </note>
+ <section remap="h3">
+ <title><indexterm>
+ <primary>quilt</primary>
+ </indexterm>Provisioning the Build Machine and Installing Dependencies</title>
+ <note>
+ <para>This example is specific for building a modified Red Hat Enteprise Linux 6 kernel for
+ the Lustre file system server. It is assumed that the operating system has been freshly
+ installed on a machine with the hostname <literal>rhel6-master</literal>.</para>
+ </note>
+ <para>To provision the build machine and install dependencies, complete these steps.</para>
+ <orderedlist>
+ <listitem>
+ <para>Log in as <systemitem>root</systemitem>.</para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>Install the kernel development tools:</para>
+ <para>
+ <screen># yum -y groupinstall "Development Tools"</screen>
+ </para>
+ <note>
+ <para>If the Development Tools group is not available, you will need to satisfy the dependencies for the build manually.</para>
+ </note>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297215" xreflabel=""/> Lustre modules compiled for the Linux kernel</para>
+ <listitem>
+ <para>Install additional dependencies:</para>
+ <para>
+ <screen># yum -y install xmlto asciidoc elfutils-libelf-devel
+ zlib-devel binutils-devel newt-devel python-devel hmaccalc
+ perl-ExtUtils-Embed</screen>
+ </para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>(Optional) Install EPEL:</para>
+ <para>
+ <screen># wget
+ http://download.fedoraproject.org/pub/epel/6/x86_64/
+ epel-release-6-8.noarch.rpm
+# rpm -ivh ./epel-release-5-4.noarch.rpm
+</screen>
+ </para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297216" xreflabel=""/> Lustre utilities required for Lustre configuration</para>
+ <listitem>
+ <para>(Optional) Install quilt:</para>
+ <para>
+ <screen># yum -y install quilt
+</screen>
+ </para>
+ <note>
+ <para>
+ <package>newt-devel</package> may not be available for Red Hat Enterprise Linux 6. One
+ option is to download the <link xmlns:xlink="http://www.w3.org/1999/xlink"
+ xlink:href="http://mirror.centos.org/centos/6/os/x86_64/Packages/newt-devel-0.52.11-3.el6.x86_64.rpm"
+ >newt-devel</link>, <link xmlns:xlink="http://www.w3.org/1999/xlink"
+ xlink:href="http://mirror.centos.org/centos/6/os/x86_64/Packages/slang-devel-2.2.1-1.el6.x86_64.rpm"
+ >slang-devel</link>, and <link xmlns:xlink="http://www.w3.org/1999/xlink"
+ xlink:href="http://mirror.centos.org/centos/6/os/i386/Packages/asciidoc-8.4.5-4.1.el6.noarch.rpm"
+ >asciidoc</link> RPMs from the CentOS Enterprise Linux Distribution and install
+ using:</para>
+ <para>
+ <screen>yum --nogpgcheck localinstall
+ ./newt-devel-0.52.11-3.el6.x86_64.rpm
+ ./slang-devel-2.2.1-1.el6.x86_64.rpm
+ ./asciidoc-8.4.5-4.1.el6.noarch.rpm
+</screen>
+ </para>
+ </note>
</listitem>
-<listitem>
- <para> </para>
+ </orderedlist>
+ </section>
+ <section remap="h3" xml:id="PreparingLustreSource">
+ <title>Preparing the Lustre Source</title>
+ <para>To prepare the Lustre source, complete these steps.</para>
+ <orderedlist>
+ <listitem>
+ <para>Create a user called <systemitem>build</systemitem> with a home directory
+ <systemitem>/home/build</systemitem>:</para>
+ <para>
+ <screen># useradd -m build
+</screen>
+ </para>
</listitem>
-</itemizedlist>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1304577" xreflabel=""/>The installation procedure involves several steps:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302403" xreflabel=""/> Patching the core kernel</para>
+ <listitem>
+ <para>Switch to the user called <systemitem>build</systemitem> and change the directory to
+ the users home directory:</para>
+ <para>
+ <screen># su build
+$ cd $HOME
+</screen>
+ </para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>Get the MASTER branch of the Lustre software from the Lustre software git
+ repository:</para>
+ <para>
+ <screen># git clone git://git.hpdd.intel.com/fs/lustre-release.git
+# cd lustre-release
+</screen>
+ </para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302413" xreflabel=""/> Configuring the kernel to work with Lustre</para>
+ <listitem>
+ <para>Run:</para>
+ <para>
+ <screen>sh ./autogen.sh
+</screen>
+ </para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>Resolve any outstanding dependencies. </para>
+ <para><screen># sh ./autogen.sh</screen>When <package>autogen.sh</package> completes
+ successfully, a response similar to the following is displayed: </para>
+ <para>
+ <screen>Checking for a complete tree...
+checking forautomake-1.9>= 1.9... found 1.9.6
+...
+...
+configure.ac:10: installing `./config.sub'
+configure.ac:12: installing `./install-sh'
+configure.ac:12: installing `./missing'
+Running autoconf</screen>
+ </para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302420" xreflabel=""/> Creating Lustre and kernel RPMs from source code.</para>
+ </orderedlist>
+ </section>
+ <section remap="h3">
+ <title>Preparing the Kernel Source</title>
+ <para>To build the kernel using <literal>rpmbuild</literal> (a tool specific to RPM-based
+ distributions), complete these steps.</para>
+ <orderedlist>
+ <listitem>
+ <para>Get the kernel source: </para>
+ <para>
+ <screen># cd $HOME
+# mkdir -p kernel/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
+# cd kernel
+# echo '%_topdir %(echo $HOME)/kernel/rpmbuild'> ~/.rpmmacros
+</screen>
+ </para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>Install the kernel source (enter on one line):</para>
+ <para>
+ <screen># rpm -ivh http://ftp.redhat.com/pub/redhat/linux/enterprise/
+ 6Server/en/os/SRPMS/kernel-2.6.32-358.14.1.el6.src.rpm
+ 2>&1 | grep -v mockb</screen>
+ </para>
+ <note>
+ <para>It is intended that the Lustre software master branch be kept up-to-date with the
+ most recent kernel distributions. However, a delay may occur between a periodic update
+ to a kernel distribution and a corresponding update of the master branch. The most
+ recent supported kernel in the master branch can be found in the source directory in
+ <systemitem>lustre/kernel_patches/which_patch</systemitem>. If the master branch is
+ not current with the latest distribution, download the most recent kernel RPMs from
+ the vendor's download site.</para>
+ </note>
</listitem>
-</itemizedlist>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1299593" xreflabel=""/>When using third-party network hardware with Lustre, the third-party modules (typically, the drivers) must be linked against the Linux kernel. The LNET modules in Lustre also need these references. To meet these requirements, a specific process must be followed to install and recompile Lustre. See <link xl:href="InstallingLustrefrSourceCode.html#50438210_27248">Installing Lustre with a Third-Party Network Stack</link>, for an example showing how to install Lustre 1.6.6 using the Myricom MX 1.2.7 driver. The same process can be used for other third-party network stacks.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
+ <listitem>
+ <para>Prepare the source using <literal>rpmbuild</literal>:</para>
+ <para>
+ <screen># cd ~/kernel/rpmbuild
+# rpmbuild -bp --target=`uname -m` ./SPECS/kernel.spec
+</screen>
+ </para>
+ <para>The text displayed ends with the following:</para>
+ <para>
+ <screen>...
+gpg: Total number processed: 1
+gpg: imported: 1
++ gpg --homedir . --export --keyring ./kernel.pub Red
+gpg: WARNING: unsafe permissions on homedir `.'
++ gcc -o scripts/bin2c scripts/bin2c.c
++ scripts/bin2c ksign_def_public_key __initdata
++ cd ..
++ exit 0</screen>
+ </para>
+ </listitem>
+ </orderedlist>
+ <para>The kernel source with the Red Hat Enterprise Linux patches applied is now residing in
+ the directory
+ <systemitem>/home/build/kernel/rpmbuild/BUILD/kernel-2.6.32-358.14.1.el6.x86_64/</systemitem></para>
</section>
- <section remap="h2">
- <title>29.2 <anchor xml:id="dbdoclet.50438210_65411" xreflabel=""/>Patching the Kernel</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296830" xreflabel=""/>If you are using non-standard hardware, plan to apply a Lustre patch, or have another reason not to use packaged Lustre binaries, you have to apply several Lustre patches to the core kernel and run the Lustre configure script against the kernel.</para>
- <section remap="h3">
- <title><anchor xml:id="dbdoclet.50438210_pgfId-1296831" xreflabel=""/>29.2.1 Introducing the Quilt Utility</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296832" xreflabel=""/>To simplify the process of applying Lustre patches to the kernel, we recommend that you use the Quilt utility.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296833" xreflabel=""/>Quilt manages a stack of patches on a single source tree. A series file lists the patch files and the order in which they are applied. Patches are applied, incrementally, on the base tree and all preceding patches. You can:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302485" xreflabel=""/> Apply patches from the stack (quilt push)</para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302495" xreflabel=""/> Remove patches from the stack (quilt pop)</para>
- </listitem>
-<listitem>
- <para> </para>
+ <section remap="h3">
+ <title>Patching the Kernel Source with the Lustre Code</title>
+ <para>To patch the kernel source with the Lustre code, complete these steps.</para>
+ <orderedlist>
+ <listitem>
+ <para>Go to the directory containing the kernel source:</para>
+ <para>
+ <screen>#cd ~/kernel/rpmbuild/BUILD/kernel-2.6.32-358.14.1.el6/
+ linux-2.6.32-358.14.1.el6.x86_64/</screen>
+ </para>
+ </listitem>
+ <listitem>
+ <para>Edit the <filename>Makefile</filename> file in this directory to add a unique build
+ id to be able to ascertain that the kernel is booted. Modify line 4 as shown
+ below:</para>
+ <para>
+ <screen>EXTRAVERSION = -358.14.1.el6_lustre.x86_64</screen>
+ </para>
+ </listitem>
+ <listitem>
+ <para>Overwrite the <filename>.config</filename> file in this directory with the Lustre
+ <filename>.config</filename> file:</para>
+ <para>
+ <screen># cp ~/lustre-release/lustre/kernel_patches/kernel_configs/
+ kernel-2.6.32-2.6-rhel6-x86_64.config ./.config</screen>
+ </para>
+ </listitem>
+ <listitem>
+ <para>Link the Lustre series and patches:</para>
+ <para>
+ <screen># ln -s ~/lustre-release/lustre/kernel_patches/series/
+ 2.6-rhel6.series series
+# ln -s ~/lustre-release/lustre/kernel_patches/patches patches</screen>
+ </para>
+ </listitem>
+ <listitem>
+ <para>Apply the patches to the kernel source using <package>quilt</package>:</para>
+ <para>
+ <screen># quilt push -av</screen>
+ </para>
+ <para>The following is displayed:</para>
+ <screen>...
+...
+patching file fs/jbd2/transaction.c
+Hunk #3succeeded at 1222(offset 3lines).
+Hunk #4succeeded at 1357(offset 3lines).
+Now at patch patches/jbd2-jcberr-2.6-rhel6.patch</screen>
+ <note><para>If quilt is not installed, the following steps would be equivalent:
+<screen>for PATCH in $(cat series); do patch -p1 < patches/$PATCH; done</screen></para></note>
+ </listitem>
+ </orderedlist>
+ </section>
+ </section>
+ <section xml:id="dbdoclet.50438210_47529">
+ <title>
+ <indexterm>
+ <primary>installing</primary>
+ <secondary>from source</secondary>
+ <tertiary>installing Lustre RPMs</tertiary>
+ </indexterm> Building the Lustre RPMs</title>
+ <para>This section describes how to configure the patched kernel to work with the Lustre
+ software and how to create and install the Lustre packages (RPMs).</para>
+ <section>
+ <title>Building a New Kernel</title>
+ <para>To build a new kernel as an RPM, complete these steps.</para>
+ <orderedlist>
+ <listitem>
+ <para>In the kernel source directory, build a kernel rpm:</para>
+ <screen># cd ~/kernel/rpmbuild/BUILD/kernel-2.6.32-131.2.1.el6/
+ linux-2.6.32-131.2.1.el6.x86_64/
+# make oldconfig || make menuconfig
+# make include/asm
+# make include/linux/version.h
+# make SUBDIRS=scripts
+# make include/linux/utsrelease.h
+# make rpm</screen>
+ <para>A successful build returns text similar to the following:</para>
+ <screen>...
+...
+Wrote: /home/build/kernel/rpmbuild/SRPMS/
+ kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.src.rpm
+Wrote: /home/build/kernel/rpmbuild/RPMS/x86_64/
+ kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64.rpm
+Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.f73m1V
++ umask 022+ cd /home/build/kernel/rpmbuild/BUILD
++ cd kernel-2.6.32-358.14.1.el6_lustre.x86_64
++ rm -rf /home/build/kernel/rpmbuild/BUILDROOT/
+ kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64
++ exit 0
+rm ../kernel-2.6.3-358.14.1.el6_lustre.x86_64.tar.gz</screen>
+ <note>
+ <para>If a request to generate more entropy appears, some disk or keyboard I/O needs to
+ be generated. You can generate entropy by entering the following on another
+ terminal:<screen># grep -Ri 'any_text' /usr</screen></para>
+ </note>
+ </listitem>
+ </orderedlist>
+ <para>A fresh kernel RPM can now be found at
+ <systemitem>~/kernel/rpmbuild/RPMS/x86_64/kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64.rpm</systemitem>.</para>
+ </section>
+ <section>
+ <title>Configuring and Building Lustre RPMs</title>
+ <para>To configure and build a set of Lustre RPMs, complete these steps.<orderedlist>
+ <listitem>
+ <para>Configure the Lustre source:</para>
+ <para>
+ <screen># cd ~/lustre-release/
+# ./configure --with-linux=/home/build/kernel/rpmbuild/BUILD/
+ kernel-2.6.32-358.14.1.el6_lustre.x86_64/</screen>
+ </para>
+ <para>Text similar to the following is displayed:</para>
+ <para>
+ <screen>...
+...
+LLCPPFLAGS: -D__arch_lib__ -D_LARGEFILE64_SOURCE=1
+CFLAGS: -g -O2 -Werror
+EXTRA_KCFLAGS: -include /home/build/lustre-release/config.h -g
+ -I/home/build/lustre-release
+/libcfs/include -I/home/build/lustre-release/lnet/include
+ -I/home/build/lustre-release/
+lustre/include
+LLCFLAGS: -g -Wall -fPIC -D_GNU_SOURCE
+Type 'make' to build Lustre.</screen>
+ </para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302502" xreflabel=""/> Query the contents of the series file (quilt series), the contents of the stack (quilt applied, quilt previous, quilt top), and the patches that are not applied at a particular moment (quilt next, quilt unapplied).</para>
+ <listitem>
+ <para>Create the RPMs:</para>
+ <para>
+ <screen># make rpms</screen>
+ </para>
+ <para>Text similar to the following is displayed:</para>
+ <para>
+ <screen>...
+...
+Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.TsLWpD
++ umask 022
++ cd /home/build/kernel/rpmbuild/BUILD
++ cd lustre-2.0.61
++ rm -rf /home/build/kernel/rpmbuild/BUILDROOT/
+ lustre-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64
++ exit 0
+make[1]: Leaving directory `/home/build/lustre-release'</screen>
+ </para>
</listitem>
-<listitem>
- <para> </para>
+ </orderedlist></para>
+ <para>The resulting RPMs are in
+ <systemitem>~build/kernel/rpmbuild/RPMS/x86_64/</systemitem>.<screen>kernel-2.6.3-358.14.1.el6_lustre.x86_64-1.x86_64.rpm
+lustre-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
+lustre-debuginfo-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
+lustre-ldiskfs-3.3.0-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
+lustre-ldiskfs-debuginfo-3.3.0-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
+lustre-modules-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
+lustre-source-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
+lustre-tests-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm</screen></para>
+ </section>
+ <section xml:id="InstallingLustreKernel">
+ <title><indexterm>
+ <primary>installing</primary>
+ <secondary>from source</secondary>
+ <tertiary>installing the Lustre kernel</tertiary>
+ </indexterm>Installing the Lustre Kernel</title>
+ <para>To install the Lustre kernel, complete these steps.<orderedlist>
+ <listitem>
+ <para>As <systemitem>root</systemitem>, install the kernel:</para>
+ <para>
+ <screen># rpm -ivh ~build/kernel/rpmbuild/RPMS/x86_64/
+ kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64.rpm</screen>
+ </para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302509" xreflabel=""/> Edit and refresh (update) patches with Quilt, as well as revert inadvertent changes, and fork or clone patches and show the diffs before and after work.</para>
+ <listitem>
+ <para>Create <literal>initrd</literal> using <literal>dracut</literal>:</para>
+ <para>
+ <screen># /sbin/new-kernel-pkg --package kernel --mkinitrd --dracut
+ --depmod --install 2.6.32-358.14.1.el6_lustre.x86_64</screen>
+ </para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>Reboot the system:</para>
+ <para>
+ <screen>reboot</screen>
+ </para>
+ <para>A login prompt such as that shown below indicates
+ success:<screen>Red Hat Enterprise Linux Server release 6.0(Santiago)
+Kernel 2.6.3-358.14.1.el6_lustre.x86_64 on an x86_64
+
+client-10login:
+</screen></para>
</listitem>
-</itemizedlist>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302510" xreflabel=""/>A variety of Quilt packages (RPMs, SRPMs and tarballs) are available from various sources. Use the most recent version you can find. Quilt depends on several other utilities, e.g., the coreutils RPM that is only available in RedHat 9. For other RedHat kernels, you have to get the required packages to successfully install Quilt. If you cannot locate a Quilt package or fulfill its dependencies, you can build Quilt from a tarball, available at the Quilt project website:</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296837" xreflabel=""/><link xl:href="http://savannah.nongnu.org/projects/quilt">http://savannah.nongnu.org/projects/quilt</link></para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296838" xreflabel=""/>For additional information on using Quilt, including its commands, see <link xl:href="http://www.suse.de/~agruen/quilt.pdf">Introduction to Quilt</link> and the <link xl:href="http://linux.die.net/man/1/quilt">quilt(1) man page.</link></para>
- </section>
- <section remap="h3">
- <title><anchor xml:id="dbdoclet.50438210_pgfId-1296842" xreflabel=""/>29.2.2 <anchor xml:id="dbdoclet.50438210_44148" xreflabel=""/>Get the Lustre Source and Unpatched Kernel</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302564" xreflabel=""/>The Lustre Engineering Team has targeted several Linux kernels for use with Lustre servers (MDS/OSS) and provides a series of patches for each one. The Lustre patches are maintained in the kernel_patch directory bundled with the Lustre source code.</para>
- <informaltable frame="none">
- <tgroup cols="2">
- <colspec colname="c1" colwidth="5*"/>
- <colspec colname="c2" colwidth="95*"/>
-
-
- <tbody>
- <row>
- <entry><para><inlinemediaobject><imageobject role="html">
- <imagedata fileref="./shared/caution.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/caution.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></para></entry>
- <entry><para><emphasis role="bold">Caution -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1305236" xreflabel=""/>Lustre contains kernel modifications which interact with storage devices and may introduce security issues and data loss if not installed, configured and administered correctly. Before installing Lustre, be cautious and back up ALL data.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis> Each patch series has been tailored to a specific kernel version, and may or may not apply cleanly to other versions of the kernel.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305199" xreflabel=""/>To obtain the Lustre source and unpatched kernel:</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305200" xreflabel=""/> 1. Verify that all of the Lustre installation requirements have been met.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305210" xreflabel=""/>For more information on these prerequisites, see:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305217" xreflabel=""/> Hardware requirements in <link xl:href="SettingUpLustreSystem.html#50438256_38751">Chapter 5</link>: <link xl:href="SettingUpLustreSystem.html#50438256_66186">Setting Up a Lustre File System</link></para>
+ </orderedlist></para>
+ </section>
+ <section>
+ <title><indexterm>
+ <primary>installing</primary>
+ <secondary>from source</secondary>
+ <tertiary>building Lustre with OFED</tertiary>
+ </indexterm>Building a Lustre File System with a Third-party Network Stack on OFED
+ (Optional)</title>
+ <para>When using third-party network hardware, you must follow a specific process to install
+ and recompile the Lustre software. This section provides an installation example that shows
+ how to build Lustre software release 2.4.x when using the OpenFabrics Enterprise
+ Distribution (OFED*) version 3.5-1 open-source software. The same process can be used for
+ other third-party network stacks by replacing the OFED installation procedure and using the
+ appropriate <literal>--with</literal> option when configuring the Lustre source code.<orderedlist>
+ <listitem>
+ <para>Install the Lustre kernel (see <xref xmlns:xlink="http://www.w3.org/1999/xlink"
+ linkend="InstallingLustreKernel"/>).</para>
</listitem>
-<listitem>
- <para> </para>
+ <listitem>
+ <para>Compile and install the OFED stack. As root, enter:</para>
+ <screen># cd /usr/src
+# tar –zxvf OFED-3.5-1.tgz
+# cd OFED-3.5-1
+# ./install.pl</screen>
+ <para>A directory <literal>/usr/src/compat-rdma</literal> is created.</para>
</listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305220" xreflabel=""/> Software and environmental requirements in <link xl:href="InstallingLustre.html#50438261_99193">Preparing to Install the Lustre Software</link></para>
+ <listitem>
+ <para>Configure and build Lustre source code:</para>
+ <orderedlist numeration="loweralpha">
+ <listitem>
+ <para>Install the Lustre source. This can be done via RPM or git (see <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="PreparingLustreSource"
+ />).</para>
+ </listitem>
+ <listitem>
+ <para>Configure and build the Lustre source code</para>
+ <screen>$ cd ~/lustre-release
+$ sh ./autogen.sh</screen>
+ </listitem>
+ </orderedlist>
</listitem>
-<listitem>
- <para> </para>
+ </orderedlist></para>
+ <para>The <literal>./configure --help</literal> command shows a list of all the
+ <literal>--with</literal> options. All third-party network stacks are built in this
+ manner.<screen>$ ./configure --with-linux=/usr/src/kernels/$(uname –r) --with-o2ib=/usr/src/compat-rdma
+$ make
+$ make rpms</screen></para>
+ <para>The <literal>make rpms</literal> command output shows the location of the generated
+ RPMs.</para>
+ </section>
+ </section>
+ <section xml:id="section_g4d_yby_lk">
+ <title><indexterm>
+ <primary>installing</primary>
+ <secondary>from source</secondary>
+ <tertiary>testing a Lustre file system</tertiary>
+ </indexterm>Installing and Testing a Lustre File System</title>
+ <para>This section describes how to install the Lustre RPMs and run the Lustre test
+ suite.</para>
+ <section>
+ <title>Installing <package>e2fsprogs</package></title>
+ <para>The <package>e2fsprogs</package> package is required on all Lustre file system server nodes. To download
+ and install <literal>e2fsprogs</literal>, complete these steps.<orderedlist>
+ <listitem>
+ <para>Download <literal>e2fsprogs</literal> from the <link
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xlink:href="https://wiki.hpdd.intel.com/display/PUB/Lustre+Releases">Lustre
+ Releases</link> repository.</para>
</listitem>
-</itemizedlist>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305206" xreflabel=""/> 2. Download the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305253" xreflabel=""/>On the <link xl:href="http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L">Lustre download site</link>, select a version of Lustre to download and then select “Source†as the platform.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296849" xreflabel=""/> 3. Download the unpatched kernel.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296857" xreflabel=""/>For convenience, Oracle maintains an archive of unpatched kernel sources at:</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305277" xreflabel=""/><link xl:href="http://downloads.lustre.org/public/kernels/">http://downloads.lustre.org/public/kernels/</link><anchor xml:id="dbdoclet.50438210_28487" xreflabel=""/></para>
- </section>
- <section remap="h3">
- <title><anchor xml:id="dbdoclet.50438210_pgfId-1305280" xreflabel=""/>29.2.3 Patch the Kernel</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305429" xreflabel=""/>This procedure describes how to use Quilt to apply the Lustre patches to the kernel. To illustrate the steps in this procedure, a RHEL 5 kernel is patched for Lustre 1.6.5.1.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305430" xreflabel=""/> 1. Unpack the Lustre source and kernel to separate source trees.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296865" xreflabel=""/>a. Unpack the Lustre source.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296866" xreflabel=""/>For this procedure, we assume that the resulting source tree is in /tmp/lustre-1.6.5.1</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296867" xreflabel=""/>b. Unpack the kernel.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296868" xreflabel=""/>For this procedure, we assume that the resulting source tree (also known as the destination tree) is in /tmp/kernels/linux-2.6.18</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296869" xreflabel=""/> 2. Select a config file for your kernel, located in the kernel_configs directory (lustre/kernel_patches/kernel_config).</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296870" xreflabel=""/>The kernel_config directory contains the .config files, which are named to indicate the kernel and architecture with which they are associated. For example, the configuration file for the 2.6.18 kernel shipped with RHEL 5 (suitable for i686 SMP systems) is kernel-2.6.18-2.6-rhel5-i686-smp.config.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296871" xreflabel=""/> 3. Select the series file for your kernel, located in the series directory (lustre/kernel_patches/series).</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296872" xreflabel=""/>The series file contains the patches that need to be applied to the kernel.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296873" xreflabel=""/> 4. Set up the necessary symlinks between the kernel patches and the Lustre source.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296874" xreflabel=""/>This example assumes that the Lustre source files are unpacked under /tmp/lustre-1.6.5.1 and you have chosen the 2.6-rhel5.series file). Run:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1296875" xreflabel=""/>$ cd /tmp/kernels/linux-2.6.18
-<anchor xml:id="dbdoclet.50438210_pgfId-1296876" xreflabel=""/>$ rm -f patches series
-<anchor xml:id="dbdoclet.50438210_pgfId-1296877" xreflabel=""/>$ ln -s /tmp/lustre-1.6.5.1/lustre/kernel_patches/series/2.6-\ rhel5.series\
- ./series
-<anchor xml:id="dbdoclet.50438210_pgfId-1296878" xreflabel=""/>$ ln -s /tmp/lustre-1.6.5.1/lustre/kernel_patches/patches .
+ <listitem>
+ <para>Install the <literal>e2fsprogs</literal> package:</para>
+ <para>
+ <screen># rpm -Uvh
+ ./e2fsprogs-1.42.7.wc2-7.el6.x86_64.rpm
+ ./e2fsprogs-libs-1.42.7.wc2-7.el6.x86_64.rpm
+
</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296879" xreflabel=""/> 5. Use Quilt to apply the patches in the selected series file to the unpatched kernel. Run:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1296880" xreflabel=""/>$ cd /tmp/kernels/linux-2.6.18
-<anchor xml:id="dbdoclet.50438210_pgfId-1296881" xreflabel=""/>$ quilt push -av
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296882" xreflabel=""/>The patched destination tree acts as a base Linux source tree for Lustre.</para>
- </section>
+ </para>
+ </listitem>
+ </orderedlist></para>
</section>
- <section remap="h2">
- <title>29.3 <anchor xml:id="dbdoclet.50438210_47529" xreflabel=""/>Creating and Installing the Lustre Packages</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1296886" xreflabel=""/>After patching the kernel, configure it to work with Lustre, create the Lustre packages (RPMs) and install them.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297162" xreflabel=""/> 1. Configure the patched kernel to run with Lustre. Run:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297163" xreflabel=""/>$ cd <path to kernel tree>
-<anchor xml:id="dbdoclet.50438210_pgfId-1297079" xreflabel=""/>$ cp /boot/config-‘uname -r‘ .config
-<anchor xml:id="dbdoclet.50438210_pgfId-1297080" xreflabel=""/>$ make oldconfig || make menuconfig
-<anchor xml:id="dbdoclet.50438210_pgfId-1297083" xreflabel=""/>$ make include/asm
-<anchor xml:id="dbdoclet.50438210_pgfId-1297084" xreflabel=""/>$ make include/linux/version.h
-<anchor xml:id="dbdoclet.50438210_pgfId-1297085" xreflabel=""/>$ make SUBDIRS=scripts
-<anchor xml:id="dbdoclet.50438210_pgfId-1297086" xreflabel=""/>$ make include/linux/utsrelease.h
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297087" xreflabel=""/> 2. Run the Lustre configure script against the patched kernel and create the Lustre packages.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297088" xreflabel=""/>$ cd <path to lustre source tree>
-<anchor xml:id="dbdoclet.50438210_pgfId-1297089" xreflabel=""/>$ ./configure --with-linux=<path to kernel tree>
-<anchor xml:id="dbdoclet.50438210_pgfId-1297090" xreflabel=""/>$ make rpms
+ <section>
+ <title>Installing the Lustre RPMs</title>
+ <para>To install the Lustre RPMs, complete these steps as <systemitem>root</systemitem>:<orderedlist>
+ <listitem>
+ <para>Make the directory containing the Lustre RPMs your current directory:</para>
+ <para>
+ <screen># cd /home/build/kernel/rpmbuild/RPMS/x86_64/
</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297091" xreflabel=""/>This creates a set of .rpms in /usr/src/redhat/RPMS/<arch> with an appended date-stamp. The SuSE path is /usr/src/packages.</para>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1298281" xreflabel=""/>You do not need to run the Lustre configure script against an unpatched kernel.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297092" xreflabel=""/><emphasis role="bold">Example set of RPMs:</emphasis></para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297093" xreflabel=""/>lustre-1.6.5.1-\2.6.18_53.xx.xx.el5_lustre.1.6.5.1.custom_20081021.i686.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1297094" xreflabel=""/>
-<anchor xml:id="dbdoclet.50438210_pgfId-1297095" xreflabel=""/>lustre-debuginfo-1.6.5.1-\2.6.18_53.xx.xx.el5_lustre.1.6.5.1.custom_2008102\
-1.i686.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1297096" xreflabel=""/>
-<anchor xml:id="dbdoclet.50438210_pgfId-1297097" xreflabel=""/>lustre-modules-1.6.5.1-\2.6.18_53.xx.xxel5_lustre.1.6.5.1.custom_20081021.i\
-686.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1297098" xreflabel=""/>
-<anchor xml:id="dbdoclet.50438210_pgfId-1297099" xreflabel=""/>lustre-source-1.6.5.1-\2.6.18_53.xx.xx.el5_lustre.1.6.5.1.custom_20081021.i\
-686.rpm
-</screen>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1297101" xreflabel=""/>If the steps to create the RPMs fail, contact Lustre Support by reporting a bug. See <link xl:href="LustreTroubleshooting.html#50438198_30989">Reporting a Lustre Bug</link>.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1297102" xreflabel=""/>Several features and packages are available that extend the core functionality of Lustre. These features/packages can be enabled at the build time by issuing appropriate arguments to the configure command. For a list of these features and packages, run ./configure -help in the Lustre source tree. The configs/ directory of the kernel source contains the config files matching each the kernel version. Copy one to .config at the root of the kernel tree.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297103" xreflabel=""/> 3. <anchor xml:id="dbdoclet.50438210_41207" xreflabel=""/>Create the kernel package. Navigate to the kernel source directory and run:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297104" xreflabel=""/>$ make rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297106" xreflabel=""/>Example result:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297107" xreflabel=""/>kernel-2.6.95.0.3.EL_lustre.1.6.5.1custom-1.i686.rpm
-</screen>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1297108" xreflabel=""/><link xl:href="InstallingLustrefrSourceCode.html#50438210_41207">Step 3</link> is only valid for RedHat and SuSE kernels. If you are using a stock Linux kernel, you need to get a script to create the kernel RPM.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297755" xreflabel=""/> 4. Install the Lustre packages.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302732" xreflabel=""/>Some Lustre packages are installed on servers (MDS and OSSs), and others are installed on Lustre clients. For guidance on where to install specific packages, see <link xl:href="InstallingLustre.html#50438261_21654">TABLE 8-1</link> in <link xl:href="InstallingLustre.html#50438261_99193">Preparing to Install the Lustre Software</link>. which lists required packages and for each package, where to install it. Depending on the selected platform, not all of the packages listed in <link xl:href="InstallingLustre.html#50438261_21654">TABLE 8-1</link> need to be installed.</para>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438210_pgfId-1302707" xreflabel=""/>Running the patched server kernel on the clients is optional. It is not necessary unless the clients will be used for multiple purposes, for example, to run as a client and an OST.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1302716" xreflabel=""/>Lustre packages should be installed in this order:</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297644" xreflabel=""/>a. Install the kernel, modules and ldiskfs packages.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297645" xreflabel=""/>Navigate to the directory where the RPMs are stored, and use the rpm -ivh command to install the kernel, module and ldiskfs packages.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299031" xreflabel=""/>$ rpm -ivh kernel-lustre-smp-<ver> \
-<anchor xml:id="dbdoclet.50438210_pgfId-1299032" xreflabel=""/>kernel-ib-<ver> \
-<anchor xml:id="dbdoclet.50438210_pgfId-1299033" xreflabel=""/>lustre-modules-<ver> \
-<anchor xml:id="dbdoclet.50438210_pgfId-1299034" xreflabel=""/>lustre-ldiskfs-<ver>
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297650" xreflabel=""/>b. Install the utilities/userspace packages.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297651" xreflabel=""/>Use the rpm -ivh command to install the utilities packages. For example:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297652" xreflabel=""/>$ rpm -ivh lustre-<ver>
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297653" xreflabel=""/>c. Install the e2fsprogs package.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297654" xreflabel=""/>Make sure the e2fsprogs package is unpacked, and use the rpm -i command to install it. For example:</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1297655" xreflabel=""/>$ rpm -i e2fsprogs-<ver>
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1297656" xreflabel=""/>d. (Optional) If you want to add optional packages to your Lustre system, install them now.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1298263" xreflabel=""/> 5. Verify that the boot loader (grub.conf or lilo.conf) has been updated to load the patched kernel.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1298264" xreflabel=""/> 6. Reboot the patched clients and the servers.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1298265" xreflabel=""/>a. If you applied the patched kernel to any clients, reboot them.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1298266" xreflabel=""/>Unpatched clients do not need to be rebooted.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1298267" xreflabel=""/>b. Reboot the servers.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1298268" xreflabel=""/>Once all the machines have rebooted, the next steps are to configure Lustre Networking (LNET) and the Lustre file system. See <link xl:href="ConfiguringLustre.html#50438267_88428">Chapter 10</link>: <link xl:href="ConfiguringLustre.html#50438267_66186">Configuring Lustre</link>.</para>
+ </para>
+ </listitem>
+ <listitem>
+ <para>Install the RPMs:</para>
+ <para>
+ <screen># rpm -ivh lustre-ldiskfs-3.3.0-2.6.32-358.14.1.el6_lustre.x86_64*
+# rpm -ivh lustre-modules-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64*
+# rpm -ivh lustre-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_*
+# rpm -ivh lustre-tests-*</screen>
+ </para>
+ </listitem>
+ </orderedlist></para>
</section>
- <section remap="h2">
- <title>29.4 <anchor xml:id="dbdoclet.50438210_27248" xreflabel=""/>Installing Lustre with a Third-Party Network Stack</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299781" xreflabel=""/>When using third-party network hardware, you must follow a specific process to install and recompile Lustre. This section provides an installation example, describing how to install Lustre 1.6.6 while using the Myricom MX 1.2.7 driver. The same process is used for other third-party network stacks, by replacing MX-specific references in <link xl:href="InstallingLustrefrSourceCode.html#50438210_12366">Step 2</link> with the stack-specific build and using the proper --with option when configuring the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299638" xreflabel=""/> 1. Compile and install the Lustre kernel.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299642" xreflabel=""/>a. Install the necessary build tools.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305395" xreflabel=""/>GCC and related tools must be installed. For more information, see <link xl:href="InstallingLustre.html#50438261_37079">Required Software</link>.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299645" xreflabel=""/>$ yum install rpm-build redhat-rpm-config
-<anchor xml:id="dbdoclet.50438210_pgfId-1300055" xreflabel=""/>$ mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
-<anchor xml:id="dbdoclet.50438210_pgfId-1300058" xreflabel=""/>$ echo '%_topdir %(echo $HOME)/rpmbuild' > .rpmmacros
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299650" xreflabel=""/>b. Install the patched Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300116" xreflabel=""/>This RPM is available at the <link xl:href="http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L">Lustre download site</link>.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1300294" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305349" xreflabel=""/>kernel-lustre-source-2.6.18-92.1.10.el5_lustre.1.6.6.x86_64.rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300117" xreflabel=""/>a. Build the Linux kernel RPM.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299659" xreflabel=""/>$ cd /usr/src/linux-2.6.18-92.1.10.el5_lustre.1.6.6
-<anchor xml:id="dbdoclet.50438210_pgfId-1300080" xreflabel=""/>$ make distclean
-<anchor xml:id="dbdoclet.50438210_pgfId-1300083" xreflabel=""/>$ make oldconfig dep bzImage modules
-<anchor xml:id="dbdoclet.50438210_pgfId-1300086" xreflabel=""/>$ cp /boot/config-`uname -r` .config
-<anchor xml:id="dbdoclet.50438210_pgfId-1300089" xreflabel=""/>$ make oldconfig || make menuconfig
-<anchor xml:id="dbdoclet.50438210_pgfId-1300092" xreflabel=""/>$ make include/asm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300095" xreflabel=""/>$ make include/linux/version.h
-<anchor xml:id="dbdoclet.50438210_pgfId-1300098" xreflabel=""/>$ make SUBDIRS=scripts
-<anchor xml:id="dbdoclet.50438210_pgfId-1300101" xreflabel=""/>$ make rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299670" xreflabel=""/>b. Install the Linux kernel RPM.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300130" xreflabel=""/>If you are building a set of RPMs for a cluster installation, this step is not necessary. Source RPMs are only needed on the build machine.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299674" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305534" xreflabel=""/>~/rpmbuild/kernel-lustre-2.6.18-92.1.10.el5_\lustre.1.6.6.x86_64.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300142" xreflabel=""/>$ mkinitrd /boot/2.6.18-92.1.10.el5_lustre.1.6.6
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299677" xreflabel=""/>c. Update the boot loader (/etc/grub.conf) with the new kernel boot information.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299679" xreflabel=""/>$ /sbin/shutdown 0 -r
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299681" xreflabel=""/> 2. <anchor xml:id="dbdoclet.50438210_12366" xreflabel=""/>Compile and install the MX stack.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299686" xreflabel=""/>$ cd /usr/src/
-<anchor xml:id="dbdoclet.50438210_pgfId-1299907" xreflabel=""/>$ gunzip mx_1.2.7.tar.gz (can be obtained from www.myri.com/scs/)
-<anchor xml:id="dbdoclet.50438210_pgfId-1299910" xreflabel=""/>$ tar -xvf mx_1.2.7.tar
-<anchor xml:id="dbdoclet.50438210_pgfId-1299913" xreflabel=""/>$ cd mx-1.2.7
-<anchor xml:id="dbdoclet.50438210_pgfId-1299916" xreflabel=""/>$ ln -s common include
-<anchor xml:id="dbdoclet.50438210_pgfId-1299919" xreflabel=""/>$ ./configure --with-kernel-lib
-<anchor xml:id="dbdoclet.50438210_pgfId-1299922" xreflabel=""/>$ make
-<anchor xml:id="dbdoclet.50438210_pgfId-1299925" xreflabel=""/>$ make install
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299696" xreflabel=""/> 3. Compile and install the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299698" xreflabel=""/>a. Install the Lustre source (this can be done via RPM or tarball). The source file is available at the <link xl:href="http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L">Lustre download site</link>. This example shows installation via the tarball.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299700" xreflabel=""/>$ cd /usr/src/
-<anchor xml:id="dbdoclet.50438210_pgfId-1300181" xreflabel=""/>$ gunzip lustre-1.6.6.tar.gz
-<anchor xml:id="dbdoclet.50438210_pgfId-1300184" xreflabel=""/>$ tar -xvf lustre-1.6.6.tar
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299706" xreflabel=""/>b. Configure and build the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300191" xreflabel=""/>The ./configure --help command shows a list of all of the --with options. All third-party network stacks are built in this manner.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299711" xreflabel=""/>$ cd lustre-1.6.6
-<anchor xml:id="dbdoclet.50438210_pgfId-1300366" xreflabel=""/>$ ./configure --with-linux=/usr/src/linux \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305509" xreflabel=""/>--with-mx=/usr/src/mx-1.2.7
-<anchor xml:id="dbdoclet.50438210_pgfId-1300369" xreflabel=""/>$ make
-<anchor xml:id="dbdoclet.50438210_pgfId-1300372" xreflabel=""/>$ make rpms
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299716" xreflabel=""/>The make rpms command output shows the location of the generated RPMs</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299718" xreflabel=""/> 4. Use the rpm -ivh command to install the RPMS.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299720" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305550" xreflabel=""/>lustre-1.6.6-2.6.18_92.1.10.el5_lustre.1.6.6smp.x86_64.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300246" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305560" xreflabel=""/>lustre-modules-1.6.6-2.6.18_92.1.10.el5_lustre.1.6.6\
-<anchor xml:id="dbdoclet.50438210_pgfId-1305564" xreflabel=""/>smp.x86_64.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300249" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305568" xreflabel=""/>lustre-ldiskfs-3.0.6-2.6.18_92.1.10.el5_lustre.1.6.6\
-<anchor xml:id="dbdoclet.50438210_pgfId-1305572" xreflabel=""/>smp.x86_64.rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299726" xreflabel=""/> 5. Add the following lines to the /etc/modprobe.conf file.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299727" xreflabel=""/>options kmxlnd hosts=/etc/hosts.mxlnd
-<anchor xml:id="dbdoclet.50438210_pgfId-1300264" xreflabel=""/>options lnet networks=mx0(myri0),tcp0(eth0)
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299730" xreflabel=""/> 6. Populate the myri0 configuration with the proper IP addresses.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299731" xreflabel=""/>vim /etc/sysconfig/network-scripts/myri0
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299733" xreflabel=""/> 7. Add the following line to the /etc/hosts.mxlnd file.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299734" xreflabel=""/>$ IP HOST BOARD EP_ID
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299736" xreflabel=""/> 8. Start Lustre.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300394" xreflabel=""/>Once all the machines have rebooted, the next steps are to configure Lustre Networking (LNET) and the Lustre file system. See <link xl:href="ConfiguringLustre.html#50438267_88428">Chapter 10</link>: <link xl:href="InstallingLustrefrSourceCode.html#50438210_67391">Installing Lustre from Source Code</link>.</para>
- <!--
-Begin SiteCatalyst code version: G.5.
--->
- <!--
-End SiteCatalyst code version: G.5.
--->
- <informaltable frame="none">
- <tgroup cols="3">
- <colspec colname="c1" colwidth="33*"/>
- <colspec colname="c2" colwidth="33*"/>
- <colspec colname="c3" colwidth="33*"/>
-
-
-
- <tbody>
- <row>
- <entry align="left"><para>Lustre 2.0 Operations Manual</para></entry>
- <entry align="right"><para>821-2076-10</para></entry>
- <entry align="right" valign="top"><para><link xl:href="index.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/toc01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/toc01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="VI_Reference.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/prev01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/prev01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="LustreRecovery.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/next01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/next01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="ix.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/index01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/index01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link></para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><link xl:href=""/></para>
- <para><link xl:href="copyright.html">Copyright</link> © 2011, Oracle and/or its affiliates. All rights reserved.</para>
+ <section>
+ <title>Running the Test Suite</title>
+ <para>To test a single node Lustre file system installation, complete these steps.<orderedlist>
+ <listitem>
+ <para>Mount the Lustre file system.</para>
+ <para>
+ <screen># /usr/lib64/lustre/tests/llmount.sh</screen>
+ </para>
+ <para>Text similar to the following will be
+ displayed:<screen>Loading modules from /usr/lib64/lustre/tests/..
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+gss/krb5 is not supported
+Formatting mgs, mds, osts
+Format mds1: /tmp/lustre-mdt1
+Format ost1: /tmp/lustre-ost1
+Format ost2: /tmp/lustre-ost2
+Checking servers environments
+Checking clients rhel6-master environments
+Loading modules from /usr/lib64/lustre/tests/..
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+gss/krb5 is not supported
+Setup mgs, mdt, osts
+Starting mds1: -o loop,user_xattr,acl
+/tmp/lustre-mdt1 /mnt/mds1
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Started lustre-MDT0000
+Starting ost1: -o loop /tmp/lustre-ost1 /mnt/ost1
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Started lustre-OST0000
+Starting ost2: -o loop /tmp/lustre-ost2 /mnt/ost2
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Started lustre-OST0001
+Starting client: rhel5-build: -o user_xattr,acl,flock
+rhel6-master@tcp:/lustre /mnt/lustre
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Using TIMEOUT=20
+disable quota as required</screen></para>
+ <para>The Lustre file system is now available at
+ <systemitem>/mnt/lustre</systemitem>.</para>
+ <warning><para>The Lustre file system available at <literal>/mnt/lustre</literal> is a test file system built from small loopback devices in <literal>/tmp</literal> and not for production use.</para></warning>
+ <note>
+ <para>If you see the error below, associate the IP address of a non-loopback interface
+ with the name of your machine in the file
+ <filename>/etc/hosts</filename>.<screen>mkfs.lustre: Can't parse NID 'rhel6-master@tcp'</screen></para>
+ </note>
+ </listitem>
+ </orderedlist></para>
</section>
</section>
-</article>
+</chapter>
git://git.whamcloud.com / doc / manual.git / blobdiff