- <section remap="h2">
- <title>29.4 <anchor xml:id="dbdoclet.50438210_27248" xreflabel=""/>Installing Lustre with a Third-Party Network Stack</title>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299781" xreflabel=""/>When using third-party network hardware, you must follow a specific process to install and recompile Lustre. This section provides an installation example, describing how to install Lustre 1.6.6 while using the Myricom MX 1.2.7 driver. The same process is used for other third-party network stacks, by replacing MX-specific references in <link xl:href="InstallingLustrefrSourceCode.html#50438210_12366">Step 2</link> with the stack-specific build and using the proper --with option when configuring the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299638" xreflabel=""/> 1. Compile and install the Lustre kernel.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299642" xreflabel=""/>a. Install the necessary build tools.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1305395" xreflabel=""/>GCC and related tools must be installed. For more information, see <link xl:href="InstallingLustre.html#50438261_37079">Required Software</link>.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299645" xreflabel=""/>$ yum install rpm-build redhat-rpm-config
-<anchor xml:id="dbdoclet.50438210_pgfId-1300055" xreflabel=""/>$ mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
-<anchor xml:id="dbdoclet.50438210_pgfId-1300058" xreflabel=""/>$ echo '%_topdir %(echo $HOME)/rpmbuild' > .rpmmacros
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299650" xreflabel=""/>b. Install the patched Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300116" xreflabel=""/>This RPM is available at the <link xl:href="http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L">Lustre download site</link>.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1300294" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305349" xreflabel=""/>kernel-lustre-source-2.6.18-92.1.10.el5_lustre.1.6.6.x86_64.rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300117" xreflabel=""/>a. Build the Linux kernel RPM.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299659" xreflabel=""/>$ cd /usr/src/linux-2.6.18-92.1.10.el5_lustre.1.6.6
-<anchor xml:id="dbdoclet.50438210_pgfId-1300080" xreflabel=""/>$ make distclean
-<anchor xml:id="dbdoclet.50438210_pgfId-1300083" xreflabel=""/>$ make oldconfig dep bzImage modules
-<anchor xml:id="dbdoclet.50438210_pgfId-1300086" xreflabel=""/>$ cp /boot/config-`uname -r` .config
-<anchor xml:id="dbdoclet.50438210_pgfId-1300089" xreflabel=""/>$ make oldconfig || make menuconfig
-<anchor xml:id="dbdoclet.50438210_pgfId-1300092" xreflabel=""/>$ make include/asm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300095" xreflabel=""/>$ make include/linux/version.h
-<anchor xml:id="dbdoclet.50438210_pgfId-1300098" xreflabel=""/>$ make SUBDIRS=scripts
-<anchor xml:id="dbdoclet.50438210_pgfId-1300101" xreflabel=""/>$ make rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299670" xreflabel=""/>b. Install the Linux kernel RPM.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300130" xreflabel=""/>If you are building a set of RPMs for a cluster installation, this step is not necessary. Source RPMs are only needed on the build machine.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299674" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305534" xreflabel=""/>~/rpmbuild/kernel-lustre-2.6.18-92.1.10.el5_\lustre.1.6.6.x86_64.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300142" xreflabel=""/>$ mkinitrd /boot/2.6.18-92.1.10.el5_lustre.1.6.6
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299677" xreflabel=""/>c. Update the boot loader (/etc/grub.conf) with the new kernel boot information.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299679" xreflabel=""/>$ /sbin/shutdown 0 -r
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299681" xreflabel=""/> 2. <anchor xml:id="dbdoclet.50438210_12366" xreflabel=""/>Compile and install the MX stack.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299686" xreflabel=""/>$ cd /usr/src/
-<anchor xml:id="dbdoclet.50438210_pgfId-1299907" xreflabel=""/>$ gunzip mx_1.2.7.tar.gz (can be obtained from www.myri.com/scs/)
-<anchor xml:id="dbdoclet.50438210_pgfId-1299910" xreflabel=""/>$ tar -xvf mx_1.2.7.tar
-<anchor xml:id="dbdoclet.50438210_pgfId-1299913" xreflabel=""/>$ cd mx-1.2.7
-<anchor xml:id="dbdoclet.50438210_pgfId-1299916" xreflabel=""/>$ ln -s common include
-<anchor xml:id="dbdoclet.50438210_pgfId-1299919" xreflabel=""/>$ ./configure --with-kernel-lib
-<anchor xml:id="dbdoclet.50438210_pgfId-1299922" xreflabel=""/>$ make
-<anchor xml:id="dbdoclet.50438210_pgfId-1299925" xreflabel=""/>$ make install
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299696" xreflabel=""/> 3. Compile and install the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299698" xreflabel=""/>a. Install the Lustre source (this can be done via RPM or tarball). The source file is available at the <link xl:href="http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L">Lustre download site</link>. This example shows installation via the tarball.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299700" xreflabel=""/>$ cd /usr/src/
-<anchor xml:id="dbdoclet.50438210_pgfId-1300181" xreflabel=""/>$ gunzip lustre-1.6.6.tar.gz
-<anchor xml:id="dbdoclet.50438210_pgfId-1300184" xreflabel=""/>$ tar -xvf lustre-1.6.6.tar
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299706" xreflabel=""/>b. Configure and build the Lustre source code.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300191" xreflabel=""/>The ./configure --help command shows a list of all of the --with options. All third-party network stacks are built in this manner.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299711" xreflabel=""/>$ cd lustre-1.6.6
-<anchor xml:id="dbdoclet.50438210_pgfId-1300366" xreflabel=""/>$ ./configure --with-linux=/usr/src/linux \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305509" xreflabel=""/>--with-mx=/usr/src/mx-1.2.7
-<anchor xml:id="dbdoclet.50438210_pgfId-1300369" xreflabel=""/>$ make
-<anchor xml:id="dbdoclet.50438210_pgfId-1300372" xreflabel=""/>$ make rpms
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299716" xreflabel=""/>The make rpms command output shows the location of the generated RPMs</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299718" xreflabel=""/> 4. Use the rpm -ivh command to install the RPMS.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299720" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305550" xreflabel=""/>lustre-1.6.6-2.6.18_92.1.10.el5_lustre.1.6.6smp.x86_64.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300246" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305560" xreflabel=""/>lustre-modules-1.6.6-2.6.18_92.1.10.el5_lustre.1.6.6\
-<anchor xml:id="dbdoclet.50438210_pgfId-1305564" xreflabel=""/>smp.x86_64.rpm
-<anchor xml:id="dbdoclet.50438210_pgfId-1300249" xreflabel=""/>$ rpm -ivh \
-<anchor xml:id="dbdoclet.50438210_pgfId-1305568" xreflabel=""/>lustre-ldiskfs-3.0.6-2.6.18_92.1.10.el5_lustre.1.6.6\
-<anchor xml:id="dbdoclet.50438210_pgfId-1305572" xreflabel=""/>smp.x86_64.rpm
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299726" xreflabel=""/> 5. Add the following lines to the /etc/modprobe.conf file.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299727" xreflabel=""/>options kmxlnd hosts=/etc/hosts.mxlnd
-<anchor xml:id="dbdoclet.50438210_pgfId-1300264" xreflabel=""/>options lnet networks=mx0(myri0),tcp0(eth0)
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299730" xreflabel=""/> 6. Populate the myri0 configuration with the proper IP addresses.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299731" xreflabel=""/>vim /etc/sysconfig/network-scripts/myri0
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299733" xreflabel=""/> 7. Add the following line to the /etc/hosts.mxlnd file.</para>
- <screen><anchor xml:id="dbdoclet.50438210_pgfId-1299734" xreflabel=""/>$ IP HOST BOARD EP_ID
-</screen>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1299736" xreflabel=""/> 8. Start Lustre.</para>
- <para><anchor xml:id="dbdoclet.50438210_pgfId-1300394" xreflabel=""/>Once all the machines have rebooted, the next steps are to configure Lustre Networking (LNET) and the Lustre file system. See <link xl:href="ConfiguringLustre.html#50438267_88428">Chapter 10</link>: <link xl:href="InstallingLustrefrSourceCode.html#50438210_67391">Installing Lustre from Source Code</link>.</para>
- <!--
-Begin SiteCatalyst code version: G.5.
--->
- <!--
-End SiteCatalyst code version: G.5.
--->
- <informaltable frame="none">
- <tgroup cols="3">
- <colspec colname="c1" colwidth="33*"/>
- <colspec colname="c2" colwidth="33*"/>
- <colspec colname="c3" colwidth="33*"/>
-
-
-
- <tbody>
- <row>
- <entry align="left"><para>Lustre 2.0 Operations Manual</para></entry>
- <entry align="right"><para>821-2076-10</para></entry>
- <entry align="right" valign="top"><para><link xl:href="index.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/toc01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/toc01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="VI_Reference.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/prev01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/prev01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="LustreRecovery.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/next01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/next01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link><link xl:href="ix.html"><inlinemediaobject><imageobject role="html">
- <imagedata contentdepth="26" contentwidth="30" fileref="./shared/index01.gif" scalefit="1"/>
- </imageobject>
-<imageobject role="fo">
- <imagedata contentdepth="100%" contentwidth="" depth="" fileref="./shared/index01.gif" scalefit="1" width="100%"/>
- </imageobject>
-</inlinemediaobject></link></para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para><link xl:href=""/></para>
- <para><link xl:href="copyright.html">Copyright</link> © 2011, Oracle and/or its affiliates. All rights reserved.</para>
+ <section>
+ <title>Running the Test Suite</title>
+ <para>To test a single node Lustre file system installation, complete these steps.<orderedlist>
+ <listitem>
+ <para>Mount the Lustre file system.</para>
+ <para>
+ <screen># /usr/lib64/lustre/tests/llmount.sh</screen>
+ </para>
+ <para>Text similar to the following will be
+ displayed:<screen>Loading modules from /usr/lib64/lustre/tests/..
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+gss/krb5 is not supported
+Formatting mgs, mds, osts
+Format mds1: /tmp/lustre-mdt1
+Format ost1: /tmp/lustre-ost1
+Format ost2: /tmp/lustre-ost2
+Checking servers environments
+Checking clients rhel6-master environments
+Loading modules from /usr/lib64/lustre/tests/..
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+gss/krb5 is not supported
+Setup mgs, mdt, osts
+Starting mds1: -o loop,user_xattr,acl
+/tmp/lustre-mdt1 /mnt/mds1
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Started lustre-MDT0000
+Starting ost1: -o loop /tmp/lustre-ost1 /mnt/ost1
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Started lustre-OST0000
+Starting ost2: -o loop /tmp/lustre-ost2 /mnt/ost2
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Started lustre-OST0001
+Starting client: rhel5-build: -o user_xattr,acl,flock
+rhel6-master@tcp:/lustre /mnt/lustre
+debug=0x33f0404
+subsystem_debug=0xffb7e3ff
+debug_mb=10
+Using TIMEOUT=20
+disable quota as required</screen></para>
+ <para>The Lustre file system is now available at
+ <systemitem>/mnt/lustre</systemitem>.</para>
+ <warning><para>The Lustre file system available at <literal>/mnt/lustre</literal> is a test file system built from small loopback devices in <literal>/tmp</literal> and not for production use.</para></warning>
+ <note>
+ <para>If you see the error below, associate the IP address of a non-loopback interface
+ with the name of your machine in the file
+ <filename>/etc/hosts</filename>.<screen>mkfs.lustre: Can't parse NID 'rhel6-master@tcp'</screen></para>
+ </note>
+ </listitem>
+ </orderedlist></para>