From 6b16c761d29a6edb17d1afaa5e4ad4965bd7c803 Mon Sep 17 00:00:00 2001 From: Richard Henwood Date: Thu, 29 Aug 2013 18:23:07 -0500 Subject: [PATCH] LUDOC-130 fix: improvements for accuracy. This patch updates LUDOC-130 work to enhance accuracy. The improvments implemented in the patch were provided against a recently landed patch http://review.whamcloud.com/#/c/6608/ Signed-off-by: Richard Henwood Change-Id: I1fc7c5b1a71fe293b5308b1ba61c4e5151494222 Reviewed-on: http://review.whamcloud.com/7511 Tested-by: Hudson --- InstallingLustre.xml | 6 +- InstallingLustreFromSourceCode.xml | 144 ++++++++++++++++--------------------- LustreProgrammingInterfaces.xml | 2 +- UpgradingLustre.xml | 34 +++++---- 4 files changed, 82 insertions(+), 104 deletions(-) diff --git a/InstallingLustre.xml b/InstallingLustre.xml index f44ca2d..5e3d213 100644 --- a/InstallingLustre.xml +++ b/InstallingLustre.xml @@ -370,13 +370,17 @@ + Download the e2fsprogs RPMs for your platform from the Lustre Releases + repository. + + Download the Lustre server RPMs for your platform from the Lustre Releases repository. See for a list of required packages. - Install the Lustre server packages on all Lustre servers (MGS, MDSs, and + Install the Lustre server and e2fsprogs packages on all Lustre servers (MGS, MDSs, and OSSs). Log onto a Lustre server as the root user diff --git a/InstallingLustreFromSourceCode.xml b/InstallingLustreFromSourceCode.xml index 102795c..f4208dc 100644 --- a/InstallingLustreFromSourceCode.xml +++ b/InstallingLustreFromSourceCode.xml @@ -14,17 +14,16 @@ - + It is recommended that you install from prebuild RPMs of the Lustre software unless you need to customize the Lustre server kernel or will be using an Linux kernel that has not been - tested with the Lustre software. + tested with the Lustre software. Instructions on installing pre-built RPMs are in For a list of supported Linux distributions and architectures, see . Prebuild RPMs - are available in the Lustre Releases + are available in the Lustre Releases repository. For information about installing Lustre RPMs, see . @@ -35,19 +34,17 @@ To install Lustre from source code, the following are required: - An x86_64 machine with a fresh installation of a CentOS (or Red Hat*) 6 Linux - operating system. + A x86_64 machine with a fresh installation of a Linux operating system. + This section of the Manual only describes the specific procedure against building on Red Hat* 6 Linux and derivatives. For a complete set of supported platforms please visit .The server kernel should be one of the supported kernel versions but it is not necessary to run a vendor distribution to build the Lustre software. - Access to the Lustre - Releases repository. This repository contains Lustre software patches and a test - suite. + Access to the Lustre software git repository at http://git.hpdd.intel.com/. - Access to a recent version of (Recommended) Access to a recent version of EPEL containing the - quilt utility used for managing a series of patches. + quilt utility used for managing a series of patches. + The use of quilt is optional for building Lustre, and is detected at configure time if it is installed. Otherwise the patch utility is used to apply ldiskfs patches to the kernel source. Quilt is useful if you intend to modify the Lustre kernel patches. (Recommended) At least 1 GB memory on the machine @@ -82,14 +79,16 @@ environment, including build tools, the Lustre source, and the Linux kernel source. It then describes how to apply Lustre patches to the Linux kernel. - A patched Linux kernel is not required on Lustre clients. + A patched Linux kernel is NOT required in order to build the +Lustre client code. The steps below for building the Linux kernel are only +required if a new kernel is needed for a Lustre server.
<indexterm> <primary>quilt</primary> </indexterm>Provisioning the Build Machine and Installing Dependencies - This example procedure assumes a Red Hat* Enteprise Linux* 6 operating system has been + This example is specific for building a modified Red Hat* Enteprise Linux* 6 kernel for the Lustre file system server. It is assumed that the operating system has been freshly installed on a machine with the hostname rhel6-master. To provision the build machine and install dependencies, complete these steps. @@ -103,14 +102,7 @@ # yum -y groupinstall "Development Tools" - If the Development Tools group is not available, you may need to install the - packages individually using: - - # yum -y install automake xmlto asciidoc - elfutils-libelf-devel zlib-devel binutils-devel - newt-devel python-devel hmaccalc perl-ExtUtils-Embed - rpm-build make gcc redhat-rpm-config patchutils git - + If the Development Tools group is not available, you will need to satisfy the dependencies for the build manually. @@ -122,17 +114,17 @@ - Install EPEL: + (Optional) Install EPEL: # wget - http://download.fedoraproject.org/pub/epel/5/x86_64/ - epel-release-5-4.noarch.rpm + http://download.fedoraproject.org/pub/epel/6/x86_64/ + epel-release-6-8.noarch.rpm # rpm -ivh ./epel-release-5-4.noarch.rpm - Install quilt: + (Optional) Install quilt: # yum -y install quilt @@ -171,18 +163,17 @@ - Switch to the user called build and change the directory to - the $HOME/build directory: + Switch to the user called build and change the directory to the users home directory: # su build -# cd $HOME +$ cd $HOME - Get the MASTER branch of the Lustre software from the Lustre repository: + Get the MASTER branch of the Lustre software from the Lustre software git repository: - # git clone git://git.whamcloud.com/fs/lustre-release.git + # git clone git://git.hpdd.intel.com/fs/lustre-release.git # cd lustre-release @@ -230,15 +221,15 @@ Running autoconf Install the kernel source (enter on one line): # rpm -ivh http://ftp.redhat.com/pub/redhat/linux/enterprise/ - 6Server/en/os/SRPMS/kernel-2.6.32-131.2.1.el6.src.rpm + 6Server/en/os/SRPMS/kernel-2.6.32-358.14.1.el6.src.rpm 2>&1 | grep -v mockb - It is intended that the Lustre software MASTER branch be kept up-to-date with the + It is intended that the Lustre software master branch be kept up-to-date with the most recent kernel distributions. However, a delay may occur between a periodic update - to a kernel distribution and a corresponding update of the MASTER branch. The most - recent supported kernel in the MASTER branch can be found in the source directory in - lustre/kernel_patches/which_patch. If the MASTER branch is + to a kernel distribution and a corresponding update of the master branch. The most + recent supported kernel in the master branch can be found in the source directory in + lustre/kernel_patches/which_patch. If the master branch is not current with the latest distribution, download the most recent kernel RPMs from the vendor's download site. @@ -266,7 +257,7 @@ gpg: WARNING: unsafe permissions on homedir `.' The kernel source with the Red Hat Enterprise Linux patches applied is now residing in the directory - /home/build/kernel/rpmbuild/BUILD/kernel-2.6.32-131.2.1.el6/linux-2.6.32-131.2.1.el6.x86_64/ + /home/build/kernel/rpmbuild/BUILD/kernel-2.6.32-358.14.1.el6.x86_64/
Patching the Kernel Source with the Lustre Code @@ -275,8 +266,8 @@ gpg: WARNING: unsafe permissions on homedir `.' Go to the directory containing the kernel source: - #cd ~/kernel/rpmbuild/BUILD/kernel-2.6.32-131.2.1.el6/ - linux-2.6.32-131.2.1.el6.x86_64/ + #cd ~/kernel/rpmbuild/BUILD/kernel-2.6.32-358.14.1.el6/ + linux-2.6.32-358.14.1.el6.x86_64/ @@ -284,7 +275,7 @@ gpg: WARNING: unsafe permissions on homedir `.' id to be able to ascertain that the kernel is booted. Modify line 4 as shown below: - EXTRAVERSION = .lustremaster + EXTRAVERSION = -358.14.1.el6_lustre.x86_64 @@ -315,6 +306,8 @@ patching file fs/jbd2/transaction.c Hunk #3succeeded at 1222(offset 3lines). Hunk #4succeeded at 1357(offset 3lines). Now at patch patches/jbd2-jcberr-2.6-rhel6.patch + If quilt is not installed, the following steps would be equivalent: +for PATCH in $(cat series); do patch -p1 < patches/$PATCH; done
@@ -346,16 +339,16 @@ Now at patch patches/jbd2-jcberr-2.6-rhel6.patch ... ... Wrote: /home/build/kernel/rpmbuild/SRPMS/ - kernel-2.6.32lustremaster-1.src.rpm + kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.src.rpm Wrote: /home/build/kernel/rpmbuild/RPMS/x86_64/ - kernel-2.6.32.lustremaster-1.x86_64.rpm + kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.f73m1V + umask 022+ cd /home/build/kernel/rpmbuild/BUILD -+ cd kernel-2.6.32lustremaster ++ cd kernel-2.6.32-358.14.1.el6_lustre.x86_64 + rm -rf /home/build/kernel/rpmbuild/BUILDROOT/ - kernel-2.6.32.lustremaster-1.x86_64 + kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64 + exit 0 -rm ../kernel-2.6.32lustremaster.tar.gz +rm ../kernel-2.6.3-358.14.1.el6_lustre.x86_64.tar.gz If a request to generate more entropy appears, some disk or keyboard I/O needs to be generated. You can generate entropy by entering the following on another @@ -364,7 +357,7 @@ rm ../kernel-2.6.32lustremaster.tar.gz A fresh kernel RPM can now be found at - ~/kernel/rpmbuild/RPMS/x86_64/kernel-2.6.32.lustremaster-1.x86_64.rpm. + ~/kernel/rpmbuild/RPMS/x86_64/kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64.rpm.
Configuring and Building Lustre RPMs @@ -374,7 +367,7 @@ rm ../kernel-2.6.32lustremaster.tar.gz # cd ~/lustre-release/ # ./configure --with-linux=/home/build/kernel/rpmbuild/BUILD/ - kernel-2.6.32.lustremaster/ + kernel-2.6.32-358.14.1.el6_lustre.x86_64/ Text similar to the following is displayed: @@ -405,21 +398,21 @@ Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.TsLWpD + cd /home/build/kernel/rpmbuild/BUILD + cd lustre-2.0.61 + rm -rf /home/build/kernel/rpmbuild/BUILDROOT/ - lustre-2.0.61-2.6.32_lustremaster_g0533e7b.x86_64 + lustre-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64 + exit 0 make[1]: Leaving directory `/home/build/lustre-release' The resulting RPMs are in - ~build/kernel/rpmbuild/RPMS/x86_64/.kernel-2.6.32lustremaster-1.x86_64.rpm -lustre-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm -lustre-debuginfo-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm -lustre-ldiskfs-3.3.0-2.6.32.lustremaster_g0533e7b.x86_64.rpm -lustre-ldiskfs-debuginfo-3.3.0-2.6.32.lustremaster_g0533e7b.x86_64.rpm -lustre-modules-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm -lustre-source-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm -lustre-tests-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm + ~build/kernel/rpmbuild/RPMS/x86_64/.kernel-2.6.3-358.14.1.el6_lustre.x86_64-1.x86_64.rpm +lustre-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm +lustre-debuginfo-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm +lustre-ldiskfs-3.3.0-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm +lustre-ldiskfs-debuginfo-3.3.0-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm +lustre-modules-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm +lustre-source-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm +lustre-tests-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_g0533e7b.x86_64.rpm
Installing the Lustre Kernel @@ -428,33 +421,15 @@ lustre-tests-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm As root, install the kernel: # rpm -ivh ~build/kernel/rpmbuild/RPMS/x86_64/ - kernel-2.6.32.lustremaster-1.x86_64.rpm + kernel-2.6.32-358.14.1.el6_lustre.x86_64-1.x86_64.rpm Create initrd using dracut: # /sbin/new-kernel-pkg --package kernel --mkinitrd --dracut - --depmod --install 2.6.32.lustremaster - - - - (Optional) To mount the Lustre file system at boot, modify - /etc/fstab to add required entries. - In Lustre Release 2.4, the script - /etc/init.d/lustre has been provided to allow the Lustre file - system to be enabled as a service. To enable the service, enter the command - chkconfig lustre on. - - - (Optional) Customize the LNET configuration: - - # vi /etc/modprobe.d/lustre.conf + --depmod --install 2.6.32-358.14.1.el6_lustre.x86_64 - In Lustre Release 2.4, the script - /etc/init.d/lnet has been provided to allow LNET to be enabled as - a service (for example, on a router). To enable this service, enter the command - chkconfig lnet on. Reboot the system: @@ -463,7 +438,7 @@ lustre-tests-2.0.61-2.6.32.lustremaster_g0533e7b.x86_64.rpm A login prompt such as that shown below indicates success:Red Hat Enterprise Linux Server release 6.0(Santiago) -Kernel 2.6.32lustremaster on an x86_64 +Kernel 2.6.3-358.14.1.el6_lustre.x86_64 on an x86_64 client-10login: @@ -477,7 +452,7 @@ client-10login: suite.
Installing <package>e2fsprogs</package> - The e2fsprogs package is required to run the test suite. To download + The e2fsprogs package is required on all Lustre file system server nodes. To download and install e2fsprogs, complete these steps. Download e2fsprogs from the Install the e2fsprogs package: # rpm -Uvh - ./e2fsprogs-1.42.6.wc2-7.el6.x86_64.rpm - ./e2fsprogs-libs-1.42.6.wc2-7.el6.x86_64.rpm + ./e2fsprogs-1.42.7.wc2-7.el6.x86_64.rpm + ./e2fsprogs-libs-1.42.7.wc2-7.el6.x86_64.rpm @@ -510,9 +485,9 @@ client-10login: Install the RPMs: - # rpm -ivh lustre-ldiskfs-3.3.0-2.6.32.lustremaster* -# rpm -ivh lustre-modules-2.0.61-2.6.32.lustremaster* -# rpm -ivh lustre-2.0.61-2.6.32.lustremaster_* + # rpm -ivh lustre-ldiskfs-3.3.0-2.6.32-358.14.1.el6_lustre.x86_64* +# rpm -ivh lustre-modules-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64* +# rpm -ivh lustre-2.0.61-2.6.32-358.14.1.el6_lustre.x86_64_* # rpm -ivh lustre-tests-* @@ -522,7 +497,7 @@ client-10login: Running the Test Suite To test a single node Lustre file system installation, complete these steps. - Run the test suite. + Mount the Lustre file system. # /usr/lib64/lustre/tests/llmount.sh @@ -567,6 +542,7 @@ Using TIMEOUT=20 disable quota as required The Lustre file system is now available at /mnt/lustre. + The Lustre file system available at /mnt/lustre is a test file system built from small loopback devices in /tmp and not for production use. If you see the error below, associate the IP address of a non-loopback interface with the name of your machine in the file diff --git a/LustreProgrammingInterfaces.xml b/LustreProgrammingInterfaces.xml index cf67023..732ae1f 100644 --- a/LustreProgrammingInterfaces.xml +++ b/LustreProgrammingInterfaces.xml @@ -30,7 +30,7 @@
Synopsis - The MDS uses the utility lctl get_param + The MDS uses the utility as specified by lctl get_param mdt.${FSNAME}-MDT{xxxx}.identity_upcall to look up the supplied UID in order to retrieve the user's supplementary group membership. The result is temporarily cached in the kernel (for five minutes, by default) to avoid the overhead of calling into userspace diff --git a/UpgradingLustre.xml b/UpgradingLustre.xml index e8aa5f0..a7738e9 100644 --- a/UpgradingLustre.xml +++ b/UpgradingLustre.xml @@ -23,23 +23,21 @@ upgrading Release Interoperability and Upgrade Requirements - Lustre 2.x release (major) + Lustre software version 2.x release (major) upgrade: All servers must be upgraded at the same time, while some or all clients may be upgraded. - All servers must be be upgraded to Red Hat* Enterprise Linux* 6 (RHEL 6) or other - compatible Linux distribution. See the Linux Test Matrix at All servers must be be upgraded to a Linux kernel supported by the Lustre software. See the Linux Test Matrix at for a - list of tested Linux distros. + list of tested Lustre distributions. - Clients to be upgraded to the Lustre 2.4 release or higher must be running RHEL 6 or - other compatible Linux distribution. See the Linux Test Matrix at Clients to be upgraded to the Lustre 2.4 release or higher must be running a compatible Linux distribution. See the Linux Test Matrix at for a - list of tested Linux distros. + list of tested Linux distributions. Lustre 2.X.y release (minor) @@ -82,11 +80,11 @@ it:tune2fs -O large_xattrOnce the wide striping feature is enabled and in use on the MDT, it is not possible to directly downgrade the MDT file system to an earlier version of the Lustre software that does not support wide striping. The only way to disable - wide striping is to delete all files with an extended attribute - (xattr), unmount the MDT, and run the following command to + wide striping is to delete all files with a large_xattr extended attribute, + unmount the MDT, and run the following command to turn off the large-xattr - option:tune2fs -O ^large_xattr - + option:tune2fs -O ^large_xattr + In Lustre release 2.4, a new feature allows using multiple MDTs, which can each serve one or more remote sub-directories in the file system. The root directory @@ -128,8 +126,8 @@ - Upgrade the Linux operating system on all servers to RHEL 6 or other compatible - (tested) distribution and reboot. See the Linux Test Matrix at Upgrade the Linux operating system on all servers to a compatible + (tested) Linux distribution and reboot. See the Linux Test Matrix at . @@ -158,7 +156,7 @@ Verify the packages are installed correctly: - rpm -qa|egrep "lustre|wc"|sort + rpm -qa|egrep "lustre|wc" @@ -194,7 +192,7 @@ Verify the packages were installed correctly: - # rpm -qa|egrep "lustre|kernel"|sort + # rpm -qa|egrep "lustre|kernel" @@ -227,7 +225,7 @@ (Optional) - If you are upgrading to Lustre release 2.3, or a release previous to Lustre release 2.3, + If you are upgrading to Lustre software release 2.3 or higher from Lustre software version 2.2 or earlier and want to enable the quota feature, complete these steps: @@ -330,7 +328,7 @@ conf_param $FSNAME.quota.ost=$QUOTA_TYPE Verify the packages are installed correctly: - rpm -qa|egrep "lustre|wc"|sort + rpm -qa|egrep "lustre|wc" @@ -364,7 +362,7 @@ conf_param $FSNAME.quota.ost=$QUOTA_TYPE Verify the packages were installed correctly: - # rpm -qa|egrep "lustre|kernel"|sort + # rpm -qa|egrep "lustre|kernel" -- 1.8.3.1