X-Git-Url: https://git.whamcloud.com/?a=blobdiff_plain;f=UpgradingLustre.xml;h=d009f88e6d221eebe6e80816e932df1b4cf9b08f;hb=c485024a05dc155017460cf49055dae43c5bb47d;hp=3c236f9078085127aec1a331adc29f99d6366989;hpb=d61df1724de9b8c6ae47d54e625ed4a4f78d3fcc;p=doc%2Fmanual.git diff --git a/UpgradingLustre.xml b/UpgradingLustre.xml index 3c236f9..d009f88 100644 --- a/UpgradingLustre.xml +++ b/UpgradingLustre.xml @@ -1,177 +1,215 @@ - - Upgrading a Lustre* File System - This chapter describes interoperability between Lustre* software releases. It also provides - procedures for upgrading from a Lustre 1.8 release to a Lustre 2.x release, from a Lustre 2.x - release to a more recent Lustre 2.x release (major release upgrade), and from a Lustre 2.X.y - release to 2.X.y (minor release upgrade). It includes the following sections: + + + Upgrading a Lustre File System + This chapter describes interoperability between Lustre software + releases. It also provides procedures for upgrading from Lustre software + release 1.8 to Lustre software release 2.x , from a Lustre software release + 2.x to a more recent Lustre software release 2.x (major release upgrade), and + from a a Lustre software release 2.x.y to a more recent Lustre software + release 2.x.y (minor release upgrade). It includes the following + sections: - + + + - + + + - + + +
- <indexterm> - <primary>Lustre</primary> - <secondary>upgrading</secondary> - <see>upgrading</see> - </indexterm><indexterm> - <primary>upgrading</primary> - </indexterm>Release Interoperability and Upgrade Requirements - Lustre software version 2.x release (major) - upgrade: + + <indexterm> + <primary>Lustre</primary> + <secondary>upgrading</secondary> + <see>upgrading</see> + </indexterm> + <indexterm> + <primary>upgrading</primary> + </indexterm>Release Interoperability and Upgrade Requirements + + + Lustre software release 2.x (major) + upgrade: + + - All servers must be upgraded at the same time, while some or all clients may be - upgraded. + All servers must be upgraded at the same time, while some or + all clients may be upgraded independently of the servers. - All servers must be be upgraded to a Linux kernel supported by the Lustre software. See the Linux Test Matrix at for a - list of tested Lustre distributions. + All servers must be be upgraded to a Linux kernel supported by + the Lustre software. See the Lustre Release Notes for your Lustre + version for a list of tested Linux distributions. - Clients to be upgraded to the Lustre 2.4 release or higher must be running a compatible Linux distribution. See the Linux Test Matrix at for a - list of tested Linux distributions. + Clients to be upgraded must be running a compatible Linux + distribution as described in the Release Notes. - - Lustre 2.X.y release (minor) - upgrade: + + + + + Lustre software release 2.x.y release (minor) + upgrade: + + - All servers must be upgraded at the same time, while some or all clients may be - upgraded. + All servers must be upgraded at the same time, while some or all + clients may be upgraded. - Rolling upgrades are supported for minor releases allowing individual servers and - clients to be upgraded without stopping the Lustre file system. + Rolling upgrades are supported for minor releases allowing + individual servers and clients to be upgraded without stopping the + Lustre file system.
- <indexterm> - <primary>upgrading</primary> - <secondary>major release (2.x to 2.x)</secondary> - </indexterm><indexterm> - <primary>wide striping</primary> - </indexterm><indexterm> - <primary>MDT</primary> - <secondary>multiple MDSx</secondary> - </indexterm><indexterm> - <primary>large_xattr</primary> - </indexterm><indexterm> - <primary>wide striping</primary> - <secondary>large_xattr</secondary> - </indexterm>Upgrading to Lustre Release 2.x (Major Release) - The procedure for upgrading a Lustre release 2.x file system to a more recent 2.x release - of the Lustre software is described in this section. + + <indexterm> + <primary>upgrading</primary> + <secondary>major release (2.x to 2.x)</secondary> + </indexterm> + <indexterm> + <primary>wide striping</primary> + </indexterm> + <indexterm> + <primary>MDT</primary> + <secondary>multiple MDSs</secondary> + </indexterm> + <indexterm> + <primary>large_xattr</primary> + <secondary>ea_inode</secondary> + </indexterm> + <indexterm> + <primary>wide striping</primary> + <secondary>large_xattr</secondary> + <tertiary>ea_inode</tertiary> + </indexterm>Upgrading to Lustre Software Release 2.x (Major + Release) + The procedure for upgrading from a Lustre software release 2.x to a + more recent 2.x release of the Lustre software is described in this + section. - This procedure can also be used to upgrade Lustre release 1.8.6-wc1 or later to any - Lustre release 2.x. To upgrade other versions of 1.8.x, contact your support - provider. + This procedure can also be used to upgrade Lustre software release + 1.8.6-wc1 or later to any Lustre software release 2.x. To upgrade other + versions of Lustre software release 1.8.x, contact your support + provider. - In Lustre release 2.2, a feature has been added that allows striping - across up to 2000 OSTs. By default, this "wide striping" feature is disabled. It is - activated by setting the large_xattr option on the MDT using either - mkfs.lustre or tune2fs. For example after upgrading - an existing file system to Lustre release 2.2 or later, wide striping can be enabled by - running the following command on the MDT device before mounting - it:tune2fs -O large_xattrOnce the wide striping feature is enabled and in - use on the MDT, it is not possible to directly downgrade the MDT file system to an earlier - version of the Lustre software that does not support wide striping. To disable wide striping: - - Delete all wide-striped files. - OR - Use lfs_migrate with the option -c - stripe_count (set stripe_count - to 160) to move the files to another location. - - - Unmount the MDT. - - - Run the following command to turn off the large_xattr - option:tune2fs -O ^large_xattr - - - + In Lustre software release 2.2, a feature has been added for + ldiskfs-based MDTs that allows striping a single file across up to 2000 + OSTs. By default, this "wide striping" feature is disabled. It is + activated by setting the ea_inode option on the MDT + using either mkfs.lustre or tune2fs. + For example after upgrading an existing file system to Lustre software + release 2.2 or later, wide striping can be enabled by running the + following command on the MDT device before mounting it: + tune2fs -O large_xattr + Once the wide striping feature is enabled and in use on the MDT, it is + not possible to directly downgrade the MDT file system to an earlier + version of the Lustre software that does not support wide striping. To + disable wide striping: + + + Delete all wide-striped files, OR + use lfs_migrate -c 160 (or fewer stripes) + to migrate the files to use fewer OSTs. This does not affect the + total number of OSTs that the whole filesystem can access. + + + Unmount the MDT. + + + Run the following command to turn off the + large_xattr option: + tune2fs -O ^large_xattr + + Using either + mkfs.lustre or + tune2fs with + large_xattr or + ea_inode option reseults in + ea_inode in the file system feature list. + - To generate a list of all files with more than 160 stripes use lfs - find with the --stripe-count - option:lfs find ${mountpoint} --stripe-count=+160 + To generate a list of all files with more than 160 stripes use + lfs find with the + --stripe-count option: + lfs find ${mountpoint} --stripe-count=+160 - In Lustre release 2.4, a new feature allows using multiple MDTs, which can each serve - one or more remote sub-directories in the file system. The root directory - is always located on MDT0. - Note that clients running a release prior to the Lustre 2.4 release can only see the - namespace hosted by MDT0 and will return an IO error if an attempt is made to access a - directory on another MDT. + In Lustre software release 2.4, a new feature allows using multiple + MDTs, which can each serve one or more remote sub-directories in the file + system. The + root directory is always located on MDT0. + Note that clients running a release prior to the Lustre software + release 2.4 can only see the namespace hosted by MDT0 and will return an + IO error if an attempt is made to access a directory on another + MDT. - To upgrade a Lustre release 2.x file system to a more recent major release, complete these - steps: + To upgrade a Lustre software release 2.x to a more recent major + release, complete these steps: - Create a complete, restorable file system backup. + Create a complete, restorable file system backup. - Before installing the Lustre software, back up ALL data. The Lustre software - contains kernel modifications that interact with storage devices and may introduce - security issues and data loss if not installed, configured, or administered properly. If - a full backup of the file system is not practical, a device-level backup of the MDT file - system is recommended. See for a procedure. + Before installing the Lustre software, back up ALL data. The + Lustre software contains kernel modifications that interact with + storage devices and may introduce security issues and data loss if + not installed, configured, or administered properly. If a full backup + of the file system is not practical, a device-level backup of the MDT + file system is recommended. See + for a procedure. - Shut down the file system by unmounting all clients and servers in the order shown - below (unmounting a block device causes the Lustre software to be shut down on that - node): - - - Unmount the clients. On each client node, run: - umount -a -t lustre - - - Unmount the MDT. On the MDS node, run: - umount -a -t lustre - - - Unmount all the OSTs. On each OSS node, run: - umount -a -t lustre - - + Shut down the entire filesystem by following + Upgrade the Linux operating system on all servers to a compatible - (tested) Linux distribution and reboot. See the Linux Test Matrix at . + (tested) Linux distribution and reboot. - Upgrade the Linux operating system on all clients to RHEL 6 or other compatible - (tested) distribution and reboot. See the Linux Test Matrix at . + Upgrade the Linux operating system on all clients to Red Hat + Enterprise Linux 6 or other compatible (tested) distribution and + reboot. - Download the Lustre server RPMs for your platform from the Lustre Releases - repository. See for a list of required packages. + Download the Lustre server RPMs for your platform from the + + Lustre Releasesrepository. See + for a list of required packages. - Install the Lustre server packages on all Lustre servers (MGS, MDSs, and OSSs). + Install the Lustre server packages on all Lustre servers (MGS, + MDSs, and OSSs). - Log onto a Lustre server as the root user + Log onto a Lustre server as the + root user - Use the yum command to install the packages: + Use the + yum command to install the packages: - # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... + # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... @@ -186,28 +224,33 @@ - Download the Lustre client RPMs for your platform from the Lustre Releases - repository. See for a list of required packages. + Download the Lustre client RPMs for your platform from the + + Lustre Releasesrepository. See + for a list of required packages. - The version of the kernel running on a Lustre client must be the same as the version - of the lustre-client-modules-ver package - being installed. If not, a compatible kernel must be installed on the client before the - Lustre client packages are installed. + The version of the kernel running on a Lustre client must be + the same as the version of the + lustre-client-modules- + verpackage being installed. If not, a + compatible kernel must be installed on the client before the Lustre + client packages are installed. - Install the Lustre client packages on each of the Lustre clients to be - upgraded. + Install the Lustre client packages on each of the Lustre clients + to be upgraded. - Log onto a Lustre client as the root user. + Log onto a Lustre client as the + root user. - Use the yum command to install the packages: + Use the + yum command to install the packages: - # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... + # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... @@ -222,142 +265,182 @@ - (Optional) For upgrades to Lustre release 2.2 or higher, to enable - wide striping on an existing MDT, run the following command on the MDT - :mdt# tune2fs -O large_xattr device + (Optional) For upgrades to Lustre software release 2.2 or higher, + to enable wide striping on an existing MDT, run the following command + on the MDT: + tune2fs -O ea_inode /dev/mdtdev + + For more information about wide striping, see + . - (Optional) For upgrades to Lustre release 2.4 or higher, to format an - additional MDT, complete these steps: - - Determine the index used for the first MDT (each MDT must have unique index). - Enter:client$ lctl dl | grep mdc + (Optional) For upgrades to Lustre software release 2.4 or higher, + to format an additional MDT, complete these steps: + + + Determine the index used for the first MDT (each MDT must + have unique index). Enter: + client$ lctl dl | grep mdc 36 UP mdc lustre-MDT0000-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5 - In this example, the next available index is 1. - - - Add the new block device as a new MDT at the next available index by entering - (on one - line):mds# mkfs.lustre --reformat --fsname=filesystem_name --mdt \ - --mgsnode=mgsnode --index 1 /dev/mdt1_device - - - + In this example, the next available index is 1. + + + Add the new block device as a new MDT at the next available + index by entering (on one line): + mds# mkfs.lustre --reformat --fsname=filesystem_name --mdt \ + --mgsnode=mgsnode --index 1 +/dev/mdt1_device + + - (Optional) If you are upgrading to Lustre software release 2.3 or higher from Lustre - software version 2.2 or earlier and want to enable the quota feature, complete these - steps: - - Before setting up the file system, enter on both the MDS and - OSTs:tunefs.lustre --quota - - - When setting up the file system, - enter:conf_param $FSNAME.quota.mdt=$QUOTA_TYPE + (Optional) If you are upgrading to Lustre software release 2.3 or + higher from Lustre software release 2.2 or earlier and want to enable + the quota feature, complete these steps: + + + Before setting up the file system, enter on both the MDS and + OSTs: + tunefs.lustre --quota + + + (Optional) If you are upgrading before Lustre software release + 2.10, to enable the project quota feature enter the following on every + ldiskfs backend target: + tune2fs –O project /dev/dev + + Enabling the project feature will prevent + the filesystem from being used by older versions of ldiskfs, so it + should only be enabled if the project quota feature is required and/or + after it is known that the upgraded release does not need to be + downgraded. + + + When setting up the file system, enter: + conf_param $FSNAME.quota.mdt=$QUOTA_TYPE conf_param $FSNAME.quota.ost=$QUOTA_TYPE - - + + - (Optional) If you are upgrading from Lustre release 1.8, you must manually enable the - FID-in-dirent feature. On the MDS, - enter:tune2fs –O dirdata /dev/mdtdev + (Optional) If you are upgrading from Lustre software release 1.8, + you must manually enable the FID-in-dirent feature. On the MDS, enter: + tune2fs –O dirdata /dev/mdtdev - This step is not reversible. Do not complete this step until you are sure you will - not be downgrading the Lustre software. + This step is not reversible. Do not complete this step until + you are sure you will not be downgrading the Lustre software. - This step only enables FID-in-dirent for newly created files. If you are upgrading to - Lustre release 2.4, you can use LFSCK 1.5 to enable FID-in-dirent for existing files. For - more information about FID-in-dirent and related functionalities in LFSCK 1.5, see . + This step only enables FID-in-dirent for newly + created files. If you are upgrading to Lustre software release 2.4, + you can use namespace LFSCK to enable FID-in-dirent for the existing + files. For the case of upgrading from Lustre software release 1.8, it is + important to note that if you do NOT enable dirdata via + the tune2fs command above, the namespace LFSCK will NOT + generate FID-in-dirent for the existing files. For more information about + FID-in-dirent and related functionalities in LFSCK, see + . - Start the Lustre file system by starting the components in the order shown in the - following steps: + Start the Lustre file system by starting the components in the + order shown in the following steps: - Mount the MGT. On the MGS, runmgs# mount -a -t lustre + Mount the MGT. On the MGS, run + mgs# mount -a -t lustre - Mount the MDT(s). On each MDT, run:mds# mount -a -t lustre + Mount the MDT(s). On each MDT, run: + mds# mount -a -t lustre Mount all the OSTs. On each OSS node, run: oss# mount -a -t lustre - This command assumes that all the OSTs are listed in the - /etc/fstab file. OSTs that are not listed in the - /etc/fstab file, must be mounted individually by running the - mount command: - mount -t lustre /dev/block_device /mount_point + This command assumes that all the OSTs are listed in the + /etc/fstab file. OSTs that are not listed in + the + /etc/fstab file, must be mounted individually + by running the mount command: + mount -t lustre /dev/block_device/mount_point - Mount the file system on the clients. On each client node, run: + Mount the file system on the clients. On each client node, + run: client# mount -a -t lustre - The mounting order described in the steps above must be followed for the intial mount - and registration of a Lustre file system after an upgrade. For a normal start of a Lustre - file system, the mounting order is MGT, OSTs, MDT(s), clients. + The mounting order described in the steps above must be followed + for the initial mount and registration of a Lustre file system after an + upgrade. For a normal start of a Lustre file system, the mounting order + is MGT, OSTs, MDT(s), clients. - If you have a problem upgrading a Lustre file system, see for some ways - to get help. + If you have a problem upgrading a Lustre file system, see + for some ways to get help.
- <indexterm> - <primary>upgrading</primary> - <secondary>2.X.y to 2.X.y (minor release)</secondary> - </indexterm><indexterm/>Upgrading to a Lustre Release 2.X.y (Minor Release) - Rolling upgrades are supported for upgrading from any Lustre release 2.X.y to a more - recent 2.X.y release. This allows the Lustre file system to continue to run while individual - servers (or their failover partners) and clients are upgraded one at a time. The procedure for - upgrading a Lustre release 2.X.y file system to a more recent minor release is described in - this section. - To upgrade Lustre release 2.X.y to a more recent minor release, complete these - steps: + + <indexterm> + <primary>upgrading</primary> + <secondary>2.X.y to 2.X.y (minor release)</secondary> + </indexterm>Upgrading to Lustre Software Release 2.x.y (Minor + Release) + Rolling upgrades are supported for upgrading from any Lustre software + release 2.x.y to a more recent Lustre software release 2.X.y. This allows + the Lustre file system to continue to run while individual servers (or + their failover partners) and clients are upgraded one at a time. The + procedure for upgrading a Lustre software release 2.x.y to a more recent + minor release is described in this section. + To upgrade Lustre software release 2.x.y to a more recent minor + release, complete these steps: - Create a complete, restorable file system backup. + Create a complete, restorable file system backup. - Before installing the Lustre software, back up ALL data. The Lustre software - contains kernel modifications that interact with storage devices and may introduce - security issues and data loss if not installed, configured, or administered properly. If - a full backup of the file system is not practical, a device-level backup of the MDT file - system is recommended. See for a procedure. + Before installing the Lustre software, back up ALL data. The + Lustre software contains kernel modifications that interact with + storage devices and may introduce security issues and data loss if + not installed, configured, or administered properly. If a full backup + of the file system is not practical, a device-level backup of the MDT + file system is recommended. See + for a procedure. - Download the Lustre server RPMs for your platform from the Lustre Releases - repository. See for a list of required packages. + Download the Lustre server RPMs for your platform from the + + Lustre Releasesrepository. See + for a list of required packages. - For a rolling upgrade, complete any procedures required to keep the Lustre file system - running while the server to be upgraded is offline, such as failing over a primary server - to its secondary partner. + For a rolling upgrade, complete any procedures required to keep + the Lustre file system running while the server to be upgraded is + offline, such as failing over a primary server to its secondary + partner. - Unmount the Lustre server to be upgraded (MGS, MDS, or OSS) + Unmount the Lustre server to be upgraded (MGS, MDS, or + OSS) Install the Lustre server packages on the Lustre server. - Log onto the Lustre server as the root user + Log onto the Lustre server as the + root user - Use the yum command to install the packages: + Use the + yum command to install the packages: - # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... + # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... @@ -368,7 +451,8 @@ conf_param $FSNAME.quota.ost=$QUOTA_TYPE Mount the Lustre server to restart the Lustre software on the - server:server# mount -a -t lustre + server: + server# mount -a -t lustre Repeat these steps on each Lustre server. @@ -376,22 +460,25 @@ conf_param $FSNAME.quota.ost=$QUOTA_TYPE - Download the Lustre client RPMs for your platform from the Lustre Releases - repository. See for a list of required packages. + Download the Lustre client RPMs for your platform from the + + Lustre Releasesrepository. See + for a list of required packages. - Install the Lustre client packages on each of the Lustre clients to be - upgraded. + Install the Lustre client packages on each of the Lustre clients + to be upgraded. - Log onto a Lustre client as the root user. + Log onto a Lustre client as the + root user. - Use the yum command to install the packages: + Use the + yum command to install the packages: - # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... + # yum --nogpgcheck install pkg1.rpm pkg2.rpm ... @@ -402,7 +489,8 @@ conf_param $FSNAME.quota.ost=$QUOTA_TYPE Mount the Lustre client to restart the Lustre software on the - client:client# mount -a -t lustre + client: + client# mount -a -t lustre Repeat these steps on each Lustre client. @@ -410,8 +498,9 @@ conf_param $FSNAME.quota.ost=$QUOTA_TYPE - If you have a problem upgrading a Lustre file system, see for some - suggestions for how to get help. + If you have a problem upgrading a Lustre file system, see + for some suggestions for how to get + help.