From 3eba9c153757b350882c28161e45b4e5815617b2 Mon Sep 17 00:00:00 2001 From: Joseph Gmitter Date: Mon, 20 Nov 2017 11:13:09 -0500 Subject: [PATCH] LUDOC-161 zfs: Backup/Restore of ZFS backend filesystems This patch documents how to backup/restore ZFS backend filesystems at the file system level using 'tar'. The existing section on backing up an OST or MDT at the filesystem level was updated to capture zfs. Also, a section has been added to discuss migrating between ZFS and ldiskfs backends using the filesystem level backup/restore process. This work has landed to master under LU-7585. Signed-off-by: Joseph Gmitter Change-Id: I20619e42c7313657599788a23caf4e867f714db6 Reviewed-on: https://review.whamcloud.com/30172 Reviewed-by: James Nunez Tested-by: Jenkins --- BackupAndRestore.xml | 201 +++++++++++++++++++++++++++++++++++++++++--------- LustreMaintenance.xml | 22 +++--- LustreOperations.xml | 2 +- 3 files changed, 180 insertions(+), 45 deletions(-) diff --git a/BackupAndRestore.xml b/BackupAndRestore.xml index 136afc6..f7c97ee 100644 --- a/BackupAndRestore.xml +++ b/BackupAndRestore.xml @@ -20,12 +20,12 @@ xml:id="backupandrestore"> - + - + @@ -437,7 +437,7 @@ Changelog records consumed: 42 restore is detected (see LU-957), and file-level backup is supported (see - ). + ). If hardware replacement is the reason for the backup or if a spare storage device is available, it is possible to do a raw copy of the MDT or @@ -474,7 +474,7 @@ Changelog records consumed: 42 backup device, and is still much better than not having any backup at all. -
+
<indexterm> <primary>backup</primary> @@ -483,10 +483,10 @@ Changelog records consumed: 42</screen> <indexterm> <primary>backup</primary> <secondary>MDT file system</secondary> - </indexterm>Backing Up an OST or MDT (ldiskfs File System Level) + Backing Up an OST or MDT (Backend File System Level) This procedure provides an alternative to backup or migrate the data of an OST or MDT at the file level. At the file-level, unused space is - omitted from the backed up and the process may be completed quicker with + omitted from the backup and the process may be completed quicker with a smaller total backup size. Backing up a single OST device is not necessarily the best way to perform backups of the Lustre file system, since the files stored in the backup are not usable without metadata stored @@ -494,37 +494,85 @@ Changelog records consumed: 42 it is the preferred method for migration of OST devices, especially when it is desirable to reformat the underlying file system with different configuration options or to reduce fragmentation. - - Prior to Lustre software release 2.3, the only successful way to - perform an MDT backup and restore is to do a device-level backup as is - described in + + Prior to Lustre software release 2.3, the + only successful way to perform an MDT backup and restore was to do a + device-level backup as described in . The ability to do MDT file-level backups is not available for Lustre software release 2.0 through 2.2, because restoration of the Object Index (OI) file does not - return the MDT to a functioning state. - Since Lustre software release 2.3, + return the MDT to a functioning state. + Since Lustre software release 2.3, Object Index files are automatically rebuilt at first mount after a restore is detected (see LU-957), - so file-level MDT restore is supported. - - For Lustre software release 2.3 and newer with MDT file-level backup - support, substitute - mdt for - ost in the instructions below. - - - - Make a mountpoint for the file - system. - - [oss]# mkdir -p /mnt/ost - - - - Mount the file system. - + so file-level MDT restore is supported. +
+ + <indexterm> + <primary>backup</primary> + <secondary>index objects</secondary> + </indexterm>Backing Up an OST or MDT (Backend File System Level) + Prior to Lustre software release 2.11.0, we can only do the backend + file system level backup and restore process for ldiskfs-based systems. + The ability to perform a zfs-based MDT/OST file system level backup and + restore is introduced beginning in Lustre software release 2.11.0. + Differing from an ldiskfs-based system, index objects must be backed up + before the unmount of the target (MDT or OST) in order to be able to + restore the file system successfully. To enable index backup on the + target, execute the following command on the target server: + # lctl set_param osd-zfs.${fsname}-${target}.index_backup=1 + ${target} is composed of the target type + (MDT or OST) plus the target index, such as MDT0000, + OST0001, and so on. + The index_backup is also valid for an ldiskfs-based system, + that will be used when migrating data between ldiskfs-based and + zfs-based systems as described in . + +
+
+ + <indexterm> + <primary>backup</primary> + <secondary>OST and MDT</secondary> + </indexterm>Backing Up an OST or MDT + For Lustre software release 2.3 and newer with MDT file-level backup + support, substitute mdt for ost + in the instructions below. + + + Umount the target + + + Make a mountpoint for the file system. + + [oss]# mkdir -p /mnt/ost + + + Mount the file system. + For ldiskfs-based systems: [oss]# mount -t ldiskfs /dev/{ostdev} /mnt/ost + For zfs-based systems: + + + Import the pool for the target if it is exported. For example: + [oss]# zpool import lustre-ost [-d ${ostdev_dir}] + + + + Enable the canmount property on the target + filesystem. For example: + [oss]# zfs set canmount=on ${fsname}-ost/ost + You also can specify the mountpoint property. By default, it will + be: /${fsname}-ost/ost + + + + Mount the target as 'zfs'. For example: + [oss]# zfs mount ${fsname}-ost/ost + + + @@ -614,8 +662,9 @@ trusted.fid= \ +
-
+
<indexterm> <primary>backup</primary> @@ -626,15 +675,38 @@ trusted.fid= \ <orderedlist> <listitem> <para>Format the new device.</para> - <screen>[oss]# mkfs.lustre --ost --index {<emphasis>OST index</emphasis>} {<emphasis>other options</emphasis>} /dev/<emphasis>{newdev}</emphasis></screen> + <screen>[oss]# mkfs.lustre --ost --index {<emphasis>OST index</emphasis>} +--replace --fstype=${fstype} {<emphasis>other options</emphasis>} /dev/<emphasis>{newdev}</emphasis></screen> </listitem> <listitem> - <para>Set the file system label.</para> + <para>Set the file system label (<emphasis role="bold">ldiskfs-based + systems only</emphasis>).</para> <screen>[oss]# e2label {fsname}-OST{index in hex} /mnt/ost</screen> </listitem> <listitem> <para>Mount the file system.</para> + <para>For ldiskfs-based systems:</para> <screen>[oss]# mount -t ldiskfs /dev/<emphasis>{newdev}</emphasis> /mnt/ost</screen> + <para>For zfs-based systems:</para> + <orderedlist> + <listitem> + <para>Import the pool for the target if it is exported. For example: + </para> + <screen>[oss]# zpool import lustre-ost [-d ${ostdev_dir}]</screen> + </listitem> + <listitem> + <para>Enable the canmount property on the target filesystem. For + example:</para> + <screen>[oss]# zfs set canmount=on ${fsname}-ost/ost</screen> + <para>You also can specify the <literal>mountpoint</literal> + property. By default, it will be: + <literal>/${fsname}-ost/ost</literal></para> + </listitem> + <listitem> + <para>Mount the target as 'zfs'. For example:</para> + <screen>[oss]# zfs mount ${fsname}-ost/ost</screen> + </listitem> + </orderedlist> </listitem> <listitem> <para>Change to the new file system mount point.</para> @@ -669,8 +741,8 @@ trusted.fid= \ 0x0d822200000000004a8a73e500000000808a0100000000000000000000000000</screen> </listitem> <listitem> - <para>Remove old OI files.</para> - <screen>[oss]# rm -f oi.16*</screen> + <para>Remove old OI and LFSCK files.</para> + <screen>[oss]# rm -rf oi.16* lfsck_* LFSCK</screen> </listitem> <listitem> <para>Remove old CATALOGS.</para> @@ -696,6 +768,23 @@ trusted.fid= \ <listitem> <para>Unmount the new file system.</para> <screen>[oss]# umount /mnt/ost</screen> + <note><para>If the restored system has a different NID from the backup + system, please change the NID. For detail, please refer to + <xref linkend="dbdoclet.changingservernid" />. For example:</para> + <screen>[oss]# mount -t lustre -o nosvc ${fsname}-ost/ost /mnt/ost +[oss]# lctl replace_nids ${fsname}-OSTxxxx $new_nids +[oss]# umount /mnt/ost</screen></note> + </listitem> + <listitem> + <para>Mount the target as <literal>lustre</literal>.</para> + <para>Usually, we will use the <literal>-o abort_recov</literal> option + to skip unnecessary recovery. For example:</para> + <screen>[oss]# mount -t lustre -o abort_recov #{fsname}-ost/ost /mnt/ost</screen> + <para>Lustre can detect the restore automatically when mounting the + target, and then trigger OI scrub to rebuild the OIs and index objects + asynchronously in the background. You can check the OI scrub status + with the following command:</para> + <screen>[oss]# lctl get_param -n osd-${fstype}.${fsname}-${target}.oi_scrub</screen> </listitem> </orderedlist> <para condition='l23'>If the file system was used between the time the @@ -987,4 +1076,46 @@ fstab passwds </note> </section> </section> + <section xml:id="migrate_backends" condition="l2B"> + <title> + <indexterm> + <primary>backup</primary> + <secondary>ZFS ZPL</secondary> + </indexterm>Migration Between ZFS and ldiskfs Target Filesystems + + Beginning with Lustre 2.11.0, it is possible to migrate between + ZFS and ldiskfs backends. For migrating OSTs, it is best to use + lfs find/lfs_migrate to empty out + an OST while the filesystem is in use and then reformat it with the new + fstype. For instructions on removing the OST, please see + . +
+ + <indexterm> + <primary>backup</primary> + <secondary>ZFS to ldiskfs</secondary> + </indexterm>Migrate from a ZFS to an ldiskfs based filesystem + The first step of the process is to make a ZFS backend backup + using tar as described in + . + Next, restore the backup to an ldiskfs-based system as described + in . +
+
+ + <indexterm> + <primary>backup</primary> + <secondary>ZFS to ldiskfs</secondary> + </indexterm>Migrate from an ldiskfs to a ZFS based filesystem + The first step of the process is to make an ldiskfs backend backup + using tar as described in + . + Caution:For a migration from + ldiskfs to zfs, it is required to enable index_backup before the + unmount of the target. This is an additional step for a regular + ldiskfs-based backup/restore and easy to be missed. + Next, restore the backup to an ldiskfs-based system as described + in . +
+
diff --git a/LustreMaintenance.xml b/LustreMaintenance.xml index 9b220ed..dfeaa6f 100644 --- a/LustreMaintenance.xml +++ b/LustreMaintenance.xml @@ -1,4 +1,4 @@ - + Lustre Maintenance Once you have the Lustre file system up and running, you can use the procedures in this section to perform these basic Lustre maintenance tasks: @@ -15,7 +15,7 @@ - + @@ -215,7 +215,7 @@ Regenerating Lustre Configuration Logs After the writeconf command is run, the configuration logs are re-generated as servers restart.
-
+
<indexterm><primary>maintenance</primary><secondary>changing a NID</secondary></indexterm> Changing a Server NID In Lustre software release 2.3 or earlier, the tunefs.lustre @@ -609,8 +609,8 @@ oss# mount -t ldiskfs /dev/ost_device /mnt/ostIf the original OST is still available, it is best to follow the OST backup and restore procedure given in either , or - and - . + and + . To replace an OST that was removed from service due to corruption or hardware failure, the replacement OST needs to be formatted using mkfs.lustre, and the Lustre file system configuration @@ -727,10 +727,14 @@ Changing the Address of a Failover Node linkend="configuringfailover"/>.
- <indexterm><primary>maintenance</primary><secondary>separate a combined MGS/MDT</secondary></indexterm> -Separate a combined MGS/MDT - These instructions assume the MGS node will be the same as the MDS node. For instructions on how to move MGS to a different node, see . - These instructions are for doing the split without shutting down other servers and clients. + <indexterm><primary>maintenance</primary><secondary>separate a + combined MGS/MDT</secondary></indexterm> + Separate a combined MGS/MDT + These instructions assume the MGS node will be the same as the MDS + node. For instructions on how to move MGS to a different node, see + . + These instructions are for doing the split without shutting down + other servers and clients. Stop the MDS. diff --git a/LustreOperations.xml b/LustreOperations.xml index d5f2e31..7a6b82c 100644 --- a/LustreOperations.xml +++ b/LustreOperations.xml @@ -923,7 +923,7 @@ tune2fs [-m reserved_blocks_percent] /dev/ To copy the contents of an existing OST to a new OST (or an old MDT to a new MDT), follow the process for either OST/MDT backups in or - . + . For more information on removing a MDT, see .
-- 1.8.3.1