X-Git-Url: https://git.whamcloud.com/?a=blobdiff_plain;f=LustreMaintenance.xml;h=ac0a3bf79487f8f99206cb8f016aaec2ef8fceda;hb=00c99af814574fe85ae7bea886d9fffcce4d0261;hp=c95f7f295e5ae90e9b2870ac59df47311c18abae;hpb=25106a04c49c4acef6322b68ce22217fac9e523c;p=doc%2Fmanual.git diff --git a/LustreMaintenance.xml b/LustreMaintenance.xml index c95f7f2..ac0a3bf 100644 --- a/LustreMaintenance.xml +++ b/LustreMaintenance.xml @@ -1,4 +1,7 @@ - + + Lustre Maintenance Once you have the Lustre file system up and running, you can use the procedures in this section to perform these basic Lustre maintenance tasks: @@ -62,6 +65,9 @@ + + +
@@ -412,7 +418,7 @@ Adding a New OST to a Lustre File System Add a new OST by using mkfs.lustre as when the filesystem was first formatted, see - for details. Each new OST + for details. Each new OST must have a unique index number, use lctl dl to see a list of all OSTs. For example, to add a new OST at index 12 to the testfs filesystem run following commands @@ -440,7 +446,7 @@ oss# mount -t lustre /dev/sda /mnt/testfs/ost12 system on OST0004 that are larger than 4GB in size to other OSTs, enter: client# lfs find /test --ost test-OST0004 -size +4G | lfs_migrate -y - See for details. + See for details.
@@ -470,14 +476,14 @@ Removing and Restoring MDTs and OSTs A hard drive has failed and a RAID resync/rebuild is underway, though the OST can also be marked degraded by the RAID system to avoid allocating new files on the slow OST which - can reduce performance, see + can reduce performance, see for more details. OST is nearing its space capacity, though the MDS will already try to avoid allocating new files on overly-full OSTs if possible, - see for details. + see for details. @@ -619,14 +625,45 @@ client$ lfs getstripe --mdt-index /mnt/lustre/local_dir0 If there is not expected to be a replacement for this OST in - the near future, permanently deactivate it on all clients and - the MDS by running the following command on the MGS: - mgs# lctl conf_param ost_name.osc.active=0 + the near future, permanently deactivate it on all clients and + the MDS by running the following command on the MGS: + mgs# lctl conf_param ost_name.osc.active=0 A deactivated OST still appears in the file system - configuration, though a replacement OST can be created using the + configuration, though a replacement OST can be created that + re-uses the same OST index with the mkfs.lustre --replace option, see . + To totally remove the OST from the filesystem configuration, + the OST configuration records should be found in the startup + logs by running the command + "lctl --device MGS llog_print fsname-client" + on the MGS (and also + "... $fsname-MDTxxxx" + for all the MDTs) to list all attach, + setup, add_osc, + add_pool, and other records related to the + removed OST(s). Once the index value is + known for each configuration record, the command + "lctl --device MGS llog_cancel llog_name -i index " + will drop that record from the configuration log + llog_name for each of the + fsname-client and + fsname-MDTxxxx + configuration logs so that new mounts will no longer process it. + If a whole OSS is being removed, theadd_uuid + records for the OSS should similarly be canceled. + +mgs# lctl --device MGS llog_print testfs-client | egrep "192.168.10.99@tcp|OST0003" +- { index: 135, event: add_uuid, nid: 192.168.10.99@tcp(0x20000c0a80a63), node: 192.168.10.99@tcp } +- { index: 136, event: attach, device: testfs-OST0003-osc, type: osc, UUID: testfs-clilov_UUID } +- { index: 137, event: setup, device: testfs-OST0003-osc, UUID: testfs-OST0003_UUID, node: 192.168.10.99@tcp } +- { index: 138, event: add_osc, device: testfs-clilov, ost: testfs-OST0003_UUID, index: 3, gen: 1 } +mgs# lctl --device MGS llog_cancel testfs-client -i 138 +mgs# lctl --device MGS llog_cancel testfs-client -i 137 +mgs# lctl --device MGS llog_cancel testfs-client -i 136 + + @@ -679,7 +716,7 @@ oss# mount -t ldiskfs /dev/ost_device /mnt/ost Restoring OST Configuration Files If the original OST is still available, it is best to follow the OST backup and restore procedure given in either - , or + , or and . To replace an OST that was removed from service due to corruption @@ -721,7 +758,7 @@ oss# mount -t ldiskfs /dev/new_ost_dev / Recreate the OST configuration files, if unavailable. Follow the procedure in - to recreate the LAST_ID + to recreate the LAST_ID file for this OST index. The last_rcvd file will be recreated when the OST is first mounted using the default parameters, which are normally correct for all file systems. The @@ -872,4 +909,26 @@ touch: cannot touch ‘test_file’: Read-only file system mds# lctl set_param mdt.fs-MDT0000.readonly=0 mdt.fs-MDT0000.readonly=0 +
+ <indexterm><primary>maintenance</primary> + <secondary>Tune fallocate</secondary></indexterm> + Tune Fallocate for ldiskfs + This section shows how to tune/enable/disable fallocate for + ldiskfs OSTs. + The default mode=0 is the standard + "allocate unwritten extents" behavior used by ext4. This is by far the + fastest for space allocation, but requires the unwritten extents to be + split and/or zeroed when they are overwritten. + The OST fallocate mode=1 can also be set to use + "zeroed extents", which may be handled by "WRITE SAME", "TRIM zeroes data", + or other low-level functionality in the underlying block device. + mode=-1 completely disables fallocate. + Example: To completely disable fallocate + lctl set_param osd-ldiskfs.*.fallocate_zero_blocks=-1 + Example: To enable fallocate to use 'zeroed extents' + lctl set_param osd-ldiskfs.*.fallocate_zero_blocks=1 +
+