From: Richard Henwood Date: Wed, 16 Jul 2014 18:53:16 +0000 (-0500) Subject: LUDOC-249 defrag: lfs_migrate can be used to defrag. X-Git-Tag: 2.8.0~16 X-Git-Url: https://git.whamcloud.com/?a=commitdiff_plain;h=6104427ada621d448b01d99ef9ba902337727779;p=doc%2Fmanual.git LUDOC-249 defrag: lfs_migrate can be used to defrag. Alert the audience that lfs_migrate can be used to defragment a file, provided there is suffecient unfragmented free space available. Wrap lines for sane reading. Change-Id: I3f88debdd2a125a9216a9b2d8b4169426a2a99da Signed-off-by: Richard Henwood Reviewed-on: http://review.whamcloud.com/11119 Tested-by: Jenkins --- diff --git a/UserUtilities.xml b/UserUtilities.xml index a7d9910..308393c 100644 --- a/UserUtilities.xml +++ b/UserUtilities.xml @@ -1,25 +1,39 @@ - + + User Utilities This chapter describes user utilities and includes the following sections: - + + + - + + + - + + + - + + + - + + +
- <indexterm><primary>lfs</primary></indexterm> + <title> + <indexterm> + <primary>lfs</primary> + </indexterm> <literal>lfs</literal> The lfs utility can be used for user configuration routines and monitoring. @@ -99,17 +113,23 @@ lfs help - Option + + Option + - Description + + Description + - changelog + + changelog + Shows the metadata changes on an MDT. Start and end points are optional. The --follow option blocks on new changes; this option is only valid when run directly on the MDT node. @@ -117,7 +137,9 @@ lfs help - changelog_clear + + changelog_clear + Indicates that changelog records previous to endrec are no longer of interest to a particular consumer id, potentially allowing the MDT to free up disk space. An endrec of 0 indicates the current last record. Changelog consumers must be registered on the MDT node using lctl. @@ -144,10 +166,12 @@ lfs help By default, the usage of all mounted Lustre file systems is reported. If the path option is included, only the usage for the specified file system is reported. If the -h option is included, the - output is printed in human-readable format, using SI base-2 suffixes for Mega-, Giga-, Tera-, Peta-, or - Exabytes. + output is printed in human-readable format, using SI base-2 suffixes for + Mega-, + Giga-, + Tera-, + Peta-, or + Exabytes. If the --lazy option is specified, any OST that is currently disconnected from the client will be skipped. Using the --lazy option prevents the df output from @@ -159,7 +183,9 @@ lfs help - find + + find + Searches the directory tree rooted at the given directory/filename for files that match the given parameters. @@ -184,10 +210,12 @@ lfs help -   +   - --ctime + + --ctime + File status was last changed N*24 hours ago. @@ -195,10 +223,12 @@ lfs help -   +   - --mtime + + --mtime + File data was last modified N*24 hours ago. @@ -206,10 +236,12 @@ lfs help -   +   - --obd + + --obd + File has an object on a specific OST(s). @@ -217,10 +249,12 @@ lfs help -   +   - --size + + --size + File has a size in bytes, or kilo-, Mega-, Giga-, Tera-, Peta- or Exabytes if a suffix is given. @@ -228,10 +262,12 @@ lfs help -   +   - --type + + --type + File has the type - block, character, directory, pipe, file, symlink, socket @@ -240,10 +276,12 @@ lfs help -   +   - --uid + + --uid + File has a specific numeric user ID. @@ -251,10 +289,12 @@ lfs help -   +   - --user + + --user + File owned by a specific user (numeric user ID allowed). @@ -262,10 +302,12 @@ lfs help -   +   - --gid + + --gid + File has a specific group ID. @@ -273,10 +315,12 @@ lfs help -   +   - --group + + --group + File belongs to a specific group (numeric group ID allowed). @@ -284,7 +328,7 @@ lfs help -   +   --maxdepth @@ -295,10 +339,10 @@ lfs help -   +   - --print / --print0 + --print / --print0 Prints the full filename, followed by a new line or NULL character correspondingly. @@ -306,7 +350,9 @@ lfs help - osts [path] + + osts [path] + Lists all OSTs for the file system. If a path located on a mounted Lustre file @@ -316,10 +362,12 @@ lfs help - getname [path...] + + getname [path...] + - List each Lustre file system instance associated with each Lustre mount + List each Lustre file system instance associated with each Lustre mount point. If no path is specified, all Lustre mount points are interrogated. If a list of paths is provided, the instance of each path is provided. If the path is not a Lustre instance 'No such device' is returned. @@ -327,7 +375,9 @@ lfs help - getstripe + + getstripe + Lists striping information for a given filename or directory. By default, the stripe count, stripe size and offset are returned. @@ -336,15 +386,17 @@ lfs help printed without substituting the file system default values for unspecified fields. If the striping EA is not set, 0, 0, and -1 will be printed for the stripe count, size, and offset respectively. - The -M prints the index of the MDT for a given directory. See . + The -M prints the index of the MDT for a given directory. See . -   +   - --obd ost_name + + --obd ost_name + Lists files that have an object on a specific OST. @@ -352,21 +404,25 @@ lfs help -   +   - --quiet + + --quiet + - Lists details about the file's object ID information. + Lists details about the file's object ID information. -   +   - --verbose + + --verbose + Prints additional striping information. @@ -374,10 +430,12 @@ lfs help -   +   - --count + + --count + Lists the stripe count (how many OSTs to use). @@ -385,10 +443,12 @@ lfs help -   +   - --index + + --index + Lists the index for each OST in the file system. @@ -396,10 +456,12 @@ lfs help -   +   - --offset + + --offset + Lists the OST index on which file striping starts. @@ -407,10 +469,12 @@ lfs help -   +   - --pool + + --pool + Lists the pools to which a file belongs. @@ -418,10 +482,12 @@ lfs help -   +   - --size + + --size + Lists the stripe size (how much data to write to one OST before moving to the next OST). @@ -429,10 +495,12 @@ lfs help -   +   - --directory + + --directory + Lists entries about a specified directory instead of its contents (in the same manner as ls -d). @@ -440,10 +508,12 @@ lfs help -   +   - --recursive + + --recursive + Recurses into all sub-directories. @@ -451,20 +521,25 @@ lfs help - setstripe + + setstripe + - Create new files with a specific file layout (stripe pattern) configuration. - The file cannot exist prior to using setstripe. A directory must exist prior to using setstripe. - + Create new files with a specific file layout (stripe + pattern) configuration.The file cannot exist prior to using + setstripe. A directory must exist prior to using + setstripe. -   +   - --count stripe_cnt + + --count stripe_cnt + Number of OSTs over which to stripe a file. A stripe_cnt of 0 uses the file system-wide default stripe count (default is 1). A stripe_cnt of -1 stripes over all available OSTs. @@ -472,23 +547,23 @@ lfs help -   +   - --size stripe_size - The default stripe-size is 0. The default start-ost is -1. Do NOT confuse them! If you set start-ost to 0, all new file creations occur on OST 0 (seldom a good idea). -   + --size stripe_sizeThe default stripe-size is 0. The default start-ost is -1. Do NOT confuse them! If you set start-ost to 0, all new file creations occur on OST 0 (seldom a good idea).  - Number of bytes to store on an OST before moving to the next OST. A stripe_size of 0 uses the file system's default stripe size, (default is 1 MB). Can be specified with k (KB), m (MB), or g (GB), respectively. + Number of bytes to store on an OST before moving to the next OST. A stripe_size of 0 uses the file system's default stripe size, (default is 1 MB). Can be specified with k (KB), m (MB), or g (GB), respectively. -   +   - --index --offset start_ost_index + + --index --offset start_ost_index + The OST index (base 10, starting at 0) on which to start striping for this file. A start_ost_index value of -1 allows the MDS to choose the starting index. This is the default value, and it means that the MDS selects the starting OST as it wants. We strongly recommend selecting this default, as it allows space and load balancing to be done by the MDS as needed. The start_ost_index value has no relevance on whether the MDS will use round-robin or QoS weighted allocation for the remaining stripes in the file. @@ -496,10 +571,12 @@ lfs help -   +   - --pool pool + + --pool pool + Name of the pre-defined pool of OSTs (see ) that will be used for striping. The stripe_cnt, stripe_size and start_ost values are used as well. The start-ost value must be part of the pool or an error is returned. @@ -507,7 +584,9 @@ lfs help - setstripe -d + + setstripe -d + Deletes default striping on the specified directory. @@ -515,24 +594,37 @@ lfs help - poollist {filesystem} [.poolname]|{pathname} + + poollist {filesystem} [.poolname]|{pathname} + - Lists pools in the file system or pathname, or OSTs in the file system's pool. + Lists pools in the file system or pathname, or OSTs in the file system's pool. - quota [-q] [-v] [-o obd_uuid|-i mdt_idx|-I ost_idx] [-u|-g uname|uid|gname|gid] /mount_point -   + + quota [-q] [-v] [-o obd_uuid|-i mdt_idx|-I ost_idx] [-u|-g uname|uid|gname|gid] /mount_point + +   - Displays disk usage and limits, either for the full file system or for objects on a specific OBD. A user or group name or an ID can be specified. If both user and group are omitted, quotas for the current UID/GID are shown. The -q option disables printing of additional descriptions (including column titles). It fills in blank spaces in the ''grace'' column with zeros (when there is no grace period set), to ensure that the number of columns is consistent. The -v option provides more verbose (per-OBD statistics) output. + Displays disk usage and limits, either for the full file + system or for objects on a specific OBD. A user or group name or an ID can be + specified. If both user and group are omitted, quotas for the current UID/GID + are shown. The -q option disables printing of additional + descriptions (including column titles). It fills in blank spaces in the + grace column with zeros (when there is no grace period set), + to ensure that the number of columns is consistent. The -v + option provides more verbose (per-OBD statistics) output. - quota -t -u|-g /mount_point + + quota -t -u|-g /mount_point + Displays block and inode grace times for user (-u) or group (-g) quotas. @@ -540,15 +632,19 @@ lfs help - quotachown + + quotachown + - Changes the file's owner and group on OSTs of the specified file system. + Changes the file's owner and group on OSTs of the specified file system. - quotacheck [-ugf] /mount_point + + quotacheck [-ugf] /mount_point + Scans the specified file system for disk usage, and creates or updates quota files. Options specify quota for users (-u), groups (-g), and force (-f). @@ -556,7 +652,9 @@ lfs help - quotaon [-ugf] /mount_point + + quotaon [-ugf] /mount_point + Turns on file system quotas. Options specify quota for users (-u), groups (-g), and force (-f). @@ -564,7 +662,9 @@ lfs help - quotaoff [-ugf] /mount_point + + quotaoff [-ugf] /mount_point + Turns off file system quotas. Options specify quota for users (-u), groups (-g), and force (-f). @@ -572,7 +672,9 @@ lfs help - quotainv [-ug] [-f] /mount_point + + quotainv [-ug] [-f] /mount_point + Clears quota files (administrative quota files if used without -f, operational quota files otherwise), all of their quota entries for users (-u) or groups (-g). After running quotainv, you must run quotacheck before using quotas. @@ -583,26 +685,30 @@ lfs help - setquota -u|-g uname|uid|gname|gid} [--block-softlimit block_softlimit] [--block-hardlimit block_hardlimit] [--inode-softlimit inode_softlimit] [--inode-hardlimit inode_hardlimit] /mount_point + + setquota -u|-g uname|uid|gname|gid} [--block-softlimit block_softlimit] [--block-hardlimit block_hardlimit] [--inode-softlimit inode_softlimit] [--inode-hardlimit inode_hardlimit] /mount_point + - Sets file system quotas for users or groups. Limits can be specified with --{block|inode}-{softlimit|hardlimit} or their short equivalents -b, -B, -i, -I. Users can set 1, 2, 3 or 4 limits. - The old setquota interface is supported, but it may be - removed in a future Lustre software release. - Also, limits can be specified with special suffixes, -b, -k, -m, -g, -t, and -p to indicate units of 1, 2^10, 2^20, 2^30, 2^40 and 2^50, respectively. By default, the block limits unit is 1 kilobyte (1,024), and block limits are always kilobyte-grained (even if specified in bytes). See . + Sets file system quotas for users or groups. Limits can be specified with --{block|inode}-{softlimit|hardlimit} or their short equivalents -b, -B, -i, -I. Users can set 1, 2, 3 or 4 limits.The old setquota interface is supported, but it may be + removed in a future Lustre software release.Also, limits can be specified with special suffixes, -b, -k, -m, -g, -t, and -p to indicate units of 1, 2^10, 2^20, 2^30, 2^40 and 2^50, respectively. By default, the block limits unit is 1 kilobyte (1,024), and block limits are always kilobyte-grained (even if specified in bytes). See . - setquota -t -u|-g [--block-grace block_grace] [--inode-grace inode_grace] /mount_point + + setquota -t -u|-g [--block-grace block_grace] [--inode-grace inode_grace] /mount_point + - Sets the file system quota grace times for users or groups. Grace time is specified in 'XXwXXdXXhXXmXXs' format or as an integer seconds value. See . + Sets the file system quota grace times for users or groups. Grace time is specified in 'XXwXXdXXhXXmXXs' format or as an integer seconds value. See . - help + + help + Provides brief help on various lfs arguments. @@ -610,7 +716,9 @@ lfs help - exit/quit + + exit/quit + Quits the interactive lfs session. @@ -644,7 +752,7 @@ lfs help $ lfs df -i List space or inode usage for a specific OST pool. $ lfs df --pool filesystem[.pool] | pathname - List quotas of user 'bob'. + List quotas of user 'bob'. $ lfs quota -u bob /mnt/lustre Show grace times for user quotas on /mnt/lustre. $ lfs quota -t -u /mnt/lustre @@ -656,7 +764,7 @@ lfs help $ lfs quotaon -ug /mnt/lustre Turns off quotas of user and group. $ lfs quotaoff -ug /mnt/lustre - Sets quotas of user 'bob', with a 1 GB block quota hardlimit and a 2 GB block quota softlimit. + Sets quotas of user 'bob', with a 1 GB block quota hardlimit and a 2 GB block quota softlimit. $ lfs setquota -u bob --block-softlimit 2000000 --block-hardlimit 1000000 /mnt/lustre Sets grace times for user quotas: 1000 seconds for block quotas, 1 week and 4 days for inode quotas. $ lfs setquota -t -u --block-grace 1000 --inode-grace 1w4d /mnt/lustre @@ -671,19 +779,24 @@ lfs help Finds all directories/files associated with poolA. $ lfs find /mnt/lustre --pool poolA Finds all directories/files not associated with a pool. - $ lfs find /mnt//lustre --pool "" + $ lfs find /mnt//lustre --pool "" Finds all directories/files associated with pool. - $ lfs find /mnt/lustre ! --pool "" + $ lfs find /mnt/lustre ! --pool "" Associates a directory with the pool my_pool, so all new files and directories are created in the pool. $ lfs setstripe --pool my_pool /mnt/lustre/dir
See Also - + + +
- <indexterm><primary>lfs_migrate</primary></indexterm> + <title> + <indexterm> + <primary>lfs_migrate</primary> + </indexterm> <literal>lfs_migrate</literal> The lfs_migrate utility is a simple tool to migrate files between Lustre OSTs. @@ -693,10 +806,55 @@ lfs help
Description - The lfs_migrate utility is a simple tool to assist migration of files between Lustre OSTs. The utility copies each specified file to a new file, verifies the file contents have not changed, and then renames the new file to the original filename. This allows balanced space usage between OSTs, moving files of OSTs that are starting to show hardware problems (though are still functional) or OSTs that will be discontinued. - Because lfs_migrate is not closely integrated with the MDS, it cannot determine whether a file is currently open and/or in-use by other applications or nodes. This makes it UNSAFE for use on files that might be modified by other applications, since the migrated file is only a copy of the current file. This results in the old file becoming an open-unlinked file and any modifications to that file are lost. - Files to be migrated can be specified as command-line arguments. If a directory is specified on the command-line then all files within the directory are migrated. If no files are specified on the command-line, then a list of files is read from the standard input, making lfs_migrate suitable for use with lfs find to locate files on specific OSTs and/or matching other file attributes. - The current file allocation policies on the MDS dictate where the new files are placed, taking into account whether specific OSTs have been disabled on the MDS via lctl (preventing new files from being allocated there), whether some OSTs are overly full (reducing the number of files placed on those OSTs), or if there is a specific default file striping for the target directory (potentially changing the stripe count, stripe size, OST pool, or OST index of a new file). + The lfs_migrate utility is a simple tool to + assist migration of files between Lustre OSTs. The utility copies each + specified file to a new file, verifies the file contents have not changed, and + then renames the new file to the original filename. This allows balanced space + usage between OSTs, moving files off OSTs that are starting to show hardware + problems (though are still functional) or OSTs that will be + discontinued. + + For versions of Lustre before 2.5, lfs_migrate is not + closely integrated with the MDS, it cannot determine whether a file is + currently open and/or in-use by other applications or nodes. This makes it + UNSAFE for use on files that might be modified by other applications, since the + migrated file is only a copy of the current file. This results in the old file + becoming an open-unlinked file and any modifications to that file are + lost. + + Files to be migrated can be specified as command-line arguments. If a + directory is specified on the command-line then all files within the directory + are migrated. If no files are specified on the command-line, then a list of + files is read from the standard input, making lfs_migrate + suitable for use with lfs find to locate files on specific + OSTs and/or matching other file attributes. + The current file allocation policies on the MDS dictate where the new + files are placed, taking into account whether specific OSTs have been disabled + on the MDS via lctl (preventing new files from being + allocated there), whether some OSTs are overly full (reducing the number of + files placed on those OSTs), or if there is a specific default file striping + for the target directory (potentially changing the stripe count, stripe size, + OST pool, or OST index of a new file). + + The lfs_migrate utility can also be used in some cases + to reduce file + fragmentationfragmentation. File + fragmentation will typically reduce Lustre file system performance. File + fragmentation may be observed on an aged file system and will commonly occur if + the file was written by many threads. Provided there is sufficient free space + (or if it was written when the file system was nearly full) + that is less fragmented than the file being copied, re-writing a file with + lfs_migrate will result in a migrated file with reduced + fragmentation. The tool filefrag can be used to report file + fragmentation. See + + + As long as a file has extent lengths of tens of megabytes + (read_bandwidth * seek_time) or more, the read + performance for the file will not be significantly impacted by fragmentation, + since the read pipeline can be filled by large reads from disk even with an + occasional disk seek. +
Options @@ -708,10 +866,14 @@ lfs help - Option + + Option + - Description + + Description + @@ -719,7 +881,8 @@ lfs help - -c stripecount + -c stripecount + Restripe file using the specified stripe count. This option may not be @@ -729,7 +892,8 @@ lfs help - -h + -h + Display help information. @@ -755,31 +919,37 @@ lfs help - -q + + -q + Run quietly (does not print filenames or status). - -R + + -R + Restripe file using default directory striping instead of keeping striping. This option may not be specified at the same time as the -c option. - -s + -s + Skip file data comparison after migrate. Default is to compare migrated file against original to verify correctness. - -y + -y + - Answer 'y' to usage warning without prompting + Answer 'y' to usage warning without prompting (for scripts, use with caution). @@ -796,11 +966,16 @@ lfs help
See Also - + + +
- <indexterm><primary>filefrag</primary></indexterm> + <title> + <indexterm> + <primary>filefrag</primary> + </indexterm> <literal>filefrag</literal> The e2fsprogs package contains the filefrag tool which reports the extent of file fragmentation. @@ -833,6 +1008,11 @@ lfs help can fully utilize the disk disk bandwidth even with occasional seeks. + In default mode The default mode is faster than the + verbose/extent mode., filefrag returns the + number of physically discontiguous extents in the file. In extent or verbose + mode, each extent is printed with details. For a Lustre file system, the + extents are printed in device offset order, not logical offset order.
Options @@ -844,17 +1024,23 @@ lfs help - Option + + Option + - Description + + Description + - -b + + -b + Uses the 1024-byte blocksize for the output. By default, this blocksize is @@ -863,7 +1049,9 @@ lfs help - -e + + -e + Uses the extent mode when printing the output. This is @@ -872,7 +1060,9 @@ lfs help - -l + + -l + Displays extents in LUN offset order. This is the only @@ -881,7 +1071,9 @@ lfs help - -s + + -s + Synchronizes any unwritten file data to disk before @@ -890,7 +1082,9 @@ lfs help - -v + + -v + Prints the file's layout in verbose mode when checking @@ -929,7 +1123,10 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes)
- <indexterm><primary>mount</primary></indexterm> + <title> + <indexterm> + <primary>mount</primary> + </indexterm> <literal>mount</literal> The standard mount(8) Linux command is used to mount a Lustre file @@ -943,17 +1140,23 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Server options + + Server options + - Description + + Description + - abort_recov + + abort_recov + Aborts recovery when starting a target @@ -961,7 +1164,9 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - nosvc + + nosvc + Starts only MGS/MGC servers @@ -969,7 +1174,9 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - nomgs + + nomgs + Start a MDT with a co-located MGS without starting the MGS @@ -977,7 +1184,9 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - exclude + + exclude + Starts with a dead OST @@ -985,7 +1194,9 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - md_stripe_cache_size + + md_stripe_cache_size + Sets the stripe cache size for server side disk with a striped raid configuration @@ -1001,17 +1212,23 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Client options + + Client options + - Description + + Description + - flock/noflock/localflock + + flock/noflock/localflock + Enables/disables global flock or local flock support @@ -1019,7 +1236,9 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - user_xattr/nouser_xattr + + user_xattr/nouser_xattr + Enables/disables user-extended attributes @@ -1027,15 +1246,19 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - user_fid2path/nouser_fid2path + + user_fid2path/nouser_fid2path + - Enables/disables FID to path translation by regular users + Enables/disables FID to path translation by regular users - retry= + + retry= + Number of times a client will retry to mount the file system