From: Richard Henwood Date: Wed, 9 Dec 2015 18:47:13 +0000 (-0600) Subject: LUDOC-263 wrap: wrap lines for review convenience. X-Git-Tag: 2.9.0~34 X-Git-Url: https://git.whamcloud.com/?p=doc%2Fmanual.git;a=commitdiff_plain;h=93cda492417285364795cb14506a4cc6259c6700 LUDOC-263 wrap: wrap lines for review convenience. As subject. Change-Id: I1f92f65bdd1bb45f80f9c621de1b131be60ce5f1 Signed-off-by: Richard Henwood Reviewed-on: http://review.whamcloud.com/17530 Tested-by: Jenkins --- diff --git a/UserUtilities.xml b/UserUtilities.xml index 308393c..b05e054 100644 --- a/UserUtilities.xml +++ b/UserUtilities.xml @@ -1,34 +1,9 @@ - - + + User Utilities - This chapter describes user utilities and includes the following sections: - - - - - - - - - - - - - - - - - - - - - - - - - - - + This chapter describes user utilities.
<indexterm> @@ -36,10 +11,13 @@ </indexterm> <literal>lfs</literal> - The lfs utility can be used for user configuration routines and monitoring. + The + lfs utility can be used for user configuration routines + and monitoring.
Synopsis - lfs + +lfs lfs changelog [--follow] mdt_name [startrec [endrec]] lfs changelog_clear mdt_name id endrec lfs check mds|osts|servers @@ -55,17 +33,17 @@ lfs getname [-h]|[path...] lfs getstripe [--obd|-O ost_name] [--quiet|-q] [--verbose|-v] [--count|-c] [--index|-i | --offset|-o] [--size|-s] [--pool|-p] [--directory|-d] - [--recursive|-r] [--raw|-R] [-M] dirname|filename ... + [--recursive|-r] [--raw|-R] [-M] + dirname|filename ... lfs setstripe [--size|-s stripe_size] [--count|-c stripe_count] [--index|-i|--offset|-o start_ost_index] [--pool|-p pool] dirname|filename lfs setstripe -d dir lfs osts [path] -lfs poollist filesystem[.pool]|pathname +lfs poollist filesystem[.pool]| pathname lfs quota [-q] [-v] [-h] [-o obd_uuid|-I ost_idx|-i mdt_idx] - [-u username|uid|-g group|gid] - /mount_point + [-u username|uid|-g group|gid] /mount_point lfs quota -t -u|-g /mount_point lfs quotacheck [-ug] /mount_point lfs quotachown [-i] /mount_point @@ -82,34 +60,53 @@ lfs setquota -u|--user|-g|--group uname|uid|gname|gid [-b block_softlimit] [-B block_hardlimit] [-i inode-softlimit] [-I inode_hardlimit] /mount_point -lfs setquota -t -u|-g - [--block-grace block_grace] +lfs setquota -t -u|-g [--block-grace block_grace] [--inode-grace inode_grace] /mount_point -lfs setquota -t -u|-g - [-b block_grace] [-i inode_grace] +lfs setquota -t -u|-g [-b block_grace] [-i inode_grace] /mount_point lfs help - In the above example, the /mount_point parameter refers to the mount point of the Lustre file system. + In the above example, the + + /mount_point + parameter refers to the mount point of the Lustre file + system. - The old lfs quota output was very detailed and contained cluster-wide quota statistics (including cluster-wide limits for a user/group and cluster-wide usage for a user/group), as well as statistics for each MDS/OST. Now, lfs quota has been updated to provide only cluster-wide statistics, by default. To obtain the full report of cluster-wide limits, usage and statistics, use the -v option with lfs quota. + The old lfs quota output was very detailed and contained + cluster-wide quota statistics (including cluster-wide limits for a + user/group and cluster-wide usage for a user/group), as well as + statistics for each MDS/OST. Now, + lfs quota has been updated to provide only + cluster-wide statistics, by default. To obtain the full report of + cluster-wide limits, usage and statistics, use the + -v option with + lfs quota.
Description - The lfs utility is used to create a new file with a specific striping pattern, determine the default striping pattern, gather the extended attributes (object numbers and location) for a specific file, find files with specific attributes, list OST information or set quota limits. It can be invoked interactively without any arguments or in a non-interactive mode with one of the supported arguments. + The + lfs utility is used to create a new file with a + specific striping pattern, determine the default striping pattern, gather + the extended attributes (object numbers and location) for a specific + file, find files with specific attributes, list OST information or set + quota limits. It can be invoked interactively without any arguments or in + a non-interactive mode with one of the supported arguments.
Options - The various lfs options are listed and described below. For a complete list of available options, type help at the lfs prompt. + The various + lfs options are listed and described below. For a + complete list of available options, type help at the + lfs prompt. - - - + + + @@ -132,7 +129,10 @@ lfs help - Shows the metadata changes on an MDT. Start and end points are optional. The --follow option blocks on new changes; this option is only valid when run directly on the MDT node. + Shows the metadata changes on an MDT. Start and end + points are optional. The + --follow option blocks on new changes; this + option is only valid when run directly on the MDT node. @@ -142,75 +142,113 @@ lfs help - Indicates that changelog records previous to endrec are no longer of interest to a particular consumer id, potentially allowing the MDT to free up disk space. An endrec of 0 indicates the current last record. Changelog consumers must be registered on the MDT node using lctl. + Indicates that changelog records previous to + + endrec + are no longer of interest to a particular consumer + + id + , potentially allowing the MDT to free up disk space. + An + + endrec + of 0 indicates the current last record. Changelog + consumers must be registered on the MDT node using + lctl. - check + check - Displays the status of MDS or OSTs (as specified in the command) or all servers (MDS and OSTs). + Displays the status of MDS or OSTs (as specified in the + command) or all servers (MDS and OSTs). - df [-i] [-h] [--pool|-p - fsname[.pool] - [path] [--lazy] - - - Use -i to report file system disk space usage or inode - usage of each MDT or OST or, if a pool is specified with the -p - option, a subset of OSTs. - By default, the usage of all mounted Lustre file systems is reported. If the - path option is included, only the usage for the specified - file system is reported. If the -h option is included, the - output is printed in human-readable format, using SI base-2 suffixes for - Mega-, - Giga-, - Tera-, - Peta-, or - Exabytes. - If the --lazy option is specified, any OST that is - currently disconnected from the client will be skipped. Using the - --lazy option prevents the df output from - being blocked when an OST is offline. Only the space on the OSTs that can - currently be accessed are returned. The llite.*.lazystatfs - tunable can be enabled to make this the default behaviour for all - statfs() operations. + df [-i] [-h] [--pool|-p + fsname[. + pool] [ + path] [--lazy] + + + Use + -i to report file system disk space usage or + inode usage of each MDT or OST or, if a pool is specified with + the + -p option, a subset of OSTs. + By default, the usage of all mounted Lustre file systems + is reported. If the + path option is included, only the usage for + the specified file system is reported. If the + -h option is included, the output is printed + in human-readable format, using SI base-2 suffixes for + Mega-, + Giga-, + Tera-, + Peta-, or + Exabytes. + If the + --lazy option is specified, any OST that is + currently disconnected from the client will be skipped. Using + the + --lazy option prevents the + df output from being blocked when an OST is + offline. Only the space on the OSTs that can currently be + accessed are returned. The + llite.*.lazystatfs tunable can be enabled to + make this the default behaviour for all + statfs() operations. - find + find - Searches the directory tree rooted at the given directory/filename for files that match the given parameters. - Using ! before an option negates its meaning (files NOT matching the parameter). Using + before a numeric value means files with the parameter OR MORE. Using - before a numeric value means files with the parameter OR LESS. + Searches the directory tree rooted at the given + directory/filename for files that match the given + parameters. + Using + ! before an option negates its meaning (files + NOT matching the parameter). Using + + before a numeric value means files with the + parameter OR MORE. Using + - before a numeric value means files with the + parameter OR LESS. - + - --atime + --atime - File was last accessed N*24 hours ago. (There is no guarantee that atime is kept coherent across the cluster.) - OSTs store a transient atime that is updated when clients - do read requests. Permanent atime is written to the MDS when - the file is closed. However, on-disk atime is only updated if it is more than 60 - seconds old (/proc/fs/lustre/mds/*/max_atime_diff). The Lustre - software considers the latest atime from all OSTs. If a - setattr is set by user, then it is updated on both the MDS - and OST, allowing the atime to go backward. + File was last accessed N*24 hours ago. (There is no + guarantee that + atime is kept coherent across the + cluster.) + OSTs store a transient + atime that is updated when clients do read + requests. Permanent + atime is written to the MDS when the file is + closed. However, on-disk atime is only updated if it is more + than 60 seconds old ( + /proc/fs/lustre/mds/*/max_atime_diff). The + Lustre software considers the latest + atime from all OSTs. If a + setattr is set by user, then it is updated on + both the MDS and OST, allowing the + atime to go backward. -   +   @@ -223,7 +261,7 @@ lfs help -   +   @@ -236,7 +274,7 @@ lfs help -   +   @@ -249,7 +287,7 @@ lfs help -   +   @@ -257,12 +295,13 @@ lfs help - File has a size in bytes, or kilo-, Mega-, Giga-, Tera-, Peta- or Exabytes if a suffix is given. + File has a size in bytes, or kilo-, Mega-, Giga-, Tera-, + Peta- or Exabytes if a suffix is given. -   +   @@ -270,13 +309,14 @@ lfs help - File has the type - block, character, directory, pipe, file, symlink, socket - or door (used in Solaris operating system). + File has the type - block, character, directory, pipe, + file, symlink, socket or door (used in Solaris operating + system). -   +   @@ -289,7 +329,7 @@ lfs help -   +   @@ -297,12 +337,13 @@ lfs help - File owned by a specific user (numeric user ID allowed). + File owned by a specific user (numeric user ID + allowed). -   +   @@ -315,7 +356,7 @@ lfs help -   +   @@ -323,29 +364,35 @@ lfs help - File belongs to a specific group (numeric group ID allowed). + File belongs to a specific group (numeric group ID + allowed). -   +   - --maxdepth + - + -maxdepth - Limits find to descend at most N levels of the directory tree. + Limits find to descend at most N levels of the directory + tree. -   +   - --print / --print0 + + --print/ + --print0 - Prints the full filename, followed by a new line or NULL character correspondingly. + Prints the full filename, followed by a new line or NULL + character correspondingly. @@ -355,9 +402,9 @@ lfs help - Lists all OSTs for the file system. If a path located on a mounted Lustre file - system is specified, then only OSTs belonging to this file system are - displayed. + Lists all OSTs for the file system. If a path located on + a mounted Lustre file system is specified, then only OSTs + belonging to this file system are displayed. @@ -367,10 +414,11 @@ lfs help - List each Lustre file system instance associated with each Lustre mount - point. If no path is specified, all Lustre mount points are interrogated. If a - list of paths is provided, the instance of each path is provided. If the path is - not a Lustre instance 'No such device' is returned. + List each Lustre file system instance associated with + each Lustre mount point. If no path is specified, all Lustre + mount points are interrogated. If a list of paths is provided, + the instance of each path is provided. If the path is not a + Lustre instance 'No such device' is returned. @@ -380,22 +428,36 @@ lfs help - Lists striping information for a given filename or directory. By default, the stripe count, stripe size and offset are returned. - If you only want specific striping information, then the options of --count,--size,--index or --offset plus various combinations of these options can be used to retrieve specific information. - If the --raw option is specified, the stripe information is - printed without substituting the file system default values for unspecified - fields. If the striping EA is not set, 0, 0, and -1 will be printed for the stripe - count, size, and offset respectively. - The -M prints the index of the MDT for a given directory. See . + Lists striping information for a given filename or + directory. By default, the stripe count, stripe size and offset + are returned. + If you only want specific striping information, then the + options of + --count, + --size, + --index or + --offset plus various combinations of these + options can be used to retrieve specific information. + If the + --raw option is specified, the stripe + information is printed without substituting the file system + default values for unspecified fields. If the striping EA is + not set, 0, 0, and -1 will be printed for the stripe count, + size, and offset respectively. + The + -M prints the index of the MDT for a given + directory. See + . -   +   - --obd ost_name + --obd + ost_name @@ -404,7 +466,7 @@ lfs help -   +   @@ -412,12 +474,13 @@ lfs help - Lists details about the file's object ID information. + Lists details about the file's object ID + information. -   +   @@ -430,7 +493,7 @@ lfs help -   +   @@ -443,7 +506,7 @@ lfs help -   +   @@ -456,7 +519,7 @@ lfs help -   +   @@ -469,7 +532,7 @@ lfs help -   +   @@ -482,7 +545,7 @@ lfs help -   +   @@ -490,12 +553,13 @@ lfs help - Lists the stripe size (how much data to write to one OST before moving to the next OST). + Lists the stripe size (how much data to write to one OST + before moving to the next OST). -   +   @@ -503,12 +567,14 @@ lfs help - Lists entries about a specified directory instead of its contents (in the same manner as ls -d). + Lists entries about a specified directory instead of its + contents (in the same manner as + ls -d). -   +   @@ -526,15 +592,19 @@ lfs help - Create new files with a specific file layout (stripe - pattern) configuration.The file cannot exist prior to using - setstripe. A directory must exist prior to using - setstripe. + Create new files with a specific file layout (stripe + pattern) configuration. + + The file cannot exist prior to using + setstripe. A directory must exist prior to + using + setstripe. + -   +   @@ -542,23 +612,38 @@ lfs help - Number of OSTs over which to stripe a file. A stripe_cnt of 0 uses the file system-wide default stripe count (default is 1). A stripe_cnt of -1 stripes over all available OSTs. + Number of OSTs over which to stripe a file. A + stripe_cnt of 0 uses the file system-wide + default stripe count (default is 1). A + stripe_cnt of -1 stripes over all available + OSTs. -   +   - --size stripe_sizeThe default stripe-size is 0. The default start-ost is -1. Do NOT confuse them! If you set start-ost to 0, all new file creations occur on OST 0 (seldom a good idea).  + + --size stripe_size + + The default stripe-size is 0. The default start-ost is + -1. Do NOT confuse them! If you set start-ost to 0, all new + file creations occur on OST 0 (seldom a good idea). +   - Number of bytes to store on an OST before moving to the next OST. A stripe_size of 0 uses the file system's default stripe size, (default is 1 MB). Can be specified with k (KB), m (MB), or g (GB), respectively. + Number of bytes to store on an OST before moving to the + next OST. A stripe_size of 0 uses the file system's default + stripe size, (default is 1 MB). Can be specified with + k(KB), + m(MB), or + g(GB), respectively. -   +   @@ -566,20 +651,37 @@ lfs help - The OST index (base 10, starting at 0) on which to start striping for this file. A start_ost_index value of -1 allows the MDS to choose the starting index. This is the default value, and it means that the MDS selects the starting OST as it wants. We strongly recommend selecting this default, as it allows space and load balancing to be done by the MDS as needed. The start_ost_index value has no relevance on whether the MDS will use round-robin or QoS weighted allocation for the remaining stripes in the file. + The OST index (base 10, starting at 0) on which to start + striping for this file. A start_ost_index value of -1 allows + the MDS to choose the starting index. This is the default + value, and it means that the MDS selects the starting OST as it + wants. We strongly recommend selecting this default, as it + allows space and load balancing to be done by the MDS as + needed. The + start_ost_index value has no relevance on + whether the MDS will use round-robin or QoS weighted allocation + for the remaining stripes in the file. -   +   - --pool pool + --pool + pool - Name of the pre-defined pool of OSTs (see ) that will be used for striping. The stripe_cnt, stripe_size and start_ost values are used as well. The start-ost value must be part of the pool or an error is returned. + Name of the pre-defined pool of OSTs (see + ) that will be used + for striping. The + stripe_cnt, + stripe_size and + start_ost values are used as well. The + start-ost value must be part of the pool or an error is + returned. @@ -589,45 +691,61 @@ lfs help - Deletes default striping on the specified directory. + Deletes default striping on the specified + directory. - poollist {filesystem} [.poolname]|{pathname} + poollist {filesystem} + [.poolname]|{pathname} - Lists pools in the file system or pathname, or OSTs in the file system's pool. + Lists pools in the file system or pathname, or OSTs in + the file system's pool. - quota [-q] [-v] [-o obd_uuid|-i mdt_idx|-I ost_idx] [-u|-g uname|uid|gname|gid] /mount_point + quota [-q] [-v] [-o + obd_uuid|-i + mdt_idx|-I + ost_idx] [-u|-g + uname|uid|gname|gid] + /mount_point -   +   - Displays disk usage and limits, either for the full file - system or for objects on a specific OBD. A user or group name or an ID can be - specified. If both user and group are omitted, quotas for the current UID/GID - are shown. The -q option disables printing of additional - descriptions (including column titles). It fills in blank spaces in the - grace column with zeros (when there is no grace period set), - to ensure that the number of columns is consistent. The -v - option provides more verbose (per-OBD statistics) output. + Displays disk usage and limits, either for the full file + system or for objects on a specific OBD. A user or group name + or an ID can be specified. If both user and group are omitted, + quotas for the current UID/GID are shown. The + -q option disables printing of additional + descriptions (including column titles). It fills in blank + spaces in the + grace column with zeros (when there is no + grace period set), to ensure that the number of columns is + consistent. The + -v option provides more verbose (per-OBD + statistics) output. - quota -t -u|-g /mount_point + quota -t + -u|-g + /mount_point - Displays block and inode grace times for user (-u) or group (-g) quotas. + Displays block and inode grace times for user ( + -u) or group ( + -g) quotas. @@ -637,71 +755,130 @@ lfs help - Changes the file's owner and group on OSTs of the specified file system. + Changes the file's owner and group on OSTs of the + specified file system. - quotacheck [-ugf] /mount_point + quotacheck [-ugf] + /mount_point - Scans the specified file system for disk usage, and creates or updates quota files. Options specify quota for users (-u), groups (-g), and force (-f). + Scans the specified file system for disk usage, and + creates or updates quota files. Options specify quota for users + ( + -u), groups ( + -g), and force ( + -f). - quotaon [-ugf] /mount_point + quotaon [-ugf] + /mount_point - Turns on file system quotas. Options specify quota for users (-u), groups (-g), and force (-f). + Turns on file system quotas. Options specify quota for + users ( + -u), groups ( + -g), and force ( + -f). - quotaoff [-ugf] /mount_point + quotaoff [-ugf] + /mount_point - Turns off file system quotas. Options specify quota for users (-u), groups (-g), and force (-f). + Turns off file system quotas. Options specify quota for + users ( + -u), groups ( + -g), and force ( + -f). - quotainv [-ug] [-f] /mount_point + quotainv [-ug] [-f] + /mount_point - Clears quota files (administrative quota files if used without -f, operational quota files otherwise), all of their quota entries for users (-u) or groups (-g). After running quotainv, you must run quotacheck before using quotas. + Clears quota files (administrative quota files if used + without + -f, operational quota files otherwise), all + of their quota entries for users ( + -u) or groups ( + -g). After running + quotainv, you must run + quotacheck before using quotas. - Use extreme caution when using this command; its results cannot be undone. + Use extreme caution when using this command; its + results cannot be undone. - setquota -u|-g uname|uid|gname|gid} [--block-softlimit block_softlimit] [--block-hardlimit block_hardlimit] [--inode-softlimit inode_softlimit] [--inode-hardlimit inode_hardlimit] /mount_point + setquota -u|-g + + uname|uid|gname|gid}[--block-softlimit + block_softlimit] + [--block-hardlimit + block_hardlimit] + [--inode-softlimit + inode_softlimit] + [--inode-hardlimit + inode_hardlimit] + /mount_point - Sets file system quotas for users or groups. Limits can be specified with --{block|inode}-{softlimit|hardlimit} or their short equivalents -b, -B, -i, -I. Users can set 1, 2, 3 or 4 limits.The old setquota interface is supported, but it may be - removed in a future Lustre software release.Also, limits can be specified with special suffixes, -b, -k, -m, -g, -t, and -p to indicate units of 1, 2^10, 2^20, 2^30, 2^40 and 2^50, respectively. By default, the block limits unit is 1 kilobyte (1,024), and block limits are always kilobyte-grained (even if specified in bytes). See . + Sets file system quotas for users or groups. Limits can + be specified with + --{block|inode}-{softlimit|hardlimit} or + their short equivalents + -b, + -B, + -i, + -I. Users can set 1, 2, 3 or 4 limits. + + The old + setquota interface is supported, but it may + be removed in a future Lustre software release. + Also, limits can be specified with special suffixes, + -b, -k, -m, -g, -t, and -p to indicate units of 1, 2^10, 2^20, + 2^30, 2^40 and 2^50, respectively. By default, the block limits + unit is 1 kilobyte (1,024), and block limits are always + kilobyte-grained (even if specified in bytes). See + . - setquota -t -u|-g [--block-grace block_grace] [--inode-grace inode_grace] /mount_point + setquota -t -u|-g [--block-grace + block_grace] [--inode-grace + inode_grace] + /mount_point - Sets the file system quota grace times for users or groups. Grace time is specified in 'XXwXXdXXhXXmXXs' format or as an integer seconds value. See . + Sets the file system quota grace times for users or + groups. Grace time is specified in ' + XXwXXdXXhXXmXXs' format or as an integer + seconds value. See + . @@ -711,7 +888,8 @@ lfs help - Provides brief help on various lfs arguments. + Provides brief help on various + lfs arguments. @@ -721,7 +899,8 @@ lfs help - Quits the interactive lfs session. + Quits the interactive + lfs session. @@ -730,65 +909,141 @@ lfs help
Examples - Creates a file striped on two OSTs with 128 KB on each stripe. - $ lfs setstripe -s 128k -c 2 /mnt/lustre/file1 - Deletes a default stripe pattern on a given directory. New files use the default striping pattern. - $ lfs setstripe -d /mnt/lustre/dir + Creates a file striped on two OSTs with 128 KB on each + stripe. + +$ lfs setstripe -s 128k -c 2 /mnt/lustre/file1 + + Deletes a default stripe pattern on a given directory. New files + use the default striping pattern. + +$ lfs setstripe -d /mnt/lustre/dir + Lists the detailed object allocation of a given file. - $ lfs getstripe -v /mnt/lustre/file1 - List all the mounted Lustre file systems and corresponding Lustre instances. - $ lfs getname - Efficiently lists all files in a given directory and its subdirectories. - $ lfs find /mnt/lustre - Recursively lists all regular files in a given directory more than 30 days old. - $ lfs find /mnt/lustre -mtime +30 -type f -print - Recursively lists all files in a given directory that have objects on OST2-UUID. The lfs check servers command checks the status of all servers (MDT and OSTs). - $ lfs find --obd OST2-UUID /mnt/lustre/ + +$ lfs getstripe -v /mnt/lustre/file1 + + List all the mounted Lustre file systems and corresponding Lustre + instances. + +$ lfs getname + + Efficiently lists all files in a given directory and its + subdirectories. + +$ lfs find /mnt/lustre + + Recursively lists all regular files in a given directory more than + 30 days old. + +$ lfs find /mnt/lustre -mtime +30 -type f -print + + Recursively lists all files in a given directory that have objects + on OST2-UUID. The lfs check servers command checks the status of all + servers (MDT and OSTs). + +$ lfs find --obd OST2-UUID /mnt/lustre/ + Lists all OSTs in the file system. - $ lfs osts + +$ lfs osts + Lists space usage per OST and MDT in human-readable format. - $ lfs df -h + +$ lfs df -h + Lists inode usage per OST and MDT. - $ lfs df -i + +$ lfs df -i + List space or inode usage for a specific OST pool. - $ lfs df --pool filesystem[.pool] | pathname + +$ lfs df --pool +filesystem[. +pool] | +pathname + List quotas of user 'bob'. - $ lfs quota -u bob /mnt/lustre - Show grace times for user quotas on /mnt/lustre. - $ lfs quota -t -u /mnt/lustre + +$ lfs quota -u bob /mnt/lustre + + Show grace times for user quotas on + /mnt/lustre. + +$ lfs quota -t -u /mnt/lustre + Changes file owner and group. - $ lfs quotachown -i /mnt/lustre - Checks quotas for user and group. Turns on quotas after making the check. - $ lfs quotacheck -ug /mnt/lustre + +$ lfs quotachown -i /mnt/lustre + + Checks quotas for user and group. Turns on quotas after making the + check. + +$ lfs quotacheck -ug /mnt/lustre + Turns on quotas of user and group. - $ lfs quotaon -ug /mnt/lustre + +$ lfs quotaon -ug /mnt/lustre + Turns off quotas of user and group. - $ lfs quotaoff -ug /mnt/lustre - Sets quotas of user 'bob', with a 1 GB block quota hardlimit and a 2 GB block quota softlimit. - $ lfs setquota -u bob --block-softlimit 2000000 --block-hardlimit 1000000 /mnt/lustre - Sets grace times for user quotas: 1000 seconds for block quotas, 1 week and 4 days for inode quotas. - $ lfs setquota -t -u --block-grace 1000 --inode-grace 1w4d /mnt/lustre + +$ lfs quotaoff -ug /mnt/lustre + + Sets quotas of user 'bob', with a 1 GB block quota hardlimit and a + 2 GB block quota softlimit. + +$ lfs setquota -u bob --block-softlimit 2000000 --block-hardlimit 1000000 +/mnt/lustre + + Sets grace times for user quotas: 1000 seconds for block quotas, 1 + week and 4 days for inode quotas. + +$ lfs setquota -t -u --block-grace 1000 --inode-grace 1w4d /mnt/lustre + Checks the status of all servers (MDT, OST) - $ lfs check servers - Creates a file striped on two OSTs from the pool my_pool - $ lfs setstripe --pool my_pool -c 2 /mnt/lustre/file - Lists the pools defined for the mounted Lustre file system /mnt/lustre - $ lfs poollist /mnt/lustre/ - Lists the OSTs which are members of the pool my_pool in file system my_fs - $ lfs poollist my_fs.my_pool - Finds all directories/files associated with poolA. - $ lfs find /mnt/lustre --pool poolA + +$ lfs check servers + + Creates a file striped on two OSTs from the pool + my_pool + +$ lfs setstripe --pool my_pool -c 2 /mnt/lustre/file + + Lists the pools defined for the mounted Lustre file system + /mnt/lustre + +$ lfs poollist /mnt/lustre/ + + Lists the OSTs which are members of the pool + my_pool in file system + my_fs + +$ lfs poollist my_fs.my_pool + + Finds all directories/files associated with + poolA. + +$ lfs find /mnt/lustre --pool poolA + Finds all directories/files not associated with a pool. - $ lfs find /mnt//lustre --pool "" + +$ lfs find /mnt//lustre --pool "" + Finds all directories/files associated with pool. - $ lfs find /mnt/lustre ! --pool "" - Associates a directory with the pool my_pool, so all new files and directories are created in the pool. - $ lfs setstripe --pool my_pool /mnt/lustre/dir + +$ lfs find /mnt/lustre ! --pool "" + + Associates a directory with the pool + my_pool, so all new files and directories are created + in the pool. + +$ lfs setstripe --pool my_pool /mnt/lustre/dir +
See Also - +
@@ -799,70 +1054,85 @@ lfs help lfs_migrate - The lfs_migrate utility is a simple tool to migrate files between Lustre OSTs. + The + lfs_migrate utility is a simple tool to migrate files + between Lustre OSTs.
Synopsis - lfs_migrate [-c stripecount] [-h] [-l] [-n] [-q] [-R] [-s] [-y] [file|directory ...] + +lfs_migrate [-c stripecount] [-h] [-l] [-n] [-q] [-R] [-s] [-y] +[file|directory ...] +
Description - The lfs_migrate utility is a simple tool to - assist migration of files between Lustre OSTs. The utility copies each - specified file to a new file, verifies the file contents have not changed, and - then renames the new file to the original filename. This allows balanced space - usage between OSTs, moving files off OSTs that are starting to show hardware - problems (though are still functional) or OSTs that will be + The + lfs_migrate utility is a simple tool to assist + migration of files between Lustre OSTs. The utility copies each specified + file to a new file, verifies the file contents have not changed, and then + renames the new file to the original filename. This allows balanced space + usage between OSTs, moving files off OSTs that are starting to show + hardware problems (though are still functional) or OSTs that will be discontinued. - For versions of Lustre before 2.5, lfs_migrate is not - closely integrated with the MDS, it cannot determine whether a file is - currently open and/or in-use by other applications or nodes. This makes it - UNSAFE for use on files that might be modified by other applications, since the - migrated file is only a copy of the current file. This results in the old file - becoming an open-unlinked file and any modifications to that file are + For versions of Lustre before 2.5, + lfs_migrate is not closely integrated with the MDS, + it cannot determine whether a file is currently open and/or in-use by + other applications or nodes. This makes it UNSAFE for use on files that + might be modified by other applications, since the migrated file is + only a copy of the current file. This results in the old file becoming + an open-unlinked file and any modifications to that file are lost. - Files to be migrated can be specified as command-line arguments. If a - directory is specified on the command-line then all files within the directory - are migrated. If no files are specified on the command-line, then a list of - files is read from the standard input, making lfs_migrate - suitable for use with lfs find to locate files on specific - OSTs and/or matching other file attributes. - The current file allocation policies on the MDS dictate where the new - files are placed, taking into account whether specific OSTs have been disabled - on the MDS via lctl (preventing new files from being - allocated there), whether some OSTs are overly full (reducing the number of - files placed on those OSTs), or if there is a specific default file striping - for the target directory (potentially changing the stripe count, stripe size, - OST pool, or OST index of a new file). + Files to be migrated can be specified as command-line arguments. If + a directory is specified on the command-line then all files within the + directory are migrated. If no files are specified on the command-line, + then a list of files is read from the standard input, making + lfs_migrate suitable for use with + lfs find to locate files on specific OSTs and/or + matching other file attributes. + The current file allocation policies on the MDS dictate where the + new files are placed, taking into account whether specific OSTs have been + disabled on the MDS via + lctl(preventing new files from being allocated there), + whether some OSTs are overly full (reducing the number of files placed on + those OSTs), or if there is a specific default file striping for the + target directory (potentially changing the stripe count, stripe size, OST + pool, or OST index of a new file). - The lfs_migrate utility can also be used in some cases - to reduce file - fragmentationfragmentation. File - fragmentation will typically reduce Lustre file system performance. File - fragmentation may be observed on an aged file system and will commonly occur if - the file was written by many threads. Provided there is sufficient free space - (or if it was written when the file system was nearly full) - that is less fragmented than the file being copied, re-writing a file with - lfs_migrate will result in a migrated file with reduced - fragmentation. The tool filefrag can be used to report file - fragmentation. See + The + lfs_migrate utility can also be used in some cases to + reduce file + + fragmentation + fragmentation. File fragmentation will typically reduce + Lustre file system performance. File fragmentation may be observed on + an aged file system and will commonly occur if the file was written by + many threads. Provided there is sufficient free space (or if it was + written when the file system was nearly full) that is less fragmented + than the file being copied, re-writing a file with + lfs_migrate will result in a migrated file with + reduced fragmentation. The tool + filefrag can be used to report file fragmentation. + See + - As long as a file has extent lengths of tens of megabytes - (read_bandwidth * seek_time) or more, the read - performance for the file will not be significantly impacted by fragmentation, - since the read pipeline can be filled by large reads from disk even with an - occasional disk seek. + As long as a file has extent lengths of tens of megabytes ( + read_bandwidth * seek_time) or more, the + read performance for the file will not be significantly impacted by + fragmentation, since the read pipeline can be filled by large reads + from disk even with an occasional disk seek.
Options - Options supporting lfs_migrate are described below. + Options supporting + lfs_migrate are described below. - - + + @@ -881,12 +1151,14 @@ lfs help - -c stripecount + -c + stripecount - Restripe file using the specified stripe count. This option may not be - specified at the same time as the -R option. + Restripe file using the specified stripe count. This + option may not be specified at the same time as the + -R option. @@ -904,9 +1176,11 @@ lfs help -l - Migrate files with hard links (skips, by default). Files with multiple hard - links are split into multiple separate files by lfs_migrate, so - they are skipped, by default, to avoid breaking the hard links. + Migrate files with hard links (skips, by default). Files + with multiple hard links are split into multiple separate files + by + lfs_migrate, so they are skipped, by + default, to avoid breaking the hard links. @@ -931,16 +1205,18 @@ lfs help -R - Restripe file using default directory striping instead of keeping striping. - This option may not be specified at the same time as the -c - option. + Restripe file using default directory striping instead of + keeping striping. This option may not be specified at the same + time as the + -c option. -s - Skip file data comparison after migrate. Default is to compare migrated file - against original to verify correctness. + Skip file data comparison after migrate. Default is to + compare migrated file against original to verify + correctness. @@ -949,8 +1225,9 @@ lfs help - Answer 'y' to usage warning without prompting - (for scripts, use with caution). + Answer ' + y' to usage warning without prompting (for + scripts, use with caution). @@ -959,15 +1236,21 @@ lfs help
Examples - Rebalances all files in /mnt/lustre/dir. - $ lfs_migrate /mnt/lustre/file - Migrates files in /test filesystem on OST004 larger than 4 GB in size. - $ lfs find /test -obd test-OST004 -size +4G | lfs_migrate -y + Rebalances all files in + /mnt/lustre/dir. + +$ lfs_migrate /mnt/lustre/file + + Migrates files in /test filesystem on OST004 larger than 4 GB in + size. + +$ lfs find /test -obd test-OST004 -size +4G | lfs_migrate -y +
See Also - +
@@ -978,49 +1261,64 @@ lfs help filefrag - The e2fsprogs package contains the filefrag tool which reports the extent of file fragmentation. + The + e2fsprogs package contains the + filefrag tool which reports the extent of file + fragmentation.
Synopsis - filefrag [ -belsv ] [ files... ] + +filefrag [ -belsv ] [ files... ] +
Description - The filefrag utility reports the extent of fragmentation in a given file. - The filefrag utility obtains the extent information - from Lustre files using the FIEMAP ioctl, which is - efficient and fast, even for very large files. - In default mode - The default mode is faster than the verbose/extent mode since - it only counts the number of extents. - , filefrag prints the number of physically - discontiguous extents in the file. In extent or verbose mode, each - extent is printed with details such as the blocks allocated on each OST. - For a Lustre file system, the extents are printed in device offset order - (i.e. all of the extents for one OST first, then the next OST, etc.), - not file logical offset order. If the file logical offset order was - used, the Lustre striping would make the output very verbose and - difficult to see if there was file fragmentation or not. + The + filefrag utility reports the extent of fragmentation in + a given file. The + filefrag utility obtains the extent information from + Lustre files using the + FIEMAP ioctl, which is efficient and fast, even for + very large files. + In default mode + + The default mode is faster than the verbose/extent mode since it + only counts the number of extents. + , + filefrag prints the number of physically discontiguous + extents in the file. In extent or verbose mode, each extent is printed + with details such as the blocks allocated on each OST. For a Lustre file + system, the extents are printed in device offset order (i.e. all of the + extents for one OST first, then the next OST, etc.), not file logical + offset order. If the file logical offset order was used, the Lustre + striping would make the output very verbose and difficult to see if there + was file fragmentation or not. Note that as long as a file has extent lengths of tens of - megabytes or more (i.e. read_bandwidth * seek_time > - extent_length), the read performance for the file will - not be significantly impacted by fragmentation, since file readahead - can fully utilize the disk disk bandwidth even with occasional seeks. - + megabytes or more (i.e. + read_bandwidth * seek_time > + extent_length), the read performance for the file will + not be significantly impacted by fragmentation, since file readahead + can fully utilize the disk disk bandwidth even with occasional + seeks. - In default mode The default mode is faster than the - verbose/extent mode., filefrag returns the - number of physically discontiguous extents in the file. In extent or verbose - mode, each extent is printed with details. For a Lustre file system, the - extents are printed in device offset order, not logical offset order. + In default mode + + The default mode is faster than the verbose/extent mode. + , + filefrag returns the number of physically discontiguous + extents in the file. In extent or verbose mode, each extent is printed + with details. For a Lustre file system, the extents are printed in device + offset order, not logical offset order.
Options - The options and descriptions for the filefrag utility are listed below. + The options and descriptions for the + filefrag utility are listed below. - - + + @@ -1043,8 +1341,9 @@ lfs help - Uses the 1024-byte blocksize for the output. By default, this blocksize is - used by the Lustre file system, since OSTs may use different block sizes. + Uses the 1024-byte blocksize for the output. By default, + this blocksize is used by the Lustre file system, since OSTs + may use different block sizes. @@ -1054,7 +1353,7 @@ lfs help - Uses the extent mode when printing the output. This is + Uses the extent mode when printing the output. This is the default for Lustre files in verbose mode. @@ -1065,7 +1364,7 @@ lfs help - Displays extents in LUN offset order. This is the only + Displays extents in LUN offset order. This is the only available mode for Lustre. @@ -1099,10 +1398,13 @@ lfs help
Examples Lists default output. - $ filefrag /mnt/lustre/foo -/mnt/lustre/foo: 13 extents found + +$ filefrag /mnt/lustre/foo +/mnt/lustre/foo: 13 extents found + Lists verbose output in extent format. - $ filefrag -v /mnt/lustre/foo + +$ filefrag -v /mnt/lustre/foo Filesystem type is: bd00bd0 File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: @@ -1119,7 +1421,8 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) 10: 1097728.. 1220607: 2805866496..2805989375: 122880: 0002: network 11: 1220608.. 1343487: 2805997568..2806120447: 122880: 0002: network 12: 1343488.. 1433599: 2806128640..2806218751: 90112: 0002: network -/mnt/lustre/foo: 13 extents found +/mnt/lustre/foo: 13 extents found +
@@ -1129,14 +1432,16 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) mount - The standard mount(8) Linux command is used to mount a Lustre file - system. When mounting a Lustre file system, mount(8) executes the - /sbin/mount.lustre command to complete the mount. The mount command - supports these options specific to a Lustre file system: + The standard + mount(8) Linux command is used to mount a Lustre file + system. When mounting a Lustre file system, mount(8) executes the + /sbin/mount.lustre command to complete the mount. The + mount command supports these options specific to a Lustre file + system: - - + + @@ -1179,7 +1484,8 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Start a MDT with a co-located MGS without starting the MGS + Start a MDT with a co-located MGS without starting the + MGS @@ -1199,7 +1505,8 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Sets the stripe cache size for server side disk with a striped raid configuration + Sets the stripe cache size for server side disk with a + striped raid configuration @@ -1207,8 +1514,8 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - - + + @@ -1251,7 +1558,8 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Enables/disables FID to path translation by regular users + Enables/disables FID to path translation by + regular users @@ -1261,7 +1569,8 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Number of times a client will retry to mount the file system + Number of times a client will retry to mount the file + system @@ -1270,13 +1579,23 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes)
Handling Timeouts - Timeouts are the most common cause of hung applications. After a timeout involving an MDS or failover OST, applications attempting to access the disconnected resource wait until the connection gets established. - When a client performs any remote operation, it gives the server a reasonable amount of time to respond. If a server does not reply either due to a down network, hung server, or any other reason, a timeout occurs which requires a recovery. - If a timeout occurs, a message (similar to this one), appears on the console of the client, and in /var/log/messages: - LustreError: 26597:(client.c:810:ptlrpc_expire_one_request()) @@@ timeout + Timeouts are the most common cause of hung applications. After a + timeout involving an MDS or failover OST, applications attempting to access + the disconnected resource wait until the connection gets + established. + When a client performs any remote operation, it gives the server a + reasonable amount of time to respond. If a server does not reply either due + to a down network, hung server, or any other reason, a timeout occurs which + requires a recovery. + If a timeout occurs, a message (similar to this one), appears on the + console of the client, and in + /var/log/messages: + +LustreError: 26597:(client.c:810:ptlrpc_expire_one_request()) @@@ timeout req@a2d45200 x5886/t0 o38->mds_svc_UUID@NID_mds_UUID:12 lens 168/64 ref 1 fl -RPC:/0/0 rc 0 +RPC:/0/0 rc 0 +