X-Git-Url: https://git.whamcloud.com/?p=fs%2Flustre-release.git;a=blobdiff_plain;f=lustre%2Fdoc%2Flfs-setdirstripe.1;h=c1be84f3db6bf09a135b1300e1e991f75f86e912;hp=fda8be38ef786440b222107029d368477e28da6c;hb=a205334da5fcb150ba42505f735c26206321af9b;hpb=8c914986bf415a535153f77e53e630707f8e1a1c

diff --git a/lustre/doc/lfs-setdirstripe.1 b/lustre/doc/lfs-setdirstripe.1
index fda8be3..c1be84f 100644
--- a/lustre/doc/lfs-setdirstripe.1
+++ b/lustre/doc/lfs-setdirstripe.1
@@ -2,13 +2,15 @@
 .SH NAME
 lfs setdirstripe, mkdir \- set striping pattern of a directory.
 .SH SYNOPSIS
-.B lfs setdirstripe [\fR-cdDHioTx\fR] \fIDIR\fR...
+.B lfs setdirstripe [\fR-cdDhHioTxX\fR] \fIDIR\fR...
 .br
 .SH DESCRIPTION
-Create a striped directory with specified striping pattern. This lfs utility
-is similar to
-.BR lfs setstripe
-, but is used to create striped directory.
+Create a striped directory with specified striping pattern. This
+.B lfs
+sub-command is similar to
+.BR "lfs setstripe" ,
+but is used to create a striped directory or set the default layout for
+subdirectories.
 Can also be used to create directory with a foreign (free format) striping pattern (see
 .BR --foreign
 and
@@ -23,15 +25,23 @@ Stripe the new directory over
 .I COUNT
 MDTs.
 .TP
+.BR \-h ", " \-\-help
+Print usage message.
+.TP
 .BR \-i ", " \-\-mdt\-index =\fISTART_MDT_INDEX\fR[,\fIMDT_INDEX ...]
 Use the MDT whose index is
 .I START_MDT_INDEX
 as the master/starting MDT for the directory. If multiple
 .I MDT_INDEX
 values are given, then the stripes will be allocated on the specified
-MDT indices.  If index -1 (default) is used, it will randomly pick
-.I COUNT
-less full MDTs.
+MDT indices.  If
+.B -1
+(default) is used, the client will round-robin subdirectory creation
+across all MDTs if their free space is within
+.B lod.*.mdt_qos_threshold_rr
+percent of each other, otherwise the client will prefer to select
+.B COUNT
+MDT(s) proportional to the free space and inodes on each.
 .TP
 .BR \-H ", " \-\-mdt-hash =\fIHASH_TYPE\fR
 Use
@@ -39,18 +49,16 @@ Use
 for the striped directory.
 .RS 1.2i
 .TP
+.B crush
+CRUSH hash algorithm.  This is a consistent hash
+algorithm, so minimum sub files need to relocate
+during directory restripe.
+.TP
 .B fnv_1a_64
 Fowler-Noll-Vo (FNV-1a) hash algorithm.  This provides
 reasonably uniform, but not cryptographically strong,
 hashing of the filename. (default)
 .TP
-.B space
-This can only be set on plain directory default striping.
-Newly created sub-directories will be distributed on all
-MDTs by MDT space usage. Note that this is suggested to
-be set on lustre ROOT, so that all sub-directories under
-lustre ROOT are distributed among all MDTs.
-.TP
 .B all_char
 Sum of ASCII characters modulo number of MDTs. This
 provides weak hashing of the filename, and is suitable
@@ -68,14 +76,18 @@ Set the default striping pattern of subdirectories. Newly created
 sub-directories will use the new default striping pattern,
 but existing sub-directories will not be affected.  The newly
 created sub-directories will also inherit the specified default
-striping pattern. Only default stripe count is supported for now.
+striping pattern.
 
-Note that striping all directories across all MDTs by default is not
+Note that striping all directories across all MDTs by default is
+.B not
 recommended at this time, as the clients will have to do more RPCs to
 create and access each directory, hurting performance rather than
-improving it.  Default striped directories are preferred for cases
-where large subdirectories will be created (e.g. file-per-process
-job output directories).
+improving it.  Default striped directories are preferred only for
+parent directories where large subdirectories will be created
+(e.g. file-per-process job output directories).  Instead, using
+.B -c 1 -i -1
+on top-level directories balances mkdir therein over MDTs automatically
+without causing all subdirectories to be remote by default.
 .TP
 .BR \-o ", " \-\-mode =\fIMODE\fR
 Set the file access permissions of the new directory to the specified
@@ -94,36 +106,87 @@ option) striping. Where
 specifies a known foreign type (like
 .BR none
 ,
-.BR daos
+.BR symlink
 , ...) or a 32-bit numeric type.
 .TP
 .BR \-\-flags =\fI<hex>\fR
-Specify a bitmask of flags.
+Specify a numeric bitmask of type-specific layout flags for the foreign layout.
 .TP
 .BR \-x ", " \-\-xattr =\fISTRING\fR
 Specify a string to be used as a foreign (free format) striping.
+.TP
+.BR \-X ", " \-\-max-inherit = \fIMAX_INHERIT
+Set the inherit depth of default directory layout. If non-zero, then
+.I MAX_INHERIT
+is the number of subdirectory levels for which this default layout is inherited,
+up to a maximum of 250 levels, and is decremented by one when copying the
+default layout to each new subdirectory, until zero and the default layout is
+no longer copied. A
+.I MAX_INHERIT
+of -1 means the default layout is inherited for all subdirectories.
+.TP
+.BR \-\-max-inherit-rr = \fIMAX_INHERIT_RR
+Set the round-robin inherit depth of the default directory layout, only when
+.I START_MDT_INDEX
+is -1.  If
+.I MAX_INHERIT_RR
+is zero (unset), then new subdirectories are preferentially created on MDTs
+with more free space and inodes if the MDTs are imbalanced.  If non-zero, then
+.I MAX_INHERIT_RR
+is the number of subdirectory levels, up to a maximum of 250 levels, for which
+new subdirectories will be created in a round-robin manner across all available
+MDTs, rather than using MDT space balancing for new subdirectories.  Otherwise,
+.I MAX_INHERIT_RR
+is decremented by one when copying the default layout to each new subdirectory,
+until zero and round-robin is no longer used.  It may be useful to set
+.I MAX_INHERIT_RR
+to 2 or 3 when setting the default directory layout on the root of a
+.B new
+filesystem, so that top-level subdirectories are immediately spread
+across MDTs, rather than waiting for the MDTs to become imbalanced.
+Round-robin subdirectory creation is unlikely to be useful for existing
+filesystems that already have many files and imbalanced MDTs, since space
+balancing will already happen, and this will unnecessarily increase the number
+of remote subdirectories (increasing overhead) without any benefit.
 .SH NOTE
 .PP
+If neither
+.B -c
+or
+.B -i
+are specified,
+.B lfs mkdir
+will default to
+.B -c 1 -i -1
+and create the directory on an MDT that is less full than the others.
+.PP
 The
 .B lfs setdirstripe
-command is only executable by root unless
+command is only usable by root unless the
 .B "mdt.*.enable_remote_dir_gid"
 is set on the MDS via
 .B lctl set_param
-to be either a non-zero GID to limit it to a single group
-(e.g. "operator" or "admin"), or "-1" to allow any group
-to create remote directories.
+to be either a non-zero numeric GID to limit it to a single group (e.g. the
+.BR "operator " or " admin"
+GID), or
+.B "-1"
+to allow any user/group to create remote directories.  By default, it is
+.B "0"
+to limit remote/striped directories to only the root user.
 
-The root of the file system is on MDT0000, and directories and files inherit the
-MDT of their parent directory unless a different MDT is specified with this
-command.
+The root directory of the file system is on MDT0000, and a filesystem-wide
+default directory layout is set on it by default:
+.B lfs setdirstripe -D -i -1 -c 1 --max-inherit -1 --max-inherit-rr 3
+, which is a space balanced directory layout, therefore new directories will be
+created on all MDTs by inode and space usage, and the top three level of new
+directories will be created in roundrobin mode if system is balanced.  If user
+doesn't want automatic space balance, deleting this default directory layout
+won't work, because it will be recreated if not set, instead user can run
+.B lfs setdirstripe -D -i -1 -c 1 --max-inherit 1 <mountpoint>
+to limit space balance in the top level, or run
+.B lfs setdirstripe -D -i 0 -c 1 --max-inherit 1 <mountpoint>
+to disable it.
 
-By default, only directories on MDT0000 can contain directories that are not on
-the same MDT.  However, if "mdt.*.enable_remote_dir" is set non-zero on an MDT
-then it will allow creating remote directories that have parents other than
-MDT0000. This is restricted to avoid creating directory trees that have
-intermediate path components on a series different MDTs and become unavailable
-if any of the intermediate MDTs are offline.
 .SH EXAMPLES
 .TP
 .B $ lfs setdirstripe -c 2 -i 1 -H all_char /mnt/lustre/dir1
@@ -132,12 +195,12 @@ This creates a directory striped on two MDTs, whose first stripe is on
 (MDT index 1), and whose hash type is
 .BR all_char .
 .TP
-.B $ lfs mkdir --foreign=daos --xattr PUUID:CUUID /mnt/lustre/dir1
+.B $ lfs mkdir --foreign=symlink --xattr PUUID/CUUID /mnt/lustre/dir1
 This creates
 .B dir1
 with foreign (non-lustre/free format)
-.B PUUID:CUUID
-striping/LMV EA.
+.B PUUID/CUUID
+striping/LMV EA value (symlink type).
 .SH AVAILABILITY
 The
 .B lfs setdirstripe