-.TH LFS-SETDIRSTRIPE 1 2014-06-08 "Lustre" "Lustre Utilities"
+.TH LFS-SETDIRSTRIPE 1 2017-11-07 "Lustre" "Lustre Utilities"
.SH NAME
lfs setdirstripe, mkdir \- set striping pattern of a directory.
.SH SYNOPSIS
-.B lfs setdirstripe
-[\fIOPTION\fR]... \fIDIR\fR...
+.B lfs setdirstripe [\fR-cdDhHioTxX\fR] \fIDIR\fR...
.br
.SH DESCRIPTION
-Create a striped directory with specified striping pattern. This lfs utility
-is similar to
-.B lfs setstripe
-, but is used to create striped directory.
+Create a striped directory with specified striping pattern. This
+.B lfs
+sub-command is similar to
+.BR "lfs setstripe" ,
+but is used to create a striped directory or set the default layout for
+subdirectories.
+Can also be used to create directory with a foreign (free format) striping pattern (see
+.BR --foreign
+and
+.BR --xattr
+options).
.B lfs mkdir
is an alias for this command.
.SH OPTIONS
.TP
-\fB\-c\fR, \fB\-\-count\fR=\fICOUNT\fR
-Stripe over
+.BR \-c ", " \-T ", " \-\-mdt\-count =\fICOUNT\fR
+Stripe the new directory over
.I COUNT
MDTs.
.TP
-\fB\-i\fR, \fB\-\-index\fR=\fISTART_MDT_INDEX\fR
-Stripe from the MDT, whose index is
-.I START_MDT_INDEX.
+.BR \-h ", " \-\-help
+Print usage message.
+.TP
+.BR \-i ", " \-\-mdt\-index =\fISTART_MDT_INDEX\fR[,\fIMDT_INDEX ...]
+Use the MDT whose index is
+.I START_MDT_INDEX
+as the master/starting MDT for the directory. If multiple
+.I MDT_INDEX
+values are given, then the stripes will be allocated on the specified
+MDT indices. If
+.B -1
+(default) is used, the client will round-robin subdirectory creation
+across all MDTs if their free space is within
+.B lod.*.mdt_qos_threshold_rr
+percent of each other, otherwise the client will prefer to select
+.B COUNT
+MDT(s) proportional to the free space and inodes on each.
.TP
-\fB\-t\fR, \fB\-\-hash\-type\fR=\fIHASH_TYPE\fR
-the hash_type of the striped directory.
+.BR \-H ", " \-\-mdt-hash =\fIHASH_TYPE\fR
+Use
+.I hash_type
+for the striped directory.
.RS 1.2i
.TP
+.B crush
+CRUSH hash algorithm. This is a consistent hash
+algorithm, so minimum sub files need to relocate
+during directory restripe.
+.TP
.B fnv_1a_64
Fowler-Noll-Vo (FNV-1a) hash algorithm. This provides
reasonably uniform, but not cryptographically strong,
perfectly uniform distribution (e.g. sequential numbers).
.RE
.TP
-.B \-D, \-\-default_stripe
-Set the default striping pattern of the directory. Newly created
+.BR \-d ", " \-\-delete
+Delete the default striping layout from the directory. New subdirectories
+created in this directory will inherit the global default directory layout
+(by default they will not be striped).
+.TP
+.BR \-D ", " \-\-default
+Set the default striping pattern of subdirectories. Newly created
sub-directories will use the new default striping pattern,
but existing sub-directories will not be affected. The newly
-created sub-directories will also inherit the current default
-striping pattern. Only default stripe count is supported for now.
-.TP
-\fB\-m\fR, \fB\-\-mode\fR=\fIMODE\fR
-Set the file access mode of the new directory to the specified
-numeric value (typically octal) as specified in
-.BR chmod (1),
-and is not affected by the current
+created sub-directories will also inherit the specified default
+striping pattern.
+
+Note that striping all directories across all MDTs by default is
+.B not
+recommended at this time, as the clients will have to do more RPCs to
+create and access each directory, hurting performance rather than
+improving it. Default striped directories are preferred only for
+parent directories where large subdirectories will be created
+(e.g. file-per-process job output directories). Instead, using
+.B -c 1 -i -1
+on top-level directories balances mkdir therein over MDTs automatically
+without causing all subdirectories to be remote by default.
+.TP
+.BR \-o ", " \-\-mode =\fIMODE\fR
+Set the file access permissions of the new directory to the specified
+numeric
+.I MODE
+(typically octal), as with
+.BR chmod (1).
+It is not affected by the current
.BR umask (1p).
+.TP
+.BR \-\-foreign[=type]
+Create a directory with a foreign (non-Lustre/free format, see
+.BR \-\-xattr
+option) striping. Where
+.BR type
+specifies a known foreign type (like
+.BR none
+,
+.BR symlink
+, ...) or a 32-bit numeric type.
+.TP
+.BR \-\-flags =\fI<hex>\fR
+Specify a numeric bitmask of type-specific layout flags for the foreign layout.
+.TP
+.BR \-x ", " \-\-xattr =\fISTRING\fR
+Specify a string to be used as a foreign (free format) striping.
+.TP
+.BR \-X ", " \-\-max-inherit = \fIMAX_INHERIT
+Set the inherit depth of default directory layout. If non-zero, then
+.I MAX_INHERIT
+is the number of subdirectory levels for which this default layout is inherited,
+up to a maximum of 250 levels, and is decremented by one when copying the
+default layout to each new subdirectory, until zero and the default layout is
+no longer copied. A
+.I MAX_INHERIT
+of -1 means the default layout is inherited for all subdirectories.
+.TP
+.BR \-\-max-inherit-rr = \fIMAX_INHERIT_RR
+Set the round-robin inherit depth of the default directory layout, only when
+.I START_MDT_INDEX
+is -1. If
+.I MAX_INHERIT_RR
+is zero (unset), then new subdirectories are preferentially created on MDTs
+with more free space and inodes if the MDTs are imbalanced. If non-zero, then
+.I MAX_INHERIT_RR
+is the number of subdirectory levels, up to a maximum of 250 levels, for which
+new subdirectories will be created in a round-robin manner across all available
+MDTs, rather than using MDT space balancing for new subdirectories. Otherwise,
+.I MAX_INHERIT_RR
+is decremented by one when copying the default layout to each new subdirectory,
+until zero and round-robin is no longer used. It may be useful to set
+.I MAX_INHERIT_RR
+to 2 or 3 when setting the default directory layout on the root of a
+.B new
+filesystem, so that top-level subdirectories are immediately spread
+across MDTs, rather than waiting for the MDTs to become imbalanced.
+Round-robin subdirectory creation is unlikely to be useful for existing
+filesystems that already have many files and imbalanced MDTs, since space
+balancing will already happen, and this will unnecessarily increase the number
+of remote subdirectories (increasing overhead) without any benefit.
.SH NOTE
.PP
-The "lfs setdirstripe" command is only executable by root unless
-"mdt.*.enable_remote_dir_gid" is set via "lctl set_param" to be either a
-non-zero GID to limit it to a single group (e.g. "operator" or "admin"),
-or "-1" to allow any group to create remote directories.
+If neither
+.B -c
+or
+.B -i
+are specified,
+.B lfs mkdir
+will default to
+.B -c 1 -i -1
+and create the directory on an MDT that is less full than the others.
+.PP
+The
+.B lfs setdirstripe
+command is only usable by root unless the
+.B "mdt.*.enable_remote_dir_gid"
+is set on the MDS via
+.B lctl set_param
+to be either a non-zero numeric GID to limit it to a single group (e.g. the
+.BR "operator " or " admin"
+GID), or
+.B "-1"
+to allow any user/group to create remote directories. By default, it is
+.B "0"
+to limit remote/striped directories to only the root user.
-The root of the file system is on MDT0000, and directories and files inherit the
-MDT of their parent directory unless a different MDT is specified with this
-command.
+The root directory of the file system is on MDT0000, and a filesystem-wide
+default directory layout is set on it by default:
+.B lfs setdirstripe -D -i -1 -c 1 --max-inherit -1 --max-inherit-rr 3
+, which is a space balanced directory layout, therefore new directories will be
+created on all MDTs by inode and space usage, and the top three level of new
+directories will be created in roundrobin mode if system is balanced. If user
+doesn't want automatic space balance, deleting this default directory layout
+won't work, because it will be recreated if not set, instead user can run
+.B lfs setdirstripe -D -i -1 -c 1 --max-inherit 1 <mountpoint>
+to limit space balance in the top level, or run
+.B lfs setdirstripe -D -i 0 -c 1 --max-inherit 1 <mountpoint>
+to disable it.
-By default, only directories on MDT0000 can contain directories that are not on
-the same MDT. However, if "mdt.*.enable_remote_dir" is set non-zero on an MDT
-then it will allow creating remote directories that have parents other than
-MDT0000. This is restricted to avoid creating directory trees that have
-intermediate path components on a series different MDTs and become unavailable
-if any of the intermediate MDTs are offline.
.SH EXAMPLES
.TP
-.B $ lfs setdirstripe -c 2 -i 1 -t all_char /mnt/lustre/dir1
+.B $ lfs setdirstripe -c 2 -i 1 -H all_char /mnt/lustre/dir1
This creates a directory striped on two MDTs, whose first stripe is on
.B MDT0001
(MDT index 1), and whose hash type is
.BR all_char .
-.SH BUGS
-Default stripe cannot be deleted once it is set.
+.TP
+.B $ lfs mkdir --foreign=symlink --xattr PUUID/CUUID /mnt/lustre/dir1
+This creates
+.B dir1
+with foreign (non-lustre/free format)
+.B PUUID/CUUID
+striping/LMV EA value (symlink type).
.SH AVAILABILITY
-The lfs setdirstripe command is part of the Lustre filesystem.
+The
+.B lfs setdirstripe
+command is part of the Lustre filesystem.
.SH SEE ALSO
+.BR lctl (8),
.BR lfs (1),
.BR lfs-getdirstripe (1),
.BR lfs-setstripe (1),
git://git.whamcloud.com / fs / lustre-release.git / blobdiff