LU-15743 utils: add --xattr option to lfs find

[fs/lustre-release.git] / lustre / doc / lfs_migrate.1

diff --git a/lustre/doc/lfs_migrate.1 b/lustre/doc/lfs_migrate.1
index 892c265..fc9ffab 100644 (file)
--- a/lustre/doc/lfs_migrate.1
+++ b/lustre/doc/lfs_migrate.1
@@ -1,42 +1,37 @@
 .TH lfs_migrate 1 "Dec 19, 2017" Lustre "utilities"
 .SH NAME
 .B lfs_migrate
 .TH lfs_migrate 1 "Dec 19, 2017" Lustre "utilities"
 .SH NAME
 .B lfs_migrate

-\- simple tool to migrate files between Lustre OSTs
+\- migrate files between Lustre OSTs

 .SH SYNOPSIS
 .B lfs_migrate
 .SH SYNOPSIS
 .B lfs_migrate

-.RB [ --dry-run ]
-.RB [ -h ]
+.RB [ "-A " [ -C \fI<cap> \fR] [ -M \fI<min_free> \fR] [ -X \fI<max_free> \fR]]
+.RB [ --dry-run | -n ]
+.RB [ --help | -h ]

 .RB [ --no-rsync | --rsync ]
 .RB [ --no-rsync | --rsync ]

-.RB [ -q ]
-.RB [ -R ]
-.RB [ -s ]
-.RB [ -v ]
-.RB [ -y ]
+.RB [ --pool | -p \fI<pool> \fR]
+.RB [ --quiet | -q ]
+.RB [ --restripe | -R ]
+.RB [ --stripe-count | -c \fI<stripe_count> \fR]
+.RB [ --stripe-size | -S \fI<stripe_size> \fR]
+.RB [ --skip | -s ]
+.RB [ --verbose | -v ]
+.RB [ --yes | -y ]
+.RB [ -D ]

 .RB [ -0 ]
 .RB [ -0 ]

-.RI [ FILE | DIR ]...
+.RI [ FILE | DIR ] ...

 .br
 .SH DESCRIPTION
 .B lfs_migrate
 .br
 .SH DESCRIPTION
 .B lfs_migrate

-is a simple tool to assist migration of files between Lustre OSTs.  It
-is simply copying each specified file to a new file, verifying the file
-contents have not changed, and then renaming the new file back to the
-original filename.  This allows balancing space usage between OSTs, moving
-files off OSTs that are starting to show hardware problems (though are still
-functional), or OSTs that will be removed from the filesystem.
-.PP
-Because
-.B lfs_migrate
-is
-.B not
-yet closely integrated with the MDS, it cannot determine whether a file
-is currently open and/or in-use by other applications or nodes.  That makes
-it
-.B
-UNSAFE
-for use on files that might be modified by other applications, since the
-migrated file is only a copy of the current file, and this will result in
-the old file becoming an open-unlinked file and any modifications to that
-file will be lost.
+is a tool to assist migration of files between Lustre OSTs, possibly also
+restriping the files as it goes. It copies each specified file to a new file,
+verifies the file contents have not changed, and then replaces the original
+filename with the new file (either atomically via
+.BR lfs-migrate (1)
+on Lustre 2.5 and later, or
+.BR mv (1)
+on older versions of Lustre). This allows balancing space usage between OSTs,
+moving files off OSTs that are starting to show hardware problems but are still
+functional, or OSTs that will be removed from the filesystem.

 .PP
 Files to be migrated can be specified as command-line arguments.  If a
 directory is specified on the command-line then all files within that
 .PP
 Files to be migrated can be specified as command-line arguments.  If a
 directory is specified on the command-line then all files within that


@@ -44,19 +39,25 @@ directory are migrated.  If no files are specified on the command-line,
 then a list of files is read from the standard input, making
 .B lfs_migrate
 suitable for use with
 then a list of files is read from the standard input, making
 .B lfs_migrate
 suitable for use with

-.BR lfs (1) " find"
-to locate files on specific OSTs and/or matching other file attributes.
+.BR lfs-find (1)
+to locate files on specific OSTs and/or matching other file attributes,
+or any other tools that generate a list of files.

 .PP
 .PP

-Any options and arguments not explicitly recognized by the script are passed
-through to the
+Any options and arguments not explicitly recognized by
+.B lfs_migrate
+are passed through to the underlying

 .B lfs migrate
 command, see
 .B lfs migrate
 command, see

-.BR lfs-migrate (1).
-To maintain backward compatibility, the \fI-n \fRoption is used by the script
-for a dry-run, and is not passed to
+.BR lfs-migrate (1)
+and
+.BR lfs-setstripe (1)
+for a complete list of options.
+.PP
+To maintain backward compatibility, the \fI-n \fRoption is used by the
+script to indicate a dry-run (no modifications made), and is not passed to

 .B lfs migrate
 as the non-block option.  To specify non-block, use the long option
 .B lfs migrate
 as the non-block option.  To specify non-block, use the long option

-.IR --non-block .
+.BR --non-block .

 .PP
 The current file allocation policies on MDS dictate where the new files
 are placed, taking into account whether specific OSTs have been disabled
 .PP
 The current file allocation policies on MDS dictate where the new files
 are placed, taking into account whether specific OSTs have been disabled


@@ -69,69 +70,123 @@ directory (potentially changing the stripe count, stripe size, OST pool,
 or OST index of a new file).
 .SH OPTIONS
 .TP
 or OST index of a new file).
 .SH OPTIONS
 .TP

-.B \\--dry-run
+.B \\--dry-run|-n

 Only print the names of files to be migrated.
 .TP
 Only print the names of files to be migrated.
 .TP

-.B \\-h
-Display help information.
+.B \\-D
+Do not use direct I/O to copy file contents.
+.TP
+.B \\-A
+Automatically determine the stripe count for the file, using the algorithm
+count = sqrt(filesize_in_GB) + 1.  This option may not be specified at the
+same time as the \fB-c \fRor \\-R \fRoptions.
+.TP
+.B \\--stripe-count|-c \fI<stripe_count>
+Restripe file using the specified \fIstripe_count\fR. This option may not be
+specified at the same time as the \fB-A \fRor \fB-R \fRoptions.
+.TP
+.B \\-C \fI<cap>
+When \fB-A \fRis set, limit the migrated file to use on each OST at most
+1/\fIcap \fRof the available space of the smallest OST.  If this option is not
+set, a default value of 100 is used, limiting the object size to 1% of available
+space.
+.TP
+.B \\--help|-h
+Display usage message.

 .TP
 .B \\--no-rsync
 Do not fall back to using rsync if
 .TP
 .B \\--no-rsync
 Do not fall back to using rsync if

-.BR lfs (1) " migrate" " fails."
-Cannot be used at the same time as \fI--rsync\fR.
+.BR lfs-migrate (1) " fails."
+Cannot be used at the same time as \fB--rsync\fR.

 .TP
 .TP

-.B \\-q
+.B \\--min-free|-M \fI<min_free>
+When \fB-A \fRis set, only consider OSTs with free space greater than the
+\fImin_free \fRvalue to be available for migration.  The value is specified in
+KB. If this option is not set, a default of 256MB is used.
+.TP
+.BR \\--pool | -q \fI<pool>
+Migrate files to specified pool.
+.TP
+.BR \\--quiet | -q

 Run quietly (don't print filenames or status).
 .TP
 .B \\--rsync
 Force rsync to be used instead of
 Run quietly (don't print filenames or status).
 .TP
 .B \\--rsync
 Force rsync to be used instead of

-.BR lfs (1) " migrate" .
-May not be used at the same time as \fI--no-rsync\fR.
+.BR lfs-migrate (1) .
+May not be used at the same time as
+.BR --no-rsync .

 .TP
 .TP

-.B \\-R
+.B \\--restripe|-R

 Restripe file using default directory striping instead of keeping striping.
 Restripe file using default directory striping instead of keeping striping.

-This option may not be specified at the same time as the -c or -S options
-(these options are passed through to
+This option may not be specified at the same time as the \fB-A\fR, \fB-c\fR, or
+\fB-S \fRoptions.  (these options are passed through to

 .BR "lfs migrate" ,
 and are therefore not listed here).
 .TP
 .BR "lfs migrate" ,
 and are therefore not listed here).
 .TP

-.B \\-s
+.B \\--skip|-s

 Skip file data comparison after migrate.  Default is to compare migrated file
 against original to verify correctness.
 .TP
 Skip file data comparison after migrate.  Default is to compare migrated file
 against original to verify correctness.
 .TP

-.B \\-v
+.B \\--stripe-size|-S
+.I <stripe_size>
+Restripe file using the specified stripe size. This option may not be
+specified at the same time as the \fB-R \fRoption.
+.TP
+.B \\--verbose|-v

 Show verbose debug messages.
 .TP
 Show verbose debug messages.
 .TP

-.B \\-y
-Answer 'y' to usage warning without prompting (for scripts, use with caution).
+.B \\--max-free|-X \fI<max_free>
+When \fB-A \fRis set, \fImax_free \fRis the maximum amount of free space that
+can be considered available for the migration of the file on each OST.  The
+value is specified in KB.  This option is useful for testing, by simulating
+OSTs that are nearly full.
+.TP
+.B \\--yes|-y
+Answer 'y' to usage warning without prompt (--rsync only, use with caution).

 .TP
 .B \\-0
 Input file names on stdin are separated by a null character.
 .SH EXAMPLES
 To rebalance all files within
 .TP
 .B \\-0
 Input file names on stdin are separated by a null character.
 .SH EXAMPLES
 To rebalance all files within

-.I /testfs/jobs/2011
-(which are known not to be modified by in-use programs):
+.IR /testfs/jobs/2011 :

 .IP
 lfs_migrate /testfs/jobs/2011
 .PP
 To migrate files within the
 .I /testfs
 filesystem on OST0004 (perhaps because it is much more full than other OSTs),
 .IP
 lfs_migrate /testfs/jobs/2011
 .PP
 To migrate files within the
 .I /testfs
 filesystem on OST0004 (perhaps because it is much more full than other OSTs),

-larger than 4GB (because it is more efficient to just migrate large files),
-and older than two days (to avoid files that are in use, though this is NOT
-a guarantee the files are not being modified, that is workload specific):
+larger than 4GB (because it is more efficient to migrate a few large files than
+many small ones), and older than two days (to avoid files that are in use) to
+use auto-restriping for these files, after disabling file creation on
+testfs-OST0004 (this is needed on all MDS nodes):

 .IP
 .IP

-lfs find /testfs -obd test-OST0004 -size +4G -mtime +2d |
-    lfs_migrate -y
-.SH KNOWN BUGS
-Eventually, this functionality will be integrated into
-.BR lfs (1)
-itself and will integrate with the MDS layout locking to make it safe
-in the presence of opened files and ongoing file IO.
+.nf
+mds# lctl set_param osp.testfs-OST0004*.max_create_count=0
+client# lfs find /testfs -obd testfs-OST0004 -size +4G -mtime +2d | lfs_migrate -A
+mds# lctl set_param osp.testfs-OST0004*.max_create_count=20000
+.fi
+.PP
+To use automatic restriping, and limit the object size per OST to 5% of
+current free space:
+.IP
+lfs_migrate -A -C 20 /testfs/jobs/2011
+.SH NOTES
+In versions prior to 2.5,
+.B lfs_migrate
+is
+.B not
+closely integrated with the MDS, and cannot determine whether a file
+is currently open and/or in-use by other applications or nodes.  That makes
+it
+.B UNSAFE
+for use on files that might be modified by other applications, since the
+migrated file is only a copy of the current file. This will result in the
+old file becoming an open-unlinked file, and any modifications to that file
+will be lost.

 .SH AVAILABILITY
 .B lfs_migrate
 .SH AVAILABILITY
 .B lfs_migrate

-is part of the 
-.BR Lustre (7) 
+is part of the
+.BR Lustre (7)

 filesystem package.  Added in the 1.8.4 release.
 .SH SEE ALSO
 .BR lfs (1)
 filesystem package.  Added in the 1.8.4 release.
 .SH SEE ALSO
 .BR lfs (1)