X-Git-Url: https://git.whamcloud.com/?a=blobdiff_plain;f=lustre%2Fdoc%2Flfs-migrate.1;h=fb72541fedd4252b471eae513fa3101ae330932f;hb=05e6ccd344e7eba44e43230fa2fa0a1b3b6115c4;hp=d3e21113bffd02c4b1dd0e9635f1aaae445a81e9;hpb=af4dfdeb689a5c8b86184480525fec9a6d36699a;p=fs%2Flustre-release.git diff --git a/lustre/doc/lfs-migrate.1 b/lustre/doc/lfs-migrate.1 index d3e2111..fb72541 100644 --- a/lustre/doc/lfs-migrate.1 +++ b/lustre/doc/lfs-migrate.1 @@ -3,62 +3,153 @@ lfs migrate \- migrate files or directories between MDTs or OSTs. .SH SYNOPSIS .B lfs migrate -\fI-m mdt_idx\fR [-v|--verbose] \fI\fR +.RB [ -h "] [" -v ] +.RI [ SETSTRIPE_OPTIONS " ... ]" +.RI < file "> ..." .br -.B lfs migrate -.RI [OPTIONS] ... \fR... +.B lfs migrate -m \fIstart_mdt_index +.RB [ -cHv ] +.RI < directory > .br .SH DESCRIPTION -Migrate MDT inodes or OST objects between MDTs and OSTs respectively. For the -case of MDT inode migration: +Migrate OST objects or MDT inodes between MDTs and OSTs respectively. +.P +The +.B lfs migrate +command can be used for moving files from one (or more) OSTs to other +OSTs (e.g. for space balancing between OSTs, or to evacuate an OST for +hardware reasons), to change the stripe count or other layout parameters +of a file (e.g. to increase the bandwidth of a file by striping it over +multiple OSTs), or to move the file between different classes of storage +(e.g. SSD vs. HDD OSTs, or local vs. remote OSTs in different pools). +.P +In OST object migration mode, the command supports the same +.I SETSTRIPE_OPTIONS +listed in +.BR lfs-setstripe (1) +to specify the layout of the target file. The migrate command differs +from +.B lfs setstripe +in that +.B lfs migrate +will copy the data from the existing file(s) using the new layout parameters +to the new OST(s). In contrast, +.B lfs setstripe +is used for creating new (empty) files with the specified layout. +.SH OST MIGRATE OPTIONS +For OST object migration, there additional options available: .TP -.B migrate -m|--mdt-index \fIdirectory\fR -.br -Move the file metadata (inode) from one MDT to MDT with index \fBmdt_idx\fR. -Migration of striped directories or individual files between MDTs is not -currently supported. Only the root user can migrate directories. Files that -have been archived by HSM or are currently open are skipped by MDT inode -migration. Access to files within the directory is blocked until migration is -complete. +.BR -b , --block +Block access to the file by other applications during data migration +(default). This prevents other processes from accessing the file during +migration, which prevents data data writes to the old file objects from +being lost. This should be used if an OST needs to be completely emptied +prior to its removal, to ensure all requested files are migrated off the +OST. .TP -\fIWARNING\fR -A migrated file or directory will have a new inode number and FID. As -a consequence, files archieved by Lustre HSM cannot currently be migrated -and migrated files that have a new inode number may confuse backup tools. +.BR -h , --help +Print usage message. .TP -For the case of OST object migration: +.BR -n , --non-block +Abort migration if concurrent file access is detected. This can be +used with OST space balancing migration to avoid interfering with file +access by applications if there is not a requirement to migrate any +particular file to the new layout. .TP -.B migrate -.B [--stripe-count|-c \fIstripe_count\fR] -.B [--stripe-index|-i \fIstart_ost_idx\fR] -.B [--stripe-size|-S \fIstripe_size\fR] -.B [--pool|-p \fIpool_name\fR] -.B [--ost-list|-o \fIost_indices\fR] -.B [--block|-b] -.B [--non-block|-n] \fIfile|directory\fR -.br -Migrate can also be used for OST objects by omitting \fI-m\fR option. In this -mode, the command has identical options to -.BR lfs-setstripe (1). -The difference between migrate and setstripe is that \fImigrate\fR will -re-layout the data in existing files using the new layout parameter by copying -the data from the existing OST(s) to the new OST(s). In contrast, -\fIsetstripe\fR is used for creating new files with the specified layout. For -more information, see setstripe in lfs(1). +.BR -D , --non-direct +Do +.B not +use +.B O_DIRECT +read and write operations when migrating a file. The +.B O_DIRECT +option avoids data copy from kernel buffers into userspace, which can +impose CPU and memory overhead on the copy operation, but makes read and +write operations synchronous. Using the +.B --non-direct +option uses buffered read/write operations, which may improve migration +speed at the cost of more CPU and memory overhead. +.TP +.BR -v , --verbose +Print each filename as it is migrated. .P -NOTE: lfs migrate has a complementary script +NOTE: +.B lfs migrate +has a complementary .B lfs_migrate -which is used to provide extra functionality when migrating file data -between OSTs and has a separate man page. +script which is used to provide extra functionality when migrating file +data between OSTs and has a separate man page. See +.BR lfs_migrate (1) +for details. +.SH MDT MIGRATE OPTIONS +.TP +.BR -m , --mdt-index=\fIstart_mdt_index\fR +Directory will be migrated to MDTs starting with +.I start_mdt_index +, or specific MDTs if multiple MDTs are specified in a comma-seperated list. +This is useful if new MDTs have been added to a filesystem and existing user or +project directories should be migrated off old MDTs to balance the space usage +and future metadata workload. If +.I start_mdt_index +is set to -1, the MDT will be chosen by space and inode usage. +.TP +.BR -c , --mdt-count=\fICOUNT\fR +Directory will be migrated to +.I COUNT +MDTs. +.TP +.BR -H , --mdt-hash=\fIHASH_TYPE\fR +Use +.I HASH_TYPE +for the new layout. +.RS 1.2i +.TP +.B all_char (type 1) +Sum of ASCII characters modulo number of MDTs. This +provides weak hashing of the filename, and is suitable +for only testing or when the input is known to have +perfectly uniform distribution (e.g. sequential numbers). +.TP +.B fnv_1a_64 (type 2) +Fowler-Noll-Vo (FNV-1a) hash algorithm. This provides +reasonably uniform, but not cryptographically strong, +hashing of the filename. (default) +.TP +.B crush (type 3) +CRUSH hash algorithm. This is a consistent hash +algorithm, so minimum sub files need to relocate +during directory restripe. +.RE +.P +Only the root user can migrate directories. Files that have been archived by +HSM or are currently opened will fail to migrate, user can run the same migrate +command again to finish migration when files are ready. Both inode and +directory entry will be migrated. During migration directory and sub files can +be accessed like normal ones. +.TP +\fIWARNING\fR +A migrated file or directory will have a new FID, and hence a new inode +number. As a consequence, files archived by Lustre HSM that depend on +the FID as the identifier in the HSM archive cannot currently be migrated. +Having a new inode number may also cause backup tools to consider the +migrated file(s) to be a new, and cause them to be backed up again. +.P .SH EXAMPLES -Move the inodes contained in ./testremote from their current MDT to the -MDT with index 0: -.HP -.B $ lfs migrate -m 0 ./testremote +.TP +.B $ lfs migrate -c 2 /mnt/lustre/file1 +This migrates the file into a new layout with 2 stripes. +.TP +.B $ lfs migrate -E 64M -c 1 -E 256M -c 4 -E -1 -c -1 /mnt/lustre/file1 +This migrates the file into a three component composite layout. +.TP +.B $ lfs migrate -m 0,2 ./testremote +Move the inodes contained in directory ./testremote from their current +MDT to the MDT with index 0 and 2. .SH AUTHOR The lfs command is part of the Lustre filesystem. .SH SEE ALSO .BR lfs (1), +.BR lfs-setstripe (1), .BR lfs-setdirstripe (1), .BR lfs-getdirstripe (1), .BR lfs-mkdir (1),