1 .TH LFS-MIGRATE 1 2021-11-08 "Lustre" "Lustre Utilities"
3 lfs-migrate \- migrate files or directories between MDTs or OSTs.
7 .RI [ SETSTRIPE_OPTIONS " ... ]"
10 .B lfs migrate -m \fISTART_MDT_INDEX
15 Migrate OST objects between OSTs for the specified
17 or recursively migrate
19 and all inodes/directories therein between MDTs.
20 .SH OST MIGRATE OPTIONS
24 command can be used for moving files from one (or more) OSTs to other
25 OSTs (e.g. for space balancing between OSTs, or to evacuate an OST for
26 hardware reasons), to change the stripe count or other layout parameters
27 of a file (e.g. to increase the bandwidth of a file by striping it over
28 multiple OSTs), or to move the file between different classes of storage
29 (e.g. SSD vs. HDD OSTs, or local vs. remote OSTs in different pools).
31 In OST object migration mode, the command supports the same
35 to specify the layout of the target file. The migrate command differs from
39 will copy the data from the existing file(s) using the new layout parameters
40 to the new OST(s). In contrast,
42 is used for creating new (empty) files with the specified layout.
43 For OST object migration, there additional options available:
46 Block access to the file by other applications during data migration
47 (default). This prevents other processes from accessing the file during
48 migration, which prevents data data writes to the old file objects from
49 being lost. This should be used if an OST needs to be completely emptied
50 prior to its removal, to ensure all requested files are migrated off the
57 Abort migration if concurrent file access is detected. This can be
58 used with OST space balancing migration to avoid interfering with file
59 access by applications if there is not a requirement to migrate any
60 particular file to the new layout.
67 read and write operations when migrating a file. The
69 option avoids data copy from kernel buffers into userspace, which can
70 impose CPU and memory overhead on the copy operation, but makes read and
71 write operations synchronous. Using the
73 option uses buffered read/write operations, which may improve migration
74 speed at the cost of more CPU and memory overhead.
76 This option cannot be used on encrypted files when the encryption key is not
77 available. It will result in
82 Limit how much file system bandwidth a migrate job can consume.
85 Output verbose stats about migrate job progress in YAML format.
88 Controls how often the stats are output; this defaults to 5 seconds.
91 Print each filename as it is migrated.
97 script which is used to provide extra functionality when migrating file
98 data between OSTs and has a separate man page. See
101 .SH MDT MIGRATE OPTIONS
103 .BR -m , --mdt-index=\fIMDT_INDEX [, \fIMDT_INDEX ,...]
106 .B and all subdirectories and inodes
107 will be migrated to the MDT with the specified
109 This is useful if new MDTs have been added to a filesystem and existing user or
110 project directories should be migrated off old MDTs to balance the space usage
111 and future metadata workload. If
113 is -1, the MDT index will be balanced by free space and inodes among
114 available MDTs. If multiple
116 values are specified in a comma-seperated list, then all
117 subdirectories will be
119 across all of the specified MDT indices as if an equivalent
123 .BR -c , --mdt-count=\fICOUNT\fR
124 All directories and subdirectories in the tree will be striped across
128 as the primary MDT for the directory. If
133 directory stripes will be chosen from MDTs proportional to the amount
134 of free inodes and space on each MDT. If multiple
136 values are specified in a comma-separated list, then the number of specified
142 Only migrate the specified \fIDIRECTORY\fR and the non-directory inodes that are
143 directly located within it.
144 Similar to '\fBls -d\fR' and '\fBlfs getstripe -d\fR'.
146 .BR -H , --mdt-hash=\fIHASH_TYPE\fR
149 for the new directory layout.
153 Sum of ASCII characters modulo number of MDTs. This
154 provides weak hashing of the filename, and is suitable
155 for only testing or when the input is known to have
156 perfectly uniform distribution (e.g. sequential numbers).
158 .B fnv_1a_64 (type 2)
159 Fowler-Noll-Vo (FNV-1a) hash algorithm. This provides
160 reasonably uniform, but not cryptographically strong,
161 hashing of the filename. (default)
164 CRUSH hash algorithm. This is a consistent hash
165 algorithm, so minimum sub files need to relocate
166 during directory restripe.
171 Only the root user can migrate directories. Files that have been archived by
172 HSM or are currently opened will fail to migrate, user can run the same migrate
173 command again to finish migration when files are ready. Both inode and
174 directory entry will be migrated. During migration directory and sub files can
175 be accessed like normal ones, but the migration itself cannot be interrupted.
178 It is not currently possible to migrate files with an
180 component (Data-on-MDT, DoM). If it is necessary to migrate such files off
181 a particular MDT, they must first be migrated to have a non-DoM file layout
182 and then the inodes migrated separately. See
184 for details on how to migrate DoM files between MDTs.
187 Each migrated file or directory will have a new FID, and hence a new inode
188 number. As a consequence, files archived by Lustre HSM that depend on
189 the FID as the identifier in the HSM archive cannot currently be migrated.
190 Having a new inode number may also cause backup tools to consider the
191 migrated file(s) to be a new, and cause them to be backed up again.
195 .B $ lfs migrate -c 2 /mnt/lustre/file1
196 This migrates the data in
198 into a new layout with 2 stripes.
200 .B $ lfs migrate -E 256M -c 1 -E 16G -c 4 -E eof -c 40 /mnt/lustre/file2
202 This migrates the data in
204 into a three component composite layout (number of stripes depends on
207 .B # lfs migrate -m 0,2 testremote
209 Recursively move the subdirectories and inodes contained in directory
211 from its current MDT to MDT0000 and MDT0002. The
213 directory and all of its subdirectories will be striped across both MDTs.
215 .B $ lfs migrate -m 0,2 -d ./testremote
216 Move ./testremote and the first level of sub files from their current MDT
217 to the MDT with index 0 and 2. Different from above case, the layout of
218 subdirectories under ./testremote won't be changed.
220 .B # lfs setstripe -E 256M -c 1 -E 16G -c 4 -E eof -c 40 topdir
221 Set a default PFL layout (without any DoM component) on the directory
224 .B # lfs find dir -type f -L mdt -0 | xargs -0 lfs migrate --copy topdir
225 then find and migrate all regular files that have an
227 component to copy the default layout from the specified
230 .B # lfs migrate -m 2 topdir
232 and finally migrate the directory
234 and all files and subdirectories in that tree to MDT0002. This allows
235 migrating files with DoM components off an MDT.
237 The lfs command is part of the Lustre filesystem.
240 .BR lfs-setstripe (1),
241 .BR lfs-setdirstripe (1),
242 .BR lfs-getdirstripe (1),