1 <?xml version="1.0" encoding="utf-8"?>
2 <chapter xmlns="http://docbook.org/ns/docbook"
3 xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
5 xmlns:xi="http://www.w3.org/2001/XInclude">
6 <title xml:id="userutilities.title">User Utilities</title>
7 <para>This chapter describes user utilities.</para>
8 <section xml:id="dbdoclet.50438206_94597">
11 <primary>lfs</primary>
13 <literal>lfs</literal>
16 <literal>lfs</literal> utility can be used for user configuration routines
17 and monitoring.</para>
19 <title>Synopsis</title>
22 lfs changelog [--follow] <replaceable>mdt_name</replaceable> [startrec [endrec]]
23 lfs changelog_clear <replaceable>mdt_name id endrec</replaceable>
24 lfs check <replaceable>mds|osts|servers</replaceable>
25 lfs df [-i] [-h] [--pool]-p <replaceable>fsname</replaceable>[.<replaceable>pool</replaceable>] [<replaceable>path</replaceable>] [--lazy]
26 lfs find [[!] --atime|-A [-+]N] [[!] --mtime|-M [-+]N]
27 [[!] --ctime|-C [-+]N] [--maxdepth|-D N] [--name|-n <replaceable>pattern</replaceable>]
28 [--print|-p] [--print0|-P] [[!] --obd|-O <replaceable>ost_name</replaceable>[,<replaceable>ost_name...</replaceable>]]
29 [[!] --size|-S [+-]N[kMGTPE]] --type |-t {bcdflpsD}]
30 [[!] --gid|-g|--group|-G <replaceable>gname|gid</replaceable>]
31 [[!] --uid|-u|--user|-U <replaceable>uname|uid</replaceable>]
32 <replaceable>dirname|filename</replaceable>
33 lfs getname [-h]|[path...]
34 lfs getstripe [--obd|-O <replaceable>ost_name</replaceable>] [--quiet|-q] [--verbose|-v]
35 [--stripe-count|-c] [--stripe-index|-i]
36 [--stripe-size|-s] [--pool|-p] [--directory|-d]
37 [--mdt-index|-M] [--recursive|-r] [--raw|-R]
39 <replaceable>dirname|filename</replaceable> ...
40 lfs setstripe [--size|-s stripe_size] [--count|-c <replaceable>stripe_count</replaceable>]
41 [--stripe-index|-i <replaceable>start_ost_index</replaceable>]
42 [--ost-list|-o <replaceable>ost_indicies</replaceable>]
43 [--pool|-p <replaceable>pool</replaceable>]
44 <replaceable>dirname|filename</replaceable>
45 lfs setstripe -d <replaceable>dir</replaceable>
47 lfs pool_list <replaceable>filesystem</replaceable>[.<replaceable>pool</replaceable>]| <replaceable>pathname</replaceable>
48 lfs quota [-q] [-v] [-h] [-o <replaceable>obd_uuid</replaceable>|-I <replaceable>ost_idx</replaceable>|-i <replaceable>mdt_idx</replaceable>]
49 [-u <replaceable>username|uid|-g</replaceable> <replaceable>group|gid</replaceable>|-p <replaceable>projid</replaceable>] <replaceable>/mount_point</replaceable>
50 lfs quota -t -u|-g|-p <replaceable>/mount_point</replaceable>
51 lfs quotacheck [-ug] <replaceable>/mount_point</replaceable>
52 lfs quotachown [-i] <replaceable>/mount_point</replaceable>
53 lfs quotainv [-ug] [-f] <replaceable>/mount_point</replaceable>
54 lfs quotaon [-ugf] <replaceable>/mount_point</replaceable>
55 lfs quotaoff [-ug] <replaceable>/mount_point</replaceable>
56 lfs setquota {-u|--user|-g|--group|-p|--project} <replaceable>uname|uid|gname|gid|projid</replaceable>
57 [--block-softlimit <replaceable>block_softlimit</replaceable>]
58 [--block-hardlimit <replaceable>block_hardlimit</replaceable>]
59 [--inode-softlimit <replaceable>inode_softlimit</replaceable>]
60 [--inode-hardlimit <replaceable>inode_hardlimit</replaceable>]
61 <replaceable>/mount_point</replaceable>
62 lfs setquota -u|--user|-g|--group|-p|--project <replaceable>uname|uid|gname|gid|projid</replaceable>
63 [-b <replaceable>block_softlimit</replaceable>] [-B <replaceable>block_hardlimit</replaceable>]
64 [-i <replaceable>inode-softlimit</replaceable>] [-I <replaceable>inode_hardlimit</replaceable>]
65 <replaceable>/mount_point</replaceable>
66 lfs setquota -t -u|-g|-p [--block-grace <replaceable>block_grace</replaceable>]
67 [--inode-grace <replaceable>inode_grace</replaceable>]
68 <replaceable>/mount_point</replaceable>
69 lfs setquota -t -u|-g|-p [-b <replaceable>block_grace</replaceable>] [-i <replaceable>inode_grace</replaceable>]
70 <replaceable>/mount_point</replaceable>
74 <para>In the above example, the
76 <replaceable>/mount_point</replaceable>
77 </literal> parameter refers to the mount point of the Lustre file
81 <para>The old lfs quota output was very detailed and contained
82 cluster-wide quota statistics (including cluster-wide limits for a
83 user/group and cluster-wide usage for a user/group), as well as
84 statistics for each MDS/OST. Now,
85 <literal>lfs quota</literal> has been updated to provide only
86 cluster-wide statistics, by default. To obtain the full report of
87 cluster-wide limits, usage and statistics, use the
88 <literal>-v</literal> option with
89 <literal>lfs quota</literal>.</para>
93 <title>Description</title>
95 <literal>lfs</literal> utility is used to create a new file with a
96 specific striping pattern, determine the default striping pattern, gather
97 the extended attributes (object numbers and location) for a specific
98 file, find files with specific attributes, list OST information or set
99 quota limits. It can be invoked interactively without any arguments or in
100 a non-interactive mode with one of the supported arguments.</para>
103 <title>Options</title>
105 <literal>lfs</literal> options are listed and described below. For a
106 complete list of available options, type help at the
107 <literal>lfs</literal> prompt.</para>
108 <informaltable frame="all">
110 <colspec colname="c1" colwidth="20*" />
111 <colspec colname="c2" colwidth="30*" />
112 <colspec colname="c3" colwidth="50*" />
115 <entry nameend="c2" namest="c1">
117 <emphasis role="bold">Option</emphasis>
122 <emphasis role="bold">Description</emphasis>
129 <entry nameend="c2" namest="c1">
131 <literal>changelog</literal>
135 <para>Shows the metadata changes on an MDT. Start and end
136 points are optional. The
137 <literal>--follow</literal> option blocks on new changes; this
138 option is only valid when run directly on the MDT node.</para>
142 <entry nameend="c2" namest="c1">
144 <literal>changelog_clear</literal>
148 <para>Indicates that changelog records previous to
150 <replaceable>endrec</replaceable>
151 </literal> are no longer of interest to a particular consumer
153 <replaceable>id</replaceable>
154 </literal>, potentially allowing the MDT to free up disk space.
157 <replaceable>endrec</replaceable>
158 </literal> of 0 indicates the current last record. Changelog
159 consumers must be registered on the MDT node using
160 <literal>lctl</literal>.</para>
164 <entry nameend="c2" namest="c1">
165 <literal>check</literal>
168 <para>Displays the status of MDS or OSTs (as specified in the
169 command) or all servers (MDS and OSTs).</para>
173 <entry nameend="c2" namest="c1">
174 <literal>df [-i] [-h] [--pool|-p
175 <replaceable>fsname</replaceable>[.
176 <replaceable>pool</replaceable>] [
177 <replaceable>path</replaceable>] [--lazy]</literal>
181 <literal>-i</literal> to report file system disk space usage or
182 inode usage of each MDT or OST or, if a pool is specified with
184 <literal>-p</literal> option, a subset of OSTs.</para>
185 <para>By default, the usage of all mounted Lustre file systems
187 <literal>path</literal> option is included, only the usage for
188 the specified file system is reported. If the
189 <literal>-h</literal> option is included, the output is printed
190 in human-readable format, using SI base-2 suffixes for
191 <emphasis role="bold">M</emphasis>ega-,
192 <emphasis role="bold">G</emphasis>iga-,
193 <emphasis role="bold">T</emphasis>era-,
194 <emphasis role="bold">P</emphasis>eta-, or
195 <emphasis role="bold">E</emphasis>xabytes.</para>
197 <literal>--lazy</literal> option is specified, any OST that is
198 currently disconnected from the client will be skipped. Using
200 <literal>--lazy</literal> option prevents the
201 <literal>df</literal> output from being blocked when an OST is
202 offline. Only the space on the OSTs that can currently be
203 accessed are returned. The
204 <literal>llite.*.lazystatfs</literal> tunable can be enabled to
205 make this the default behaviour for all
206 <literal>statfs()</literal> operations.</para>
210 <entry nameend="c2" namest="c1">
212 <literal>find</literal>
216 <para>Searches the directory tree rooted at the given
217 directory/filename for files that match the given
220 <literal>!</literal> before an option negates its meaning (files
221 NOT matching the parameter). Using
222 <literal>+</literal> before a numeric value means files with the
223 parameter OR MORE. Using
224 <literal>-</literal> before a numeric value means files with the
225 parameter OR LESS.</para>
231 <literal>--atime</literal>
234 <para>File was last accessed N*24 hours ago. (There is no
236 <literal>atime</literal> is kept coherent across the
238 <para>OSTs store a transient
239 <literal>atime</literal> that is updated when clients do read
241 <literal>atime</literal> is written to the MDS when the file is
242 closed. However, on-disk atime is only updated if it is more
243 than 60 seconds old (
244 <literal>/proc/fs/lustre/mds/*/max_atime_diff</literal>). The
245 Lustre software considers the latest
246 <literal>atime</literal> from all OSTs. If a
247 <literal>setattr</literal> is set by user, then it is updated on
248 both the MDS and OST, allowing the
249 <literal>atime</literal> to go backward.</para>
258 <literal>--ctime</literal>
262 <para>File status was last changed N*24 hours ago.</para>
271 <literal>--mtime</literal>
275 <para>File data was last modified N*24 hours ago.</para>
284 <literal>--obd</literal>
288 <para>File has an object on a specific OST(s).</para>
297 <literal>--size</literal>
301 <para>File has a size in bytes, or kilo-, Mega-, Giga-, Tera-,
302 Peta- or Exabytes if a suffix is given.</para>
311 <literal>--type</literal>
315 <para>File has the type - block, character, directory, pipe,
316 file, symlink, socket or door (used in Solaris operating
326 <literal>--uid</literal>
330 <para>File has a specific numeric user ID.</para>
339 <literal>--user</literal>
343 <para>File owned by a specific user (numeric user ID
353 <literal>--gid</literal>
357 <para>File has a specific group ID.</para>
366 <literal>--group</literal>
370 <para>File belongs to a specific group (numeric group ID
380 <literal>-maxdepth</literal></para>
383 <para>Limits find to descend at most N levels of the directory
393 <literal>--print</literal>/
394 <literal>--print0</literal></para>
397 <para>Prints the full filename, followed by a new line or NULL
398 character correspondingly.</para>
402 <entry nameend="c2" namest="c1">
404 <literal>osts [path]</literal>
408 <para>Lists all OSTs for the file system. If a path located on
409 a mounted Lustre file system is specified, then only OSTs
410 belonging to this file system are displayed.</para>
414 <entry nameend="c2" namest="c1">
416 <literal>getname [path...]</literal>
420 <para>List each Lustre file system instance associated with
421 each Lustre mount point. If no path is specified, all Lustre
422 mount points are interrogated. If a list of paths is provided,
423 the instance of each path is provided. If the path is not a
424 Lustre instance 'No such device' is returned.</para>
428 <entry nameend="c2" namest="c1">
430 <literal>getstripe</literal>
434 <para>Lists striping information for a given filename or
435 directory. By default, the stripe count, stripe size and offset
437 <para>If you only want specific striping information, then the
439 <literal>--stripe-count</literal>,
440 <literal>--stripe-size</literal>,
441 <literal>--stripe-index</literal>,
442 <literal>--layout</literal>, or
443 <literal>--pool</literal> plus various combinations of these
444 options can be used to retrieve specific information.</para>
446 <literal>--raw</literal> option is specified, the stripe
447 information is printed without substituting the file system
448 default values for unspecified fields. If the striping EA is
449 not set, 0, 0, and -1 will be printed for the stripe count,
450 size, and offset respectively.</para>
451 <para condition="l24">The
452 <literal>--mdt-index</literal> prints the index of the MDT for a given
454 <xref linkend="dbdoclet.rmremotedir" />.</para>
464 <replaceable>ost_name</replaceable></literal>
468 <para>Lists files that have an object on a specific OST.</para>
477 <literal>--quiet</literal>
481 <para>Lists details about the file's object ID
491 <literal>--verbose</literal>
495 <para>Prints additional striping information.</para>
504 <literal>--count</literal>
508 <para>Lists the stripe count (how many OSTs to use).</para>
517 <literal>--index</literal>
521 <para>Lists the index for each OST in the file system.</para>
530 <literal>--offset</literal>
534 <para>Lists the OST index on which file striping starts.</para>
543 <literal>--pool</literal>
547 <para>Lists the pools to which a file belongs.</para>
556 <literal>--size</literal>
560 <para>Lists the stripe size (how much data to write to one OST
561 before moving to the next OST).</para>
570 <literal>--directory</literal>
574 <para>Lists entries about a specified directory instead of its
575 contents (in the same manner as
576 <literal>ls -d</literal>).</para>
585 <literal>--recursive</literal>
589 <para>Recurses into all sub-directories.</para>
593 <entry nameend="c2" namest="c1">
595 <literal>setstripe</literal>
599 <para>Create new files with a specific file layout (stripe
600 pattern) configuration.
602 <para>The file cannot exist prior to using
603 <literal>setstripe</literal>. A directory must exist prior to
605 <literal>setstripe</literal>.</para>
615 <literal>--count stripe_cnt</literal>
619 <para>Number of OSTs over which to stripe a file. A
620 <literal>stripe_cnt</literal> of 0 uses the file system-wide
621 default stripe count (default is 1). A
622 <literal>stripe_cnt</literal> of -1 stripes over all available
632 <literal>--size stripe_size</literal>
634 <para>The default stripe-size is 0. The default start-ost is
635 -1. Do NOT confuse them! If you set start-ost to 0, all new
636 file creations occur on OST 0 (seldom a good idea).</para>
637 </footnote> </para>
640 <para>Number of bytes to store on an OST before moving to the
641 next OST. A stripe_size of 0 uses the file system's default
642 stripe size, (default is 1 MB). Can be specified with
643 <emphasis role="bold">k</emphasis>(KB),
644 <emphasis role="bold">m</emphasis>(MB), or
645 <emphasis role="bold">g</emphasis>(GB), respectively.</para>
654 <literal>--stripe-index start_ost_index</literal>
658 <para>The OST index (base 10, starting at 0) on which to start
659 striping for this file. A start_ost_index value of -1 allows
660 the MDS to choose the starting index. This is the default
661 value, and it means that the MDS selects the starting OST as it
662 wants. We strongly recommend selecting this default, as it
663 allows space and load balancing to be done by the MDS as
665 <literal>start_ost_index</literal> value has no relevance on
666 whether the MDS will use round-robin or QoS weighted allocation
667 for the remaining stripes in the file.</para>
676 <literal>--ost-index ost_indices</literal>
680 <para>This option is used to specify the exact stripe
681 layout on the the file system. <literal>ost_indices</literal>
682 is a list of OSTs referenced by their indices and index ranges
683 separated by commas, e.g. <literal>1,2-4,7</literal>.</para>
693 <replaceable>pool</replaceable></literal>
697 <para>Name of the pre-defined pool of OSTs (see
698 <xref linkend="dbdoclet.50438219_38274" />) that will be used
700 <literal>stripe_cnt</literal>,
701 <literal>stripe_size</literal> and
702 <literal>start_ost</literal> values are used as well. The
703 start-ost value must be part of the pool or an error is
708 <entry nameend="c2" namest="c1">
710 <literal>setstripe -d</literal>
714 <para>Deletes default striping on the specified
719 <entry nameend="c2" namest="c1">
721 <literal>pool_list {filesystem}[.poolname]|{pathname}</literal>
725 <para>Lists pools in the file system or pathname, or OSTs in
726 the file system's pool.</para>
730 <entry nameend="c2" namest="c1">
732 <literal>quota [-q] [-v] [-o
733 <replaceable>obd_uuid</replaceable>|-i
734 <replaceable>mdt_idx</replaceable>|-I
735 <replaceable>ost_idx</replaceable>] [-u|-g|-p
736 <replaceable>uname|uid|gname|gid|projid]</replaceable>
737 <replaceable>/mount_point</replaceable></literal>
742 <para>Displays disk usage and limits, either for the full file
743 system or for objects on a specific OBD. A user or group name
744 or an usr, group and project ID can be specified. If all user,
745 group project ID are omitted, quotas for the current UID/GID
746 are shown. The <literal>-q</literal> option disables printing
747 of additional descriptions (including column titles). It fills
748 in blank spaces in the
749 <literal>grace</literal> column with zeros (when there is no
750 grace period set), to ensure that the number of columns is
752 <literal>-v</literal> option provides more verbose (per-OBD
753 statistics) output.</para>
757 <entry nameend="c2" namest="c1">
760 <replaceable>-u|-g|-p</replaceable>
761 <replaceable>/mount_point</replaceable></literal>
765 <para>Displays block and inode grace times for user (
766 <literal>-u</literal>) or group (
767 <literal>-g</literal>) or project (
768 <literal>-p</literal>) quotas.</para>
772 <entry nameend="c2" namest="c1">
774 <literal>quotachown</literal>
778 <para>Changes the file's owner and group on OSTs of the
779 specified file system.</para>
783 <entry nameend="c2" namest="c1">
785 <literal>quotacheck [-ugf]
786 <replaceable>/mount_point</replaceable></literal>
790 <para>Scans the specified file system for disk usage, and
791 creates or updates quota files. Options specify quota for users
793 <literal>-u</literal>), groups (
794 <literal>-g</literal>), and force (
795 <literal>-f</literal>).</para>
799 <entry nameend="c2" namest="c1">
801 <literal>quotaon [-ugf]
802 <replaceable>/mount_point</replaceable></literal>
806 <para>Turns on file system quotas. Options specify quota for
808 <literal>-u</literal>), groups (
809 <literal>-g</literal>), and force (
810 <literal>-f</literal>).</para>
814 <entry nameend="c2" namest="c1">
816 <literal>quotaoff [-ugf]
817 <replaceable>/mount_point</replaceable></literal>
821 <para>Turns off file system quotas. Options specify quota for
823 <literal>-u</literal>), groups (
824 <literal>-g</literal>), and force (
825 <literal>-f</literal>).</para>
829 <entry nameend="c2" namest="c1">
831 <literal>quotainv [-ug] [-f]
832 <replaceable>/mount_point</replaceable></literal>
836 <para>Clears quota files (administrative quota files if used
838 <literal>-f</literal>, operational quota files otherwise), all
839 of their quota entries for users (
840 <literal>-u</literal>) or groups (
841 <literal>-g</literal>). After running
842 <literal>quotainv</literal>, you must run
843 <literal>quotacheck</literal> before using quotas.</para>
845 <para>Use extreme caution when using this command; its
846 results cannot be undone.</para>
851 <entry nameend="c2" namest="c1">
853 <literal>setquota {-u|-g|-p
854 <replaceable>uname|uid|gname|gid|projid}</replaceable>
856 <replaceable>block_softlimit</replaceable>]
858 <replaceable>block_hardlimit</replaceable>]
860 <replaceable>inode_softlimit</replaceable>]
862 <replaceable>inode_hardlimit</replaceable>]
863 <replaceable>/mount_point</replaceable></literal>
867 <para>Sets file system quotas for users, groups or one project.
868 Limits can be specified with
869 <literal>--{block|inode}-{softlimit|hardlimit}</literal> or
870 their short equivalents
871 <literal>-b</literal>,
872 <literal>-B</literal>,
873 <literal>-i</literal>,
874 <literal>-I</literal>. Users can set 1, 2, 3 or 4 limits.
877 <literal>setquota</literal> interface is supported, but it may
878 be removed in a future Lustre software release.</para>
879 </footnote>Also, limits can be specified with special suffixes,
880 -b, -k, -m, -g, -t, and -p to indicate units of 1, 2^10, 2^20,
881 2^30, 2^40 and 2^50, respectively. By default, the block limits
882 unit is 1 kilobyte (1,024), and block limits are always
883 kilobyte-grained (even if specified in bytes). See
884 <xref linkend="dbdoclet.50438206_11903" />.</para>
888 <entry nameend="c2" namest="c1">
890 <literal>setquota -t -u|-g|-p [--block-grace
891 <replaceable>block_grace</replaceable>] [--inode-grace
892 <replaceable>inode_grace</replaceable>]
893 <replaceable>/mount_point</replaceable></literal>
897 <para>Sets the file system quota grace times for users or
898 groups. Grace time is specified in '
899 <literal>XXwXXdXXhXXmXXs</literal>' format or as an integer
901 <xref linkend="dbdoclet.50438206_11903" />.</para>
905 <entry nameend="c2" namest="c1">
907 <literal>help</literal>
911 <para>Provides brief help on various
912 <literal>lfs</literal> arguments.</para>
916 <entry nameend="c2" namest="c1">
918 <literal>exit/quit</literal>
922 <para>Quits the interactive
923 <literal>lfs</literal> session.</para>
930 <section remap="h5" xml:id="dbdoclet.50438206_11903">
931 <title>Examples</title>
932 <para>Creates a file striped on two OSTs with 128 KB on each
935 $ lfs setstripe -s 128k -c 2 /mnt/lustre/file1
937 <para>Deletes a default stripe pattern on a given directory. New files
938 use the default striping pattern.</para>
940 $ lfs setstripe -d /mnt/lustre/dir
942 <para>Lists the detailed object allocation of a given file.</para>
944 $ lfs getstripe -v /mnt/lustre/file1
946 <para>List all the mounted Lustre file systems and corresponding Lustre
951 <para>Efficiently lists all files in a given directory and its
952 subdirectories.</para>
954 $ lfs find /mnt/lustre
956 <para>Recursively lists all regular files in a given directory more than
959 $ lfs find /mnt/lustre -mtime +30 -type f -print
961 <para>Recursively lists all files in a given directory that have objects
962 on OST2-UUID. The lfs check servers command checks the status of all
963 servers (MDT and OSTs).</para>
965 $ lfs find --obd OST2-UUID /mnt/lustre/
967 <para>Lists all OSTs in the file system.</para>
971 <para>Lists space usage per OST and MDT in human-readable format.</para>
975 <para>Lists inode usage per OST and MDT.</para>
979 <para>List space or inode usage for a specific OST pool.</para>
982 <replaceable>filesystem</replaceable>[.
983 <replaceable>pool</replaceable>] |
984 <replaceable>pathname</replaceable>
986 <para>List quotas of user 'bob'.</para>
988 $ lfs quota -u bob /mnt/lustre
990 <para>List quotas of project ID '1'.</para>
992 $ lfs quota -p 1 /mnt/lustre
994 <para>Show grace times for user quotas on
995 <literal>/mnt/lustre</literal>.</para>
997 $ lfs quota -t -u /mnt/lustre
999 <para>Changes file owner and group.</para>
1001 $ lfs quotachown -i /mnt/lustre
1003 <para>Checks quotas for user and group. Turns on quotas after making the
1006 $ lfs quotacheck -ug /mnt/lustre
1008 <para>Turns on quotas of user and group.</para>
1010 $ lfs quotaon -ug /mnt/lustre
1012 <para>Turns off quotas of user and group.</para>
1014 $ lfs quotaoff -ug /mnt/lustre
1016 <para>Sets quotas of user 'bob', with a 1 GB block quota hardlimit and a
1017 2 GB block quota softlimit.</para>
1019 $ lfs setquota -u bob --block-softlimit 2000000 --block-hardlimit 1000000
1022 <para>Sets grace times for user quotas: 1000 seconds for block quotas, 1
1023 week and 4 days for inode quotas.</para>
1025 $ lfs setquota -t -u --block-grace 1000 --inode-grace 1w4d /mnt/lustre
1027 <para>Checks the status of all servers (MDT, OST)</para>
1031 <para>Creates a file striped on two OSTs from the pool
1032 <literal>my_pool</literal></para>
1034 $ lfs setstripe --pool my_pool -c 2 /mnt/lustre/file
1036 <para>Lists the pools defined for the mounted Lustre file system
1037 <literal>/mnt/lustre</literal></para>
1039 $ lfs pool_list /mnt/lustre/
1041 <para>Lists the OSTs which are members of the pool
1042 <literal>my_pool</literal> in file system
1043 <literal>my_fs</literal></para>
1045 $ lfs pool_list my_fs.my_pool
1047 <para>Finds all directories/files associated with
1048 <literal>poolA</literal>.</para>
1050 $ lfs find /mnt/lustre --pool poolA
1052 <para>Finds all directories/files not associated with a pool.</para>
1054 $ lfs find /mnt//lustre --pool ""
1056 <para>Finds all directories/files associated with pool.</para>
1058 $ lfs find /mnt/lustre ! --pool ""
1060 <para>Associates a directory with the pool
1061 <literal>my_pool</literal>, so all new files and directories are created
1064 $ lfs setstripe --pool my_pool /mnt/lustre/dir
1067 <section remap="h5">
1068 <title>See Also</title>
1070 <xref linkend="dbdoclet.50438219_38274" />
1074 <section xml:id="dbdoclet.50438206_42260">
1077 <primary>lfs_migrate</primary>
1079 <literal>lfs_migrate</literal>
1082 <literal>lfs_migrate</literal> utility is a simple tool to migrate files
1083 between Lustre OSTs.</para>
1084 <section remap="h5">
1085 <title>Synopsis</title>
1087 lfs_migrate [-c <replaceable>stripecount</replaceable>] [-h] [-l] [-n] [-q] [-R] [-s] [-y]
1088 [file|directory ...]
1091 <section remap="h5">
1092 <title>Description</title>
1094 <literal>lfs_migrate</literal> utility is a simple tool to assist
1095 migration of files between Lustre OSTs. The utility copies each specified
1096 file to a new file, verifies the file contents have not changed, and then
1097 renames the new file to the original filename. This allows balanced space
1098 usage between OSTs, moving files off OSTs that are starting to show
1099 hardware problems (though are still functional) or OSTs that will be
1100 discontinued.</para>
1102 <para>For versions of Lustre before 2.5,
1103 <literal>lfs_migrate</literal> is not closely integrated with the MDS,
1104 it cannot determine whether a file is currently open and/or in-use by
1105 other applications or nodes. This makes it UNSAFE for use on files that
1106 might be modified by other applications, since the migrated file is
1107 only a copy of the current file. This results in the old file becoming
1108 an open-unlinked file and any modifications to that file are
1111 <para>Files to be migrated can be specified as command-line arguments. If
1112 a directory is specified on the command-line then all files within the
1113 directory are migrated. If no files are specified on the command-line,
1114 then a list of files is read from the standard input, making
1115 <literal>lfs_migrate</literal> suitable for use with
1116 <literal>lfs</literal> find to locate files on specific OSTs and/or
1117 matching other file attributes.</para>
1118 <para>The current file allocation policies on the MDS dictate where the
1119 new files are placed, taking into account whether specific OSTs have been
1120 disabled on the MDS via
1121 <literal>lctl</literal>(preventing new files from being allocated there),
1122 whether some OSTs are overly full (reducing the number of files placed on
1123 those OSTs), or if there is a specific default file striping for the
1124 target directory (potentially changing the stripe count, stripe size, OST
1125 pool, or OST index of a new file).</para>
1128 <literal>lfs_migrate</literal> utility can also be used in some cases to
1131 <primary>fragmentation</primary>
1132 </indexterm>fragmentation. File fragmentation will typically reduce
1133 Lustre file system performance. File fragmentation may be observed on
1134 an aged file system and will commonly occur if the file was written by
1135 many threads. Provided there is sufficient free space (or if it was
1136 written when the file system was nearly full) that is less fragmented
1137 than the file being copied, re-writing a file with
1138 <literal>lfs_migrate</literal> will result in a migrated file with
1139 reduced fragmentation. The tool
1140 <literal>filefrag</literal> can be used to report file fragmentation.
1142 <xref linkend="dbdoclet.50438206_75125" /></para>
1145 <para>As long as a file has extent lengths of tens of megabytes (
1146 <replaceable>read_bandwidth * seek_time</replaceable>) or more, the
1147 read performance for the file will not be significantly impacted by
1148 fragmentation, since the read pipeline can be filled by large reads
1149 from disk even with an occasional disk seek.</para>
1152 <section remap="h5">
1153 <title>Options</title>
1154 <para>Options supporting
1155 <literal>lfs_migrate</literal> are described below.</para>
1156 <informaltable frame="all">
1158 <colspec colname="c1" colwidth="50*" />
1159 <colspec colname="c2" colwidth="50*" />
1164 <emphasis role="bold">Option</emphasis>
1169 <emphasis role="bold">Description</emphasis>
1179 <replaceable>stripecount</replaceable></literal>
1183 <para>Restripe file using the specified stripe count. This
1184 option may not be specified at the same time as the
1185 <literal>-R</literal> option.</para>
1191 <literal>-h</literal>
1195 <para>Display help information.</para>
1200 <literal>-l</literal>
1203 <para>Migrate files with hard links (skips, by default). Files
1204 with multiple hard links are split into multiple separate files
1206 <literal>lfs_migrate</literal>, so they are skipped, by
1207 default, to avoid breaking the hard links.</para>
1212 <literal>-n</literal>
1215 <para>Only print the names of files to be migrated.</para>
1221 <literal>-q</literal>
1225 <para>Run quietly (does not print filenames or status).</para>
1230 <literal>-R</literal>
1232 <entry>Restripe file using default directory striping instead of
1233 keeping striping. This option may not be specified at the same
1235 <literal>-c</literal> option.</entry>
1239 <literal>-s</literal>
1241 <entry>Skip file data comparison after migrate. Default is to
1242 compare migrated file against original to verify
1243 correctness.</entry>
1248 <literal>-y</literal>
1253 <literal>y</literal>' to usage warning without prompting (for
1254 scripts, use with caution).</para>
1261 <section remap="h5">
1262 <title>Examples</title>
1263 <para>Rebalances all files in
1264 <literal>/mnt/lustre/dir</literal>.</para>
1266 $ lfs_migrate /mnt/lustre/file
1268 <para>Migrates files in /test filesystem on OST004 larger than 4 GB in
1271 $ lfs find /test -obd test-OST004 -size +4G | lfs_migrate -y
1274 <section remap="h5">
1275 <title>See Also</title>
1277 <xref linkend="dbdoclet.50438206_94597" />
1281 <section xml:id="dbdoclet.50438206_75125">
1284 <primary>filefrag</primary>
1286 <literal>filefrag</literal>
1289 <literal>e2fsprogs</literal> package contains the
1290 <literal>filefrag</literal> tool which reports the extent of file
1291 fragmentation.</para>
1292 <section remap="h5">
1293 <title>Synopsis</title>
1295 filefrag [ -belsv ] [ files... ]
1298 <section remap="h5">
1299 <title>Description</title>
1301 <literal>filefrag</literal> utility reports the extent of fragmentation in
1303 <literal>filefrag</literal> utility obtains the extent information from
1304 Lustre files using the
1305 <literal>FIEMAP ioctl</literal>, which is efficient and fast, even for
1306 very large files.</para>
1307 <para>In default mode
1309 <para>The default mode is faster than the verbose/extent mode since it
1310 only counts the number of extents.</para>
1312 <literal>filefrag</literal> prints the number of physically discontiguous
1313 extents in the file. In extent or verbose mode, each extent is printed
1314 with details such as the blocks allocated on each OST. For a Lustre file
1315 system, the extents are printed in device offset order (i.e. all of the
1316 extents for one OST first, then the next OST, etc.), not file logical
1317 offset order. If the file logical offset order was used, the Lustre
1318 striping would make the output very verbose and difficult to see if there
1319 was file fragmentation or not.</para>
1321 <para>Note that as long as a file has extent lengths of tens of
1322 megabytes or more (i.e.
1323 <replaceable>read_bandwidth * seek_time >
1324 extent_length</replaceable>), the read performance for the file will
1325 not be significantly impacted by fragmentation, since file readahead
1326 can fully utilize the disk disk bandwidth even with occasional
1329 <para>In default mode
1331 <para>The default mode is faster than the verbose/extent mode.</para>
1333 <literal>filefrag</literal> returns the number of physically discontiguous
1334 extents in the file. In extent or verbose mode, each extent is printed
1335 with details. For a Lustre file system, the extents are printed in device
1336 offset order, not logical offset order.</para>
1338 <section remap="h5">
1339 <title>Options</title>
1340 <para>The options and descriptions for the
1341 <literal>filefrag</literal> utility are listed below.</para>
1342 <informaltable frame="all">
1344 <colspec colname="c1" colwidth="50*" />
1345 <colspec colname="c2" colwidth="50*" />
1350 <emphasis role="bold">Option</emphasis>
1355 <emphasis role="bold">Description</emphasis>
1364 <literal>-b</literal>
1368 <para>Uses the 1024-byte blocksize for the output. By default,
1369 this blocksize is used by the Lustre file system, since OSTs
1370 may use different block sizes.</para>
1376 <literal>-e</literal>
1380 <para>Uses the extent mode when printing the output. This is
1381 the default for Lustre files in verbose mode.</para>
1387 <literal>-l</literal>
1391 <para>Displays extents in LUN offset order. This is the only
1392 available mode for Lustre.</para>
1398 <literal>-s</literal>
1402 <para>Synchronizes any unwritten file data to disk before
1403 requesting the mapping.</para>
1409 <literal>-v</literal>
1413 <para>Prints the file's layout in verbose mode when checking
1414 file fragmentation, including the logical to physical mapping
1415 for each extent in the file and the OST index.</para>
1422 <section remap="h5">
1423 <title>Examples</title>
1424 <para>Lists default output.</para>
1426 $ filefrag /mnt/lustre/foo
1427 /mnt/lustre/foo: 13 extents found
1429 <para>Lists verbose output in extent format.</para>
1431 $ filefrag -v /mnt/lustre/foo
1432 Filesystem type is: bd00bd0
1433 File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes)
1434 ext: device_logical: physical_offset: length: dev: flags:
1435 0: 0.. 122879: 2804679680..2804802559: 122880: 0002: network
1436 1: 122880.. 245759: 2804817920..2804940799: 122880: 0002: network
1437 2: 245760.. 278527: 2804948992..2804981759: 32768: 0002: network
1438 3: 278528.. 360447: 2804982784..2805064703: 81920: 0002: network
1439 4: 360448.. 483327: 2805080064..2805202943: 122880: 0002: network
1440 5: 483328.. 606207: 2805211136..2805334015: 122880: 0002: network
1441 6: 606208.. 729087: 2805342208..2805465087: 122880: 0002: network
1442 7: 729088.. 851967: 2805473280..2805596159: 122880: 0002: network
1443 8: 851968.. 974847: 2805604352..2805727231: 122880: 0002: network
1444 9: 974848.. 1097727: 2805735424..2805858303: 122880: 0002: network
1445 10: 1097728.. 1220607: 2805866496..2805989375: 122880: 0002: network
1446 11: 1220608.. 1343487: 2805997568..2806120447: 122880: 0002: network
1447 12: 1343488.. 1433599: 2806128640..2806218751: 90112: 0002: network
1448 /mnt/lustre/foo: 13 extents found
1452 <section xml:id="dbdoclet.50438206_86244">
1455 <primary>mount</primary>
1457 <literal>mount</literal>
1460 <literal>mount(8)</literal> Linux command is used to mount a Lustre file
1461 system. When mounting a Lustre file system, mount(8) executes the
1462 <literal>/sbin/mount.lustre</literal> command to complete the mount. The
1463 mount command supports these options specific to a Lustre file
1465 <informaltable frame="all">
1467 <colspec colname="c1" colwidth="50*" />
1468 <colspec colname="c2" colwidth="50*" />
1473 <emphasis role="bold">Server options</emphasis>
1478 <emphasis role="bold">Description</emphasis>
1487 <literal>abort_recov</literal>
1491 <para>Aborts recovery when starting a target</para>
1497 <literal>nosvc</literal>
1501 <para>Starts only MGS/MGC servers</para>
1507 <literal>nomgs</literal>
1511 <para>Start a MDT with a co-located MGS without starting the
1518 <literal>exclude</literal>
1522 <para>Starts with a dead OST</para>
1528 <literal>md_stripe_cache_size</literal>
1532 <para>Sets the stripe cache size for server side disk with a
1533 striped raid configuration</para>
1539 <informaltable frame="all">
1541 <colspec colname="c1" colwidth="50*" />
1542 <colspec colname="c2" colwidth="50*" />
1547 <emphasis role="bold">Client options</emphasis>
1552 <emphasis role="bold">Description</emphasis>
1561 <literal>flock/noflock/localflock</literal>
1565 <para>Enables/disables global flock or local flock support</para>
1571 <literal>user_xattr/nouser_xattr</literal>
1575 <para>Enables/disables user-extended attributes</para>
1581 <literal>user_fid2path/nouser_fid2path</literal>
1585 <para condition="l23">Enables/disables FID to path translation by
1586 regular users</para>
1592 <literal>retry=</literal>
1596 <para>Number of times a client will retry to mount the file
1604 <section xml:id="dbdoclet.50438206_56217">
1605 <title>Handling Timeouts</title>
1606 <para>Timeouts are the most common cause of hung applications. After a
1607 timeout involving an MDS or failover OST, applications attempting to access
1608 the disconnected resource wait until the connection gets
1610 <para>When a client performs any remote operation, it gives the server a
1611 reasonable amount of time to respond. If a server does not reply either due
1612 to a down network, hung server, or any other reason, a timeout occurs which
1613 requires a recovery.</para>
1614 <para>If a timeout occurs, a message (similar to this one), appears on the
1615 console of the client, and in
1616 <literal>/var/log/messages</literal>:</para>
1618 LustreError: 26597:(client.c:810:ptlrpc_expire_one_request()) @@@ timeout
1620 req@a2d45200 x5886/t0 o38->mds_svc_UUID@NID_mds_UUID:12 lens 168/64 ref 1 fl