SystemConfigurationUtilities.xml

   1 <?xml version='1.0' encoding='UTF-8'?>
   2 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="systemconfigurationutilities">
   3   <title xml:id="systemconfigurationutilities.title">System Configuration Utilities</title>
   4   <para>This chapter includes system configuration utilities and includes the following sections:</para>
   5   <itemizedlist>
   6     <listitem>
   7       <para><xref linkend="dbdoclet.50438219_55923"/></para>
   8     </listitem>
   9     <listitem>
  10       <para><xref linkend="dbdoclet.50438219_76969"/></para>
  11     </listitem>
  12     <listitem>
  13       <para><xref linkend="dbdoclet.50438219_38274"/></para>
  14     </listitem>
  15     <listitem>
  16       <para><xref linkend="dbdoclet.50438219_58217"/></para>
  17     </listitem>
  18     <listitem>
  19       <para><xref linkend="dbdoclet.50438219_44971"/></para>
  20     </listitem>
  21     <listitem>
  22       <para><xref linkend="dbdoclet.50438219_84890"/></para>
  23     </listitem>
  24     <listitem>
  25       <para><xref linkend="dbdoclet.50438219_90386"/></para>
  26     </listitem>
  27     <listitem>
  28       <para><xref linkend="dbdoclet.50438219_23232"/></para>
  29     </listitem>
  30     <listitem>
  31       <para><xref linkend="dbdoclet.50438219_23648"/></para>
  32     </listitem>
  33     <listitem>
  34       <para><xref linkend="dbdoclet.50438219_64286"/></para>
  35     </listitem>
  36     <listitem>
  37       <para><xref linkend="dbdoclet.50438219_90218"/></para>
  38     </listitem>
  39     <listitem>
  40       <para><xref linkend="dbdoclet.50438219_54734"/></para>
  41     </listitem>
  42     <listitem>
  43       <para><xref linkend="dbdoclet.50438219_63667"/></para>
  44     </listitem>
  45     <listitem>
  46       <para><xref linkend="dbdoclet.50438219_75432"/></para>
  47     </listitem>
  48     <listitem>
  49       <para><xref linkend="dbdoclet.50438219_12635"/></para>
  50     </listitem>
  51     <listitem>
  52       <para><xref linkend="dbdoclet.50438219_82679"/></para>
  53     </listitem>
  54     <listitem>
  55       <para><xref linkend="dbdoclet.50438219_51496"/></para>
  56     </listitem>
  57     <listitem>
  58       <para><xref linkend="dbdoclet.50438219_39574"/></para>
  59     </listitem>
  60     <listitem>
  61       <para><xref linkend="dbdoclet.50438219_99928"/></para>
  62     </listitem>
  63   </itemizedlist>
  64   <section xml:id="dbdoclet.50438219_55923">
  65       <title><indexterm><primary>e2scan</primary></indexterm>
  66                   e2scan</title>
  67     <para>The e2scan utility is an ext2 file system-modified inode scan program. The e2scan program uses libext2fs to find inodes with ctime or mtime newer than a given time and prints out their pathname. Use e2scan to efficiently generate lists of files that have been modified. The e2scan tool is included in the e2fsprogs package, located at:</para>
  68     <para><link xl:href="http://downloads.hpdd.intel.com/public/e2fsprogs/latest/">http://downloads.hpdd.intel.com/public/e2fsprogs/latest/</link></para>
  69     <section remap="h5">
  70       <title>Synopsis</title>
  71       <screen>e2scan [options] [-f file] block_device</screen>
  72     </section>
  73     <section remap="h5">
  74       <title>Description</title>
  75       <para>When invoked, the e2scan utility iterates all inodes on the block device, finds modified inodes, and prints their inode numbers. A similar iterator, using libext2fs(5), builds a table (called parent database) which lists the parent node for each inode. With a lookup function, you can reconstruct modified pathnames from root.</para>
  76     </section>
  77     <section remap="h5">
  78       <title>Options</title>
  79       <informaltable frame="all">
  80         <tgroup cols="2">
  81           <colspec colname="c1" colwidth="50*"/>
  82           <colspec colname="c2" colwidth="50*"/>
  83           <thead>
  84             <row>
  85               <entry>
  86                 <para><emphasis role="bold">Option</emphasis></para>
  87               </entry>
  88               <entry>
  89                 <para><emphasis role="bold">Description</emphasis></para>
  90               </entry>
  91             </row>
  92           </thead>
  93           <tbody>
  94             <row>
  95               <entry>
  96                 <para> <literal>-b <replaceable>inode buffer blocks</replaceable></literal></para>
  97               </entry>
  98               <entry>
  99                 <para> Sets the readahead inode blocks to get excellent performance when scanning the block device.</para>
 100                 <para>&#160;</para>
 101               </entry>
 102             </row>
 103             <row>
 104               <entry>
 105                 <para> <literal>-o <replaceable>output file</replaceable></literal></para>
 106               </entry>
 107               <entry>
 108                 <para> If an output file is specified, modified pathnames are written to this file. Otherwise, modified parameters are written to stdout.</para>
 109               </entry>
 110             </row>
 111             <row>
 112               <entry>
 113                 <para> <literal>-t <replaceable>inode</replaceable>| <replaceable>pathname</replaceable></literal></para>
 114               </entry>
 115               <entry>
 116                 <para> Sets the e2scan type if type is inode. The e2scan utility prints modified inode numbers to stdout. By default, the type is set as pathname.</para>
 117                 <para>The e2scan utility lists modified pathnames based on modified inode numbers.</para>
 118               </entry>
 119             </row>
 120             <row>
 121               <entry>
 122                 <para> <literal>-u</literal></para>
 123               </entry>
 124               <entry>
 125                 <para> Rebuilds the parent database from scratch. Otherwise, the current parent database is used.</para>
 126               </entry>
 127             </row>
 128           </tbody>
 129         </tgroup>
 130       </informaltable>
 131     </section>
 132   </section>
 133   <section xml:id="dbdoclet.50438219_76969">
 134     <title><indexterm><primary>l_getidentity</primary></indexterm>
 135 l_getidentity</title>
 136     <para>The l_getidentity utility handles Lustre user / group cache upcall.</para>
 137     <section remap="h5">
 138       <title>Synopsis</title>
 139       <screen>l_getidentity ${FSNAME}-MDT{xxxx} {uid}</screen>
 140     </section>
 141     <section remap="h5">
 142       <title>Description</title>
 143       <para>The group upcall file contains the path to an executable file that is invoked to resolve
 144         a numeric UID to a group membership list. This utility opens
 145           <literal>/proc/fs/lustre/mdt/${FSNAME}-MDT{xxxx}/identity_info</literal> and writes the
 146         related <literal>identity_downcall_data</literal> structure (see <xref
 147           linkend="dbdoclet.50438291_33759"/>.) The data is persisted with <literal>lctl set_param
 148           mdt.${FSNAME}-MDT{xxxx}.identity_info</literal>.</para>
 149       <para>The l_getidentity utility is the reference implementation of the user or group cache upcall.</para>
 150     </section>
 151     <section remap="h5">
 152       <title>Options</title>
 153       <informaltable frame="all">
 154         <tgroup cols="2">
 155           <colspec colname="c1" colwidth="50*"/>
 156           <colspec colname="c2" colwidth="50*"/>
 157           <thead>
 158             <row>
 159               <entry>
 160                 <para><emphasis role="bold">Option</emphasis></para>
 161               </entry>
 162               <entry>
 163                 <para><emphasis role="bold">Description</emphasis></para>
 164               </entry>
 165             </row>
 166           </thead>
 167           <tbody>
 168             <row>
 169               <entry>
 170                 <para>
 171                   <literal>${FSNAME}-MDT{xxxx}</literal></para>
 172               </entry>
 173               <entry>
 174                 <para> Metadata server target name</para>
 175               </entry>
 176             </row>
 177             <row>
 178               <entry>
 179                 <para> <literal>uid</literal></para>
 180               </entry>
 181               <entry>
 182                 <para> User identifier</para>
 183               </entry>
 184             </row>
 185           </tbody>
 186         </tgroup>
 187       </informaltable>
 188     </section>
 189     <section remap="h5">
 190       <title>Files</title>
 191       <para>The l_getidentity files are located at:</para>
 192       <screen>/proc/fs/lustre/mdt/${FSNAME}-MDT{xxxx}/identity_upcall</screen>
 193     </section>
 194   </section>
 195   <section xml:id="dbdoclet.50438219_38274">
 196     <title><indexterm><primary>lctl</primary></indexterm>
 197 lctl</title>
 198     <para>The lctl utility is used for root control and configuration. With lctl you can directly control Lustre via an ioctl interface, allowing various configuration, maintenance and debugging features to be accessed.</para>
 199     <section remap="h5">
 200       <title>Synopsis</title>
 201       <screen>lctl [--device <replaceable>devno</replaceable>] <replaceable>command [args]</replaceable></screen>
 202     </section>
 203     <section remap="h5">
 204       <title>Description</title>
 205       <para>The lctl utility can be invoked in interactive mode by issuing the lctl command. After that, commands are issued as shown below. The most common lctl commands are:</para>
 206       <screen>dl
 207 dk
 208 device
 209 network <replaceable>up|down</replaceable>
 210 list_nids
 211 ping <replaceable>nid</replaceable>help
 212 quit</screen>
 213       <para>For a complete list of available commands, type <literal>help</literal> at the <literal>lctl</literal> prompt. To get basic help on command meaning and syntax, type <literal>help <replaceable>command</replaceable></literal>. Command completion is activated with the TAB key (depending on compile options), and command history is available via the up- and down-arrow keys.</para>
 214       <para>For non-interactive use, use the second invocation, which runs the command after connecting to the device.</para>
 215     </section>
 216     <section remap="h5">
 217       <title>Setting Parameters with lctl</title>
 218       <para>Lustre parameters are not always accessible using the procfs interface, as it is platform-specific. As a solution, lctl {get,set}_param has been introduced as a platform-independent interface to the Lustre tunables. Avoid direct references to /proc/{fs,sys}/{lustre,lnet}. For future portability, use lctl {get,set}_param .</para>
 219       <para>When the file system is running, use the <literal>lctl set_param</literal> command on the affected node(s) to <emphasis>temporarily</emphasis> set parameters (mapping to items in /proc/{fs,sys}/{lnet,lustre}). The <literal>lctl set_param</literal> command uses this syntax:</para>
 220       <screen>lctl set_param [-n] [-P] [-d] <replaceable>obdtype.obdname.property</replaceable>=<replaceable>value</replaceable></screen>
 221       <para>For example:</para>
 222       <screen>mds# lctl set_param mdt.testfs-MDT0000.identity_upcall=NONE</screen>
 223       <para condition='l25'>Use <literal>-P</literal> option to set parameters permanently. Option <literal>-d </literal>deletes permanent parameters. For example:
 224               <screen>mgs# lctl set_param -P mdt.testfs-MDT0000.identity_upcall=NONE
 225 mgs# lctl set_param -P -d mdt.testfs-MDT0000.identity_upcall</screen></para>
 226       <para>Many permanent parameters can be set with <literal>lctl conf_param</literal>. In general, <literal>lctl conf_param</literal> can be used to specify any OBD device parameter settable in a /proc/fs/lustre file. The <literal>lctl conf_param</literal> command must be run on the MGS node, and uses this syntax:</para>
 227       <screen><replaceable>obd|fsname</replaceable>.obdtype.property=<replaceable>value</replaceable>) </screen>
 228       <para>For example:</para>
 229       <screen>mgs# lctl conf_param testfs-MDT0000.mdt.identity_upcall=NONE
 230 $ lctl conf_param testfs.llite.max_read_ahead_mb=16 </screen>
 231       <caution>
 232         <para>The <literal>lctl conf_param</literal> command <emphasis>permanently</emphasis> sets parameters in the file system configuration for all nodes of the specified type.</para>
 233       </caution>
 234       <para>To get current Lustre parameter settings, use the <literal>lctl get_param</literal> command on the desired node with the same parameter name as <literal>lctl set_param</literal>:</para>
 235       <screen>lctl get_param [-n] <replaceable>obdtype.obdname.parameter</replaceable></screen>
 236       <para>For example:</para>
 237       <screen>mds# lctl get_param mdt.testfs-MDT0000.identity_upcall</screen>
 238       <para>To list Lustre parameters that are available to set, use the <literal>lctl list_param</literal> command, with this syntax:</para>
 239       <screen>lctl list_param [-R] [-F] <replaceable>obdtype.obdname.*</replaceable></screen>
 240       <para>For example, to list all of the parameters on the MDT:</para>
 241       <screen>oss# lctl list_param -RF mdt</screen>
 242       <para>For more information on using lctl to set temporary and permanent parameters, see <xref linkend="dbdoclet.50438194_51490"/>.</para>
 243       <para><emphasis role="bold">Network Configuration</emphasis></para>
 244       <informaltable frame="all">
 245         <tgroup cols="2">
 246           <colspec colname="c1" colwidth="50*"/>
 247           <colspec colname="c2" colwidth="50*"/>
 248           <thead>
 249             <row>
 250               <entry>
 251                 <para><emphasis role="bold">Option</emphasis></para>
 252               </entry>
 253               <entry>
 254                 <para><emphasis role="bold">Description</emphasis></para>
 255               </entry>
 256             </row>
 257           </thead>
 258           <tbody>
 259             <row>
 260               <entry>
 261                 <para> <literal>network up|down|tcp|elan|myrinet</literal></para>
 262               </entry>
 263               <entry>
 264                 <para> Starts or stops LNet, or selects a network type for other <literal>lctl</literal> LNet commands.</para>
 265               </entry>
 266             </row>
 267             <row>
 268               <entry>
 269                 <para> <literal>list_nids</literal></para>
 270               </entry>
 271               <entry>
 272                 <para> Prints all NIDs on the local node. LNet must be running.</para>
 273               </entry>
 274             </row>
 275             <row>
 276               <entry>
 277                 <para> <literal>which_nid <replaceable>nidlist</replaceable></literal></para>
 278               </entry>
 279               <entry>
 280                 <para> From a list of NIDs for a remote node, identifies the NID on which interface communication will occur.</para>
 281               </entry>
 282             </row>
 283             <row>
 284               <entry>
 285                 <para> <literal>ping <replaceable>nid</replaceable></literal></para>
 286               </entry>
 287               <entry>
 288                 <para> Checks LNet connectivity via an LNet ping. This uses the fabric appropriate to the specified NID.</para>
 289               </entry>
 290             </row>
 291             <row>
 292               <entry>
 293                 <para> <literal>interface_list</literal></para>
 294               </entry>
 295               <entry>
 296                 <para> Prints the network interface information for a given <emphasis>network</emphasis> type.</para>
 297               </entry>
 298             </row>
 299             <row>
 300               <entry>
 301                 <para> <literal>peer_list</literal></para>
 302               </entry>
 303               <entry>
 304                 <para> Prints the known peers for a given <emphasis>network</emphasis> type.</para>
 305               </entry>
 306             </row>
 307             <row>
 308               <entry>
 309                 <para> <literal>conn_list</literal></para>
 310               </entry>
 311               <entry>
 312                 <para> Prints all the connected remote NIDs for a given <emphasis>network</emphasis> type.</para>
 313               </entry>
 314             </row>
 315             <row>
 316               <entry>
 317                 <para> <literal>active_tx</literal></para>
 318               </entry>
 319               <entry>
 320                 <para> This command prints active transmits. It is only used for the Elan <emphasis>network</emphasis> type.</para>
 321               </entry>
 322             </row>
 323             <row>
 324               <entry>
 325                 <para> <literal>route_list</literal></para>
 326               </entry>
 327               <entry>
 328                 <para> Prints the complete routing table.</para>
 329               </entry>
 330             </row>
 331           </tbody>
 332         </tgroup>
 333       </informaltable>
 334       <para><emphasis role="bold">Device Selection</emphasis></para>
 335       <informaltable frame="all">
 336         <tgroup cols="3">
 337           <colspec colname="c1" colwidth="33*"/>
 338           <colspec colname="c2" colwidth="33*"/>
 339           <colspec colname="c3" colwidth="33*"/>
 340           <thead>
 341             <row>
 342               <entry>
 343                 <para><emphasis role="bold">Option</emphasis></para>
 344               </entry>
 345               <entry>
 346                 <para><emphasis role="bold">&#160;</emphasis></para>
 347               </entry>
 348               <entry>
 349                 <para><emphasis role="bold">Description</emphasis></para>
 350               </entry>
 351             </row>
 352           </thead>
 353           <tbody>
 354             <row>
 355               <entry>
 356                 <para> <literal>device <replaceable>devname</replaceable></literal></para>
 357               </entry>
 358               <entry>
 359                 <para> &#160;</para>
 360               </entry>
 361               <entry>
 362                 <para> This selects the specified OBD device. All other commands depend on the device being set.</para>
 363               </entry>
 364             </row>
 365             <row>
 366               <entry>
 367                 <para> <literal>device_list</literal></para>
 368               </entry>
 369               <entry>
 370                 <para> &#160;</para>
 371               </entry>
 372               <entry>
 373                 <para> Shows the local Lustre OBDs, a/k/a <literal>dl</literal>.</para>
 374               </entry>
 375             </row>
 376           </tbody>
 377         </tgroup>
 378       </informaltable>
 379       <para><emphasis role="bold">Device Operations</emphasis></para>
 380       <informaltable frame="all">
 381         <tgroup cols="3">
 382           <colspec colname="c1" colwidth="33*"/>
 383           <colspec colname="c2" colwidth="33*"/>
 384           <colspec colname="c3" colwidth="33*"/>
 385           <thead>
 386             <row>
 387               <entry nameend="c2" namest="c1">
 388                 <para><emphasis role="bold">Option</emphasis></para>
 389               </entry>
 390               <entry>
 391                 <para><emphasis role="bold">Description</emphasis></para>
 392               </entry>
 393             </row>
 394           </thead>
 395           <tbody>
 396             <row>
 397               <entry nameend="c2" namest="c1">
 398                 <para> <literal>list_param [-F|-R] <replaceable>parameter</replaceable> <replaceable>[parameter ...]</replaceable></literal></para>
 399               </entry>
 400               <entry>
 401                 <para> Lists the Lustre or LNet parameter name.</para>
 402                 <para>&#160;</para>
 403               </entry>
 404             </row>
 405             <row>
 406               <entry>
 407                 <para> &#160;</para>
 408               </entry>
 409               <entry>
 410                 <para> <literal>-F</literal></para>
 411               </entry>
 412               <entry>
 413                 <para> Adds &apos;/&apos;, &apos;@&apos; or &apos;=&apos; for directories, symlinks and writeable files, respectively.</para>
 414               </entry>
 415             </row>
 416             <row>
 417               <entry>
 418                 <para> &#160;</para>
 419               </entry>
 420               <entry>
 421                 <para> <literal>-R</literal></para>
 422               </entry>
 423               <entry>
 424                 <para> Recursively lists all parameters under the specified path. If <literal>param_path</literal> is unspecified, all parameters are shown.</para>
 425               </entry>
 426             </row>
 427             <row>
 428               <entry nameend="c2" namest="c1">
 429                 <para> <literal>get_param [-n|-N|-F] <replaceable>parameter</replaceable> <replaceable>[parameter ...]</replaceable></literal></para>
 430               </entry>
 431               <entry>
 432                 <para> Gets the value of a Lustre or LNet parameter from the specified path.</para>
 433               </entry>
 434             </row>
 435             <row>
 436               <entry>
 437                 <para> &#160;</para>
 438               </entry>
 439               <entry>
 440                 <para> <literal>-n</literal></para>
 441               </entry>
 442               <entry>
 443                 <para> Prints only the parameter value and not the parameter name.</para>
 444               </entry>
 445             </row>
 446             <row>
 447               <entry>
 448                 <para> &#160;</para>
 449               </entry>
 450               <entry>
 451                 <para> <literal>-N</literal></para>
 452               </entry>
 453               <entry>
 454                 <para> Prints only matched parameter names and not the values; especially useful when using patterns.</para>
 455               </entry>
 456             </row>
 457             <row>
 458               <entry>
 459                 <para> &#160;</para>
 460               </entry>
 461               <entry>
 462                 <para> <literal>-F</literal></para>
 463               </entry>
 464               <entry>
 465                 <para> When <literal>-N</literal> is specified, adds &apos;/&apos;, &apos;@&apos; or &apos;=&apos; for directories, symlinks and writeable files, respectively.</para>
 466               </entry>
 467             </row>
 468             <row>
 469               <entry nameend="c2" namest="c1">
 470                 <para> <literal>set_param [-n] <replaceable>parameter</replaceable>=<replaceable>value</replaceable></literal></para>
 471               </entry>
 472               <entry>
 473                 <para> Sets the value of a Lustre or LNet parameter from the specified path.</para>
 474               </entry>
 475             </row>
 476             <row>
 477               <entry>
 478                 <para> &#160;</para>
 479               </entry>
 480               <entry>
 481                 <para> <literal>-n</literal></para>
 482               </entry>
 483               <entry>
 484                 <para> Disables printing of the key name when printing values.</para>
 485               </entry>
 486             </row>
 487             <row>
 488               <entry nameend="c2" namest="c1">
 489                 <para><literal>conf_param [-d] <replaceable>device|fsname</replaceable> <replaceable>parameter</replaceable>=<replaceable>value</replaceable></literal></para>
 490               </entry>
 491               <entry>
 492                 <para> Sets a permanent configuration parameter for any device via the MGS. This command must be run on the MGS node.</para>
 493                 <para>All writeable parameters under <literal>lctl list_param</literal> (e.g. <literal>lctl list_param -F osc.*.* | grep</literal> =) can be permanently set using <literal>lctl conf_param</literal>, but the format is slightly different. For <literal>conf_param</literal>, the device is specified first, then the obdtype. Wildcards are not supported. Additionally, failover nodes may be added (or removed), and some system-wide parameters may be set as well (sys.at_max, sys.at_min, sys.at_extra, sys.at_early_margin, sys.at_history, sys.timeout, sys.ldlm_timeout). For system-wide parameters, <replaceable>device</replaceable> is ignored.</para>
 494                 <para>For more information on setting permanent parameters and <literal>lctl conf_param</literal> command examples, see <xref linkend="dbdoclet.50438194_64195"/> (Setting Permanent Parameters).</para>
 495               </entry>
 496             </row>
 497             <row>
 498               <entry>
 499                 <para> &#160;</para>
 500               </entry>
 501               <entry>
 502                 <para><literal>-d <replaceable>device|fsname</replaceable>.<replaceable>parameter</replaceable></literal></para>
 503                 <para>&#160;</para>
 504               </entry>
 505               <entry>
 506                 <para> Deletes a parameter setting (use the default value at the next restart). A null value for <replaceable>value</replaceable> also deletes the parameter setting.</para>
 507               </entry>
 508             </row>
 509             <row>
 510               <entry nameend="c2" namest="c1">
 511                 <para> <literal>activate</literal></para>
 512               </entry>
 513               <entry>
 514                 <para> Re-activates an import after the deactivate operation. This setting is only effective until the next restart (see <literal>conf_param</literal>).</para>
 515               </entry>
 516             </row>
 517             <row>
 518               <entry nameend="c2" namest="c1">
 519                 <para> <literal>deactivate</literal></para>
 520               </entry>
 521               <entry>
 522                 <para> Deactivates an import, in particular meaning do not assign new file stripes to an OSC. Running lctl deactivate on the MDS stops new objects from being allocated on the OST. Running lctl deactivate on Lustre clients causes them to return -EIO when accessing objects on the OST instead of waiting for recovery.</para>
 523               </entry>
 524             </row>
 525             <row>
 526               <entry nameend="c2" namest="c1">
 527                 <para> <literal>abort_recovery</literal></para>
 528               </entry>
 529               <entry>
 530                 <para> Aborts the recovery process on a re-starting MDT or OST.</para>
 531               </entry>
 532             </row>
 533           </tbody>
 534         </tgroup>
 535       </informaltable>
 536       <note>
 537         <para>Lustre tunables are not always accessible using the procfs interface, as it is platform-specific. As a solution, <literal>lctl {get,set,list}_param</literal> has been introduced as a platform-independent interface to the Lustre tunables. Avoid direct references to <literal>/proc/{fs,sys}/{lustre,lnet}</literal>. For future portability, use <literal>lctl {get,set,list}_param</literal> instead.</para>
 538       </note>
 539       <para><emphasis role="bold">Virtual Block Device Operations</emphasis></para>
 540       <para>Lustre can emulate a virtual block device upon a regular file. This emulation is needed when you are trying to set up a swap space via the file.</para>
 541       <informaltable frame="all">
 542         <tgroup cols="2">
 543           <colspec colname="c1" colwidth="50*"/>
 544           <colspec colname="c2" colwidth="50*"/>
 545           <thead>
 546             <row>
 547               <entry>
 548                 <para><emphasis role="bold">Option</emphasis></para>
 549               </entry>
 550               <entry>
 551                 <para><emphasis role="bold">Description</emphasis></para>
 552               </entry>
 553             </row>
 554           </thead>
 555           <tbody>
 556             <row>
 557               <entry>
 558                 <para><literal>blockdev_attach <replaceable>filename</replaceable> <replaceable>/dev/lloop_device</replaceable></literal></para>
 559               </entry>
 560               <entry>
 561                 <para> Attaches a regular Lustre file to a block device. If the device node does not exist, <literal>lctl</literal> creates it. It is recommend that a device node is created by <literal>lctl</literal> since the emulator uses a dynamical major number.</para>
 562               </entry>
 563             </row>
 564             <row>
 565               <entry>
 566                 <para><literal>blockdev_detach <replaceable>/dev/lloop_device</replaceable></literal></para>
 567               </entry>
 568               <entry>
 569                 <para> Detaches the virtual block device.</para>
 570               </entry>
 571             </row>
 572             <row>
 573               <entry>
 574                 <para><literal>blockdev_info <replaceable>/dev/lloop_device</replaceable></literal></para>
 575               </entry>
 576               <entry>
 577                 <para> Provides information about the Lustre file attached to the device node.</para>
 578               </entry>
 579             </row>
 580           </tbody>
 581         </tgroup>
 582       </informaltable>
 583       <para><emphasis role="bold">Changelogs</emphasis></para>
 584       <informaltable frame="all">
 585         <tgroup cols="2">
 586           <colspec colname="c1" colwidth="50*"/>
 587           <colspec colname="c2" colwidth="50*"/>
 588           <thead>
 589             <row>
 590               <entry>
 591                 <para><emphasis role="bold">Option</emphasis></para>
 592               </entry>
 593               <entry>
 594                 <para><emphasis role="bold">Description</emphasis></para>
 595               </entry>
 596             </row>
 597           </thead>
 598           <tbody>
 599             <row>
 600               <entry>
 601                 <para> <literal>changelog_register</literal></para>
 602               </entry>
 603               <entry>
 604                 <para> Registers a new changelog user for a particular device.
 605                 Changelog entries are saved persistently on the MDT with each
 606                 filesystem operation, and are only purged beyond all registered
 607                 user&apos;s minimum set point (see
 608                 <literal>lfs changelog_clear</literal>). This may cause the
 609                 Changelog to consume a large amount of space, eventually
 610                 filling the MDT, if a changelog user is registered but never
 611                 consumes those records.</para>
 612               </entry>
 613             </row>
 614             <row>
 615               <entry>
 616                 <para>changelog_deregister <replaceable>id</replaceable></para>
 617               </entry>
 618               <entry>
 619                 <para> Unregisters an existing changelog user. If the
 620                 user&apos;s &quot;clear&quot; record number is the minimum for
 621                 the device, changelog records are purged until the next minimum.
 622                 </para>
 623               </entry>
 624             </row>
 625           </tbody>
 626         </tgroup>
 627       </informaltable>
 628       <para><emphasis role="bold">Debug</emphasis></para>
 629       <informaltable frame="all">
 630         <tgroup cols="2">
 631           <colspec colname="c1" colwidth="50*"/>
 632           <colspec colname="c2" colwidth="50*"/>
 633           <thead>
 634             <row>
 635               <entry>
 636                 <para><emphasis role="bold">Option</emphasis></para>
 637               </entry>
 638               <entry>
 639                 <para><emphasis role="bold">Description</emphasis></para>
 640               </entry>
 641             </row>
 642           </thead>
 643           <tbody>
 644             <row>
 645               <entry>
 646                 <para> <literal>debug_daemon</literal></para>
 647               </entry>
 648               <entry>
 649                 <para> Starts and stops the debug daemon, and controls the output filename and size.</para>
 650               </entry>
 651             </row>
 652             <row>
 653               <entry>
 654                 <para> <literal>debug_kernel <replaceable>[file]</replaceable> [raw]</literal></para>
 655               </entry>
 656               <entry>
 657                 <para> Dumps the kernel debug buffer to stdout or a file.</para>
 658               </entry>
 659             </row>
 660             <row>
 661               <entry>
 662                 <para><literal>debug_file <replaceable>input_file</replaceable> <replaceable>[output_file]</replaceable></literal></para>
 663               </entry>
 664               <entry>
 665                 <para> Converts the kernel-dumped debug log from binary to plain text format.</para>
 666               </entry>
 667             </row>
 668             <row>
 669               <entry>
 670                 <para> <literal>clear</literal></para>
 671               </entry>
 672               <entry>
 673                 <para> Clears the kernel debug buffer.</para>
 674               </entry>
 675             </row>
 676             <row>
 677               <entry>
 678                 <para> <literal>mark <replaceable>text</replaceable></literal></para>
 679               </entry>
 680               <entry>
 681                 <para> Inserts marker text in the kernel debug buffer.</para>
 682               </entry>
 683             </row>
 684             <row>
 685               <entry>
 686                 <para> <literal>filter <replaceable>subsystem_id|debug_mask</replaceable></literal></para>
 687               </entry>
 688               <entry>
 689                 <para> Filters kernel debug messages by subsystem or mask.</para>
 690               </entry>
 691             </row>
 692             <row>
 693               <entry>
 694                 <para> <literal>show <replaceable>subsystem_id|debug_mask</replaceable></literal></para>
 695               </entry>
 696               <entry>
 697                 <para> Shows specific types of messages.</para>
 698               </entry>
 699             </row>
 700             <row>
 701               <entry>
 702                 <para> <literal>debug_list <replaceable>subsystems|types</replaceable></literal></para>
 703               </entry>
 704               <entry>
 705                 <para> Lists all subsystem and debug types.</para>
 706               </entry>
 707             </row>
 708             <row>
 709               <entry>
 710                 <para> <literal>modules <replaceable>path</replaceable></literal></para>
 711               </entry>
 712               <entry>
 713                 <para> Provides GDB-friendly module information.</para>
 714               </entry>
 715             </row>
 716           </tbody>
 717         </tgroup>
 718       </informaltable>
 719     </section>
 720     <section remap="h5">
 721       <title>Options</title>
 722       <para>Use the following options to invoke lctl.</para>
 723       <informaltable frame="all">
 724         <tgroup cols="2">
 725           <colspec colname="c1" colwidth="50*"/>
 726           <colspec colname="c2" colwidth="50*"/>
 727           <thead>
 728             <row>
 729               <entry>
 730                 <para><emphasis role="bold">Option</emphasis></para>
 731               </entry>
 732               <entry>
 733                 <para><emphasis role="bold">Description</emphasis></para>
 734               </entry>
 735             </row>
 736           </thead>
 737           <tbody>
 738             <row>
 739               <entry>
 740                 <para> <literal>--device</literal></para>
 741               </entry>
 742               <entry>
 743                 <para> Device to be used for the operation (specified by name or number). See device_list.</para>
 744               </entry>
 745             </row>
 746             <row>
 747               <entry>
 748                 <para> <literal>--ignore_errors | ignore_errors</literal></para>
 749               </entry>
 750               <entry>
 751                 <para> Ignores errors during script processing.</para>
 752               </entry>
 753             </row>
 754           </tbody>
 755         </tgroup>
 756       </informaltable>
 757     </section>
 758     <section remap="h5">
 759       <title>Examples</title>
 760       <para><literal>lctl</literal></para>
 761       <screen>$ lctl
 762 lctl &gt; dl
 763    0 UP mgc MGC192.168.0.20@tcp btbb24e3-7deb-2ffa-eab0-44dffe00f692 5
 764    1 UP ost OSS OSS_uuid 3
 765    2 UP obdfilter testfs-OST0000 testfs-OST0000_UUID 3
 766 lctl &gt; dk /tmp/log Debug log: 87 lines, 87 kept, 0 dropped.
 767 lctl &gt; quit</screen>
 768     </section>
 769     <section remap="h5">
 770       <title>See Also</title>
 771       <itemizedlist>
 772         <listitem>
 773           <para> <xref linkend="dbdoclet.50438219_75432"/> </para>
 774         </listitem>
 775         <listitem>
 776           <para> <xref linkend="dbdoclet.50438219_12635"/> </para>
 777         </listitem>
 778         <listitem>
 779           <para> <xref linkend="dbdoclet.50438219_38274"/> </para>
 780         </listitem>
 781         <listitem>
 782           <para> <xref linkend="dbdoclet.50438206_94597"/> </para>
 783         </listitem>
 784       </itemizedlist>
 785     </section>
 786   </section>
 787   <section xml:id="dbdoclet.50438219_58217">
 788     <title><indexterm><primary>ll_decode_filter_fid</primary></indexterm>
 789 ll_decode_filter_fid</title>
 790     <para>The ll_decode_filter_fid utility displays the Lustre object ID and MDT parent FID.</para>
 791     <section remap="h5">
 792       <title>Synopsis</title>
 793       <screen>ll_decode_filter_fid object_file [object_file ...]</screen>
 794     </section>
 795     <section remap="h5">
 796       <title>Description</title>
 797       <para>The ll_decode_filter_fid utility decodes and prints the Lustre OST object ID, MDT FID,
 798         stripe index for the specified OST object(s), which is stored in the &quot;trusted.fid&quot;
 799         attribute on each OST object. This is accessible to <literal>ll_decode_filter_fid</literal>
 800         when the OST file system is mounted locally as type ldiskfs for maintenance.</para>
 801       <para>The &quot;trusted.fid&quot; extended attribute is stored on each OST object when it is first modified (data written or attributes set), and is not accessed or modified by Lustre after that time.</para>
 802       <para>The OST object ID (objid) is useful in case of OST directory corruption, though normally the ll_recover_lost_found_objs(8) utility is able to reconstruct the entire OST object directory hierarchy. The MDS FID can be useful to determine which MDS inode an OST object is (or was) used by. The stripe index can be used in conjunction with other OST objects to reconstruct the layout of a file even if the MDT inode was lost.</para>
 803     </section>
 804     <section remap="h5">
 805       <title>Examples</title>
 806       <screen>root@oss1# cd /mnt/ost/lost+found
 807 root@oss1# ll_decode_filter_fid #12345[4,5,8]
 808 #123454: objid=690670 seq=0 parent=[0x751c5:0xfce6e605:0x0]
 809 #123455: objid=614725 seq=0 parent=[0x18d11:0xebba84eb:0x1]
 810 #123458: objid=533088 seq=0 parent=[0x21417:0x19734d61:0x0]</screen>
 811       <para>This shows that the three files in lost+found have decimal object IDs - 690670, 614725, and 533088, respectively. The object sequence number (formerly object group) is 0 for all current OST objects.</para>
 812       <para>The MDT parent inode FIDs are hexadecimal numbers of the form sequence:oid:idx. Since the sequence number is below 0x100000000 in all these cases, the FIDs are in the legacy Inode and Generation In FID (IGIF) namespace and are mapped directly to the MDT inode = seq and generation = oid values; the MDT inodes are 0x751c5, 0x18d11, and 0x21417 respectively. For objects with MDT parent sequence numbers above 0x200000000, this indicates that the FID needs to be mapped via the MDT Object Index (OI) file on the MDT to determine the internal inode number.</para>
 813       <para>The idx field shows the stripe number of this OST object in the Lustre RAID-0 striped file.</para>
 814     </section>
 815     <section remap="h5">
 816       <title>See Also</title>
 817       <para><xref linkend="dbdoclet.50438219_44971"/></para>
 818     </section>
 819   </section>
 820   <section xml:id="dbdoclet.50438219_44971">
 821     <title><indexterm><primary>ll_recover_lost_found_objs</primary></indexterm>
 822 ll_recover_lost_found_objs</title>
 823     <para>The ll_recover_lost_found_objs utility helps recover Lustre OST objects (file data) from a lost and found directory and return them to their correct locations.</para>
 824     <note>
 825       <para>Running the ll_recover_lost_found_objs tool is not strictly necessary to bring an OST back online, it just avoids losing access to objects that were moved to the lost and found directory due to directory corruption.</para>
 826     </note>
 827     <section remap="h5">
 828       <title>Synopsis</title>
 829       <screen>$ ll_recover_lost_found_objs [-hv] -d directory</screen>
 830     </section>
 831     <section remap="h5">
 832       <title>Description</title>
 833       <para>The first time Lustre modifies an object, it saves the MDS inode number and the objid as an extended attribute on the object, so in case of directory corruption of the OST, it is possible to recover the objects. Running e2fsck fixes the corrupted OST directory, but it puts all of the objects into a lost and found directory, where they are inaccessible to Lustre. Use the ll_recover_lost_found_objs utility to recover all (or at least most) objects from a lost and found directory and return them to the O/0/d* directories.</para>
 834       <para>To use ll_recover_lost_found_objs, mount the file system locally (using the <literal>-t ldiskfs</literal>, or <literal>-t zfs</literal> command), run the utility and then unmount it again. The OST must not be mounted by Lustre when ll_recover_lost_found_objs is run.</para>
 835       <para condition="l26">This utility is not needed for 2.6 and later,
 836       since the <literal>LFSCK</literal> online scanning will move objects
 837       from <literal>lost+found</literal> to the proper place in the OST.</para>
 838     </section>
 839     <section remap="h5">
 840       <title>Options</title>
 841       <informaltable frame="all">
 842         <tgroup cols="2">
 843           <colspec colname="c1" colwidth="50*"/>
 844           <colspec colname="c2" colwidth="50*"/>
 845           <thead>
 846             <row>
 847               <entry>
 848                 <para><emphasis role="bold">Option</emphasis></para>
 849               </entry>
 850               <entry>
 851                 <para><emphasis role="bold">Description</emphasis></para>
 852               </entry>
 853             </row>
 854           </thead>
 855           <tbody>
 856             <row>
 857               <entry>
 858                 <para> <literal>-h</literal></para>
 859               </entry>
 860               <entry>
 861                 <para> Prints a help message</para>
 862               </entry>
 863             </row>
 864             <row>
 865               <entry>
 866                 <para> <literal>-v</literal></para>
 867               </entry>
 868               <entry>
 869                 <para> Increases verbosity</para>
 870               </entry>
 871             </row>
 872             <row>
 873               <entry>
 874                 <para> <literal>-d <replaceable>directory</replaceable></literal></para>
 875               </entry>
 876               <entry>
 877                 <para> Sets the lost and found directory path</para>
 878               </entry>
 879             </row>
 880           </tbody>
 881         </tgroup>
 882       </informaltable>
 883     </section>
 884     <section remap="h5">
 885       <title>Example</title>
 886       <screen>ll_recover_lost_found_objs -d /mnt/ost/lost+found </screen>
 887     </section>
 888   </section>
 889   <section xml:id="dbdoclet.50438219_84890">
 890     <title><indexterm><primary>llodbstat</primary></indexterm>
 891 llobdstat</title>
 892     <para>The llobdstat utility displays OST statistics.</para>
 893     <section remap="h5">
 894       <title>Synopsis</title>
 895       <screen>llobdstat ost_name [interval]</screen>
 896     </section>
 897     <section remap="h5">
 898       <title>Description</title>
 899       <para>The llobdstat utility displays a line of OST statistics for the given ost_name every interval seconds. It should be run directly on an OSS node. Type <literal>CTRL-C</literal> to stop statistics printing.</para>
 900     </section>
 901     <section remap="h5">
 902       <title>Example</title>
 903       <screen># llobdstat liane-OST0002 1
 904 /usr/bin/llobdstat on /proc/fs/lustre/obdfilter/liane-OST0002/stats
 905 Processor counters run at 2800.189 MHz
 906 Read: 1.21431e+07, Write: 9.93363e+08, create/destroy: 24/1499, stat: 34, p\
 907 unch: 18
 908 [NOTE: cx: create, dx: destroy, st: statfs, pu: punch ]
 909 Timestamp Read-delta ReadRate Write-delta WriteRate
 910 --------------------------------------------------------
 911 1217026053 0.00MB 0.00MB/s 0.00MB 0.00MB/s
 912 1217026054 0.00MB 0.00MB/s 0.00MB 0.00MB/s
 913 1217026055 0.00MB 0.00MB/s 0.00MB 0.00MB/s
 914 1217026056 0.00MB 0.00MB/s 0.00MB 0.00MB/s
 915 1217026057 0.00MB 0.00MB/s 0.00MB 0.00MB/s
 916 1217026058 0.00MB 0.00MB/s 0.00MB 0.00MB/s
 917 1217026059 0.00MB 0.00MB/s 0.00MB 0.00MB/s st:1</screen>
 918     </section>
 919     <section remap="h5">
 920       <title>Files</title>
 921       <screen>/proc/fs/lustre/obdfilter/<replaceable>ostname</replaceable>/stats</screen>
 922     </section>
 923   </section>
 924   <section xml:id="dbdoclet.50438219_90386">
 925     <title><indexterm><primary>llog_reader</primary></indexterm>
 926 llog_reader</title>
 927     <para>The llog_reader utility translates a Lustre configuration log into human-readable form.</para>
 928     <section remap="h5">
 929       <title>Synopsis</title>
 930       <screen>llog_reader filename</screen>
 931     </section>
 932     <section remap="h5">
 933       <title>Description</title>
 934       <para>The llog_reader utility parses the binary format of Lustre&apos;s on-disk configuration logs. Llog_reader can only read logs; use tunefs.lustre to write to them.</para>
 935       <para>To examine a log file on a stopped Lustre server, mount its backing file system as ldiskfs or zfs, then use llog_reader to dump the log file&apos;s contents, for example:</para>
 936       <screen>mount -t ldiskfs /dev/sda /mnt/mgs
 937 llog_reader /mnt/mgs/CONFIGS/tfs-client</screen>
 938       <para>To examine the same log file on a running Lustre server, use the ldiskfs-enabled debugfs utility (called debug.ldiskfs on some distributions) to extract the file, for example:</para>
 939       <screen>debugfs -c -R &apos;dump CONFIGS/tfs-client /tmp/tfs-client&apos; /dev/sda
 940 llog_reader /tmp/tfs-client</screen>
 941       <caution>
 942         <para>Although they are stored in the CONFIGS directory, mountdata files do not use the configuration log format and will confuse the llog_reader utility.</para>
 943       </caution>
 944     </section>
 945     <section remap="h5">
 946       <title>See Also</title>
 947       <para><xref linkend="dbdoclet.50438219_39574"/></para>
 948     </section>
 949   </section>
 950   <section xml:id="dbdoclet.50438219_23232">
 951     <title><indexterm><primary>llstat</primary></indexterm>
 952 llstat</title>
 953     <para>The llstat utility displays Lustre statistics.</para>
 954     <section remap="h5">
 955       <title>Synopsis</title>
 956       <screen>llstat [-c] [-g] [-i <replaceable>interval</replaceable>] <replaceable>stats_file
 957 </replaceable></screen>
 958     </section>
 959     <section remap="h5">
 960       <title>Description</title>
 961       <para>The llstat utility displays statistics from any of the Lustre statistics files that share a common format and are updated at <literal>interval</literal> seconds. To stop statistics printing, use <literal>ctrl</literal>-<literal>c.</literal></para>
 962     </section>
 963     <section remap="h5">
 964       <title>Options</title>
 965       <informaltable frame="all">
 966         <tgroup cols="2">
 967           <colspec colname="c1" colwidth="50*"/>
 968           <colspec colname="c2" colwidth="50*"/>
 969           <thead>
 970             <row>
 971               <entry>
 972                 <para><emphasis role="bold">Option</emphasis></para>
 973               </entry>
 974               <entry>
 975                 <para><emphasis role="bold">Description</emphasis></para>
 976               </entry>
 977             </row>
 978           </thead>
 979           <tbody>
 980             <row>
 981               <entry>
 982                 <para> <literal>-c</literal></para>
 983               </entry>
 984               <entry>
 985                 <para> Clears the statistics file.</para>
 986               </entry>
 987             </row>
 988             <row>
 989               <entry>
 990                 <para> <literal>-i</literal></para>
 991               </entry>
 992               <entry>
 993                 <para> Specifies the polling period (in seconds).</para>
 994               </entry>
 995             </row>
 996             <row>
 997               <entry>
 998                 <para> <literal>-g</literal></para>
 999               </entry>
1000               <entry>
1001                 <para> Specifies graphable output format.</para>
1002               </entry>
1003             </row>
1004             <row>
1005               <entry>
1006                 <para> <literal>-h</literal></para>
1007               </entry>
1008               <entry>
1009                 <para> Displays help information.</para>
1010               </entry>
1011             </row>
1012             <row>
1013               <entry>
1014                 <para> <literal>stats_file</literal></para>
1015               </entry>
1016               <entry>
1017                 <para> Specifies either the full path to a statistics file or the shorthand reference, <literal>mds</literal> or <literal>ost</literal></para>
1018               </entry>
1019             </row>
1020           </tbody>
1021         </tgroup>
1022       </informaltable>
1023     </section>
1024     <section remap="h5">
1025       <title>Example</title>
1026       <para>To monitor /proc/fs/lustre/ost/OSS/ost/stats at 1 second intervals, run;</para>
1027       <screen>llstat -i 1 ost</screen>
1028     </section>
1029     <section remap="h5">
1030       <title>Files</title>
1031       <para>The llstat files are located at:</para>
1032       <screen>/proc/fs/lustre/mdt/MDS/*/stats
1033 /proc/fs/lustre/mdt/*/exports/*/stats
1034 /proc/fs/lustre/mdc/*/stats
1035 /proc/fs/lustre/ldlm/services/*/stats
1036 /proc/fs/lustre/ldlm/namespaces/*/pool/stats
1037 /proc/fs/lustre/mgs/MGS/exports/*/stats
1038 /proc/fs/lustre/ost/OSS/*/stats
1039 /proc/fs/lustre/osc/*/stats
1040 /proc/fs/lustre/obdfilter/*/exports/*/stats
1041 /proc/fs/lustre/obdfilter/*/stats
1042 /proc/fs/lustre/llite/*/stats
1043 </screen>
1044     </section>
1045   </section>
1046   <section xml:id="dbdoclet.50438219_23648">
1047     <title><indexterm><primary>llverdev</primary></indexterm>
1048 llverdev</title>
1049     <para>The llverdev verifies a block device is functioning properly over its full size.</para>
1050     <section remap="h5">
1051       <title>Synopsis</title>
1052       <screen>llverdev [-c <replaceable>chunksize</replaceable>] [-f] [-h] [-o <replaceable>offset</replaceable>] [-l] [-p] [-r] [-t <replaceable>timestamp</replaceable>] [-v] [-w] <replaceable>device</replaceable></screen>
1053     </section>
1054     <section remap="h5">
1055       <title>Description</title>
1056       <para>Sometimes kernel drivers or hardware devices have bugs that prevent them from accessing the full device size correctly, or possibly have bad sectors on disk or other problems which prevent proper data storage. There are often defects associated with major system boundaries such as 2^32 bytes, 2^31 sectors, 2^31 blocks, 2^32 blocks, etc.</para>
1057       <para>The llverdev utility writes and verifies a unique test pattern across the entire device to ensure that data is accessible after it was written, and that data written to one part of the disk is not overwriting data on another part of the disk.</para>
1058       <para>It is expected that llverdev will be run on large size devices (TB). It is always better to run llverdev in verbose mode, so that device testing can be easily restarted from the point where it was stopped.</para>
1059       <para>Running a full verification can be time-consuming for very large devices. We recommend starting with a partial verification to ensure that the device is minimally sane before investing in a full verification.</para>
1060     </section>
1061     <section remap="h5">
1062       <title>Options</title>
1063       <informaltable frame="all">
1064         <tgroup cols="3">
1065           <colspec colname="c1" colwidth="33*"/>
1066           <colspec colname="c2" colwidth="33*"/>
1067           <colspec colname="c3" colwidth="33*"/>
1068           <thead>
1069             <row>
1070               <entry>
1071                 <para><emphasis role="bold">Option</emphasis></para>
1072               </entry>
1073               <entry>
1074                 <para><emphasis role="bold">&#160;</emphasis></para>
1075               </entry>
1076               <entry>
1077                 <para><emphasis role="bold">Description</emphasis></para>
1078               </entry>
1079             </row>
1080           </thead>
1081           <tbody>
1082             <row>
1083               <entry nameend="c2" namest="c1">
1084                 <para> <literal>-c|--chunksize</literal></para>
1085               </entry>
1086               <entry>
1087                 <para> I/O chunk size in bytes (default value is 1048576).</para>
1088               </entry>
1089             </row>
1090             <row>
1091               <entry nameend="c2" namest="c1">
1092                 <para> <literal>-f|--force</literal></para>
1093               </entry>
1094               <entry>
1095                 <para> Forces the test to run without a confirmation that the device will be overwritten and all data will be permanently destroyed.</para>
1096               </entry>
1097             </row>
1098             <row>
1099               <entry nameend="c2" namest="c1">
1100                 <para> <literal>-h|--help</literal></para>
1101               </entry>
1102               <entry>
1103                 <para> Displays a brief help message.</para>
1104               </entry>
1105             </row>
1106             <row>
1107               <entry nameend="c2" namest="c1">
1108                 <para> <literal>-o <replaceable>offset</replaceable></literal></para>
1109               </entry>
1110               <entry>
1111                 <para> Offset (in kilobytes) of the start of the test (default value is 0).</para>
1112               </entry>
1113             </row>
1114             <row>
1115               <entry nameend="c2" namest="c1">
1116                 <para> <literal>-l|--long</literal></para>
1117               </entry>
1118               <entry>
1119                 <para> Runs a full check, writing and then reading and verifying every block on the disk.</para>
1120               </entry>
1121             </row>
1122             <row>
1123               <entry nameend="c2" namest="c1">
1124                 <para> <literal>-p|--partial</literal></para>
1125               </entry>
1126               <entry>
1127                 <para> Runs a partial check, only doing periodic checks across the device (1 GB steps).</para>
1128               </entry>
1129             </row>
1130             <row>
1131               <entry nameend="c2" namest="c1">
1132                 <para> <literal>-r|--read</literal></para>
1133               </entry>
1134               <entry>
1135                 <para> Runs the test in read (verify) mode only, after having previously run the test in <literal>-w</literal> mode.</para>
1136               </entry>
1137             </row>
1138             <row>
1139               <entry nameend="c2" namest="c1">
1140                 <para> <literal>-t <replaceable>timestamp</replaceable></literal></para>
1141               </entry>
1142               <entry>
1143                 <para> Sets the test start time as printed at the start of a previously-interrupted
1144                   test to ensure that validation data is the same across the entire file system
1145                   (default value is the current time()).</para>
1146               </entry>
1147             </row>
1148             <row>
1149               <entry nameend="c2" namest="c1">
1150                 <para> <literal>-v|--verbose</literal></para>
1151               </entry>
1152               <entry>
1153                 <para> Runs the test in verbose mode, listing each read and write operation.</para>
1154               </entry>
1155             </row>
1156             <row>
1157               <entry nameend="c2" namest="c1">
1158                 <para> <literal>-w|--write</literal></para>
1159               </entry>
1160               <entry>
1161                 <para> Runs the test in write (test-pattern) mode (default runs both read and write).</para>
1162               </entry>
1163             </row>
1164           </tbody>
1165         </tgroup>
1166       </informaltable>
1167     </section>
1168     <section remap="h5">
1169       <title>Examples</title>
1170       <para>Runs a partial device verification on /dev/sda:</para>
1171       <screen>llverdev -v -p /dev/sda
1172 llverdev: permanently overwrite all data on /dev/sda (yes/no)? y
1173 llverdev: /dev/sda is 4398046511104 bytes (4096.0 GB) in size
1174 Timestamp: 1009839028
1175 Current write offset: 4096 kB</screen>
1176       <para>Continues an interrupted verification at offset 4096kB from the start of the device, using the same timestamp as the previous run:</para>
1177       <screen>llverdev -f -v -p --offset=4096 --timestamp=1009839028 /dev/sda
1178 llverdev: /dev/sda is 4398046511104 bytes (4096.0 GB) in size
1179 Timestamp: 1009839028
1180 write complete
1181 read complete </screen>
1182     </section>
1183   </section>
1184   <section xml:id="dbdoclet.50438219_64286">
1185     <title><indexterm><primary>lshowmount</primary></indexterm>
1186 lshowmount</title>
1187     <para>The lshowmount utility shows Lustre exports.</para>
1188     <section remap="h5">
1189       <title>Synopsis</title>
1190       <screen>lshowmount [-ehlv]</screen>
1191     </section>
1192     <section remap="h5">
1193       <title>Description</title>
1194       <para>The lshowmount utility shows the hosts that have Lustre mounted to a server. This utility looks for exports from the MGS, MDS, and obdfilter.</para>
1195     </section>
1196     <section remap="h5">
1197       <title>Options</title>
1198       <informaltable frame="all">
1199         <tgroup cols="2">
1200           <colspec colname="c1" colwidth="50*"/>
1201           <colspec colname="c2" colwidth="50*"/>
1202           <thead>
1203             <row>
1204               <entry>
1205                 <para><emphasis role="bold">Option</emphasis></para>
1206               </entry>
1207               <entry>
1208                 <para><emphasis role="bold">Description</emphasis></para>
1209               </entry>
1210             </row>
1211           </thead>
1212           <tbody>
1213             <row>
1214               <entry>
1215                 <para> <literal>-e|--enumerate</literal></para>
1216               </entry>
1217               <entry>
1218                 <para> Causes lshowmount to list each client mounted on a separate line instead of trying to compress the list of clients into a hostrange string.</para>
1219               </entry>
1220             </row>
1221             <row>
1222               <entry>
1223                 <para> <literal>-h|--help</literal></para>
1224               </entry>
1225               <entry>
1226                 <para> Causes lshowmount to print out a usage message.</para>
1227               </entry>
1228             </row>
1229             <row>
1230               <entry>
1231                 <para> <literal>-l|--lookup</literal></para>
1232               </entry>
1233               <entry>
1234                 <para> Causes lshowmount to try to look up the hostname for NIDs that look like IP addresses.</para>
1235               </entry>
1236             </row>
1237             <row>
1238               <entry>
1239                 <para> <literal>-v|--verbose</literal></para>
1240               </entry>
1241               <entry>
1242                 <para> Causes lshowmount to output export information for each service instead of only displaying the aggregate information for all Lustre services on the server.</para>
1243               </entry>
1244             </row>
1245           </tbody>
1246         </tgroup>
1247       </informaltable>
1248     </section>
1249     <section remap="h5">
1250       <title>Files</title>
1251       <screen>/proc/fs/lustre/mgs/<replaceable>server</replaceable>/exports/<replaceable>uuid</replaceable>/nid
1252 /proc/fs/lustre/mds/<replaceable>server</replaceable>/exports/<replaceable>uuid</replaceable>/nid
1253 /proc/fs/lustre/obdfilter/<replaceable>server</replaceable>/exports/<replaceable>uuid</replaceable>/nid</screen>
1254     </section>
1255   </section>
1256   <section xml:id="dbdoclet.50438219_90218">
1257     <title><indexterm><primary>lst</primary></indexterm>
1258 lst</title>
1259     <para>The lst utility starts LNet self-test.</para>
1260     <section remap="h5">
1261       <title>Synopsis</title>
1262       <screen>lst</screen>
1263     </section>
1264     <section remap="h5">
1265       <title>Description</title>
1266       <para>LNet self-test helps site administrators confirm that Lustre Networking (LNet) has been properly installed and configured. The self-test also confirms that LNet and the network software and hardware underlying it are performing as expected.</para>
1267       <para>Each LNet self-test runs in the context of a session. A node can be associated with only one session at a time, to ensure that the session has exclusive use of the nodes on which it is running. A session is create, controlled and monitored from a single node; this is referred to as the self-test console.</para>
1268       <para>Any node may act as the self-test console. Nodes are named and allocated to a self-test session in groups. This allows all nodes in a group to be referenced by a single name.</para>
1269       <para>Test configurations are built by describing and running test batches. A test batch is a named collection of tests, with each test composed of a number of individual point-to-point tests running in parallel. These individual point-to-point tests are instantiated according to the test type, source group, target group and distribution specified when the test is added to the test batch.</para>
1270     </section>
1271     <section remap="h5">
1272       <title>Modules</title>
1273       <para>To run LNet self-test, load these modules: libcfs, lnet, lnet_selftest and any one of the klnds (ksocklnd, ko2iblnd...). To load all necessary modules, run modprobe lnet_selftest, which recursively loads the modules on which lnet_selftest depends.</para>
1274       <para>There are two types of nodes for LNet self-test: the console node and test nodes. Both node types require all previously-specified modules to be loaded. (The userspace test node does not require these modules).</para>
1275       <para>Test nodes can be in either kernel or in userspace. A console user can invite a kernel test node to join the test session by running lst add_group NID, but the user cannot actively add a userspace test node to the test session. However, the console user can passively accept a test node to the test session while the test node runs lst client to connect to the console.</para>
1276     </section>
1277     <section remap="h5">
1278       <title>Utilities</title>
1279       <para>LNet self-test includes two user utilities, lst and lstclient.</para>
1280       <para>lst is the user interface for the self-test console (run on the console node). It provides a list of commands to control the entire test system, such as create session, create test groups, etc.</para>
1281       <para>lstclient is the userspace self-test program which is linked with userspace LNDs and LNet. A user can invoke lstclient to join a self-test session:</para>
1282       <screen>lstclient -sesid CONSOLE_NID group NAME</screen>
1283     </section>
1284     <section remap="h5">
1285       <title>Example Script</title>
1286       <para>This is a sample LNet self-test script which simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an IB network (connected via LNet routers), with half the clients reading and half the clients writing.</para>
1287       <screen>#!/bin/bash
1288 export LST_SESSION=$$
1289 lst new_session read/write
1290 lst add_group servers 192.168.10.[8,10,12-16]@tcp
1291 lst add_group readers 192.168.1.[1-253/2]@o2ib
1292 lst add_group writers 192.168.1.[2-254/2]@o2ib
1293 lst add_batch bulk_rw
1294 lst add_test --batch bulk_rw --from readers --to servers     brw read check\
1295 =simple size=1M
1296 lst add_test --batch bulk_rw --from writers --to servers     brw write chec\
1297 k=full size=4K
1298 # start running
1299 lst run bulk_rw
1300 # display server stats for 30 seconds
1301 lst stat servers &amp; sleep 30; kill $!
1302 # tear down
1303 lst end_session </screen>
1304     </section>
1305   </section>
1306   <section xml:id="dbdoclet.50438219_54734">
1307     <title><indexterm><primary>lustre_rmmod.sh</primary></indexterm>
1308 lustre_rmmod.sh</title>
1309     <para>The lustre_rmmod.sh utility removes all Lustre and LNet modules (assuming no Lustre services are running). It is located in /usr/bin.</para>
1310     <note>
1311       <para>The lustre_rmmod.sh utility does not work if Lustre modules are being used or if you have manually run the lctl network up command.</para>
1312     </note>
1313   </section>
1314   <section xml:id="dbdoclet.50438219_63667">
1315     <title><indexterm><primary>lustre_rsync</primary></indexterm>
1316 lustre_rsync</title>
1317     <para>The lustre_rsync utility synchronizes (replicates) a Lustre file system to a target file system.</para>
1318     <section remap="h5">
1319       <title>Synopsis</title>
1320       <screen>lustre_rsync --source|-s <replaceable>src</replaceable> --target|-t <replaceable>tgt</replaceable>
1321    --mdt|-m <replaceable>mdt</replaceable> [--user|-u <replaceable>userid</replaceable>]
1322    [--xattr|-x <replaceable>yes|no</replaceable>] [--verbose|-v]
1323    [--statuslog|-l <replaceable>log</replaceable>] [--dry-run] [--abort-on-err]
1324
1325 lustre_rsync --statuslog|-l <replaceable>log</replaceable>
1326
1327 lustre_rsync --statuslog|-l <replaceable>log</replaceable> --source|-s <replaceable>source</replaceable>
1328    --target|-t <replaceable>tgt</replaceable> --mdt|-m <replaceable>mdt</replaceable></screen>
1329     </section>
1330     <section remap="h5">
1331       <title>Description</title>
1332       <para>The lustre_rsync utility is designed to synchronize (replicate) a Lustre file system (source) to another file system (target). The target can be a Lustre file system or any other type, and is a normal, usable file system. The synchronization operation is efficient and does not require directory walking, as lustre_rsync uses Lustre MDT changelogs to identify changes in the Lustre file system.</para>
1333       <para>Before using lustre_rsync:</para>
1334       <itemizedlist>
1335         <listitem>
1336           <para>A changelog user must be registered (see lctl (8) changelog_register)</para>
1337         </listitem>
1338       </itemizedlist>
1339       <para>- AND -</para>
1340       <itemizedlist>
1341         <listitem>
1342           <para>Verify that the Lustre file system (source) and the replica file system (target) are identical before the changelog user is registered. If the file systems are discrepant, use a utility, e.g. regular rsync (not lustre_rsync) to make them identical.</para>
1343         </listitem>
1344       </itemizedlist>
1345     </section>
1346     <section remap="h5">
1347       <title>Options</title>
1348       <informaltable frame="all">
1349         <tgroup cols="2">
1350           <colspec colname="c1" colwidth="50*"/>
1351           <colspec colname="c2" colwidth="50*"/>
1352           <thead>
1353             <row>
1354               <entry>
1355                 <para><emphasis role="bold">Option</emphasis></para>
1356               </entry>
1357               <entry>
1358                 <para><emphasis role="bold">Description</emphasis></para>
1359               </entry>
1360             </row>
1361           </thead>
1362           <tbody>
1363             <row>
1364               <entry>
1365                 <para> <literal>--source=<replaceable>src</replaceable></literal></para>
1366               </entry>
1367               <entry>
1368                 <para> The path to the root of the Lustre file system (source) which will be synchronized. This is a mandatory option if a valid status log created during a previous synchronization operation (--statuslog) is not specified.</para>
1369               </entry>
1370             </row>
1371             <row>
1372               <entry>
1373                 <para> <literal>--target=<replaceable>tgt</replaceable></literal></para>
1374               </entry>
1375               <entry>
1376                 <para> The path to the root where the source file system will be synchronized (target). This is a mandatory option if the status log created during a previous synchronization operation (--statuslog) is not specified. This option can be repeated if multiple synchronization targets are desired.</para>
1377               </entry>
1378             </row>
1379             <row>
1380               <entry>
1381                 <para> <literal>--mdt=<replaceable>mdt</replaceable></literal></para>
1382               </entry>
1383               <entry>
1384                 <para> The metadata device to be synchronized. A changelog user must be registered for this device. This is a mandatory option if a valid status log created during a previous synchronization operation (--statuslog) is not specified.</para>
1385               </entry>
1386             </row>
1387             <row>
1388               <entry>
1389                 <para> <literal>--user=<replaceable>userid</replaceable></literal></para>
1390               </entry>
1391               <entry>
1392                 <para> The changelog user ID for the specified MDT. To use lustre_rsync, the changelog user must be registered. For details, see the changelog_register parameter in the lctl man page. This is a mandatory option if a valid status log created during a previous synchronization operation (--statuslog) is not specified.</para>
1393               </entry>
1394             </row>
1395             <row>
1396               <entry>
1397                 <para> <literal>--statuslog=<replaceable>log</replaceable></literal></para>
1398               </entry>
1399               <entry>
1400                 <para> A log file to which synchronization status is saved. When lustre_rsync starts, the state of a previous replication is read from here. If the status log from a previous synchronization operation is specified, otherwise mandatory options like --source, --target and --mdt options may be skipped. By specifying options like --source, --target and/or --mdt in addition to the --statuslog option, parameters in the status log can be overridden. Command line options take precedence over options in the status log.</para>
1401               </entry>
1402             </row>
1403             <row>
1404               <entry>
1405                 <para> <literal>--xattr<replaceable>yes|no</replaceable></literal></para>
1406               </entry>
1407               <entry>
1408                 <para> Specifies whether extended attributes (xattrs) are synchronized or not. The default is to synchronize extended attributes.</para>
1409                 <para>NOTE: Disabling xattrs causes Lustre striping information not to be synchronized.</para>
1410               </entry>
1411             </row>
1412             <row>
1413               <entry>
1414                 <para> <literal>--verbose</literal></para>
1415               </entry>
1416               <entry>
1417                 <para> Produces a verbose output.</para>
1418               </entry>
1419             </row>
1420             <row>
1421               <entry>
1422                 <para> <literal>--dry-run</literal></para>
1423               </entry>
1424               <entry>
1425                 <para> Shows the output of lustre_rsync commands (copy, mkdir, etc.) on the target file system without actually executing them.</para>
1426               </entry>
1427             </row>
1428             <row>
1429               <entry>
1430                 <para> <literal>--abort-on-err</literal></para>
1431               </entry>
1432               <entry>
1433                 <para> Shows the output of lustre_rsync commands (copy, mkdir, etc.) on the target file system without actually executing them.</para>
1434               </entry>
1435             </row>
1436           </tbody>
1437         </tgroup>
1438       </informaltable>
1439     </section>
1440     <section remap="h5">
1441       <title>Examples</title>
1442       <para>Register a changelog user for an MDT (e.g., MDT lustre-MDT0000).</para>
1443       <screen>$ ssh
1444 $ MDS lctl changelog_register \
1445            --device lustre-MDT0000 -n
1446 cl1</screen>
1447       <para>Synchronize/replicate a Lustre file system (/mnt/lustre) to a target file system (/mnt/target).</para>
1448       <screen>$ lustre_rsync --source=/mnt/lustre --target=/mnt/target \
1449            --mdt=lustre-MDT0000 --user=cl1 \
1450            --statuslog replicate.log  --verbose
1451 Lustre filesystem: lustre
1452 MDT device: lustre-MDT0000
1453 Source: /mnt/lustre
1454 Target: /mnt/target
1455 Statuslog: sync.log
1456 Changelog registration: cl1
1457 Starting changelog record: 0
1458 Errors: 0
1459 lustre_rsync took 1 seconds
1460 Changelog records consumed: 22
1461 </screen>
1462       <para>After the file system undergoes changes, synchronize the changes with the target file system. Only the statuslog name needs to be specified, as it has all the parameters passed earlier.</para>
1463       <screen>$ lustre_rsync --statuslog replicate.log --verbose
1464 Replicating Lustre filesystem: lustre
1465 MDT device: lustre-MDT0000
1466 Source: /mnt/lustre
1467 Target: /mnt/target
1468 Statuslog: replicate.log
1469 Changelog registration: cl1
1470 Starting changelog record: 22
1471 Errors: 0
1472 lustre_rsync took 2 seconds
1473 Changelog records consumed: 42</screen>
1474       <para>Synchronize a Lustre file system (/mnt/lustre) to two target file systems (/mnt/target1 and /mnt/target2).</para>
1475       <screen>$ lustre_rsync --source=/mnt/lustre \
1476    --target=/mnt/target1 --target=/mnt/target2 \
1477    --mdt=lustre-MDT0000 --user=cl1
1478    --statuslog replicate.log</screen>
1479     </section>
1480     <section remap="h5">
1481       <title>See Also</title>
1482       <para><xref linkend="dbdoclet.50438206_94597"/></para>
1483     </section>
1484   </section>
1485   <section xml:id="dbdoclet.50438219_75432">
1486     <title><indexterm><primary>mkfs.lustre</primary></indexterm>
1487 mkfs.lustre</title>
1488     <para>The <literal>mkfs.lustre</literal> utility formats a disk for a Lustre service.</para>
1489     <section remap="h5">
1490       <title>Synopsis</title>
1491       <screen>mkfs.lustre <replaceable>target_type</replaceable> [options] <replaceable>device</replaceable></screen>
1492       <para>where <replaceable>target_type</replaceable> is one of the following:</para>
1493       <informaltable frame="all">
1494         <tgroup cols="2">
1495           <colspec colname="c1" colwidth="50*"/>
1496           <colspec colname="c2" colwidth="50*"/>
1497           <thead>
1498             <row>
1499               <entry>
1500                 <para><emphasis role="bold">Option</emphasis></para>
1501               </entry>
1502               <entry>
1503                 <para><emphasis role="bold">Description</emphasis></para>
1504               </entry>
1505             </row>
1506           </thead>
1507           <tbody>
1508             <row>
1509               <entry>
1510                 <para> <literal>--ost</literal></para>
1511               </entry>
1512               <entry>
1513                 <para> Object storage target (OST)</para>
1514               </entry>
1515             </row>
1516             <row>
1517               <entry>
1518                 <para> <literal>--mdt</literal></para>
1519               </entry>
1520               <entry>
1521                 <para> Metadata storage target (MDT)</para>
1522               </entry>
1523             </row>
1524             <row>
1525               <entry>
1526                 <para> <literal>--network=<replaceable>net,...</replaceable></literal></para>
1527               </entry>
1528               <entry>
1529                 <para> Network(s) to which to restrict this OST/MDT. This option can be repeated as necessary.</para>
1530               </entry>
1531             </row>
1532             <row>
1533               <entry>
1534                 <para> <literal>--mgs</literal></para>
1535               </entry>
1536               <entry>
1537                 <para> Configuration management service (MGS), one per site. This service can be
1538                   combined with one <literal>--mdt</literal> service by specifying both
1539                   types.</para>
1540               </entry>
1541             </row>
1542           </tbody>
1543         </tgroup>
1544       </informaltable>
1545     </section>
1546     <section remap="h5">
1547       <title>Description</title>
1548       <para><literal>mkfs.lustre</literal> is used to format a disk device for use as part of a
1549         Lustre file system. After formatting, a disk can be mounted to start the Lustre service
1550         defined by this command.</para>
1551       <para>When the file system is created, parameters can simply be added as a
1552           <literal>--param</literal> option to the <literal>mkfs.lustre</literal> command. See <xref
1553           linkend="dbdoclet.50438194_17237"/>.</para>
1554       <informaltable frame="all">
1555         <tgroup cols="3">
1556           <colspec colname="c1" colwidth="1*"/>
1557           <colspec colname="c2" colwidth="1*"/>
1558           <colspec colname="c3" colwidth="3*"/>
1559           <thead>
1560             <row>
1561               <entry nameend="c2" namest="c1">
1562                 <para><emphasis role="bold">Option</emphasis></para>
1563               </entry>
1564               <entry>
1565                 <para><emphasis role="bold">Description</emphasis></para>
1566               </entry>
1567             </row>
1568           </thead>
1569           <tbody>
1570             <row>
1571               <entry nameend="c2" namest="c1">
1572                 <para> <literal>--backfstype=<replaceable>fstype</replaceable></literal></para>
1573               </entry>
1574               <entry>
1575                 <para> Forces a particular format for the backing file system such as ldiskfs (the default) or zfs.</para>
1576               </entry>
1577             </row>
1578             <row>
1579               <entry nameend="c2" namest="c1">
1580                 <para> <literal>--comment=<replaceable>comment</replaceable></literal></para>
1581               </entry>
1582               <entry>
1583                 <para> Sets a user comment about this disk, ignored by the Lustre software.</para>
1584               </entry>
1585             </row>
1586             <row>
1587               <entry nameend="c2" namest="c1">
1588                 <para> <literal>--device-size=<replaceable>#</replaceable>>KB</literal></para>
1589               </entry>
1590               <entry>
1591                 <para>Sets the device size for loop devices.</para>
1592               </entry>
1593             </row>
1594             <row>
1595               <entry nameend="c2" namest="c1">
1596                 <para> <literal>--dryrun</literal></para>
1597               </entry>
1598               <entry>
1599                 <para>Only prints what would be done; it does not affect the disk.</para>
1600               </entry>
1601             </row>
1602             <row>
1603               <entry nameend="c2" namest="c1"
1604                     ><literal>--servicenode=<replaceable>nid,...</replaceable></literal></entry>
1605               <entry>Sets the NID(s) of all service nodes, including primary and failover partner
1606                 service nodes. The <literal>--servicenode</literal> option cannot be used with
1607                   <literal>--failnode</literal> option. See <xref
1608                   xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/> for
1609                 more details.</entry>
1610             </row>
1611             <row>
1612               <entry nameend="c2" namest="c1">
1613                 <para> <literal>--failnode=<replaceable>nid,...</replaceable></literal></para>
1614               </entry>
1615               <entry>
1616                 <para>Sets the NID(s) of a failover service node for a primary server for a target.
1617                   The <literal>--failnode</literal> option cannot be used with
1618                     <literal>--servicenode</literal> option. See <xref
1619                     xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/>
1620                   for more details.<note>
1621                     <para>When the <literal>--failnode</literal> option is used, certain
1622                       restrictions apply (see <xref xmlns:xlink="http://www.w3.org/1999/xlink"
1623                         linkend="dbdoclet.50438188_92688"/>).</para>
1624                   </note></para>
1625               </entry>
1626             </row>
1627             <row>
1628               <entry nameend="c2" namest="c1">
1629                 <para> <literal>--fsname=<replaceable>filesystem_name</replaceable></literal></para>
1630               </entry>
1631               <entry>
1632                 <para> The Lustre file system of which this service/node will be a part. The default
1633                   file system name is <literal>lustre</literal>.</para>
1634                 <para>&#160;</para>
1635                 <note>
1636                   <para>The file system name is limited to 8 characters.</para>
1637                 </note>
1638               </entry>
1639             </row>
1640             <row>
1641               <entry nameend="c2" namest="c1">
1642                 <para>
1643                   <literal>--index=<replaceable>index_number</replaceable></literal></para>
1644               </entry>
1645               <entry>
1646                 <para>Specifies the OST or MDT number (0...N). This allows mapping between the OSS
1647                   and MDS node and the device on which the OST or MDT is located.</para>
1648               </entry>
1649             </row>
1650             <row>
1651               <entry nameend="c2" namest="c1">
1652                 <para> <literal>--mkfsoptions=<replaceable>opts</replaceable></literal></para>
1653               </entry>
1654               <entry>
1655                 <para>  Formats options for the backing file system. For example, ext3 options could be set here.</para>
1656               </entry>
1657             </row>
1658             <row>
1659               <entry nameend="c2" namest="c1">
1660                 <para> <literal>--mountfsoptions=<replaceable>opts</replaceable></literal></para>
1661               </entry>
1662               <entry>
1663                 <para>  Sets the mount options used when the backing file system is mounted.</para>
1664                 <warning><para>Unlike earlier versions of <literal>mkfs.lustre</literal>, this version completely replaces
1665                     the default mount options with those specified on the command line, and issues a
1666                     warning on stderr if any default mount options are omitted.</para></warning>
1667                 <para>The defaults for ldiskfs are:</para>
1668                 <para>MGS/MDT: <literal>errors=remount-ro,iopen_nopriv,user_xattr</literal></para>
1669                 <para>OST: <literal>errors=remount-ro,extents,mballoc</literal></para>
1670                 <para condition='l25'>OST: <literal>errors=remount-ro</literal></para>
1671                 <para>Use care when altering the default mount options.</para>
1672               </entry>
1673             </row>
1674             <row>
1675               <entry nameend="c2" namest="c1">
1676                 <para> <literal>--network=<replaceable>net,...</replaceable></literal></para>
1677                 <para>&#160;</para>
1678               </entry>
1679               <entry>
1680                 <para>  Network(s) to which to restrict this OST/MDT. This option can be repeated as necessary.</para>
1681               </entry>
1682             </row>
1683             <row>
1684               <entry nameend="c2" namest="c1">
1685                 <para> <literal>--mgsnode=<replaceable>nid,...</replaceable></literal></para>
1686               </entry>
1687               <entry>
1688                 <para> Sets the NIDs of the MGS node, required for all targets other than the MGS.</para>
1689               </entry>
1690             </row>
1691             <row>
1692               <entry nameend="c2" namest="c1">
1693                 <para> <literal>--param <replaceable>key</replaceable>=<replaceable>value</replaceable></literal></para>
1694               </entry>
1695               <entry>
1696                 <para> Sets the permanent parameter <replaceable>key</replaceable> to value <replaceable>value</replaceable>. This option can be repeated as necessary. Typical options might include:</para>
1697               </entry>
1698             </row>
1699             <row>
1700               <entry>
1701                 <para> &#160;</para>
1702               </entry>
1703               <entry>
1704                 <para> <literal>--param sys.timeout=40</literal>></para>
1705               </entry>
1706               <entry>
1707                 <para> System obd timeout.</para>
1708               </entry>
1709             </row>
1710             <row>
1711               <entry>
1712                 <para> &#160;</para>
1713               </entry>
1714               <entry>
1715                 <para> <literal>--param lov.stripesize=2M</literal></para>
1716               </entry>
1717               <entry>
1718                 <para> Default stripe size.</para>
1719               </entry>
1720             </row>
1721             <row>
1722               <entry>
1723                 <para> &#160;</para>
1724               </entry>
1725               <entry>
1726                 <para> <literal>param lov.stripecount=2</literal></para>
1727               </entry>
1728               <entry>
1729                 <para> Default stripe count.</para>
1730               </entry>
1731             </row>
1732             <row>
1733               <entry>
1734                 <para> &#160;</para>
1735               </entry>
1736               <entry>
1737                 <para> <literal>--param failover.mode=failout</literal></para>
1738               </entry>
1739               <entry>
1740                 <para> Returns errors instead of waiting for recovery.</para>
1741               </entry>
1742             </row>
1743             <row>
1744               <entry nameend="c2" namest="c1">
1745                 <para> <literal>--quiet</literal></para>
1746               </entry>
1747               <entry>
1748                 <para> Prints less information.</para>
1749               </entry>
1750             </row>
1751             <row>
1752               <entry nameend="c2" namest="c1">
1753                 <para> <literal>--reformat</literal></para>
1754               </entry>
1755               <entry>
1756                 <para> Reformats an existing Lustre disk.</para>
1757               </entry>
1758             </row>
1759             <row>
1760               <entry nameend="c2" namest="c1">
1761                 <para> <literal>--stripe-count-hint=stripes</literal></para>
1762               </entry>
1763               <entry>
1764                 <para> Used to optimize the MDT&apos;s inode size.</para>
1765               </entry>
1766             </row>
1767             <row>
1768               <entry nameend="c2" namest="c1">
1769                 <para> <literal>--verbose</literal></para>
1770               </entry>
1771               <entry>
1772                 <para> Prints more information.</para>
1773               </entry>
1774             </row>
1775           </tbody>
1776         </tgroup>
1777       </informaltable>
1778     </section>
1779     <section remap="h5">
1780       <title>Examples</title>
1781       <para>Creates a combined MGS and MDT for file system <literal>testfs</literal> on, e.g., node <literal>cfs21</literal>:</para>
1782       <screen>mkfs.lustre --fsname=testfs --mdt --mgs /dev/sda1</screen>
1783       <para>Creates an OST for file system <literal>testfs</literal> on any node (using the above
1784         MGS):</para>
1785       <screen>mkfs.lustre --fsname=testfs --mgsnode=cfs21@tcp0 --ost --index=0 /dev/sdb</screen>
1786       <para>Creates a standalone MGS on, e.g., node <literal>cfs22</literal>:</para>
1787       <screen>mkfs.lustre --mgs /dev/sda1</screen>
1788       <para>Creates an MDT for file system <literal>myfs1</literal> on any node (using the above MGS):</para>
1789       <screen>mkfs.lustre --fsname=myfs1 --mdt --mgsnode=cfs22@tcp0 /dev/sda2</screen>
1790     </section>
1791     <section remap="h5">
1792       <title>See Also</title>
1793       <itemizedlist>
1794         <listitem>
1795           <para><xref linkend="dbdoclet.50438219_75432"/>mkfs.lustre, </para>
1796         </listitem>
1797         <listitem>
1798           <para><xref linkend="dbdoclet.50438219_12635"/>mount.lustre, </para>
1799         </listitem>
1800         <listitem>
1801           <para><xref linkend="dbdoclet.50438206_94597"/>lfs</para>
1802         </listitem>
1803       </itemizedlist>
1804     </section>
1805   </section>
1806   <section xml:id="dbdoclet.50438219_12635">
1807     <title><indexterm><primary>mount.lustre</primary></indexterm>
1808 mount.lustre</title>
1809     <para>The mount.lustre utility starts a Lustre client or target service.</para>
1810     <section remap="h5">
1811       <title>Synopsis</title>
1812       <screen>mount -t lustre [-o options] device mountpoint
1813 </screen>
1814     </section>
1815     <section remap="h5">
1816       <title>Description</title>
1817       <para>The mount.lustre utility starts a Lustre client or target service. This program should not be called directly; rather, it is a helper program invoked through mount(8), as shown above. Use the umount command to stop Lustre clients and targets.</para>
1818       <para>There are two forms for the device option, depending on whether a client or a target service is started:</para>
1819       <informaltable frame="all">
1820         <tgroup cols="2">
1821           <colspec colname="c1" colwidth="50*"/>
1822           <colspec colname="c2" colwidth="50*"/>
1823           <thead>
1824             <row>
1825               <entry>
1826                 <para><emphasis role="bold">Option</emphasis></para>
1827               </entry>
1828               <entry>
1829                 <para><emphasis role="bold">Description</emphasis></para>
1830               </entry>
1831             </row>
1832           </thead>
1833           <tbody>
1834             <row>
1835               <entry>
1836                 <para> <literal><replaceable>mgsname</replaceable>:/<replaceable>fsname</replaceable><replaceable>[/subdir]</replaceable></literal></para>
1837               </entry>
1838               <entry>
1839               <para> Mounts the Lustre file system named
1840               <replaceable>fsname</replaceable> (optionally starting at
1841               subdirectory <replaceable>subdir</replaceable> within the
1842               filesystem, if specified) on the client at the directory
1843               <replaceable>mountpoint</replaceable>, by contacting the Lustre
1844               Management Service at <replaceable>mgsname</replaceable>.  The
1845               format for <replaceable>mgsname</replaceable> is defined below. A
1846               client file system can be listed in <literal>fstab(5)</literal>
1847               for automatic mount at boot time, is usable like any local file
1848               system, and provides a full POSIX standard-compliant interface.
1849               </para>
1850               </entry>
1851             </row>
1852             <row>
1853               <entry>
1854                 <para> <replaceable>block_device</replaceable></para>
1855               </entry>
1856               <entry>
1857                 <para> Starts the target service defined by the
1858                 <literal>mkfs.lustre(8)</literal> command on the physical disk
1859                 <replaceable>block_device</replaceable>.  The
1860                 <replaceable>block_device</replaceable> may be specified using
1861                 <literal>-L <replaceable>label</replaceable></literal> to find
1862                 the first block device with that label (e.g.
1863                 <literal>testfs-MDT0000</literal>), or by UUID using the
1864                 <literal>-U <replaceable>uuid</replaceable></literal> option.
1865                 Care should be taken if there is a device-level backup of the
1866                 target filesystem on the same node, which would have a
1867                 duplicate label and UUID if it has not been changed with
1868                 <literal>tune2fs(8)</literal> or similar.  The mounted target
1869                 service filesystem mounted at
1870                 <replaceable>mountpoint</replaceable> is only useful for
1871                 <literal>df(1)</literal> operations and appears in
1872                 <literal>/proc/mounts</literal> to show the device is in use.
1873                 </para>
1874               </entry>
1875             </row>
1876           </tbody>
1877         </tgroup>
1878       </informaltable>
1879     </section>
1880     <section remap="h5">
1881       <title>Options</title>
1882       <informaltable frame="all">
1883         <tgroup cols="2">
1884           <colspec colname="c1" colwidth="50*"/>
1885           <colspec colname="c2" colwidth="50*"/>
1886           <thead>
1887             <row>
1888               <entry>
1889                 <para><emphasis role="bold">Option</emphasis></para>
1890               </entry>
1891               <entry>
1892                 <para><emphasis role="bold">Description</emphasis></para>
1893               </entry>
1894             </row>
1895           </thead>
1896           <tbody>
1897             <row>
1898               <entry>
1899                 <para> <literal>mgsname=<replaceable>mgsnode</replaceable>[:<replaceable>mgsnode</replaceable>]</literal></para>
1900               </entry>
1901               <entry>
1902                 <para><replaceable>mgsname</replaceable> is a colon-separated
1903                 list of <replaceable>mgsnode</replaceable> names where the MGS
1904                 service may run.  Multiple <replaceable>mgsnode</replaceable>
1905                 values can be specified if the MGS service is configured for
1906                 HA failover and may be running on any one of the nodes.
1907                 </para>
1908               </entry>
1909             </row>
1910             <row>
1911               <entry>
1912                 <para> <literal>mgsnode=<replaceable>mgsnid</replaceable>[,<replaceable>mgsnid</replaceable>]</literal></para>
1913               </entry>
1914               <entry>
1915                 <para> Each <replaceable>mgsnode</replaceable> may specify a
1916                 comma-separated list of NIDs, if there are different LNet
1917                 interfaces for that <literal>mgsnode</literal>.
1918                 </para>
1919               </entry>
1920             </row>
1921             <row>
1922               <entry>
1923                 <para> <literal>mgssec=<replaceable>flavor</replaceable></literal></para>
1924               </entry>
1925               <entry>
1926                 <para>Specifies the encryption flavor for the initial network
1927                 RPC connection to the MGS.  Non-security flavors are:
1928                 <literal>null</literal>, <literal>plain</literal>, and
1929                 <literal>gssnull</literal>, which respectively disable, or
1930                 have no encryption or integrity features for testing purposes.
1931                 Kerberos flavors are: <literal>krb5n</literal>,
1932                 <literal>krb5a</literal>, <literal>krb5i</literal>, and
1933                 <literal>krb5p</literal>.  Shared-secret key flavors are:
1934                 <literal>skn</literal>, <literal>ska</literal>,
1935                 <literal>ski</literal>, and <literal>skpi</literal>, see the
1936                 <xref linkend="lustressk"/> for more details.  The security
1937                 flavor for client-to-server connections is specified in the
1938                 filesystem configuration that the client fetches from the MGS.
1939                 </para>
1940               </entry>
1941             </row>
1942             <row>
1943               <entry>
1944                 <para> <literal>skpath=<replaceable>file|directory</replaceable></literal></para>
1945               </entry>
1946               <entry>
1947                 <para condition='l29'>
1948                 Path to a file or directory with the keyfile(s) to load for
1949                 this mount command.  Keys are inserted into the
1950                 <literal>KEY_SPEC_SESSION_KEYRING</literal> keyring in the
1951                 kernel with a description containing
1952                 <literal>lustre:</literal> and a suffix which depends on
1953                 whether the context of the mount command is for an MGS,
1954                 MDT/OST, or client.
1955                 </para>
1956               </entry>
1957             </row>
1958             <row>
1959               <entry>
1960                 <para> <literal>exclude=<replaceable>ostlist</replaceable></literal></para>
1961               </entry>
1962               <entry>
1963                 <para>Starts a client or MDT with a colon-separated list of
1964                 known inactive OSTs that it will not try to connect to.</para>
1965               </entry>
1966             </row>
1967           </tbody>
1968         </tgroup>
1969       </informaltable>
1970       <para>In addition to the standard mount(8) options, Lustre understands
1971       the following client-specific options:</para>
1972       <informaltable frame="all">
1973         <tgroup cols="2">
1974           <colspec colname="c1" colwidth="50*"/>
1975           <colspec colname="c2" colwidth="50*"/>
1976           <thead>
1977             <row>
1978               <entry>
1979                 <para><emphasis role="bold">Option</emphasis></para>
1980               </entry>
1981               <entry>
1982                 <para><emphasis role="bold">Description</emphasis></para>
1983               </entry>
1984             </row>
1985           </thead>
1986           <tbody>
1987             <row>
1988               <entry>
1989                 <para><literal>always_ping</literal></para>
1990               </entry>
1991               <entry>
1992                 <para condition='l29'>The client will periodically ping the server when it is
1993                 idle, even if the server <literal>ptlrpc</literal> module
1994                 is configured with the <literal>suppress_pings</literal>
1995                 option.  This allows clients to reliably use the filesystem
1996                 even if they are not part of an external client health
1997                 monitoring mechanism.
1998                 </para>
1999               </entry>
2000             </row>
2001             <row>
2002               <entry>
2003                 <para> <literal>flock</literal></para>
2004               </entry>
2005               <entry>
2006                 <para>Enables advisory file locking support between
2007                 participating applications using the <literal>flock(2)</literal>
2008                 system call.  This causes file locking to be coherent across all
2009                 client nodes also using this mount option.  This is useful if
2010                 applications need coherent userspace file locking across
2011                 multiple client nodes, but also imposes communications overhead
2012                 in order to maintain locking consistency between client nodes.
2013                 </para>
2014               </entry>
2015             </row>
2016             <row>
2017               <entry>
2018                 <para> <literal>localflock</literal></para>
2019               </entry>
2020               <entry>
2021                 <para>Enables client-local <literal>flock(2)</literal> support,
2022                 using only client-local advisory file locking.  This is faster
2023                 than using the global <literal>flock</literal> option, and can
2024                 be used for applications that depend on functioning
2025                 <literal>flock(2)</literal> but run only on a single node.
2026                 It has minimal overhead using only the Linux kernel's locks.
2027                 </para>
2028               </entry>
2029             </row>
2030             <row>
2031               <entry>
2032                 <para> <literal>noflock</literal></para>
2033               </entry>
2034               <entry>
2035                 <para>Disables <literal>flock(2)</literal> support entirely,
2036                 and is the default option. Applications calling
2037                 <literal>flock(2)</literal> get an
2038                 <literal>ENOSYS</literal> error. It is up to the administrator
2039                 to choose either the <literal>localflock</literal> or
2040                 <literal>flock</literal> mount option based on their
2041                 requirements.  It is possible to mount clients with different
2042                 options, and only those mounted with <literal>flock</literal>
2043                 will be coherent amongst each other.
2044                 </para>
2045               </entry>
2046             </row>
2047             <row>
2048               <entry>
2049                 <para> <literal>lazystatfs</literal></para>
2050               </entry>
2051               <entry>
2052                 <para>Allows <literal>statfs(2)</literal> (as used by
2053                 <literal>df(1)</literal> and <literal>lfs-df(1)</literal>) to
2054                 return even if some OST or MDT is unresponsive or has been
2055                 temporarily or permanently disabled in the configuration.
2056                 This avoids blocking until all of the targets are available.
2057                 This is the default behavior since Lustre 2.9.0.
2058                 </para>
2059               </entry>
2060             </row>
2061             <row>
2062               <entry>
2063                 <para> <literal>nolazystatfs</literal></para>
2064               </entry>
2065               <entry>
2066                 <para>Requires that <literal>statfs(2)</literal> block until all
2067                 OSTs and MDTs are available and have returned space usage.
2068                 </para>
2069               </entry>
2070             </row>
2071             <row>
2072               <entry>
2073                 <para> <literal>user_xattr</literal></para>
2074               </entry>
2075               <entry>
2076                 <para>Enables get/set of extended attributes by regular users
2077                 in the <literal>user.*</literal> namespace. See the
2078                 <literal>attr(5)</literal> manual page for more details.
2079                 </para>
2080               </entry>
2081             </row>
2082             <row>
2083               <entry>
2084                 <para> <literal>nouser_xattr</literal></para>
2085               </entry>
2086               <entry>
2087                 <para>Disables use of extended attributes in the
2088                 <literal>user.*</literal> namespace by regular users. Root
2089                 and system processes can still use extended attributes.</para>
2090               </entry>
2091             </row>
2092             <row>
2093               <entry>
2094                 <para> <literal>verbose</literal></para>
2095               </entry>
2096               <entry>
2097                 <para> Enable extra mount/umount console messages.</para>
2098               </entry>
2099             </row>
2100             <row>
2101               <entry>
2102                 <para> <literal>noverbose</literal></para>
2103               </entry>
2104               <entry>
2105                 <para> Disable mount/umount console messages.</para>
2106               </entry>
2107             </row>
2108             <row>
2109               <entry>
2110                 <para> <literal>user_fid2path</literal></para>
2111               </entry>
2112               <entry>
2113                 <para condition='l23'>Enable FID to path translation by regular
2114                 users. Note: This option allows a potential security hole
2115                 because it allows regular users direct access to a file by its
2116                 File ID, bypassing POSIX path-based permission checks which
2117                 could otherwise prevent the user from accessing a file in a
2118                 directory that they do not have access to. Regular permission
2119                 checks are still performed on the file itself, so the user
2120                 cannot access a file to which they have no access rights.
2121                 </para>
2122               </entry>
2123             </row>
2124             <row>
2125               <entry>
2126                 <para> <literal>nouser_fid2path</literal></para>
2127               </entry>
2128               <entry>
2129                 <para condition='l23'> Disable FID to path translation by
2130                 regular users. Root and processes with
2131                 <literal>CAP_DAC_READ_SEARCH</literal> can still perform FID
2132                 to path translation.
2133                 </para>
2134               </entry>
2135             </row>
2136           </tbody>
2137         </tgroup>
2138       </informaltable>
2139       <para>In addition to the standard mount options and backing disk type
2140       (e.g. ldiskfs) options, Lustre understands the following server-specific
2141       mount options:</para>
2142       <informaltable frame="all">
2143         <tgroup cols="2">
2144           <colspec colname="c1" colwidth="50*"/>
2145           <colspec colname="c2" colwidth="50*"/>
2146           <thead>
2147             <row>
2148               <entry>
2149                 <para><emphasis role="bold">Option</emphasis></para>
2150               </entry>
2151               <entry>
2152                 <para><emphasis role="bold">Description</emphasis></para>
2153               </entry>
2154             </row>
2155           </thead>
2156           <tbody>
2157             <row>
2158               <entry>
2159                 <para> <literal>nosvc</literal></para>
2160               </entry>
2161               <entry>
2162                 <para>  Starts the MGC (and MGS, if co-located) for a target service, not the actual service.</para>
2163               </entry>
2164             </row>
2165             <row>
2166               <entry>
2167                 <para> <literal>nomgs</literal></para>
2168               </entry>
2169               <entry>
2170                 <para>  Starts only the MDT (with a co-located MGS), without starting the MGS.</para>
2171               </entry>
2172             </row>
2173             <row>
2174               <entry>
2175                 <para> <literal>abort_recov</literal></para>
2176               </entry>
2177               <entry>
2178                 <para>  Aborts client recovery on that server and starts the target service immediately.</para>
2179               </entry>
2180             </row>
2181             <row>
2182               <entry>
2183                 <para> <literal>max_sectors_kb=<replaceable>KB</replaceable></literal></para>
2184               </entry>
2185               <entry>
2186                 <para condition='l210'>Sets the block device parameter
2187                 <literal>max_sectors_kb</literal> limit for the MDT or OST
2188                 target being mounted to specified maximum number of kilobytes.
2189                 When <literal>max_sectors_kb</literal> isn't specified as a
2190                 mount option, it will automatically be set to the
2191                 <literal>max_hw_sectors_kb</literal> (up to a maximum of 16MiB)
2192                 for that block device. This default behavior is suited for
2193                 most users. When <literal>max_sectors_kb=0</literal> is used,
2194                 the current value for this tunable will be kept.
2195                 </para>
2196               </entry>
2197             </row>
2198             <row>
2199               <entry>
2200                 <para> <literal>md_stripe_cache_size</literal></para>
2201               </entry>
2202               <entry>
2203                 <para>  Sets the stripe cache size for server-side disk with a striped RAID configuration.</para>
2204               </entry>
2205             </row>
2206             <row>
2207               <entry>
2208                 <para> <literal>recovery_time_soft=<replaceable>timeout</replaceable></literal></para>
2209               </entry>
2210               <entry>
2211                 <para>Allows <literal>timeout</literal> seconds for clients to
2212                 reconnect for recovery after a server crash. This timeout is
2213                 incrementally extended if it is about to expire and the server
2214                 is still handling new connections from recoverable clients.
2215                 </para>
2216                 <para>The default soft recovery timeout is 3 times the value
2217                 of the Lustre timeout parameter (see
2218                 <xref linkend="section_c24_nt5_dl"/>). The default Lustre
2219                 timeout is 100 seconds, which would make the soft recovery
2220                 timeout default to 300 seconds (5 minutes). The soft recovery
2221                 timeout is set at mount time and will not change if the Lustre
2222                 timeout is changed after mount time.
2223                 </para>
2224               </entry>
2225             </row>
2226             <row>
2227               <entry>
2228                 <para> <literal>recovery_time_hard=<replaceable>timeout</replaceable></literal></para>
2229               </entry>
2230               <entry>
2231                 <para>The server is allowed to incrementally extend its timeout
2232                 up to a hard maximum of <replaceable>timeout</replaceable>
2233                 seconds.
2234                 </para>
2235                 <para>The default hard recovery timeout is 9 times the value
2236                 of the Lustre timeout parameter (see
2237                 <xref linkend="section_c24_nt5_dl"/>). The default Lustre
2238                 timeout is 100 seconds, which would make the hard recovery
2239                 timeout default to 900 seconds (15 minutes). The hard recovery
2240                 timeout is set at mount time and will not change if the Lustre
2241                 timeout is changed after mount time.
2242                 </para>
2243               </entry>
2244             </row>
2245             <row>
2246               <entry>
2247                 <para> <literal>noscrub</literal></para>
2248               </entry>
2249               <entry>
2250                 <para>Typically the MDT will detect restoration from a
2251                 file-level backup during mount. This mount option prevents
2252                 the OI Scrub from starting automatically when the MDT is
2253                 mounted. Manually starting LFSCK after mounting provides finer
2254                 control over the starting conditions. This mount option also
2255                 prevents OI scrub from occurring automatically when OI
2256                 inconsistency is detected (see
2257                 <xref linkend="dbdoclet.lfsck_auto_scrub"/>).
2258                 </para>
2259               </entry>
2260             </row>
2261           </tbody>
2262         </tgroup>
2263       </informaltable>
2264     </section>
2265     <section remap="h5">
2266       <title>Examples</title>
2267       <para>Starts a client for the Lustre file system
2268       <replaceable>chipfs</replaceable> at mount point
2269       <replaceable>/mnt/chip</replaceable>. The Management Service is running on
2270       a node reachable from this client via the cfs21@tcp0 NID.</para>
2271       <screen>mount -t lustre cfs21@tcp0:/chipfs /mnt/chip</screen>
2272       <para condition='l29'>Similar to the above example, but mounting a
2273       subdirectory under <replaceable>chipfs</replaceable> as a fileset.
2274       <screen>mount -t lustre cfs21@tcp0:/chipfs/v1_0 /mnt/chipv1_0</screen>
2275       </para>
2276       <para>Starts the Lustre metadata target service from /dev/sda1 on mount point /mnt/test/mdt.</para>
2277       <screen>mount -t lustre /dev/sda1 /mnt/test/mdt</screen>
2278       <para>Starts the testfs-MDT0000 service (using the disk label), but aborts the recovery process.</para>
2279       <screen>mount -t lustre -L testfs-MDT0000 -o abort_recov /mnt/test/mdt</screen>
2280     </section>
2281     <section remap="h5">
2282       <title>See Also</title>
2283       <itemizedlist>
2284         <listitem>
2285           <para>  <xref linkend="dbdoclet.50438219_75432"/></para>
2286         </listitem>
2287         <listitem>
2288           <para>  <xref linkend="dbdoclet.50438219_39574"/></para>
2289         </listitem>
2290         <listitem>
2291           <para>  <xref linkend="dbdoclet.50438219_38274"/></para>
2292         </listitem>
2293         <listitem>
2294           <para>  <xref linkend="dbdoclet.50438206_94597"/></para>
2295         </listitem>
2296       </itemizedlist>
2297     </section>
2298   </section>
2299   <section xml:id="dbdoclet.50438219_82679">
2300     <title><indexterm><primary>plot-llstat</primary></indexterm>
2301 plot-llstat</title>
2302     <para>The plot-llstat utility plots Lustre statistics.</para>
2303     <section remap="h5">
2304       <title>Synopsis</title>
2305       <screen>plot-llstat results_filename [parameter_index]
2306 </screen>
2307     </section>
2308     <section remap="h5">
2309       <title>Description</title>
2310       <para>The plot-llstat utility generates a CSV file and instruction files for gnuplot from the output of llstat. Since llstat is generic in nature, plot-llstat is also a generic script. The value of parameter_index can be 1 for count per interval, 2 for count per second (default setting) or 3 for total count.</para>
2311       <para>The plot-llstat utility creates a .dat (CSV) file using the number of operations specified by the user. The number of operations equals the number of columns in the CSV file. The values in those columns are equal to the corresponding value of parameter_index in the output file.</para>
2312       <para>The plot-llstat utility also creates a .scr file that contains instructions for gnuplot to plot the graph. After generating the .dat and .scr files, the plot-llstat tool invokes gnuplot to display the graph.</para>
2313     </section>
2314     <section remap="h5">
2315       <title>Options</title>
2316       <informaltable frame="all">
2317         <tgroup cols="2">
2318           <colspec colname="c1" colwidth="50*"/>
2319           <colspec colname="c2" colwidth="50*"/>
2320           <thead>
2321             <row>
2322               <entry>
2323                 <para><emphasis role="bold">Option</emphasis></para>
2324               </entry>
2325               <entry>
2326                 <para><emphasis role="bold">Description</emphasis></para>
2327               </entry>
2328             </row>
2329           </thead>
2330           <tbody>
2331             <row>
2332               <entry>
2333                 <para> <literal>results_filename</literal></para>
2334               </entry>
2335               <entry>
2336                 <para> Output generated by plot-llstat</para>
2337               </entry>
2338             </row>
2339             <row>
2340               <entry>
2341                 <para> <literal>parameter_index</literal></para>
2342                 <para>&#160;</para>
2343               </entry>
2344               <entry>
2345                 <para> Value of parameter_index can be:</para>
2346                 <para> 1 - count per interval</para>
2347                 <para> 2 - count per second (default setting)</para>
2348                 <para> 3 - total count</para>
2349               </entry>
2350             </row>
2351           </tbody>
2352         </tgroup>
2353       </informaltable>
2354     </section>
2355     <section remap="h5">
2356       <title>Example</title>
2357       <screen>llstat -i2 -g -c lustre-OST0000 &gt; log
2358 plot-llstat log 3</screen>
2359     </section>
2360   </section>
2361   <section xml:id="dbdoclet.50438219_51496">
2362     <title><indexterm><primary>routerstat</primary></indexterm>
2363 routerstat</title>
2364     <para>The routerstat utility prints Lustre router statistics.</para>
2365     <section remap="h5">
2366       <title>Synopsis</title>
2367       <screen>routerstat [<replaceable>interval</replaceable>]</screen>
2368     </section>
2369     <section remap="h5">
2370       <title>Description</title>
2371       <para>The routerstat utility displays LNet router statistics. If no <literal><replaceable>interval</replaceable></literal> is specified, then statistics are sampled and printed only one time. Otherwise, statistics are sampled and printed at the specified <literal><replaceable>interval</replaceable></literal> (in seconds).</para>
2372     </section>
2373     <section remap="h5">
2374       <title>Output</title>
2375       <para>The routerstat output includes the following fields:</para>
2376       <informaltable frame="all">
2377         <tgroup cols="2">
2378           <colspec colname="c1" colwidth="50*"/>
2379           <colspec colname="c2" colwidth="50*"/>
2380           <thead>
2381             <row>
2382               <entry>
2383                 <para><emphasis role="bold">Output</emphasis></para>
2384               </entry>
2385               <entry>
2386                 <para><emphasis role="bold">Description</emphasis></para>
2387               </entry>
2388             </row>
2389           </thead>
2390           <tbody>
2391             <row>
2392               <entry>
2393                 <para> <literal>M</literal></para>
2394               </entry>
2395               <entry>
2396                 <para> Number of messages currently being processed by LNet (The maximum number of messages ever processed by LNet concurrently)</para>
2397               </entry>
2398             </row>
2399             <row>
2400               <entry>
2401                 <para> <literal>E</literal></para>
2402               </entry>
2403               <entry>
2404                 <para> Number of LNet errors</para>
2405               </entry>
2406             </row>
2407             <row>
2408               <entry>
2409                 <para> <literal>S</literal></para>
2410               </entry>
2411               <entry>
2412                 <para> Total size (length) of messages sent in bytes/ Number of messages sent</para>
2413               </entry>
2414             </row>
2415             <row>
2416               <entry>
2417                 <para> <literal>R</literal></para>
2418               </entry>
2419               <entry>
2420                 <para> Total size (length) of messages received in bytes/ Number of messages received</para>
2421               </entry>
2422             </row>
2423             <row>
2424               <entry>
2425                 <para> <literal>F</literal></para>
2426               </entry>
2427               <entry>
2428                 <para> Total size (length) of messages routed in bytes/ Number of messages routed</para>
2429               </entry>
2430             </row>
2431             <row>
2432               <entry>
2433                 <para> <literal>D</literal></para>
2434               </entry>
2435               <entry>
2436                 <para> Total size (length) of messages dropped in bytes/ Number of messages dropped</para>
2437               </entry>
2438             </row>
2439           </tbody>
2440         </tgroup>
2441       </informaltable>
2442       <para>When an <literal><replaceable>interval</replaceable></literal> is specified, additional lines of statistics are printed including the following fields:</para>
2443       <informaltable frame="all">
2444         <tgroup cols="2">
2445           <colspec colname="c1" colwidth="50*"/>
2446           <colspec colname="c2" colwidth="50*"/>
2447           <thead>
2448             <row>
2449               <entry>
2450                 <para><emphasis role="bold">Output</emphasis></para>
2451               </entry>
2452               <entry>
2453                 <para><emphasis role="bold">Description</emphasis></para>
2454               </entry>
2455             </row>
2456           </thead>
2457           <tbody>
2458             <row>
2459               <entry>
2460                 <para> <literal>M</literal></para>
2461               </entry>
2462               <entry>
2463                 <para> Number of messages currently being processed by LNet (The maximum number of messages ever processed by LNet concurrently)</para>
2464               </entry>
2465             </row>
2466             <row>
2467               <entry>
2468                 <para> <literal>E</literal></para>
2469               </entry>
2470               <entry>
2471                 <para> Number of LNet errors per second</para>
2472               </entry>
2473             </row>
2474             <row>
2475               <entry>
2476                 <para> <literal>S</literal></para>
2477               </entry>
2478               <entry>
2479                 <para> Rate of data sent in Mbytes per second/ Count of messages sent per second</para>
2480               </entry>
2481             </row>
2482             <row>
2483               <entry>
2484                 <para> <literal>R</literal></para>
2485               </entry>
2486               <entry>
2487                 <para> Rate of data received in Mbytes per second/ Count of messages received per second</para>
2488               </entry>
2489             </row>
2490             <row>
2491               <entry>
2492                 <para> <literal>F</literal></para>
2493               </entry>
2494               <entry>
2495                 <para> Rate of data routed in Mbytes per second/ Count of messages routed per second</para>
2496               </entry>
2497             </row>
2498             <row>
2499               <entry>
2500                 <para> <literal>D</literal></para>
2501               </entry>
2502               <entry>
2503                 <para> Rate of data dropped in Mbytes per second/ Count of messages dropped per second</para>
2504               </entry>
2505             </row>
2506           </tbody>
2507         </tgroup>
2508       </informaltable>
2509     </section>
2510     <section remap="h5">
2511       <title>Example</title>
2512       <screen># routerstat 1
2513 M 0(13) E 0 S 117379184/4250 R 878480/4356 F 0/0 D 0/0
2514 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2515 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2516 M   0( 13) E 0 S    8.00/     8 R    0.00/    16 F    0.00/     0 D 0.00/0
2517 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2518 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2519 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2520 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2521 M   0( 13) E 0 S    8.00/     8 R    0.00/    16 F    0.00/     0 D 0.00/0
2522 M   0( 13) E 0 S    7.00/     7 R    0.00/    14 F    0.00/     0 D 0.00/0
2523 ...</screen>
2524     </section>
2525     <section remap="h5">
2526       <title>Files</title>
2527       <para>The routerstat utility extracts statistics data from:</para>
2528       <screen>/proc/sys/lnet/stats</screen>
2529     </section>
2530   </section>
2531   <section xml:id="dbdoclet.50438219_39574">
2532     <title><indexterm><primary>tunefs.lustre</primary></indexterm>
2533 tunefs.lustre</title>
2534     <para>The tunefs.lustre utility modifies configuration information on a Lustre target disk.</para>
2535     <section remap="h5">
2536       <title>Synopsis</title>
2537       <screen>tunefs.lustre [options] <replaceable>/dev/device</replaceable></screen>
2538     </section>
2539     <section remap="h5">
2540       <title>Description</title>
2541       <para>tunefs.lustre is used to modify configuration information on a Lustre target disk. This does not reformat the disk or erase the target information, but modifying the configuration information can result in an unusable file system.</para>
2542       <caution>
2543         <para>Changes made here affect a file system only when the target is mounted the next time.</para>
2544       </caution>
2545       <para>With tunefs.lustre, parameters are &quot;additive&quot; -- new parameters are specified in addition to old parameters, they do not replace them. To erase all old tunefs.lustre parameters and just use newly-specified parameters, run:</para>
2546       <screen>$ tunefs.lustre --erase-params --param=<replaceable>new_parameters</replaceable> </screen>
2547       <para>The tunefs.lustre command can be used to set any parameter settable in a /proc/fs/lustre file and that has its own OBD device, so it can be specified as <replaceable>{obd|fsname}.obdtype.proc_file_name=value</replaceable>. For example:</para>
2548       <screen>$ tunefs.lustre --param mdt.identity_upcall=NONE /dev/sda1</screen>
2549     </section>
2550     <section remap="h5">
2551       <title>Options</title>
2552       <para>The tunefs.lustre options are listed and explained below.</para>
2553       <informaltable frame="all">
2554         <tgroup cols="2">
2555           <colspec colname="c1" colwidth="50*"/>
2556           <colspec colname="c2" colwidth="50*"/>
2557           <thead>
2558             <row>
2559               <entry>
2560                 <para><emphasis role="bold">Option</emphasis></para>
2561               </entry>
2562               <entry>
2563                 <para><emphasis role="bold">Description</emphasis></para>
2564               </entry>
2565             </row>
2566           </thead>
2567           <tbody>
2568             <row>
2569               <entry>
2570                 <para> <literal>--comment=<replaceable>comment</replaceable></literal></para>
2571               </entry>
2572               <entry>
2573                 <para> Sets a user comment about this disk, ignored by Lustre.</para>
2574               </entry>
2575             </row>
2576             <row>
2577               <entry>
2578                 <para> <literal>--dryrun</literal></para>
2579               </entry>
2580               <entry>
2581                 <para> Only prints what would be done; does not affect the disk.</para>
2582               </entry>
2583             </row>
2584             <row>
2585               <entry>
2586                 <para> <literal>--erase-params</literal></para>
2587               </entry>
2588               <entry>
2589                 <para> Removes all previous parameter information.</para>
2590               </entry>
2591             </row>
2592             <row>
2593               <entry>
2594                 <literal>--servicenode=<replaceable>nid,...</replaceable></literal></entry>
2595               <entry>Sets the NID(s) of all service nodes, including primary and failover partner
2596                 service nodes. The <literal>--servicenode</literal> option cannot be used with
2597                   <literal>--failnode</literal> option. See <xref
2598                   xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/> for
2599                 more details.</entry>
2600             </row>
2601             <row>
2602               <entry>
2603                 <para> <literal>--failnode=<replaceable>nid,...</replaceable></literal></para>
2604               </entry>
2605               <entry>
2606                 <para>Sets the NID(s) of a failover service node for a primary server for a target.
2607                   The <literal>--failnode</literal> option cannot be used with
2608                     <literal>--servicenode</literal> option. See <xref
2609                     xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/>
2610                   for more details.<note>
2611                     <para>When the <literal>--failnode</literal> option is used, certain
2612                       restrictions apply (see <xref xmlns:xlink="http://www.w3.org/1999/xlink"
2613                         linkend="dbdoclet.50438188_92688"/>).</para>
2614                   </note></para>
2615               </entry>
2616             </row>
2617             <row>
2618               <entry>
2619                 <para> <literal>--fsname=<replaceable>filesystem_name</replaceable></literal></para>
2620               </entry>
2621               <entry>
2622                 <para> The Lustre file system of which this service will be a part. The default file
2623                   system name is <literal>lustre</literal>.</para>
2624               </entry>
2625             </row>
2626             <row>
2627               <entry>
2628                 <para> <literal>--index=<replaceable>index</replaceable></literal></para>
2629               </entry>
2630               <entry>
2631                 <para> Forces a particular OST or MDT index.</para>
2632               </entry>
2633             </row>
2634             <row>
2635               <entry>
2636                 <para> <literal>--mountfsoptions=<replaceable>opts</replaceable></literal></para>
2637               </entry>
2638               <entry>
2639                 <para> Sets the mount options used when the backing file system is mounted.</para>
2640                 <warning><para> Unlike earlier versions of tunefs.lustre, this version completely replaces the existing mount options with those specified on the command line, and issues a warning on stderr if any default mount options are omitted.</para></warning>
2641                 <para>The defaults for ldiskfs are:</para>
2642                 <para>MGS/MDT: <literal>errors=remount-ro,iopen_nopriv,user_xattr</literal></para>
2643                 <para>OST: <literal>errors=remount-ro,extents,mballoc</literal></para>
2644                 <para condition='l25'>OST: <literal>errors=remount-ro</literal></para>
2645                 <para>Do not alter the default mount options unless you know what you are doing.</para>
2646               </entry>
2647             </row>
2648             <row>
2649               <entry>
2650                 <para> <literal>--network=<replaceable>net,...</replaceable></literal></para>
2651               </entry>
2652               <entry>
2653                 <para> Network(s) to which to restrict this OST/MDT. This option can be repeated as necessary.</para>
2654               </entry>
2655             </row>
2656             <row>
2657               <entry>
2658                 <para> <literal>--mgs</literal></para>
2659               </entry>
2660               <entry>
2661                 <para> Adds a configuration management service to this target.</para>
2662               </entry>
2663             </row>
2664             <row>
2665               <entry>
2666                 <para> <literal>--msgnode=<replaceable>nid,...</replaceable></literal></para>
2667               </entry>
2668               <entry>
2669                 <para> Sets the NID(s) of the MGS node; required for all targets other than the MGS.</para>
2670               </entry>
2671             </row>
2672             <row>
2673               <entry>
2674                 <para> <literal>--nomgs</literal></para>
2675               </entry>
2676               <entry>
2677                 <para> Removes a configuration management service to this target.</para>
2678               </entry>
2679             </row>
2680             <row>
2681               <entry>
2682                 <para> <literal>--quiet</literal></para>
2683               </entry>
2684               <entry>
2685                 <para> Prints less information.</para>
2686               </entry>
2687             </row>
2688             <row>
2689               <entry>
2690                 <para> <literal>--verbose</literal></para>
2691               </entry>
2692               <entry>
2693                 <para> Prints more information.</para>
2694               </entry>
2695             </row>
2696             <row>
2697               <entry>
2698                 <para> <literal>--writeconf</literal></para>
2699               </entry>
2700               <entry>
2701                 <para> Erases all configuration logs for the file system to which this MDT belongs,
2702                   and regenerates them. This is dangerous operation. All clients must be unmounted
2703                   and servers for this file system should be stopped. All targets (OSTs/MDTs) must
2704                   then be restarted to regenerate the logs. No clients should be started until all
2705                   targets have restarted.</para>
2706                 <para>The correct order of operations is:</para>
2707                 <orderedlist>
2708                   <listitem>
2709                     <para>Unmount all clients on the file system</para>
2710                   </listitem>
2711                   <listitem>
2712                     <para>Unmount the MDT and all OSTs on the file system</para>
2713                   </listitem>
2714                   <listitem>
2715                     <para>Run <literal>tunefs.lustre --writeconf
2716                         <replaceable>device</replaceable></literal> on every server</para>
2717                   </listitem>
2718                   <listitem>
2719                     <para>Mount the MDT and OSTs</para>
2720                   </listitem>
2721                   <listitem>
2722                     <para>Mount the clients</para>
2723                   </listitem>
2724                 </orderedlist>
2725               </entry>
2726             </row>
2727           </tbody>
2728         </tgroup>
2729       </informaltable>
2730     </section>
2731     <section remap="h5">
2732       <title>Examples</title>
2733       <para>Change the MGS&apos;s NID address. (This should be done on each target disk, since they should all contact the same MGS.)</para>
2734       <screen>tunefs.lustre --erase-param --mgsnode=<replaceable>new_nid</replaceable> --writeconf /dev/sda</screen>
2735       <para>Add a failover NID location for this target.</para>
2736       <screen>tunefs.lustre --param=&quot;failover.node=192.168.0.13@tcp0&quot; /dev/sda </screen>
2737     </section>
2738     <section remap="h5">
2739       <title>See Also</title>
2740       <itemizedlist>
2741         <listitem>
2742           <para><xref linkend="dbdoclet.50438219_75432"/></para>
2743         </listitem>
2744         <listitem>
2745           <para><xref linkend="dbdoclet.50438219_12635"/></para>
2746         </listitem>
2747         <listitem>
2748           <para><xref linkend="dbdoclet.50438219_38274"/></para>
2749         </listitem>
2750         <listitem>
2751           <para><xref linkend="dbdoclet.50438206_94597"/></para>
2752         </listitem>
2753       </itemizedlist>
2754     </section>
2755   </section>
2756   <section xml:id="dbdoclet.50438219_99928">
2757       <title><indexterm><primary>utilities</primary><secondary>system config</secondary></indexterm>
2758 Additional System Configuration Utilities</title>
2759     <para>This section describes additional system configuration utilities for Lustre.</para>
2760     <section remap="h3">
2761       <title><indexterm><primary>utilities</primary><secondary>application profiling</secondary></indexterm>
2762 Application Profiling Utilities</title>
2763       <para>The following utilities are located in /usr/bin.</para>
2764       <para><literal>lustre_req_history.sh</literal></para>
2765       <para>The lustre_req_history.sh utility (run from a client), assembles as much Lustre RPC request history as possible from the local node and from the servers that were contacted, providing a better picture of the coordinated network activity.</para>
2766     </section>
2767     <section remap="h3">
2768       <title>More /proc Statistics for Application Profiling</title>
2769       <para>The following utilities provide additional statistics.</para>
2770       <para><literal>vfs_ops_stats</literal></para>
2771       <para>The client vfs_ops_stats utility tracks Linux VFS operation calls into Lustre for a single PID, PPID, GID or everything.</para>
2772       <screen>/proc/fs/lustre/llite/*/vfs_ops_stats
2773 /proc/fs/lustre/llite/*/vfs_track_[pid|ppid|gid]
2774 </screen>
2775       <para><literal>extents_stats</literal></para>
2776       <para>The client extents_stats utility shows the size distribution of I/O calls from the client (cumulative and by process).</para>
2777       <screen>/proc/fs/lustre/llite/*/extents_stats, extents_stats_per_process
2778 </screen>
2779       <para><literal>offset_stats</literal></para>
2780       <para>The client offset_stats utility shows the read/write seek activity of a client by offsets and ranges.</para>
2781       <screen>/proc/fs/lustre/llite/*/offset_stats
2782 </screen>
2783       <para>Lustre includes per-client and improved MDT statistics:</para>
2784       <itemizedlist>
2785         <listitem>
2786           <para> Per-client statistics tracked on the servers</para>
2787         </listitem>
2788       </itemizedlist>
2789       <para>Each MDS and OSS now tracks LDLM and operations statistics for
2790       every connected client, for comparisons and simpler collection of
2791       distributed job statistics.</para>
2792       <screen>/proc/fs/lustre/mds|obdfilter/*/exports/
2793 </screen>
2794       <itemizedlist>
2795         <listitem>
2796           <para> Improved MDT statistics</para>
2797         </listitem>
2798       </itemizedlist>
2799       <para>More detailed MDT operations statistics are collected for better
2800       profiling.</para>
2801       <screen>/proc/fs/lustre/mdt/*/md_stats
2802 </screen>
2803     </section>
2804     <section remap="h3">
2805       <title><indexterm><primary>utilities</primary><secondary>debugging</secondary></indexterm><indexterm><primary>debug</primary><secondary>utilities</secondary></indexterm>
2806
2807 Testing / Debugging Utilities</title>
2808       <para>Lustre offers the following test and debugging utilities.</para>
2809       <section remap="h5">
2810         <title><indexterm><primary>lr_reader</primary></indexterm>
2811 lr_reader</title>
2812         <para>The lr_reader utility translates the content of the <literal>last_rcvd</literal> and <literal>reply_data</literal> files into human-readable form.</para>
2813         <para>The following utilities are part of the Lustre I/O kit. For more information, see <xref linkend="benchmarkingtests"/>.</para>
2814       </section>
2815       <section remap="h5">
2816         <title><indexterm>
2817             <primary>sgpdd-survey</primary>
2818           </indexterm> sgpdd-survey</title>
2819         <para>The <literal>sgpdd-survey</literal> utility tests &apos;bare metal&apos; performance,
2820           bypassing as much of the kernel as possible. The <literal>sgpdd-survey</literal> tool does
2821           not require Lustre, but it does require the sgp_dd package.</para>
2822         <caution>
2823           <para>The <literal>sgpdd-survey</literal> utility erases all data on the device.</para>
2824         </caution>
2825       </section>
2826       <section remap="h5">
2827         <title><indexterm>
2828             <primary>obdfilter-survey</primary>
2829           </indexterm>obdfilter-survey</title>
2830         <para>The <literal>obdfilter-survey</literal> utility is a shell script that tests
2831           performance of isolated OSTS, the network via echo clients, and an end-to-end test.</para>
2832       </section>
2833       <section remap="h5">
2834         <title><indexterm><primary>ior-survey</primary></indexterm>ior-survey</title>
2835         <para>The ior-survey utility is a script used to run the IOR benchmark. Lustre includes IOR version 2.8.6.</para>
2836       </section>
2837       <section remap="h5">
2838         <title><indexterm>
2839             <primary>ost-survey</primary>
2840           </indexterm>ost-survey</title>
2841         <para>The <literal>ost-survey</literal> utility is an OST performance survey that tests
2842           client-to-disk performance of the individual OSTs in a Lustre file system.</para>
2843       </section>
2844       <section remap="h5">
2845         <title><indexterm><primary>stats-collect</primary></indexterm>stats-collect</title>
2846         <para>The stats-collect utility contains scripts used to collect application profiling information from Lustre clients and servers.</para>
2847       </section>
2848     </section>
2849     <section remap="h3" condition='l29'>
2850       <title><indexterm><primary>fileset</primary></indexterm>Fileset Feature</title>
2851       <para> With the fileset feature, Lustre now provides subdirectory mount
2852       support.  Subdirectory mounts, also referred to as filesets, allow a
2853       client to mount a child directory of a parent filesystem, thereby limiting
2854       the filesystem namespace visibility on a specific client.  A common use
2855       case is for a client to use a subdirectory mount when there is a desire to
2856       limit the visibility of the entire filesystem namesapce to aid in the
2857       prevention of accidental file deletions outside of the subdirectory
2858       mount.</para>
2859       <para>It is important to note that invocation of the subdirectory mount is
2860       voluntary by the client and not does prevent access to files that are
2861       visible in multiple subdirectory mounts via hard links.  Furthermore, it
2862       does not prevent the client from subsequently mounting the whole file
2863       system without a subdirectory being specified.</para>
2864       <figure xml:id="understandinglustre.fig.fileset">
2865         <title>
2866         <indexterm>
2867           <primary>Lustre</primary>
2868           <secondary>fileset</secondary>
2869         </indexterm>Lustre fileset</title>
2870         <mediaobject>
2871           <imageobject>
2872             <imagedata scalefit="1" width="100%"
2873             fileref="./figures/fileset.png" />
2874           </imageobject>
2875           <textobject>
2876             <phrase>Lustre file system fileset feature</phrase>
2877           </textobject>
2878         </mediaobject>
2879       </figure>
2880       <section remap="h4">
2881         <title>Examples</title>
2882         <para>The following example will mount the
2883         <literal>chipfs</literal> filesystem on client1 and create a
2884         subdirectory <literal>v1_1</literal> within that filesystem.  Client2
2885         will then mount only the <literal>v1_1</literal> subdirectory as a
2886         fileset, thereby limiting access to anything else in the
2887         <literal>chipfs</literal> filesystem from client2.</para>
2888         <screen>client1# mount -t lustre mgs@tcp:/chipfs /mnt/chip
2889 client1# mkdir /mnt/chip/v1_1</screen>
2890         <screen>client2# mount -t lustre mgs@tcp:/chipfs/v1_1 /mnt/chipv1_1</screen>
2891     <para>You can check the created mounts in /etc/mtab. It should look like
2892     the following:</para>
2893     <screen><replaceable>client1</replaceable>
2894 mds@tcp0:/chipfs/ /mnt/chip lustre rw         0       0
2895 </screen><screen>
2896 <replaceable>client2</replaceable>
2897 mds@tcp0:/chipfs/v1_1 /mnt/chipv1_1 lustre rw         0       0</screen>
2898         <para>Create a directory under the /mnt/chip mount, and get its FID</para>
2899     <screen>client1# mkdir /mnt/chip/v1_2
2900 client1# lfs path2fid /mnt/chip/v1_2
2901 [0x200000400:0x2:0x0]
2902 </screen>
2903         <para>If you try resolve the FID of the <literal>/mnt/chip/v1_2</literal>
2904     path (as created in the example above) on client2, an error will be returned
2905     as the FID can not be resolved on client2 since it is not part of the
2906     mounted fileset on that client.  Recall that the fileset on client2 mounted
2907     the <literal>v1_1</literal> subdirectory beneath the top level
2908     <literal>chipfs</literal> filesystem.
2909     </para>
2910     <screen>client2# lfs fid2path /mnt/chip/v1_2 [0x200000400:0x2:0x0]
2911 fid2path: error on FID [0x200000400:0x2:0x0]: No such file or directory</screen>
2912     <para>Subdirectory mounts do not have the <literal>.lustre</literal>
2913     pseudo directory, which prevents clients from opening or accessing files
2914     only by FID.</para>
2915     <screen>client1# ls /mnt/chipfs/.lustre
2916         fid  lost+found</screen>
2917     <screen>client2# ls /mnt/chipv1_1/.lustre
2918         ls: cannot access /mnt/chipv1_1/.lustre: No such file or directory
2919     </screen>
2920       </section>
2921     </section>
2922   </section>
2923 </chapter>