<?xml version='1.0' encoding='UTF-8'?>
-<!-- This document was created with Syntext Serna Free. -->
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="systemconfigurationutilities">
<title xml:id="systemconfigurationutilities.title">System Configuration Utilities</title>
<para>This chapter includes system configuration utilities and includes the following sections:</para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">-b inode buffer blocks</emphasis></para>
+ <para> <literal>-b <replaceable>inode buffer blocks</replaceable></literal></para>
</entry>
<entry>
<para> Sets the readahead inode blocks to get excellent performance when scanning the block device.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-o output file</emphasis></para>
+ <para> <literal>-o <replaceable>output file</replaceable></literal></para>
</entry>
<entry>
<para> If an output file is specified, modified pathnames are written to this file. Otherwise, modified parameters are written to stdout.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-t inode | pathname</emphasis></para>
+ <para> <literal>-t <replaceable>inode</replaceable>| <replaceable>pathname</replaceable></literal></para>
</entry>
<entry>
<para> Sets the e2scan type if type is inode. The e2scan utility prints modified inode numbers to stdout. By default, the type is set as pathname.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-u</emphasis></para>
+ <para> <literal>-u</literal></para>
</entry>
<entry>
<para> Rebuilds the parent database from scratch. Otherwise, the current parent database is used.</para>
<para>The l_getidentity utility handles Lustre user / group cache upcall.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>l_getidentity {mdtname} {uid}</screen>
+ <screen>l_getidentity ${FSNAME}-MDT{xxxx} {uid}</screen>
</section>
<section remap="h5">
<title>Description</title>
- <para>The group upcall file contains the path to an executable file that, when properly installed, is invoked to resolve a numeric UID to a group membership list. This utility should complete the mds_grp_downcall_data structure and write it to the /proc/fs/lustre/mdt/${FSNAME}-MDT{xxxx}/identity_info pseudo-file.</para>
+ <para>The group upcall file contains the path to an executable file that is invoked to resolve
+ a numeric UID to a group membership list. This utility opens
+ <literal>/proc/fs/lustre/mdt/${FSNAME}-MDT{xxxx}/identity_info</literal> and writes the
+ related <literal>identity_downcall_data</literal> structure (see <xref
+ linkend="dbdoclet.50438291_33759"/>.) The data is persisted with <literal>lctl set_param
+ mdt.${FSNAME}-MDT{xxxx}.identity_info</literal>.</para>
<para>The l_getidentity utility is the reference implementation of the user or group cache upcall.</para>
</section>
<section remap="h5">
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">mdtname</emphasis></para>
+ <para>
+ <literal>${FSNAME}-MDT{xxxx}</literal></para>
</entry>
<entry>
<para> Metadata server target name</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">uid</emphasis></para>
+ <para> <literal>uid</literal></para>
</entry>
<entry>
<para> User identifier</para>
<para>The lctl utility is used for root control and configuration. With lctl you can directly control Lustre via an ioctl interface, allowing various configuration, maintenance and debugging features to be accessed.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>lctl
-lctl --device <devno> <command [args]></screen>
+ <screen>lctl [--device <replaceable>devno</replaceable>] <replaceable>command [args]</replaceable></screen>
</section>
<section remap="h5">
<title>Description</title>
<screen>dl
dk
device
-network <<emphasis>up/down</emphasis>>
+network <replaceable>up|down</replaceable>
list_nids
-ping <emphasis>nid</emphasis>help
+ping <replaceable>nid</replaceable>help
quit</screen>
- <para>For a complete list of available commands, type help at the lctl prompt. To get basic help on command meaning and syntax, type help<emphasis>command</emphasis>. Command completion is activated with the TAB key, and command history is available via the up- and down-arrow keys.</para>
+ <para>For a complete list of available commands, type <literal>help</literal> at the <literal>lctl</literal> prompt. To get basic help on command meaning and syntax, type <literal>help <replaceable>command</replaceable></literal>. Command completion is activated with the TAB key (depending on compile options), and command history is available via the up- and down-arrow keys.</para>
<para>For non-interactive use, use the second invocation, which runs the command after connecting to the device.</para>
</section>
<section remap="h5">
<title>Setting Parameters with lctl</title>
<para>Lustre parameters are not always accessible using the procfs interface, as it is platform-specific. As a solution, lctl {get,set}_param has been introduced as a platform-independent interface to the Lustre tunables. Avoid direct references to /proc/{fs,sys}/{lustre,lnet}. For future portability, use lctl {get,set}_param .</para>
- <para>When the file system is running, use the lctl set_param command to set temporary parameters (mapping to items in /proc/{fs,sys}/{lnet,lustre}). The lctl set_param command uses this syntax:</para>
- <screen>lctl set_param [-n] <obdtype>.<obdname>.<proc_file_name>=<value></screen>
+ <para>When the file system is running, use the <literal>lctl set_param</literal> command on the affected node(s) to <emphasis>temporarily</emphasis> set parameters (mapping to items in /proc/{fs,sys}/{lnet,lustre}). The <literal>lctl set_param</literal> command uses this syntax:</para>
+ <screen>lctl set_param [-n] [-P] [-d] <replaceable>obdtype.obdname.property</replaceable>=<replaceable>value</replaceable></screen>
<para>For example:</para>
- <screen>$ lctl set_param ldlm.namespaces.*osc*.lru_size=$((NR_CPU*100))</screen>
- <para>Many permanent parameters can be set with lctl conf_param. In general, lctl conf_param can be used to specify any parameter settable in a /proc/fs/lustre file, with its own OBD device. The lctl conf_param command uses this syntax:</para>
- <screen><obd|fsname>.<obdtype>.<proc_file_name>=<value>) </screen>
+ <screen>mds# lctl set_param mdt.testfs-MDT0000.identity_upcall=NONE</screen>
+ <para condition='l25'>Use <literal>-P</literal> option to set parameters permanently. Option <literal>-d </literal>deletes permanent parameters. For example:
+ <screen>mgs# lctl set_param -P mdt.testfs-MDT0000.identity_upcall=NONE
+mgs# lctl set_param -P -d mdt.testfs-MDT0000.identity_upcall</screen></para>
+ <para>Many permanent parameters can be set with <literal>lctl conf_param</literal>. In general, <literal>lctl conf_param</literal> can be used to specify any OBD device parameter settable in a /proc/fs/lustre file. The <literal>lctl conf_param</literal> command must be run on the MGS node, and uses this syntax:</para>
+ <screen><replaceable>obd|fsname</replaceable>.obdtype.property=<replaceable>value</replaceable>) </screen>
<para>For example:</para>
- <screen>$ lctl conf_param testfs-MDT0000.mdt.group_upcall=NONE
+ <screen>mgs# lctl conf_param testfs-MDT0000.mdt.identity_upcall=NONE
$ lctl conf_param testfs.llite.max_read_ahead_mb=16 </screen>
<caution>
- <para>The lctlconf_param command permanently sets parameters in the file system configuration.</para>
+ <para>The <literal>lctl conf_param</literal> command <emphasis>permanently</emphasis> sets parameters in the file system configuration for all nodes of the specified type.</para>
</caution>
- <para>To get current Lustre parameter settings, use the lctl get_param command with this syntax:</para>
- <screen>lctl get_param [-n] <obdtype>.<obdname>.<proc_file_name></screen>
+ <para>To get current Lustre parameter settings, use the <literal>lctl get_param</literal> command on the desired node with the same parameter name as <literal>lctl set_param</literal>:</para>
+ <screen>lctl get_param [-n] <replaceable>obdtype.obdname.parameter</replaceable></screen>
<para>For example:</para>
- <screen>$ lctl get_param -n ost.*.ost_io.timeouts </screen>
- <para>To list Lustre parameters that are available to set, use the lctl list_param command, with this syntax:</para>
- <screen>lctl list_param [-n] <obdtype>.<obdname></screen>
- <para>For example:</para>
- <screen>$ lctl list_param obdfilter.lustre-OST0000</screen>
- <para>For more information on using lctl to set temporary and permanent parameters, see <xref linkend="dbdoclet.50438194_51490"/> (Setting Parameters with lctl).</para>
+ <screen>mds# lctl get_param mdt.testfs-MDT0000.identity_upcall</screen>
+ <para>To list Lustre parameters that are available to set, use the <literal>lctl list_param</literal> command, with this syntax:</para>
+ <screen>lctl list_param [-R] [-F] <replaceable>obdtype.obdname.*</replaceable></screen>
+ <para>For example, to list all of the parameters on the MDT:</para>
+ <screen>oss# lctl list_param -RF mdt</screen>
+ <para>For more information on using lctl to set temporary and permanent parameters, see <xref linkend="dbdoclet.50438194_51490"/>.</para>
<para><emphasis role="bold">Network Configuration</emphasis></para>
<informaltable frame="all">
<tgroup cols="2">
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">network</emphasis> <<emphasis>up/down</emphasis>>|<<emphasis>tcp/elan/myrinet</emphasis>></para>
+ <para> <literal>network up|down|tcp|elan</literal></para>
</entry>
<entry>
- <para> Starts or stops LNET, or selects a network type for other <emphasis role="bold">lctl</emphasis> LNET commands.</para>
+ <para> Starts or stops LNet, or selects a network type for other <literal>lctl</literal> LNet commands.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">list_nids</emphasis></para>
+ <para> <literal>list_nids</literal></para>
</entry>
<entry>
- <para> Prints all NIDs on the local node. LNET must be running.</para>
+ <para> Prints all NIDs on the local node. LNet must be running.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">which_nid</emphasis> <<emphasis>nidlist</emphasis>></para>
+ <para> <literal>which_nid <replaceable>nidlist</replaceable></literal></para>
</entry>
<entry>
<para> From a list of NIDs for a remote node, identifies the NID on which interface communication will occur.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">ping</emphasis> <<emphasis>nid</emphasis>></para>
+ <para> <literal>ping <replaceable>nid</replaceable></literal></para>
</entry>
<entry>
- <para> Checks LNET connectivity via an LNET ping. This uses the fabric appropriate to the specified NID.</para>
+ <para> Checks LNet connectivity via an LNet ping. This uses the fabric appropriate to the specified NID.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">interface_list</emphasis></para>
+ <para> <literal>interface_list</literal></para>
</entry>
<entry>
- <para> Prints the network interface information for a given <emphasis role="bold">network</emphasis> type.</para>
+ <para> Prints the network interface information for a given <emphasis>network</emphasis> type.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">peer_list</emphasis></para>
+ <para> <literal>peer_list</literal></para>
</entry>
<entry>
- <para> Prints the known peers for a given <emphasis role="bold">network</emphasis> type.</para>
+ <para> Prints the known peers for a given <emphasis>network</emphasis> type.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">conn_list</emphasis></para>
+ <para> <literal>conn_list</literal></para>
</entry>
<entry>
- <para> Prints all the connected remote NIDs for a given <emphasis role="bold">network</emphasis> type.</para>
+ <para> Prints all the connected remote NIDs for a given <emphasis>network</emphasis> type.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">active_tx</emphasis></para>
+ <para> <literal>active_tx</literal></para>
</entry>
<entry>
- <para> This command prints active transmits. It is only used for the Elan <emphasis role="bold">network</emphasis> type.</para>
+ <para> This command prints active transmits. It is only used for the Elan <emphasis>network</emphasis> type.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">route_list</emphasis></para>
+ <para> <literal>route_list</literal></para>
</entry>
<entry>
<para> Prints the complete routing table.</para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">device</emphasis> <<emphasis>devname</emphasis>></para>
+ <para> <literal>device <replaceable>devname</replaceable></literal></para>
</entry>
<entry>
<para>  </para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">device_list</emphasis></para>
+ <para> <literal>device_list</literal></para>
</entry>
<entry>
<para>  </para>
</entry>
<entry>
- <para> Shows the local Lustre OBDs, a/k/a <emphasis role="bold">dl</emphasis>.</para>
+ <para> Shows the local Lustre OBDs, a/k/a <literal>dl</literal>.</para>
</entry>
</row>
</tbody>
<tbody>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">list_param</emphasis><emphasis>[-F|-R]</emphasis> <<emphasis>param_path ...</emphasis>></para>
+ <para> <literal>list_param [-F|-R] <replaceable>parameter</replaceable> <replaceable>[parameter ...]</replaceable></literal></para>
</entry>
<entry>
- <para> Lists the Lustre or LNET parameter name.</para>
+ <para> Lists the Lustre or LNet parameter name.</para>
<para> </para>
</entry>
</row>
<para>  </para>
</entry>
<entry>
- <para> -<emphasis role="bold">F</emphasis></para>
+ <para> <literal>-F</literal></para>
</entry>
<entry>
<para> Adds '/', '@' or '=' for directories, symlinks and writeable files, respectively.</para>
<para>  </para>
</entry>
<entry>
- <para> -<emphasis role="bold">R</emphasis></para>
+ <para> <literal>-R</literal></para>
</entry>
<entry>
- <para> Recursively lists all parameters under the specified path. If <emphasis>param_path</emphasis> is unspecified, all parameters are shown.</para>
+ <para> Recursively lists all parameters under the specified path. If <literal>param_path</literal> is unspecified, all parameters are shown.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">get_param</emphasis><emphasis>[-n|-N|-F]</emphasis> <<emphasis>param_path ...</emphasis>></para>
+ <para> <literal>get_param [-n|-N|-F] <replaceable>parameter</replaceable> <replaceable>[parameter ...]</replaceable></literal></para>
</entry>
<entry>
- <para> Gets the value of a Lustre or LNET parameter from the specified path.</para>
+ <para> Gets the value of a Lustre or LNet parameter from the specified path.</para>
</entry>
</row>
<row>
<para>  </para>
</entry>
<entry>
- <para> <emphasis role="bold">-n</emphasis></para>
+ <para> <literal>-n</literal></para>
</entry>
<entry>
<para> Prints only the parameter value and not the parameter name.</para>
<para>  </para>
</entry>
<entry>
- <para> <emphasis role="bold">-N</emphasis></para>
+ <para> <literal>-N</literal></para>
</entry>
<entry>
<para> Prints only matched parameter names and not the values; especially useful when using patterns.</para>
<para>  </para>
</entry>
<entry>
- <para> <emphasis role="bold">-F</emphasis></para>
+ <para> <literal>-F</literal></para>
</entry>
<entry>
- <para> When <emphasis role="bold">-N</emphasis> is specified, adds '/', '@' or '=' for directories, symlinks and writeable files, respectively.</para>
+ <para> When <literal>-N</literal> is specified, adds '/', '@' or '=' for directories, symlinks and writeable files, respectively.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">set_param</emphasis><emphasis>[-n]</emphasis><emphasis><param_path=value...></emphasis></para>
+ <para> <literal>set_param [-n] <replaceable>parameter</replaceable>=<replaceable>value</replaceable></literal></para>
</entry>
<entry>
- <para> Sets the value of a Lustre or LNET parameter from the specified path.</para>
+ <para> Sets the value of a Lustre or LNet parameter from the specified path.</para>
</entry>
</row>
<row>
<para>  </para>
</entry>
<entry>
- <para> <emphasis role="bold">-n</emphasis></para>
+ <para> <literal>-n</literal></para>
</entry>
<entry>
<para> Disables printing of the key name when printing values.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">conf_param</emphasis><emphasis>[-d] <device|fsname>.<parameter>=<value></emphasis></para>
+ <para><literal>conf_param [-d] <replaceable>device|fsname</replaceable> <replaceable>parameter</replaceable>=<replaceable>value</replaceable></literal></para>
</entry>
<entry>
<para> Sets a permanent configuration parameter for any device via the MGS. This command must be run on the MGS node.</para>
- <para>All writeable parameters under <emphasis role="bold">lctl list_param</emphasis> (e.g. <emphasis>lctl list_param -F osc.*.* | grep</emphasis> =) can be permanently set using <emphasis role="bold">lctl conf_param</emphasis>, but the format is slightly different. For <emphasis role="bold">conf_param</emphasis>, the device is specified first, then the obdtype. Wildcards are not supported. Additionally, failover nodes may be added (or removed), and some system-wide parameters may be set as well (sys.at_max, sys.at_min, sys.at_extra, sys.at_early_margin, sys.at_history, sys.timeout, sys.ldlm_timeout). For system-wide parameters, <device> is ignored.</para>
- <para>For more information on setting permanent parameters and <emphasis role="bold">lctl conf_param</emphasis> command examples, see <xref linkend="dbdoclet.50438194_64195"/> (Setting Permanent Parameters).</para>
+ <para>All writeable parameters under <literal>lctl list_param</literal> (e.g. <literal>lctl list_param -F osc.*.* | grep</literal> =) can be permanently set using <literal>lctl conf_param</literal>, but the format is slightly different. For <literal>conf_param</literal>, the device is specified first, then the obdtype. Wildcards are not supported. Additionally, failover nodes may be added (or removed), and some system-wide parameters may be set as well (sys.at_max, sys.at_min, sys.at_extra, sys.at_early_margin, sys.at_history, sys.timeout, sys.ldlm_timeout). For system-wide parameters, <replaceable>device</replaceable> is ignored.</para>
+ <para>For more information on setting permanent parameters and <literal>lctl conf_param</literal> command examples, see <xref linkend="dbdoclet.50438194_64195"/> (Setting Permanent Parameters).</para>
</entry>
</row>
<row>
<para>  </para>
</entry>
<entry>
- <para> <emphasis role="bold">-d <device|fsname>.<parameter></emphasis></para>
+ <para><literal>-d <replaceable>device|fsname</replaceable>.<replaceable>parameter</replaceable></literal></para>
<para> </para>
</entry>
<entry>
- <para> Deletes a parameter setting (use the default value at the next restart). A null value for <emphasis role="bold"><value></emphasis> also deletes the parameter setting.</para>
+ <para> Deletes a parameter setting (use the default value at the next restart). A null value for <replaceable>value</replaceable> also deletes the parameter setting.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">activate</emphasis></para>
+ <para> <literal>activate</literal></para>
</entry>
<entry>
- <para> Re-activates an import after the deactivate operation. This setting is only effective until the next restart (see <emphasis role="bold">conf_param</emphasis>).</para>
+ <para> Re-activates an import after the deactivate operation. This setting is only effective until the next restart (see <literal>conf_param</literal>).</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">deactivate</emphasis></para>
+ <para> <literal>deactivate</literal></para>
</entry>
<entry>
<para> Deactivates an import, in particular meaning do not assign new file stripes to an OSC. Running lctl deactivate on the MDS stops new objects from being allocated on the OST. Running lctl deactivate on Lustre clients causes them to return -EIO when accessing objects on the OST instead of waiting for recovery.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">abort_recovery</emphasis></para>
+ <para> <literal>abort_recovery</literal></para>
</entry>
<entry>
<para> Aborts the recovery process on a re-starting MDT or OST.</para>
</tgroup>
</informaltable>
<note>
- <para>Lustre tunables are not always accessible using the procfs interface, as it is platform-specific. As a solution, lctl {get,set,list}_param has been introduced as a platform-independent interface to the Lustre tunables. Avoid direct references to /proc/{fs,sys}/{lustre,lnet}. For future portability, use lctl {get,set,list}_param instead.</para>
+ <para>Lustre tunables are not always accessible using the procfs interface, as it is platform-specific. As a solution, <literal>lctl {get,set,list}_param</literal> has been introduced as a platform-independent interface to the Lustre tunables. Avoid direct references to <literal>/proc/{fs,sys}/{lustre,lnet}</literal>. For future portability, use <literal>lctl {get,set,list}_param</literal> instead.</para>
</note>
<para><emphasis role="bold">Virtual Block Device Operations</emphasis></para>
<para>Lustre can emulate a virtual block device upon a regular file. This emulation is needed when you are trying to set up a swap space via the file.</para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">blockdev_attach</emphasis><emphasis><file name> <device node></emphasis></para>
+ <para><literal>blockdev_attach <replaceable>filename</replaceable> <replaceable>/dev/lloop_device</replaceable></literal></para>
</entry>
<entry>
- <para> Attaches a regular Lustre file to a block device. If the device node does not exist, <emphasis role="bold">lctl</emphasis> creates it. We recommend that you create the device node by <emphasis role="bold">lctl</emphasis> since the emulator uses a dynamical major number.</para>
+ <para> Attaches a regular Lustre file to a block device. If the device node does not exist, <literal>lctl</literal> creates it. It is recommend that a device node is created by <literal>lctl</literal> since the emulator uses a dynamical major number.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">blockdev_detach</emphasis><emphasis><device node></emphasis></para>
+ <para><literal>blockdev_detach <replaceable>/dev/lloop_device</replaceable></literal></para>
</entry>
<entry>
<para> Detaches the virtual block device.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">blockdev_info</emphasis><emphasis><device node></emphasis></para>
+ <para><literal>blockdev_info <replaceable>/dev/lloop_device</replaceable></literal></para>
</entry>
<entry>
<para> Provides information about the Lustre file attached to the device node.</para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">changelog_register</emphasis></para>
+ <para> <literal>changelog_register</literal></para>
</entry>
<entry>
- <para> Registers a new changelog user for a particular device. Changelog entries are not purged beyond a registered user's set point (see <emphasis role="bold">lfs changelog_clear</emphasis>).</para>
+ <para> Registers a new changelog user for a particular device.
+ Changelog entries are saved persistently on the MDT with each
+ filesystem operation, and are only purged beyond all registered
+ user's minimum set point (see
+ <literal>lfs changelog_clear</literal>). This may cause the
+ Changelog to consume a large amount of space, eventually
+ filling the MDT, if a changelog user is registered but never
+ consumes those records.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">changelog_deregister</emphasis><emphasis><id></emphasis></para>
+ <para>changelog_deregister <replaceable>id</replaceable></para>
</entry>
<entry>
- <para> Unregisters an existing changelog user. If the user's "clear" record number is the minimum for the device, changelog records are purged until the next minimum.</para>
+ <para> Unregisters an existing changelog user. If the
+ user's "clear" record number is the minimum for
+ the device, changelog records are purged until the next minimum.
+ </para>
</entry>
</row>
</tbody>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">debug_daemon</emphasis></para>
+ <para> <literal>debug_daemon</literal></para>
</entry>
<entry>
<para> Starts and stops the debug daemon, and controls the output filename and size.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">debug_kernel</emphasis><emphasis>[file] [raw]</emphasis></para>
+ <para> <literal>debug_kernel <replaceable>[file]</replaceable> [raw]</literal></para>
</entry>
<entry>
<para> Dumps the kernel debug buffer to stdout or a file.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">debug_file</emphasis><emphasis><input> [output]</emphasis></para>
+ <para><literal>debug_file <replaceable>input_file</replaceable> <replaceable>[output_file]</replaceable></literal></para>
</entry>
<entry>
<para> Converts the kernel-dumped debug log from binary to plain text format.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">clear</emphasis></para>
+ <para> <literal>clear</literal></para>
</entry>
<entry>
<para> Clears the kernel debug buffer.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">mark</emphasis><emphasis><text></emphasis></para>
+ <para> <literal>mark <replaceable>text</replaceable></literal></para>
</entry>
<entry>
<para> Inserts marker text in the kernel debug buffer.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">filter</emphasis><emphasis><subsystem id/debug mask></emphasis></para>
+ <para> <literal>filter <replaceable>subsystem_id|debug_mask</replaceable></literal></para>
</entry>
<entry>
<para> Filters kernel debug messages by subsystem or mask.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">show</emphasis><emphasis><subsystem id/debug mask></emphasis></para>
+ <para> <literal>show <replaceable>subsystem_id|debug_mask</replaceable></literal></para>
</entry>
<entry>
<para> Shows specific types of messages.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">debug_list</emphasis><emphasis><subs/types></emphasis></para>
+ <para> <literal>debug_list <replaceable>subsystems|types</replaceable></literal></para>
</entry>
<entry>
<para> Lists all subsystem and debug types.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">modules</emphasis><emphasis><path></emphasis></para>
+ <para> <literal>modules <replaceable>path</replaceable></literal></para>
</entry>
<entry>
<para> Provides GDB-friendly module information.</para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">--device</emphasis></para>
+ <para> <literal>--device</literal></para>
</entry>
<entry>
<para> Device to be used for the operation (specified by name or number). See device_list.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--ignore_errors | ignore_errors</emphasis></para>
+ <para> <literal>--ignore_errors | ignore_errors</literal></para>
</entry>
<entry>
<para> Ignores errors during script processing.</para>
</section>
<section remap="h5">
<title>Examples</title>
- <para><emphasis role="bold">lctl</emphasis></para>
+ <para><literal>lctl</literal></para>
<screen>$ lctl
lctl > dl
0 UP mgc MGC192.168.0.20@tcp btbb24e3-7deb-2ffa-eab0-44dffe00f692 5
</section>
<section remap="h5">
<title>Description</title>
- <para>The ll_decode_filter_fid utility decodes and prints the Lustre OST object ID, MDT FID, stripe index for the specified OST object(s), which is stored in the "trusted.fid" attribute on each OST object. This is accessible to ll_decode_filter_fid when the OST filesystem is mounted locally as type ldiskfs for maintenance.</para>
+ <para>The ll_decode_filter_fid utility decodes and prints the Lustre OST object ID, MDT FID,
+ stripe index for the specified OST object(s), which is stored in the "trusted.fid"
+ attribute on each OST object. This is accessible to <literal>ll_decode_filter_fid</literal>
+ when the OST file system is mounted locally as type ldiskfs for maintenance.</para>
<para>The "trusted.fid" extended attribute is stored on each OST object when it is first modified (data written or attributes set), and is not accessed or modified by Lustre after that time.</para>
<para>The OST object ID (objid) is useful in case of OST directory corruption, though normally the ll_recover_lost_found_objs(8) utility is able to reconstruct the entire OST object directory hierarchy. The MDS FID can be useful to determine which MDS inode an OST object is (or was) used by. The stripe index can be used in conjunction with other OST objects to reconstruct the layout of a file even if the MDT inode was lost.</para>
</section>
#123455: objid=614725 seq=0 parent=[0x18d11:0xebba84eb:0x1]
#123458: objid=533088 seq=0 parent=[0x21417:0x19734d61:0x0]</screen>
<para>This shows that the three files in lost+found have decimal object IDs - 690670, 614725, and 533088, respectively. The object sequence number (formerly object group) is 0 for all current OST objects.</para>
- <para>The MDT parent inode FIDs are hexdecimal numbers of the form sequence:oid:idx. Since the sequence number is below 0x100000000 in all these cases, the FIDs are in the legacy Inode and Generation In FID (IGIF) namespace and are mapped directly to the MDT inode = seq and generation = oid values; the MDT inodes are 0x751c5, 0x18d11, and 0x21417 respectively. For objects with MDT parent sequence numbers above 0x200000000, this indicates that the FID needs to be mapped via the MDT Object Index (OI) file on the MDT to determine the internal inode number.</para>
+ <para>The MDT parent inode FIDs are hexadecimal numbers of the form sequence:oid:idx. Since the sequence number is below 0x100000000 in all these cases, the FIDs are in the legacy Inode and Generation In FID (IGIF) namespace and are mapped directly to the MDT inode = seq and generation = oid values; the MDT inodes are 0x751c5, 0x18d11, and 0x21417 respectively. For objects with MDT parent sequence numbers above 0x200000000, this indicates that the FID needs to be mapped via the MDT Object Index (OI) file on the MDT to determine the internal inode number.</para>
<para>The idx field shows the stripe number of this OST object in the Lustre RAID-0 striped file.</para>
</section>
<section remap="h5">
<para><xref linkend="dbdoclet.50438219_44971"/></para>
</section>
</section>
- <section xml:id="dbdoclet.50438219_44971">
+ <section xml:id="dbdoclet.50438219_44971" condition='l28'>
<title><indexterm><primary>ll_recover_lost_found_objs</primary></indexterm>
ll_recover_lost_found_objs</title>
- <para>The ll_recover_lost_found_objs utility helps recover Lustre OST objects (file data) from a lost and found directory and return them to their correct locations.</para>
- <note>
- <para>Running the ll_recover_lost_found_objs tool is not strictly necessary to bring an OST back online, it just avoids losing access to objects that were moved to the lost and found directory due to directory corruption.</para>
+ <para>The <literal>ll_recover_lost_found_objs</literal> utility was
+ used to help recover Lustre OST objects (file data) from the
+ <literal>lost+found</literal> directory of an OST and return them to
+ their correct locations based on information stored in the
+ <literal>trusted.fid</literal> extended attribute stored on every
+ OST object containing data.</para>
+ <note condition="l26"><para>This utility is not needed with Lustre 2.6
+ and later, and is removed in Lustre 2.8 since <literal>LFSCK</literal>
+ online scanning will automatically move objects from
+ <literal>lost+found</literal> to the proper place in the OST.</para>
+ </note>
+ <note condition='l25'>
+ <para>The <literal>ll_recover_lost_found_objs</literal> tool is not
+ strictly necessary to bring an OST back online, it just avoids losing
+ access to objects that were moved to the lost+found directory due to
+ directory corruption on the OST.</para>
</note>
<section remap="h5">
<title>Synopsis</title>
</section>
<section remap="h5">
<title>Description</title>
- <para>The first time Lustre writes to an object, it saves the MDS inode number and the objid as an extended attribute on the object, so in case of directory corruption of the OST, it is possible to recover the objects. Running e2fsck fixes the corrupted OST directory, but it puts all of the objects into a lost and found directory, where they are inaccessible to Lustre. Use the ll_recover_lost_found_objs utility to recover all (or at least most) objects from a lost and found directory and return them to the O/0/d* directories.</para>
- <para>To use ll_recover_lost_found_objs, mount the file system locally (using the -t ldiskfs command), run the utility and then unmount it again. The OST must not be mounted by Lustre when ll_recover_lost_found_objs is run.</para>
+ <para>The first time Lustre modifies an object, it saves the MDS inode number and the objid as an extended attribute on the object, so in case of directory corruption of the OST, it is possible to recover the objects. Running e2fsck fixes the corrupted OST directory, but it puts all of the objects into a lost and found directory, where they are inaccessible to Lustre. Use the ll_recover_lost_found_objs utility to recover all (or at least most) objects from a lost and found directory and return them to the O/0/d* directories.</para>
+ <para>To use ll_recover_lost_found_objs, mount the file system locally (using the <literal>-t ldiskfs</literal>, or <literal>-t zfs</literal> command), run the utility and then unmount it again. The OST must not be mounted by Lustre when ll_recover_lost_found_objs is run.</para>
</section>
<section remap="h5">
<title>Options</title>
<para><emphasis role="bold">Option</emphasis></para>
</entry>
<entry>
- <para><emphasis role="bold"> Description</emphasis></para>
+ <para><emphasis role="bold">Description</emphasis></para>
</entry>
</row>
</thead>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">-h</emphasis></para>
+ <para> <literal>-h</literal></para>
</entry>
<entry>
<para> Prints a help message</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-v</emphasis></para>
+ <para> <literal>-v</literal></para>
</entry>
<entry>
<para> Increases verbosity</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-d</emphasis><emphasis> directory</emphasis></para>
+ <para> <literal>-d <replaceable>directory</replaceable></literal></para>
</entry>
<entry>
<para> Sets the lost and found directory path</para>
</section>
<section remap="h5">
<title>Description</title>
- <para>The llobdstat utility displays a line of OST statistics for the given ost_name every interval seconds. It should be run directly on an OSS node. Type <emphasis>CTRL-C</emphasis> to stop statistics printing.</para>
+ <para>The llobdstat utility displays a line of OST statistics for the given ost_name every interval seconds. It should be run directly on an OSS node. Type <literal>CTRL-C</literal> to stop statistics printing.</para>
</section>
<section remap="h5">
<title>Example</title>
</section>
<section remap="h5">
<title>Files</title>
- <screen>/proc/fs/lustre/obdfilter/<ostname>/stats</screen>
+ <screen>/proc/fs/lustre/obdfilter/<replaceable>ostname</replaceable>/stats</screen>
</section>
</section>
<section xml:id="dbdoclet.50438219_90386">
<title><indexterm><primary>llog_reader</primary></indexterm>
llog_reader</title>
- <para>The llog_reader utility parses Lustre's on-disk configuration logs.</para>
+ <para>The llog_reader utility translates a Lustre configuration log into human-readable form.</para>
<section remap="h5">
<title>Synopsis</title>
<screen>llog_reader filename</screen>
<section remap="h5">
<title>Description</title>
<para>The llog_reader utility parses the binary format of Lustre's on-disk configuration logs. Llog_reader can only read logs; use tunefs.lustre to write to them.</para>
- <para>To examine a log file on a stopped Lustre server, mount its backing file system as ldiskfs, then use llog_reader to dump the log file's contents, for example:</para>
+ <para>To examine a log file on a stopped Lustre server, mount its backing file system as ldiskfs or zfs, then use llog_reader to dump the log file's contents, for example:</para>
<screen>mount -t ldiskfs /dev/sda /mnt/mgs
llog_reader /mnt/mgs/CONFIGS/tfs-client</screen>
<para>To examine the same log file on a running Lustre server, use the ldiskfs-enabled debugfs utility (called debug.ldiskfs on some distributions) to extract the file, for example:</para>
<para>The llstat utility displays Lustre statistics.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>llstat [-c] [-g] [-i interval] stats_file
-</screen>
+ <screen>llstat [-c] [-g] [-i <replaceable>interval</replaceable>] <replaceable>stats_file
+</replaceable></screen>
</section>
<section remap="h5">
<title>Description</title>
- <para>The llstat utility displays statistics from any of the Lustre statistics files that share a common format and are updated at <emphasis>interval</emphasis> seconds. To stop statistics printing, use <emphasis>ctrl</emphasis>-<emphasis>c.</emphasis></para>
+ <para>The llstat utility displays statistics from any of the Lustre statistics files that share a common format and are updated at <literal>interval</literal> seconds. To stop statistics printing, use <literal>ctrl</literal>-<literal>c.</literal></para>
</section>
<section remap="h5">
<title>Options</title>
<para><emphasis role="bold">Option</emphasis></para>
</entry>
<entry>
- <para><emphasis role="bold"> Description</emphasis></para>
+ <para><emphasis role="bold">Description</emphasis></para>
</entry>
</row>
</thead>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">-c</emphasis></para>
+ <para> <literal>-c</literal></para>
</entry>
<entry>
<para> Clears the statistics file.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-i</emphasis></para>
+ <para> <literal>-i</literal></para>
</entry>
<entry>
<para> Specifies the polling period (in seconds).</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-g</emphasis></para>
+ <para> <literal>-g</literal></para>
</entry>
<entry>
<para> Specifies graphable output format.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-h</emphasis></para>
+ <para> <literal>-h</literal></para>
</entry>
<entry>
<para> Displays help information.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">stats_file</emphasis></para>
+ <para> <literal>stats_file</literal></para>
</entry>
<entry>
- <para> Specifies either the full path to a statistics file or the shorthand reference, <emphasis role="bold">mds</emphasis> or <emphasis role="bold">ost</emphasis></para>
+ <para> Specifies either the full path to a statistics file or the shorthand reference, <literal>mds</literal> or <literal>ost</literal></para>
</entry>
</row>
</tbody>
<title>Files</title>
<para>The llstat files are located at:</para>
<screen>/proc/fs/lustre/mdt/MDS/*/stats
-/proc/fs/lustre/mds/*/exports/*/stats
+/proc/fs/lustre/mdt/*/exports/*/stats
/proc/fs/lustre/mdc/*/stats
/proc/fs/lustre/ldlm/services/*/stats
/proc/fs/lustre/ldlm/namespaces/*/pool/stats
<para>The llverdev verifies a block device is functioning properly over its full size.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>llverdev [-c <emphasis>chunksize</emphasis>] [-f] [-h] [-o <emphasis>offset</emphasis>] [-l] [-p] [-r] [-t <emphasis>timestamp</emphasis>] [-v] [-w] <emphasis>device</emphasis></screen>
+ <screen>llverdev [-c <replaceable>chunksize</replaceable>] [-f] [-h] [-o <replaceable>offset</replaceable>] [-l] [-p] [-r] [-t <replaceable>timestamp</replaceable>] [-v] [-w] <replaceable>device</replaceable></screen>
</section>
<section remap="h5">
<title>Description</title>
<tbody>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-c|--chunksize</emphasis></para>
+ <para> <literal>-c|--chunksize</literal></para>
</entry>
<entry>
<para> I/O chunk size in bytes (default value is 1048576).</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-f|--force</emphasis></para>
+ <para> <literal>-f|--force</literal></para>
</entry>
<entry>
<para> Forces the test to run without a confirmation that the device will be overwritten and all data will be permanently destroyed.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-h|--help</emphasis></para>
+ <para> <literal>-h|--help</literal></para>
</entry>
<entry>
<para> Displays a brief help message.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-o</emphasis><emphasis>offset</emphasis></para>
+ <para> <literal>-o <replaceable>offset</replaceable></literal></para>
</entry>
<entry>
<para> Offset (in kilobytes) of the start of the test (default value is 0).</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-l|--long</emphasis></para>
+ <para> <literal>-l|--long</literal></para>
</entry>
<entry>
<para> Runs a full check, writing and then reading and verifying every block on the disk.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-p|--partial</emphasis></para>
+ <para> <literal>-p|--partial</literal></para>
</entry>
<entry>
<para> Runs a partial check, only doing periodic checks across the device (1 GB steps).</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-r|--read</emphasis></para>
+ <para> <literal>-r|--read</literal></para>
</entry>
<entry>
- <para> Runs the test in read (verify) mode only, after having previously run the test in <emphasis role="bold">-w</emphasis> mode.</para>
+ <para> Runs the test in read (verify) mode only, after having previously run the test in <literal>-w</literal> mode.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-t</emphasis><emphasis>timestamp</emphasis></para>
+ <para> <literal>-t <replaceable>timestamp</replaceable></literal></para>
</entry>
<entry>
- <para> Sets the test start time as printed at the start of a previously-interrupted test to ensure that validation data is the same across the entire filesystem (default value is the current time()).</para>
+ <para> Sets the test start time as printed at the start of a previously-interrupted
+ test to ensure that validation data is the same across the entire file system
+ (default value is the current time()).</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-v|--verbose</emphasis></para>
+ <para> <literal>-v|--verbose</literal></para>
</entry>
<entry>
<para> Runs the test in verbose mode, listing each read and write operation.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">-w|--write</emphasis></para>
+ <para> <literal>-w|--write</literal></para>
</entry>
<entry>
<para> Runs the test in write (test-pattern) mode (default runs both read and write).</para>
</section>
<section remap="h5">
<title>Description</title>
- <para>The lshowmount utility shows the hosts that have Lustre mounted to a server. Ths utility looks for exports from the MGS, MDS, and obdfilter.</para>
+ <para>The lshowmount utility shows the hosts that have Lustre mounted to a server. This utility looks for exports from the MGS, MDS, and obdfilter.</para>
</section>
<section remap="h5">
<title>Options</title>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">-e|--enumerate</emphasis></para>
+ <para> <literal>-e|--enumerate</literal></para>
</entry>
<entry>
<para> Causes lshowmount to list each client mounted on a separate line instead of trying to compress the list of clients into a hostrange string.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-h|--help</emphasis></para>
+ <para> <literal>-h|--help</literal></para>
</entry>
<entry>
<para> Causes lshowmount to print out a usage message.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-l|--lookup</emphasis></para>
+ <para> <literal>-l|--lookup</literal></para>
</entry>
<entry>
<para> Causes lshowmount to try to look up the hostname for NIDs that look like IP addresses.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">-v|--verbose</emphasis></para>
+ <para> <literal>-v|--verbose</literal></para>
</entry>
<entry>
<para> Causes lshowmount to output export information for each service instead of only displaying the aggregate information for all Lustre services on the server.</para>
</section>
<section remap="h5">
<title>Files</title>
- <screen>/proc/fs/lustre/mgs/<server>/exports/<uuid>/nid /proc/fs/lustre/mds/<server>/expo\
-rts/<uuid>/nid /proc/fs/lustre/obdfilter/<server>/exports/<uuid>/nid</screen>
+ <screen>/proc/fs/lustre/mgs/<replaceable>server</replaceable>/exports/<replaceable>uuid</replaceable>/nid
+/proc/fs/lustre/mds/<replaceable>server</replaceable>/exports/<replaceable>uuid</replaceable>/nid
+/proc/fs/lustre/obdfilter/<replaceable>server</replaceable>/exports/<replaceable>uuid</replaceable>/nid</screen>
</section>
</section>
<section xml:id="dbdoclet.50438219_90218">
<title><indexterm><primary>lst</primary></indexterm>
lst</title>
- <para>The lst utility starts LNET self-test.</para>
+ <para>The lst utility starts LNet self-test.</para>
<section remap="h5">
<title>Synopsis</title>
<screen>lst</screen>
</section>
<section remap="h5">
<title>Description</title>
- <para>LNET self-test helps site administrators confirm that Lustre Networking (LNET) has been properly installed and configured. The self-test also confirms that LNET and the network software and hardware underlying it are performing as expected.</para>
- <para>Each LNET self-test runs in the context of a session. A node can be associated with only one session at a time, to ensure that the session has exclusive use of the nodes on which it is running. A session is create, controlled and monitored from a single node; this is referred to as the self-test console.</para>
+ <para>LNet self-test helps site administrators confirm that Lustre Networking (LNet) has been properly installed and configured. The self-test also confirms that LNet and the network software and hardware underlying it are performing as expected.</para>
+ <para>Each LNet self-test runs in the context of a session. A node can be associated with only one session at a time, to ensure that the session has exclusive use of the nodes on which it is running. A session is create, controlled and monitored from a single node; this is referred to as the self-test console.</para>
<para>Any node may act as the self-test console. Nodes are named and allocated to a self-test session in groups. This allows all nodes in a group to be referenced by a single name.</para>
<para>Test configurations are built by describing and running test batches. A test batch is a named collection of tests, with each test composed of a number of individual point-to-point tests running in parallel. These individual point-to-point tests are instantiated according to the test type, source group, target group and distribution specified when the test is added to the test batch.</para>
</section>
<section remap="h5">
<title>Modules</title>
- <para>To run LNET self-test, load these modules: libcfs, lnet, lnet_selftest and any one of the klnds (ksocklnd, ko2iblnd...). To load all necessary modules, run modprobe lnet_selftest, which recursively loads the modules on which lnet_selftest depends.</para>
- <para>There are two types of nodes for LNET self-test: the console node and test nodes. Both node types require all previously-specified modules to be loaded. (The userspace test node does not require these modules).</para>
+ <para>To run LNet self-test, load these modules: libcfs, lnet, lnet_selftest and any one of the klnds (ksocklnd, ko2iblnd...). To load all necessary modules, run modprobe lnet_selftest, which recursively loads the modules on which lnet_selftest depends.</para>
+ <para>There are two types of nodes for LNet self-test: the console node and test nodes. Both node types require all previously-specified modules to be loaded. (The userspace test node does not require these modules).</para>
<para>Test nodes can be in either kernel or in userspace. A console user can invite a kernel test node to join the test session by running lst add_group NID, but the user cannot actively add a userspace test node to the test session. However, the console user can passively accept a test node to the test session while the test node runs lst client to connect to the console.</para>
</section>
<section remap="h5">
<title>Utilities</title>
- <para>LNET self-test includes two user utilities, lst and lstclient.</para>
+ <para>LNet self-test includes two user utilities, lst and lstclient.</para>
<para>lst is the user interface for the self-test console (run on the console node). It provides a list of commands to control the entire test system, such as create session, create test groups, etc.</para>
- <para>lstclient is the userspace self-test program which is linked with userspace LNDs and LNET. A user can invoke lstclient to join a self-test session:</para>
+ <para>lstclient is the userspace self-test program which is linked with userspace LNDs and LNet. A user can invoke lstclient to join a self-test session:</para>
<screen>lstclient -sesid CONSOLE_NID group NAME</screen>
</section>
<section remap="h5">
<title>Example Script</title>
- <para>This is a sample LNET self-test script which simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an IB network (connected via LNET routers), with half the clients reading and half the clients writing.</para>
+ <para>This is a sample LNet self-test script which simulates the traffic pattern of a set of Lustre servers on a TCP network, accessed by Lustre clients on an IB network (connected via LNet routers), with half the clients reading and half the clients writing.</para>
<screen>#!/bin/bash
export LST_SESSION=$$
lst new_session read/write
<section xml:id="dbdoclet.50438219_54734">
<title><indexterm><primary>lustre_rmmod.sh</primary></indexterm>
lustre_rmmod.sh</title>
- <para>The lustre_rmmod.sh utility removes all Lustre and LNET modules (assuming no Lustre services are running). It is located in /usr/bin.</para>
+ <para>The lustre_rmmod.sh utility removes all Lustre and LNet modules (assuming no Lustre services are running). It is located in /usr/bin.</para>
<note>
<para>The lustre_rmmod.sh utility does not work if Lustre modules are being used or if you have manually run the lctl network up command.</para>
</note>
<para>The lustre_rsync utility synchronizes (replicates) a Lustre file system to a target file system.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>lustre_rsync --source|-s <src> --target|-t <tgt>
- --mdt|-m <mdt> [--user|-u <user id>]
- [--xattr|-x <yes|no>] [--verbose|-v]
- [--statuslog|-l <log>] [--dry-run] [--abort-on-err]
+ <screen>lustre_rsync --source|-s <replaceable>src</replaceable> --target|-t <replaceable>tgt</replaceable>
+ --mdt|-m <replaceable>mdt</replaceable> [--user|-u <replaceable>userid</replaceable>]
+ [--xattr|-x <replaceable>yes|no</replaceable>] [--verbose|-v]
+ [--statuslog|-l <replaceable>log</replaceable>] [--dry-run] [--abort-on-err]
-lustre_rsync --statuslog|-l <log>
+lustre_rsync --statuslog|-l <replaceable>log</replaceable>
-lustre_rsync --statuslog|-l <log> --source|-s <source>
- --target|-t <tgt> --mdt|-m <mdt></screen>
+lustre_rsync --statuslog|-l <replaceable>log</replaceable> --source|-s <replaceable>source</replaceable>
+ --target|-t <replaceable>tgt</replaceable> --mdt|-m <replaceable>mdt</replaceable></screen>
</section>
<section remap="h5">
<title>Description</title>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">--source=</emphasis><emphasis><src></emphasis></para>
+ <para> <literal>--source=<replaceable>src</replaceable></literal></para>
</entry>
<entry>
<para> The path to the root of the Lustre file system (source) which will be synchronized. This is a mandatory option if a valid status log created during a previous synchronization operation (--statuslog) is not specified.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--target=</emphasis><emphasis><tgt></emphasis></para>
+ <para> <literal>--target=<replaceable>tgt</replaceable></literal></para>
</entry>
<entry>
<para> The path to the root where the source file system will be synchronized (target). This is a mandatory option if the status log created during a previous synchronization operation (--statuslog) is not specified. This option can be repeated if multiple synchronization targets are desired.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--mdt=</emphasis><emphasis><mdt></emphasis></para>
+ <para> <literal>--mdt=<replaceable>mdt</replaceable></literal></para>
</entry>
<entry>
<para> The metadata device to be synchronized. A changelog user must be registered for this device. This is a mandatory option if a valid status log created during a previous synchronization operation (--statuslog) is not specified.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--user=</emphasis><emphasis><user id></emphasis></para>
+ <para> <literal>--user=<replaceable>userid</replaceable></literal></para>
</entry>
<entry>
<para> The changelog user ID for the specified MDT. To use lustre_rsync, the changelog user must be registered. For details, see the changelog_register parameter in the lctl man page. This is a mandatory option if a valid status log created during a previous synchronization operation (--statuslog) is not specified.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--statuslog=</emphasis><emphasis><log></emphasis></para>
+ <para> <literal>--statuslog=<replaceable>log</replaceable></literal></para>
</entry>
<entry>
- <para> A log file to which synchronization status is saved. When lustre_rsync starts, the state of a previous replication is read from here. If the status log from a previous synchronization operation is specified, otherwise mandatory options like --source, --target and --mdt options may be skipped. By specifying options like --source, --target and/or --mdt in addition to the --statuslog option, parameters in the status log can be overriden. Command line options take precedence over options in the status log.</para>
+ <para> A log file to which synchronization status is saved. When lustre_rsync starts, the state of a previous replication is read from here. If the status log from a previous synchronization operation is specified, otherwise mandatory options like --source, --target and --mdt options may be skipped. By specifying options like --source, --target and/or --mdt in addition to the --statuslog option, parameters in the status log can be overridden. Command line options take precedence over options in the status log.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--xattr</emphasis><emphasis><yes|no></emphasis></para>
+ <para> <literal>--xattr<replaceable>yes|no</replaceable></literal></para>
</entry>
<entry>
<para> Specifies whether extended attributes (xattrs) are synchronized or not. The default is to synchronize extended attributes.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--verbose</emphasis></para>
+ <para> <literal>--verbose</literal></para>
</entry>
<entry>
<para> Produces a verbose output.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--dry-run</emphasis></para>
+ <para> <literal>--dry-run</literal></para>
</entry>
<entry>
<para> Shows the output of lustre_rsync commands (copy, mkdir, etc.) on the target file system without actually executing them.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--abort-on-err</emphasis></para>
+ <para> <literal>--abort-on-err</literal></para>
</entry>
<entry>
<para> Shows the output of lustre_rsync commands (copy, mkdir, etc.) on the target file system without actually executing them.</para>
<section xml:id="dbdoclet.50438219_75432">
<title><indexterm><primary>mkfs.lustre</primary></indexterm>
mkfs.lustre</title>
- <para>The mkfs.lustre utility formats a disk for a Lustre service.</para>
+ <para>The <literal>mkfs.lustre</literal> utility formats a disk for a Lustre service.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>mkfs.lustre <target_type> [options] device</screen>
- <para>where <target_type> is one of the following:</para>
+ <screen>mkfs.lustre <replaceable>target_type</replaceable> [options] <replaceable>device</replaceable></screen>
+ <para>where <replaceable>target_type</replaceable> is one of the following:</para>
<informaltable frame="all">
<tgroup cols="2">
<colspec colname="c1" colwidth="50*"/>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">--ost</emphasis></para>
+ <para> <literal>--ost</literal></para>
</entry>
<entry>
- <para> Object Storage Target (OST)</para>
+ <para> Object storage target (OST)</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--mdt</emphasis></para>
+ <para> <literal>--mdt</literal></para>
</entry>
<entry>
- <para> Metadata Storage Target (MDT)</para>
+ <para> Metadata storage target (MDT)</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--network=</emphasis><emphasis>net,...</emphasis></para>
+ <para> <literal>--network=<replaceable>net,...</replaceable></literal></para>
</entry>
<entry>
<para> Network(s) to which to restrict this OST/MDT. This option can be repeated as necessary.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--mgs</emphasis></para>
+ <para> <literal>--mgs</literal></para>
</entry>
<entry>
- <para> Configuration Management Service (MGS), one per site. This service can be combined with one <emphasis role="bold">--mdt</emphasis> service by specifying both types.</para>
+ <para> Configuration management service (MGS), one per site. This service can be
+ combined with one <literal>--mdt</literal> service by specifying both
+ types.</para>
</entry>
</row>
</tbody>
</section>
<section remap="h5">
<title>Description</title>
- <para>mkfs.lustre is used to format a disk device for use as part of a Lustre file system. After formatting, a disk can be mounted to start the Lustre service defined by this command.</para>
- <para>When the file system is created, parameters can simply be added as a --param option to the mkfs.lustre command. See <xref linkend="dbdoclet.50438194_17237"/>.</para>
+ <para><literal>mkfs.lustre</literal> is used to format a disk device for use as part of a
+ Lustre file system. After formatting, a disk can be mounted to start the Lustre service
+ defined by this command.</para>
+ <para>When the file system is created, parameters can simply be added as a
+ <literal>--param</literal> option to the <literal>mkfs.lustre</literal> command. See <xref
+ linkend="dbdoclet.50438194_17237"/>.</para>
<informaltable frame="all">
<tgroup cols="3">
- <colspec colname="c1" colwidth="33*"/>
- <colspec colname="c2" colwidth="33*"/>
- <colspec colname="c3" colwidth="33*"/>
+ <colspec colname="c1" colwidth="1*"/>
+ <colspec colname="c2" colwidth="1*"/>
+ <colspec colname="c3" colwidth="3*"/>
<thead>
<row>
<entry nameend="c2" namest="c1">
<tbody>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--backfstype</emphasis>=<emphasis>fstype</emphasis></para>
+ <para> <literal>--backfstype=<replaceable>fstype</replaceable></literal></para>
</entry>
<entry>
- <para> Forces a particular format for the backing file system (such as ext3, ldiskfs).</para>
+ <para> Forces a particular format for the backing file system such as ldiskfs (the default) or zfs.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--comment</emphasis>=<emphasis>comment</emphasis></para>
+ <para> <literal>--comment=<replaceable>comment</replaceable></literal></para>
</entry>
<entry>
- <para> Sets a user comment about this disk, ignored by Lustre.</para>
+ <para> Sets a user comment about this disk, ignored by the Lustre software.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--device-size</emphasis>=<emphasis>KB</emphasis></para>
+ <para> <literal>--device-size=<replaceable>#</replaceable>>KB</literal></para>
</entry>
<entry>
- <para> Sets the device size for loop devices.</para>
+ <para>Sets the device size for loop devices.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--dryrun</emphasis></para>
+ <para> <literal>--dryrun</literal></para>
</entry>
<entry>
- <para> Only prints what would be done; it does not affect the disk.</para>
+ <para>Only prints what would be done; it does not affect the disk.</para>
</entry>
</row>
<row>
- <entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--failnode</emphasis>=<emphasis>nid</emphasis>,...</para>
- </entry>
- <entry>
- <para> Sets the NID(s) of a failover partner. This option can be repeated as needed.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: Cannot be used with <emphasis role="bold">--servicenode</emphasis>.</para>
- </entry>
+ <entry nameend="c2" namest="c1"
+ ><literal>--servicenode=<replaceable>nid,...</replaceable></literal></entry>
+ <entry>Sets the NID(s) of all service nodes, including primary and failover partner
+ service nodes. The <literal>--servicenode</literal> option cannot be used with
+ <literal>--failnode</literal> option. See <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/> for
+ more details.</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--servicenode</emphasis>=<emphasis>nid</emphasis>,...</para>
+ <para> <literal>--failnode=<replaceable>nid,...</replaceable></literal></para>
</entry>
<entry>
- <para> Sets the NID(s) of all service node, including failover partner as well as primary node service nids. This option can be repeated as needed.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: Cannot be used with <emphasis role="bold">--failnode</emphasis>.</para>
+ <para>Sets the NID(s) of a failover service node for a primary server for a target.
+ The <literal>--failnode</literal> option cannot be used with
+ <literal>--servicenode</literal> option. See <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/>
+ for more details.<note>
+ <para>When the <literal>--failnode</literal> option is used, certain
+ restrictions apply (see <xref xmlns:xlink="http://www.w3.org/1999/xlink"
+ linkend="dbdoclet.50438188_92688"/>).</para>
+ </note></para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--fsname</emphasis>=<emphasis>filesystem_name</emphasis></para>
+ <para> <literal>--fsname=<replaceable>filesystem_name</replaceable></literal></para>
</entry>
<entry>
- <para> The Lustre file system of which this service/node will be a part. The default file system name is 'lustreâ€.</para>
+ <para> The Lustre file system of which this service/node will be a part. The default
+ file system name is <literal>lustre</literal>.</para>
<para> </para>
<note>
<para>The file system name is limited to 8 characters.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--index</emphasis>=<emphasis>index</emphasis></para>
+ <para>
+ <literal>--index=<replaceable>index_number</replaceable></literal></para>
</entry>
<entry>
- <para> Specifies the OST or MDT number. This should always be used when formatting OSTs, in order to ensure that there is a simple mapping between the OST index and the OSS node and device it is located on.</para>
+ <para>Specifies the OST or MDT number (0...N). This allows mapping between the OSS
+ and MDS node and the device on which the OST or MDT is located.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--mkfsoptions</emphasis>=<emphasis>opts</emphasis></para>
+ <para> <literal>--mkfsoptions=<replaceable>opts</replaceable></literal></para>
</entry>
<entry>
<para> Formats options for the backing file system. For example, ext3 options could be set here.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--mountfsoptions</emphasis>=<emphasis>opts</emphasis></para>
+ <para> <literal>--mountfsoptions=<replaceable>opts</replaceable></literal></para>
</entry>
<entry>
<para> Sets the mount options used when the backing file system is mounted.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: Unlike earlier versions of mkfs.lustre, this version completely replaces the default mount options with those specified on the command line, and issues a warning on stderr if any default mount options are omitted.</para>
+ <warning><para>Unlike earlier versions of <literal>mkfs.lustre</literal>, this version completely replaces
+ the default mount options with those specified on the command line, and issues a
+ warning on stderr if any default mount options are omitted.</para></warning>
<para>The defaults for ldiskfs are:</para>
- <para>OST: <emphasis>errors=remount-ro</emphasis>;</para>
- <para>MGS/MDT: <emphasis>errors=remount-ro,iopen_nopriv,user_xattr</emphasis></para>
- <para>Do not alter the default mount options unless you know what you are doing.</para>
+ <para>MGS/MDT: <literal>errors=remount-ro,iopen_nopriv,user_xattr</literal></para>
+ <para>OST: <literal>errors=remount-ro,extents,mballoc</literal></para>
+ <para condition='l25'>OST: <literal>errors=remount-ro</literal></para>
+ <para>Use care when altering the default mount options.</para>
</entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--network</emphasis>=<emphasis>net</emphasis>,...</para>
+ <para> <literal>--network=<replaceable>net,...</replaceable></literal></para>
<para> </para>
</entry>
<entry>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--mgsnode</emphasis>=<emphasis>nid</emphasis>,...</para>
+ <para> <literal>--mgsnode=<replaceable>nid,...</replaceable></literal></para>
</entry>
<entry>
<para> Sets the NIDs of the MGS node, required for all targets other than the MGS.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--param</emphasis><emphasis>key</emphasis>=<emphasis>value</emphasis></para>
+ <para> <literal>--param <replaceable>key</replaceable>=<replaceable>value</replaceable></literal></para>
</entry>
<entry>
- <para> Sets the permanent parameter <emphasis>key</emphasis> to value <emphasis>value</emphasis>. This option can be repeated as necessary. Typical options might include:</para>
+ <para> Sets the permanent parameter <replaceable>key</replaceable> to value <replaceable>value</replaceable>. This option can be repeated as necessary. Typical options might include:</para>
</entry>
</row>
<row>
<para>  </para>
</entry>
<entry>
- <para> --<emphasis>param sys.timeout=40</emphasis></para>
+ <para> <literal>--param sys.timeout=40</literal>></para>
</entry>
<entry>
<para> System obd timeout.</para>
<para>  </para>
</entry>
<entry>
- <para> --<emphasis>param lov.stripesize=2M</emphasis></para>
+ <para> <literal>--param lov.stripesize=2M</literal></para>
</entry>
<entry>
<para> Default stripe size.</para>
<para>  </para>
</entry>
<entry>
- <para> --<emphasis>param lov.stripecount=2</emphasis></para>
+ <para> <literal>param lov.stripecount=2</literal></para>
</entry>
<entry>
<para> Default stripe count.</para>
<para>  </para>
</entry>
<entry>
- <para> --<emphasis>param failover.mode=failout</emphasis></para>
+ <para> <literal>--param failover.mode=failout</literal></para>
</entry>
<entry>
<para> Returns errors instead of waiting for recovery.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--quiet</emphasis></para>
+ <para> <literal>--quiet</literal></para>
</entry>
<entry>
<para> Prints less information.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--reformat</emphasis></para>
+ <para> <literal>--reformat</literal></para>
</entry>
<entry>
<para> Reformats an existing Lustre disk.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--stripe-count-hint=stripes</emphasis></para>
+ <para> <literal>--stripe-count-hint=stripes</literal></para>
</entry>
<entry>
<para> Used to optimize the MDT's inode size.</para>
</row>
<row>
<entry nameend="c2" namest="c1">
- <para> <emphasis role="bold">--verbose</emphasis></para>
+ <para> <literal>--verbose</literal></para>
</entry>
<entry>
<para> Prints more information.</para>
</section>
<section remap="h5">
<title>Examples</title>
- <para>Creates a combined MGS and MDT for file system <emphasis role="bold">testfs</emphasis> on, e.g., node <emphasis role="bold">cfs21</emphasis>:</para>
+ <para>Creates a combined MGS and MDT for file system <literal>testfs</literal> on, e.g., node <literal>cfs21</literal>:</para>
<screen>mkfs.lustre --fsname=testfs --mdt --mgs /dev/sda1</screen>
- <para>Creates an OST for file system <emphasis role="bold">testfs</emphasis> on any node (using the above MGS):</para>
+ <para>Creates an OST for file system <literal>testfs</literal> on any node (using the above
+ MGS):</para>
<screen>mkfs.lustre --fsname=testfs --mgsnode=cfs21@tcp0 --ost --index=0 /dev/sdb</screen>
- <para>Creates a standalone MGS on, e.g., node <emphasis role="bold">cfs22</emphasis>:</para>
+ <para>Creates a standalone MGS on, e.g., node <literal>cfs22</literal>:</para>
<screen>mkfs.lustre --mgs /dev/sda1</screen>
- <para>Creates an MDT for file system <emphasis role="bold">myfs1</emphasis> on any node (using the above MGS):</para>
+ <para>Creates an MDT for file system <literal>myfs1</literal> on any node (using the above MGS):</para>
<screen>mkfs.lustre --fsname=myfs1 --mdt --mgsnode=cfs22@tcp0 /dev/sda2</screen>
</section>
<section remap="h5">
<para>The mount.lustre utility starts a Lustre client or target service.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>mount -t lustre [-o options] directory
+ <screen>mount -t lustre [-o options] device mountpoint
</screen>
</section>
<section remap="h5">
<tbody>
<row>
<entry>
- <para> <emphasis><mgsspec>:/<fsname></emphasis></para>
- <para> </para>
+ <para> <literal><replaceable>mgsname</replaceable>:/<replaceable>fsname</replaceable><replaceable>[/subdir]</replaceable></literal></para>
</entry>
<entry>
- <para> Mounts the Lustre file system named <emphasis>fsname</emphasis> on the client by contacting the Management Service at <emphasis>mgsspec</emphasis> on the pathname given by <emphasis>directory</emphasis>. The format for <emphasis>mgsspec</emphasis> is defined below. A mounted client file system appears in fstab(5) and is usable, like any local file system, and provides a full POSIX-compliant interface.</para>
+ <para> Mounts the Lustre file system named
+ <replaceable>fsname</replaceable> (optionally starting at
+ subdirectory <replaceable>subdir</replaceable> within the
+ filesystem, if specified) on the client at the directory
+ <replaceable>mountpoint</replaceable>, by contacting the Lustre
+ Management Service at <replaceable>mgsname</replaceable>. The
+ format for <replaceable>mgsname</replaceable> is defined below. A
+ client file system can be listed in <literal>fstab(5)</literal>
+ for automatic mount at boot time, is usable like any local file
+ system, and provides a full POSIX standard-compliant interface.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis><disk_device></emphasis></para>
+ <para> <replaceable>block_device</replaceable></para>
</entry>
<entry>
- <para> Starts the target service defined by the mkfs.lustre command on the physical disk <emphasis>disk_device</emphasis>. A mounted target service file system is only useful for df(1) operations and appears in fstab(5) to show the device is in use.</para>
+ <para> Starts the target service defined by the
+ <literal>mkfs.lustre(8)</literal> command on the physical disk
+ <replaceable>block_device</replaceable>. The
+ <replaceable>block_device</replaceable> may be specified using
+ <literal>-L <replaceable>label</replaceable></literal> to find
+ the first block device with that label (e.g.
+ <literal>testfs-MDT0000</literal>), or by UUID using the
+ <literal>-U <replaceable>uuid</replaceable></literal> option.
+ Care should be taken if there is a device-level backup of the
+ target filesystem on the same node, which would have a
+ duplicate label and UUID if it has not been changed with
+ <literal>tune2fs(8)</literal> or similar. The mounted target
+ service filesystem mounted at
+ <replaceable>mountpoint</replaceable> is only useful for
+ <literal>df(1)</literal> operations and appears in
+ <literal>/proc/mounts</literal> to show the device is in use.
+ </para>
</entry>
</row>
</tbody>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold"><mgsspec>:</emphasis>=<emphasis><mgsnode>[:<mgsnode>]</emphasis></para>
- <para> </para>
+ <para> <literal>mgsname=<replaceable>mgsnode</replaceable>[:<replaceable>mgsnode</replaceable>]</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>mgsname</replaceable> is a colon-separated
+ list of <replaceable>mgsnode</replaceable> names where the MGS
+ service may run. Multiple <replaceable>mgsnode</replaceable>
+ values can be specified if the MGS service is configured for
+ HA failover and may be running on any one of the nodes.
+ </para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>mgsnode=<replaceable>mgsnid</replaceable>[,<replaceable>mgsnid</replaceable>]</literal></para>
</entry>
<entry>
- <para> The MGS specification may be a colon-separated list of nodes.</para>
+ <para> Each <replaceable>mgsnode</replaceable> may specify a
+ comma-separated list of NIDs, if there are different LNet
+ interfaces for that <literal>mgsnode</literal>.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold"><mgsnode>:</emphasis>=<emphasis><mgsnid>[,<mgsnid>]</emphasis></para>
+ <para> <literal>mgssec=<replaceable>flavor</replaceable></literal></para>
</entry>
<entry>
- <para> Each node may be specified by a comma-separated list of NIDs.</para>
+ <para>Specifies the encryption flavor for the initial network
+ RPC connection to the MGS. Non-security flavors are:
+ <literal>null</literal>, <literal>plain</literal>, and
+ <literal>gssnull</literal>, which respectively disable, or
+ have no encryption or integrity features for testing purposes.
+ Kerberos flavors are: <literal>krb5n</literal>,
+ <literal>krb5a</literal>, <literal>krb5i</literal>, and
+ <literal>krb5p</literal>. Shared-secret key flavors are:
+ <literal>skn</literal>, <literal>ska</literal>,
+ <literal>ski</literal>, and <literal>skpi</literal>, see the
+ <xref linkend="lustressk"/> for more details. The security
+ flavor for client-to-server connections is specified in the
+ filesystem configuration that the client fetches from the MGS.
+ </para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>skpath=<replaceable>file|directory</replaceable></literal></para>
+ </entry>
+ <entry>
+ <para condition='l29'>
+ Path to a file or directory with the keyfile(s) to load for
+ this mount command. Keys are inserted into the
+ <literal>KEY_SPEC_SESSION_KEYRING</literal> keyring in the
+ kernel with a description containing
+ <literal>lustre:</literal> and a suffix which depends on
+ whether the context of the mount command is for an MGS,
+ MDT/OST, or client.
+ </para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>exclude=<replaceable>ostlist</replaceable></literal></para>
+ </entry>
+ <entry>
+ <para>Starts a client or MDT with a colon-separated list of
+ known inactive OSTs that it will not try to connect to.</para>
</entry>
</row>
</tbody>
</tgroup>
</informaltable>
- <para>In addition to the standard mount options, Lustre understands the following client-specific options:</para>
+ <para>In addition to the standard mount(8) options, Lustre understands
+ the following client-specific options:</para>
<informaltable frame="all">
<tgroup cols="2">
<colspec colname="c1" colwidth="50*"/>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">flock</emphasis></para>
+ <para><literal>always_ping</literal></para>
+ </entry>
+ <entry>
+ <para condition='l29'>The client will periodically ping the server when it is
+ idle, even if the server <literal>ptlrpc</literal> module
+ is configured with the <literal>suppress_pings</literal>
+ option. This allows clients to reliably use the filesystem
+ even if they are not part of an external client health
+ monitoring mechanism.
+ </para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>flock</literal></para>
+ </entry>
+ <entry>
+ <para>Enables advisory file locking support between
+ participating applications using the <literal>flock(2)</literal>
+ system call. This causes file locking to be coherent across all
+ client nodes also using this mount option. This is useful if
+ applications need coherent userspace file locking across
+ multiple client nodes, but also imposes communications overhead
+ in order to maintain locking consistency between client nodes.
+ </para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>localflock</literal></para>
</entry>
<entry>
- <para> Enables full flock support, coherent across all client nodes.</para>
+ <para>Enables client-local <literal>flock(2)</literal> support,
+ using only client-local advisory file locking. This is faster
+ than using the global <literal>flock</literal> option, and can
+ be used for applications that depend on functioning
+ <literal>flock(2)</literal> but run only on a single node.
+ It has minimal overhead using only the Linux kernel's locks.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">localflock</emphasis></para>
+ <para> <literal>noflock</literal></para>
</entry>
<entry>
- <para> Enables local flock support, using only client-local flock (faster, for applications that require flock, but do not run on multiple nodes).</para>
+ <para>Disables <literal>flock(2)</literal> support entirely,
+ and is the default option. Applications calling
+ <literal>flock(2)</literal> get an
+ <literal>ENOSYS</literal> error. It is up to the administrator
+ to choose either the <literal>localflock</literal> or
+ <literal>flock</literal> mount option based on their
+ requirements. It is possible to mount clients with different
+ options, and only those mounted with <literal>flock</literal>
+ will be coherent amongst each other.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">noflock</emphasis></para>
+ <para> <literal>lazystatfs</literal></para>
</entry>
<entry>
- <para> Disables flock support entirely. Applications calling flock get an error. It is up to the administrator to choose either <emphasis role="bold">localflock</emphasis> (fastest, low impact, not coherent between nodes) or <emphasis role="bold">flock</emphasis> (slower, performance impact for use, coherent between nodes).</para>
+ <para>Allows <literal>statfs(2)</literal> (as used by
+ <literal>df(1)</literal> and <literal>lfs-df(1)</literal>) to
+ return even if some OST or MDT is unresponsive or has been
+ temporarily or permanently disabled in the configuration.
+ This avoids blocking until all of the targets are available.
+ This is the default behavior since Lustre 2.9.0.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">user_xattr</emphasis></para>
+ <para> <literal>nolazystatfs</literal></para>
</entry>
<entry>
- <para> Enables get/set of extended attributes by regular users. See the attr(5) manual page.</para>
+ <para>Requires that <literal>statfs(2)</literal> block until all
+ OSTs and MDTs are available and have returned space usage.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">nouser_xattr</emphasis></para>
+ <para> <literal>user_xattr</literal></para>
</entry>
<entry>
- <para> Disables use of extended attributes by regular users. Root and system processes can still use extended attributes.</para>
+ <para>Enables get/set of extended attributes by regular users
+ in the <literal>user.*</literal> namespace. See the
+ <literal>attr(5)</literal> manual page for more details.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">acl</emphasis></para>
+ <para> <literal>nouser_xattr</literal></para>
</entry>
<entry>
- <para> Enables POSIX Access Control List support. See the acl(5) manual page.</para>
+ <para>Disables use of extended attributes in the
+ <literal>user.*</literal> namespace by regular users. Root
+ and system processes can still use extended attributes.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">noacl</emphasis></para>
+ <para> <literal>verbose</literal></para>
</entry>
<entry>
- <para> Disables Access Control List support.</para>
+ <para> Enable extra mount/umount console messages.</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>noverbose</literal></para>
+ </entry>
+ <entry>
+ <para> Disable mount/umount console messages.</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>user_fid2path</literal></para>
+ </entry>
+ <entry>
+ <para>Enable FID-to-path translation by regular users.
+ </para>
+ <note><para>This option allows a potential security hole because
+ it allows regular users direct access to a file by its Lustre
+ File ID. This bypasses POSIX path-based permission checks,
+ and could allow the user to access a file in a directory that
+ they do not have access to. Regular POSIX file mode and ACL
+ permission checks are still performed on the file itself, so
+ users cannot access a file to which they have no permission.
+ </para></note>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>nouser_fid2path</literal></para>
+ </entry>
+ <entry>
+ <para> Disable FID to path translation by
+ regular users. Root and processes with
+ <literal>CAP_DAC_READ_SEARCH</literal> can still perform FID
+ to path translation.
+ </para>
</entry>
</row>
</tbody>
</tgroup>
</informaltable>
- <para>In addition to the standard mount options and backing disk type (e.g. ext3) options, Lustre understands the following server-specific options:</para>
+ <para>In addition to the standard mount options and backing disk type
+ (e.g. ldiskfs) options, Lustre understands the following server-specific
+ mount options:</para>
<informaltable frame="all">
<tgroup cols="2">
<colspec colname="c1" colwidth="50*"/>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">nosvc</emphasis></para>
+ <para> <literal>nosvc</literal></para>
</entry>
<entry>
<para> Starts the MGC (and MGS, if co-located) for a target service, not the actual service.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">nomsgs</emphasis></para>
+ <para> <literal>nomgs</literal></para>
</entry>
<entry>
<para> Starts only the MDT (with a co-located MGS), without starting the MGS.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">exclude=<ostlist></emphasis></para>
- </entry>
- <entry>
- <para> Starts a client or MDT with a colon-separated list of known inactive OSTs.</para>
- </entry>
- </row>
- <row>
- <entry>
- <para> <emphasis role="bold">nosvc</emphasis></para>
+ <para> <literal>abort_recov</literal></para>
</entry>
<entry>
- <para> Only starts the MGC (and MGS, if co-located) for a target service, not the actual service.</para>
+ <para> Aborts client recovery on that server and starts the target service immediately.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">nomsgs</emphasis></para>
+ <para> <literal>max_sectors_kb=<replaceable>KB</replaceable></literal></para>
</entry>
<entry>
- <para> Starts a MDT with a co-located MGS, without starting the MGS.</para>
+ <para condition='l210'>Sets the block device parameter
+ <literal>max_sectors_kb</literal> limit for the MDT or OST
+ target being mounted to specified maximum number of kilobytes.
+ When <literal>max_sectors_kb</literal> isn't specified as a
+ mount option, it will automatically be set to the
+ <literal>max_hw_sectors_kb</literal> (up to a maximum of 16MiB)
+ for that block device. This default behavior is suited for
+ most users. When <literal>max_sectors_kb=0</literal> is used,
+ the current value for this tunable will be kept.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">exclude</emphasis>=<emphasis>ostlist</emphasis></para>
+ <para> <literal>md_stripe_cache_size</literal></para>
</entry>
<entry>
- <para> Starts a client or MDT with a (colon-separated) list of known inactive OSTs.</para>
- </entry>
- </row>
- <row>
- <entry>
- <para> <emphasis role="bold">abort_recov</emphasis></para>
- </entry>
- <entry>
- <para> Aborts client recovery and starts the target service immediately.</para>
+ <para> Sets the stripe cache size for server-side disk with a striped RAID configuration.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">md_stripe_cache_size</emphasis></para>
+ <para> <literal>recovery_time_soft=<replaceable>timeout</replaceable></literal></para>
</entry>
<entry>
- <para> Sets the stripe cache size for server-side disk with a striped RAID configuration.</para>
+ <para>Allows <literal>timeout</literal> seconds for clients to
+ reconnect for recovery after a server crash. This timeout is
+ incrementally extended if it is about to expire and the server
+ is still handling new connections from recoverable clients.
+ </para>
+ <para>The default soft recovery timeout is 3 times the value
+ of the Lustre timeout parameter (see
+ <xref linkend="section_c24_nt5_dl"/>). The default Lustre
+ timeout is 100 seconds, which would make the soft recovery
+ timeout default to 300 seconds (5 minutes). The soft recovery
+ timeout is set at mount time and will not change if the Lustre
+ timeout is changed after mount time.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">recovery_time_soft</emphasis>=<emphasis>timeout</emphasis></para>
+ <para> <literal>recovery_time_hard=<replaceable>timeout</replaceable></literal></para>
</entry>
<entry>
- <para> Allows <emphasis>timeout</emphasis> seconds for clients to reconnect for recovery after a server crash. This timeout is incrementally extended if it is about to expire and the server is still handling new connections from recoverable clients. The default soft recovery timeout is 300 seconds (5 minutes).</para>
+ <para>The server is allowed to incrementally extend its timeout
+ up to a hard maximum of <replaceable>timeout</replaceable>
+ seconds.
+ </para>
+ <para>The default hard recovery timeout is 9 times the value
+ of the Lustre timeout parameter (see
+ <xref linkend="section_c24_nt5_dl"/>). The default Lustre
+ timeout is 100 seconds, which would make the hard recovery
+ timeout default to 900 seconds (15 minutes). The hard recovery
+ timeout is set at mount time and will not change if the Lustre
+ timeout is changed after mount time.
+ </para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">recovery_time_hard</emphasis>=<emphasis>timeout</emphasis></para>
+ <para> <literal>noscrub</literal></para>
</entry>
<entry>
- <para> The server is allowed to incrementally extend its timeout up to a hard maximum of <emphasis>timeout</emphasis> seconds. The default hard recovery timeout is set to 900 seconds (15 minutes).</para>
+ <para>Typically the MDT will detect restoration from a
+ file-level backup during mount. This mount option prevents
+ the OI Scrub from starting automatically when the MDT is
+ mounted. Manually starting LFSCK after mounting provides finer
+ control over the starting conditions. This mount option also
+ prevents OI scrub from occurring automatically when OI
+ inconsistency is detected (see
+ <xref linkend="dbdoclet.lfsck_auto_scrub"/>).
+ </para>
</entry>
</row>
</tbody>
</section>
<section remap="h5">
<title>Examples</title>
- <para>Starts a client for the Lustre file system testfs at mount point /mnt/myfilesystem. The Management Service is running on a node reachable from this client via the cfs21@tcp0 NID.</para>
- <screen>mount -t lustre cfs21@tcp0:/testfs /mnt/myfilesystem</screen>
+ <para>Starts a client for the Lustre file system
+ <replaceable>chipfs</replaceable> at mount point
+ <replaceable>/mnt/chip</replaceable>. The Management Service is running on
+ a node reachable from this client via the cfs21@tcp0 NID.</para>
+ <screen>mount -t lustre cfs21@tcp0:/chipfs /mnt/chip</screen>
+ <para condition='l29'>Similar to the above example, but mounting a
+ subdirectory under <replaceable>chipfs</replaceable> as a fileset.
+ <screen>mount -t lustre cfs21@tcp0:/chipfs/v1_0 /mnt/chipv1_0</screen>
+ </para>
<para>Starts the Lustre metadata target service from /dev/sda1 on mount point /mnt/test/mdt.</para>
<screen>mount -t lustre /dev/sda1 /mnt/test/mdt</screen>
<para>Starts the testfs-MDT0000 service (using the disk label), but aborts the recovery process.</para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">results_filename</emphasis></para>
+ <para> <literal>results_filename</literal></para>
</entry>
<entry>
<para> Output generated by plot-llstat</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">parameter_index</emphasis></para>
+ <para> <literal>parameter_index</literal></para>
<para> </para>
</entry>
<entry>
<para>The routerstat utility prints Lustre router statistics.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>routerstat [interval]</screen>
+ <screen>routerstat [<replaceable>interval</replaceable>]</screen>
</section>
<section remap="h5">
<title>Description</title>
- <para>The routerstat utility watches LNET router statistics. If no <emphasis>interval</emphasis> is specified, then statistics are sampled and printed only one time. Otherwise, statistics are sampled and printed at the specified <emphasis>interval</emphasis> (in seconds).</para>
+ <para>The routerstat utility displays LNet router statistics. If no <literal><replaceable>interval</replaceable></literal> is specified, then statistics are sampled and printed only one time. Otherwise, statistics are sampled and printed at the specified <literal><replaceable>interval</replaceable></literal> (in seconds).</para>
</section>
<section remap="h5">
- <title>Options</title>
+ <title>Output</title>
<para>The routerstat output includes the following fields:</para>
<informaltable frame="all">
<tgroup cols="2">
<thead>
<row>
<entry>
- <para><emphasis role="bold">Option</emphasis></para>
+ <para><emphasis role="bold">Output</emphasis></para>
</entry>
<entry>
<para><emphasis role="bold">Description</emphasis></para>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">M</emphasis></para>
+ <para> <literal>M</literal></para>
</entry>
<entry>
- <para> msgs_alloc(msgs_max)</para>
+ <para> Number of messages currently being processed by LNet (The maximum number of messages ever processed by LNet concurrently)</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">E</emphasis></para>
+ <para> <literal>E</literal></para>
</entry>
<entry>
- <para> errors</para>
+ <para> Number of LNet errors</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">S</emphasis></para>
+ <para> <literal>S</literal></para>
</entry>
<entry>
- <para> send_count/send_length</para>
+ <para> Total size (length) of messages sent in bytes/ Number of messages sent</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">R</emphasis></para>
+ <para> <literal>R</literal></para>
</entry>
<entry>
- <para> recv_count/recv_length</para>
+ <para> Total size (length) of messages received in bytes/ Number of messages received</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">F</emphasis></para>
+ <para> <literal>F</literal></para>
</entry>
<entry>
- <para> route_count/route_length</para>
+ <para> Total size (length) of messages routed in bytes/ Number of messages routed</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">D</emphasis></para>
+ <para> <literal>D</literal></para>
</entry>
<entry>
- <para> drop_count/drop_length</para>
+ <para> Total size (length) of messages dropped in bytes/ Number of messages dropped</para>
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+ <para>When an <literal><replaceable>interval</replaceable></literal> is specified, additional lines of statistics are printed including the following fields:</para>
+ <informaltable frame="all">
+ <tgroup cols="2">
+ <colspec colname="c1" colwidth="50*"/>
+ <colspec colname="c2" colwidth="50*"/>
+ <thead>
+ <row>
+ <entry>
+ <para><emphasis role="bold">Output</emphasis></para>
+ </entry>
+ <entry>
+ <para><emphasis role="bold">Description</emphasis></para>
+ </entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <para> <literal>M</literal></para>
+ </entry>
+ <entry>
+ <para> Number of messages currently being processed by LNet (The maximum number of messages ever processed by LNet concurrently)</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>E</literal></para>
+ </entry>
+ <entry>
+ <para> Number of LNet errors per second</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>S</literal></para>
+ </entry>
+ <entry>
+ <para> Rate of data sent in Mbytes per second/ Count of messages sent per second</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>R</literal></para>
+ </entry>
+ <entry>
+ <para> Rate of data received in Mbytes per second/ Count of messages received per second</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>F</literal></para>
+ </entry>
+ <entry>
+ <para> Rate of data routed in Mbytes per second/ Count of messages routed per second</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para> <literal>D</literal></para>
+ </entry>
+ <entry>
+ <para> Rate of data dropped in Mbytes per second/ Count of messages dropped per second</para>
</entry>
</row>
</tbody>
</informaltable>
</section>
<section remap="h5">
+ <title>Example</title>
+ <screen># routerstat 1
+M 0(13) E 0 S 117379184/4250 R 878480/4356 F 0/0 D 0/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 8.00/ 8 R 0.00/ 16 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 8.00/ 8 R 0.00/ 16 F 0.00/ 0 D 0.00/0
+M 0( 13) E 0 S 7.00/ 7 R 0.00/ 14 F 0.00/ 0 D 0.00/0
+...</screen>
+ </section>
+ <section remap="h5">
<title>Files</title>
<para>The routerstat utility extracts statistics data from:</para>
<screen>/proc/sys/lnet/stats</screen>
<para>The tunefs.lustre utility modifies configuration information on a Lustre target disk.</para>
<section remap="h5">
<title>Synopsis</title>
- <screen>tunefs.lustre [options] <device></screen>
+ <screen>tunefs.lustre [options] <replaceable>/dev/device</replaceable></screen>
</section>
<section remap="h5">
<title>Description</title>
<para>Changes made here affect a file system only when the target is mounted the next time.</para>
</caution>
<para>With tunefs.lustre, parameters are "additive" -- new parameters are specified in addition to old parameters, they do not replace them. To erase all old tunefs.lustre parameters and just use newly-specified parameters, run:</para>
- <screen>$ tunefs.lustre --erase-params --param=<new parameters> </screen>
- <para>The tunefs.lustre command can be used to set any parameter settable in a /proc/fs/lustre file and that has its own OBD device, so it can be specified as <obd|fsname>.<obdtype>.<proc_file_name>=<value>. For example:</para>
- <screen>$ tunefs.lustre --param mdt.group_upcall=NONE /dev/sda1</screen>
+ <screen>$ tunefs.lustre --erase-params --param=<replaceable>new_parameters</replaceable> </screen>
+ <para>The tunefs.lustre command can be used to set any parameter settable in a /proc/fs/lustre file and that has its own OBD device, so it can be specified as <replaceable>{obd|fsname}.obdtype.proc_file_name=value</replaceable>. For example:</para>
+ <screen>$ tunefs.lustre --param mdt.identity_upcall=NONE /dev/sda1</screen>
</section>
<section remap="h5">
<title>Options</title>
<tbody>
<row>
<entry>
- <para> <emphasis role="bold">--comment</emphasis>=<emphasis>comment</emphasis></para>
+ <para> <literal>--comment=<replaceable>comment</replaceable></literal></para>
</entry>
<entry>
<para> Sets a user comment about this disk, ignored by Lustre.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--dryrun</emphasis></para>
+ <para> <literal>--dryrun</literal></para>
</entry>
<entry>
<para> Only prints what would be done; does not affect the disk.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--erase-params</emphasis></para>
+ <para> <literal>--erase-params</literal></para>
</entry>
<entry>
<para> Removes all previous parameter information.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--failnode</emphasis>=<emphasis>nid</emphasis>,...</para>
- </entry>
- <entry>
- <para> Sets the NID(s) of a failover partner. This option can be repeated as needed.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: Cannot be used with <emphasis role="bold">--servicenode</emphasis>.</para>
- </entry>
+ <literal>--servicenode=<replaceable>nid,...</replaceable></literal></entry>
+ <entry>Sets the NID(s) of all service nodes, including primary and failover partner
+ service nodes. The <literal>--servicenode</literal> option cannot be used with
+ <literal>--failnode</literal> option. See <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/> for
+ more details.</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--servicenode</emphasis>=<emphasis>nid</emphasis>,...</para>
+ <para> <literal>--failnode=<replaceable>nid,...</replaceable></literal></para>
</entry>
<entry>
- <para> Sets the NID(s) of all service node, including failover partner as well as local service nids. This option can be repeated as needed.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: Cannot be used with <emphasis role="bold">--failnode</emphasis>.</para>
+ <para>Sets the NID(s) of a failover service node for a primary server for a target.
+ The <literal>--failnode</literal> option cannot be used with
+ <literal>--servicenode</literal> option. See <xref
+ xmlns:xlink="http://www.w3.org/1999/xlink" linkend="dbdoclet.50438188_92688"/>
+ for more details.<note>
+ <para>When the <literal>--failnode</literal> option is used, certain
+ restrictions apply (see <xref xmlns:xlink="http://www.w3.org/1999/xlink"
+ linkend="dbdoclet.50438188_92688"/>).</para>
+ </note></para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--fsname</emphasis>=<emphasis>filesystem_name</emphasis></para>
+ <para> <literal>--fsname=<replaceable>filesystem_name</replaceable></literal></para>
</entry>
<entry>
- <para> The Lustre file system of which this service will be a part. The default file system name is 'lustreâ€.</para>
+ <para> The Lustre file system of which this service will be a part. The default file
+ system name is <literal>lustre</literal>.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--index</emphasis>=index</para>
+ <para> <literal>--index=<replaceable>index</replaceable></literal></para>
</entry>
<entry>
<para> Forces a particular OST or MDT index.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--mountfsoptions</emphasis>=opts</para>
+ <para> <literal>--mountfsoptions=<replaceable>opts</replaceable></literal></para>
</entry>
<entry>
<para> Sets the mount options used when the backing file system is mounted.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: Unlike earlier versions of tunefs.lustre, this version completely replaces the existing mount options with those specified on the command line, and issues a warning on stderr if any default mount options are omitted.</para>
+ <warning><para> Unlike earlier versions of tunefs.lustre, this version completely replaces the existing mount options with those specified on the command line, and issues a warning on stderr if any default mount options are omitted.</para></warning>
<para>The defaults for ldiskfs are:</para>
- <para>OST: <emphasis>errors=remount-ro,mballoc,extents</emphasis>;</para>
- <para>MGS/MDT: <emphasis>errors=remount-ro,iopen_nopriv,user_xattr</emphasis></para>
+ <para>MGS/MDT: <literal>errors=remount-ro,iopen_nopriv,user_xattr</literal></para>
+ <para>OST: <literal>errors=remount-ro,extents,mballoc</literal></para>
+ <para condition='l25'>OST: <literal>errors=remount-ro</literal></para>
<para>Do not alter the default mount options unless you know what you are doing.</para>
</entry>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--network</emphasis>=net,...</para>
+ <para> <literal>--network=<replaceable>net,...</replaceable></literal></para>
</entry>
<entry>
<para> Network(s) to which to restrict this OST/MDT. This option can be repeated as necessary.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--mgs</emphasis></para>
+ <para> <literal>--mgs</literal></para>
</entry>
<entry>
<para> Adds a configuration management service to this target.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--msgnode</emphasis>=<emphasis>nid</emphasis>,...</para>
+ <para> <literal>--msgnode=<replaceable>nid,...</replaceable></literal></para>
</entry>
<entry>
<para> Sets the NID(s) of the MGS node; required for all targets other than the MGS.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--nomgs</emphasis></para>
+ <para> <literal>--nomgs</literal></para>
</entry>
<entry>
<para> Removes a configuration management service to this target.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--quiet</emphasis></para>
+ <para> <literal>--quiet</literal></para>
</entry>
<entry>
<para> Prints less information.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--verbose</emphasis></para>
+ <para> <literal>--verbose</literal></para>
</entry>
<entry>
<para> Prints more information.</para>
</row>
<row>
<entry>
- <para> <emphasis role="bold">--writeconf</emphasis></para>
+ <para> <literal>--writeconf</literal></para>
</entry>
<entry>
- <para> Erases all configuration logs for the file system to which this MDT belongs, and regenerates them. This is dangerous operation. All clients must be unmounted and servers for this file system should be stopped. All targets (OSTs/MDTs) must then be restarted to regenerate the logs. No clients should be started until all targets have restarted.</para>
- <para> </para>
+ <para> Erases all configuration logs for the file system to which this MDT belongs,
+ and regenerates them. This is dangerous operation. All clients must be unmounted
+ and servers for this file system should be stopped. All targets (OSTs/MDTs) must
+ then be restarted to regenerate the logs. No clients should be started until all
+ targets have restarted.</para>
<para>The correct order of operations is:</para>
- <para>* Unmount all clients on the file system</para>
- <para>* Unmount the MDT and all OSTs on the file system</para>
- <para>* Run <emphasis role="bold">tunefs.lustre --writeconf <device></emphasis> on every server</para>
- <para>* Mount the MDT and OSTs</para>
- <para>* Mount the clients</para>
+ <orderedlist>
+ <listitem>
+ <para>Unmount all clients on the file system</para>
+ </listitem>
+ <listitem>
+ <para>Unmount the MDT and all OSTs on the file system</para>
+ </listitem>
+ <listitem>
+ <para>Run <literal>tunefs.lustre --writeconf
+ <replaceable>device</replaceable></literal> on every server</para>
+ </listitem>
+ <listitem>
+ <para>Mount the MDT and OSTs</para>
+ </listitem>
+ <listitem>
+ <para>Mount the clients</para>
+ </listitem>
+ </orderedlist>
</entry>
</row>
</tbody>
<section remap="h5">
<title>Examples</title>
<para>Change the MGS's NID address. (This should be done on each target disk, since they should all contact the same MGS.)</para>
- <screen>tunefs.lustre --erase-param --mgsnode=<new_nid> --writeconf /dev/sda</screen>
+ <screen>tunefs.lustre --erase-param --mgsnode=<replaceable>new_nid</replaceable> --writeconf /dev/sda</screen>
<para>Add a failover NID location for this target.</para>
<screen>tunefs.lustre --param="failover.node=192.168.0.13@tcp0" /dev/sda </screen>
</section>
<title><indexterm><primary>utilities</primary><secondary>application profiling</secondary></indexterm>
Application Profiling Utilities</title>
<para>The following utilities are located in /usr/bin.</para>
- <para><emphasis role="bold">lustre_req_history.sh</emphasis></para>
+ <para><literal>lustre_req_history.sh</literal></para>
<para>The lustre_req_history.sh utility (run from a client), assembles as much Lustre RPC request history as possible from the local node and from the servers that were contacted, providing a better picture of the coordinated network activity.</para>
- <para><emphasis role="bold">llstat.sh</emphasis></para>
- <para>The llstat.sh utility handles a wider range of statistics files, and has command line switches to produce more graphable output.</para>
- <para><emphasis role="bold">plot-llstat.sh</emphasis></para>
- <para>The plot-llstat.sh utility plots the output from llstat.sh using gnuplot.</para>
</section>
<section remap="h3">
<title>More /proc Statistics for Application Profiling</title>
<para>The following utilities provide additional statistics.</para>
- <para><emphasis role="bold">vfs_ops_stats</emphasis></para>
+ <para><literal>vfs_ops_stats</literal></para>
<para>The client vfs_ops_stats utility tracks Linux VFS operation calls into Lustre for a single PID, PPID, GID or everything.</para>
<screen>/proc/fs/lustre/llite/*/vfs_ops_stats
/proc/fs/lustre/llite/*/vfs_track_[pid|ppid|gid]
</screen>
- <para><emphasis role="bold">extents_stats</emphasis></para>
+ <para><literal>extents_stats</literal></para>
<para>The client extents_stats utility shows the size distribution of I/O calls from the client (cumulative and by process).</para>
<screen>/proc/fs/lustre/llite/*/extents_stats, extents_stats_per_process
</screen>
- <para><emphasis role="bold">offset_stats</emphasis></para>
+ <para><literal>offset_stats</literal></para>
<para>The client offset_stats utility shows the read/write seek activity of a client by offsets and ranges.</para>
<screen>/proc/fs/lustre/llite/*/offset_stats
</screen>
<para> Per-client statistics tracked on the servers</para>
</listitem>
</itemizedlist>
- <para>Each MDT and OST now tracks LDLM and operations statistics for every connected client, for comparisons and simpler collection of distributed job statistics.</para>
+ <para>Each MDS and OSS now tracks LDLM and operations statistics for
+ every connected client, for comparisons and simpler collection of
+ distributed job statistics.</para>
<screen>/proc/fs/lustre/mds|obdfilter/*/exports/
</screen>
<itemizedlist>
<para> Improved MDT statistics</para>
</listitem>
</itemizedlist>
- <para>More detailed MDT operations statistics are collected for better profiling.</para>
- <screen>/proc/fs/lustre/mds/*/stats
+ <para>More detailed MDT operations statistics are collected for better
+ profiling.</para>
+ <screen>/proc/fs/lustre/mdt/*/md_stats
</screen>
</section>
<section remap="h3">
Testing / Debugging Utilities</title>
<para>Lustre offers the following test and debugging utilities.</para>
<section remap="h5">
- <title><indexterm><primary>loadgen</primary></indexterm>
-loadgen</title>
- <para>The Load Generator (loadgen) is a test program designed to simulate large numbers of Lustre clients connecting and writing to an OST. The loadgen utility is located at lustre/utils/loadgen (in a build directory) or at /usr/sbin/loadgen (from an RPM).</para>
- <para>Loadgen offers the ability to run this test:</para>
- <orderedlist>
- <listitem>
- <para>Start an arbitrary number of (echo) clients.</para>
- </listitem>
- <listitem>
- <para>Start and connect to an echo server, instead of a real OST.</para>
- </listitem>
- <listitem>
- <para>Create/bulk_write/delete objects on any number of echo clients simultaneously.</para>
- </listitem>
- </orderedlist>
- <para>Currently, the maximum number of clients is limited by MAX_OBD_DEVICES and the amount of memory available.</para>
- </section>
- <section remap="h5">
- <title>Usage</title>
- <para>The loadgen utility can be run locally on the OST server machine or remotely from any LNET host. The device command can take an optional NID as a parameter; if unspecified, the first local NID found is used.</para>
- <para>The obdecho module must be loaded by hand before running loadgen.</para>
- <screen># cd lustre/utils/
-# insmod ../obdecho/obdecho.ko
-# ./loadgen
-loadgen> h
-This is a test program used to simulate large numbers of clients. The echo \
-obds are used, so the obdecho module must be loaded.
-
-Typical usage would be:
-loadgen> dev lustre-OST0000 set the target device
-loadgen> start 20 start 20 echo clients
-loadgen> wr 10 5 have 10 clients do simultaneous brw_write \
-tests 5 times each
-
-Available commands are:
- device
- dl
- echosrv
- start
- verbose
- wait
- write
- help
- exit
- quit
-
-For more help type: help command-name
-loadgen>
-loadgen> device lustre-OST0000 192.168.0.21@tcp
-Added uuid OSS_UUID: 192.168.0.21@tcp
-Target OST name is 'lustre-OST0000'
-loadgen>
-loadgen> st 4
-start 0 to 4
-./loadgen: running thread #1
-./loadgen: running thread #2
-./loadgen: running thread #3
-./loadgen: running thread #4
-loadgen> wr 4 5
-Estimate 76 clients before we run out of grant space (155872K / 2097152)
-1: i0
-2: i0
-4: i0
-3: i0
-1: done (0)
-2: done (0)
-4: done (0)
-3: done (0)
-wrote 25MB in 1.419s (17.623 MB/s)
-loadgen>
-</screen>
- <para>The loadgen utility prints periodic status messages; message output can be controlled with the verbose command.</para>
- <para>To insure that a file can be written to (a requirement of write cache), OSTs reserve ("grants"), chunks of space for each newly-created file. A grant may cause an OST to report that it is out of space, even though there is plenty of space on the disk, because the space is "reserved" by other files. The loadgen utility estimates the number of simultaneous open files as the disk size divided by the grant size and reports that number when the write tests are first started.</para>
- <para><emphasis role="bold">Echo Server</emphasis></para>
- <para>The loadgen utility can start an echo server. On another node, loadgen can specify the echo server as the device, thus creating a network-only test environment.</para>
- <screen>loadgen> echosrv
-loadgen> dl
- 0 UP obdecho echosrv echosrv 3
- 1 UP ost OSS OSS 3
-</screen>
- <para>On another node:</para>
- <screen>loadgen> device echosrv cfs21@tcp
-Added uuid OSS_UUID: 192.168.0.21@tcp
-Target OST name is 'echosrv'
-loadgen> st 1
-start 0 to 1
-./loadgen: running thread #1
-loadgen> wr 1
-start a test_brw write test on X clients for Y iterations
-usage: write <num_clients> <num_iter> [<delay>]
-loadgen> wr 1 1
-loadgen>
-1: i0
-1: done (0)
-wrote 1MB in 0.029s (34.023 MB/s)
-</screen>
- <para>Scripting</para>
- <para>The threads all perform their actions in non-blocking mode; use the wait command to block for the idle state. For example:</para>
- <screen>#!/bin/bash
-./loadgen << EOF
-device lustre-OST0000
-st 1
-wr 1 10
-wait
-quit
-EOF
-</screen>
- <para><emphasis role="bold">Feature Requests</emphasis></para>
- <para>The loadgen utility is intended to grow into a more comprehensive test tool; feature requests are encouraged. The current feature requests include:</para>
- <itemizedlist>
- <listitem>
- <para> Locking simulation</para>
- </listitem>
- <listitem>
- <itemizedlist>
- <listitem>
- <para> Many (echo) clients cache locks for the specified resource at the same time.</para>
- </listitem>
- <listitem>
- <para> Many (echo) clients enqueue locks for the specified resource simultaneously.</para>
- </listitem>
- </itemizedlist>
- </listitem>
- <listitem>
- <para> obdsurvey functionality</para>
- </listitem>
- <listitem>
- <itemizedlist>
- <listitem>
- <para> Fold the Lustre I/O kit's obdsurvey script functionality into loadgen</para>
- </listitem>
- </itemizedlist>
- </listitem>
- </itemizedlist>
- </section>
- <section remap="h5">
- <title><indexterm><primary>llog_reader</primary></indexterm>
-llog_reader</title>
- <para>The llog_reader utility translates a Lustre configuration log into human-readable form.</para>
- </section>
- <section remap="h5">
- <title>Synopsis</title>
- <screen>llog_reader filename
-</screen>
- </section>
- <section remap="h5">
- <title>Description</title>
- <para>llog_reader parses the binary format of Lustre's on-disk configuration logs. It can only read the logs. Use tunefs.lustre to write to them.</para>
- <para>To examine a log file on a stopped Lustre server, mount its backing file system as ldiskfs, then use llog_reader to dump the log file's contents. For example:</para>
- <screen>mount -t ldiskfs /dev/sda /mnt/mgs
-llog_reader /mnt/mgs/CONFIGS/tfs-client
-</screen>
- <para>To examine the same log file on a running Lustre server, use the ldiskfs-enabled debugfs utility (called debug.ldiskfs on some distributions) to extract the file. For example:</para>
- <screen>debugfs -c -R 'dump CONFIGS/tfs-client /tmp/tfs-client' /dev/sda
-llog_reader /tmp/tfs-client
-</screen>
- <caution>
- <para>Although they are stored in the CONFIGS directory, mountdata files do not use the config log format and will confuse llog_reader.</para>
- </caution>
- <para><emphasis role="bold">See Also</emphasis></para>
- <para><xref linkend="dbdoclet.50438219_39574"/></para>
- </section>
- <section remap="h5">
<title><indexterm><primary>lr_reader</primary></indexterm>
lr_reader</title>
- <para>The lr_reader utility translates a last received (last_rcvd) file into human-readable form.</para>
- <para>The following utilites are part of the Lustre I/O kit. For more information, see <xref linkend="benchmarkingtests"/>.</para>
+ <para>The lr_reader utility translates the content of the <literal>last_rcvd</literal> and <literal>reply_data</literal> files into human-readable form.</para>
+ <para>The following utilities are part of the Lustre I/O kit. For more information, see <xref linkend="benchmarkingtests"/>.</para>
</section>
<section remap="h5">
- <title><indexterm><primary>sgpdd_survey</primary></indexterm>
-sgpdd_survey</title>
- <para>The sgpdd_survey utility tests 'bare metal' performance, bypassing as much of the kernel as possible. The sgpdd_survey tool does not require Lustre, but it does require the sgp_dd package.</para>
+ <title><indexterm>
+ <primary>sgpdd-survey</primary>
+ </indexterm> sgpdd-survey</title>
+ <para>The <literal>sgpdd-survey</literal> utility tests 'bare metal' performance,
+ bypassing as much of the kernel as possible. The <literal>sgpdd-survey</literal> tool does
+ not require Lustre, but it does require the sgp_dd package.</para>
<caution>
- <para>The sgpdd_survey utility erases all data on the device.</para>
+ <para>The <literal>sgpdd-survey</literal> utility erases all data on the device.</para>
</caution>
</section>
<section remap="h5">
- <title><indexterm><primary>obdfilter_survey</primary></indexterm>obdfilter_survey</title>
- <para>The obdfilter_survey utility is a shell script that tests performance of isolated OSTS, the network via echo clients, and an end-to-end test.</para>
+ <title><indexterm>
+ <primary>obdfilter-survey</primary>
+ </indexterm>obdfilter-survey</title>
+ <para>The <literal>obdfilter-survey</literal> utility is a shell script that tests
+ performance of isolated OSTS, the network via echo clients, and an end-to-end test.</para>
</section>
<section remap="h5">
<title><indexterm><primary>ior-survey</primary></indexterm>ior-survey</title>
<para>The ior-survey utility is a script used to run the IOR benchmark. Lustre includes IOR version 2.8.6.</para>
</section>
<section remap="h5">
- <title><indexterm><primary>ost_survey</primary></indexterm>ost_survey</title>
- <para>The ost_survey utility is an OST performance survey that tests client-to-disk performance of the individual OSTs in a Lustre file system.</para>
+ <title><indexterm>
+ <primary>ost-survey</primary>
+ </indexterm>ost-survey</title>
+ <para>The <literal>ost-survey</literal> utility is an OST performance survey that tests
+ client-to-disk performance of the individual OSTs in a Lustre file system.</para>
</section>
<section remap="h5">
<title><indexterm><primary>stats-collect</primary></indexterm>stats-collect</title>
<para>The stats-collect utility contains scripts used to collect application profiling information from Lustre clients and servers.</para>
</section>
</section>
- <section remap="h3">
- <title><indexterm><primary>flock</primary></indexterm>Flock Feature</title>
- <para>Lustre now includes the flock feature, which provides file locking support. Flock describes classes of file locks known as 'flocks'. Flock can apply or remove a lock on an open file as specified by the user. However, a single file may not, simultaneously, have both shared and exclusive locks.</para>
- <para>By default, the flock utility is disabled on Lustre. Two modes are available.</para>
- <informaltable frame="none">
- <tgroup cols="2">
- <colspec colname="c1" colwidth="50*"/>
- <colspec colname="c2" colwidth="50*"/>
- <tbody>
- <row>
- <entry>
- <para> <emphasis role="bold">local mode</emphasis></para>
- </entry>
- <entry>
- <para> In this mode, locks are coherent on one node (a single-node flock), but not across all clients. To enable it, use -o localflock. This is a client-mount option.</para>
- <note>
- <para>This mode does not impact performance and is appropriate for single-node databases.</para>
- </note>
- </entry>
- </row>
- <row>
- <entry>
- <para> <emphasis role="bold">consistent mode</emphasis></para>
- </entry>
- <entry>
- <para> In this mode, locks are coherent across all clients.</para>
- <para> To enable it, use the -o flock. This is a client-mount option.</para>
- <para><emphasis role="bold">CAUTION</emphasis>: This mode affects the performance of the file being flocked and may affect stability, depending on the Lustre version used. Consider using a newer Lustre version which is more stable. If the consistent mode is enabled and no applications are using flock, then it has no effect.</para>
- </entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <para>A call to use flock may be blocked if another process is holding an incompatible lock. Locks created using flock are applicable for an open file table entry. Therefore, a single process may hold only one type of lock (shared or exclusive) on a single file. Subsequent flock calls on a file that is already locked converts the existing lock to the new lock mode.</para>
+ <section remap="h3" condition='l29'>
+ <title><indexterm><primary>fileset</primary></indexterm>Fileset Feature</title>
+ <para> With the fileset feature, Lustre now provides subdirectory mount
+ support. Subdirectory mounts, also referred to as filesets, allow a
+ client to mount a child directory of a parent filesystem, thereby limiting
+ the filesystem namespace visibility on a specific client. A common use
+ case is for a client to use a subdirectory mount when there is a desire to
+ limit the visibility of the entire filesystem namesapce to aid in the
+ prevention of accidental file deletions outside of the subdirectory
+ mount.</para>
+ <para>It is important to note that invocation of the subdirectory mount is
+ voluntary by the client and not does prevent access to files that are
+ visible in multiple subdirectory mounts via hard links. Furthermore, it
+ does not prevent the client from subsequently mounting the whole file
+ system without a subdirectory being specified.</para>
+ <figure xml:id="understandinglustre.fig.fileset">
+ <title>
+ <indexterm>
+ <primary>Lustre</primary>
+ <secondary>fileset</secondary>
+ </indexterm>Lustre fileset</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata scalefit="1" width="100%"
+ fileref="./figures/fileset.png" />
+ </imageobject>
+ <textobject>
+ <phrase>Lustre file system fileset feature</phrase>
+ </textobject>
+ </mediaobject>
+ </figure>
<section remap="h4">
- <title>Example</title>
- <screen>$ mount -t lustre -o flock mds@tcp0:/lustre /mnt/client</screen>
- <para>You can check it in /etc/mtab. It should look like,</para>
- <screen>mds@tcp0:/lustre /mnt/client lustre rw,flock 0 0</screen>
+ <title>Examples</title>
+ <para>The following example will mount the
+ <literal>chipfs</literal> filesystem on client1 and create a
+ subdirectory <literal>v1_1</literal> within that filesystem. Client2
+ will then mount only the <literal>v1_1</literal> subdirectory as a
+ fileset, thereby limiting access to anything else in the
+ <literal>chipfs</literal> filesystem from client2.</para>
+ <screen>client1# mount -t lustre mgs@tcp:/chipfs /mnt/chip
+client1# mkdir /mnt/chip/v1_1</screen>
+ <screen>client2# mount -t lustre mgs@tcp:/chipfs/v1_1 /mnt/chipv1_1</screen>
+ <para>You can check the created mounts in /etc/mtab. It should look like
+ the following:</para>
+ <screen><replaceable>client1</replaceable>
+mds@tcp0:/chipfs/ /mnt/chip lustre rw 0 0
+</screen><screen>
+<replaceable>client2</replaceable>
+mds@tcp0:/chipfs/v1_1 /mnt/chipv1_1 lustre rw 0 0</screen>
+ <para>Create a directory under the /mnt/chip mount, and get its FID</para>
+ <screen>client1# mkdir /mnt/chip/v1_2
+client1# lfs path2fid /mnt/chip/v1_2
+[0x200000400:0x2:0x0]
+</screen>
+ <para>If you try resolve the FID of the <literal>/mnt/chip/v1_2</literal>
+ path (as created in the example above) on client2, an error will be returned
+ as the FID can not be resolved on client2 since it is not part of the
+ mounted fileset on that client. Recall that the fileset on client2 mounted
+ the <literal>v1_1</literal> subdirectory beneath the top level
+ <literal>chipfs</literal> filesystem.
+ </para>
+ <screen>client2# lfs fid2path /mnt/chip/v1_2 [0x200000400:0x2:0x0]
+fid2path: error on FID [0x200000400:0x2:0x0]: No such file or directory</screen>
+ <para>Subdirectory mounts do not have the <literal>.lustre</literal>
+ pseudo directory, which prevents clients from opening or accessing files
+ only by FID.</para>
+ <screen>client1# ls /mnt/chipfs/.lustre
+ fid lost+found</screen>
+ <screen>client2# ls /mnt/chipv1_1/.lustre
+ ls: cannot access /mnt/chipv1_1/.lustre: No such file or directory
+ </screen>
</section>
</section>
</section>
git://git.whamcloud.com / doc / manual.git / blobdiff