<para>To mount a client or an MDT with one or more inactive OSTs, run commands similar to this:</para>
<screen>client# mount -o exclude=testfs-OST0000 -t lustre \
uml1:/testfs /mnt/testfs
- client# cat /proc/fs/lustre/lov/testfs-clilov-*/target_obd</screen>
- <para>To activate an inactive OST on a live client or MDT, use the <literal>lctl activate</literal> command on the OSC device. For example:</para>
+ client# lctl get_param lov.testfs-clilov-*.target_obd</screen>
+ <para>To activate an inactive OST on a live client or MDT, use the
+ <literal>lctl activate</literal> command on the OSC device. For example:</para>
<screen>lctl --device 7 activate</screen>
<note>
- <para>A colon-separated list can also be specified. For example, <literal>exclude=testfs-OST0000:testfs-OST0001</literal>.</para>
+ <para>A colon-separated list can also be specified. For example,
+ <literal>exclude=testfs-OST0000:testfs-OST0001</literal>.</para>
</note>
</section>
<section xml:id="dbdoclet.50438199_15240">
<title><indexterm><primary>maintenance</primary><secondary>finding nodes</secondary></indexterm>
Finding Nodes in the Lustre File System</title>
- <para>There may be situations in which you need to find all nodes in your Lustre file system or get the names of all OSTs.</para>
+ <para>There may be situations in which you need to find all nodes in
+ your Lustre file system or get the names of all OSTs.</para>
<para>To get a list of all Lustre nodes, run this command on the MGS:</para>
- <screen># cat /proc/fs/lustre/mgs/MGS/live/*</screen>
+ <screen># lctl get_param mgs.MGS.live.*</screen>
<note>
- <para>This command must be run on the MGS.
- </para>
+ <para>This command must be run on the MGS.</para>
</note>
- <para>In this example, file system <literal>testfs</literal> has three nodes,
- <literal>testfs-MDT0000</literal>, <literal>testfs-OST0000</literal>, and
- <literal>testfs-OST0001</literal>.</para>
- <screen>cfs21:/tmp# cat /proc/fs/lustre/mgs/MGS/live/*
+ <para>In this example, file system <literal>testfs</literal> has three
+ nodes, <literal>testfs-MDT0000</literal>,
+ <literal>testfs-OST0000</literal>, and
+ <literal>testfs-OST0001</literal>.</para>
+ <screen>mgs:/root# lctl get_param mgs.MGS.live.*
fsname: testfs
flags: 0x0 gen: 26
testfs-MDT0000
testfs-OST0000
testfs-OST0001 </screen>
<para>To get the names of all OSTs, run this command on the MDS:</para>
- <screen># cat /proc/fs/lustre/lov/<replaceable>fsname</replaceable>-mdtlov/target_obd </screen>
+ <screen>mds:/root# lctl get_param lov.*-mdtlov.target_obd </screen>
<note>
- <para>This command must be run on the MDS.
- </para>
+ <para>This command must be run on the MDS.</para>
</note>
- <para>In this example, there are two OSTs, testfs-OST0000 and testfs-OST0001, which are both
- active.</para>
- <screen>cfs21:/tmp# cat /proc/fs/lustre/lov/testfs-mdtlov/target_obd
+ <para>In this example, there are two OSTs, testfs-OST0000 and
+ testfs-OST0001, which are both active.</para>
+ <screen>mgs:/root# lctl get_param lov.testfs-mdtlov.target_obd
0: testfs-OST0000_UUID ACTIVE
1: testfs-OST0001_UUID ACTIVE </screen>
</section>
<para>To change a server NID:</para>
<orderedlist>
<listitem>
- <para>Update the LNET configuration in the <literal>/etc/modprobe.conf</literal> file so the list of server NIDs is correct. Use <literal>lctl list_nids</literal> to view the list of server NIDS.</para>
+ <para>Update the LNet configuration in the <literal>/etc/modprobe.conf</literal> file so the list of server NIDs is correct. Use <literal>lctl list_nids</literal> to view the list of server NIDS.</para>
<para>The <literal>lctl list_nids</literal> command indicates which network(s) are
configured to work with the Lustre file system.</para>
</listitem>
<para>where <replaceable>devicename</replaceable> is the Lustre target name, e.g.
<literal>testfs-OST0013</literal></para>
</listitem>
- <listitem>
- <para>If the MGS and MDS share a partition, stop the MGS:</para>
- <screen>umount <replaceable>mount_point</replaceable></screen>
- </listitem>
+ <listitem>
+
<para>If the MGS and MDS share a partition, stop the MGS:</para>
+ <screen>umount <replaceable>mount_point</replaceable></screen>
+ </listitem>
</orderedlist>
- <note><para>The <literal>replace_nids</literal> command also cleans all old, invalidated records out of the configuration log, while preserving all other current settings.</para></note>
- <note><para>The previous configuration log is backed up on the MGS disk with the suffix <literal>'.bak'</literal>.</para></note>
+ <note><para>The <literal>replace_nids</literal> command also cleans
+ all old, invalidated records out of the configuration log, while
+ preserving all other current settings.</para></note>
+ <note><para>The previous configuration log is backed up on the MGS
+ disk with the suffix <literal>'.bak'</literal>.</para></note>
</section>
<section xml:id="dbdoclet.addingamdt" condition='l24'>
<title><indexterm>
<primary>maintenance</primary>
<secondary>adding an MDT</secondary>
</indexterm>Adding a New MDT to a Lustre File System</title>
- <para>Additional MDTs can be added to serve one or more remote sub-directories within the
- file system. It is possible to have multiple remote sub-directories reference the same MDT.
- However, the root directory will always be located on MDT0. To add a new MDT into the file
- system:</para>
+ <para>Additional MDTs can be added using the DNE feature to serve one
+ or more remote sub-directories within a filesystem, in order to
+ increase the total number of files that can be created in the
+ filesystem, to increase aggregate metadata performance, or to isolate
+ user or application workloads from other users of the filesystem. It
+ is possible to have multiple remote sub-directories reference the
+ same MDT. However, the root directory will always be located on
+ MDT0. To add a new MDT into the file system:</para>
<orderedlist>
<listitem>
- <para>Discover the maximum MDT index. Each MDTs must have unique index.</para>
- <screen>
+ <para>Discover the maximum MDT index. Each MDT must have unique index.</para>
+<screen>
client$ lctl dl | grep mdc
36 UP mdc testfs-MDT0000-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
37 UP mdc testfs-MDT0001-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
38 UP mdc testfs-MDT0002-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
39 UP mdc testfs-MDT0003-mdc-ffff88004edf3c00 4c8be054-144f-9359-b063-8477566eb84e 5
- </screen>
+</screen>
</listitem>
<listitem>
- <para>Add the new block device as a new MDT at the next available index. In this example, the next available index is 4.</para>
- <screen>
-mds# mkfs.lustre --reformat --fsname=<replaceable>filesystem_name</replaceable> --mdt --mgsnode=<replaceable>mgsnode</replaceable> --index 4 <replaceable>/dev/mdt4_device</replaceable>
- </screen>
+ <para>Add the new block device as a new MDT at the next available
+ index. In this example, the next available index is 4.</para>
+<screen>
+mds# mkfs.lustre --reformat --fsname=<replaceable>testfs</replaceable> --mdt --mgsnode=<replaceable>mgsnode</replaceable> --index 4 <replaceable>/dev/mdt4_device</replaceable>
+</screen>
</listitem>
<listitem>
-
<para>Mount the MDTs.</para>
- <screen>
+ <para>Mount the MDTs.</para>
+<screen>
mds# mount –t lustre <replaceable>/dev/mdt4_blockdevice</replaceable> /mnt/mdt4
- </screen>
+</screen>
+ </listitem>
+ <listitem>
+ <para>In order to start creating new files and directories on the
+ new MDT(s) they need to be attached into the namespace at one or
+ more subdirectories using the <literal>lfs mkdir</literal> command.
+ All files and directories below those created with
+ <literal>lfs mkdir</literal> will also be created on the same MDT
+ unless otherwise specified.
+ </para>
+<screen>
+client# lfs mkdir -i 3 /mnt/testfs/new_dir_on_mdt3
+client# lfs mkdir -i 4 /mnt/testfs/new_dir_on_mdt4
+client# lfs mkdir -c 4 /mnt/testfs/new_directory_striped_across_4_mdts
+</screen>
</listitem>
</orderedlist>
- </section>
+ </section>
<section xml:id="dbdoclet.50438199_22527">
<title><indexterm><primary>maintenance</primary><secondary>adding a OST</secondary></indexterm>
Adding a New OST to a Lustre File System</title>
<para>If the MDT is permanently inaccessible, <literal>lfs rmdir {directory}</literal> can be used to delete the directory entry. A normal <literal>rmdir</literal> will report an IO error due to the remote MDT being inactive. After the remote directory has been removed, the administrator should mark the MDT as permanently inactive with:</para>
<screen>lctl conf_param {MDT name}.mdc.active=0
</screen>
-<para>A user can identify the location of a remote sub-directory using the <literal>lfs</literal> utility. For example:</para>
+<para>A user can identify which MDT holds a remote sub-directory using
+the <literal>lfs</literal> utility. For example:</para>
<screen>client$ lfs getstripe -M /mnt/lustre/remote_dir1
1
client$ mkdir /mnt/lustre/local_dir0
client$ lfs getstripe -M /mnt/lustre/local_dir0
0
</screen>
- <para>The <literal>getstripe -M</literal> parameters return the index of the MDT that is serving the given directory.</para>
+ <para>The <literal>getstripe [--mdt-index|-M]</literal> parameters return
+ the index of the MDT that is serving the given directory.</para>
</section>
<section xml:id="dbdoclet.inactivemdt" condition='l24'>
<title>
<para>Deactivate the OST.</para>
<orderedlist>
<listitem>
- <para>To temporarily disable the deactivated OST, enter:
- <screen>[client]# lctl set_param osc.<replaceable>fsname</replaceable>-<replaceable>OSTnumber</replaceable>-*.active=0</screen>If
- there is expected to be a replacement OST in some short time (a few days), the OST
- can temporarily be deactivated on the clients: <note>
- <para>This setting is only temporary and will be reset if the clients or MDS are
- rebooted. It needs to be run on all clients.</para>
+ <para>
+ If there is expected to be a replacement OST in some short
+ time (a few days), the OST can temporarily be deactivated on
+ the clients using:
+ <screen>client# lctl set_param osc.<replaceable>fsname</replaceable>-OST<replaceable>number</replaceable>-*.active=0</screen>
+ <note><para>This setting is only temporary and will be reset
+ if the clients are remounted or rebooted. It needs to be run
+ on all clients.</para>
</note></para>
</listitem>
</orderedlist>
- <para>If there is not expected to be a replacement for this OST in the near future,
- permanently deactivate the OST on all clients and the MDS:
- <screen>[mgs]# lctl conf_param <replaceable>ost_name</replaceable>.osc.active=0</screen></para>
+ <para>If there is not expected to be a replacement for this OST in
+ the near future, permanently deactivate it on all clients and the MDS
+ by running the following command on the MGS:
+ <screen>mgs# lctl conf_param <replaceable>ost_name</replaceable>.osc.active=0</screen></para>
<note>
- <para>A removed OST still appears in the file system; do not create a new OST with the
- same name.</para>
+ <para>A deactivated OST still appears in the file system
+ configuration, though a new OST with the same name can be
+ created using the <literal>--replace</literal> option for
+ <literal>mkfs.lustre</literal>.</para>
</note>
</listitem>
</orderedlist>
<secondary>restoring OST config</secondary>
</indexterm> Restoring OST Configuration Files</title>
<para>If the original OST is still available, it is best to follow the OST backup and restore
- procedure given in either <xref linkend="dbdoclet.50438207_71633"/>, or <xref
- linkend="dbdoclet.50438207_21638"/> and <xref linkend="dbdoclet.50438207_22325"/>. </para>
+ procedure given in either <xref linkend="dbdoclet.backup_device"/>, or
+ <xref linkend="dbdoclet.backup_target_filesystem"/> and
+ <xref linkend="dbdoclet.restore_target_filesystem"/>.</para>
<para>To replace an OST that was removed from service due to corruption or hardware failure,
the file system needs to be formatted using <literal>mkfs.lustre</literal>, and the Lustre
file system configuration should be restored, if available. </para>
<title><indexterm>
<primary>maintenance</primary>
<secondary>reintroducing an OSTs</secondary>
- </indexterm> Returning a Deactivated OST to Service</title>
- <para> If the OST was permanently deactivated, it needs to be reactivated in the MGS
- configuration.
- <screen>mgs# lctl conf_param <replaceable>ost_name</replaceable>.osc.active=1</screen> If
- the OST was temporarily deactivated, it needs to be reactivated on the MDS and clients.
+ </indexterm>Returning a Deactivated OST to Service</title>
+ <para>If the OST was permanently deactivated, it needs to be
+ reactivated in the MGS configuration.
+ <screen>mgs# lctl conf_param <replaceable>ost_name</replaceable>.osc.active=1</screen>
+ If the OST was temporarily deactivated, it needs to be reactivated on
+ the MDS and clients.
<screen>mds# lctl --device <replaceable>lustre_devno</replaceable> activate
- client# lctl set_param osc.<replaceable>fsname</replaceable>-<replaceable>OSTnumber</replaceable>-*.active=0</screen></para>
+client# lctl set_param osc.<replaceable>fsname</replaceable>-OST<replaceable>number</replaceable>-*.active=1</screen></para>
</section>
</section>
<section xml:id="dbdoclet.50438199_77819">
<para>In the course of administering a Lustre file system, you may need to determine which
machine is serving a specific OST. It is not as simple as identifying the machine’s IP
address, as IP is only one of several networking protocols that the Lustre software uses and,
- as such, LNET does not use IP addresses as node identifiers, but NIDs instead. To identify the
+ as such, LNet does not use IP addresses as node identifiers, but NIDs instead. To identify the
NID that is serving a specific OST, run one of the following commands on a client (you do not
need to be a root user):
- <screen>client$ lctl get_param osc.<replaceable>fsname</replaceable>-<replaceable>OSTnumber</replaceable>*.ost_conn_uuid</screen>For
- example:
+ <screen>client$ lctl get_param osc.<replaceable>fsname</replaceable>-OST<replaceable>number</replaceable>*.ost_conn_uuid</screen>
+ For example:
<screen>client$ lctl get_param osc.*-OST0000*.ost_conn_uuid
osc.testfs-OST0000-osc-f1579000.ost_conn_uuid=192.168.20.1@tcp</screen>-
OR -
<para>Start the MGS.</para>
<screen>mgs# mount -t lustre <replaceable>/dev/mgs_device</replaceable> <replaceable>/mgs_mount_point</replaceable></screen>
<para>Check to make sure it knows about all your file system</para>
- <screen>cat /proc/fs/lustre/mgs/MGS/filesystems</screen>
+ <screen>mgs:/root# lctl get_param mgs.MGS.filesystems</screen>
</listitem>
<listitem>
<para>Remove the MGS option from the MDT, and set the new MGS nid.</para>
<listitem>
<para>Start the MDT.</para>
<screen>mds# mount -t lustre <replaceable>/dev/mdt_device /mdt_mount_point</replaceable></screen>
- <para>Check to make sure the MGS configuration look right</para>
- <screen>mds# cat /proc/fs/lustre/mgs/MGS/live/<replaceable>filesystem_name</replaceable></screen>
+ <para>Check to make sure the MGS configuration looks right:</para>
+ <screen>mgs# lctl get_param mgs.MGS.live.<replaceable>filesystem_name</replaceable></screen>
</listitem>
</orderedlist>
</section>
git://git.whamcloud.com / doc / manual.git / blobdiff