X-Git-Url: https://git.whamcloud.com/?a=blobdiff_plain;f=LustreOperations.xml;h=d5f2e31c07af5cb3f88c1f84767f92ebc7ad29c3;hb=feb018cdf25683c6ebbb0982f6b5c12040c0b9ec;hp=cbe0f63e0d71899cace25b74b3e61eb3d42a12af;hpb=19bb368be1efb6faa537c3d6792b9d34b4de62a7;p=doc%2Fmanual.git diff --git a/LustreOperations.xml b/LustreOperations.xml index cbe0f63..d5f2e31 100644 --- a/LustreOperations.xml +++ b/LustreOperations.xml @@ -134,22 +134,87 @@ LABEL=testfs-OST0000 /mnt/test/ost0 lustre defaults,_netdev,noauto 0 0 environment. -
+
+ + <indexterm> + <primary>operations</primary> + <secondary>shutdownLustre</secondary> + </indexterm>Stopping the Filesystem + A complete Lustre filesystem shutdown occurs by unmounting all + clients and servers in the order shown below. Please note that unmounting + a block device causes the Lustre software to be shut down on that node. + + Please note that the -a -t lustre in the + commands below is not the name of a filesystem, but rather is + specifying to unmount all entries in /etc/mtab that are of type + lustre + + Unmount the clients + On each client node, unmount the filesystem on that client + using the umount command: + umount -a -t lustre + The example below shows the unmount of the + testfs filesystem on a client node: + [root@client1 ~]# mount |grep testfs +XXX.XXX.0.11@tcp:/testfs on /mnt/testfs type lustre (rw,lazystatfs) + +[root@client1 ~]# umount -a -t lustre +[154523.177714] Lustre: Unmounted testfs-client + + Unmount the MDT and MGT + On the MGS and MDS node(s), use the umount + command: + umount -a -t lustre + The example below shows the unmount of the MDT and MGT for + the testfs filesystem on a combined MGS/MDS: + + [root@mds1 ~]# mount |grep lustre +/dev/sda on /mnt/mgt type lustre (ro) +/dev/sdb on /mnt/mdt type lustre (ro) + +[root@mds1 ~]# umount -a -t lustre +[155263.566230] Lustre: Failing over testfs-MDT0000 +[155263.775355] Lustre: server umount testfs-MDT0000 complete +[155269.843862] Lustre: server umount MGS complete + For a seperate MGS and MDS, the same command is used, first on + the MDS and then followed by the MGS. + + Unmount all the OSTs + On each OSS node, use the umount command: + + umount -a -t lustre + The example below shows the unmount of all OSTs for the + testfs filesystem on server + OSS1: + + [root@oss1 ~]# mount |grep lustre +/dev/sda on /mnt/ost0 type lustre (ro) +/dev/sdb on /mnt/ost1 type lustre (ro) +/dev/sdc on /mnt/ost2 type lustre (ro) + +[root@oss1 ~]# umount -a -t lustre +[155336.491445] Lustre: Failing over testfs-OST0002 +[155336.556752] Lustre: server umount testfs-OST0002 complete + + + For unmount command syntax for a single OST, MDT, or MGT target + please refer to +
+
<indexterm> <primary>operations</primary> <secondary>unmounting</secondary> - </indexterm>Unmounting a Server - To stop a Lustre server, use the + Unmounting a Specific Target on a Server + To stop a Lustre OST, MDT, or MGT , use the umount - /mount - point command. - For example, to stop - ost0 on mount point - /mnt/test, run: - -$ umount /mnt/test - + /mount_point command. + The example below stops an OST, ost0, on mount + point /mnt/ost0 for the testfs + filesystem: + [root@oss1 ~]# umount /mnt/ost0 +[ 385.142264] Lustre: Failing over testfs-OST0000 +[ 385.210810] Lustre: server umount testfs-OST0000 complete Gracefully stopping a server with the umount command preserves the state of the connected clients. The next time the server is started, it waits for clients to @@ -227,7 +292,7 @@ $ tunefs.lustre --param failover.mode=failout
-
+
<indexterm> <primary>operations</primary> @@ -259,7 +324,9 @@ lctl get_param obdfilter.*.degraded resets to <literal>0</literal>.</para> <para>It is recommended that this be implemented by an automated script - that monitors the status of individual RAID devices.</para> + that monitors the status of individual RAID devices, such as MD-RAID's + <literal>mdadm(8)</literal> command with the <literal>--monitor</literal> + option to mark an affected device degraded or restored.</para> </section> <section xml:id="dbdoclet.50438194_88063"> <title> @@ -387,8 +454,13 @@ client# lfs mkdir –i will leave the namespace below it inaccessible. For this reason, by default it is only possible to create remote sub-directories off MDT0. To relax this restriction and enable remote sub-directories off any MDT, an - administrator must issue the command - <literal>lctl set_param mdt.*.enable_remote_dir=1</literal>.</para> + administrator must issue the following command on the MGS: + <screen>mgs# lctl conf_param <replaceable>fsname</replaceable>.mdt.enable_remote_dir=1</screen> + For Lustre filesystem 'scratch', the command executed is: + <screen>mgs# lctl conf_param scratch.mdt.enable_remote_dir=1</screen> + To verify the configuration setting execute the following command on any + MDS: + <screen>mds# lctl get_param mdt.*.enable_remote_dir</screen></para> </warning> <para condition='l28'>With Lustre software version 2.8, a new tunable is available to allow users with a specific group ID to create @@ -397,9 +469,13 @@ client# lfs mkdir –i parameter to the 'wheel' or 'admin' group ID allows users with that GID to create and delete remote and striped directories. Setting this parameter to <literal>-1</literal> on MDT0 to permanently allow any - non-root users create and delete remote and striped directories. For - example: - <screen>lctl set_param -P mdt.*.enable_remote_dir_gid=-1</screen> + non-root users create and delete remote and striped directories. + On the MGS execute the following command: + <screen>mgs# lctl conf_param <replaceable>fsname</replaceable>.mdt.enable_remote_dir_gid=-1</screen> + For the Lustre filesystem 'scratch', the commands expands to: + <screen>mgs# lctl conf_param scratch.mdt.enable_remote_dir_gid=-1</screen>. + The change can be verified by executing the following command on every MDS: + <screen>mds# lctl get_param mdt.<replaceable>*</replaceable>.enable_remote_dir_gid</screen> </para> </section> <section xml:id="dbdoclet.lfsmkdirdne2" condition='l28'> @@ -423,7 +499,7 @@ client# lfs mkdir –i <para>The Lustre 2.8 DNE feature enables individual files in a given directory to store their metadata on separate MDTs (a <emphasis>striped directory</emphasis>) once additional MDTs have been added to the - filesystem, see <xref linkend="dbdoclet.addingamdt"/>. + filesystem, see <xref linkend="dbdoclet.adding_new_mdt"/>. The result of this is that metadata requests for files in a striped directory are serviced by multiple MDTs and metadata service load is distributed over all the MDTs that service a given @@ -634,7 +710,7 @@ lctl set_param -P -d </section> <section xml:id="dbdoclet.50438194_88217"> <title>Listing Parameters - To list Lustre or LNET parameters that are available to set, use + To list Lustre or LNet parameters that are available to set, use the lctl list_param command. For example: @@ -702,7 +778,7 @@ osc.myth-OST0004-osc-ffff8800376bdc00.cur_grant_bytes=33808384 --mgsnode= or --servicenode=). To display the NIDs of all servers in networks configured to work - with the Lustre file system, run (while LNET is running): + with the Lustre file system, run (while LNet is running): lctl list_nids @@ -732,12 +808,12 @@ client# mount -t lustre 192.168.10.1@tcp0:192.168.10.2@tcp0:/testfs \ /mnt/testfs mds0# umount /mnt/mdt mds1# mount -t lustre /dev/sda1 /mnt/test/mdt -mds1# cat /proc/fs/lustre/mds/testfs-MDT0000/recovery_status +mds1# lctl get_param mdt.testfs-MDT0000.recovery_status Where multiple NIDs are specified separated by commas (for example, 10.67.73.200@tcp,192.168.10.1@tcp), the two NIDs refer to the same host, and the Lustre software chooses the - bestone for communication. When a pair of NIDs is + best one for communication. When a pair of NIDs is separated by a colon (for example, 10.67.73.200@tcp:10.67.73.201@tcp), the two NIDs refer to two different hosts and are treated as a failover pair (the Lustre @@ -846,9 +922,9 @@ tune2fs [-m reserved_blocks_percent] /dev/ Replacing an Existing OST or MDT To copy the contents of an existing OST to a new OST (or an old MDT to a new MDT), follow the process for either OST/MDT backups in - or - . For more information on - removing a MDT, see + or + . + For more information on removing a MDT, see .