<?xml version='1.0' encoding='utf-8'?>
<chapter xmlns="http://docbook.org/ns/docbook"
-xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
-xml:id="managingfilesystemio">
+ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
+ xml:id="managingfilesystemio">
<title xml:id="managingfilesystemio.title">Managing the File System and
I/O</title>
<section xml:id="dbdoclet.50438211_17536">
OST(s) to new OST(s), the data must be migrated (copied)
to the new location. The simplest way to do this is to use the
<literal>lfs_migrate</literal> command, as described in
- <xref linkend="dbdoclet.adding_new_ost" />.</para>
+ <xref linkend="lustremaint.adding_new_ost" />.</para>
</section>
<section remap="h3">
<title>
this migration. However, you cannot abort a failed migration, or
migrate to different MDTs from previous migration command.</para>
</section>
+ <section remap="h3" condition='l2C'>
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Directory Restriping</title>
+ <para>Lustre 2.14 includs a feature to change the stripe count of an
+ existing directory. The <literal>lfs setdirstripe -c</literal> command
+ can be performed on an existing directory to change its stripe count.
+ For example, a directory <literal>/testfs/testdir</literal> is becoming
+ large, run the following command to increase its stripe count to
+ <literal>2</literal>:</para>
+ <screen>$ lfs setdirstripe -c 2 /testfs/testdir</screen>
+ <para>By default directory restriping will migrate sub-file dirents only,
+ but it won't move inodes. To enable moving both dirents and inodes, run
+ the following command on all MDS's:</para>
+ <screen>mds$ lctl set_param mdt.*.dir_restripe_nsonly=0</screen>
+ <para>It's not allowed to specify MDTs in directory restriping, instead
+ server will pick MDTs for the added stripes by space and inode usages.
+ During restriping, directory and its sub-files can be accessed like
+ normal ones, which is the same as directory migration. Similarly you
+ cannot abort a failed restriping, and server will resume the failed
+ restriping automatically when it notices an unfinished restriping.</para>
+ </section>
+ <section remap="h3" condition='l2C'>
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Directory Auto-Split</title>
+ <para>Lustre 2.14 includs a feature to automatically increase the stripe
+ count of a directory when it becomes large. This can be enabled by the
+ following command:</para>
+ <screen>mds$ lctl set_param mdt.*.enable_dir_auto_split=1</screen>
+ <para>The sub file count that triggers directory auto-split is 50k, and
+ it can be changed by the following command:</para>
+ <screen>mds$ lctl set_param mdt.*.dir_split_count=value</screen>
+ <para>The directory stripe count will be increased from 0 to 4 if it's a
+ plain directory, and from 4 to 8 upon the second split, and so on.
+ However the final stripe count won't exceed total MDT count, and it will
+ stop splitting when it's distributed among all MDTs. This delta value
+ can be changed by the following command:</para>
+ <screen>mds$ lctl set_param mdt.*.dir_split_delta=value</screen>
+ </section>
</section>
</section>
- <section xml:id="dbdoclet.50438211_75549">
+ <section xml:id="managingfilesystemio.managing_ost_pools">
<title>
<indexterm>
<primary>I/O</primary>
</caution>
<para>To create a new pool, run:</para>
<screen>
-mgs# lctl pool_new
-<replaceable>fsname</replaceable>.
-<replaceable>poolname</replaceable>
+mgs# lctl pool_new <replaceable>fsname</replaceable>.<replaceable>poolname</replaceable>
</screen>
<note>
<para>The pool name is an ASCII string up to 15 characters.</para>
</note>
<para>To add the named OST to a pool, run:</para>
<screen>
-mgs# lctl pool_add
-<replaceable>fsname</replaceable>.
-<replaceable>poolname</replaceable>
-<replaceable>ost_list</replaceable>
+mgs# lctl pool_add <replaceable>fsname</replaceable>.<replaceable>poolname</replaceable> <replaceable>ost_list</replaceable>
</screen>
<para>Where:</para>
<itemizedlist>
<para>To set striping patterns, run:</para>
<screen>
client# lfs setstripe [--size|-s stripe_size] [--offset|-o start_ost]
- [--count|-c stripe_count] [--pool|-p pool_name]
+ [--stripe-count|-c stripe_count] [--overstripe-count|-C stripe_count]
+ [--pool|-p pool_name]
<replaceable>dir|filename</replaceable>
</screen>
</listitem>
</orderedlist>
<para>If a Lustre file system administrator wants to explore this approach
- further, per-OST disk-usage statistics can be found under
- <literal>/proc/fs/lustre/osc/*/rpc_stats</literal></para>
+ further, per-OST disk-usage statistics can be found in the
+ <literal>osc.*.rpc_stats</literal> parameter file.</para>
</section>
<section xml:id="dbdoclet.50438211_80295">
<title>
</section>
</section>
<section remap="h3">
- <title>Ptlrpc Thread Pool</title>
- <para>Releases prior to Lustre software release 2.2 used two portal RPC
- daemons for each client/server pair. One daemon handled all synchronous
- IO requests, and the second daemon handled all asynchronous (non-IO)
- RPCs. The increasing use of large SMP nodes for Lustre servers exposed
- some scaling issues. The lack of threads for large SMP nodes resulted in
+ <title>PtlRPC Client Thread Pool</title>
+ <para>The use of large SMP nodes for Lustre clients
+ requires significant parallelism within the kernel to avoid
cases where a single CPU would be 100% utilized and other CPUs would be
- relativity idle. This is especially noticeable when a single client
+ relativity idle. This is especially noticeable when a single thread
traverses a large directory.</para>
- <para>Lustre software release 2.2.x implements a ptlrpc thread pool, so
- that multiple threads can be created to serve asynchronous RPC requests.
- The number of threads spawned is controlled at module load time using
- module options. By default one thread is spawned per CPU, with a minimum
- of 2 threads spawned irrespective of module options.</para>
+ <para>The Lustre client implements a PtlRPC daemon thread pool, so that
+ multiple threads can be created to serve asynchronous RPC requests, even
+ if only a single userspace thread is running. The number of ptlrpcd
+ threads spawned is controlled at module load time using module options.
+ By default two service threads are spawned per CPU socket.</para>
<para>One of the issues with thread operations is the cost of moving a
thread context from one CPU to another with the resulting loss of CPU
- cache warmth. To reduce this cost, ptlrpc threads can be bound to a CPU.
+ cache warmth. To reduce this cost, PtlRPC threads can be bound to a CPU.
However, if the CPUs are busy, a bound thread may not be able to respond
quickly, as the bound CPU may be busy with other tasks and the thread
must wait to schedule.</para>
- <para>Because of these considerations, the pool of ptlrpc threads can be
+ <para>Because of these considerations, the pool of ptlrpcd threads can be
a mixture of bound and unbound threads. The system operator can balance
the thread mixture based on system size and workload.</para>
<section>
<literal>etc/modprobe.d</literal> directory, as options for the ptlrpc
module.
<screen>
-options ptlrpcd max_ptlrpcds=XXX
+options ptlrpcd ptlrpcd_per_cpt_max=XXX
</screen></para>
- <para>Sets the number of ptlrpcd threads created at module load time.
- The default if not specified is one thread per CPU, including
- hyper-threaded CPUs. The lower bound is 2 (old prlrpcd behaviour)
+ <para>Sets the number of ptlrpcd threads created per socket.
+ The default if not specified is two threads per CPU socket, including
+ hyper-threaded CPUs. The lower bound is 2 threads per socket.
<screen>
options ptlrpcd ptlrpcd_bind_policy=[1-4]
</screen></para>
</section>
</section>
</chapter>
+<!--
+ vim:expandtab:shiftwidth=2:tabstop=8:
+ -->
git://git.whamcloud.com / doc / manual.git / blobdiff