<?xml version='1.0' encoding='utf-8'?>
<chapter xmlns="http://docbook.org/ns/docbook"
-xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
-xml:id="managingfilesystemio">
+ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
+ xml:id="managingfilesystemio">
<title xml:id="managingfilesystemio.title">Managing the File System and
I/O</title>
- <para>This chapter describes file striping and I/O options, and includes the
- following sections:</para>
- <itemizedlist>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_17536" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_75549" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_11204" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_80295" />
- </para>
- </listitem>
- <listitem>
- <para>
- <xref linkend="dbdoclet.50438211_61024" />
- </para>
- </listitem>
- </itemizedlist>
<section xml:id="dbdoclet.50438211_17536">
<title>
<indexterm>
</indexterm>Handling Full OSTs</title>
<para>Sometimes a Lustre file system becomes unbalanced, often due to
incorrectly-specified stripe settings, or when very large files are created
- that are not striped over all of the OSTs. If an OST is full and an attempt
- is made to write more information to the file system, an error occurs. The
- procedures below describe how to handle a full OST.</para>
+ that are not striped over all of the OSTs. Lustre will automatically avoid
+ allocating new files on OSTs that are full. If an OST is completely full and
+ more data is written to files already located on that OST, an error occurs.
+ The procedures below describe how to handle a full OST.</para>
<para>The MDS will normally handle space balancing automatically at file
- creation time, and this procedure is normally not needed, but may be
- desirable in certain circumstances (e.g. when creating very large files
+ creation time, and this procedure is normally not needed, but manual data
+ migration may be desirable in some cases (e.g. creating very large files
that would consume more than the total free space of the full OSTs).</para>
<section remap="h3">
<title>
client# lfs df -h
UUID bytes Used Available \
Use% Mounted on
-lustre-MDT0000_UUID 4.4G 214.5M 3.9G \
-4% /mnt/lustre[MDT:0]
-lustre-OST0000_UUID 2.0G 751.3M 1.1G \
-37% /mnt/lustre[OST:0]
-lustre-OST0001_UUID 2.0G 755.3M 1.1G \
-37% /mnt/lustre[OST:1]
-lustre-OST0002_UUID 2.0G 1.7G 155.1M \
-86% /mnt/lustre[OST:2] <-
-lustre-OST0003_UUID 2.0G 751.3M 1.1G \
-37% /mnt/lustre[OST:3]
-lustre-OST0004_UUID 2.0G 747.3M 1.1G \
-37% /mnt/lustre[OST:4]
-lustre-OST0005_UUID 2.0G 743.3M 1.1G \
-36% /mnt/lustre[OST:5]
+testfs-MDT0000_UUID 4.4G 214.5M 3.9G \
+4% /mnt/testfs[MDT:0]
+testfs-OST0000_UUID 2.0G 751.3M 1.1G \
+37% /mnt/testfs[OST:0]
+testfs-OST0001_UUID 2.0G 755.3M 1.1G \
+37% /mnt/testfs[OST:1]
+testfs-OST0002_UUID 2.0G 1.7G 155.1M \
+86% /mnt/testfs[OST:2] ****
+testfs-OST0003_UUID 2.0G 751.3M 1.1G \
+37% /mnt/testfs[OST:3]
+testfs-OST0004_UUID 2.0G 747.3M 1.1G \
+37% /mnt/testfs[OST:4]
+testfs-OST0005_UUID 2.0G 743.3M 1.1G \
+36% /mnt/testfs[OST:5]
filesystem summary: 11.8G 5.4G 5.8G \
-45% /mnt/lustre
+45% /mnt/testfs
</screen>
- <para>In this case, OST:2 is almost full and when an attempt is made to
+ <para>In this case, OST0002 is almost full and when an attempt is made to
write additional information to the file system (even with uniform
striping over all the OSTs), the write command fails as follows:</para>
<screen>
-client# lfs setstripe /mnt/lustre 4M 0 -1
-client# dd if=/dev/zero of=/mnt/lustre/test_3 bs=10M count=100
-dd: writing '/mnt/lustre/test_3': No space left on device
+client# lfs setstripe /mnt/testfs 4M 0 -1
+client# dd if=/dev/zero of=/mnt/testfs/test_3 bs=10M count=100
+dd: writing '/mnt/testfs/test_3': No space left on device
98+0 records in
97+0 records out
1017192448 bytes (1.0 GB) copied, 23.2411 seconds, 43.8 MB/s
<title>
<indexterm>
<primary>I/O</primary>
- <secondary>taking OST offline</secondary>
- </indexterm>Taking a Full OST Offline</title>
+ <secondary>disabling OST creates</secondary>
+ </indexterm>Disabling creates on a Full OST</title>
<para>To avoid running out of space in the file system, if the OST usage
is imbalanced and one or more OSTs are close to being full while there
- are others that have a lot of space, the full OSTs may optionally be
- deactivated at the MDS to prevent the MDS from allocating new objects
- there.</para>
+ are others that have a lot of space, the MDS will typically avoid file
+ creation on the full OST(s) automatically. The full OSTs may optionally
+ be deactivated manually on the MDS to ensure the MDS will not allocate
+ new objects there.</para>
<orderedlist>
<listitem>
- <para>Log into the MDS server:</para>
- <screen>
-client# ssh root@192.168.0.10
-root@192.168.0.10's password:
-Last login: Wed Nov 26 13:35:12 2008 from 192.168.0.6
-</screen>
- </listitem>
- <listitem>
- <para>Use the
- <literal>lctl dl</literal> command to show the status of all file
- system components:</para>
- <screen>
-mds# lctl dl
-0 UP mgs MGS MGS 9
-1 UP mgc MGC192.168.0.10@tcp e384bb0e-680b-ce25-7bc9-81655dd1e813 5
-2 UP mdt MDS MDS_uuid 3
-3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
-4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 5
-5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5
-6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5
-7 UP osc lustre-OST0002-osc lustre-mdtlov_UUID 5
-8 UP osc lustre-OST0003-osc lustre-mdtlov_UUID 5
-9 UP osc lustre-OST0004-osc lustre-mdtlov_UUID 5
-10 UP osc lustre-OST0005-osc lustre-mdtlov_UUID 5
-</screen>
- </listitem>
- <listitem>
- <para>Use
- <literal>lctl</literal> deactivate to take the full OST
- offline:</para>
- <screen>
-mds# lctl --device 7 deactivate
-</screen>
- </listitem>
- <listitem>
- <para>Display the status of the file system components:</para>
+ <para>Log into the MDS server and use the <literal>lctl</literal>
+ command to stop new object creation on the full OST(s):
+ </para>
<screen>
-mds# lctl dl
-0 UP mgs MGS MGS 9
-1 UP mgc MGC192.168.0.10@tcp e384bb0e-680b-ce25-7bc9-81655dd1e813 5
-2 UP mdt MDS MDS_uuid 3
-3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
-4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 5
-5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5
-6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5
-7 IN osc lustre-OST0002-osc lustre-mdtlov_UUID 5
-8 UP osc lustre-OST0003-osc lustre-mdtlov_UUID 5
-9 UP osc lustre-OST0004-osc lustre-mdtlov_UUID 5
-10 UP osc lustre-OST0005-osc lustre-mdtlov_UUID 5
+mds# lctl set_param osp.<replaceable>fsname</replaceable>-OST<replaceable>nnnn</replaceable>*.max_create_count=0
</screen>
</listitem>
</orderedlist>
- <para>The device list shows that OST0002 is now inactive. When new files
- are created in the file system, they will only use the remaining active
- OSTs. Either manual space rebalancing can be done by migrating data to
- other OSTs, as shown in the next section, or normal file deletion and
- creation can be allowed to passively rebalance the space usage.</para>
+ <para>When new files are created in the file system, they will only use
+ the remaining OSTs. Either manual space rebalancing can be done by
+ migrating data to other OSTs, as shown in the next section, or normal
+ file deletion and creation can passively rebalance the space usage.</para>
</section>
<section remap="h3">
<title>
<primary>maintenance</primary>
<secondary>full OSTs</secondary>
</indexterm>Migrating Data within a File System</title>
- <para>As stripes cannot be moved within the file system, data must be
- migrated manually by copying and renaming the file, removing the original
- file, and renaming the new file with the original file name. The simplest
- way to do this is to use the
- <literal>lfs_migrate</literal> command (see
- <xref linkend="dbdoclet.50438206_42260" />). However, the steps for
- migrating a file by hand are also shown here for reference.</para>
- <orderedlist>
- <listitem>
- <para>Identify the file(s) to be moved.</para>
- <para>In the example below, output from the
- <literal>getstripe</literal> command indicates that the file
- <literal>test_2</literal> is located entirely on OST2:</para>
- <screen>
-client# lfs getstripe /mnt/lustre/test_2
-/mnt/lustre/test_2
-obdidx objid objid group
- 2 8 0x8 0
-</screen>
- </listitem>
- <listitem>
- <para>To move single object(s), create a new copy and remove the
- original. Enter:</para>
- <screen>
-client# cp -a /mnt/lustre/test_2 /mnt/lustre/test_2.tmp
-client# mv /mnt/lustre/test_2.tmp /mnt/lustre/test_2
-</screen>
- </listitem>
- <listitem>
- <para>To migrate large files from one or more OSTs, enter:</para>
- <screen>
-client# lfs find --ost
-<replaceable>ost_name</replaceable> -size +1G | lfs_migrate -y
-</screen>
- </listitem>
- <listitem>
- <para>Check the file system balance.</para>
- <para>The
- <literal>df</literal> output in the example below shows a more
- balanced system compared to the
- <literal>df</literal> output in the example in
- <xref linkend="dbdoclet.50438211_17536" />.</para>
- <screen>
-client# lfs df -h
-UUID bytes Used Available Use% \
- Mounted on
-lustre-MDT0000_UUID 4.4G 214.5M 3.9G 4% \
- /mnt/lustre[MDT:0]
-lustre-OST0000_UUID 2.0G 1.3G 598.1M 65% \
- /mnt/lustre[OST:0]
-lustre-OST0001_UUID 2.0G 1.3G 594.1M 65% \
- /mnt/lustre[OST:1]
-lustre-OST0002_UUID 2.0G 913.4M 1000.0M 45% \
- /mnt/lustre[OST:2]
-lustre-OST0003_UUID 2.0G 1.3G 602.1M 65% \
- /mnt/lustre[OST:3]
-lustre-OST0004_UUID 2.0G 1.3G 606.1M 64% \
- /mnt/lustre[OST:4]
-lustre-OST0005_UUID 2.0G 1.3G 610.1M 64% \
- /mnt/lustre[OST:5]
-
-filesystem summary: 11.8G 7.3G 3.9G 61% \
-/mnt/lustre
-</screen>
- </listitem>
- </orderedlist>
+
+ <para>If there is a need to move the file data from the current
+ OST(s) to new OST(s), the data must be migrated (copied)
+ to the new location. The simplest way to do this is to use the
+ <literal>lfs_migrate</literal> command, as described in
+ <xref linkend="lustremaint.adding_new_ost" />.</para>
</section>
<section remap="h3">
<title>
<primary>maintenance</primary>
<secondary>bringing OST online</secondary>
</indexterm>Returning an Inactive OST Back Online</title>
- <para>Once the deactivated OST(s) no longer are severely imbalanced, due
+ <para>Once the full OST(s) no longer are severely imbalanced, due
to either active or passive data redistribution, they should be
reactivated so they will again have new files allocated on them.</para>
<screen>
-[mds]# lctl --device 7 activate
-[mds]# lctl dl
- 0 UP mgs MGS MGS 9
- 1 UP mgc MGC192.168.0.10@tcp e384bb0e-680b-ce25-7bc9-816dd1e813 5
- 2 UP mdt MDS MDS_uuid 3
- 3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
- 4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 5
- 5 UP osc lustre-OST0000-osc lustre-mdtlov_UUID 5
- 6 UP osc lustre-OST0001-osc lustre-mdtlov_UUID 5
- 7 UP osc lustre-OST0002-osc lustre-mdtlov_UUID 5
- 8 UP osc lustre-OST0003-osc lustre-mdtlov_UUID 5
- 9 UP osc lustre-OST0004-osc lustre-mdtlov_UUID 5
- 10 UP osc lustre-OST0005-osc lustre-mdtlov_UUID
+[mds]# lctl set_param osp.testfs-OST0002.max_create_count=20000
</screen>
</section>
+ <section remap="h3">
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Migrating Metadata within a Filesystem</title>
+ <section remap="h3" condition='l28'>
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Whole Directory Migration</title>
+ <para>Lustre software version 2.8 includes a feature
+ to migrate metadata (directories and inodes therein) between MDTs.
+ This migration can only be performed on whole directories. Striped
+ directories are not supported until Lustre 2.12. For example, to
+ migrate the contents of the <literal>/testfs/remotedir</literal>
+ directory from the MDT where it currently is located to MDT0000 to
+ allow that MDT to be removed, the sequence of commands is as follows:
+ </para>
+ <screen>$ cd /testfs
+$ lfs getdirstripe -m ./remotedir <lineannotation>which MDT is dir on?</lineannotation>
+1
+$ touch ./remotedir/file.{1,2,3}.txt<lineannotation>create test files</lineannotation>
+$ lfs getstripe -m ./remotedir/file.*.txt<lineannotation>check files are on MDT0001</lineannotation>
+1
+1
+1
+$ lfs migrate -m 0 ./remotedir <lineannotation>migrate testremote to MDT0000</lineannotation>
+$ lfs getdirstripe -m ./remotedir <lineannotation>which MDT is dir on now?</lineannotation>
+0
+$ lfs getstripe -m ./remotedir/file.*.txt<lineannotation>check files are on MDT0000</lineannotation>
+0
+0
+0</screen>
+ <para>For more information, see <literal>man lfs-migrate</literal>.
+ </para>
+ <warning><para>During migration each file receives a new identifier
+ (FID). As a consequence, the file will report a new inode number to
+ userspace applications. Some system tools (for example, backup and
+ archiving tools, NFS, Samba) that identify files by inode number may
+ consider the migrated files to be new, even though the contents are
+ unchanged. If a Lustre system is re-exporting to NFS, the migrated
+ files may become inaccessible during and after migration if the
+ client or server are caching a stale file handle with the old FID.
+ Restarting the NFS service will flush the local file handle cache,
+ but clients may also need to be restarted as they may cache stale
+ file handles as well.
+ </para></warning>
+ </section>
+ <section remap="h3" condition='l2C'>
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Striped Directory Migration</title>
+ <para>Lustre 2.8 included a feature to migrate metadata (directories
+ and inodes therein) between MDTs, however it did not support migration
+ of striped directories, or changing the stripe count of an existing
+ directory. Lustre 2.12 adds support for migrating and restriping
+ directories. The <literal>lfs migrate -m</literal> command can only
+ only be performed on whole directories, though it will migrate both
+ the specified directory and its sub-entries recursively.
+ For example, to migrate the contents of a large directory
+ <literal>/testfs/largedir</literal> from its current location on
+ MDT0000 to MDT0001 and MDT0003, run the following command:</para>
+ <screen>$ lfs migrate -m 1,3 /testfs/largedir</screen>
+ <para>Metadata migration will migrate file dirent and inode to other
+ MDTs, but it won't touch file data. During migration, directory and
+ its sub-files can be accessed like normal ones, though the same
+ warning above applies to tools that depend on the file inode number.
+ Migration may fail for various reasons such as MDS restart, or disk
+ full. In those cases, some of the sub-files may have been migrated to
+ the new MDTs, while others are still on the original MDT. The files
+ can be accessed normally. The same <literal>lfs migrate -m</literal>
+ command should be executed again when these issues are fixed to finish
+ this migration. However, you cannot abort a failed migration, or
+ migrate to different MDTs from previous migration command.</para>
+ </section>
+ <section remap="h3" condition='l2C'>
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Directory Restriping</title>
+ <para>Lustre 2.14 includs a feature to change the stripe count of an
+ existing directory. The <literal>lfs setdirstripe -c</literal> command
+ can be performed on an existing directory to change its stripe count.
+ For example, a directory <literal>/testfs/testdir</literal> is becoming
+ large, run the following command to increase its stripe count to
+ <literal>2</literal>:</para>
+ <screen>$ lfs setdirstripe -c 2 /testfs/testdir</screen>
+ <para>By default directory restriping will migrate sub-file dirents only,
+ but it won't move inodes. To enable moving both dirents and inodes, run
+ the following command on all MDS's:</para>
+ <screen>mds$ lctl set_param mdt.*.dir_restripe_nsonly=0</screen>
+ <para>It's not allowed to specify MDTs in directory restriping, instead
+ server will pick MDTs for the added stripes by space and inode usages.
+ During restriping, directory and its sub-files can be accessed like
+ normal ones, which is the same as directory migration. Similarly you
+ cannot abort a failed restriping, and server will resume the failed
+ restriping automatically when it notices an unfinished restriping.</para>
+ </section>
+ <section remap="h3" condition='l2C'>
+ <title><indexterm>
+ <primary>migrating metadata</primary>
+ </indexterm>Directory Auto-Split</title>
+ <para>Lustre 2.14 includs a feature to automatically increase the stripe
+ count of a directory when it becomes large. This can be enabled by the
+ following command:</para>
+ <screen>mds$ lctl set_param mdt.*.enable_dir_auto_split=1</screen>
+ <para>The sub file count that triggers directory auto-split is 50k, and
+ it can be changed by the following command:</para>
+ <screen>mds$ lctl set_param mdt.*.dir_split_count=value</screen>
+ <para>The directory stripe count will be increased from 0 to 4 if it's a
+ plain directory, and from 4 to 8 upon the second split, and so on.
+ However the final stripe count won't exceed total MDT count, and it will
+ stop splitting when it's distributed among all MDTs. This delta value
+ can be changed by the following command:</para>
+ <screen>mds$ lctl set_param mdt.*.dir_split_delta=value</screen>
+ </section>
+ </section>
</section>
- <section xml:id="dbdoclet.50438211_75549">
+ <section xml:id="managingfilesystemio.managing_ost_pools">
<title>
<indexterm>
<primary>I/O</primary>
</caution>
<para>To create a new pool, run:</para>
<screen>
-mgs# lctl pool_new
-<replaceable>fsname</replaceable>.
-<replaceable>poolname</replaceable>
+mgs# lctl pool_new <replaceable>fsname</replaceable>.<replaceable>poolname</replaceable>
</screen>
<note>
- <para>The pool name is an ASCII string up to 16 characters.</para>
+ <para>The pool name is an ASCII string up to 15 characters.</para>
</note>
<para>To add the named OST to a pool, run:</para>
<screen>
-mgs# lctl pool_add
-<replaceable>fsname</replaceable>.
-<replaceable>poolname</replaceable>
-<replaceable>ost_list</replaceable>
+mgs# lctl pool_add <replaceable>fsname</replaceable>.<replaceable>poolname</replaceable> <replaceable>ost_list</replaceable>
</screen>
<para>Where:</para>
<itemizedlist>
<literal>_UUID</literal> are missing, they are automatically added.</para>
<para>For example, to add even-numbered OSTs to
<literal>pool1</literal> on file system
- <literal>lustre</literal>, run a single command (
+ <literal>testfs</literal>, run a single command (
<literal>pool_add</literal>) to add many OSTs to the pool at one
time:</para>
<para>
<screen>
-lctl pool_add lustre.pool1 OST[0-10/2]
+lctl pool_add testfs.pool1 OST[0-10/2]
</screen>
</para>
<note>
<para>To set striping patterns, run:</para>
<screen>
client# lfs setstripe [--size|-s stripe_size] [--offset|-o start_ost]
- [--count|-c stripe_count] [--pool|-p pool_name]
+ [--stripe-count|-c stripe_count] [--overstripe-count|-C stripe_count]
+ [--pool|-p pool_name]
<replaceable>dir|filename</replaceable>
</screen>
<listitem>
<para>Add a new OST by passing on the following commands, run:</para>
<screen>
-oss# mkfs.lustre --fsname=spfs --mgsnode=mds16@tcp0 --ost --index=12 /dev/sda
-oss# mkdir -p /mnt/test/ost12
-oss# mount -t lustre /dev/sda /mnt/test/ost12
+oss# mkfs.lustre --fsname=testfs --mgsnode=mds16@tcp0 --ost --index=12 /dev/sda
+oss# mkdir -p /mnt/testfs/ost12
+oss# mount -t lustre /dev/sda /mnt/testfs/ost12
</screen>
</listitem>
<listitem>
</listitem>
</orderedlist>
<para>If a Lustre file system administrator wants to explore this approach
- further, per-OST disk-usage statistics can be found under
- <literal>/proc/fs/lustre/osc/*/rpc_stats</literal></para>
+ further, per-OST disk-usage statistics can be found in the
+ <literal>osc.*.rpc_stats</literal> parameter file.</para>
</section>
<section xml:id="dbdoclet.50438211_80295">
<title>
checksum algorithm is now in use.</para>
<screen>
$ lctl get_param osc.*.checksum_type
-osc.lustre-OST0000-osc-ffff81012b2c48e0.checksum_type=crc32 [adler]
+osc.testfs-OST0000-osc-ffff81012b2c48e0.checksum_type=crc32 [adler]
$ lctl set_param osc.*.checksum_type=crc32
-osc.lustre-OST0000-osc-ffff81012b2c48e0.checksum_type=crc32
+osc.testfs-OST0000-osc-ffff81012b2c48e0.checksum_type=crc32
$ lctl get_param osc.*.checksum_type
-osc.lustre-OST0000-osc-ffff81012b2c48e0.checksum_type=[crc32] adler
+osc.testfs-OST0000-osc-ffff81012b2c48e0.checksum_type=[crc32] adler
</screen>
</section>
</section>
<section remap="h3">
- <title>Ptlrpc Thread Pool</title>
- <para>Releases prior to Lustre software release 2.2 used two portal RPC
- daemons for each client/server pair. One daemon handled all synchronous
- IO requests, and the second daemon handled all asynchronous (non-IO)
- RPCs. The increasing use of large SMP nodes for Lustre servers exposed
- some scaling issues. The lack of threads for large SMP nodes resulted in
+ <title>PtlRPC Client Thread Pool</title>
+ <para>The use of large SMP nodes for Lustre clients
+ requires significant parallelism within the kernel to avoid
cases where a single CPU would be 100% utilized and other CPUs would be
- relativity idle. This is especially noticeable when a single client
+ relativity idle. This is especially noticeable when a single thread
traverses a large directory.</para>
- <para>Lustre software release 2.2.x implements a ptlrpc thread pool, so
- that multiple threads can be created to serve asynchronous RPC requests.
- The number of threads spawned is controlled at module load time using
- module options. By default one thread is spawned per CPU, with a minimum
- of 2 threads spawned irrespective of module options.</para>
+ <para>The Lustre client implements a PtlRPC daemon thread pool, so that
+ multiple threads can be created to serve asynchronous RPC requests, even
+ if only a single userspace thread is running. The number of ptlrpcd
+ threads spawned is controlled at module load time using module options.
+ By default two service threads are spawned per CPU socket.</para>
<para>One of the issues with thread operations is the cost of moving a
thread context from one CPU to another with the resulting loss of CPU
- cache warmth. To reduce this cost, ptlrpc threads can be bound to a CPU.
+ cache warmth. To reduce this cost, PtlRPC threads can be bound to a CPU.
However, if the CPUs are busy, a bound thread may not be able to respond
quickly, as the bound CPU may be busy with other tasks and the thread
must wait to schedule.</para>
- <para>Because of these considerations, the pool of ptlrpc threads can be
+ <para>Because of these considerations, the pool of ptlrpcd threads can be
a mixture of bound and unbound threads. The system operator can balance
the thread mixture based on system size and workload.</para>
<section>
<literal>etc/modprobe.d</literal> directory, as options for the ptlrpc
module.
<screen>
-options ptlrpcd max_ptlrpcds=XXX
+options ptlrpcd ptlrpcd_per_cpt_max=XXX
</screen></para>
- <para>Sets the number of ptlrpcd threads created at module load time.
- The default if not specified is one thread per CPU, including
- hyper-threaded CPUs. The lower bound is 2 (old prlrpcd behaviour)
+ <para>Sets the number of ptlrpcd threads created per socket.
+ The default if not specified is two threads per CPU socket, including
+ hyper-threaded CPUs. The lower bound is 2 threads per socket.
<screen>
options ptlrpcd ptlrpcd_bind_policy=[1-4]
</screen></para>
</section>
</section>
</chapter>
+<!--
+ vim:expandtab:shiftwidth=2:tabstop=8:
+ -->
git://git.whamcloud.com / doc / manual.git / blobdiff