parameters. These parameters are contained in the
<literal>/etc/modprobe.d/lustre.conf</literal> file.</para>
</note>
- <section xml:id="dbdoclet.50438272_55226">
+ <section xml:id="tuning_optimizing_service_threads">
<title>
<indexterm>
<primary>tuning</primary>
<para>
This works in a similar fashion to
binding of threads on MDS. MDS thread tuning is covered in
- <xref linkend="dbdoclet.mdsbinding" />.</para>
+ <xref linkend="mdsbinding" />.</para>
<itemizedlist>
<listitem>
<para>
</listitem>
</itemizedlist>
<para>For further details, see
- <xref linkend="dbdoclet.50438271_87260" />.</para>
+ <xref linkend="tuning_setting_thread_count" />.</para>
</section>
- <section xml:id="dbdoclet.mdstuning">
+ <section xml:id="mdstuning">
<title>
<indexterm>
<primary>tuning</primary>
</screen>
</para>
<para>For details, see
- <xref linkend="dbdoclet.50438271_87260" />.</para>
+ <xref linkend="tuning_setting_thread_count" />.</para>
<para>The number of MDS service threads started depends on system size
and the load on the server, and has a default maximum of 64. The
maximum potential number of threads (<literal>MDS_MAX_THREADS</literal>)
in providing the read page service. The read page service handles
file close and readdir operations.</para>
</listitem>
- <listitem>
- <para>
- <literal>mds_attr_num_threads</literal> controls the number of threads
- in providing the setattr service to clients running Lustre software
- release 1.8.</para>
- </listitem>
</itemizedlist>
</section>
</section>
- <section xml:id="dbdoclet.mdsbinding">
+ <section xml:id="mdsbinding">
<title>
<indexterm>
<primary>tuning</primary>
bindings are selected automatically to provide good overall performance for
a given CPU count. However, an administrator can deviate from these setting
if they choose. For details on specifying the mapping of CPU cores to
- CPTs see <xref linkend="dbdoclet.libcfstuning"/>.
+ CPTs see <xref linkend="libcfstuning"/>.
</para>
<itemizedlist>
<listitem>
to
<literal>CPT4</literal>.</para>
</listitem>
- <listitem>
- <para>
- <literal>mds_attr_num_cpts=[EXPRESSION]</literal> binds the setattr
- service threads to CPTs defined by
- <literal>EXPRESSION</literal>.</para>
- </listitem>
</itemizedlist>
- <para>Parameters must be set before module load in the file
+ <para>Parameters must be set before module load in the file
<literal>/etc/modprobe.d/lustre.conf</literal>. For example:
<example><title>lustre.conf</title>
<screen>options lnet networks=tcp0(eth0)
</example>
</para>
</section>
- <section xml:id="dbdoclet.50438272_73839">
+ <section xml:id="tuning_lnet_params">
<title>
<indexterm>
<primary>LNet</primary>
an administrator can bind an interface to one or more CPU partitions.
Bindings are specified as options to the LNet modules. For more
information on specifying module options, see
- <xref linkend="dbdoclet.50438293_15350" /></para>
+ <xref linkend="tuning_lnet_mod_params" /></para>
<para>For example,
<literal>o2ib0(ib0)[0,1]</literal> will ensure that all messages for
<literal>o2ib0</literal> will be handled by LND threads executing on
events across all CPTs. This may balance load better across the CPU but
can incur a cross CPU overhead.</para>
<para>The current policy can be changed by an administrator with
- <literal>echo
- <replaceable>value</replaceable>>
- /proc/sys/lnet/portal_rotor</literal>. There are four options for
+ <literal>lctl set_param portal_rotor=value</literal>.
+ There are four options for
<literal>
<replaceable>value</replaceable>
</literal>:</para>
be MAX.</para>
</section>
</section>
- <section xml:id="dbdoclet.libcfstuning">
+ <section xml:id="libcfstuning">
<title>
<indexterm>
<primary>tuning</primary>
</para>
</section>
</section>
- <section xml:id="dbdoclet.lndtuning">
+ <section xml:id="lndtuning">
<title>
<indexterm>
<primary>tuning</primary>
<literal>ksocklnd</literal> using the
<literal>nscheds</literal> parameter. This adjusts the number of threads for
each partition, not the overall number of threads on the LND.</para>
+ <note>
+ <para>The default number of threads for
+ <literal>ko2iblnd</literal> and
+ <literal>ksocklnd</literal> are automatically set and are chosen to
+ work well across a number of typical scenarios, for systems with both
+ high and low core counts.</para>
+ </note>
<section>
<title>ko2iblnd Tuning</title>
<para>The following table outlines the ko2iblnd module parameters to be used
</informaltable>
</section>
</section>
- <section xml:id="dbdoclet.nrstuning">
+ <section xml:id="nrstuning">
<title>
<indexterm>
<primary>tuning</primary>
</listitem>
</itemizedlist>
</section>
- <section xml:id="dbdoclet.tbftuning" condition='l26'>
+ <section xml:id="tbftuning" condition='l26'>
<title>
<indexterm>
<primary>tuning</primary>
<screen>lctl set_param x.x.x.nrs_tbf_rule=
"[reg|hp] start <replaceable>rule_name</replaceable> <replaceable>arguments</replaceable>..."
</screen>
- <para>'<replaceable>rule_name</replaceable>' is a string of the TBF
- policy rule's name and '<replaceable>arguments</replaceable>' is a
- string to specify the detailed rule according to the different types.
- </para>
+ <para>'<replaceable>rule_name</replaceable>' is a string up to 15
+ characters which identifies the TBF policy rule's name. Alphanumeric
+ characters and underscores are accepted (e.g: "test_rule_A1").
+ </para>
+ <para>'<replaceable>arguments</replaceable>' is a string to specify the
+ detailed rule according to the different types.
+ </para>
<itemizedlist>
<para>Next, the different types of TBF policies will be described.</para>
<listitem>
<para><emphasis role="bold">JobID based TBF policy</emphasis></para>
<para>For the JobID, please see
<xref xmlns:xlink="http://www.w3.org/1999/xlink"
- linkend="dbdoclet.jobstats" /> for more details.</para>
+ linkend="jobstats" /> for more details.</para>
<para>Command:</para>
<screen>lctl set_param x.x.x.nrs_tbf_rule=
"[reg|hp] start <replaceable>rule_name</replaceable> jobid={<replaceable>jobid_list</replaceable>} rate=<replaceable>rate</replaceable>"
</itemizedlist>
</section>
</section>
- <section xml:id="dbdoclet.delaytuning" condition='l2A'>
+ <section xml:id="delaytuning" condition='l2A'>
<title>
<indexterm>
<primary>tuning</primary>
</itemizedlist>
</section>
</section>
- <section xml:id="dbdoclet.50438272_25884">
+ <section xml:id="tuning_lockless_IO">
<title>
<indexterm>
<primary>tuning</primary>
<para>
<emphasis role="bold">Client-side:</emphasis>
</para>
- <screen>
-/proc/fs/lustre/llite/lustre-*
-</screen>
+ <screen>llite.<replaceable>fsname</replaceable>-*</screen>
<para>
<literal>contention_seconds</literal>-
<literal>llite</literal> inode remembers its contended state for the
<emphasis role="bold">Client-side statistics:</emphasis>
</para>
<para>The
- <literal>/proc/fs/lustre/llite/lustre-*/stats</literal> file has new
- rows for lockless I/O statistics.</para>
+ <literal>llite.<replaceable>fsname</replaceable>-*.stats</literal>
+ parameter has several entries for lockless I/O statistics.</para>
<para>
<literal>lockless_read_bytes</literal> and
<literal>lockless_write_bytes</literal>- To count the total bytes read
renegotiate the new maximum RPC size.</para></caution>
</section>
</section>
- <section xml:id="dbdoclet.50438272_80545">
+ <section xml:id="tuning_IO_small_files">
<title>
<indexterm>
<primary>tuning</primary>
</listitem>
</itemizedlist>
</section>
- <section xml:id="dbdoclet.50438272_45406">
+ <section xml:id="write_vs_read_performance">
<title>
<indexterm>
<primary>tuning</primary>