<?xml version='1.0' encoding='UTF-8'?><chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="benchmarkingtests">
- <title xml:id="benchmarkingtests.title">Benchmarking Lustre Performance (Lustre I/O Kit)</title>
- <para>This chapter describes the Lustre I/O kit, a collection of I/O benchmarking tools for a Lustre cluster, and PIOS, a parallel I/O simulator for Linux and Solaris. It includes:</para>
+ <title xml:id="benchmarkingtests.title">Benchmarking Lustre File System Performance (Lustre I/O
+ Kit)</title>
+ <para>This chapter describes the Lustre I/O kit, a collection of I/O benchmarking tools for a
+ Lustre cluster, and PIOS, a parallel I/O simulator for Linux and
+ Solaris<superscript>*</superscript> operating systems. It includes:</para>
<itemizedlist>
<listitem>
<para><xref linkend="dbdoclet.50438212_44437"/></para>
<indexterm><primary>performance</primary><see>benchmarking</see></indexterm>
Using Lustre I/O Kit Tools</title>
- <para>The tools in the Lustre I/O Kit are used to benchmark Lustre hardware and validate that it is working as expected before you install the Lustre software. It can also be used to to validate the performance of the various hardware and software layers in the cluster and also to find and troubleshoot I/O issues.</para>
+ <para>The tools in the Lustre I/O Kit are used to benchmark Lustre file system hardware and
+ validate that it is working as expected before you install the Lustre software. It can also be
+ used to to validate the performance of the various hardware and software layers in the cluster
+ and also to find and troubleshoot I/O issues.</para>
<para>Typically, performance is measured starting with single raw devices and then proceeding to groups of devices. Once raw performance has been established, other software layers are then added incrementally and tested.</para>
<section remap="h3">
<title>Contents of the Lustre I/O Kit</title>
- <para>The I/O kit contains three tests, each of which tests a progressively higher layer in the Lustre stack:</para>
+ <para>The I/O kit contains three tests, each of which tests a progressively higher layer in
+ the Lustre software stack:</para>
<itemizedlist>
<listitem>
<para><literal>sgpdd-survey</literal> - Measure basic 'bare metal' performance
issues.</para>
</listitem>
</itemizedlist>
- <para>Typically with these tests, Lustre should deliver 85-90% of the raw device performance.</para>
+ <para>Typically with these tests, a Lustre file system should deliver 85-90% of the raw device
+ performance.</para>
<para>A utility <literal>stats-collect</literal> is also provided to collect application profiling information from Lustre clients and servers. See <xref linkend="dbdoclet.50438212_58201"/> for more information.</para>
</section>
<section remap="h3">
<para>Password-free remote access to nodes in the system (provided by <literal>ssh</literal> or <literal>rsh</literal>).</para>
</listitem>
<listitem>
- <para>LNET self-test completed to test that Lustre Networking has been properly installed and configured. See <xref linkend="lnetselftest"/>.</para>
+ <para>LNET self-test completed to test that Lustre networking has been properly installed
+ and configured. See <xref linkend="lnetselftest"/>.</para>
</listitem>
<listitem>
<para>Lustre file system software installed.</para>
</listitem>
</itemizedlist>
<para>Download the Lustre I/O kit (<literal>lustre-iokit</literal>)from:</para>
- <para><link xl:href="http://downloads.whamcloud.com/">http://downloads.whamcloud.com/</link></para>
+ <para><link xl:href="http://downloads.hpdd.intel.com/">http://downloads.hpdd.intel.com/</link></para>
</section>
</section>
<section xml:id="dbdoclet.50438212_51053">
<note>
<para>If you need to create raw devices to use the <literal>sgpdd-survey</literal> tool, note
that raw device 0 cannot be used due to a bug in certain versions of the "raw"
- utility (including that shipped with RHEL4U4.)</para>
+ utility (including the version shipped with Red Hat Enterprise Linux 4U4.)</para>
</note>
<section remap="h3">
<title><indexterm><primary>benchmarking</primary><secondary>tuning storage</secondary></indexterm>Tuning Linux Storage Devices</title>
<para>The <literal>plot-obdfilter</literal> script generates from the output of the
<literal>obdfilter-survey</literal> a CSV file and parameters for importing into a
spreadsheet or gnuplot to visualize the data.</para>
- <para>To run the <literal>obdfilter-survey</literal> script, create a standard Lustre
- configuration; no special setup is needed.</para>
+ <para>To run the <literal>obdfilter-survey</literal> script, create a standard Lustre file
+ system configuration; no special setup is needed.</para>
<para><emphasis role="bold">To perform an automatic run:</emphasis></para>
<orderedlist>
<listitem>
<title><indexterm><primary>benchmarking</primary><secondary>network</secondary></indexterm>Testing Network Performance</title>
<para>The <literal>obdfilter-survey</literal> script can only be run automatically against a
network; no manual test is provided.</para>
- <para>To run the network test, a specific Lustre setup is needed. Make sure that these configuration requirements have been met.</para>
+ <para>To run the network test, a specific Lustre file system setup is needed. Make sure that
+ these configuration requirements have been met.</para>
<para><emphasis role="bold">To perform an automatic run:</emphasis></para>
<orderedlist>
<listitem>
write speeds. The <literal>ost-survey</literal> tool is used to detect anomalies between
otherwise identical disk subsystems.</para>
<note>
- <para>We have frequently discovered wide performance variations across all LUNs in a cluster. This may be caused by faulty disks, RAID parity reconstruction during the test, or faulty network hardware.</para>
+ <para>We have frequently discovered wide performance variations across all LUNs in a cluster.
+ This may be caused by faulty disks, RAID parity reconstruction during the test, or faulty
+ network hardware.</para>
</note>
<para>To run the <literal>ost-survey</literal> script, supply a file size (in KB) and the Lustre
- mount point. For example, run:</para>
+ file system mount point. For example, run:</para>
<screen>$ ./ost-survey.sh 10 /mnt/lustre
</screen>
<para>Typical output is:</para>
<section xml:id="mds_survey_ref">
<title><indexterm><primary>benchmarking</primary><secondary>MDS
performance</secondary></indexterm>Testing MDS Performance (<literal>mds-survey</literal>)</title>
- <para><literal>mds-survey</literal> is available in Lustre 2.2 and beyond. The <literal>mds-survey</literal> script tests the local metadata performance using the echo_client to drive different layers of the MDS stack: mdd, mdt, osd (current lustre version only supports mdd stack). It can be used with the following classes of operations:</para>
+ <para><literal>mds-survey</literal> is available in Lustre software release 2.2 and beyond. The
+ <literal>mds-survey</literal> script tests the local metadata performance using the
+ echo_client to drive different layers of the MDS stack: mdd, mdt, osd (the Lustre software
+ only supports mdd stack). It can be used with the following classes of operations:</para>
<itemizedlist>
<listitem>
<para>The script must be customized according to the components under test and where it should keep its working files. Customization variables are described as followed:</para>
<itemizedlist>
<listitem>
- <para>thrlo - threads to start testing. skipped if less than dir_count</para>
+ <para><literal>thrlo</literal> - threads to start testing. skipped if less than
+ <literal>dir_count</literal></para>
</listitem>
<listitem>
- <para>thrhi - maximum number of threads to test</para>
+ <para><literal>thrhi</literal> - maximum number of threads to test</para>
</listitem>
<listitem>
- <para>targets - MDT instance</para>
+ <para><literal>targets</literal> - MDT instance</para>
</listitem>
<listitem>
- <para>file_count - number of files per thread to test</para>
+ <para><literal>file_count</literal> - number of files per thread to test</para>
</listitem>
<listitem>
- <para>dir_count - total number of directories to test. Must be less than or equal to <literal>thrhi</literal></para>
+ <para><literal>dir_count</literal> - total number of directories to test. Must be less
+ than or equal to <literal>thrhi</literal></para>
</listitem>
<listitem>
- <para>stripe_count - number stripe on OST objects</para>
+ <para><literal>stripe_count </literal>- number stripe on OST objects</para>
</listitem>
<listitem>
- <para>tests_str - test operations. Must have at least "create" and "destroy"</para>
+ <para><literal>tests_str</literal> - test operations. Must have at least "create" and
+ "destroy"</para>
</listitem>
<listitem>
- <para>start_number - base number for each thread to prevent name collisions</para>
+ <para><literal>start_number</literal> - base number for each thread to prevent name
+ collisions</para>
</listitem>
<listitem>
- <para>layer - MDS stack's layer to be tested</para>
+ <para><literal>layer</literal> - MDS stack's layer to be tested</para>
</listitem>
</itemizedlist>
<para>Run without OST objects creation:</para>
<para>Setup the Lustre MDS without OST mounted. Then invoke the <literal>mds-survey</literal> script</para>
<screen>$ thrhi=64 file_count=200000 sh mds-survey</screen>
<para>Run with OST objects creation:</para>
- <para>Setup the Lustre MDS with at least one OST mounted. Then invoke the <literal>mds-survey</literal> script with stripe_count parameter</para>
+ <para>Setup the Lustre MDS with at least one OST mounted. Then invoke the
+ <literal>mds-survey</literal> script with <literal>stripe_count</literal>
+ parameter</para>
<screen>$ thrhi=64 file_count=200000 stripe_count=2 sh mds-survey</screen>
- <para>Note: a specific mdt instance can be specified using targets variable.</para>
+ <para>Note: a specific MDT instance can be specified using targets variable.</para>
<screen>$ targets=lustre-MDT0000 thrhi=64 file_count=200000 stripe_count=2 sh mds-survey</screen>
</listitem>
</orderedlist>
<para>The <literal>stats-collect</literal> utility requires:</para>
<itemizedlist>
<listitem>
- <para>Lustre to be installed and set up on your cluster</para>
+ <para>Lustre software to be installed and set up on your cluster</para>
</listitem>
<listitem>
<para>SSH and SCP access to these nodes without requiring a password</para>
<para><literal>SERVICE</literal> - Lustre OST and MDT RPC service statistics</para>
</listitem>
<listitem>
- <para><literal>BRW</literal> - OST block read/write statistics (brw_stats)</para>
+ <para><literal>BRW</literal> - OST bulk read/write statistics (brw_stats)</para>
</listitem>
<listitem>
<para><literal>SDIO</literal> - SCSI disk IO statistics (sd_iostats)</para>
git://git.whamcloud.com / doc / manual.git / blobdiff