</itemizedlist>
<para>Specific Lustre timeouts are described below.</para>
<para><literal> /proc/sys/lustre/timeout </literal></para>
- <para>This is the time period that a client waits for a server to complete an RPC (default is 100s). Servers wait half of this time for a normal client RPC to complete and a quarter of this time for a single bulk request (read or write of up to 1 MB) to complete. The client pings recoverable targets (MDS and OSTs) at one quarter of the timeout, and the server waits one and a half times the timeout before evicting a client for being "stale."</para>
+ <para>This is the time period that a client waits for a server to complete an RPC (default is 100s). Servers wait half of this time for a normal client RPC to complete and a quarter of this time for a single bulk request (read or write of up to 4 MB) to complete. The client pings recoverable targets (MDS and OSTs) at one quarter of the timeout, and the server waits one and a half times the timeout before evicting a client for being "stale."</para>
<note>
<para>Lustre sends periodic 'PING' messages to servers with which it had no communication for a specified period of time. Any network activity on the file system that triggers network traffic toward servers also works as a health check.</para>
</note>
<para>... and so on.</para>
<para>RPC stream tunables are described below.</para>
<para><literal> osc.<replaceable>osc_instance</replaceable>.max_dirty_mb </literal></para>
- <para xml:id='lustreproc.maxdirtymb'>This tunable controls how many MBs of dirty data can be written and queued up in the OSC. POSIX file writes that are cached contribute to this count. When the limit is reached, additional writes stall until previously-cached writes are written to the server. This may be changed by writing a single ASCII integer to the file. Only values between 0 and 512 are allowable. If 0 is given, no writes are cached. Performance suffers noticeably unless you use large writes (1 MB or more).</para>
+ <para xml:id='lustreproc.maxdirtymb'>This tunable controls how many MBs of dirty data can be written and queued up in the OSC. POSIX file writes that are cached contribute to this count. When the limit is reached, additional writes stall until previously-cached writes are written to the server. This may be changed by writing a single ASCII integer to the file. Only values between 0 and 2048 or 1/4 of RAM are allowable. If 0 is given, no writes are cached. Performance suffers noticeably unless you use large writes (1 MB or more).</para>
<para><literal> osc.<replaceable>osc_instance</replaceable>.cur_dirty_bytes </literal></para>
<para>This tunable is a read-only value that returns the current amount of bytes written and cached on this OSC.</para>
<para><literal> osc.<replaceable>osc_instance</replaceable>.max_pages_per_rpc </literal></para>
- <para>This tunable is the maximum number of pages that will undergo I/O in a single RPC to the OST. The minimum is a single page and the maximum for this setting is platform dependent (256 for i386/x86_64, possibly less for ia64/PPC with larger <literal>PAGE_SIZE</literal>), though generally amounts to a total of 1 MB in the RPC.</para>
+ <para>This tunable is the maximum number of pages that will undergo I/O in a single RPC to the OST. The minimum is a single page and the maximum for this setting is 1024 (for systems with 4kB <literal>PAGE_SIZE</literal>), with the default maximum of 1MB in the RPC. It is also possible to specify a units suffix (e.g. <literal>4M</literal>), so that the RPC size can be specified independently of the client <literal>PAGE_SIZE</literal>.</para>
<para><literal> osc.<replaceable>osc_instance</replaceable>.max_rpcs_in_flight </literal></para>
<para>This tunable is the maximum number of concurrent RPCs in flight from an OSC to its OST. If the OSC tries to initiate an RPC but finds that it already has the same number of RPCs outstanding, it will wait to issue further RPCs until some complete. The minimum setting is 1 and maximum setting is 256. If you are looking to improve small file I/O performance, increase the <literal>max_rpcs_in_flight</literal> value.</para>
<para>To maximize performance, the value for <literal>max_dirty_mb</literal> is recommended to be 4 * <literal>max_pages_per_rpc</literal> * <literal>max_rpcs_in_flight</literal>.</para>
<para>In addition to the MDS memory requirements mentioned in <xref linkend="dbdoclet.50438256_87676"/>, the OSS requirements include:</para>
<itemizedlist>
<listitem>
- <para><emphasis role="bold">Service threads</emphasis> : The service threads on the OSS node pre-allocate a 1 MB I/O buffer for each ost_io service thread, so these buffers do not need to be allocated and freed for each I/O request.</para>
+ <para><emphasis role="bold">Service threads</emphasis> : The service threads on the OSS node pre-allocate a 4 MB I/O buffer for each ost_io service thread, so these buffers do not need to be allocated and freed for each I/O request.</para>
</listitem>
<listitem>
<para><emphasis role="bold">OSS read cache</emphasis> : OSS read cache provides read-only caching of data on an OSS, using the regular Linux page cache to store the data. Just like caching from a regular file system in Linux, OSS read cache uses as much physical memory as is available.</para>
<title><indexterm><primary>setup</primary><secondary>memory</secondary><tertiary>OSS</tertiary></indexterm>Calculating OSS Memory Requirements</title>
<para>The minimum recommended RAM size for an OSS with two OSTs is computed below:</para>
<informalexample>
- <para>Ethernet/TCP send/receive buffers (1 MB * 512 threads) = 512 MB</para>
+ <para>Ethernet/TCP send/receive buffers (4 MB * 512 threads) = 2048 MB</para>
<para>400 MB journal size * 2 OST devices = 800 MB</para>
<para>1.5 MB read/write per OST IO thread * 512 threads = 768 MB</para>
<para>600 MB file system read cache * 2 OSTs = 1200 MB</para>
git://git.whamcloud.com / doc / manual.git / commitdiff