recovery will take as long as is needed for the single MDS to be restarted.</para>
<para>When <xref linkend="imperativerecovery"/> is enabled, clients are notified of an MDS restart (either the backup or a restored primary). Clients always may detect an MDS failure either by timeouts of in-flight requests or idle-time ping messages. In either case the clients then connect to the new backup MDS and use the Metadata Replay protocol. Metadata Replay is responsible for ensuring that the backup MDS re-acquires state resulting from transactions whose effects were made visible to clients, but which were not committed to the disk.</para>
<para>The reconnection to a new (or restarted) MDS is managed by the file system configuration loaded by the client when the file system is first mounted. If a failover MDS has been configured (using the <literal>--failnode=</literal> option to <literal>mkfs.lustre</literal> or <literal>tunefs.lustre</literal>), the client tries to reconnect to both the primary and backup MDS until one of them responds that the failed MDT is again available. At that point, the client begins recovery. For more information, see <xref linkend="metadatereplay"/>.</para>
- <para>Transaction numbers are used to ensure that operations are replayed in the order they
- were originally performed, so that they are guaranteed to succeed and present the same file
- system state as before the failure. In addition, clients inform the new server of their
- existing lock state (including locks that have not yet been granted). All metadata and lock
- replay must complete before new, non-recovery operations are permitted. In addition, only
- clients that were connected at the time of MDS failure are permitted to reconnect during the
- recovery window, to avoid the introduction of state changes that might conflict with what is
- being replayed by previously-connected clients.</para>
- <para condition="l24">Lustre software release 2.4 introduces multiple metadata targets. If
- multiple metadata targets are in use, active-active failover is possible. See <xref
- linkend="dbdoclet.mdtactiveactive"/> for more information.</para>
+ <para>Transaction numbers are used to ensure that operations are
+ replayed in the order they were originally performed, so that they
+ are guaranteed to succeed and present the same file system state as
+ before the failure. In addition, clients inform the new server of their
+ existing lock state (including locks that have not yet been granted).
+ All metadata and lock replay must complete before new, non-recovery
+ operations are permitted. In addition, only clients that were connected
+ at the time of MDS failure are permitted to reconnect during the recovery
+ window, to avoid the introduction of state changes that might conflict
+ with what is being replayed by previously-connected clients.</para>
+ <para condition="l24">Lustre software release 2.4 introduces multiple
+ metadata targets. If multiple MDTs are in use, active-active failover
+ is possible (e.g. two MDS nodes, each actively serving one or more
+ different MDTs for the same filesystem). See
+ <xref linkend="dbdoclet.mdtactiveactive"/> for more information.</para>
</section>
<section remap="h3">
<title><indexterm><primary>recovery</primary><secondary>OST failure</secondary></indexterm>OST Failure (Failover)</title>
<para>Each client request processed by the server that involves any state change (metadata update, file open, write, etc., depending on server type) is assigned a transaction number by the server that is a target-unique, monotonically increasing, server-wide 64-bit integer. The transaction number for each file system-modifying request is sent back to the client along with the reply to that client request. The transaction numbers allow the client and server to unambiguously order every modification to the file system in case recovery is needed.</para>
<para>Each reply sent to a client (regardless of request type) also contains the last
committed transaction number that indicates the highest transaction number committed to the
- file system. The <literal>ldiskfs</literal> backing file system that the Lustre software
+ file system. The <literal>ldiskfs</literal> and <literal>ZFS</literal> backing file systems that the Lustre software
uses enforces the requirement that any earlier disk operation will always be committed to
disk before a later disk operation, so the last committed transaction number also reports
that any requests with a lower transaction number have been committed to disk.</para>
<para>The lock handle can be found by walking the list of granted locks for the resource looking for one with the appropriate remote file handle (present in the re-sent request). Verify that the lock has the right mode (determined by performing the disposition/request/status analysis above) and is granted to the proper client.</para>
</section>
</section>
+ <section remap="h3" condition="l28">
+ <title>Multiple Reply Data per Client</title>
+ <para>Since Lustre 2.8, the MDS is able to save several reply data per client. The reply data are stored in the <literal>reply_data</literal> internal file of the MDT. Additionally to the XID of the request, the transaction number, the result code and the open "disposition", the reply data contains a generation number that identifies the client thanks to the content of the <literal>last_rcvd</literal> file.</para>
+ </section>
</section>
<section xml:id="versionbasedrecovery">
<title><indexterm><primary>Version-based recovery (VBR)</primary></indexterm>Version-based Recovery</title>
git://git.whamcloud.com / doc / manual.git / blobdiff