:keywords: PtlRPC, Lustre, Protocol
-:numbered!: [abstract] Abstract -------- The Lustre parallel file
-system <<lustre>> provides a global POSIX <<POSIX>> namespace for the
-computing resources of a data center. Lustre runs on Linux-based hosts
-via kernel modules, and delegates block storage management to the
-back-end servers while providing object-based storage to its
-clients. Servers are responsible for both data objects (the contents
-of actual files) and index objects (for directory information). Data
-objects are gathered on Object Storage Servers (OSSs), and index
-objects are stored on MetaData Storage Servers (MDSs). Each back-end
-storage volume is a target with Object Storage Targets (OSTs) on OSSs,
-and MetaData Storage Targets (MDTs) on MDSs. Clients assemble the
-data from the MDT(s) and OST(s) to present a single coherent
-POSIX-compliant file system. The clients and servers communicate and
-coordinate among themselves via network protocols. A low-level
-protocol, LNet, abstracts the details of the underlying networking
-hardware and presents a uniform interface, originally based on Sandia
-Portals <<PORTALS>>, to Lustre clients and servers. Lustre, in turn,
-layers its own protocol PtlRPC atop LNet. This document describes the
-Lustre protocols, including how they are implemeted via PtlRPC and the
-Lustre Distributed Lock Manager (based on the VAX/VMS Distributed Lock
-Manager <<VAX_DLM>>). This document does not describe Lustre itself in
-any detail, except where doing so explains concepts that allow this
-document to be self-contained.
+include::introduction.txt[]
-:numbered:
-
-Overview
---------
-
-'Content to be provided'
-
-Messages
---------
-
-These are the messages that traverse the network using PTLRPC.
-
-[NOTE]
-This initial list combines some actual message names or types with the
-POSIX semantic operations they are being used to implement, as well as
-a few other underlying mechanisms (cf. "grant"). A subsequent
-refinement will separate the various items and relate them to one
-another.
-
-Client-MDS RPCs for POSIX namespace operations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-'Content to be provided'
-
-=== mount ===
-
-'Content to be provided'
-
-=== unmount ===
-
-'Content to be provided'
-
-=== create ===
-
-'Content to be provided'
-
-=== open ===
-
-'Content to be provided'
-
-=== close ===
-
-'Content to be provided'
-
-=== unlink ===
-
-'Content to be provided'
-
-=== mkdir ===
-
-image:mkdir1.png[mkdir]
-
-=== rmdir ===
-
-'Content to be provided'
-
-=== rename ===
-
-'Content to be provided'
-
-=== link ===
-
-'Content to be provided'
-
-=== symlink ===
-
-'Content to be provided'
-
-=== getattr ===
-
-'Content to be provided'
-
-=== setattr ===
-
-'Content to be provided'
-
-=== statfs ===
-
-'Content to be provided'
-
-=== ... ===
-
-'Content to be provided'
-
-
-Client-MDS RPCs for internal state management
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-'Content to be provided'
-
-=== connect ===
-
-'Content to be provided'
-
-=== disconnect ===
-
-'Content to be provided'
-
-=== FLD ===
-
-'Content to be provided'
-
-=== SEQ ===
-
-'Content to be provided'
-
-=== PING ===
+include::data_types.txt[]
-'Content to be provided'
+include::connection.txt[]
-=== LDLM ===
+include::timeouts.txt[]
-'Content to be provided'
+include::file_id.txt[]
-=== ... ===
+include::ldlm.txt[]
-'Content to be provided'
+include::llog.txt[]
-Client-OSS RPCs for IO Operations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+include::recovery.txt[]
-'Content to be provided'
+include::security.txt[]
-=== read ===
+include::lustre_messages.txt[]
-'Content to be provided'
+include::lustre_operations.txt[]
-=== write ===
-
-'Content to be provided'
-
-=== truncate ===
-
-'Content to be provided'
-
-=== setattr ===
-
-'Content to be provided'
-
-=== grant ===
-
-'Content to be provided'
-
-=== ... ===
-
-'Content to be provided'
-
-MDS-OSS RPCs for internal state management
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-'Content to be provided'
-
-=== object precreation ===
-
-'Content to be provided'
-
-=== orphan recovery ===
-
-'Content to be provided'
-
-=== UID/GID change ===
-
-'Content to be provided'
-
-=== unlink ===
-
-'Content to be provided'
-
-=== ... ===
-
-'Content to be provided'
-
-MDS-OSS RPCs for quota management
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-'Content to be provided'
-
-
-MDS-OSS OUT RPCs for distributed updates
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-'Content to be provided'
-
-=== DNE1 remote directories ===
-
-'Content to be provided'
-
-=== DNE2 striped directories ===
-
-'Content to be provided'
-
-=== LFSCK2/3 verification and repair ===
-
-'Content to be provided'
-
-Message Flows
--------------
-
- Each file operation (in Lustre) generates a set of messages in a
- particular sequence. There is one sequence for any particular
- concrete operation, but under varying circumstances the same file
- operation may generate a different sequence.
-
-State Machines
---------------
-
- For each File operation, the collection of possible sequences of
- messages is governed by a state machine.
+include::file_system_operations.txt[]
:numbered!:
-[glossary]
-Glossary
---------
-Here are some common terms used in discussing Lustre, POSIX semantics,
-and the prtocols used to implement them.
-
-[glossary]
-Object Storage Server::
- An object storage server (OSS) is a computer responsible for
- running Lustre kernel services in support of managing bulk data
- objects on the underlying storage. There can be multiple OSSs in a
- Lustre file system.
-
-MetaData Server::
- A metadata server (MDS) is a computer responsible for running the
- Lustre kernel services in support of managing the POSIX-compliant
- name space and the indices associating files in that name space with
- the locations of their corresponding objects. As of v2.4 there can
- be multiple MDSs in a Lustre file system.
-
-Object Storage Target::
- An object storage target (OST) is the service provided by an OSS
- that mediates the placement of data objects on the specific
- underlying file system hardware. There can be multiple OSTs on a
- given OSS.
-
-MetaData Target::
- A metadata target (MDT) is the service provided by an MDS that
- mediates the management of name space indices on the underlying file
- system hardware. As of v2.4 there can be multiple MDTs on an MDS.
-
-server::
- A computer providing a service, such as an OSS or an MDS
-
-target::
- Storage available to be served, such as an OST or an MDT. Also the
- service being provided.
-
-protocol::
- An agreed upon formalism for communicating between two entities,
- such as between two servers or between a client and a server.
-
-client::
- A computer taking advantage of a service provided by a server, such
- as a Lustre client using MDS(s) and OSS(s) to assemble a
- POSIX-compliant file system with its namespace and data storage
- capabilities.
-
-PtlRPC::
- The protocol (or set of protocols) implemented via RPCs that is
- (are) employed by Lustre to communicate between its clients and
- servers.
-
-Remote Procedure Call::
- A mechanism for implementing operations involving one computer
- acting on the behalf of another (RPC).
-
-LNet::
- A lower level protocol employed by PtlRPC to abstract the mechanisms
- provided by various hardware centric protocols, such as TCP or
- Infiniband.
-
-[appendix]
-Concepts
---------
-
-'Content to be provided'
+include::glossary.txt[]
[appendix]
License
git://git.whamcloud.com / doc / protocol.git / blobdiff