the connection and has an import for the MGS, while the MGS has an
export for each MDS. Each MDS connects to each OST, with an import on
the MDS and an export on the OSS. This connection supports requests
-from the MDS to the OST for 'statfs' information such as size and
-access time values. Each OSS also connects to the first MDS to get
+from the MDS to the OST to create and destroy data objects, to set
+attributes (such as permission bits), and for 'statfs' information for
+precreation needs. Each OSS also connects to the first MDS to get
access to auxiliary services, with an import on the OSS and an export
-on the first MDS. The auxiliary services are: the File ID Location
+on the first MDS. The auxiliary services are: the File ID Location
Database (FLDB), the quota master service, and the sequence
-controller.
+controller. This connection for auxiliary services is a 'lightweight'
+one in that it has no replay functionality and consumes no space on
+the MDS for client data. Each MDS connects also to all other MDSs for
+DNE needs.
Finally, for some communications the roles of message initiation and
message reply are reversed. This is the case, for instance, with
is "in use". The arrival of an otherwise unprocessed message for this
target will increment the refcount. A reference by an LDLM lock that
gets taken will increment the refcount. Callback invocations and
-replay also lead to incrementing the ref_count. The next for fields -
-'exp_rpc_count', exp_cb_count', and 'exp_replay_count', and
-'exp_locks_count' - all subcategorize the 'exp_refcount' for debug
+replay also lead to incrementing the 'ref_count'. The next four fields
+- 'exp_rpc_count', exp_cb_count', and 'exp_replay_count', and
+'exp_locks_count' - all subcategorize the 'exp_refcount'. The
+reference counter keeps the export alive while there are any users of
+that export. The reference counter is also used for debug
purposes. Similarly, the 'exp_locks_list' and 'exp_locks_list_guard'
-are further debug info that lists the actual locks accounted in
+are further debug info that list the actual locks accounted for in
'exp_locks_count'.
The 'exp_client_uuid' gives the UUID of the client connected to this
The 'exp_obd' points to the 'obd_device' structure for the device that
is the target of this export.
-In the event of a call-back the export needs to have a the ability to
+In the event of an LDLM call-back the export needs to have a the ability to
initiate messages back to the client. The 'exp_imp_reverse' provides a
"reverse" import that manages this capability.
the request and its attendant locks need to be preserved until either
a) the client acknowleges recieving the reply, or b) the transaction
has been committed locally. This ensures a request can be replayed in
-the event of a failure. The reply is kept on the
-'exp_outstanding_replies' list until the LNet layer notifies the
-server that the reply has been acknowledged. A reply is kept on the
-'exp_uncommitted_replies' list until the transaction (if any) has been
-committed.
+the event of a failure. The LDLM lock is being kept until one of these
+event occurs to prevent any other modifications of the same object.
+The reply is kept on the 'exp_outstanding_replies' list until the LNet
+layer notifies the server that the reply has been acknowledged. A reply
+is kept on the 'exp_uncommitted_replies' list until the transaction
+(if any) has been committed.
The 'exp_last_committed' value keeps the transaction number of the
last committed transaction. Every reply to a client includes this
and transaction replay. For a file system modifying request, the
server composes its reply including the 'pb_pre_versions' entries in
'ptlrpc_body', which indicate the most recent updates to the
-object. The client updates the request wth teh 'pb_transno' and
+object. The client updates the request with the 'pb_transno' and
'pb_pre_versions' from the reply, and keeps that request until the
target signals that the transaction has been committed to disk. If the
client times-out without that confirmation then it will 'replay' the
request, which now includes the 'pb_pre_versions' information. During
-a replay the target checks that the object has not been further
-modified beyond those 'pb_pre_versions'. If this check fails then the
-request is out of date, and the recovery process fails for the
-connection between this client and this target. At that point the
-'exp_vbr_failed' flag is set to indicate version based recovery
-failed. This will lead to the client being evicted and this export
-being cleaned up and deleted.
+a replay the target checks that the object has the same version as
+'pb_pre_versions' in replay. If this check fails then the object can't
+be restored in the same state as it was in before failure. Usually that
+happens if the recovery process fails for the connection between some
+other client and this target, so part of change needed for this client
+wasn't restored. At that point the 'exp_vbr_failed' flag is set
+to indicate version based recovery failed. This will lead to the client
+being evicted and this export being cleaned up and deleted.
At the start of recovery both the 'exp_req_replay_needed' and
'exp_lock_replay_needed' flags are set. As request replay is completed
disk. When the export is created the 'exp_need_sync' flag is set and
the actual writing to disk is delayed. As transactions arrive from
clients (in a much less coordinated fashion) the 'exp_need_sync' flag
-indicates a need to save the export as well as the transaction. At
-that point the flag is cleared (except see below).
+indicates a need to have the export data on disk before proceeding
+with a new transaction, so as it is next updated the transaction is
+done synchronously to commit all changes on disk. At that point the
+flag is cleared (except see below).
In DNE (phase I) the export for an MDT managing the connection from
another MDT will want to always keep the 'exp_need_sync' flag set. For
As RPCs arrive they are first subjected to triage. Each request is
placed on the 'exp_hp_rpcs' list and examined to see if it is high
-priority (fixme: what constitutes high priority? PING, truncate, bulk
-I/O, ... others?). If it is not high priority then it is moved to the
-'exp_reg_prcs' list. The 'exp_rpc_lock' protects both lists from
-concurrent access.
+priority (PING, truncate, bulk I/O). If it is not high priority then
+it is moved to the 'exp_reg_prcs' list. The 'exp_rpc_lock' protects
+both lists from concurrent access.
All arriving LDLM requests get put on the 'exp_bl_list' and access to
that list is controlled via the 'exp_bl_list_lock'.
Connection Count
^^^^^^^^^^^^^^^^
-Each export maintains a connection count. Or is it just the management
-server?
+Each export maintains a connection count.
git://git.whamcloud.com / doc / protocol.git / blobdiff