5 Lustre operations are denoted by the 'pb_opc' op-code field of the
6 message preamble. Each operation is implemented as a pair of messages,
7 with the 'pb_type' field set to PTLRPC_MSG_REQUEST for requests
8 initiating the operation, and PTLRPC_MSG_REPLY for replies. Note that
9 as a general matter, the receipt by a client of the rply message only
10 assures the client hat the server has initiated the action, if
11 any. See the discussion on <<transno,transaction numbers>>
12 and <<recovery>> for how the client is given confirmation that a
13 request has been completed.
15 Command 8: OST CONNECT - Client connection to an OST
16 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
22 | obd_connect_client | obd_connect_server
25 When a client initiates a connection to a specific target on an OSS,
26 it does so by sending an 'obd_connect_client' message and awaiting the
27 reply from the OSS of an 'obd_connect_server' message. From a previous
28 interaction with the MGS the client knows the UUID of the target OST,
29 and can fill that value into the 'obd_connect_client' message.
31 The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the
32 capabilities appropriate to the client. The 'ocd_brw_size' is set to the
33 largest value for the size of an RPC that the client can handle. The
34 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client
35 considers appropriate. Other fields in the preamble and
36 'obd_connect_data' structures are zero, as is the 'lustre_handle'
39 Once the server receives the 'obd_connect_client' message on behalf of
40 the given target it replies with an 'obd_connect_server' message. In
41 that message the server sends the 'pb__handle' to uniquely
42 identify the connection for subsequent communication. The client notes
43 that handle in its import for the given target.
45 fixme: Are there circumstances that could lead to the 'status'
46 value in the reply being non-zero? What would lead to that and what
47 error values would result?
49 The target maintains the last committed transaction for a client in
50 its export for that client. If this is the first connection, then that
51 last transactiion value would just be zero. If there were previous
52 transactions for the client, then the transaction number for the last
53 such committed transaction is put in the 'pb_last_committed' field.
55 In a connection request the operation is not file system modifying, so
56 the 'pb_transno' value will be zero in the reply as well.
58 fixme: there is still some work to be done about how the fields are
61 Command 9: OST DISCONNECT - Disconnect client from an OST
62 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
71 The information exchanged in a DISCONNECT message is that normally
72 conveyed in the mesage preamble.
74 Command 33: MDS GETATTR - Get MDS Attributes
75 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
81 | mdt_body_capa | mds_getattr_server
86 Command 38: MDS CONNECT - Client connection to an MDS
87 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
93 | obd_connect_client | obd_connect_server
96 N.B. This is nearly identical to the explanation for OST_CONNECT and
97 for MGS_CONNECT. We may want to simplify and/or unify the discussion
98 and only call out how this one differees from a generic CONNECT
101 When a client initiates a connection to a specific target on an MDS,
102 it does so by sending an 'obd_connect_client' message and awaiting the
103 reply from the MDS of an 'obd_connect_server' message. From a previous
104 interaction with the MGS the client knows the UUID of the target MDT,
105 and can fill that value into the 'obd_connect_client' message.
107 The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the
108 capabilities appropriate to the client. The 'ocd_brw_size' is set to the
109 largest value for the size of an RPC that the client can handle. The
110 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client
111 considers appropriate. Other fields in the preamble and
112 'obd_connect_data' structures are zero, as is the 'lustre_handle'
115 Once the server receives the 'obd_connect_client' message on behalf of
116 the given target it replies with an 'obd_connect_server' message. In
117 that message the server sends the 'pb__handle' to uniquely
118 identify the connection for subsequent communication. The client notes
119 that handle in its import for the given target.
121 fixme: Are there circumstances that could lead to the 'status'
122 value in the reply being non-zero? What would lead to that and what
123 error values would result?
125 The target maintains the last committed transaction for a client in
126 its export for that client. If this is the first connection, then that
127 last transactiion value would just be zero. If there were previous
128 transactions for the client, then the transaction number for the last
129 such committed transaction is put in the 'pb_last_committed' field.
131 In a connection request the operation is not file system modifying, so
132 the 'pb_transno' value will be zero in the reply as well.
134 fixme: there is still some work to be done about how the fields are
137 Command 39: MDS DISCONNECT - Disconnect client from an MDS
138 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
147 The information exchanged in a DISCONNECT message is that normally
148 conveyed in the mesage preamble.
150 Command 40: MDS GETSTATUS - get the status from a target
151 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
153 The MDS_GETSTATUS command targets a specific MDT. If there are several,
154 the client will need to send a separate message for each.
160 | mdt_body_only | mdt_body_capa
163 The 'mdt_body_only' request message is one that provides information
164 about a specific MDT, identifying which (among several possible)
165 target is being asked about.
167 In the reply there is additional information about the MDT's
170 Command 41: MDS STATFS - get statfs data from the server
171 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
177 | empty | obd_statfs_server
180 The 'empty' request message is one that only has the 'ptlrpc_body'
181 data encoded. The fields have thier generic values for a request from
182 this client, with 'pb_opc' being set to MDS_STATFS (41).
184 In the reply there is, in addition to the 'ptlrpc_body', data relevant
185 to a 'statfs' system call.
187 Command 101: LDLM ENQUEUE - Enqueue a lock request
188 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
194 | ldlm_enqueue_client | ldlm_enqueue_lvb_server
197 In order to access data on disk at a target the client needs to
198 establish a lock (either concurrent for reads or exclusive for
199 writes). The client sends the 'ldlm_enqueue_client' message to the
200 server holding the target, and the server will reply with an
201 'ldlm_enqueue_server' message. (N.B. It actuallity it is an
202 'ldlm_enqueue_lvb_server' message with the length of the 'struct
203 ost_lvb' portion set to zero, which ammounts to the same thing).
205 The target UUID is just "MGS", and the client UUID is set to the
206 32byte string it gets from ... where?
208 The 'struct lustre_handle' (the fourth buffer in the message) has its
209 cookie set to .. what? It is set, but where does it come from?
211 The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the
212 capabilities appropriate to the client. The 'ocd_brw_size' is set to the
213 largest value for the size of an RPC that the client can handle. The
214 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client
215 considers appropriate. Other fields in the preamble and
216 'obd_connect_data' structures are zero.
218 Once the server receives the 'obd_connect_client' message on behalf of
219 the given target it replies with an 'obd_connect_server' message. In
220 that message the server sends the 'pb__handle' to uniquely
221 identify the connection for subsequent communication. The client notes
222 that handle in its import for the given target.
224 fixme: Are there circumstances that could lead to the 'status'
225 value in the reply being non-zero? What would lead to that and what
226 error values would result?
228 The target maintains the last committed transaction for a client in
229 its export for that client. If this is the first connection, then that
230 last transactiion value would just be zero. If there were previous
231 transactions for the client, then the transaction number for the last
232 such committed transaction is put in the 'pb_last_committed' field.
234 In a connection request the operation is not file system modifying, so
235 the 'pb_transno' value will be zero in the reply as well.
237 Command 250: MGS CONNECT - Client connection to an MGS
238 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
244 | obd_connect_client | obd_connect_server
247 When a client initiates a connection to the MGS,
248 it does so by sending an 'obd_connect_client' message and awaiting the
249 reply from the MGS of an 'obd_connect_server' message. This is the
250 first operation carried out by a client upon the issue of a 'mount'
251 command, and the target UUID is provided on the command line.
253 The target UUID is just "MGS", and the client UUID is set to the
254 32byte string it gets from ... where?
256 The 'struct lustre_handle' (the fourth buffer in the message) has its
257 cookie set to .. what? It is set, but where does it come from?
259 The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the
260 capabilities appropriate to the client. The 'ocd_brw_size' is set to the
261 largest value for the size of an RPC that the client can handle. The
262 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client
263 considers appropriate. Other fields in the preamble and
264 'obd_connect_data' structures are zero.
266 Once the server receives the 'obd_connect_client' message on behalf of
267 the given target it replies with an 'obd_connect_server' message. In
268 that message the server sends the 'pb__handle' to uniquely
269 identify the connection for subsequent communication. The client notes
270 that handle in its import for the given target.
272 fixme: Are there circumstances that could lead to the 'status'
273 value in the reply being non-zero? What would lead to that and what
274 error values would result?
276 The target maintains the last committed transaction for a client in
277 its export for that client. If this is the first connection, then that
278 last transactiion value would just be zero. If there were previous
279 transactions for the client, then the transaction number for the last
280 such committed transaction is put in the 'pb_last_committed' field.
282 In a connection request the operation is not file system modifying, so
283 the 'pb_transno' value will be zero in the reply as well.
285 Command 251: MGS DISCONNECT - Disconnect client from an MGS
286 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
288 .MGS_DISCONNECT (251)
295 N.B. The usual 'struct req_format' definition does not exist for
296 MGS_DISCONNECT. It will take a more careful code review to verify that
297 it also has 'empty' messages gong back and forth.
299 The information exchanged in a DISCONNECT message is that normally
300 conveyed in the mesage preamble.
302 Command 256: MGS CONFIG READ - Read MGS configuration info
303 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
305 .MGS_CONFIG_READ (256)
309 | mgs_config_read_client | mgs_config_read_server
312 Command 501: LLOG ORIGIN HANDLE CREATE - Create llog handle
313 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
315 .LLOG_ORIGIN_HANDLE_CREATE (510)
319 | llog_origin_handle_create_client | llogd_body_only
322 Command 502: LLOG ORIGIN HANDLE NEXT BLOCK - Read the next block
323 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
325 .LLOG_ORIGIN_HANDLE_NEXT_BLOCK (502)
329 | llogd_body_only | llog_origin_handle_next_block_server
332 Command 503: LLOG ORIGIN HANDLE READ HEADER - Read handle header
333 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
335 .LLOG_ORIGIN_HANDLE_READ_HEADER (503)
339 | llogd_body_only | llog_log_hdr_only
342 LLOG_ORIGIN_HANDLE_NEXT_BLOCK