Lustre Operations ----------------- [[lustre-operations]] Lustre operations are denoted by the 'pb_opc' op-code field of the RPC descriptor. Each operation is implemented as a pair of messages, with the 'pb_type' field set to PTLRPC_MSG_REQUEST for requests initiating the operation, and PTLRPC_MSG_REPLY for replies. Note that as a general matter, the receipt by a client of the reply message only assures the client hat the server has initiated the action, if any. See the discussion on <> and <> for how the client is given confirmation that a request has been completed. include::ost_setattr.txt[] Command 8: OST CONNECT - Client connection to an OST ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[ost-connect-rpc]] .OST_CONNECT (8) [options="header"] |==== | request | reply | obd_connect_client | obd_connect_server |==== When a client initiates a connection to a specific target on an OSS, it does so by sending an 'obd_connect_client' message and awaiting the reply from the OSS of an 'obd_connect_server' message. From a previous interaction with the MGS the client knows the UUID of the target OST, and can fill that value into the 'obd_connect_client' message. The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the capabilities appropriate to the client. The 'ocd_brw_size' is set to the largest value for the size of an RPC that the client can handle. The 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client considers appropriate. Other fields in the descriptor and 'obd_connect_data' structures are zero, as is the 'lustre_handle' element. Once the server receives the 'obd_connect_client' message on behalf of the given target it replies with an 'obd_connect_server' message. In that message the server sends the 'pb__handle' to uniquely identify the connection for subsequent communication. The client notes that handle in its import for the given target. fixme: Are there circumstances that could lead to the 'status' value in the reply being non-zero? What would lead to that and what error values would result? The target maintains the last committed transaction for a client in its export for that client. If this is the first connection, then that last transaction value would just be zero. If there were previous transactions for the client, then the transaction number for the last such committed transaction is put in the 'pb_last_committed' field. In a connection request the operation is not file system modifying, so the 'pb_transno' value will be zero in the reply as well. fixme: there is still some work to be done about how the fields are managed. Command 9: OST DISCONNECT - Disconnect client from an OST ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[ost-disconnect-rpc]] .OST_DISCONNECT (9) [options="header"] |==== | request | reply | empty | empty |==== The information exchanged in a DISCONNECT message is that normally conveyed in the RPC descriptor. include::ost_punch.txt[] include::ost_statfs.txt[] Command 33: MDS GETATTR - Get MDS Attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mds-getattr-rpc]] .MDS_GETATTR (33) [options="header"] |==== | request | reply | mdt_body_capa | mds_getattr_server |==== include::mds_reint.txt[] Command 38: MDS CONNECT - Client connection to an MDS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mds-connect-rpc]] .MDS_CONNECT (38) [options="header"] |==== | request | reply | obd_connect_client | obd_connect_server |==== N.B. This is nearly identical to the explanation for OST_CONNECT and for MGS_CONNECT. We may want to simplify and/or unify the discussion and only call out how this one differs from a generic CONNECT operation. When a client initiates a connection to a specific target on an MDS, it does so by sending an 'obd_connect_client' message and awaiting the reply from the MDS of an 'obd_connect_server' message. From a previous interaction with the MGS the client knows the UUID of the target MDT, and can fill that value into the 'obd_connect_client' message. The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the capabilities appropriate to the client. The 'ocd_brw_size' is set to the largest value for the size of an RPC that the client can handle. The 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client considers appropriate. Other fields in the descriptor and 'obd_connect_data' structures are zero, as is the 'lustre_handle' element. Once the server receives the 'obd_connect_client' message on behalf of the given target it replies with an 'obd_connect_server' message. In that message the server sends the 'pb__handle' to uniquely identify the connection for subsequent communication. The client notes that handle in its import for the given target. fixme: Are there circumstances that could lead to the 'status' value in the reply being non-zero? What would lead to that and what error values would result? The target maintains the last committed transaction for a client in its export for that client. If this is the first connection, then that last transaction value would just be zero. If there were previous transactions for the client, then the transaction number for the last such committed transaction is put in the 'pb_last_committed' field. In a connection request the operation is not file system modifying, so the 'pb_transno' value will be zero in the reply as well. fixme: there is still some work to be done about how the fields are managed. Command 39: MDS DISCONNECT - Disconnect client from an MDS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mds-disconnect-rpc]] .MDS_DISCONNECT (39) [options="header"] |==== | request | reply | empty | empty |==== The information exchanged in a DISCONNECT message is that normally conveyed in the RPC descriptor. Command 40: MDS GETSTATUS - get the status from a target ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mds-getstatus-rpc]] The MDS_GETSTATUS command targets a specific MDT. If there are several, the client will need to send a separate message for each. .MDS_GETSTATUS (40) [options="header"] |==== | request | reply | mdt_body_only | mdt_body_capa |==== The 'mdt_body_only' request message is one that provides information about a specific MDT, identifying which (among several possible) target is being asked about. In the reply there is additional information about the MDT's capabilities. include::mds_statfs.txt[] include::mds_getxattr.txt[] include::ldlm_enqueue.txt[] include::ldlm_cancel.txt[] include::ldlm_bl_callback.txt[] include::ldlm_cp_callback.txt[] include::ldlm_gl_callback.txt[] Command 250: MGS CONNECT - Client connection to an MGS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mgs-connect-rpc]] .MGS_CONNECT (250) [options="header"] |==== | request | reply | obd_connect_client | obd_connect_server |==== When a client initiates a connection to the MGS, it does so by sending an 'obd_connect_client' message and awaiting the reply from the MGS of an 'obd_connect_server' message. This is the first operation carried out by a client upon the issue of a 'mount' command, and the target UUID is provided on the command line. The target UUID is just "MGS", and the client UUID is set to the 32byte string it gets from ... where? The 'struct lustre_handle' (the fourth buffer in the message) has its cookie set to .. what? It is set, but where does it come from? The 'ocd_connect_flags' field is set to (fixme: what?) reflecting the capabilities appropriate to the client. The 'ocd_brw_size' is set to the largest value for the size of an RPC that the client can handle. The 'ocd_ibits_known' and 'ocd_checksum_types' values are set to what the client considers appropriate. Other fields in the descriptor and 'obd_connect_data' structures are zero. Once the server receives the 'obd_connect_client' message on behalf of the given target it replies with an 'obd_connect_server' message. In that message the server sends the 'pb__handle' to uniquely identify the connection for subsequent communication. The client notes that handle in its import for the given target. fixme: Are there circumstances that could lead to the 'status' value in the reply being non-zero? What would lead to that and what error values would result? The target maintains the last committed transaction for a client in its export for that client. If this is the first connection, then that last transaction value would just be zero. If there were previous transactions for the client, then the transaction number for the last such committed transaction is put in the 'pb_last_committed' field. In a connection request the operation is not file system modifying, so the 'pb_transno' value will be zero in the reply as well. Command 251: MGS DISCONNECT - Disconnect client from an MGS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mgs-disconnect-rpc]] .MGS_DISCONNECT (251) [options="header"] |==== | request | reply | empty | empty |==== N.B. The usual 'struct req_format' definition does not exist for MGS_DISCONNECT. It will take a more careful code review to verify that it also has 'empty' messages gong back and forth. The information exchanged in a DISCONNECT message is that normally conveyed in the RPC descriptor. Command 256: MGS CONFIG READ - Read MGS configuration info ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[mgs-config-read-rpc]] .MGS_CONFIG_READ (256) [options="header"] |==== | request | reply | mgs_config_read_client | mgs_config_read_server |==== Command 501: LLOG ORIGIN HANDLE CREATE - Create llog handle ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[llog-origin-handle-create-rpc]] .LLOG_ORIGIN_HANDLE_CREATE (510) [options="header"] |==== | request | reply | llog_origin_handle_create_client | llogd_body_only |==== Command 502: LLOG ORIGIN HANDLE NEXT BLOCK - Read the next block ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[llog-origin-handle-next-block-rpc]] .LLOG_ORIGIN_HANDLE_NEXT_BLOCK (502) [options="header"] |==== | request | reply | llogd_body_only | llog_origin_handle_next_block_server |==== Command 503: LLOG ORIGIN HANDLE READ HEADER - Read handle header ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[llog-origin-handle-read-header-rpc]] .LLOG_ORIGIN_HANDLE_READ_HEADER (503) [options="header"] |==== | request | reply | llogd_body_only | llog_log_hdr_only |====