Named Request Message Pairs =========================== Each request message pair is identified in the source code by a symbol that starts iwth "RQF_". Each has a name and two message formats. The request message pair's name is a string. The two message formats in a message pair are the symbols given for a request message, on the one hand, and a reply message, on the other. Names ===== The name appears as a string in the first position (argument) of a "struct req_format" declaration. The symbol for the message pair is usually the same as the name string but with "RQF_" in front of it. Thus the message pair named "MDS_CONNECT" appears in the code as the symbol "RQF_MDS_CONNECT". The name itself is only used for reporting purposes, for example in the traces decoded and reported through the Lustre wireshark extension. There are a few occasions (likely bugs) where the name and the symbol do not actually match in the way described above. Those instances are: - Both RQF_LDLM_GL_DESC_CALLBACK and RQF_LDLM_GL_CALLBACK use the name LDLM_GL_CALLBACK - RQF_LOG_CANCEL is named OBD_LOG_CANCEL - RQF_MDS_REINT_CREATE_SLAVE is named MDS_REINT_CREATE_EA - Both RQF_MDS_CLOSE and RQF_MDS_RELEASE_CLOSE are named MDS_CLOSE - Both RQF_OUT_UPDATE and RQF_OUT_UPDATE_OBJ are named OUT_UPDATE The RQF_* symbols are gathered in the: static struct req_format *req_formats[] = array in layout.c. Their declarations, with the pair's name and the names of the two actual message formats, follow soon after in the same file. Messages ======== The pair of symbols in a request message pair identify the format for each corresponding PtlRPC message, with the first of the pair being a request for action of some sort and the second being the reply appropriate to the request. See message_formats.txt for details about the structure of each message format. In many cases the host making the request will be a Lustre client, but that is not universally the case. For example, a lock callback might be initiated by the MDS and the request sent from the MDS to a Lustre client. In other cases the message request is between two Lustre servers. The message pairs always list the (symbol for the) request first and then the reply. Many of those pairs employ the words "client" and "server" in the symbol's name. This is unfortunate. Such "client" messages are not necessarily sent from Lustre clients and such "server" messages are not necessarily replies sent from Lustre servers. A symbol with the word "client" should always be thought of simply as the format of a message initiating a request. Likewise, "server" symbols simply mean the format for a message in reply to a request, whatever the actual host that sends the reply. So the message pair should be understood as a message initiating a request and the message that is anticipated in reply to that request. Op Codes ======== The name in the RQF_BLA definition will sometimes, but not always, correspond to the source code symbolic representation (in lustre/include/lustre/lustre_idl.h) of an "op code" as encoded in the "__u32 pb_opc" field of the "struct ptlrpc_body_v3". The op code symbols are defined in a collection of *_cmd_t enumerations in lustre_idl.h. Historically, PtlRPC message consturuction was somewhat ad hoc, and carried out at the point where the message was needed. A newer and more systematic construction mechanism resulted in the list of "struct req_format RQF_*" message pair declarations. There are still some 21 or so op codes defined in lustre_idl.h that have no corresponding "struct req_format RQF_BLA" in layouts.c. One example is the "MGS_CONNECT" op code. A code audit will be necessary to see if any of those are obsolete (and/or unused) op codes or if they should be updated to the new form. There are also about 50 "struct req_format RQF_BLA" message pairs defined in layout.c for which there is no corresponding "BLA" op code. A code audit will be needed to determine if any or all of them should be introduced to lustre_idl.h as new op codes or if they (some of them) are obsolete or unused. In some cases (eg. RQF_MDS_REINT_*) one op code (eb. MDS_OPEN) is overloaded for all the message pairs of that type. The following are the 94 named message pairs from layout.c. Each one defines a pair of message formats, which are identified by name. The message formats are reused and there are 95 such message formats for the 94 named message pairs. The message formats themselves are detailed in message_formats.txt. As mentioned above, the first of the pair should be thought of as the actual "request message" being made, and the second of the pair is the "reply message". struct req_format name op code ----------------- ----------------- ----------------- RQF_CONNECT CONNECT obd_connect_client obd_connect_server RQF_FLD_QUERY FLD_QUERY fld_query_client fld_query_server RQF_FLD_READ FLD_READ fld_read_client fld_read_server RQF_LDLM_BL_CALLBACK LDLM_BL_CALLBACK LDLM_BL_CALLBACK ldlm_enqueue_client empty RQF_LDLM_CALLBACK LDLM_CALLBACK ldlm_enqueue_client empty RQF_LDLM_CANCEL LDLM_CANCEL LDLM_CANCEL ldlm_enqueue_client empty RQF_LDLM_CONVERT LDLM_CONVERT LDLM_CONVERT ldlm_enqueue_client ldlm_enqueue_server RQF_LDLM_CP_CALLBACK LDLM_CP_CALLBACK LDLM_CP_CALLBACK ldlm_cp_callback_client empty RQF_LDLM_ENQUEUE LDLM_ENQUEUE LDLM_ENQUEUE ldlm_enqueue_client ldlm_enqueue_lvb_server RQF_LDLM_ENQUEUE_LVB LDLM_ENQUEUE_LVB ldlm_enqueue_client ldlm_enqueue_lvb_server RQF_LDLM_GL_CALLBACK LDLM_GL_CALLBACK LDLM_GL_CALLBACK ldlm_enqueue_client ldlm_gl_callback_server RQF_LDLM_GL_DESC_CALLBACK LDLM_GL_CALLBACK ldlm_gl_callback_desc_client ldlm_gl_callback_server RQF_LDLM_INTENT LDLM_INTENT ldlm_intent_client ldlm_intent_server RQF_LDLM_INTENT_BASIC LDLM_INTENT_BASIC ldlm_intent_basic_client ldlm_enqueue_lvb_server RQF_LDLM_INTENT_CREATE LDLM_INTENT_CREATE ldlm_intent_create_client ldlm_intent_getattr_server RQF_LDLM_INTENT_GETATTR LDLM_INTENT_GETATTR ldlm_intent_getattr_client ldlm_intent_getattr_server RQF_LDLM_INTENT_GETXATTR LDLM_INTENT_GETXATTR ldlm_intent_getxattr_client ldlm_intent_getxattr_server RQF_LDLM_INTENT_LAYOUT LDLM_INTENT_LAYOUT ldlm_intent_layout_client ldlm_enqueue_lvb_server RQF_LDLM_INTENT_OPEN LDLM_INTENT_OPEN ldlm_intent_open_client ldlm_intent_open_server RQF_LDLM_INTENT_QUOTA LDLM_INTENT_QUOTA ldlm_intent_quota_client ldlm_intent_quota_server RQF_LDLM_INTENT_UNLINK LDLM_INTENT_UNLINK ldlm_intent_unlink_client ldlm_intent_server RQF_LFSCK_NOTIFY LFSCK_NOTIFY LFSCK_NOTIFY obd_lfsck_request empty RQF_LFSCK_QUERY LFSCK_QUERY LFSCK_QUERY obd_lfsck_request obd_lfsck_reply RQF_LLOG_ORIGIN_CONNECT LLOG_ORIGIN_CONNECT llogd_conn_body_only empty RQF_LLOG_ORIGIN_HANDLE_CREATE LLOG_ORIGIN_HANDLE_CREATE llog_origin_handle_create_client llogd_body_only RQF_LLOG_ORIGIN_HANDLE_DESTROY LLOG_ORIGIN_HANDLE_DESTROY llogd_body_only llogd_body_only RQF_LLOG_ORIGIN_HANDLE_NEXT_BLOCK LLOG_ORIGIN_HANDLE_NEXT_BLOCK llogd_body_only llog_origin_handle_next_block_server RQF_LLOG_ORIGIN_HANDLE_PREV_BLOCK LLOG_ORIGIN_HANDLE_PREV_BLOCK llogd_body_only llog_origin_handle_next_block_server RQF_LLOG_ORIGIN_HANDLE_READ_HEADER LLOG_ORIGIN_HANDLE_READ_HEADER llogd_body_only llog_log_hdr_only RQF_LOG_CANCEL OBD_LOG_CANCEL OBD_LOG_CANCEL log_cancel_client empty RQF_MDS_CLOSE MDS_CLOSE MDS_CLOSE mdt_close_client mds_last_unlink_server RQF_MDS_CONNECT MDS_CONNECT MDS_CONNECT obd_connect_client obd_connect_server RQF_MDS_DISCONNECT MDS_DISCONNECT MDS_DISCONNECT empty empty RQF_MDS_DONE_WRITING MDS_DONE_WRITING MDS_DONE_WRITING mdt_close_client mdt_body_only RQF_MDS_GETATTR MDS_GETATTR MDS_GETATTR mdt_body_capa mds_getattr_server RQF_MDS_GETATTR_NAME MDS_GETATTR_NAME MDS_GETATTR_NAME mds_getattr_name_client mds_getattr_server RQF_MDS_GETSTATUS MDS_GETSTATUS MDS_GETSTATUS mdt_body_only mdt_body_capa RQF_MDS_GETXATTR MDS_GETXATTR MDS_GETXATTR mds_getxattr_client mds_getxattr_server RQF_MDS_GET_INFO MDS_GET_INFO MDS_GET_INFO mds_getinfo_client mds_getinfo_server RQF_MDS_HSM_ACTION MDS_HSM_ACTION MDS_HSM_ACTION mdt_body_capa mdt_hsm_action_server RQF_MDS_HSM_CT_REGISTER MDS_HSM_CT_REGISTER MDS_HSM_CT_REGISTER mdt_hsm_ct_register empty RQF_MDS_HSM_CT_UNREGISTER MDS_HSM_CT_UNREGISTER MDS_HSM_CT_UNREGISTER mdt_hsm_ct_unregister empty RQF_MDS_HSM_PROGRESS MDS_HSM_PROGRESS MDS_HSM_PROGRESS mdt_hsm_progress empty RQF_MDS_HSM_REQUEST MDS_HSM_REQUEST MDS_HSM_REQUEST mdt_hsm_request empty RQF_MDS_HSM_STATE_GET MDS_HSM_STATE_GET MDS_HSM_STATE_GET mdt_body_capa mdt_hsm_state_get_server RQF_MDS_HSM_STATE_SET MDS_HSM_STATE_SET MDS_HSM_STATE_SET mdt_hsm_state_set empty RQF_MDS_QUOTACHECK MDS_QUOTACHECK MDS_QUOTACHECK quotactl_only empty RQF_MDS_QUOTACTL MDS_QUOTACTL MDS_QUOTACTL quotactl_only quotactl_only RQF_MDS_READPAGE MDS_READPAGE MDS_READPAGE mdt_body_capa mdt_body_only RQF_MDS_REINT MDS_REINT MDS_REINT mds_reint_client mdt_body_only RQF_MDS_REINT_CREATE MDS_REINT_CREATE MDS_OPEN mds_reint_create_client mdt_body_capa RQF_MDS_REINT_CREATE_RMT_ACL MDS_REINT_CREATE_RMT_ACL MDS_OPEN mds_reint_create_rmt_acl_client mdt_body_capa RQF_MDS_REINT_CREATE_SLAVE MDS_REINT_CREATE_EA MDS_OPEN mds_reint_create_slave_client mdt_body_capa RQF_MDS_REINT_CREATE_SYM MDS_REINT_CREATE_SYM MDS_OPEN mds_reint_create_sym_client mdt_body_capa RQF_MDS_REINT_LINK MDS_REINT_LINK MDS_OPEN mds_reint_link_client mdt_body_only RQF_MDS_REINT_OPEN MDS_REINT_OPEN MDS_OPEN mds_reint_open_client mds_reint_open_server RQF_MDS_REINT_RENAME MDS_REINT_RENAME MDS_OPEN mds_reint_rename_client mds_last_unlink_server RQF_MDS_REINT_SETATTR MDS_REINT_SETATTR MDS_OPEN mds_reint_setattr_client mds_setattr_server RQF_MDS_REINT_SETXATTR MDS_REINT_SETXATTR MDS_OPEN mds_reint_setxattr_client mdt_body_only RQF_MDS_REINT_UNLINK MDS_REINT_UNLINK MDS_OPEN mds_reint_unlink_client mds_last_unlink_server RQF_MDS_RELEASE_CLOSE MDS_CLOSE mdt_release_close_client mds_last_unlink_server RQF_MDS_STATFS MDS_STATFS MDS_STATFS empty obd_statfs_server RQF_MDS_SWAP_LAYOUTS MDS_SWAP_LAYOUTS MDS_SWAP_LAYOUTS mdt_swap_layouts empty RQF_MDS_SYNC MDS_SYNC MDS_SYNC mdt_body_capa mdt_body_only RQF_MGS_CONFIG_READ MGS_CONFIG_READ MGS_CONFIG_READ mgs_config_read_client mgs_config_read_server RQF_MGS_SET_INFO MGS_SET_INFO MGS_SET_INFO mgs_set_info mgs_set_info RQF_MGS_TARGET_REG MGS_TARGET_REG MGS_TARGET_REG mgs_target_info_only mgs_target_info_only RQF_OBD_IDX_READ OBD_IDX_READ OBD_IDX_READ obd_idx_read_client obd_idx_read_server RQF_OBD_PING OBD_PING OBD_PING empty empty RQF_OBD_SET_INFO OBD_SET_INFO obd_set_info_client empty RQF_OST_BRW_READ OST_BRW_READ ost_brw_client ost_brw_read_server RQF_OST_BRW_WRITE OST_BRW_WRITE ost_brw_client ost_brw_write_server RQF_OST_CONNECT OST_CONNECT OST_CONNECT obd_connect_client obd_connect_server RQF_OST_CREATE OST_CREATE OST_CREATE ost_body_only ost_body_only RQF_OST_DESTROY OST_DESTROY OST_DESTROY ost_destroy_client ost_body_only RQF_OST_DISCONNECT OST_DISCONNECT OST_DISCONNECT empty empty RQF_OST_GETATTR OST_GETATTR OST_GETATTR ost_body_capa ost_body_only RQF_OST_GET_INFO OST_GET_INFO OST_GET_INFO ost_get_info_generic_client ost_get_info_generic_server RQF_OST_GET_INFO_FIEMAP OST_GET_INFO_FIEMAP ost_get_fiemap_client ost_get_fiemap_server RQF_OST_GET_INFO_LAST_FID OST_GET_INFO_LAST_FID ost_get_last_fid_client ost_get_last_fid_server RQF_OST_GET_INFO_LAST_ID OST_GET_INFO_LAST_ID ost_get_info_generic_client ost_get_last_id_server RQF_OST_PUNCH OST_PUNCH OST_PUNCH ost_body_capa ost_body_only RQF_OST_QUOTACHECK OST_QUOTACHECK OST_QUOTACHECK quotactl_only empty RQF_OST_QUOTACTL OST_QUOTACTL OST_QUOTACTL quotactl_only quotactl_only RQF_OST_SETATTR OST_SETATTR OST_SETATTR ost_body_capa ost_body_only RQF_OST_SET_GRANT_INFO OST_SET_GRANT_INFO ost_grant_shrink_client ost_body_only RQF_OST_SET_INFO_LAST_FID OST_SET_INFO_LAST_FID obd_set_info_client empty RQF_OST_STATFS OST_STATFS OST_STATFS empty obd_statfs_server RQF_OST_SYNC OST_SYNC OST_SYNC ost_body_capa ost_body_only RQF_OUT_UPDATE OUT_UPDATE_OBJ OUT_UPDATE mds_update_client mds_update_server RQF_QC_CALLBACK QC_CALLBACK quotactl_only empty RQF_QUOTA_DQACQ QUOTA_DQACQ quota_body_only quota_body_only RQF_SEC_CTX SEC_CTX empty empty RQF_SEQ_QUERY SEQ_QUERY seq_query_client seq_query_server Extra Op Codes ============== These op codes apear in some *_cmd_t enumeration in lustre_idl.h, but do not correspond to any RQF_* request format declaration in layout.c: LDLM_SET_INFO MDS_IS_SUBDIR MDS_PIN MDS_SETXATTR MDS_SET_INFO MDS_UNPIN MDS_WRITEPAGE MGS_CONNECT MGS_DISCONNECT MGS_EXCEPTION MGS_TARGET_DEL OBD_LOG_CANCEL OST_CLOSE OST_OPEN OST_QUOTA_ADJUST_QUNIT OST_READ OST_REPLY OST_SET_INFO OST_WRITE QUOTA_DQREL SEC_CTX_FINI SEC_CTX_INIT SEC_CTX_INIT_CONT A code audit will be needed to determine if they are unused or, like MGS_CONNECT, are introduced in an ad hoc fashion at the point the message is needed. If possible, the still in use op codes should be folded into the new "struct req_format" style of declaration, and their creation and management made uniform with the rest. In cases where this is impossible, the reason and the handling should both be explained clearly.