Data Structures and Defines ~~~~~~~~~~~~~~~~~~~~~~~~~~~ [[data-structs]] The following data types are used in the Lustre protocol description. .Basic Data Types [options="header"] |===== | data types | size | __u8 | an 8-bit unsigned integer | __u16 | a 16-bit unsigned integer | __u32 | a 32-bit unsigned integer | __u64 | a 64-bit unsigned integer | __s64 | a 64-bit signed integer | obd_time | an __s64 |===== The following topics introduce the various kinds of data that are represented and manipulated in Lustre messages and representations of the shared state on clients and servers. Grant ^^^^^ [[grant]] A grant value is part of a client's state for a given target. It provides an upper bound to the amount of dirty cache data the client will allow that is destined for the target. The value is established by agreement between the server and the client and represents a guarantee by the server that the target storage has space for the dirty data. The client can ask for additional grant, which the server may provide depending on how full the target is. LOV Index ^^^^^^^^^ [[lov-index]] Each target is assigned an LOV index (by the 'mkfs' command line) as the target is added to the file system. This value is stored in the MGS in order to identify its role in the file system. Transaction Number ^^^^^^^^^^^^^^^^^^ [[transno]] For each target there is a sequence of values (a strictly increasing series of numbers) where each operation that can modify the file system is assigned the next number in the series. This is the transaction number, and it imposes a strict serial ordering to all of the file system modifying operations. For file system modifying requests the server generates the next value in the sequence and informs the client of the value in the 'pb_transno' field of the 'ptlrpc_body' of its reply to the client's request. For replys to requests that do not modify the file system the 'pb_transno' field in the 'ptlrpc_body' is just set to 0. Extended Attributes ^^^^^^^^^^^^^^^^^^^ I have not figured out how so called 'eadata' buffers are handled, yet. I am told that this is not just for extended attributes, but is a generic structure. Lustre Capabilities ^^^^^^^^^^^^^^^^^^^ A 'lustre_capa' structure conveys details about the capabilities supported (or requested) between a client and a given target. I am told that this is deprecated in later version of Lustre. ---- #define CAPA_HMAC_MAX_LEN 64 struct lustre_capa { struct lu_fid lc_fid; __u64 lc_opc; __u64 lc_uid; __u64 lc_gid; __u32 lc_flags; __u32 lc_keyid; __u32 lc_timeout; __u32 lc_expiry; __u8 lc_hmac[CAPA_HMAC_MAX_LEN]; } ---- include::lustre_file_ids.txt[] MGS Configuration Reference ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ---- #define MTI_NAME_MAXLEN 64 struct mgs_config_body { char mcb_name[MTI_NAME_MAXLEN]; /* logname */ __u64 mcb_offset; /* next index of config log to request */ __u16 mcb_type; /* type of log: CONFIG_T_[CONFIG|RECOVER] */ __u8 mcb_reserved; __u8 mcb_bits; /* bits unit size of config log */ __u32 mcb_units; /* # of units for bulk transfer */ }; ---- The 'mgs_config_body' structure has information identifying to the MGS which Lustre file system the client is asking about. MGS Configuration Data ^^^^^^^^^^^^^^^^^^^^^^ ---- struct mgs_config_res { __u64 mcr_offset; /* index of last config log */ __u64 mcr_size; /* size of the log */ }; ---- The 'mgs_config_res' structure returns information about the Lustre file system. include::lustre_handle.txt[] Lustre Message Header ^^^^^^^^^^^^^^^^^^^^^ [[lustre-message-header]] Every message has an initial header that informs the receiver about the size of the rest of the message to follow along with a few other details. ---- #define LUSTRE_MSG_MAGIC_V2 0x0BD00BD3 #define MSGHDR_AT_SUPPORT 0x1 struct lustre_msg_v2 { __u32 lm_bufcount; __u32 lm_secflvr; __u32 lm_magic; __u32 lm_repsize; __u32 lm_cksum; __u32 lm_flags; __u32 lm_padding_2; __u32 lm_padding_3; __u32 lm_buflens[0]; }; #define lustre_msg lustre_msg_v2 ---- The 'lm_buffcount' field gives the number of buffers that will follow the header. The header and sequence of buffers constitutes one message. Each of the buffers is a sequence of bytes whose contents corresponds to one of the structures described in this section. There will always be at least one, and no message has more than eight. The 'lm_secflvr' field gives an indication of whether any sort of cyptographic encoding of the subsequent buffers will be in force. The value is zero if there is no "crypto" and gives a code identifying the "flavor" of crypto if it is employed. Further, if crypto is employed there will only be one buffer following (i.e. buffcount = 1), and that buffer is an encoding of what would otherwise have been the sequence of buffers normally following the header. This document will defer all discussion of cryptography. An chapter is planned that will address it separately. The 'lm_magic' field is a "magic" value (LUSTRE_MSG_MAGIC_V2) that is checked in order to positively identify that the message is intended for the use to which it is being put. That is, we are indeed dealing with a Lustre message, and not, for example, corrupted memory or a bad pointer. The 'lm_repsize' field is an indication from the sender of an action request of the maximum available space that has been set aside for any reply to the request. A reply that attempts to use more than that much space will be discarded. The 'lm_cksum' has to do with the <> settings for the cluster. Fixme: This may not be in current use. We need to verify. The 'lm_flags' field can be set to enable adaptive timeouts support with the value MSGHDR_AT_SUPPORT. The 'lm_padding*' fields are reserved for future use. The array of 'lm_bufflens' values has 'lm_bufcount' entries. Each entry corresponds to, and gives the length of, one of the buffers that will follow. The entire header is required to be a multiple of eight bytes long. Thus there may need to an extra four bytes of padding after the 'lm_bufflens' array if that array has an odd number of entries. include::ptlrpc_body.txt[] Object Based Disk UUID ^^^^^^^^^^^^^^^^^^^^^^ ---- #define UUID_MAX 40 struct obd_uuid { char uuid[UUID_MAX]; }; ---- OST ID ^^^^^^ ---- struct ost_id { union { struct ostid { __u64 oi_id; __u64 oi_seq; } oi; struct lu_fid oi_fid; } LUSTRE_ANONYMOUS_UNION_NAME; }; ----