From 907058386e795d992ab307dda9d517a2a86fac8d Mon Sep 17 00:00:00 2001 From: "Andrew C. Uselton" Date: Thu, 2 Jul 2015 11:39:04 -0500 Subject: [PATCH] LUDOC-294 protocol: Bring in new connection documentation The connect flags in connection.txt got some additional detail. Signed-off-by: Andrew C. Uselton Change-Id: Ic3dff4f664c815ab36bb46e391f92118a7d543e7 Reviewed-on: http://review.whamcloud.com/15474 Tested-by: Jenkins --- connection.txt | 161 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 155 insertions(+), 6 deletions(-) diff --git a/connection.txt b/connection.txt index 267c67e..b68b5e5 100644 --- a/connection.txt +++ b/connection.txt @@ -281,9 +281,131 @@ unnecessary RPCs for DNE. If the OBD_CONNECT_NODEVOH flag is set then the server provides no open handle for special inodes. -fixme: finish with the rest of flags - -The remaining flags are obsoleted and not used nowadays. +If the OBD_CONNECT_RMT_CLIENT is set then the client is set as +'remote' with respect to the server. The client is considered as +'local' if the user/group database on the client is identical to that +on the server, otherwise the client is set as 'remote'. This +terminology is part of Lustre Kerberos feature which is not supported +now. + +If the OBD_CONNECT_RMT_CLIENT_FORCE is set then client is set as +remote client forcefully. If the server security level doesn't support +remote clients then this client connect reply will return an -EACCESS +error. + +If the OBD_CONNECT_MDS_CAPA is set then MDS supports capability. +Capabilities are part of Lustre Kerberos. The MDS prepares the +capability when a file is opened and sends it to a client. A client +has to present a capability when it wishes to perform an operation on +that object. + +If the OBD_CONNECT_OSS_CAPA is set then OSS supports capability. +Capabilities are part of Lustre Kerberos. When the clients request the +OSS to perform a modification operations on objects the capability +authorizes these operations. + +If the OBD_CONNECT_CANCELSET is set then early batched cancels are +enabled. The ELC (Early Lock Cancel) feature allows client locks to +be cancelled prior the cancellation callback if it is clear that lock +is not needed anymore, for example after rename, after removing file +or directory, link, etc. This can reduce amount of RPCs significantly. + +If the OBD_CONNECT_AT is set then client and server use Adaptive +Timeout while request processing. Servers keep track of the RPCs +processing time and report this information back to clients to +estimate the time needed for future requests and set appropriate RPC +timeouts. + +If the OBD_CONNECT_LRU_RESIZE is set then the LRU self-adjusting is +enabled. This is set by the Lustre configurable option +--enable-lru-resize, and is enabled by default. + +If the OBD_CONNECT_FID is set then FID support is required by +server. This compatibility flag was introduced in Lustre 2.0. All +servers and clients are using FIDs nowadays. This flag is always set +on server and used to filter out clients without FID support. + +If the OBD_CONNECT_VBR is set then version based recovery is used on +server. The VBR uses an object version to track its changes on the +server and to decide if the replay can be applied during recovery +based on that version. This helps to complete recovery even if some +clients were missed or evicted. That flag is always set on server +since Lustre 1.8 and is used just to notify the server if client +doesn't support VBR. + +If the OBD_CONNECT_LOV_V3 is set then the client supports LOV vs +EA. This type of the LOV extended attribute was introduced along with +OST pools support and changed the internal structure of that EA. The +OBD_CONNECT_LOV_V3 flag notifies a server if client doesn't support +this type of LOV EA to handle requests from it properly. + +If the OBD_CONNECT_GRANT_SHRINK is set then the client can release +grant space when idle. + +If the OBD_CONNECT_SKIP_ORPHAN is set then OST doesn't reuse orphan +object ids after recovery. This connection flag is used between MDS +and OST to agree about an object pre-creation policy after MDS +recovery. If some of precreated objects weren't used but an MDT was +restarted then an OST may re-use not used objects for new pre-create +request or may not. The latter is preferred and is used by default +nowadays. + +If the OBD_CONNECT_FULL20 is set then the client is Lustre 2.x client. +Clients that are using old 1.8 format protocol conventions are not +allowed to connect. This flag should be set on all connections since +2.0, it is no longer affects behaviour and will be disabled completely +once Lustre interoperation with old clients is no longer needed. + +If the OBD_CONNECT_LAYOUTLOCK is set then the client supports layout +lock. The server will not grant a layout lock to the old clients +having no such flag. + +If the OBD_CONNECT_64BITHASH is set then the client supports 64-bit +directory hash. The server will also use 64-bit hash mode while +working with ldiskfs. + +If the OBD_CONNECT_JOBSTATS is set then the client fills jobid in +'ptlrpc_body' so server can provide extended statistics per jobid. + +If the OBD_CONNECT_UMASK is set then create uses client umask. This is +default flag for MDS but not for OST. + +If the OBD_CONNECT_LVB_TYPE is set then the variable type of LVB is +supported by a client. This flag was introduced along with DNE to +recognize DNE-aware clients. + +If the OBD_CONNECT_LIGHTWEIGHT is set then this connection is the +'lightweight' one. A lightweight connection has no entry in last_rcvd +file, so no recovery is possible, at the same time a lightweight +connection can be set up while the target is in recovery, locks can +still be acquired through this connection, although they won't be +replayed. Such type of connection is used by services like quota +manager, FLDB, etc. + +If the OBD_CONNECT_PINGLESS is set then pings can be suppressed. If +the client and server have this flag during connection and the ptlrpc +module on server has the option "suppress_pings", then pings will be +suppressed for this client. There must be an external mechanism to +notify the targets of client deaths, via the targets "evict_client" +'procfs' entries. Pings can be disabled on OSTs only. + +If the OBD_CONNECT_FLOCK_DEAD is set then the client support flock +cancellation, which is used for the flock deadlock detection mechanism. + +If the OBD_CONNECT_DISP_STRIPE is set then server returns a 'create +stripe' disposition for open request from the client. This helps to +optimize a recovery of open requests. + +If the OBD_CONNECT_OPEN_BY_FID is set then an open by FID won't pack +the name in a request. This is used by DNE. + +If the OBD_CONNECT_MDS_MDS is set then the current connection is a +MDS-MDS one. Such connections are distinguished because they provide +more functionality specific to MDS-MDS interoperation. + +If the OBD_CONNECT_IMP_RECOV is set then the Imperative Recovery is +supported. Imperative recovery means the clients are notified +explicitly when and where a failed target has restarted. The OBD_CONNECT_REQPORTAL was used to specify that client may use OST_REQUEST_PORTAL for requests to don't interfere with IO portal, @@ -306,7 +428,34 @@ The OBD_CONNECT_JOIN flag was used for the 'join files' feature, which allowed files to be concatenated. Lustre no longer supports that feature. -fixme: finish with rest of unused flags +The OBD_CONNECT_QUOTA64 was used prior Lustre 2.4 for quota purposes, +it is obsoleted due to new quota design. + +The OBD_CONNECT_REAL is not real connection flag but used locally on +client to distinguish real connection from local connections between +layers. + +The OBD_CONNECT_CHANGE_QS was used prior Lustre 2.4 for quota needs +and it is obsoleted now due to new quota design. + +If the OBD_CONNECT_EINPROGRESS is set then client handles -EINPROGRESS +RPC error properly. The quota design requires that client must resend +request with -EINPROGRESS error indefinitely, until successful +completion or another error. This flag is set on both client and +server by default. Meanwhile this flag is not checked anywere, so does +nothing. + +If the OBD_CONNECT_FLOCK_OWNER is set then 1.8 clients has fixed flock +policy and 2.x servers recognize them correctly. Meanwhile this flag +is not checked anywhere, so does nothing. + +If the OBD_CONNECT_NANOSEC_TIME is set then nanosecond timestamps are +enabled. This flag is not used nowadays, but reserved for future use. + +If the OBD_CONNECT_SHORTIO is set then short IO feature is enabled on +server. The server will avoid bulk IO for small amount of data but +data is incapsulated into ptlrpc request/reply. This flag is just +reserved for future use and does nothing nowadays. Import ^^^^^^ @@ -483,9 +632,9 @@ enumerated set of values: .enum_lustre_imp_state [options="header"] |===== -| state name | value +| state name | value | LUSTRE_IMP_CLOSED | 1 -| LUSTRE_IMP_NEW | 2 +| LUSTRE_IMP_NEW | 2 | LUSTRE_IMP_DISCON | 3 | LUSTRE_IMP_CONNECTING | 4 | LUSTRE_IMP_REPLAY | 5 -- 1.8.3.1