Whamcloud - gitweb
LU-5545 ptlrpc: false alarm in AT network latency measuring 55/12855/2
authorLiang Zhen <liang.zhen@intel.com>
Tue, 25 Nov 2014 19:21:34 +0000 (11:21 -0800)
committerOleg Drokin <oleg.drokin@intel.com>
Thu, 4 Dec 2014 20:21:41 +0000 (20:21 +0000)
If early reply of client RPC is lost and client RPC is expired and
resent, server will drop the resent RPC because it's already in
processing, server may also send reply or early reply to client,
which can still match reply buffer of the original request.
In this case, client is measuring time from resent time, but server
is reporting service time of original RPC, which is longer than
the time measured by client.

This patch is back-ported from the following one:
Lustre-commit: c238be0bd02367b634c1061a30bbe161dfd9d8e9
Lustre-change: http://review.whamcloud.com/12018

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I5bdcdee9dd6b439d5d650fa812ea65527458c590
Reviewed-on: http://review.whamcloud.com/12855
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
lustre/ptlrpc/client.c

index f3793ca..e1a467a 100644 (file)
@@ -294,14 +294,27 @@ static void ptlrpc_at_adj_net_latency(struct ptlrpc_request *req,
         time_t now = cfs_time_current_sec();
 
         LASSERT(req->rq_import);
-        at = &req->rq_import->imp_at;
+
+       if (service_time > now - req->rq_sent + 3) {
+               /* bz16408, however, this can also happen if early reply
+                * is lost and client RPC is expired and resent, early reply
+                * or reply of original RPC can still be fit in reply buffer
+                * of resent RPC, now client is measuring time from the
+                * resent time, but server sent back service time of original
+                * RPC.
+                */
+               CDEBUG((lustre_msg_get_flags(req->rq_reqmsg) & MSG_RESENT) ?
+                      D_ADAPTTO : D_WARNING,
+                      "Reported service time %u > total measured time "
+                      CFS_DURATION_T"\n", service_time,
+                      cfs_time_sub(now, req->rq_sent));
+               return;
+       }
 
         /* Network latency is total time less server processing time */
-        nl = max_t(int, now - req->rq_sent - service_time, 0) +1/*st rounding*/;
-        if (service_time > now - req->rq_sent + 3 /* bz16408 */)
-                CWARN("Reported service time %u > total measured time "
-                      CFS_DURATION_T"\n", service_time,
-                      cfs_time_sub(now, req->rq_sent));
+       nl = max_t(int, now - req->rq_sent -
+                       service_time, 0) + 1; /* st rounding */
+       at = &req->rq_import->imp_at;
 
         oldnl = at_measured(&at->iat_net_latency, nl);
         if (oldnl != 0)