LU-1431 ptlrpc: Support for over 1MB bulk I/O RPC

LU-1431 ptlrpc: Support for over 1MB bulk I/O RPC

Increase the maximum size of BRW RPCs between OSCs and OSTs to 4MB
by increasing the number of LNet Memory Descriptors assosiated with
a bulk.  Each bulk transfer is still limited to LNET_MTU_SIZE, so
that it can be passed through routers and does not break RDMA limits.

The client and server negotiate the maximum BRW RPC size at connect
time via ocd_brw_size, with the client sending the maximum size it
supports, and the server returning min(client_max, server_max) back
to the client.

For each RPC, the number of bulk MDs that are registered depends on
the actual RPC size at sending time.  The (max_brw_count - 1) is sent
to the OST in the high 16 bits of obd_ioobj.ioo_max_brw and forms a
mask for the bulk RPC XIDs.  The actual number of bulk transfers
is encoded in the BRW RPC XID.  The masked value of the request XID
determines the starting bulk transfer match bits, and the last bulk
match is the RPC XID.

For older clients this means that the starting and ending bulk match
bits are the RPC XID itself (as it always was) and the old ioo_type
(now ioo_max_brw) field being set to zero would result in a mask of
"1", which just keeps the same RPC XID as it was before.

It is important to note that the client and server do NOT share the
value of PTLRPC_BULK_OPS_MASK, or this would incorrectly mask the
RPC XID if the PTLRPC_BULK_OPS_BITS was ever increased on the server.

The actual BRW RPC size can be controlled for each client separately
by changing the client's max_pages_per_rpc.  For example, this will
set it to 4MB:

        lctl set_param osc.*.max_pages_per_rpc=1024

It is also possible to specify a units suffix, so that the RPC size
can be set independently of the client PAGE_SIZE:

        lctl set_param osc.*.max_pages_per_rpc=4M

By default, the size is 1MB for for OST BRW RPCs until more testing
is done. All other bulk I/O (e.g. MDC<->MDS) is left at 1MB.

With PTLRPC_MAX_BRW_SIZE growing larger, it doesn't make sense for
the readahead to increase the window size so dramatically at one time
Instead, limit readahead growth by the current inode i_blkbits value,
which currently defaults to 4MB.

If we want to tune the readahead growth so that it matches the actual
cl_max_pages_per_rpc value, then this can be done much more easily by
changing the per-inode i_blkbits value, since this is tied to specific
OSTs with specific RPC settings (which may be different for e.g. local
and remote OSTs), and only needs to be checked once.

Change the ras_*() functions to take the inode as an argument, and
always put it first in the argument list for consistency.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Sergii Glushchenko <sergii_glushchenko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Change-Id: I757b14a04e5d4cc053576e41e47864c743c35b8b
Xyratex-bug-id: MRP-319
Reviewed-on: http://review.whamcloud.com/4993
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>