lov_io_submit receives a range of pages, then adds pages in
to a batch until it hits a page which is not in the stripe
associated with this lov object. This means that if a
readahead page range hits the same stripe more than once,
we will issue multiple I/Os, even if the pages would fit in
one RPC.
This is unnecessary - Just submit all these pages at once.
mpirun -n 2 $IOR -s 2000 -t 47K -b 47K -k -r -E -o $FILE
Without patch:
osc.lustre-OST0001-osc-
ffff8fe82c952000.rpc_stats=
read write
pages per rpc rpcs % cum % | rpcs % cum %
1: 118 56 56 | 0 0 0
2: 0 0 56 | 0 0 0
4: 0 0 56 | 0 0 0
8: 0 0 56 | 0 0 0
16: 5 2 58 | 0 0 0
32: 0 0 58 | 0 0 0
64: 0 0 58 | 0 0 0
128: 21 10 68 | 0 0 0
256: 25 11 80 | 0 0 0
512: 10 4 85 | 0 0 0
1024: 31 14 100 | 0 0 0
osc.lustre-OST0002-osc-
ffff8fe82c952000.rpc_stats=
read write
pages per rpc rpcs % cum % | rpcs % cum %
1: 5 6 6 | 0 0 0
2: 0 0 6 | 0 0 0
4: 0 0 6 | 0 0 0
8: 0 0 6 | 0 0 0
16: 0 0 6 | 0 0 0
32: 0 0 6 | 0 0 0
64: 0 0 6 | 0 0 0
128: 19 23 29 | 0 0 0
256: 19 23 52 | 0 0 0
512: 5 6 58 | 0 0 0
1024: 34 41 100 | 0 0 0
With patch:
osc.lustre-OST0001-osc-
ffff8fe7a7227800.rpc_stats=
read write
pages per rpc rpcs % cum % | rpcs % cum %
1: 12 17 17 | 0 0 0
2: 0 0 17 | 0 0 0
4: 0 0 17 | 0 0 0
8: 0 0 17 | 0 0 0
16: 5 7 24 | 0 0 0
32: 0 0 24 | 0 0 0
64: 5 7 31 | 0 0 0
128: 6 8 40 | 0 0 0
256: 1 1 42 | 0 0 0
512: 2 2 44 | 0 0 0
1024: 38 55 100 | 0 0 0
osc.lustre-OST0002-osc-
ffff8fe7a7227800.rpc_stats=
read write
pages per rpc rpcs % cum % | rpcs % cum %
1: 0 0 0 | 0 0 0
2: 0 0 0 | 0 0 0
4: 0 0 0 | 0 0 0
8: 0 0 0 | 0 0 0
16: 0 0 0 | 0 0 0
32: 0 0 0 | 0 0 0
64: 4 7 7 | 0 0 0
128: 7 13 21 | 0 0 0
256: 0 0 21 | 0 0 0
512: 3 5 26 | 0 0 0
1024: 38 73 100 | 0 0 0
Note the much larger # of smaller RPC issued without the patch.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic10c138628c269afe57fbc57ec8c91ce990717f9
Reviewed-on: https://review.whamcloud.com/35458
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
struct lov_io_sub *sub;
struct cl_page_list *plist = &lov_env_info(env)->lti_plist;
struct cl_page *page;
+ struct cl_page *tmp;
int index;
int rc = 0;
ENTRY;
cl_page_list_move(&cl2q->c2_qin, qin, page);
index = lov_page_index(page);
- while (qin->pl_nr > 0) {
- page = cl_page_list_first(qin);
+ cl_page_list_for_each_safe(page, tmp, qin) {
+ /* this page is not on this stripe */
if (index != lov_page_index(page))
- break;
+ continue;
cl_page_list_move(&cl2q->c2_qin, qin, page);
}