From: zab Date: Fri, 5 Sep 2003 20:05:18 +0000 (+0000) Subject: b=1760 X-Git-Tag: v1_7_0_51~2^7~594 X-Git-Url: https://git.whamcloud.com/?a=commitdiff_plain;h=2f6fa930b6e734b38923b524d23894f63e7a0dbf;p=fs%2Flustre-release.git b=1760 Refactor the async ocp IO API to use less cpu on the client. With these changes streaming writes are no longer cpu-bound in llite. The major changes are to queue dirty pages for IO at commit_write() and reduce the per-page book-keeping (bye otree!). Pages are now associated with llite book-keeping for the duration of their stay in the page cache. A patch is needed in most kernels before 2.4.21 to add the removepage callback to tear down this association as pages leave the page cache. Because of this the lustre kernel version is upped to 25. now, in detail: - get rid of osc page queue plugging. instead we track pages for much longer in the queue and only fire rpcs when we have either enough to produce optimal rpcs or there are 'urgent' pages in the osc that must leave. - add some fields to the ocp to remember its position in the lov to remove lov_stripe_offset() from the profiles. - remove the nasty code for walking sb/inode dirty lists to find other pages to jam down the obd at writepage time. (yay!) - b=1755, b=1757, b=1810: make the dirty page limits and rpcs tunables really per-osc, getting rid of the fragile and redunant tracking that was going on in llite. this fixes the way commit_write sleeps for space in the cache without creating an enormous bubble in the rpc pipeline. Also cover them with osc spinlocks insted of using costly atomic ops. - b=1750: add a 'read_ahead' tunable per superblock for disabling llite read-ahead. - b=1837, b=1761: add some callbacks so that the osc can call back into llite when an rpc really fires. it wants to know if it has raced with a truncate and also wants the most recent times in an obdo. - b=1601: make commit_write's fallback to sync writes much clearer. it also is more careful about extending i_size only for successful writes. - add ll_removepage() which removes the ocp from the page and calls into the osc to remove it form the queues at truncate time. - teach read-ahead to really never block looking up pages in the page cache by calling grab_cache_page_nowait() (duh) - get rid of the otree that we used to track dirty offsets in objects. an llite level hack for prefering its i_size that includes cache extensions works until we implement the proper EOC communications to track i_size. this adds the obd_lock_contains() call until that time. - get rid of the old dead code for passing extents from ll_file_read to readpage. also old 2.6 lli experiments. --- diff --git a/lustre/kernel_patches/patches/removepage-2.4.20.patch b/lustre/kernel_patches/patches/removepage-2.4.20.patch new file mode 100644 index 0000000..cc721e1 --- /dev/null +++ b/lustre/kernel_patches/patches/removepage-2.4.20.patch @@ -0,0 +1,28 @@ + include/linux/fs.h | 1 + + mm/filemap.c | 3 +++ + 2 files changed, 4 insertions(+) + +--- linux-2.4.20-b_llpmd-l24/include/linux/fs.h~removepage-2.4.20 2003-09-05 11:45:42.000000000 -0700 ++++ linux-2.4.20-b_llpmd-l24-zab/include/linux/fs.h 2003-09-05 11:46:25.000000000 -0700 +@@ -402,6 +402,7 @@ struct address_space_operations { + int (*releasepage) (struct page *, int); + #define KERNEL_HAS_O_DIRECT /* this is for modules out of the kernel */ + int (*direct_IO)(int, struct inode *, struct kiobuf *, unsigned long, int); ++ void (*removepage)(struct page *); /* called when page gets removed from the inode */ + }; + + struct address_space { +--- linux-2.4.20-b_llpmd-l24/mm/filemap.c~removepage-2.4.20 2003-09-05 11:45:42.000000000 -0700 ++++ linux-2.4.20-b_llpmd-l24-zab/mm/filemap.c 2003-09-05 11:46:25.000000000 -0700 +@@ -95,6 +95,9 @@ static inline void remove_page_from_inod + { + struct address_space * mapping = page->mapping; + ++ if (mapping->a_ops->removepage) ++ mapping->a_ops->removepage(page); ++ + mapping->nrpages--; + list_del(&page->list); + page->mapping = NULL; + +_ diff --git a/lustre/kernel_patches/pc/removepage-2.4.20.pc b/lustre/kernel_patches/pc/removepage-2.4.20.pc new file mode 100644 index 0000000..c659e15 --- /dev/null +++ b/lustre/kernel_patches/pc/removepage-2.4.20.pc @@ -0,0 +1,2 @@ +include/linux/fs.h +mm/filemap.c diff --git a/lustre/osc/osc_create.c b/lustre/osc/osc_create.c index 3fee907..3fb9d08 100644 --- a/lustre/osc/osc_create.c +++ b/lustre/osc/osc_create.c @@ -49,7 +49,6 @@ #include #include /* for mds_objid */ -#include #include #include #include