b=1760

Refactor the async ocp IO API to use less cpu on the client.  With these
changes streaming writes are no longer cpu-bound in llite.  The major
changes are to queue dirty pages for IO at commit_write() and reduce the
per-page book-keeping (bye otree!).

Pages are now associated with llite book-keeping for the duration of their
stay in the page cache.  A patch is needed in most kernels before 2.4.21 to
add the removepage callback to tear down this association as pages leave
the page cache.  Because of this the lustre kernel version is upped to 25.

now, in detail:

- get rid of osc page queue plugging.  instead we track pages for much longer
  in the queue and only fire rpcs when we have either enough to produce optimal
  rpcs or there are 'urgent' pages in the osc that must leave.

- add some fields to the ocp to remember its position in the lov to remove
  lov_stripe_offset() from the profiles.

- remove the nasty code for walking sb/inode dirty lists to find other pages to
  jam down the obd at writepage time.  (yay!)

- b=1755, b=1757, b=1810: make the dirty page limits and rpcs tunables really
  per-osc, getting rid of the fragile and redunant tracking that was going on
  in llite.  this fixes the way commit_write sleeps for space in the cache
  without creating an enormous bubble in the rpc pipeline.  Also cover them
  with osc spinlocks insted of using costly atomic ops.

- b=1750: add a 'read_ahead' tunable per superblock for disabling llite
  read-ahead.

- b=1837, b=1761: add some callbacks so that the osc can call back into llite
  when an rpc really fires.  it wants to know if it has raced with a truncate
  and also wants the most recent times in an obdo.

- b=1601: make commit_write's fallback to sync writes much clearer.  it also is
  more careful about extending i_size only for successful writes.

- add ll_removepage() which removes the ocp from the page and calls into the
  osc to remove it form the queues at truncate time.

- teach read-ahead to really never block looking up pages in the page cache
  by calling grab_cache_page_nowait() (duh)

- get rid of the otree that we used to track dirty offsets in objects.  an
  llite level hack for prefering its i_size that includes cache extensions
  works until we implement the proper EOC communications to track i_size.
  this adds the obd_lock_contains() call until that time.

- get rid of the old dead code for passing extents from ll_file_read to
  readpage.  also old 2.6 lli experiments.

diff --git a/lustre/kernel_patches/patches/removepage-2.4.20.patch b/lustre/kernel_patches/patches/removepage-2.4.20.patch
new file mode 100644 (file)
index 0000000..cc721e1
--- /dev/null
+++ b/lustre/kernel_patches/patches/removepage-2.4.20.patch
@@ -0,0 +1,28 @@
+ include/linux/fs.h |    1 +
+ mm/filemap.c       |    3 +++
+ 2 files changed, 4 insertions(+)
+
+--- linux-2.4.20-b_llpmd-l24/include/linux/fs.h~removepage-2.4.20      2003-09-05 11:45:42.000000000 -0700
++++ linux-2.4.20-b_llpmd-l24-zab/include/linux/fs.h    2003-09-05 11:46:25.000000000 -0700
+@@ -402,6 +402,7 @@ struct address_space_operations {
+       int (*releasepage) (struct page *, int);
+ #define KERNEL_HAS_O_DIRECT /* this is for modules out of the kernel */
+       int (*direct_IO)(int, struct inode *, struct kiobuf *, unsigned long, int);
++      void (*removepage)(struct page *); /* called when page gets removed from the inode */
+ };
+ 
+ struct address_space {
+--- linux-2.4.20-b_llpmd-l24/mm/filemap.c~removepage-2.4.20    2003-09-05 11:45:42.000000000 -0700
++++ linux-2.4.20-b_llpmd-l24-zab/mm/filemap.c  2003-09-05 11:46:25.000000000 -0700
+@@ -95,6 +95,9 @@ static inline void remove_page_from_inod
+ {
+       struct address_space * mapping = page->mapping;
+ 
++      if (mapping->a_ops->removepage)
++              mapping->a_ops->removepage(page);
++      
+       mapping->nrpages--;
+       list_del(&page->list);
+       page->mapping = NULL;
+
+_

diff --git a/lustre/kernel_patches/pc/removepage-2.4.20.pc b/lustre/kernel_patches/pc/removepage-2.4.20.pc
new file mode 100644 (file)
index 0000000..c659e15
--- /dev/null
+++ b/lustre/kernel_patches/pc/removepage-2.4.20.pc
@@ -0,0 +1,2 @@
+include/linux/fs.h
+mm/filemap.c

diff --git a/lustre/osc/osc_create.c b/lustre/osc/osc_create.c
index 3fee907..3fb9d08 100644 (file)
--- a/lustre/osc/osc_create.c
+++ b/lustre/osc/osc_create.c
@@ -49,7 +49,6 @@
 
 #include <linux/kp30.h>
 #include <linux/lustre_mds.h> /* for mds_objid */
-#include <linux/lustre_otree.h>
 #include <linux/obd_ost.h>
 #include <linux/lustre_commit_confd.h>
 #include <linux/obd_lov.h>