Whamcloud - gitweb
adilger [Mon, 8 Apr 2002 20:50:06 +0000 (20:50 +0000)]
Fix typo "lusre" -> "lustre"
braam [Mon, 8 Apr 2002 18:28:50 +0000 (18:28 +0000)]
- name switch: "light" --> "lite"
"/mnt/obd" --> "/mnt/lustre"
- small changes to RPC code to handle request packet freeing later,
separate connection initialization and connect.
- rename event handlers and queues, add separate request_out and
reply_out queue
pschwan [Sat, 6 Apr 2002 17:38:35 +0000 (17:38 +0000)]
Removes all traces of mds_req, mds_rep, ost_req, and ost_rep. All subsystems
now use the functions in rpc/pack_generic.c, which are based around
struct lustre_msg. That file is small and easily understood, see it for more
details.
Now that everyone uses the same functions, the code can probably be refactored
more sensibly. This is left as an exercise for the reader.
Other, cleanups:
- we don't need to memset() after OBD_ALLOC
- the MDS and OST request and reply packets were so similar that they've been
merged, into struct mds_body and ost_body respectively.
adilger [Fri, 5 Apr 2002 20:29:39 +0000 (20:29 +0000)]
Ensure that the file type is valid.
adilger [Fri, 5 Apr 2002 20:29:12 +0000 (20:29 +0000)]
Remove use of ext2-specific inode flags to signal objid-in-i_data. Just
use the check whether it is a regular file.
Also don't use memcpy when saving objid into inode - it should be endian
converted beforehand.
adilger [Fri, 5 Apr 2002 20:26:22 +0000 (20:26 +0000)]
Ensure we undo everything in ll_read_super() on error.
adilger [Fri, 5 Apr 2002 20:13:53 +0000 (20:13 +0000)]
Add a check in new_fs() to try and avoid overwriting an in-use block device.
Not perfect, because it will only check for mounted filesystems, but better
than nothing.
adilger [Fri, 5 Apr 2002 20:06:45 +0000 (20:06 +0000)]
Add open/close methods for filterobd.
Other minor cleanups to replace "ext2obd" with "filter" in unused codepaths.
pschwan [Thu, 4 Apr 2002 21:01:46 +0000 (21:01 +0000)]
- Don't start acceptor in the background anymore
pschwan [Thu, 4 Apr 2002 18:07:17 +0000 (18:07 +0000)]
- Fixed lock tests--early indicates are that extents are working
adilger [Thu, 4 Apr 2002 09:25:29 +0000 (09:25 +0000)]
Add journal wrapping calls to all of the mds_reint operations. We also
use the mds_fs_setattr method to change the fields in the inode, as this
will properly mark the inode dirty and ensure it is written to disk.
This fixes the problem of updates to the MDS inode not being written to
disk because the dentry and inode are being discarded too quickly.
The journal wrapping of all reint operations will make the addition of
calls to atomically update last recieved trivial, so it is time well spent.
pschwan [Thu, 4 Apr 2002 04:22:42 +0000 (04:22 +0000)]
- Don't abort if fig2dev doesn't exist
- first chunk of DLM changes for new extents
- fixups to work with the new Portals tip
adilger [Wed, 3 Apr 2002 10:48:24 +0000 (10:48 +0000)]
Add skeleton definition for struct obd_run_ctxt to quiet compiler warning.
adilger [Wed, 3 Apr 2002 10:02:33 +0000 (10:02 +0000)]
Because we now return an error code from filter_prep() we have to make sure
we return 0 if we made it through OK...
adilger [Wed, 3 Apr 2002 09:57:59 +0000 (09:57 +0000)]
The start of moving the MDS root out of the filesystem root. A bit of
code reorg, and we now create ROOT and FH directories (no RCVD file yet).
Also moves MOD_INC_USE_COUNT earlier in mds_setup to avoid races.
adilger [Wed, 3 Apr 2002 08:37:25 +0000 (08:37 +0000)]
Clean up strange loop variable usage in filter_preprw() and filter_commitrw(),
and make variables more local.
adilger [Wed, 3 Apr 2002 08:08:26 +0000 (08:08 +0000)]
Have filter_prep() return an error code, and minor cleanups.
Move MOD_INC_USE_COUNT earlier in filter_setup() to avoid module unload races.
adilger [Wed, 3 Apr 2002 07:57:25 +0000 (07:57 +0000)]
Remove simple_mkdir(), push_ctxt(), and pop_ctxt() from filterobd.c and
put them in lib/simple.c so we can use them in MDS. Add simple.c to
files for obdfilter.
adilger [Wed, 3 Apr 2002 07:34:32 +0000 (07:34 +0000)]
Add .eps files to ignored files for doc/ directory.
adilger [Wed, 3 Apr 2002 07:31:18 +0000 (07:31 +0000)]
You don't need to specify the size for new_fs() if you are using a block
device (we just use the whole block device in that case).
adilger [Wed, 3 Apr 2002 07:28:55 +0000 (07:28 +0000)]
Make some attempt to clean up ptlrpc_init_svc() if there are errors setting up
the ring buffer.
adilger [Wed, 3 Apr 2002 07:26:49 +0000 (07:26 +0000)]
Move MOD_INC_USE_COUNT up in ost_setup() to avoid module unload races.
adilger [Wed, 3 Apr 2002 00:50:06 +0000 (00:50 +0000)]
Add push_ctxt and pop_ctxt to simple.c from filter.c.
adilger [Wed, 3 Apr 2002 00:45:32 +0000 (00:45 +0000)]
Move simple_mkdir() out of filterobd so that MDS can use it also.
braam [Tue, 2 Apr 2002 20:58:26 +0000 (20:58 +0000)]
- update a 3 year old message
- split the failing (uml) part of the io-failure mechanism into a separate patch
adilger [Tue, 2 Apr 2002 19:40:44 +0000 (19:40 +0000)]
Fix the setattr and rename error messages and turn off shell verbosity.
adilger [Tue, 2 Apr 2002 19:30:13 +0000 (19:30 +0000)]
Add checks into MDS failure test to verify operations failed.
braam [Tue, 2 Apr 2002 19:29:19 +0000 (19:29 +0000)]
- this Makefile.am re-builds the eps pictures and the master pdf document
braam [Tue, 2 Apr 2002 19:26:24 +0000 (19:26 +0000)]
add missing close to filter obd
adilger [Tue, 2 Apr 2002 19:23:14 +0000 (19:23 +0000)]
Re-enable writes only after MDS has been shut down.
adilger [Tue, 2 Apr 2002 19:13:25 +0000 (19:13 +0000)]
A script which sets up an ext3 testbed (ala llext3.sh), and then runs some
failure tests (setattr, create, unlink, rename) that would write to disk.
braam [Tue, 2 Apr 2002 18:45:47 +0000 (18:45 +0000)]
- add disk write instrumentation to 2.4.18 patch
pschwan [Tue, 2 Apr 2002 17:48:06 +0000 (17:48 +0000)]
- lustre_mds.h: just shrink_dcache on de
- file.c: reverse the order of the MDS/OST open RPCs
- runfailure-net: do unmount/mount after each test
braam [Tue, 2 Apr 2002 16:35:37 +0000 (16:35 +0000)]
- forgot to put the parent...
braam [Tue, 2 Apr 2002 16:34:01 +0000 (16:34 +0000)]
- second part of shrink fix
braam [Tue, 2 Apr 2002 16:33:31 +0000 (16:33 +0000)]
- part one of prune fix.
braam [Tue, 2 Apr 2002 06:13:55 +0000 (06:13 +0000)]
- add a target that automatically builds the pdf document.
- updates to the Lustre HOWTO, still incomplete
- include lock manager in master.
pschwan [Tue, 2 Apr 2002 02:23:45 +0000 (02:23 +0000)]
- runfailure-net fixes:
- create /dev/request
- the open() test needs to actually do an open()
- don't use "run 0 foo" to avoid 'No such pid' error
pschwan [Tue, 2 Apr 2002 00:03:35 +0000 (00:03 +0000)]
- fixed up lustre_mds.h to allow inclusion in userspace testreq.c
- added ptlrpc_abort_bulk() to help cleanup aborted requests
- changed giant obd_support.h enum into #defines, which will save me hours of
counting.
- fixed the ioctl interface to the MDC, so testreq works again
- gave testreq a real interface, to just run one test at a time
- ptlrpc_abort() was freeing a variable which would almost always still be NULL.
Fixed.
adilger [Mon, 1 Apr 2002 05:55:19 +0000 (05:55 +0000)]
Make the failure testing readonly stuff conditionally compiled based on
whether the kernel has been patched with support for it or not.
braam [Sun, 31 Mar 2002 04:31:55 +0000 (04:31 +0000)]
- fixes to filter obd
- move inode size update to file close time.
adilger [Fri, 29 Mar 2002 22:29:59 +0000 (22:29 +0000)]
A bunch of minor cleanups when using sizeof(). This even appears to have
caught 3 minor bugs in the code. Prompted by a couple of similar bugs that
were just found in the kernel.
Basically, with a variable like "struct foo *bar" it is preferrable to
use "sizeof(*bar)" everywhere instead of "sizeof(struct foo)". The
former isolates us from errors in case the type of "bad" ever changes.
Apparent bugs were in filter/filter_obd.c and ldlm/ldlm_resource.c (we
were using __u32 when the variable was declared as __u64). There also
appears to be a memory leak in ldlm_resource.c.
adilger [Fri, 29 Mar 2002 22:20:34 +0000 (22:20 +0000)]
Add new simple test script "runtests" which automates a bunch of the basic
tests which I do before committing stuff to CVS (copy some files, check
they are OK, shutdown+restart, check files again, etc).
Add support in "new_fs" for using pre-built filesystems for testing (this
is considerably faster than building a new fs each time on loopback).
braam [Fri, 29 Mar 2002 22:13:07 +0000 (22:13 +0000)]
- add obdfilter to our setup, cleanup and debug environment
- fix tiny bug in filter.c
adilger [Fri, 29 Mar 2002 22:10:01 +0000 (22:10 +0000)]
Clear the readonly state flags when we initially set up MDS and when we exit.
adilger [Fri, 29 Mar 2002 22:07:03 +0000 (22:07 +0000)]
Remove READPAGE_BULK_NET (unused). Without this minor change, the error
codes at the end of runfailure-net script are wrong.
It might be nice in the future to have a name->number conversion tool
which could always get this correct, but it isn't a big deal yet.
braam [Fri, 29 Mar 2002 15:19:40 +0000 (15:19 +0000)]
- add more infrastructure to handle extents and debug.
- some of this is going to change dramatically as we feel we
are handling too many resources now.
adilger [Fri, 29 Mar 2002 13:40:39 +0000 (13:40 +0000)]
Simple script to test all of the network packet loss scenarios.
adilger [Fri, 29 Mar 2002 12:57:46 +0000 (12:57 +0000)]
Patch to allow disabling devices for testing failure/recovery scenarios.
By itself this patch does not enable failure testing, but when it is
applied to the kernel, the OBD_FAIL_WRITE() call will activate it.
Patch currently supports loopback, IDE, and UML ubd failures (ignore patch
reject for ubc.c if not using UML).
Enable MDS write failure testing via "echo <num> > /proc/sys/lustre/fail_loc"
where <num> is the number of a write failure breakpoint (OBD_FAIL_MDS_*_WRITE
as defined in the enum in obd_support.h).
adilger [Fri, 29 Mar 2002 12:32:47 +0000 (12:32 +0000)]
Reinstate the delete_inode methods. We now make a copy of the superblock
operations table for the MDS superblock so we don't affect other users of
that client filesystem type.
I also removed a duplicate call to rpc_unregister_service() in mds_cleanup.
This is what was causing the "rpc unable to unregister 12" type messages at
cleanup.
adilger [Fri, 29 Mar 2002 12:02:00 +0000 (12:02 +0000)]
Set a flag in the inode if we have stored the objid there, so we know when
we should clear it. Also make clear_inode() the default cl_delete_inode
method so we don't have to do an extra check for it.
In the end, I think that the objid issue is not a source of any problems,
because ext3_delete() will only call ext3_truncate() if i_blocks is non-zero,
and it isn't (this is why we weren't seeing any errors from ext3 trying to
truncate "blocks" that are not allocated).
adilger [Fri, 29 Mar 2002 08:51:04 +0000 (08:51 +0000)]
Disable the delete_inode overloading for now. It overloads the methods
for _all_ users of that filesystem type, which is obviously bad.
adilger [Fri, 29 Mar 2002 08:28:39 +0000 (08:28 +0000)]
All of the "hack" and FIXME stuff in MDS is now handled by fs-specific methods
which are selected at mount time. This includes journaling support.
Changes OBD_FAIL_WRITE() to include the block device which should be
disabled. We conditionally compile the "make device readonly" support.
adilger [Fri, 29 Mar 2002 07:07:02 +0000 (07:07 +0000)]
Add definitions of the MDS client filesystem methods to headers and Makefile.
adilger [Fri, 29 Mar 2002 06:58:54 +0000 (06:58 +0000)]
Add in a delete_inode method for use when the inode is finally unlinked,
so that we can clear out the objid if necessary. Since delete_inode
is handled by a callback from the VFS when n_link == 0 (and we don't
want to introduce race conditions by checking this at unlink time), we
replace the client delete_inode method with a filesystem-specific MDS
delete_inode call, and then call the client delete_inode method from
within our MDS delete_inode operation.
braam [Thu, 28 Mar 2002 05:33:38 +0000 (05:33 +0000)]
- Replace the complicated waitq story with a much less complicated
semaphore
- Move semaphore after allocation
braam [Thu, 28 Mar 2002 03:57:03 +0000 (03:57 +0000)]
Try using add_wait_queue_exclusive to avoid waking all (x - 32)
suspended request threads at once.
braam [Thu, 28 Mar 2002 03:42:42 +0000 (03:42 +0000)]
- Added a queue length and waitqueue to the client structure in an
attempt to limit the number of concurrent requests outstanding at a
given time.
adilger [Thu, 28 Mar 2002 00:00:17 +0000 (00:00 +0000)]
Move mds_null.c back to mds_ext2.c - we have more than just the journaling
methods now (setattr, set_objid, get_objid, readpage), which extracts all
of the filesystem-specific hacks out of MDS. We now have files with a very
high-density hacks for ext2 and ext3, but at least we can clean it up easily.
braam [Wed, 27 Mar 2002 23:35:45 +0000 (23:35 +0000)]
There was a race between ll_unlink and ll_create_node: notably,
ll_unlink would make the inode available on the MDS before the local
dentry/inode on the client was put away. This could lead to re-using
inode numbers with a concurrent create going on.
We fixed this using the generation number. We also added the
inode/generation to unlink as a sanity check.
braam [Wed, 27 Mar 2002 22:54:23 +0000 (22:54 +0000)]
- added a find_inode function that iget4() will use to compare inodes.
find_inode simply compares generation number with what we get from MDS
- set rep->generation in mds_reint.c:mds_reint_create()
adilger [Wed, 27 Mar 2002 22:44:32 +0000 (22:44 +0000)]
Actually mount the filesystem in llmount.sh.
adilger [Wed, 27 Mar 2002 22:43:02 +0000 (22:43 +0000)]
The previously checked-in empty filesystem was dirty for some reason.
braam [Wed, 27 Mar 2002 22:04:59 +0000 (22:04 +0000)]
Increase MDS and OST request buffers to 128k each
pschwan [Wed, 27 Mar 2002 20:37:48 +0000 (20:37 +0000)]
Various llmount-client/server fixes
braam [Wed, 27 Mar 2002 20:36:24 +0000 (20:36 +0000)]
More 'rpc_register_service' removal
braam [Wed, 27 Mar 2002 20:31:22 +0000 (20:31 +0000)]
- simplified ptlrpc_service by using ME auto-unlinking
- merged ptlrpc_init_srv and rpc_register_service
adilger [Wed, 27 Mar 2002 00:46:29 +0000 (00:46 +0000)]
Simple journal abstractions for the no-journal (ext2) and ext3 cases.
adilger [Tue, 26 Mar 2002 21:14:06 +0000 (21:14 +0000)]
Empty filesystems for creating tests. Using these is _much_ faster than
using "dd" to create an empty file and then mkfs to create the filesystem.
They are also highly compressed (1000:1) so they don't use much space.
The new_fs() helper in common.sh will use these files instead of dd+mkfs
if they match the parameters given to new_fs().
adilger [Tue, 26 Mar 2002 20:57:40 +0000 (20:57 +0000)]
Set an initial value for return code.
adilger [Tue, 26 Mar 2002 20:56:59 +0000 (20:56 +0000)]
Don't dereference "lock" before it is set.
braam [Mon, 25 Mar 2002 09:21:13 +0000 (09:21 +0000)]
remove gid/rid's; minor cleanups.
pschwan [Mon, 25 Mar 2002 01:56:42 +0000 (01:56 +0000)]
- default to 5G
- run with -i 0 -i 1
pschwan [Mon, 25 Mar 2002 01:55:49 +0000 (01:55 +0000)]
- build truncate with LARGEFILE64_SOURCE
- truncate's second argument is now in KB
braam [Sun, 24 Mar 2002 23:36:24 +0000 (23:36 +0000)]
*** empty log message ***
braam [Sun, 24 Mar 2002 23:23:27 +0000 (23:23 +0000)]
- more of the locking infrastructure.
- BUG -> LBUG to work with the crashdump tools
pschwan [Sun, 24 Mar 2002 08:59:53 +0000 (08:59 +0000)]
- ll_file_open was not passing the correct file->f_flags to mdc_open. Fixed.
braam [Sat, 23 Mar 2002 08:31:50 +0000 (08:31 +0000)]
- don't read the page if we're just about to completely overwrite it
- fixed one of the page invariant BUGs
adilger [Sat, 23 Mar 2002 00:50:00 +0000 (00:50 +0000)]
Commit changes to use new setup_portals helper function.
adilger [Sat, 23 Mar 2002 00:33:13 +0000 (00:33 +0000)]
mds/handler.c, mds/mds_reint.c: fixup the ext3 MDS code a bit to build on
the older 2.4.9 kernel (i_blkbits does not exist there), and add
in the missing fixup for i_diskblocks
ALL: add in test framework for simulating errors, dropped packets, etc.
does not yet integrate with the ext3 on-disk error interface
braam [Sat, 23 Mar 2002 00:15:23 +0000 (00:15 +0000)]
Remove yet another extraneous BUG()
adilger [Sat, 23 Mar 2002 00:03:06 +0000 (00:03 +0000)]
Remove annoying output from the shell when acceptor is stopped.
adilger [Fri, 22 Mar 2002 23:56:09 +0000 (23:56 +0000)]
Fixup logic a bit.
adilger [Fri, 22 Mar 2002 23:52:08 +0000 (23:52 +0000)]
Move portals setup into a helper function "setup_portals" that uses the
enviroment variables "NETWORK={elan,tcp}", LOCALHOST, SERVER, and PORT.
adilger [Fri, 22 Mar 2002 23:45:55 +0000 (23:45 +0000)]
Simple script to run vmstat and prepend the time, so we can see when vmstat
is not being scheduled instead of thinking that something is strange.
pschwan [Fri, 22 Mar 2002 23:12:12 +0000 (23:12 +0000)]
Remove silly BUG() that catches "erroneous" writes above 4G.
adilger [Fri, 22 Mar 2002 21:28:27 +0000 (21:28 +0000)]
The ext3 MDS code. Basically, we have to hack around the fact that ext3
does not have the directories in page cache, so we need to export the
ext3_bread() function in order to ensure we read the same data as what the
filesystem itself is using. This will likely go away at some time in the
future as ext3 is ported to the pure page-cache-based directory handling
that is in ext2 (which is slightly more efficient).
pschwan [Fri, 22 Mar 2002 20:18:09 +0000 (20:18 +0000)]
use relative paths to insert the modules relative to the test script
pschwan [Fri, 22 Mar 2002 19:33:12 +0000 (19:33 +0000)]
- added non-fatal qswnal insertion to common.sh
- fixed up various scripts to use the common.sh variables
adilger [Fri, 22 Mar 2002 19:02:28 +0000 (19:02 +0000)]
Script to use an existing ext3 MDS setup. Useful for testing recoverability
of an MDS setup with llext3.sh.
adilger [Fri, 22 Mar 2002 19:01:02 +0000 (19:01 +0000)]
Script to run iozone in a loop. It probably doesn't have the right iozone
parameters that people want, but it is a start.
adilger [Fri, 22 Mar 2002 18:59:15 +0000 (18:59 +0000)]
Update rundbench script for new debug control file.
adilger [Fri, 22 Mar 2002 18:57:56 +0000 (18:57 +0000)]
Update ostreq.sh script to use new helper functions.
adilger [Fri, 22 Mar 2002 18:56:49 +0000 (18:56 +0000)]
Minor cleanup.
adilger [Fri, 22 Mar 2002 18:55:44 +0000 (18:55 +0000)]
The loopback device helper will use the next unused loopback device instead
of just starting at minor 0.
pschwan [Fri, 22 Mar 2002 18:54:22 +0000 (18:54 +0000)]
- Take advantage of new disconnect semantics to make llmountcleanup.sh a more
general script
adilger [Fri, 22 Mar 2002 18:52:06 +0000 (18:52 +0000)]
Add ext3_bread to exported symbols. Needed because ext3 does not have
directories in the page cache yet.
pschwan [Fri, 22 Mar 2002 18:31:34 +0000 (18:31 +0000)]
- Removed an overzealous BUG() that keeps us from cleaning up after C-c
adilger [Fri, 22 Mar 2002 05:45:52 +0000 (05:45 +0000)]
Indenting only.
braam [Thu, 21 Mar 2002 20:06:01 +0000 (20:06 +0000)]
- Added UUID description to network document
- fixed leaked refcount in failed osc_request startup
- small debugging message fix
- removed extraneous PtlMDUnlink
- try to remove kqswnal in llmountcleanup