2005-03-31 Cluster File Systems, Inc. * version 1.4.2 NOTE: Lustre 1.4.2 uses an incompatible network protocol than previous versions of Lustre. Please update all servers and clients to version 1.4.2 or later at the same time. You must also run "lconf --write-conf {config}.xml" on the MDS while it is stopped to update the configuration logs. * bug fixes - fix for HPUX NFS client breakage when NFS exporting Lustre (5781) - mdc_enqueue does not need max_mds_easize request buffer on send (5707) - swab llog records of type '0' so we get proper header size/idx (5861) - send llog cancel req to DLM cancel portal instead of cb portal (5515) - fix rename of one directory over another leaking an inode (5953) - avoid SetPageDirty on 2.6 (5981) - don't re-add just-being-destroyed locks to the waiting list (5653) - when creating new directories, inherit the parent's custom striping settings if present parent (3048) * miscellania - by default create 1 inode per 4kB space on MDS, per 16kB on OSTs - allow --write-conf on an MDS with different nettype than client (5619) - don't write config llogs to MDS for mounts not from that MDS (5617) - lconf should create multiple TCP connections from a client (5201) - init scripts are now turned off by default; run chkconfig --on lustre and chkconfig --on lustrefs to use them - upcalls are no longer needed for clients to recover to failover servers (3262) - add --abort-recovery option to lconf to abort recovery on device startup (6017) 2005-03-22 Cluster File Systems, Inc. * version 1.4.1 * bug fixes - don't LASSERT in ll_release on NULL lld with NFS export (4655, 5760) - hold NS lock when calling handle_ast_error->del_waiting_lock (5746) - fix setattr mtime regression from lovcleanup merge (4829, 5669) - workaround for 2.6 crash in ll_unhash_aliases (5687, 5210) - small ext3 extents cleanups and fixes (5733) - improved mballoc code, several small races and bugs fixed (5733, 5638) - kernel version 43 - fix remove_suid bugs in both 2.4 and 2.6 (5695) - avoid needless client->OST connect, fix handle mismatch (5317) - fix DLM error path that led to out-of-sync client, long delays (5779) - support common vfs-enforced mount options (nodev,nosuid,noexec) (5637) - fix several locking issues related to i_size (5492,5624,5654,5672) - don't move pending lock onto export if it is already evicted (5683) - fix kernel oops when creating .foo in unlinked directory (5548) - fix deadlock in obdfilter statistics vs. object create (5811) - use time_{before,after} to avoid timer jiffies wrap (5882) - shutdown --force/--failover stability (3607,3651,4797,5203,4834) - Do not leak request if server was not able to process it (5154) - If mds_open unable to find parent dir, make that negative lookup(5154) - don't create new directories with extent-mapping (5909, 5936) * miscellania - fix lustre/lustrefs init scripts for SuSE (patch from Scali, 5702) - don't hold the pinger_sem in ptlrpc_pinger_sending_on_import - change obd_increase_kms to obd_adjust_kms (up or down) (5654) - lconf, lmc search both /usr/lib and /usr/lib64 for Python libs (5800) - support for RHEL4 kernel on i686 (5773) - provide error messages when incompatible logs are encountered (5898) 2005-02-18 Cluster File Systems, Inc. * version 1.4.0.10 (1.4.1 release candidate 1) * bug fixes - don't keep a lock reference when lock is not granted (4238) - unsafe list practices (rarely) led to infinite eviction loop (4908) - add per-fs limit of Lustre pages in page cache, avoid OOM (4699) - drop import inflight refcount on signal_completed_replay error (5255) - unlock page after async write error during send (3677) - handle missing objects in filter_preprw_read properly (5265) - no transno return for symlink open, don't save no-trasno open (3440) - don't try to complete elan receive that already failed (4012) - free RPC server reply state on error (5406) - clean up thread from ptlrpc_start_thread() on error (5160) - readahead could read extra page into cache that wasn't ejected (5388) - prevent races in class_attach/setup/cleanup/detach (5260) - don't dereference de->d_inode after l_dput of de (5458) - use "int" for stripe value returned from lock_to_stripe (5544) - mballoc allocation and error-checking fixes in 2.6 (5504) - block device patches to fix I/O request sizes in 2.6 (5482) - look up hostnames for IB nals (5602) - 2.6 changed lock ordering of 2 semaphores, caused deadlock (5654) - don't start multiple acceptors for the same port (5277) - fix incorrect LASSERT in mds_getattr_name (5635) - export a proc file for general "ping" checking (5628) - fix "lfs check" to not block when the MDS is down (5628) * miscellania - service request history (4965) - put {ll,lov,osc}_async_page structs in a single slab (4699) - create an "evict_client" /proc entry on OSTs, like the MDS has - fix mount usage message, return errors per mount(8) (5168) - change grep [] to grep "[]" in tests so they work in more UMLs - fix ppc64/x86_64 spec to use %{_libdir} instead of /usr/lib (5389) - remove ancient LOV_MAGIC_V0 EA support (5047) - add "disk I/Os in flight" and "I/O req time" stats in obdfilter - align r/w RPCs to PTLRPC_MAX_BRW_SIZE boundaries for performance - allow readahead allocations to fail when low on memory (5383) - mmap locking landed again, after considerable improvement (2828) - add get_hostaddr() to lustreDB.py for LDAP support (5459) 2004-11-23 Cluster File Systems, Inc. * version 1.4.0 * bug fixes - send OST transaction number in read/write reply to free req (4966) - don't ASSERT in ptl_send_rpc() if we run out of memory (5119) - lock /proc/sys/portals/routes internal state, avoiding oops (4827) - the watchdog thread now runs as interruptible (5246) - flock/lockf fixes (but it's still disabled, pending 5135) - don't use EXT3 constants in llite code (5094) - memory shortage at startup could cause assertion (5176) * miscellania - reorganization of lov code - single portals codebase - Infiniband NAL - add extents/mballoc support (5025) - direct I/O reads in the obdfilter (4048) - kernel patches from LNXI for 2.6 (bluesmoke, perfctr, mtd, kexec) tbd Cluster File Systems, Inc. * version 1.2.9 * bug fixes - send OST transaction number in read/write reply to free req (4966) - don't ASSERT in ptl_send_rpc() if we run out of memory (5119) - lock /proc/sys/portals/routes internal state, avoiding oops (4827) - the watchdog thread now runs as interruptible (5246) - handle missing objects in filter_preprw_read properly (5265) - unsafe list practices (rarely) led to infinite eviction loop (4908) - drop import inflight refcount on signal_completed_replay error (5255) - unlock page after async write error during send (3677) - return original error code on reconstructed replies (3761) - no transno return for symlink open, don't save no-trasno open (3440) * miscellania - add pid to ldlm debugging output (4922) - bump the watchdog timeouts -- we can't handle 30sec yet - extra debugging for orphan dentry/inode bug (5259) 2004-11-16 Cluster File Systems, Inc. * version 1.2.8 * bug fixes - fix TCP_NODELAY bug, which caused extreme perf regression (5134) - allocate qswnal tx descriptors singly to avoid fragmentation (4504) - don't LBUG on obdo_alloc() failure, use OBD_SLAB_ALLOC() (4800) - fix NULL dereference in /proc/sys/portals/routes (4827) - allow failed mdc_close() operations to be interrupted (4561) - stop precreate on OST before MDS would time out on it (4778) - don't send partial-page writes before EOF from client (4410) - discard client grant for sub-page writes on large-page clients (4520) - don't free dentries not owned by NFS code, check generation (4806) - fix lsm leak if mds_create_objects() fails (4801) - limit debug_daemon file size, always print CERROR messages (4789) - use transno after validating reply (3892) - process timed out requests if import state changes (3754) - update mtime on OST during writes, return in glimpse (4829) - add mkfsoptions to LDAP (4679) - use ->max_readahead method instead of zapping global ra (5039) - don't interrupt __l_wait_event() during strace * miscellania - add software watchdogs to catch hung threads quickly (4941) - make lustrefs init script start after nfs is mounted - fix CWARN/ERROR duplication (4930) - return async write errors to application if possible (2248) - add /proc/sys/portal/memused (bytes allocated by PORTALS_ALLOC) - print NAL number in %x format (4645) - update barely-supported suse-2.4.21-171 series (4842) - support for sles 9 %post scripts - support for building 2.6 kernel-source packages - support for sles km_* packages 2004-10-07 Cluster File Systems, Inc. * version 1.2.7 * bug fixes - ignore -ENOENT errors in osc_destroy (3639) - notify osc create thread that OSC is being cleaned up (4600) - add nettype argument for llmount in #5d in conf-sanity.sh (3936) - reconstruct ost_handle() like mds_handle() (4657) - create a new thread to do import eviction to avoid deadlock (3969) - let lconf resolve symlinked-to devices (4629) - don't unlink "objects" from directory with default EA (4554) - hold socknal file ref over connect in case target is down (4394) - allow more than 32000 subdirectories in a single directory (3244) - fix blocks count for O_DIRECT writes (3751) - OST returns ENOSPC from object create when no space left (4539) - don't send truncate RPC if file size isn't changing (4410) - limit OSC precreate to 1/2 of value OST considers bogus (4778) - bind to privileged port in socknal and tcpnal (3689) * miscellania - rate limit CERROR/CWARN console message to avoid overload (4519) - GETFILEINFO dir ioctl returns LOV EA + MDS stat in 1 call (3327) - basic mmap support (3918) - kernel patch series update from b1_4 (4711) 2004-09-16 Cluster File Systems, Inc. * version 1.2.6 * bug fixes - avoid crash during MDS cleanup with OST shut down (2775) - fix loi_list_lock/oig_lock inversion on interrupted IO (4136) - don't use bad inodes on the MDS (3744) - dynamic object preallocation to improve recovery speed (4236) - don't hold spinlock over lock dumping or change debug flags (4401) - don't zero obd_dev when it is force cleaned (3651) - print grants to console if they go negative (4431) - "lctl deactivate" will stop automatic recovery attempts (3406) - look for existing locks in ldlm_handle_enqueue() (3764) - don't resolve lock handle twice in recovery avoiding race (4401) - revalidate should check working dir is a directory (4134) * miscellania - don't always mark "slow" obdfilter messages as errors (4418) 2004-08-24 Cluster File Systems, Inc. * version 1.2.5 * bug fixes - don't close LustreDB during write_conf until it is done (3860) - fix typo in lconf for_each_profile (3821) - allow dumping logs from multiple threads at one time (3820) - don't allow multiple threads in OSC recovery (3812) - fix debug_size parameters (3864) - fix mds_postrecov to initialize import for llog ctxt (3121) - replace config semaphore with spinlock (3306) - be sure to send a reply for a CANCEL rpc with bad export (3863) - don't allow enqueue to complete on a destroyed export (3822) - down write_lock before checking llog header bitmap (3825) - recover from lock replay timeout (3764) - up llog sem before sending rpc (3652) - reduce ns lock hold times when setting kms (3267) - change a dlm LBUG to LASSERTF, to maybe learn something (4228) - fix NULL deref and obd_dev leak on setup error (3312) - replace some LBUG about llog ops with error handling (3841) - don't match INVALID dentries from d_lookup and spin (3784) - hold dcache_lock while marking dentries INVALID and hashing (4255) - fix invalid assertion in ptlrpc_set_wait (3880) * miscellania - add libwrap support for the TCP acceptor (3996) - add /proc/sys/portals/routes for non-root route listing (3994) - allow setting MDS UUID in .xml (2580) - print the stack of a process that LBUGs (4228) 2004-07-14 Cluster File Systems, Inc. * version 1.2.4 * bug fixes - don't cleanup request in ll_file_open() on failed MDS open (3430) - make sure to unset replay flag from failed open requests (3440) - if default stripe count is 0, use OST count for inode size (3636) - update parent mtime/ctime on client for create/unlink (2611) - drop dentry ref in ext3_add_link from open_connect_dentry (3266) - free recovery state on server during a forced cleanup (3571) - unregister_reply for resent reqs (3063) - loop back devices mounting and status check on 2.6 (3563) - fix resource-creation race that can provoke i_size == 0 (3513) - don't try to use bad inodes returned from MDS/OST fs lookup (3688) - more debugging for page-accounting assertion (3746) - return -ENOENT instead of asserting if ost getattr+unlink race (3558) - avoid deadlock after precreation failure (3758) - fix race and lock order deadlock in orphan handling (3450, 3750) - add validity checks when grabbing inodes from l_ast_data (3599) * miscellania - add /proc/.../recovery_status to obdfilter (3428) - lightweight CDEBUG infrastructure, debug daemon (3668) - change default OSC RPC parameters to be better on small clusters - turn off OST read cache for files smaller than 32MB - install man pages and include them in rpms (3100) - add new init script for (un)mounting lustre filesystems (2593) - run chkconfig in %post for init scripts (3701) - drop scimac NAL (unmaintained) 2004-06-17 Cluster File Systems, Inc. * version 1.2.3 * bug fixes - clean kiobufs before and after use (3485) - strip trailing '/'s before comparing paths with /proc/mounts (3486) - remove assertions to work around "in-flight rpcs" recovery bug (3063) - change init script to fail more clearly if not run as root (1528) - allow clients to reconnect during replay (1742) - fix ns_lock/i_sem lock ordering deadlock for kms update (3477) - don't do DNS lookups on NIDs too small for IP addresses (3442) - re-awaken ptlrpcd if new requests arrive during check_set (3554) - fix cond_resched (3554) - only evict unfinished clients after recovery (3515) - allow bulk resend, prevent data loss (3570) - dynamic ptlrpc request buffer allocation (2102) - don't allow unlinking open directory if it isn't empty (2904) - set MDS/OST threads to umask 0 to not clobber client modes (3359) - remove extraneous obd dereference causing LASSERT failure (3334) - don't use get_cycles() when creating temp. files on the mds (3156) - hold i_sem when setting i_size in ll_extent_lock() (3564) - handle EEXIST for set-stripe, set proper directory name (3336) * miscellania - servers can dump a log evicting a client - lustre.dump_on_timeout=1 - fix ksocknal_fmb_callback() error messages (2918) 2004-05-27 Cluster File Systems, Inc. * version 1.2.2 * bug fixes - don't copy lvb into (possibly NULL) reply on error (2983) - don't deref dentry after dput, don't free lvb on error (2922) - use the kms to determine writeback rpc length (2947) - increment oti_logcookies when osc is inactive (2948) - update client's i_blocks count via lvb messages (2543) - handle intent open/close of special files properly (1557) - mount MDS with errors=remount-ro, like obdfilter (2009) - initialize lock handle to avoid ASSERT on error cleanup (3057) - don't use cancelling-locks' kms values (2947) - use highest lock extent for kms, not last one (2925) - don't dereference ERR_PTR() dentry in error handling path (3107) - fix thread race in portals_debug_dumplog() (3122) - create lprocfs device entries at setup instead of at attach (1519) - common AST error handler, don't evict client on completion race (3145) - zero nameidata in detach_mnt in 2.6 (3118) - verify d_inode after revalidate_special is valid in 2.6 (3116) - use lustre_put_super() to handle zconf unmounts in 2.6 (3064) - initialize RPC timeout timer earlier for 2.6 (3219) - don't dereference NULL reply buffer if mdc_close was never sent (2410) - print nal/nid for unknown nid (3258) - additional checks for oscc recovery before doing precreate (3284) - fix ll_extent_lock() error return code for 64-bit systems (3043) - don't crash in mdc_close for bad permissions on open (3285) - zero i_rdev for non-device files (3147) - clear page->private before handing to FS, better assertion (3119) - tune the read pipeline (3236) - fix incorrect decref of invalidated dentry (2350) - provide read-ahead stats and refine rpc in flight stats (3328) - don't hold journal transaction open across create RPC (3313) - update atime on MDS at close time (3265) - close LDAP connection when recovering to avoid server load (3315) - update iopen-2.6 patch with fixes from 2399,2517,2904 (3301) - don't leak open file on MDS after open resend (3325) - serialize filter_precreate and filter_destroy_precreated (3329) - loop device shouldn't call sync_dev() for nul device (3092) - clear page cache after eviction (2766) - resynchronize MDS->OST in background (2824) - refuse to mount the same filesystem twice on same mountpoint (3394) - allow llmount to create routes for mounting behind routers (3320) - push lock cancellation to blocking thread for glimpse ASTs (3409) - don't call osc_set_data_with_check() for TEST_LOCK matches (3159) - fix rare problem with rename on htree directories (3417) * miscellania - allow default OST striping configuration per directory (1414) - fix compilation for qswnal for 2.6 kernels (3125) - increase maximum number of MDS request buffers for large systems - change liblustreapi to be useful for external progs like lfsck (3098) - increase local configuration timeout for slow disks (3353) - allow configuring ldlm AST timeout - lustre.ldlm_timeout= 2004-03-22 Cluster File Systems, Inc. * version 1.2.1 * bug fixes - fixes for glimpse AST timeouts / incorrectly 0-sized files (2818) - don't overwrite extent policy data in reply if lock was blocked (2901) - drop filter export grants atomically with removal from device (2663) - del obd_self_export from work_list in class_disconnect_exports (2908) - don't LBUG if MDS recovery times out during orphan cleanup (2530) - swab reply message in mdc_close, other PPC fixes (2464) - fix destroying of named logs (2325) - overwrite old logs when running lconf --write_conf (2264) - bump LLOG_CHUNKSIZE to 8k to allow for larger clusters (2306) - fix race in target_handle_connect (2898) - mds_reint_create() should take same inode create lock (2926) - correct journal credits calculated for CANCEL_UNLINK_LOG (2931) - don't close files for self_export to avoid uninitialized obd (2936) - allow MDS with the same name as client node (2939) - hold dentry reference for closed log files for unlink (2325) - reserve space for all logs during transactions (2059) - don't evict page beyond end of stripe extent (2925) - don't oops on a deleted current working directory (2399) - handle hard links to targets without a parent properly (2517) - don't dereference NULL lock when racing during eviction (2867) - don't grow lock extents when lots of conflicting locks (2919) 2004-03-04 Cluster File Systems, Inc. * version 1.2.0 * bug fixes - account for cache space usage on clients to avoid data loss (974) - lfsck support in lustre kernel code (2349) - reduce journal credits needed for BRW writes (2370) - orphan handling to avoid losing space on client/server crashes - ptlrpcd can be blocked, stopping ALL progress (2477) - use lock value blocks to assist in proper KMS, faster stat (1021) - takes i_sem instead of DLM locks internally on obdfilter (2720) - recovery for initial connections (2355) - fixes for mds_cleanup_orphans (1934) - abort_recovery crashes MDS in b_eq (mds_unlink_orphan) (2584) - block all file creations until orphan recovery completes (1901) - client remove rq_connection from request struct (2423) - conf-sanity test_5, proper cleanup in umount log not availale (2640) - recovery timer race (2670) - mdc_close recovey bug (2532) - ptlrpc cleanup bug (2710) - mds timeout on local locks (2588) - namespace lock held during RPCs (2431) - handle interrupted sync write properly (2503) - don't try to handle a message that hasn't been replied to (2699) - client assert failure during cleanup after abort recovery (2701) - leak mdc device after failed mount (2712) - ptlrpc_check_set allows timedout requests to complete (2714) - wait for inflight reqs when ptlrpcd finishes (2710) - make sure unregistered services are removed from the srv_list - reset bulk XID's when resending them (caught by 1138 test) - unregister_bulk after timeout - fix lconf error (2694) - handle write after unfinished setstripe, stripe-only getstripe (2388) - readahead locks pages, leaves pending causing memory pressure (2673) - increase OST request buffers to 4096 on large machines (2729) - fix up permission of existing directories in simple_mkdir (2661) - init deleted item, add assertions ptlrpc_abort_inflight() (2725) - don't assign transno to errored transactions (2742) - don't delete objects on OST if given a bogus objid from MDS (2751) - handle large client PAGE_SIZE readdir on small PAGE_SIZE MDS (2777) - if rq_no_resend, then timeout request after recovery (2432) - fix MDS llog_logid record size, 64-bit array alignment (2733) - don't call usermode_helper from ptlrpcd, DEFAULT upcall (2773) - put magic in mount.lustre data, check for bad/NULL mount data (2529) - MDS recovery shouldn't delete objects that it has given out (2730) - if enqueue arrives after completion, don't clobber LVB (2819) - don't unlock pages twice when trigger_group_io returns error (2814) - don't deref NULL rq_repmsg if ldlm_handle_enqueue failed (2822) - don't write pages to disk if there was an error (1450) - don't ping imports that have recovery disabled (2676) - take buffered bytes into account when balancing socknal conn (2817) - hold a DLM lock over readdir always, use truncate_inode_pages (2706) - reconnect unlink llog connection after MDS reconnects to OST (2816) - remove little-endian swabbing of llog records (1987) - set/limit i_blksize to LL_MAX_BLKSIZE on client (2884) - retry reposting request buffers if they fail (1191) - grow extent at grant time to avoid granting a revoked lock (2809) - lock revoke doesn't evict page if covered by a second lock (2765) - disable VM readahead to avoid reading outside lock extents (2805) * miscellania - return LL_SUPER_MAGIC from statfs for the filesystem type (1972) - updated kernel patches for hp-2.4.20 kernel (2681) 2004-02-07 Cluster File Systems, Inc. * version 1.0.4 * kernel patches - fix truncated write corruption (2366) - fix for failed assertion in iopen_connect_dentry (1792,2517) * bug fixes - don't flag the ptlrpcd thread with PF_MEMALLOC (2636) - ensure len(uuid) < 37 in lmc (1171) - fix ia64 OOPS in llog_test (2255) - zero end of page at obdfilter for partial page writes (2648) - don't leave stale dentries around after renames (bug 2428) - fix timeouts when evicting a client with a single lock held (2642) - set deadline for the initial HELLO message to drain (2634) - print out dotted-quad IP addresses in the socknal (2302) * miscellania - additional debugging for MDS client eviction problem (2443) - fix mkfsoptions support for osts (2603, 2604) 2004-01-27 Cluster File Systems, Inc. * version 1.0.3 * kernel patches - add series for the vanilla 2.6.0 kernel - add series for the vanilla 2.4.24 kernel - add series for a cray x86/64 UL kernel drop - fix xattr patches for the vanilla 2.4.19 series * bug fixes - generate true UUIDs in lmc (1171) - have portals stack dumping break in UML (2466) - avoid bad dchild deref; avoid inum lock w/o creation (2362) - allocate with _NOFS in ldlm to avoid deadlock (1933) - wake callback waiting threads on client eviction (2460) - Add --ptldebug and --subsystem to lmc (1719) - update assertion to allow safe interrupt allocation - set rq_no_resend for cancel requests (2432) - recalculate ptlrpcd timeout after resend (2494) - call vfs_rmdir when removing pending directories (2368) - fix renaming a file to itself (2429) - lmc creates a default one-stripe lov (2454) - expand procfs space to handle large clusters (2326) - increase UML stack to avoid overflow - update lconf's list of debug and subsystem masks - fix lfs find --obd (2510) - /proc tunable for disabling filter read caching (2591) - stop rpm packages from altering slapd.conf (2301) - disable nagle in the socknal under 0conf (2578) - choose mds inode size based on stripe count (2572) - fix kernel-source rpm problems (2516) * miscellania - add --disable-doc to avoid pdf generation (2421) - update documentation, tests, type-os, comments - avoid format warnings on ia64 - remove the TOE NAL - tiny code cleanups by removing unused fields 2004-01-07 Cluster File Systems, Inc. * version 1.0.2 * bug fixes - fix obvious semaphore misuse in as-yet-unused setattr path (2348) - remove the most blatant lies from BUILDING file (2371) - change default debug level to reasonable production setting - reduce client side cache size to reduce cache flush time - reduce max RPCs in flight to avoid unnecessary file fragmentation - make TCP zerocopy and pinger support enabled by default (2476) - sync writes completed after process exits caused crashes (2319) - maintain correct mount count on the MDS (2356) - backout 1557, because 2316 wasn't really fixed - better file I/O statistics gathering in /proc - don't take unnecessary, deadlock-inducing bug in readpage (2383) - another kernel patch to fix zero-copy TCP function export - don't take duplicate lock when processing re-sent getattr (2420) - lctl uses obd_self_export instead of creating new conn (2353) - MDS/OST recovery case which requires object creation asserted (2425) - move lfs from /usr/sbin to /usr/bin in packages - fix race between mds_client_add and mds_client_free (2417) - use kmalloc instead of slabs in portals (2430) - don't create duplicate records when a failover MDS is present (2442) - remove unnecessary mount age check (2332) - don't remove directory inodes from locks prematurely (2451) - don't break if MDS service name is the same as hostname (2103) - fix races in client write RPC generation when cache full (2482) 2003-12-13 Cluster File Systems, Inc. * version 1.0.1 * bug fixes - remove now-unused request->rq_obd (278) - if an allocation fails, print out how much memory we've used (1933) - use PORTAL_SLAB_ALLOC for structures, to get GFP_MEMALLOC (1933) - add the "configurable stack size" patch to most series files (1256) - ability to write large log records, for 100+ OST configs (2306) - fix NULL deref when filter_prep fails (2314) - fix operator precedence error in filter_sync - dynamic allocation of socknal TX descriptors (2315) - fix a missed case in the GFP_MEMALLOC patch, can cause deadlock (2310) - fix gcc 2.96 compilation problem in xattr kernel patch (2294) - ensure that CWARN messages in Portals always get to the syslog - __init/__exit are not for prototype decls (ldlm_init/exit) - x86-64 compile warning fixes - fix gateway LMC keyword conflict (2318) - fix MDS lock inversions in getattr/reint paths (1844) - fix a rare lock re-ordering bug, which caused deadlock (2322) - fix i_sem/journal inversion in fsfilt_ext3_write_record (2306) - DLM race condition prevented some lock evictions (2328) - ENOMEM detection and retry on socknal sends (2230) - use GFP_NOFS throughout Lustre, to combat ENOMEM (2230) - move osc_rpcd into ptlrpc, for use in MDC and others (2329) - protect MDS inode fsdata with stronger locking; fixes assertion (2313) - better error messages when a client is rejected during recovery (1505) - avoid cancelling locks which were never granted, after failure (2330) - fix i_sem/journal inversion in mds_client_add (2333) - fix truncate/getattr lock cycle deadlock (2334) - use rpcd to send close; allows resend after timeout, avoid leak (1897) - fix two rare exit paths which could leak an l_lock() ref (2321) - fencepost error in MDS/OST orphan recovery (2226) - make log record alignment 8 bytes (1988) - lstripe now fails when requested offset > ost_count (2237) - ensure that all kernel series have a complete list.h (1607) - fix crashes in special-file operations (2316) - lctl create/brw OID mismatch, caused by obsolete filter loop (2339) * miscellania - allow configurable automake binary, for testing new versions - small update to the lfs documentation 2003-12-03 Cluster File Systems, Inc. * version 1.0.0 * fix negative export reference count in fsfilt_sync (2312) 2003-12-01 Cluster File Systems, Inc. * release candidate 0.9.1 * bug fixes - orphans are moved into the PENDING directory for possible recovery - replayed opens now open by fid for orphan/rename safety (1042) - last close of an orphan inode generates a transno (683) - chdir() and mount() now pin the directory entry (1020) - avoid CERROR in normal ll_setattr_raw() error case (1500) - discard very old requests without processing them (1502) - remove some common, well-understood CERRORs (1505) - require O_DIRECT I/O to be page-sized to workaround IA64 crash (1609) - clear "grant" flags in OST replies until OST grant code lands (1644) - fix read performance by not clobbering i_blksize on client (1598) - fix __ldlm_handle2lock oops by not dereferencing lock after PUT (1625) - make LRU size a /proc tunable, clears locks when reduced (707) - fix some lprocfs rot that prevented ptlbd from loading (1732) - server locks take references on exports now (1558) - build fixes for 2.4.20-rh trees (1663) - return an error from lov_create if all OSCs are inactive (1751) - fix import levels when a reconnect happens without a timeout (1597) - exit early from mds_open if we get a lookup error (1749) - partial page read at EOF wouldn't wait for disk before sending (1642) - avoid NULL deref in obdfilter when reading page past EOF (1592) - avoid LASSERT in ll_intent_lock if server failed very early (1090) - fix LBUG in ll_it_open_error with rc = -2 (1861) - write/truncate lock inversion (1639) - Don't auto-load obdclass, portals modules during cleanup (1495) - fix timestamps from jumping to "now" (1763) - extra journal assertions (1648) - add an extra multiunlink test (1771) - fix read_record/write_record API (1776) - fix leak of offset_extent, possible incorrect i_size later (1772) - fix lasserts in mis-matched transnos during open-unlink testing (1541) - Debugging for the kqswnal_get_idle_tx problems (1820) - Allow recovery to be attempted multiple times (1536) - Write out MDS last_rcvd file after it is first created (1600) - Fix tx_descriptor leak in failed transmit situations (1827) - ext3 journaling fixes for assertion failure after IO error (1871) - class_export_put() on freed export after completion AST error (1896) - Fix revalidate looping in VFS (1322) - Don't access a freed export during MDS_REINT timeout (1521) - Add open-unlink recovery support on the MDS (1673,1764) - Return an error if no MDS data was read from last_rcvd (1946) - Fix for lookup "." or ".." crash on error (1932,1931,1935) - Don't setup a disk device that doesn't match exported UUID (317) - Reduce bulk RPC timeout to avoid cascading client/OST failures (1845) - avoid committing NULL handle in force close - local.sh is now a one-stripe LOV configuration - POSIX utime.4 -EPERM on FIFO not owned by user (56) - fix ext3 htree duplicate directory entry corruption (1516) - POSIX creat.13, fstat.1, open.18, stat.3 new file atime/mtime (2020) - update to new LOV EA format (2097) - interoperability for different PAGE_SIZE/wordsize (686,1821,1343,2042) 2003-06-15 Phil Schwan * version v0_7 * bug fixes - imports and exports cleanup too early, need refcounts (349, 879, 1045) - per-import/export recovery handling (958, 931, 959) - multiple last-rcvd slots, for serving multiple FSes (949) - connections are again shared between multiple imp/exports (963, 964) - "umount -f" would hang if any requests needed to be sent (393, 978) - avoid pinning large req buffer by copying for queued messages (989) - add "uuid" to "lctl device" command to help upcalls (991) - "open" RPCs with transnos would confuse recovery counters (1037) - do proper endian conversion of all wire messages (288, 340, 891) - remove OST bulk get LBUGs, fix ost_brw_write cleanup (1126) - call waiting locks callback from LDLM recovery thread (1127, 1151) - fix ptlrpc_connection leak in target_handle_connect (1174) - fix import refcounting bug in OST and MDS cleanup (1134) - if an invalid-at-open-time OSC returned before close(), LBUG (1150) - fix very unlikely obd_types race condition (501) - remove osc_open hack for echo_client (1187) - we leaked exports/dlmimps for forcibly disconnected clients (1143) - a failure in read_inode2 leads to deadlock (1139) - cancel ack-locks as soon as transaction is committed (1072) - fix major leaks and crashes in the bulk I/O path (937, 1057) - make sure to commitrw after any preprw to avoid deadlock (1162) - failing to execute a file in a lustre FS would lock inode (1203) - small DEBUG_REQ fix to avoid dereferencing a NULL (1227) - don't ASSERT while cleaning up an incompletely-setup obd (1248) - obd_uuid2tgt would walk off the end of the list (1255) - on IA64 the osc would give portals incorrect bulk size (1258) - fix debug daemon ioctl interface; allows daemon on ia64 (1274) - fix lock inversion caused by new llite matching code (1282) - limit the number of dirty pages on a client to 10MB (1286) - timed out locks were not being corrected cancelled (1289) - fix O_DIRECT above 4GB on IA-32 (1292) * major user-visible changes - fail out/fail over policy now controlled by the upcall (993) * protocol changes - add OBD_PING to check server availability and failure (954) - lustre messages are now sent in sending host order (288, 340, 891) - add eadatalen to MDS getattr reply (340) - OST read replies may contain second buffer, with per-page status (593) 2003-03-11 Phil Schwan * version v0_6 * bug fixes - LDLM_DEBUG macro fix, for gcc 3.2 (850) - failed open()s could cause deadlock; fixed (867, 869) - stop cancelling OST locks when files are closed (481) - overlapping XID spaces caused network corruption (851, 853) - fix unsafe fsfilt counter arithmetic; change to atomic_t - setattr_raw added, to do single-RPC, server-side setattrs - lmc/lconf syntax change for OST UUIDs - fix crashy race condition between ptlrpc_free_req and osc_close - don't use request in mdc_enqueue if we hit a timeout (889) - don't set the inode i_size for regular files from the MDS (896) - handle out of order completion AST (842) - don't LBUG if a lock request times out after receiving AST (913) - avoid d_rehash race in ll_find_alias by rehashing inside dcache_lock - if a bad lock AST arrives, send an error instead of dropping entirely - return 0 from revalidate2 if ll_intent_lock returns -EINTR (912) - fix leak in bulk IO when only partially completed (899, 900, 926) - fix O_DIRECT for ia64 (55) - (almost) eliminate Lustre-kernel-thread effects on load average (722) - C-z after timeout could hang a process forever; fixed (977) * Features - client-side I/O cache (678, 924, 929, 941, 970) * protocol changes - READPAGE and SETATTRs which don't take server-side locks get their own portal 2003-02-11 Phil Schwan * version v0_5_20 * bug fixes - Fix ldlm_lock_match on the MDS to avoid matching remote locks (592) - Fix fsfilt_extN_readpage() to read a full page of directory entries, or fake the remainder if PAGE_SIZE != blocksize (500) - Avoid extra mdc_getattr() in ll_intent_lock when possible (534, 604) - Fix imbalanced LOV object allocation and out-of-bound access (469) - Most intent operations were removed, in favour of a new RPC mode that does a single RPC to the server and bypasses most of the VFS - All LDLM resource ID arrays were removed in favour of ldlm_res_id - Aggressively cancel local locks on DLM servers - mds_reint_unlink sends EA to the client if it's the last nlink. client uses that EA to unlink OST objects. - mds_reint_{rename,unlink,link} were rewritten to take ordered locks - recursive symlinks were fixed (439) - fixed NULL deref in DEBUG_REQ - filter_update_lastobjid no longer calls sync, which annoyed extN - fixed multi-client small-writes to a single file problem (445) - fixed mtime updates during file writes (607) - fixed vector writes on obdfilter causing problems when ENOSPC (670) - fixed bug in obd_brw_read/write() (under guise of testing 367) - fixed Linux OST size reporting problem (444, 656) - OST now updates object mtime with writes or setattr (607, 619) - client verifies file size before zeroing page past EOF (445) - OST now writes last allocated objid to disk with allocation (108) - LOV on echo now works (409) * protocol changes - mds_reint_unlink sends a new buffer, with the EA included. this buffer is only valid if body->valid & OBD_MD_FLEASIZE, which is only set if a regular file was being unlinked, and it was the last link - use PtlGet from the target for bulk writes (315) - OST now updates object mtime with writes or setattr (607, 619) - LDLM now has a grant-time callback to revalidate locked items, if necessary (604) - Many MDS operations were reorganized to combat race conditions * other changes - Merge b_intel branch (updated lprocfs code) - now at /proc/fs/lustre - configure check to avoid gcc version 2.96 20000731-2.96-98) (606) 2003-01-06 Andreas Dilger * version v0_5_19 * bug fixes - Fully reactivate OST imports after reconnection (512, others) - Make sure client sees our -ENOTCONN from mds_handle (513 - partial) - More graceful error handling for truncating on dead OST (515) - Don't error out unless we're actually accessing dead stripes (474) - Fix garbage sizes when stripes are missing (410) - LRU counters were broken, causing constant lock purge (433, 432) - garbage on read from stripes with failed OSTs (441) - mark OSCs as active before reconnecting during recovery (438) - lov_enqueue and lov_cancel need to handle inactive OSTs (403) - lfind did not preserve OST order in output (443) - symlinks cause hung clients, incorrect data (439) - stop dereferencing request after dropping refcount (457) - don't LASSERT(spin_is_locked) on non-SMP (455) - fixes for many rename() bugs - fstat didn't correctly synchronize attributes (399) - server must handle lock cancellation during blocking AST prep (487) - bulk descriptors were free()d too soon (511) - fix paths in lconf, which would load incorrect modules (451, 507) - fix confusing lconf 'host not found' error message (386) - fix lock order deadlock on OST (O/R i_sem before journal ops, 478) - fix race condition in mdc_blocking_ast() for inode access (526) - fix lov_unpackmd() unpacking wrong number of stripes (537) - fix lov_set_osc_active() marking wrong OSC inactive (440) - fix bad lstripe lov_unpackmd() assertion (fix layering too) (527) - fix multiple writes of stripe MD to MDS (358, maybe 519) - fix lstripe in several ways (kernel side) (527) - fix request leak in ldlm_cli_enqueue (262) - incorrect OSC was marked inactive after OST failure - call mds_fs_cleanup before unmounting filesystem (524) - fix races between taking ns_lock and ldlm_lock_change_resource - fix races updating LOV export open file list - fix lov_enqueue error path, avoid decref-ing bad lock handle - fix recovery NULL deref in ldlm_cli_cancel_unused - fix some DLM races by using new hash table for lock handles (419) - permit the client to specify desired inodes, at replay - duplicate requests when we queue them for replay reintegration - fix last_rcvd offset calculation - sync after each recovered transaction, so we always make progress - never, not always, ERESTART requests without transnos - store the lov_desc in the MDS, so we don't depend on getlovinfo to set it - skip replay if the MDS says that the client is already connected - don't check for a recovery-enabled export to match lctl's UUID - don't INC_USE_COUNT for phantom exports - don't crash when cleaning up phantom exports (567) - don't double-finish or set replay data for errored mdc_open requests - abort requests when they time out, so we don't get old replies - send/receive replies for AST messages again - if the client says that it doesn't have the lock, cancel it on the server - if we timeout during I/O, don't try to cancel an in-use lock; instead mark it as destroyed, it will all work out when decref is called - fix module use counts (22, 581) * protocol changes - ASTs now expect a reply (server cancels lock on error reply) 2002-12-02 Andreas Dilger * version v0_5_18 * bug fixes - fix many simultaneous client startup (392) - fix dentry->d_it clobbering - credentials weren't being shipped for readdir/getattr operations - remove invalid assertions triggered during some concurrent MD updates - proper Lustre versions added (336, 389) - fix memory leak for create error case (398) - fix LOV locking bug that would get cli/srv out of sync - fix echo client over LOV (409) - fix dbench 2, extN refcount problem (170, 258, 356, 418) - fix double-O_EXCL intent crash (424) - avoid sending multiple lock CANCELs (352) * Features - MDS can do multi-client recovery (modulo bugs in new code) * Documentation - many updates, edits, cleanups 2002-11-18 Phil Schwan * version v0_5_17 * bug fixes - fix null d_it dereference (346) - fix full OST/dbench hang (333) - fix permission problem with file removal (286) - fix removal of OSCs from LOV when they fail - fix NULL deref during bulk timeout (214) - fix problems related to multiple filesystems on one MDS (241) - fixed serious subtle metadata locking bugs - free locks on clients when inodes are removed due to memory pressure (201) - fix inode pointer in lock data (285) - partial support for multiple MDS on a single host (241) - data locks weren't cancelled at clear_inode time (290, 311) - intent locks could lead to unbounded lock growth (205) - added a maximum lock count, an LRU list, and a flusher - fix multiple rename (365) - properly abstracted the echo client - OSC locked 1 byte too many; fixed - rewrote brw callback code: - fixed recovery bugs related to LOVs (306) - fixed too-many-pages-in-one-write crash (191) - fixed (again) crash in sync_io_timeout (214) - probably fixed callback-related race (385) * protocol change - Add capability to MDS protocol - LDLM cancellations and callbacks on different portals 2002-10-28 Andreas Dilger * version v0_5_16 * bug fixes: - limit client IOV size to PTL_MD_MAX_IOV (611336, 191) - defer open object destruction to close time (601981, 138) - open/close OST file handle in obdo (OBD_MD_FLHANDLE) (601981, 138) - move LDLM_ENQUEUE/CONVERT back to MDS portal (625069) - abstract ll_lookup2, fix ll_revalidate2 to use abstraction (256) - don't call obd_setattr in ll_file_release for destroyed objects * protocol change to lustre_msg: move |version| and add |flags| * protocol change to osc_punch: "start" in "o_size", "end" in "o_blocks" * lock replay: for LDLM_FL_REPLAY trust client to do right thing * added replay of create, unlink, link and rename operations during MDS failover; recovery should be much more robust now * remove failed OSCs from LOVs (only lov_create uses this so far) * the lustre-HOWTO was brought (more) up to date (582544) 2002-10-23 Phil Schwan * version v0_5_15 * bug fixes: - in-use dentries weren't being reused properly (617851) - prevent multiple LDLM setup (599178) - fix LOV size calculations for truncate (617853) - fix client handling of MDS intent errors (POSIX) - fix permission bug in lovstripe.c test (624321) - fix MDS thread deadlock - move LDLM handler to DLM portal (625069) - truncate past end of file could corrupt data - proper cleanup after timeouts, crashes, etc (592524, 550815) - a race in recovery could return ETIMEDOUT to apps (623947) - building outside the source directory was fixed * the lustre-HOWTO was brought (more) up to date (582544) * major progress was made on recovery functionality 2002-10-10 Phil Schwan * version v0_5_14 * bug fixes: - recovery deadlock fix - rm -rf causes LBUG fix (617817) - file open by multiple tasks fix (618962) - directory permissions bugs (602707 and 620007) - journal_stop fixed with locking (611313) - O_APPEND failures resolved (618273, perhaps 614459) - lconf PATH fix (619770) - IA64 build fix (621450) - RPC buffer sizes scale with amount of memory 2002-10-01 Phil Schwan * version v0_5_13 * bug fixes: - locks would be cancelled without throwing away data pages, resulting in inconsistent data (605627) - inode attributes were not always being refreshed (605627, 612449) - lconf now continues to cleanup after lctl reports an error - MDS now enforces user permissions (602707) - lprocfs cleanup fixed, but not yet enabled (614157) - fixed infinite server hang, should a client not respond to an AST - avoid going into recovery if user calls readlink() with a buffer that's too small (613941) - AST RPCs no longer require replies (614867) -- this may be changed - don't crash server if client sends an IOV that's too big (611336) - fixed lock conversion deadlock (611892) - fixed the following of symlinks (614622) * recovery: the server can remove locks from a client that dies, other clients can make progress * more extN patch fixes * compile-time configurable ptlrpc buffer allocations * documentation - collaborative read cache document - Lustre Lite Performance CDR document-in-progress 2002-09-20 Andreas Dilger * version v0_5_12 * bug fix - fix typo in patch-2.4.18 2002-09-20 Andreas Dilger * version v0_5_11 * bug fixes - clear ptlrpc request each time in handle_incoming_request() - unlink of files now destroys the object on the OST 2002-09-19 Peter Braam * version 0_5_10 * add hard link support * change obdfile creation method * kernel patch changed 2002-09-19 Peter Braam * version 0_5_9 * bug fix - stack overflow bug in extN fixed 2002-09-18 Andreas Dilger * version 0_5_8 * documentation updates - add man pages for config tools - update tests/README to describe testing with new config tools - finish metadata API descriptions * bug fixes and cleanups - statfs workaround for 16TB limit - LOV stripe allocation improved, can stripe on subset of OSTs - LOV file size/IO offset was wrong for files > 4GB in size - object EA data was being dropped, caused files to be unreadable - memory overflow with non-LOV OST caused memory corruption - fixed regression tests to work with new config tools, obdfilter - fixed bug when directory size became larger than 1 block - fixed bug (for single client case) when PWD was deleted - invalidate local directory pages when doing intent-based ops - avoid LDLM oops when lock callback contained bad data 2002-09-09 Andreas Dilger * version 0_5_7 * documentation updates * bug fixes and cleanups - configuration tools - LOV - imports/exports - 64-bit compile warnings - 64-bit internal statfs data - many more * test_brw on persistent OST devices * MDS recovery * lprocfs (disabled) 2002-09-04 Andreas Dilger * version 0_5_6 * documentation updates * bug fixes and cleanups * configuration tools 2002-08-30 Peter J. Braam * version v0_5_5 * many small fixes to 0_5_4 * io/network handling * thinkos in MDS operations 2002-08-24 Peter J. Braam * version v0_5_4 * crucial basic fixes to 0.5.3 * IOR, Iozone work over Elan * EOF locks added 2002-08-07 Phil Schwan * version 0_5_3, our first alpha * we use the new Portals iovs * documentation updates * bug fixes and cleanups * small changes in the DLM wire protocol 2002-07-25 Peter J. Braam * version 0_5_1 with some initial stability, * locking on MD and file I/O. * documentation updates * several bug fixes since 0.5.0 * small changes in wire protocol 2002-07-18 Phil Schwan * version v0_4_5 * delivered as Lustre Light Alpha * fixed a crash after handling invalid MDS requests * fixed directory pages for architectures with non-4k pages sizes 2002-07-11 Andreas Dilger * release version v0_4_4 * Moves TCP acceptor to be on port 2432 (unused Coda port) instead of 1234. * Fixes a number of interruption problems with OST operations. * Update documentation for portals header changes * Move all wire protocol structs/defines to lustre_idl.h * Fixes symlink length bug. * Add tcpdump to repository. 2002-07-05 Andreas Dilger * release version v0_4_3 * Fixes statfs for inodes on extN. * Fixes bug in runtests which would delete /etc/hosts. * Use 64-bit object IDs wherever possible (not into VFS though) Remove ost_get_info, which is unused by lustre, and out of date. 2002-07-03 Peter Braam * release version v0_4_2 Fixes a lookup error (type not passed) * move forward to head of Portals * move forward to latest Lustre kernel 2002-06-25 Peter Braam * release version v0_4_1. Hopefully stable on single node use.