1 2005-03-31 Cluster File Systems, Inc. <info@clusterfs.com>
4 - fix deadlock in obdfilter statistics vs. object create (5811)
5 - fix for HPUX NFS client breakage when NFS exporting Lustre (5781)
7 - by default create 1 inode per 4kB space on MDS, per 16kB on OSTs
9 2005-03-18 Cluster File Systems, Inc. <info@clusterfs.com>
12 - don't LASSERT in ll_release on NULL lld with NFS export (4655, 5760)
13 - hold NS lock when calling handle_ast_error->del_waiting_lock (5746)
14 - fix setattr mtime regression from lovcleanup merge (4829, 5669)
15 - workaround for 2.6 crash in ll_unhash_aliases (5687, 5210)
16 - small ext3 extents cleanups and fixes (5733)
17 - improved mballoc code, several small races and bugs fixed (5733, 5638)
18 - kernel version 43 - fix remove_suid bugs in both 2.4 and 2.6 (5695)
19 - avoid needless client->OST connect, fix handle mismatch (5317)
20 - fix DLM error path that led to out-of-sync client, long delays (5779)
21 - support common vfs-enforced mount options (nodev,nosuid,noexec) (5637)
22 - fix several locking issues related to i_size (5492,5624,5654,5672)
23 - don't move pending lock onto export if it is already evicted (5683)
24 - fix kernel oops when creating .foo in unlinked directory (5548)
26 - fix lustre/lustrefs init scripts for SuSE (patch from Scali, 5702)
27 - don't hold the pinger_sem in ptlrpc_pinger_sending_on_import
28 - change obd_increase_kms to obd_adjust_kms (up or down) (5654)
29 - lconf, lmc search both /usr/lib and /usr/lib64 for Python libs (5800)
31 2005-02-18 Cluster File Systems, Inc. <info@clusterfs.com>
32 * version 1.4.0.10 (1.4.1 release candidate 1)
34 - don't keep a lock reference when lock is not granted (4238)
35 - unsafe list practices (rarely) led to infinite eviction loop (4908)
36 - add per-fs limit of Lustre pages in page cache, avoid OOM (4699)
37 - drop import inflight refcount on signal_completed_replay error (5255)
38 - unlock page after async write error during send (3677)
39 - handle missing objects in filter_preprw_read properly (5265)
40 - no transno return for symlink open, don't save no-trasno open (3440)
41 - don't try to complete elan receive that already failed (4012)
42 - free RPC server reply state on error (5406)
43 - clean up thread from ptlrpc_start_thread() on error (5160)
44 - readahead could read extra page into cache that wasn't ejected (5388)
45 - prevent races in class_attach/setup/cleanup/detach (5260)
46 - don't dereference de->d_inode after l_dput of de (5458)
47 - use "int" for stripe value returned from lock_to_stripe (5544)
48 - mballoc allocation and error-checking fixes in 2.6 (5504)
49 - block device patches to fix I/O request sizes in 2.6 (5482)
50 - look up hostnames for IB nals (5602)
51 - 2.6 changed lock ordering of 2 semaphores, caused deadlock (5654)
52 - don't start multiple acceptors for the same port (5277)
53 - fix incorrect LASSERT in mds_getattr_name (5635)
54 - export a proc file for general "ping" checking (5628)
55 - fix "lfs check" to not block when the MDS is down (5628)
57 - service request history (4965)
58 - put {ll,lov,osc}_async_page structs in a single slab (4699)
59 - create an "evict_client" /proc entry on OSTs, like the MDS has
60 - fix mount usage message, return errors per mount(8) (5168)
61 - change grep [] to grep "[]" in tests so they work in more UMLs
62 - fix ppc64/x86_64 spec to use %{_libdir} instead of /usr/lib (5389)
63 - remove ancient LOV_MAGIC_V0 EA support (5047)
64 - add "disk I/Os in flight" and "I/O req time" stats in obdfilter
65 - align r/w RPCs to PTLRPC_MAX_BRW_SIZE boundaries for performance
66 - allow readahead allocations to fail when low on memory (5383)
67 - mmap locking landed again, after considerable improvement (2828)
69 2004-11-23 Cluster File Systems, Inc. <info@clusterfs.com>
72 - send OST transaction number in read/write reply to free req (4966)
73 - don't ASSERT in ptl_send_rpc() if we run out of memory (5119)
74 - lock /proc/sys/portals/routes internal state, avoiding oops (4827)
75 - the watchdog thread now runs as interruptible (5246)
76 - flock/lockf fixes (but it's still disabled, pending 5135)
77 - don't use EXT3 constants in llite code (5094)
78 - memory shortage at startup could cause assertion (5176)
80 - reorganization of lov code
81 - single portals codebase
83 - add extents/mballoc support (5025)
84 - direct I/O reads in the obdfilter (4048)
85 - kernel patches from LNXI for 2.6 (bluesmoke, perfctr, mtd, kexec)
87 tbd Cluster File Systems, Inc. <info@clusterfs.com>
90 - send OST transaction number in read/write reply to free req (4966)
91 - don't ASSERT in ptl_send_rpc() if we run out of memory (5119)
92 - lock /proc/sys/portals/routes internal state, avoiding oops (4827)
93 - the watchdog thread now runs as interruptible (5246)
94 - handle missing objects in filter_preprw_read properly (5265)
95 - unsafe list practices (rarely) led to infinite eviction loop (4908)
96 - drop import inflight refcount on signal_completed_replay error (5255)
97 - unlock page after async write error during send (3677)
98 - return original error code on reconstructed replies (3761)
99 - no transno return for symlink open, don't save no-trasno open (3440)
101 - add pid to ldlm debugging output (4922)
102 - bump the watchdog timeouts -- we can't handle 30sec yet
103 - extra debugging for orphan dentry/inode bug (5259)
105 2004-11-16 Cluster File Systems, Inc. <info@clusterfs.com>
108 - fix TCP_NODELAY bug, which caused extreme perf regression (5134)
109 - allocate qswnal tx descriptors singly to avoid fragmentation (4504)
110 - don't LBUG on obdo_alloc() failure, use OBD_SLAB_ALLOC() (4800)
111 - fix NULL dereference in /proc/sys/portals/routes (4827)
112 - allow failed mdc_close() operations to be interrupted (4561)
113 - stop precreate on OST before MDS would time out on it (4778)
114 - don't send partial-page writes before EOF from client (4410)
115 - discard client grant for sub-page writes on large-page clients (4520)
116 - don't free dentries not owned by NFS code, check generation (4806)
117 - fix lsm leak if mds_create_objects() fails (4801)
118 - limit debug_daemon file size, always print CERROR messages (4789)
119 - use transno after validating reply (3892)
120 - process timed out requests if import state changes (3754)
121 - update mtime on OST during writes, return in glimpse (4829)
122 - add mkfsoptions to LDAP (4679)
123 - use ->max_readahead method instead of zapping global ra (5039)
124 - don't interrupt __l_wait_event() during strace
126 - add software watchdogs to catch hung threads quickly (4941)
127 - make lustrefs init script start after nfs is mounted
128 - fix CWARN/ERROR duplication (4930)
129 - return async write errors to application if possible (2248)
130 - add /proc/sys/portal/memused (bytes allocated by PORTALS_ALLOC)
131 - print NAL number in %x format (4645)
132 - update barely-supported suse-2.4.21-171 series (4842)
133 - support for sles 9 %post scripts
134 - support for building 2.6 kernel-source packages
135 - support for sles km_* packages
137 2004-10-07 Cluster File Systems, Inc. <info@clusterfs.com>
140 - ignore -ENOENT errors in osc_destroy (3639)
141 - notify osc create thread that OSC is being cleaned up (4600)
142 - add nettype argument for llmount in #5d in conf-sanity.sh (3936)
143 - reconstruct ost_handle() like mds_handle() (4657)
144 - create a new thread to do import eviction to avoid deadlock (3969)
145 - let lconf resolve symlinked-to devices (4629)
146 - don't unlink "objects" from directory with default EA (4554)
147 - hold socknal file ref over connect in case target is down (4394)
148 - allow more than 32000 subdirectories in a single directory (3244)
149 - fix blocks count for O_DIRECT writes (3751)
150 - OST returns ENOSPC from object create when no space left (4539)
151 - don't send truncate RPC if file size isn't changing (4410)
152 - limit OSC precreate to 1/2 of value OST considers bogus (4778)
153 - bind to privileged port in socknal and tcpnal (3689)
155 - rate limit CERROR/CWARN console message to avoid overload (4519)
156 - GETFILEINFO dir ioctl returns LOV EA + MDS stat in 1 call (3327)
157 - basic mmap support (3918)
158 - kernel patch series update from b1_4 (4711)
160 2004-09-16 Cluster File Systems, Inc. <info@clusterfs.com>
163 - avoid crash during MDS cleanup with OST shut down (2775)
164 - fix loi_list_lock/oig_lock inversion on interrupted IO (4136)
165 - don't use bad inodes on the MDS (3744)
166 - dynamic object preallocation to improve recovery speed (4236)
167 - don't hold spinlock over lock dumping or change debug flags (4401)
168 - don't zero obd_dev when it is force cleaned (3651)
169 - print grants to console if they go negative (4431)
170 - "lctl deactivate" will stop automatic recovery attempts (3406)
171 - look for existing locks in ldlm_handle_enqueue() (3764)
172 - don't resolve lock handle twice in recovery avoiding race (4401)
173 - revalidate should check working dir is a directory (4134)
175 - don't always mark "slow" obdfilter messages as errors (4418)
177 2004-08-24 Cluster File Systems, Inc. <info@clusterfs.com>
180 - don't close LustreDB during write_conf until it is done (3860)
181 - fix typo in lconf for_each_profile (3821)
182 - allow dumping logs from multiple threads at one time (3820)
183 - don't allow multiple threads in OSC recovery (3812)
184 - fix debug_size parameters (3864)
185 - fix mds_postrecov to initialize import for llog ctxt (3121)
186 - replace config semaphore with spinlock (3306)
187 - be sure to send a reply for a CANCEL rpc with bad export (3863)
188 - don't allow enqueue to complete on a destroyed export (3822)
189 - down write_lock before checking llog header bitmap (3825)
190 - recover from lock replay timeout (3764)
191 - up llog sem before sending rpc (3652)
192 - reduce ns lock hold times when setting kms (3267)
193 - change a dlm LBUG to LASSERTF, to maybe learn something (4228)
194 - fix NULL deref and obd_dev leak on setup error (3312)
195 - replace some LBUG about llog ops with error handling (3841)
196 - don't match INVALID dentries from d_lookup and spin (3784)
197 - hold dcache_lock while marking dentries INVALID and hashing (4255)
198 - fix invalid assertion in ptlrpc_set_wait (3880)
200 - add libwrap support for the TCP acceptor (3996)
201 - add /proc/sys/portals/routes for non-root route listing (3994)
202 - allow setting MDS UUID in .xml (2580)
203 - print the stack of a process that LBUGs (4228)
205 2004-07-14 Cluster File Systems, Inc. <info@clusterfs.com>
208 - don't cleanup request in ll_file_open() on failed MDS open (3430)
209 - make sure to unset replay flag from failed open requests (3440)
210 - if default stripe count is 0, use OST count for inode size (3636)
211 - update parent mtime/ctime on client for create/unlink (2611)
212 - drop dentry ref in ext3_add_link from open_connect_dentry (3266)
213 - free recovery state on server during a forced cleanup (3571)
214 - unregister_reply for resent reqs (3063)
215 - loop back devices mounting and status check on 2.6 (3563)
216 - fix resource-creation race that can provoke i_size == 0 (3513)
217 - don't try to use bad inodes returned from MDS/OST fs lookup (3688)
218 - more debugging for page-accounting assertion (3746)
219 - return -ENOENT instead of asserting if ost getattr+unlink race (3558)
220 - avoid deadlock after precreation failure (3758)
221 - fix race and lock order deadlock in orphan handling (3450, 3750)
222 - add validity checks when grabbing inodes from l_ast_data (3599)
224 - add /proc/.../recovery_status to obdfilter (3428)
225 - lightweight CDEBUG infrastructure, debug daemon (3668)
226 - change default OSC RPC parameters to be better on small clusters
227 - turn off OST read cache for files smaller than 32MB
228 - install man pages and include them in rpms (3100)
229 - add new init script for (un)mounting lustre filesystems (2593)
230 - run chkconfig in %post for init scripts (3701)
231 - drop scimac NAL (unmaintained)
233 2004-06-17 Cluster File Systems, Inc. <info@clusterfs.com>
236 - clean kiobufs before and after use (3485)
237 - strip trailing '/'s before comparing paths with /proc/mounts (3486)
238 - remove assertions to work around "in-flight rpcs" recovery bug (3063)
239 - change init script to fail more clearly if not run as root (1528)
240 - allow clients to reconnect during replay (1742)
241 - fix ns_lock/i_sem lock ordering deadlock for kms update (3477)
242 - don't do DNS lookups on NIDs too small for IP addresses (3442)
243 - re-awaken ptlrpcd if new requests arrive during check_set (3554)
244 - fix cond_resched (3554)
245 - only evict unfinished clients after recovery (3515)
246 - allow bulk resend, prevent data loss (3570)
247 - dynamic ptlrpc request buffer allocation (2102)
248 - don't allow unlinking open directory if it isn't empty (2904)
249 - set MDS/OST threads to umask 0 to not clobber client modes (3359)
250 - remove extraneous obd dereference causing LASSERT failure (3334)
251 - don't use get_cycles() when creating temp. files on the mds (3156)
252 - hold i_sem when setting i_size in ll_extent_lock() (3564)
253 - handle EEXIST for set-stripe, set proper directory name (3336)
255 - servers can dump a log evicting a client - lustre.dump_on_timeout=1
256 - fix ksocknal_fmb_callback() error messages (2918)
258 2004-05-27 Cluster File Systems, Inc. <info@clusterfs.com>
261 - don't copy lvb into (possibly NULL) reply on error (2983)
262 - don't deref dentry after dput, don't free lvb on error (2922)
263 - use the kms to determine writeback rpc length (2947)
264 - increment oti_logcookies when osc is inactive (2948)
265 - update client's i_blocks count via lvb messages (2543)
266 - handle intent open/close of special files properly (1557)
267 - mount MDS with errors=remount-ro, like obdfilter (2009)
268 - initialize lock handle to avoid ASSERT on error cleanup (3057)
269 - don't use cancelling-locks' kms values (2947)
270 - use highest lock extent for kms, not last one (2925)
271 - don't dereference ERR_PTR() dentry in error handling path (3107)
272 - fix thread race in portals_debug_dumplog() (3122)
273 - create lprocfs device entries at setup instead of at attach (1519)
274 - common AST error handler, don't evict client on completion race (3145)
275 - zero nameidata in detach_mnt in 2.6 (3118)
276 - verify d_inode after revalidate_special is valid in 2.6 (3116)
277 - use lustre_put_super() to handle zconf unmounts in 2.6 (3064)
278 - initialize RPC timeout timer earlier for 2.6 (3219)
279 - don't dereference NULL reply buffer if mdc_close was never sent (2410)
280 - print nal/nid for unknown nid (3258)
281 - additional checks for oscc recovery before doing precreate (3284)
282 - fix ll_extent_lock() error return code for 64-bit systems (3043)
283 - don't crash in mdc_close for bad permissions on open (3285)
284 - zero i_rdev for non-device files (3147)
285 - clear page->private before handing to FS, better assertion (3119)
286 - tune the read pipeline (3236)
287 - fix incorrect decref of invalidated dentry (2350)
288 - provide read-ahead stats and refine rpc in flight stats (3328)
289 - don't hold journal transaction open across create RPC (3313)
290 - update atime on MDS at close time (3265)
291 - close LDAP connection when recovering to avoid server load (3315)
292 - update iopen-2.6 patch with fixes from 2399,2517,2904 (3301)
293 - don't leak open file on MDS after open resend (3325)
294 - serialize filter_precreate and filter_destroy_precreated (3329)
295 - loop device shouldn't call sync_dev() for nul device (3092)
296 - clear page cache after eviction (2766)
297 - resynchronize MDS->OST in background (2824)
298 - refuse to mount the same filesystem twice on same mountpoint (3394)
299 - allow llmount to create routes for mounting behind routers (3320)
300 - push lock cancellation to blocking thread for glimpse ASTs (3409)
301 - don't call osc_set_data_with_check() for TEST_LOCK matches (3159)
302 - fix rare problem with rename on htree directories (3417)
304 - allow default OST striping configuration per directory (1414)
305 - fix compilation for qswnal for 2.6 kernels (3125)
306 - increase maximum number of MDS request buffers for large systems
307 - change liblustreapi to be useful for external progs like lfsck (3098)
308 - increase local configuration timeout for slow disks (3353)
309 - allow configuring ldlm AST timeout - lustre.ldlm_timeout=<seconds>
311 2004-03-22 Cluster File Systems, Inc. <info@clusterfs.com>
314 - fixes for glimpse AST timeouts / incorrectly 0-sized files (2818)
315 - don't overwrite extent policy data in reply if lock was blocked (2901)
316 - drop filter export grants atomically with removal from device (2663)
317 - del obd_self_export from work_list in class_disconnect_exports (2908)
318 - don't LBUG if MDS recovery times out during orphan cleanup (2530)
319 - swab reply message in mdc_close, other PPC fixes (2464)
320 - fix destroying of named logs (2325)
321 - overwrite old logs when running lconf --write_conf (2264)
322 - bump LLOG_CHUNKSIZE to 8k to allow for larger clusters (2306)
323 - fix race in target_handle_connect (2898)
324 - mds_reint_create() should take same inode create lock (2926)
325 - correct journal credits calculated for CANCEL_UNLINK_LOG (2931)
326 - don't close files for self_export to avoid uninitialized obd (2936)
327 - allow MDS with the same name as client node (2939)
328 - hold dentry reference for closed log files for unlink (2325)
329 - reserve space for all logs during transactions (2059)
330 - don't evict page beyond end of stripe extent (2925)
331 - don't oops on a deleted current working directory (2399)
332 - handle hard links to targets without a parent properly (2517)
333 - don't dereference NULL lock when racing during eviction (2867)
334 - don't grow lock extents when lots of conflicting locks (2919)
336 2004-03-04 Cluster File Systems, Inc. <info@clusterfs.com>
339 - account for cache space usage on clients to avoid data loss (974)
340 - lfsck support in lustre kernel code (2349)
341 - reduce journal credits needed for BRW writes (2370)
342 - orphan handling to avoid losing space on client/server crashes
343 - ptlrpcd can be blocked, stopping ALL progress (2477)
344 - use lock value blocks to assist in proper KMS, faster stat (1021)
345 - takes i_sem instead of DLM locks internally on obdfilter (2720)
346 - recovery for initial connections (2355)
347 - fixes for mds_cleanup_orphans (1934)
348 - abort_recovery crashes MDS in b_eq (mds_unlink_orphan) (2584)
349 - block all file creations until orphan recovery completes (1901)
350 - client remove rq_connection from request struct (2423)
351 - conf-sanity test_5, proper cleanup in umount log not availale (2640)
352 - recovery timer race (2670)
353 - mdc_close recovey bug (2532)
354 - ptlrpc cleanup bug (2710)
355 - mds timeout on local locks (2588)
356 - namespace lock held during RPCs (2431)
357 - handle interrupted sync write properly (2503)
358 - don't try to handle a message that hasn't been replied to (2699)
359 - client assert failure during cleanup after abort recovery (2701)
360 - leak mdc device after failed mount (2712)
361 - ptlrpc_check_set allows timedout requests to complete (2714)
362 - wait for inflight reqs when ptlrpcd finishes (2710)
363 - make sure unregistered services are removed from the srv_list
364 - reset bulk XID's when resending them (caught by 1138 test)
365 - unregister_bulk after timeout
366 - fix lconf error (2694)
367 - handle write after unfinished setstripe, stripe-only getstripe (2388)
368 - readahead locks pages, leaves pending causing memory pressure (2673)
369 - increase OST request buffers to 4096 on large machines (2729)
370 - fix up permission of existing directories in simple_mkdir (2661)
371 - init deleted item, add assertions ptlrpc_abort_inflight() (2725)
372 - don't assign transno to errored transactions (2742)
373 - don't delete objects on OST if given a bogus objid from MDS (2751)
374 - handle large client PAGE_SIZE readdir on small PAGE_SIZE MDS (2777)
375 - if rq_no_resend, then timeout request after recovery (2432)
376 - fix MDS llog_logid record size, 64-bit array alignment (2733)
377 - don't call usermode_helper from ptlrpcd, DEFAULT upcall (2773)
378 - put magic in mount.lustre data, check for bad/NULL mount data (2529)
379 - MDS recovery shouldn't delete objects that it has given out (2730)
380 - if enqueue arrives after completion, don't clobber LVB (2819)
381 - don't unlock pages twice when trigger_group_io returns error (2814)
382 - don't deref NULL rq_repmsg if ldlm_handle_enqueue failed (2822)
383 - don't write pages to disk if there was an error (1450)
384 - don't ping imports that have recovery disabled (2676)
385 - take buffered bytes into account when balancing socknal conn (2817)
386 - hold a DLM lock over readdir always, use truncate_inode_pages (2706)
387 - reconnect unlink llog connection after MDS reconnects to OST (2816)
388 - remove little-endian swabbing of llog records (1987)
389 - set/limit i_blksize to LL_MAX_BLKSIZE on client (2884)
390 - retry reposting request buffers if they fail (1191)
391 - grow extent at grant time to avoid granting a revoked lock (2809)
392 - lock revoke doesn't evict page if covered by a second lock (2765)
393 - disable VM readahead to avoid reading outside lock extents (2805)
395 - return LL_SUPER_MAGIC from statfs for the filesystem type (1972)
396 - updated kernel patches for hp-2.4.20 kernel (2681)
398 2004-02-07 Cluster File Systems, Inc. <info@clusterfs.com>
401 - fix truncated write corruption (2366)
402 - fix for failed assertion in iopen_connect_dentry (1792,2517)
404 - don't flag the ptlrpcd thread with PF_MEMALLOC (2636)
405 - ensure len(uuid) < 37 in lmc (1171)
406 - fix ia64 OOPS in llog_test (2255)
407 - zero end of page at obdfilter for partial page writes (2648)
408 - don't leave stale dentries around after renames (bug 2428)
409 - fix timeouts when evicting a client with a single lock held (2642)
410 - set deadline for the initial HELLO message to drain (2634)
411 - print out dotted-quad IP addresses in the socknal (2302)
413 - additional debugging for MDS client eviction problem (2443)
414 - fix mkfsoptions support for osts (2603, 2604)
416 2004-01-27 Cluster File Systems, Inc. <info@clusterfs.com>
419 - add series for the vanilla 2.6.0 kernel
420 - add series for the vanilla 2.4.24 kernel
421 - add series for a cray x86/64 UL kernel drop
422 - fix xattr patches for the vanilla 2.4.19 series
424 - generate true UUIDs in lmc (1171)
425 - have portals stack dumping break in UML (2466)
426 - avoid bad dchild deref; avoid inum lock w/o creation (2362)
427 - allocate with _NOFS in ldlm to avoid deadlock (1933)
428 - wake callback waiting threads on client eviction (2460)
429 - Add --ptldebug and --subsystem to lmc (1719)
430 - update assertion to allow safe interrupt allocation
431 - set rq_no_resend for cancel requests (2432)
432 - recalculate ptlrpcd timeout after resend (2494)
433 - call vfs_rmdir when removing pending directories (2368)
434 - fix renaming a file to itself (2429)
435 - lmc creates a default one-stripe lov (2454)
436 - expand procfs space to handle large clusters (2326)
437 - increase UML stack to avoid overflow
438 - update lconf's list of debug and subsystem masks
439 - fix lfs find --obd (2510)
440 - /proc tunable for disabling filter read caching (2591)
441 - stop rpm packages from altering slapd.conf (2301)
442 - disable nagle in the socknal under 0conf (2578)
443 - choose mds inode size based on stripe count (2572)
444 - fix kernel-source rpm problems (2516)
446 - add --disable-doc to avoid pdf generation (2421)
447 - update documentation, tests, type-os, comments
448 - avoid format warnings on ia64
450 - tiny code cleanups by removing unused fields
452 2004-01-07 Cluster File Systems, Inc. <info@clusterfs.com>
455 - fix obvious semaphore misuse in as-yet-unused setattr path (2348)
456 - remove the most blatant lies from BUILDING file (2371)
457 - change default debug level to reasonable production setting
458 - reduce client side cache size to reduce cache flush time
459 - reduce max RPCs in flight to avoid unnecessary file fragmentation
460 - make TCP zerocopy and pinger support enabled by default (2476)
461 - sync writes completed after process exits caused crashes (2319)
462 - maintain correct mount count on the MDS (2356)
463 - backout 1557, because 2316 wasn't really fixed
464 - better file I/O statistics gathering in /proc
465 - don't take unnecessary, deadlock-inducing bug in readpage (2383)
466 - another kernel patch to fix zero-copy TCP function export
467 - don't take duplicate lock when processing re-sent getattr (2420)
468 - lctl uses obd_self_export instead of creating new conn (2353)
469 - MDS/OST recovery case which requires object creation asserted (2425)
470 - move lfs from /usr/sbin to /usr/bin in packages
471 - fix race between mds_client_add and mds_client_free (2417)
472 - use kmalloc instead of slabs in portals (2430)
473 - don't create duplicate records when a failover MDS is present (2442)
474 - remove unnecessary mount age check (2332)
475 - don't remove directory inodes from locks prematurely (2451)
476 - don't break if MDS service name is the same as hostname (2103)
477 - fix races in client write RPC generation when cache full (2482)
479 2003-12-13 Cluster File Systems, Inc. <info@clusterfs.com>
482 - remove now-unused request->rq_obd (278)
483 - if an allocation fails, print out how much memory we've used (1933)
484 - use PORTAL_SLAB_ALLOC for structures, to get GFP_MEMALLOC (1933)
485 - add the "configurable stack size" patch to most series files (1256)
486 - ability to write large log records, for 100+ OST configs (2306)
487 - fix NULL deref when filter_prep fails (2314)
488 - fix operator precedence error in filter_sync
489 - dynamic allocation of socknal TX descriptors (2315)
490 - fix a missed case in the GFP_MEMALLOC patch, can cause deadlock (2310)
491 - fix gcc 2.96 compilation problem in xattr kernel patch (2294)
492 - ensure that CWARN messages in Portals always get to the syslog
493 - __init/__exit are not for prototype decls (ldlm_init/exit)
494 - x86-64 compile warning fixes
495 - fix gateway LMC keyword conflict (2318)
496 - fix MDS lock inversions in getattr/reint paths (1844)
497 - fix a rare lock re-ordering bug, which caused deadlock (2322)
498 - fix i_sem/journal inversion in fsfilt_ext3_write_record (2306)
499 - DLM race condition prevented some lock evictions (2328)
500 - ENOMEM detection and retry on socknal sends (2230)
501 - use GFP_NOFS throughout Lustre, to combat ENOMEM (2230)
502 - move osc_rpcd into ptlrpc, for use in MDC and others (2329)
503 - protect MDS inode fsdata with stronger locking; fixes assertion (2313)
504 - better error messages when a client is rejected during recovery (1505)
505 - avoid cancelling locks which were never granted, after failure (2330)
506 - fix i_sem/journal inversion in mds_client_add (2333)
507 - fix truncate/getattr lock cycle deadlock (2334)
508 - use rpcd to send close; allows resend after timeout, avoid leak (1897)
509 - fix two rare exit paths which could leak an l_lock() ref (2321)
510 - fencepost error in MDS/OST orphan recovery (2226)
511 - make log record alignment 8 bytes (1988)
512 - lstripe now fails when requested offset > ost_count (2237)
513 - ensure that all kernel series have a complete list.h (1607)
514 - fix crashes in special-file operations (2316)
515 - lctl create/brw OID mismatch, caused by obsolete filter loop (2339)
517 - allow configurable automake binary, for testing new versions
518 - small update to the lfs documentation
520 2003-12-03 Cluster File Systems, Inc. <info@clusterfs.com>
522 * fix negative export reference count in fsfilt_sync (2312)
524 2003-12-01 Cluster File Systems, Inc. <info@clusterfs.com>
525 * release candidate 0.9.1
527 - orphans are moved into the PENDING directory for possible recovery
528 - replayed opens now open by fid for orphan/rename safety (1042)
529 - last close of an orphan inode generates a transno (683)
530 - chdir() and mount() now pin the directory entry (1020)
531 - avoid CERROR in normal ll_setattr_raw() error case (1500)
532 - discard very old requests without processing them (1502)
533 - remove some common, well-understood CERRORs (1505)
534 - require O_DIRECT I/O to be page-sized to workaround IA64 crash (1609)
535 - clear "grant" flags in OST replies until OST grant code lands (1644)
536 - fix read performance by not clobbering i_blksize on client (1598)
537 - fix __ldlm_handle2lock oops by not dereferencing lock after PUT (1625)
538 - make LRU size a /proc tunable, clears locks when reduced (707)
539 - fix some lprocfs rot that prevented ptlbd from loading (1732)
540 - server locks take references on exports now (1558)
541 - build fixes for 2.4.20-rh trees (1663)
542 - return an error from lov_create if all OSCs are inactive (1751)
543 - fix import levels when a reconnect happens without a timeout (1597)
544 - exit early from mds_open if we get a lookup error (1749)
545 - partial page read at EOF wouldn't wait for disk before sending (1642)
546 - avoid NULL deref in obdfilter when reading page past EOF (1592)
547 - avoid LASSERT in ll_intent_lock if server failed very early (1090)
548 - fix LBUG in ll_it_open_error with rc = -2 (1861)
549 - write/truncate lock inversion (1639)
550 - Don't auto-load obdclass, portals modules during cleanup (1495)
551 - fix timestamps from jumping to "now" (1763)
552 - extra journal assertions (1648)
553 - add an extra multiunlink test (1771)
554 - fix read_record/write_record API (1776)
555 - fix leak of offset_extent, possible incorrect i_size later (1772)
556 - fix lasserts in mis-matched transnos during open-unlink testing (1541)
557 - Debugging for the kqswnal_get_idle_tx problems (1820)
558 - Allow recovery to be attempted multiple times (1536)
559 - Write out MDS last_rcvd file after it is first created (1600)
560 - Fix tx_descriptor leak in failed transmit situations (1827)
561 - ext3 journaling fixes for assertion failure after IO error (1871)
562 - class_export_put() on freed export after completion AST error (1896)
563 - Fix revalidate looping in VFS (1322)
564 - Don't access a freed export during MDS_REINT timeout (1521)
565 - Add open-unlink recovery support on the MDS (1673,1764)
566 - Return an error if no MDS data was read from last_rcvd (1946)
567 - Fix for lookup "." or ".." crash on error (1932,1931,1935)
568 - Don't setup a disk device that doesn't match exported UUID (317)
569 - Reduce bulk RPC timeout to avoid cascading client/OST failures (1845)
570 - avoid committing NULL handle in force close
571 - local.sh is now a one-stripe LOV configuration
572 - POSIX utime.4 -EPERM on FIFO not owned by user (56)
573 - fix ext3 htree duplicate directory entry corruption (1516)
574 - POSIX creat.13, fstat.1, open.18, stat.3 new file atime/mtime (2020)
575 - update to new LOV EA format (2097)
576 - interoperability for different PAGE_SIZE/wordsize (686,1821,1343,2042)
578 2003-06-15 Phil Schwan <phil@clusterfs.com>
581 - imports and exports cleanup too early, need refcounts (349, 879, 1045)
582 - per-import/export recovery handling (958, 931, 959)
583 - multiple last-rcvd slots, for serving multiple FSes (949)
584 - connections are again shared between multiple imp/exports (963, 964)
585 - "umount -f" would hang if any requests needed to be sent (393, 978)
586 - avoid pinning large req buffer by copying for queued messages (989)
587 - add "uuid" to "lctl device" command to help upcalls (991)
588 - "open" RPCs with transnos would confuse recovery counters (1037)
589 - do proper endian conversion of all wire messages (288, 340, 891)
590 - remove OST bulk get LBUGs, fix ost_brw_write cleanup (1126)
591 - call waiting locks callback from LDLM recovery thread (1127, 1151)
592 - fix ptlrpc_connection leak in target_handle_connect (1174)
593 - fix import refcounting bug in OST and MDS cleanup (1134)
594 - if an invalid-at-open-time OSC returned before close(), LBUG (1150)
595 - fix very unlikely obd_types race condition (501)
596 - remove osc_open hack for echo_client (1187)
597 - we leaked exports/dlmimps for forcibly disconnected clients (1143)
598 - a failure in read_inode2 leads to deadlock (1139)
599 - cancel ack-locks as soon as transaction is committed (1072)
600 - fix major leaks and crashes in the bulk I/O path (937, 1057)
601 - make sure to commitrw after any preprw to avoid deadlock (1162)
602 - failing to execute a file in a lustre FS would lock inode (1203)
603 - small DEBUG_REQ fix to avoid dereferencing a NULL (1227)
604 - don't ASSERT while cleaning up an incompletely-setup obd (1248)
605 - obd_uuid2tgt would walk off the end of the list (1255)
606 - on IA64 the osc would give portals incorrect bulk size (1258)
607 - fix debug daemon ioctl interface; allows daemon on ia64 (1274)
608 - fix lock inversion caused by new llite matching code (1282)
609 - limit the number of dirty pages on a client to 10MB (1286)
610 - timed out locks were not being corrected cancelled (1289)
611 - fix O_DIRECT above 4GB on IA-32 (1292)
612 * major user-visible changes
613 - fail out/fail over policy now controlled by the upcall (993)
615 - add OBD_PING to check server availability and failure (954)
616 - lustre messages are now sent in sending host order (288, 340, 891)
617 - add eadatalen to MDS getattr reply (340)
618 - OST read replies may contain second buffer, with per-page status (593)
620 2003-03-11 Phil Schwan <phil@clusterfs.com>
623 - LDLM_DEBUG macro fix, for gcc 3.2 (850)
624 - failed open()s could cause deadlock; fixed (867, 869)
625 - stop cancelling OST locks when files are closed (481)
626 - overlapping XID spaces caused network corruption (851, 853)
627 - fix unsafe fsfilt counter arithmetic; change to atomic_t
628 - setattr_raw added, to do single-RPC, server-side setattrs
629 - lmc/lconf syntax change for OST UUIDs
630 - fix crashy race condition between ptlrpc_free_req and osc_close
631 - don't use request in mdc_enqueue if we hit a timeout (889)
632 - don't set the inode i_size for regular files from the MDS (896)
633 - handle out of order completion AST (842)
634 - don't LBUG if a lock request times out after receiving AST (913)
635 - avoid d_rehash race in ll_find_alias by rehashing inside dcache_lock
636 - if a bad lock AST arrives, send an error instead of dropping entirely
637 - return 0 from revalidate2 if ll_intent_lock returns -EINTR (912)
638 - fix leak in bulk IO when only partially completed (899, 900, 926)
639 - fix O_DIRECT for ia64 (55)
640 - (almost) eliminate Lustre-kernel-thread effects on load average (722)
641 - C-z after timeout could hang a process forever; fixed (977)
643 - client-side I/O cache (678, 924, 929, 941, 970)
645 - READPAGE and SETATTRs which don't take server-side locks get
648 2003-02-11 Phil Schwan <phil@clusterfs.com>
651 - Fix ldlm_lock_match on the MDS to avoid matching remote locks (592)
652 - Fix fsfilt_extN_readpage() to read a full page of directory
653 entries, or fake the remainder if PAGE_SIZE != blocksize (500)
654 - Avoid extra mdc_getattr() in ll_intent_lock when possible (534, 604)
655 - Fix imbalanced LOV object allocation and out-of-bound access (469)
656 - Most intent operations were removed, in favour of a new RPC mode
657 that does a single RPC to the server and bypasses most of the VFS
658 - All LDLM resource ID arrays were removed in favour of ldlm_res_id
659 - Aggressively cancel local locks on DLM servers
660 - mds_reint_unlink sends EA to the client if it's the last nlink.
661 client uses that EA to unlink OST objects.
662 - mds_reint_{rename,unlink,link} were rewritten to take ordered locks
663 - recursive symlinks were fixed (439)
664 - fixed NULL deref in DEBUG_REQ
665 - filter_update_lastobjid no longer calls sync, which annoyed extN
666 - fixed multi-client small-writes to a single file problem (445)
667 - fixed mtime updates during file writes (607)
668 - fixed vector writes on obdfilter causing problems when ENOSPC (670)
669 - fixed bug in obd_brw_read/write() (under guise of testing 367)
670 - fixed Linux OST size reporting problem (444, 656)
671 - OST now updates object mtime with writes or setattr (607, 619)
672 - client verifies file size before zeroing page past EOF (445)
673 - OST now writes last allocated objid to disk with allocation (108)
674 - LOV on echo now works (409)
676 - mds_reint_unlink sends a new buffer, with the EA included. this
677 buffer is only valid if body->valid & OBD_MD_FLEASIZE, which is only
678 set if a regular file was being unlinked, and it was the last link
679 - use PtlGet from the target for bulk writes (315)
680 - OST now updates object mtime with writes or setattr (607, 619)
681 - LDLM now has a grant-time callback to revalidate locked items, if
683 - Many MDS operations were reorganized to combat race conditions
685 - Merge b_intel branch (updated lprocfs code) - now at /proc/fs/lustre
686 - configure check to avoid gcc version 2.96 20000731-2.96-98) (606)
688 2003-01-06 Andreas Dilger <adilger@clusterfs.com>
691 - Fully reactivate OST imports after reconnection (512, others)
692 - Make sure client sees our -ENOTCONN from mds_handle (513 - partial)
693 - More graceful error handling for truncating on dead OST (515)
694 - Don't error out unless we're actually accessing dead stripes (474)
695 - Fix garbage sizes when stripes are missing (410)
696 - LRU counters were broken, causing constant lock purge (433, 432)
697 - garbage on read from stripes with failed OSTs (441)
698 - mark OSCs as active before reconnecting during recovery (438)
699 - lov_enqueue and lov_cancel need to handle inactive OSTs (403)
700 - lfind did not preserve OST order in output (443)
701 - symlinks cause hung clients, incorrect data (439)
702 - stop dereferencing request after dropping refcount (457)
703 - don't LASSERT(spin_is_locked) on non-SMP (455)
704 - fixes for many rename() bugs
705 - fstat didn't correctly synchronize attributes (399)
706 - server must handle lock cancellation during blocking AST prep (487)
707 - bulk descriptors were free()d too soon (511)
708 - fix paths in lconf, which would load incorrect modules (451, 507)
709 - fix confusing lconf 'host not found' error message (386)
710 - fix lock order deadlock on OST (O/R i_sem before journal ops, 478)
711 - fix race condition in mdc_blocking_ast() for inode access (526)
712 - fix lov_unpackmd() unpacking wrong number of stripes (537)
713 - fix lov_set_osc_active() marking wrong OSC inactive (440)
714 - fix bad lstripe lov_unpackmd() assertion (fix layering too) (527)
715 - fix multiple writes of stripe MD to MDS (358, maybe 519)
716 - fix lstripe in several ways (kernel side) (527)
717 - fix request leak in ldlm_cli_enqueue (262)
718 - incorrect OSC was marked inactive after OST failure
719 - call mds_fs_cleanup before unmounting filesystem (524)
720 - fix races between taking ns_lock and ldlm_lock_change_resource
721 - fix races updating LOV export open file list
722 - fix lov_enqueue error path, avoid decref-ing bad lock handle
723 - fix recovery NULL deref in ldlm_cli_cancel_unused
724 - fix some DLM races by using new hash table for lock handles (419)
725 - permit the client to specify desired inodes, at replay
726 - duplicate requests when we queue them for replay reintegration
727 - fix last_rcvd offset calculation
728 - sync after each recovered transaction, so we always make progress
729 - never, not always, ERESTART requests without transnos
730 - store the lov_desc in the MDS, so we don't depend on getlovinfo to
732 - skip replay if the MDS says that the client is already connected
733 - don't check for a recovery-enabled export to match lctl's UUID
734 - don't INC_USE_COUNT for phantom exports
735 - don't crash when cleaning up phantom exports (567)
736 - don't double-finish or set replay data for errored mdc_open requests
737 - abort requests when they time out, so we don't get old replies
738 - send/receive replies for AST messages again
739 - if the client says that it doesn't have the lock, cancel it on the
741 - if we timeout during I/O, don't try to cancel an in-use lock; instead
742 mark it as destroyed, it will all work out when decref is called
743 - fix module use counts (22, 581)
745 - ASTs now expect a reply (server cancels lock on error reply)
747 2002-12-02 Andreas Dilger <adilger@clusterfs.com>
750 - fix many simultaneous client startup (392)
751 - fix dentry->d_it clobbering
752 - credentials weren't being shipped for readdir/getattr operations
753 - remove invalid assertions triggered during some concurrent MD
755 - proper Lustre versions added (336, 389)
756 - fix memory leak for create error case (398)
757 - fix LOV locking bug that would get cli/srv out of sync
758 - fix echo client over LOV (409)
759 - fix dbench 2, extN refcount problem (170, 258, 356, 418)
760 - fix double-O_EXCL intent crash (424)
761 - avoid sending multiple lock CANCELs (352)
763 - MDS can do multi-client recovery (modulo bugs in new code)
765 - many updates, edits, cleanups
767 2002-11-18 Phil Schwan <phil@clusterfs.com>
770 - fix null d_it dereference (346)
771 - fix full OST/dbench hang (333)
772 - fix permission problem with file removal (286)
773 - fix removal of OSCs from LOV when they fail
774 - fix NULL deref during bulk timeout (214)
775 - fix problems related to multiple filesystems on one MDS (241)
776 - fixed serious subtle metadata locking bugs
777 - free locks on clients when inodes are removed due to memory
779 - fix inode pointer in lock data (285)
780 - partial support for multiple MDS on a single host (241)
781 - data locks weren't cancelled at clear_inode time (290, 311)
782 - intent locks could lead to unbounded lock growth (205)
783 - added a maximum lock count, an LRU list, and a flusher
784 - fix multiple rename (365)
785 - properly abstracted the echo client
786 - OSC locked 1 byte too many; fixed
787 - rewrote brw callback code:
788 - fixed recovery bugs related to LOVs (306)
789 - fixed too-many-pages-in-one-write crash (191)
790 - fixed (again) crash in sync_io_timeout (214)
791 - probably fixed callback-related race (385)
793 - Add capability to MDS protocol
794 - LDLM cancellations and callbacks on different portals
796 2002-10-28 Andreas Dilger <adilger@clusterfs.com>
799 - limit client IOV size to PTL_MD_MAX_IOV (611336, 191)
800 - defer open object destruction to close time (601981, 138)
801 - open/close OST file handle in obdo (OBD_MD_FLHANDLE) (601981, 138)
802 - move LDLM_ENQUEUE/CONVERT back to MDS portal (625069)
803 - abstract ll_lookup2, fix ll_revalidate2 to use abstraction (256)
804 - don't call obd_setattr in ll_file_release for destroyed objects
805 * protocol change to lustre_msg: move |version| and add |flags|
806 * protocol change to osc_punch: "start" in "o_size", "end" in "o_blocks"
807 * lock replay: for LDLM_FL_REPLAY trust client to do right thing
808 * added replay of create, unlink, link and rename operations during
809 MDS failover; recovery should be much more robust now
810 * remove failed OSCs from LOVs (only lov_create uses this so far)
811 * the lustre-HOWTO was brought (more) up to date (582544)
813 2002-10-23 Phil Schwan <phil@clusterfs.com>
816 - in-use dentries weren't being reused properly (617851)
817 - prevent multiple LDLM setup (599178)
818 - fix LOV size calculations for truncate (617853)
819 - fix client handling of MDS intent errors (POSIX)
820 - fix permission bug in lovstripe.c test (624321)
821 - fix MDS thread deadlock - move LDLM handler to DLM portal (625069)
822 - truncate past end of file could corrupt data
823 - proper cleanup after timeouts, crashes, etc (592524, 550815)
824 - a race in recovery could return ETIMEDOUT to apps (623947)
825 - building outside the source directory was fixed
826 * the lustre-HOWTO was brought (more) up to date (582544)
827 * major progress was made on recovery functionality
829 2002-10-10 Phil Schwan <phil@clusterfs.com>
832 - recovery deadlock fix
833 - rm -rf causes LBUG fix (617817)
834 - file open by multiple tasks fix (618962)
835 - directory permissions bugs (602707 and 620007)
836 - journal_stop fixed with locking (611313)
837 - O_APPEND failures resolved (618273, perhaps 614459)
838 - lconf PATH fix (619770)
839 - IA64 build fix (621450)
840 - RPC buffer sizes scale with amount of memory
842 2002-10-01 Phil Schwan <phil@clusterfs.com>
845 - locks would be cancelled without throwing away data pages,
846 resulting in inconsistent data (605627)
847 - inode attributes were not always being refreshed (605627, 612449)
848 - lconf now continues to cleanup after lctl reports an error
849 - MDS now enforces user permissions (602707)
850 - lprocfs cleanup fixed, but not yet enabled (614157)
851 - fixed infinite server hang, should a client not respond to an AST
852 - avoid going into recovery if user calls readlink() with a buffer
853 that's too small (613941)
854 - AST RPCs no longer require replies (614867) -- this may be changed
855 - don't crash server if client sends an IOV that's too big (611336)
856 - fixed lock conversion deadlock (611892)
857 - fixed the following of symlinks (614622)
858 * recovery: the server can remove locks from a client that dies, other
859 clients can make progress
860 * more extN patch fixes
861 * compile-time configurable ptlrpc buffer allocations
863 - collaborative read cache document
864 - Lustre Lite Performance CDR document-in-progress
866 2002-09-20 Andreas Dilger <adilger@clusterfs.com>
869 - fix typo in patch-2.4.18
871 2002-09-20 Andreas Dilger <adilger@clusterfs.com>
874 - clear ptlrpc request each time in handle_incoming_request()
875 - unlink of files now destroys the object on the OST
877 2002-09-19 Peter Braam <braam@clusterfs.com>
879 * add hard link support
880 * change obdfile creation method
881 * kernel patch changed
883 2002-09-19 Peter Braam <braam@clusterfs.com>
886 - stack overflow bug in extN fixed
888 2002-09-18 Andreas Dilger <adilger@clusterfs.com>
890 * documentation updates
891 - add man pages for config tools
892 - update tests/README to describe testing with new config tools
893 - finish metadata API descriptions
894 * bug fixes and cleanups
895 - statfs workaround for 16TB limit
896 - LOV stripe allocation improved, can stripe on subset of OSTs
897 - LOV file size/IO offset was wrong for files > 4GB in size
898 - object EA data was being dropped, caused files to be unreadable
899 - memory overflow with non-LOV OST caused memory corruption
900 - fixed regression tests to work with new config tools, obdfilter
901 - fixed bug when directory size became larger than 1 block
902 - fixed bug (for single client case) when PWD was deleted
903 - invalidate local directory pages when doing intent-based ops
904 - avoid LDLM oops when lock callback contained bad data
906 2002-09-09 Andreas Dilger <adilger@clusterfs.com>
908 * documentation updates
909 * bug fixes and cleanups
910 - configuration tools
913 - 64-bit compile warnings
914 - 64-bit internal statfs data
916 * test_brw on persistent OST devices
920 2002-09-04 Andreas Dilger <adilger@clusterfs.com>
922 * documentation updates
923 * bug fixes and cleanups
924 * configuration tools
926 2002-08-30 Peter J. Braam <braam@clusterfs.com>
929 * many small fixes to 0_5_4
930 * io/network handling
931 * thinkos in MDS operations
933 2002-08-24 Peter J. Braam <braam@clusterfs.com>
936 * crucial basic fixes to 0.5.3
937 * IOR, Iozone work over Elan
940 2002-08-07 Phil Schwan <phil@clusterfs.com>
941 * version 0_5_3, our first alpha
942 * we use the new Portals iovs
943 * documentation updates
944 * bug fixes and cleanups
945 * small changes in the DLM wire protocol
947 2002-07-25 Peter J. Braam <braam@clusterfs.com>
948 * version 0_5_1 with some initial stability,
949 * locking on MD and file I/O.
950 * documentation updates
951 * several bug fixes since 0.5.0
952 * small changes in wire protocol
954 2002-07-18 Phil Schwan <phil@clusterfs.com>
956 * delivered as Lustre Light Alpha
957 * fixed a crash after handling invalid MDS requests
958 * fixed directory pages for architectures with non-4k pages sizes
960 2002-07-11 Andreas Dilger <adilger@clusterfs.com>
961 * release version v0_4_4
962 * Moves TCP acceptor to be on port 2432 (unused Coda port) instead
964 * Fixes a number of interruption problems with OST operations.
965 * Update documentation for portals header changes
966 * Move all wire protocol structs/defines to lustre_idl.h
967 * Fixes symlink length bug.
968 * Add tcpdump to repository.
970 2002-07-05 Andreas Dilger <adilger@clusterfs.com>
971 * release version v0_4_3
972 * Fixes statfs for inodes on extN.
973 * Fixes bug in runtests which would delete /etc/hosts.
974 * Use 64-bit object IDs wherever possible (not into VFS though)
975 Remove ost_get_info, which is unused by lustre, and out of date.
977 2002-07-03 Peter Braam <braam@clusterfs.com>
978 * release version v0_4_2 Fixes a lookup error (type not passed)
979 * move forward to head of Portals
980 * move forward to latest Lustre kernel
982 2002-06-25 Peter Braam <braam@clusterfs.com>
983 * release version v0_4_1. Hopefully stable on single node use.