1 tbd Sun Microsystems, Inc.
3 * Support for networks:
4 socklnd - any kernel supported by Lustre,
5 qswlnd - Qsnet kernel modules 5.20 and later,
6 openiblnd - IbGold 1.8.2,
7 o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
8 viblnd - Voltaire ibhost 3.4.5 and later,
9 ciblnd - Topspin 3.2.0,
10 iiblnd - Infiniserv 3.3 + PathBits patch,
11 gmlnd - GM 2.1.22 and later,
12 mxlnd - MX 1.2.1 or later,
13 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
21 Bugzilla : 13621, 15983
22 Description: Protocol V2 of o2iblnd
23 Details : o2iblnd V2 has several new features:
24 . map-on-demand: map-on-demand is disabled by default, it can
25 be enabled by using modparam "map_on_demand=@value@", @value@
26 should >= 0 and < 256, 0 will disable map-on-demand, any other
27 valid value will enable map-on-demand.
28 Oi2blnd will create FMR or physical MR for RDMA if fragments of
30 Enable map-on-demand will take less memory for new connection,
31 but a little more CPU for RDMA.
32 . iWARP : to support iWARP, please enable map-on-demand, 32 and 64
33 are recommanded value. iWARP will probably fail for value >=128.
34 . OOB NOOP message: to resolve deadlock on router.
35 . tunable peer_credits_hiw: (high water to return credits),
36 default value of peer_credits_hiw equals to (peer_credits -1),
37 user can change it between peer_credits/2 and (peer_credits - 1).
38 Lower value is recommended for high latency network.
39 . tunable message queue size: it always equals to peer_credits,
40 higher value is recommended for high latency network.
41 . It's compatible with earlier version of o2iblnd
45 Description: Fixing 'running out of ports' issue
46 Details : Add a delay before next reconnect attempt in ksocklnd in
47 the case of lost race. Limit the frequency of query-requests
48 in lnet. Improved handling of 'dead peer' notifications in
53 Description: Change ptllnd timeout and watchdog timers
54 Details : Add ptltrace_on_nal_failed and bump ptllnd timeout to match
59 Description: One down Lustre FS hangs ALL mounted Lustre filesystems
60 Details : Shared routing enhancements - peer health detection.
62 Severity : enhancement
64 Description: acceptor.c cleanup
65 Details : Code duplication in acceptor.c for the cases of kernel and
66 user-space removed. User-space libcfs tcpip primitives
67 uniformed to have prototypes similar to kernel ones. Minor
68 cosmetic changes in usocklnd to use cfs_socket_t as
69 representation of socket.
73 Description: IB path MTU mistakenly set to 1st path MTU when ib_mtu is off
74 Details : See comment 46 in bug 11245 for details - it's indeed a bug
75 introduced by the original 11245 fix.
79 Description: uptllnd credit overflow fix
80 Details : kptl_msg_t::ptlm_credits could be overflown by uptllnd since
85 Description: socklnd protocol version 3
86 Details : With current protocol V2, connections on router can be
87 blocked and can't receive any incoming messages when there is no
88 more router buffer, so ZC-ACK can't be handled (LNet message
89 can't be finalized) and will cause deadlock on router.
90 Protocol V3 has a dedicated connection for emergency messages
91 like ZC-ACK to router, messages on this dedicated connection
92 don't need any credit so will never be blocked. Also, V3 can send
93 keepalive ping in specified period for router healthy checking.
95 -------------------------------------------------------------------------------
97 12-31-2008 Sun Microsystems, Inc.
99 * Support for networks:
100 socklnd - any kernel supported by Lustre,
101 qswlnd - Qsnet kernel modules 5.20 and later,
102 openiblnd - IbGold 1.8.2,
103 o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
104 viblnd - Voltaire ibhost 3.4.5 and later,
105 ciblnd - Topspin 3.2.0,
106 iiblnd - Infiniserv 3.3 + PathBits patch,
107 gmlnd - GM 2.1.22 and later,
108 mxlnd - MX 1.2.1 or later,
109 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
113 Description: workaround for OOM from o2iblnd
114 Details : OFED needs allocate big chunk of memory for QP while creating
115 connection for o2iblnd, OOM can happen if no such a contiguous
117 QP size is decided by concurrent_sends and max_fragments of
118 o2iblnd, now we permit user to specify smaller value for
119 concurrent_sends of o2iblnd(i.e: concurrent_sends=7), which
120 will decrease memory block size required by creating QP.
124 Description: Support Zerocopy receive of Chelsio device
125 Details : Chelsio driver can support zerocopy for iov[1] if it's
126 contiguous and large enough.
130 Description: fix credit flow deadlock in uptllnd
134 Description: finalize network operation in reasonable time
135 Details : conf-sanity test_32a couldn't stop ost and mds because it
136 tried to access non-existent peer and tcp connect took
137 quite long before timing out.
141 Description: Continuous recovery on 33 of 413 nodes after lustre oss failure
142 Details : Lost reference on conn prevents peer from being destroyed, which
143 could prevent new peer creation if peer count has reached upper
148 Description: LNET Selftest results in Soft lockup on OSS CPU
149 Details : only hits when 8 or more o2ib clients involved and a session is
150 torn down with 'lst end_session' without preceeding 'lst stop'.
154 Description: concurrent_sends in IB LNDs should not be changeable at run time
155 Details : concurrent_sends in IB LNDs should not be changeable at run time
159 Description: ptl_send_rpc hits LASSERT when ptl_send_buf fails
160 Details : only hits under out-of-memory situations
163 -------------------------------------------------------------------------------
165 2009-02-07 Sun Microsystems, Inc.
167 * Support for networks:
168 socklnd - any kernel supported by Lustre,
169 qswlnd - Qsnet kernel modules 5.20 and later,
170 openiblnd - IbGold 1.8.2,
171 o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
172 viblnd - Voltaire ibhost 3.4.5 and later,
173 ciblnd - Topspin 3.2.0,
174 iiblnd - Infiniserv 3.3 + PathBits patch,
175 gmlnd - GM 2.1.22 and later,
176 mxlnd - MX 1.2.1 or later,
177 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
180 Description: workaround for OOM from o2iblnd
181 Details : OFED needs allocate big chunk of memory for QP while creating
182 connection for o2iblnd, OOM can happen if no such a contiguous
184 QP size is decided by concurrent_sends and max_fragments of
185 o2iblnd, now we permit user to specify smaller value for
186 concurrent_sends of o2iblnd(i.e: concurrent_sends=7), which
187 will decrease memory block size required by creating QP.
191 Description: Support Zerocopy receive of Chelsio device
192 Details : Chelsio driver can support zerocopy for iov[1] if it's
193 contiguous and large enough.
196 Description: fix credit flow deadlock in uptllnd
200 Description: finalize network operation in reasonable time
201 Details : conf-sanity test_32a couldn't stop ost and mds because it
202 tried to access non-existent peer and tcp connect took
203 quite long before timing out.
207 Description: Continuous recovery on 33 of 413 nodes after lustre oss failure
208 Details : Lost reference on conn prevents peer from being destroyed, which
209 could prevent new peer creation if peer count has reached upper
214 Description: LNET Selftest results in Soft lockup on OSS CPU
215 Details : only hits when 8 or more o2ib clients involved and a session is
216 torn down with 'lst end_session' without preceeding 'lst stop'.
220 Description: concurrent_sends in IB LNDs should not be changeable at run time
221 Details : concurrent_sends in IB LNDs should not be changeable at run time
223 -------------------------------------------------------------------------------
225 11-03-2008 Sun Microsystems, Inc.
227 * Support for networks:
228 socklnd - any kernel supported by Lustre,
229 qswlnd - Qsnet kernel modules 5.20 and later,
230 openiblnd - IbGold 1.8.2,
231 o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
232 viblnd - Voltaire ibhost 3.4.5 and later,
233 ciblnd - Topspin 3.2.0,
234 iiblnd - Infiniserv 3.3 + PathBits patch,
235 gmlnd - GM 2.1.22 and later,
236 mxlnd - MX 1.2.1 or later,
237 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
241 Description: ptl_send_rpc hits LASSERT when ptl_send_buf fails
242 Details : only hits under out-of-memory situations
244 -------------------------------------------------------------------------------
246 04-26-2008 Sun Microsystems, Inc.
248 * Support for networks:
249 socklnd - any kernel supported by Lustre,
250 qswlnd - Qsnet kernel modules 5.20 and later,
251 openiblnd - IbGold 1.8.2,
252 o2iblnd - OFED 1.1 and 1.2.0, 1.2.5
253 viblnd - Voltaire ibhost 3.4.5 and later,
254 ciblnd - Topspin 3.2.0,
255 iiblnd - Infiniserv 3.3 + PathBits patch,
256 gmlnd - GM 2.1.22 and later,
257 mxlnd - MX 1.2.1 or later,
258 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
262 Description: excessive debug information removed
263 Details : excessive debug information removed
267 Description: ksocknal_create_conn() hit ASSERTION during connection race
268 Details : ksocknal_create_conn() hit ASSERTION during connection race
272 Description: ksocknal_send_hello() hit ASSERTION while connecting race
273 Details : ksocknal_send_hello() hit ASSERTION while connecting race
277 Description: o2iblnd/ptllnd credit deadlock in a routed config.
278 Details : o2iblnd/ptllnd credit deadlock in a routed config.
282 Description: High load after starting lnet
283 Details : gmlnd should sleep in rx thread in interruptible way. Otherwise,
284 uptime utility reports high load that looks confusingly.
288 Description: ksocklnd fails to establish connection if accept_port is high
289 Details : PID remapping must not be done for active (outgoing) connections
292 --------------------------------------------------------------------------------
294 2008-01-11 Sun Microsystems, Inc.
296 * Support for networks:
297 socklnd - any kernel supported by Lustre,
298 qswlnd - Qsnet kernel modules 5.20 and later,
299 openiblnd - IbGold 1.8.2,
300 o2iblnd - OFED 1.1 and 1.2.0, 1.2.5
301 viblnd - Voltaire ibhost 3.4.5 and later,
302 ciblnd - Topspin 3.2.0,
303 iiblnd - Infiniserv 3.3 + PathBits patch,
304 gmlnd - GM 2.1.22 and later,
305 mxlnd - MX 1.2.1 or later,
306 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
310 Description: liblustre network error
311 Details : liblustre clients should understand LNET_ACCEPT_PORT environment
312 variable even if they don't start lnet acceptor.
316 Description: Strange message from lnet (Ignoring prediction from the future)
317 Details : Incorrect calculation of peer's last_alive value in ksocklnd
319 --------------------------------------------------------------------------------
321 2007-12-07 Cluster File Systems, Inc. <info@clusterfs.com>
323 * Support for networks:
324 socklnd - any kernel supported by Lustre,
325 qswlnd - Qsnet kernel modules 5.20 and later,
326 openiblnd - IbGold 1.8.2,
327 o2iblnd - OFED 1.1 and 1.2.0, 1.2.5.
328 viblnd - Voltaire ibhost 3.4.5 and later,
329 ciblnd - Topspin 3.2.0,
330 iiblnd - Infiniserv 3.3 + PathBits patch,
331 gmlnd - GM 2.1.22 and later,
332 mxlnd - MX 1.2.1 or later,
333 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
337 Description: ASSERTION(me == md->md_me) failed in lnet_match_md()
341 Description: increase send queue size for ciblnd/openiblnd
345 Description: new userspace socklnd
346 Details : Old userspace tcpnal that resided in lnet/ulnds/socklnd replaced
347 with new one - usocklnd.
349 Severity : enhancement
351 Description: Console message flood
352 Details : Make cdls ratelimiting more tunable by adding several tunable in
353 procfs /proc/sys/lnet/console_{min,max}_delay_centisecs and
354 /proc/sys/lnet/console_backoff.
356 --------------------------------------------------------------------------------
358 2007-09-27 Cluster File Systems, Inc. <info@clusterfs.com>
360 * Support for networks:
361 socklnd - any kernel supported by Lustre,
362 qswlnd - Qsnet kernel modules 5.20 and later,
363 openiblnd - IbGold 1.8.2,
364 o2iblnd - OFED 1.1 and 1.2,
365 viblnd - Voltaire ibhost 3.4.5 and later,
366 ciblnd - Topspin 3.2.0,
367 iiblnd - Infiniserv 3.3 + PathBits patch,
368 gmlnd - GM 2.1.22 and later,
369 mxlnd - MX 1.2.1 or later,
370 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
374 Description: /proc/sys/lnet has non-sysctl entries
375 Details : Updating dump_kernel/daemon_file/debug_mb to use sysctl variables
379 Description: TOE Kernel panic by ksocklnd
380 Details : offloaded sockets provide their own implementation of sendpage,
381 can't call tcp_sendpage() directly
385 Description: kibnal_shutdown() doesn't finish; lconf --cleanup hangs
386 Details : races between lnd_shutdown and peer creation prevent
387 lnd_shutdown from finishing.
391 Description: open files rlimit 1024 reached while liblustre testing
392 Details : ulnds/socklnd must close open socket after unsuccessful
397 Description: build error
398 Details : fix typos in gmlnd, ptllnd and viblnd
400 --------------------------------------------------------------------------------
402 2007-07-30 Cluster File Systems, Inc. <info@clusterfs.com>
404 * Support for networks:
405 socklnd - kernels up to 2.6.16,
406 qswlnd - Qsnet kernel modules 5.20 and later,
407 openiblnd - IbGold 1.8.2,
408 o2iblnd - OFED 1.1 and 1.2
409 viblnd - Voltaire ibhost 3.4.5 and later,
410 ciblnd - Topspin 3.2.0,
411 iiblnd - Infiniserv 3.3 + PathBits patch,
412 gmlnd - GM 2.1.22 and later,
413 mxlnd - MX 1.2.1 or later,
414 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
416 --------------------------------------------------------------------------------
418 2007-06-21 Cluster File Systems, Inc. <info@clusterfs.com>
420 * Support for networks:
421 socklnd - kernels up to 2.6.16,
422 qswlnd - Qsnet kernel modules 5.20 and later,
423 openiblnd - IbGold 1.8.2,
425 viblnd - Voltaire ibhost 3.4.5 and later,
426 ciblnd - Topspin 3.2.0,
427 iiblnd - Infiniserv 3.3 + PathBits patch,
428 gmlnd - GM 2.1.22 and later,
429 mxlnd - MX 1.2.1 or later,
430 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
434 Description: Initialize cpumask before use
438 Description: ASSERTION failures when upgrading to the patchless zero-copy
440 Details : This bug affects "rolling upgrades", causing an inconsistent
441 protocol version negotiation and subsequent assertion failure
442 during rolling upgrades after the first wave of upgrades.
446 Details : Change "dropped message" CERRORs to D_NETERROR so they are
447 logged instead of creating "console chatter" when a lustre
448 timeout races with normal RPC completion.
451 Details : lnet_clear_peer_table can wait forever if user forgets to
455 Details : libcfs_id2str should check pid against LNET_PID_ANY.
459 Description: added LNET self test
460 Details : landing b_self_test
465 Description: cfs_duration_{u,n}sec() wrongly calculate nanosecond part of
467 Details : do_div() macro is used incorrectly.
469 2007-04-23 Cluster File Systems, Inc. <info@clusterfs.com>
473 Description: make panic on lbug configurable
477 Description: Add OFED1.2 support to o2iblnd
478 Details : o2iblnd depends on OFED's modules, if out-tree OFED's modules
479 are installed (other than kernel's in-tree infiniband), there
480 could be some problem while insmod o2iblnd (mismatch CRC of
482 If extra Module.symvers is supported in kernel (i.e, 2.6.17),
483 this link provides solution:
484 https://bugs.openfabrics.org/show_bug.cgi?id=355
485 if extra Module.symvers is not supported in kernel, we will
486 have to run the script in bug 12316 to update
487 $LINUX/module.symvers before building o2iblnd.
488 More details about this are in bug 12316.
490 ------------------------------------------------------------------------------
492 2007-04-01 Cluster File Systems, Inc. <info@clusterfs.com>
493 * version 1.4.10 / 1.6.0
494 * Support for networks:
495 socklnd - kernels up to 2.6.16,
496 qswlnd - Qsnet kernel modules 5.20 and later,
497 openiblnd - IbGold 1.8.2,
499 viblnd - Voltaire ibhost 3.4.5 and later,
500 ciblnd - Topspin 3.2.0,
501 iiblnd - Infiniserv 3.3 + PathBits patch,
502 gmlnd - GM 2.1.22 and later,
503 mxlnd - MX 1.2.1 or later,
504 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
508 Description: Ptllnd didn't init kptllnd_data.kptl_idle_txs before it could be
509 possibly accessed in kptllnd_shutdown. Ptllnd should init
510 kptllnd_data.kptl_ptlid2str_lock before calling kptllnd_ptlid2str.
514 Description: gmlnd ignored some transmit errors when finalizing lnet messages.
518 Description: ptllnd logs a piece of incorrect debug info in kptllnd_peer_handle_hello.
522 Description: the_lnet.ln_finalizing was not set when the current thread is
523 about to complete messages. It only affects multi-threaded
529 Description: Changed the default kqswlnd ntxmsg=512
534 Description: Assertion failure in kernel ptllnd caused by posting passive
535 bulk buffers before connection establishment complete.
540 Description: A race in kernel ptllnd between deleting a peer and posting
541 new communications for it could hang communications -
542 manifesting as "Unexpectedly long timeout" messages.
547 Description: Kernel ptllnd lock ordering issue could hang a node.
552 Description: node crash on socket teardown race
555 Frequency : 'lctl peer_list' issued on a mx net
557 Description: Enable lctl's peer_list for MXLND
560 Frequency : after Ptllnd timeouts and portals congestion
562 Description: Credit overflows
563 Details : This was a bug in ptllnd connection establishment. The fix
564 implements better peer stamps to disambiguate connection
565 establishment and ensure both peers enter the credit flow
566 state machine consistently.
571 Description: kptllnd didn't propagate some network errors up to LNET
572 Details : This bug was spotted while investigating 11394. The fix
573 ensures network errors on sends and bulk transfers are
574 propagated to LNET/lustre correctly.
576 Severity : enhancement
578 Description: Fixed console chatter in case of -ETIMEDOUT.
580 Severity : enhancement
582 Description: Added D_NETTRACE for recording network packet history
583 (initially only for ptllnd). Also a separate userspace
584 ptllnd facility to gather history which should really be
585 covered by D_NETTRACE too, if only CDEBUG recorded history in
591 Description: o2iblnd handle early RDMA_CM_EVENT_DISCONNECTED.
592 Details : If the fabric is lossy, an RDMA_CM_EVENT_DISCONNECTED
593 callback can occur before a connection has actually been
594 established. This caused an assertion failure previously.
596 Severity : enhancement
598 Description: Multiple instances for o2iblnd
599 Details : Allow multiple instances of o2iblnd to enable networking over
600 multiple HCAs and routing between them.
604 Description: lnet deadlock in router_checker
605 Details : turned ksnd_connd_lock, ksnd_reaper_lock, and ksock_net_t:ksnd_lock
606 into BH locks to eliminate potential deadlock caused by
607 ksocknal_data_ready() preempting code holding these locks.
611 Description: Millions of failed socklnd connection attempts cause a very slow FS
612 Details : added a new route flag ksnr_scheduled to distinguish from
613 ksnr_connecting, so that a peer connection request is only turned
614 down for race concerns when an active connection to the same peer
615 is under progress (instead of just being scheduled).
617 ------------------------------------------------------------------------------
619 2007-02-09 Cluster File Systems, Inc. <info@clusterfs.com>
621 * Support for networks:
622 socklnd - kernels up to 2.6.16
623 qswlnd - Qsnet kernel modules 5.20 and later
624 openiblnd - IbGold 1.8.2
626 viblnd - Voltaire ibhost 3.4.5 and later
627 ciblnd - Topspin 3.2.0
628 iiblnd - Infiniserv 3.3 + PathBits patch
629 gmlnd - GM 2.1.22 and later
630 mxlnd - MX 1.2.1 or later
631 ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
634 Severity : major on XT3
636 Description: libcfs overwrites /proc/sys/portals
637 Details : libcfs created a symlink from /proc/sys/portals to
638 /proc/sys/lnet for backwards compatibility. This is no
639 longer required and makes the Cray portals /proc variables
644 Description: OFED FMR API change
645 Details : This changes parameter usage to reflect a change in
646 ib_fmr_pool_map_phys() between OFED 1.0 and OFED 1.1. Note
647 that FMR support is only used in experimental versions of the
648 o2iblnd - this change does not affect standard usage at all.
650 Severity : enhancement
652 Description: new ko2iblnd module parameter: ib_mtu
653 Details : the default IB MTU of 2048 performs badly on 23108 Tavor
654 HCAs. You can avoid this problem by setting the MTU to 1024
655 using this module parameter.
657 Severity : enhancement
658 Bugzilla : 11118/11620
659 Description: ptllnd small request message buffer alignment fix
660 Details : Set the PTL_MD_LOCAL_ALIGN8 option on small message receives.
661 Round up small message size on sends in case this option
662 is not supported. 11620 was a defect in the initial
663 implementation which effectively asserted all peers had to be
664 running the correct protocol version which was fixed by always
665 NAK-ing such requests and handling any misalignments they
670 Description: When kib(nal|lnd)_del_peer() is called upon a peer whose
671 ibp_tx_queue is not empty, kib(nal|lnd)_destroy_peer()'s
672 'LASSERT(list_empty(&peer->ibp_tx_queue))' will fail.
674 Severity : enhancement
676 Description: Patchless ZC(zero copy) socklnd
677 Details : New protocol for socklnd, socklnd can support zero copy without
678 kernel patch, it's compatible with old socklnd. Checksum is
679 moved from tunables to modparams.
683 Description: When ksocknal_del_peer() is called upon a peer whose
684 ksnp_tx_queue is not empty, ksocknal_destroy_peer()'s
685 'LASSERT(list_empty(&peer->ksnp_tx_queue))' will fail.
688 Frequency : when ptlrpc is under heavy use and runs out of request buffer
690 Description: In lnet_match_blocked_msg(), md can be used without holding a
694 Frequency : very rarely
696 Description: If ksocknal_lib_setup_sock() fails, a ref on peer is lost.
697 If connd connects a route which has been closed by
698 ksocknal_shutdown(), ksocknal_create_routes() may create new
699 routes which hold references on the peer, causing shutdown
700 process to wait for peer to disappear forever.
702 Severity : enhancement
704 Description: Dump XT3 portals traces on kptllnd timeout
705 Details : Set the kptllnd module parameter "ptltrace_on_timeout=1" to
706 dump Cray portals debug traces to a file. The kptllnd module
707 parameter "ptltrace_basename", default "/tmp/lnet-ptltrace",
708 is the basename of the dump file.
711 Frequency : infrequent
713 Description: kernel ptllnd fix bug in connection re-establishment
714 Details : Kernel ptllnd could produce protocol errors e.g. illegal
715 matchbits and/or violate the credit flow protocol when trying
716 to re-establish a connection with a peer after an error or
719 Severity : enhancement
721 Description: Allow /proc/sys/lnet/debug to be set symbolically
722 Details : Allow debug and subsystem debug values to be read/set by name
723 in addition to numerically, for ease of use.
726 Frequency : only in configurations with LNET routers
728 Description: routes automatically marked down and recovered
729 Details : In configurations with LNET routers if a router fails routers
730 now actively try to recover routes that are down, unless they
731 are marked down by an administrator.
733 ------------------------------------------------------------------------------
735 2006-12-09 Cluster File Systems, Inc. <info@clusterfs.com>
738 Frequency : very rarely, in configurations with LNET routers and TCP
740 Description: incorrect data written to files on OSTs
741 Details : In certain high-load conditions incorrect data may be written
742 to files on the OST when using TCP networks.
744 ------------------------------------------------------------------------------
746 2006-07-31 Cluster File Systems, Inc. <info@clusterfs.com>
748 - rework CDEBUG messages rate-limiting mechanism b=10375
749 - add per-socket tunables for socklnd if the kernel is patched b=10327
751 ------------------------------------------------------------------------------
753 2006-02-15 Cluster File Systems, Inc. <info@clusterfs.com>
755 - fix use of portals/lnet pid to avoid dropping RPCs b=10074
756 - iiblnd wasn't mapping all memory, resulting in comms errors b=9776
757 - quiet LNET startup LNI message for liblustre b=10128
758 - Better console error messages if 'ip2nets' can't match an IP address
759 - Fixed overflow/use-before-set bugs in linux-time.h
760 - Fixed ptllnd bug that wasn't initialising rx descriptors completely
761 - LNET teardown failed an assertion about the route table being empty
762 - Fixed a crash in LNetEQPoll(<invalid handle>)
763 - Future protocol compatibility work (b_rls146_lnetprotovrsn)
764 - improve debug message for liblustre/Catamount nodes (b=10116)
766 2005-10-10 Cluster File Systems, Inc. <info@clusterfs.com>
767 * Configuration change for the XT3
768 The PTLLND is now used to run Lustre over Portals on the XT3.
769 The configure option(s) --with-cray-portals are no longer
770 used. Rather --with-portals=<path-to-portals-includes> is
771 used to enable building on the XT3. In addition to enable
772 XT3 specific features the option --enable-cray-xt3 must be
775 2005-10-10 Cluster File Systems, Inc. <info@clusterfs.com>
776 * Portals has been removed, replaced by LNET.
777 LNET is new networking infrastructure for Lustre, it includes a
778 reorganized network configuration mode (see the user
779 documentation for full details) as well as support for routing
780 between different network fabrics. Lustre Networking Devices
781 (LNDS) for the supported network fabrics have also been created
782 for this new infrastructure.
784 2005-08-08 Cluster File Systems, Inc. <info@clusterfs.com>
789 Frequency : rare (large Voltaire clusters only)
791 Description: the default number of reserved transmit descriptors was too low
792 for some large clusters
793 Details : As a workaround, the number was increased. A proper fix includes
796 2005-06-02 Cluster File Systems, Inc. <info@clusterfs.com>
801 Frequency : occasional (large-scale events, cluster reboot, network failure)
803 Description: too many error messages on console obscure actual problem and
804 can slow down/panic server, or cause recovery to fail repeatedly
805 Details : enable rate-limiting of console error messages, and some messages
806 that were console errors now only go to the kernel log
808 Severity : enhancement
810 Description: add /proc/sys/portals/catastrophe entry which will report if
811 that node has previously LBUGged
813 2005-04-06 Cluster File Systems, Inc. <info@clusterfs.com>
815 - update gmnal to use PTL_MTU, fix module refcounting (b=5786)
817 2005-04-04 Cluster File Systems, Inc. <info@clusterfs.com>
819 - handle error return code in kranal_check_fma_rx() (5915,6054)
821 2005-02-04 Cluster File Systems, Inc. <info@clusterfs.com>
823 - update vibnal (Voltaire IB NAL)
824 - update gmnal (Myrinet NAL), gmnalid
826 2005-02-04 Eric Barton <eeb@bartonsoftware.com>
828 * Landed portals:b_port_step as follows...
830 - removed CFS_DECL_SPIN*
831 just use 'spinlock_t' and initialise with spin_lock_init()
833 - removed CFS_DECL_MUTEX*
834 just use 'struct semaphore' and initialise with init_mutex()
836 - removed CFS_DECL_RWSEM*
837 just use 'struct rw_semaphore' and initialise with init_rwsem()
839 - renamed cfs_sleep_chan -> cfs_waitq
840 cfs_sleep_link -> cfs_waitlink
842 - fixed race in linux version of arch-independent socknal
843 (the ENOMEM/EAGAIN decision).
845 - Didn't fix problems in Darwin version of arch-independent socknal
846 (resetting socket callbacks, eager ack hack, ENOMEM/EAGAIN decision)
848 - removed libcfs types from non-socknal header files (only some types
849 in the header files had been changed; the .c files hadn't been