LU-10391 lnet: use %pISc for formatting IP addresses The Linux kernel's printf functionality understands %pIS to means that a the address in a 'struct sockaddr' should be formated, either as IPv4 or IPv6. For IPv6, the verbose format showing all 16 bytes whether zero or not is used. To get the more familiar "compressed" format where strings of :0000: are replaced with ::, we need to add the 'c' flag. This is ignored for IPv4. When requesting the port as well ("%pISp), the 'c' and 'p' can appear in either order. So this patch changes all %pIS to %pISc as we always want the compressed format. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: Ida17f5008e06a00c5460cf7161ed07de8fa7a65d Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48685 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-10391 socklnd: add hello message version 4 KSOCK_PROTO_V4 uses a 'hello' message that contains lnet_hdr_nid16 with 16 byte addresses Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I52a36739d3a84dc059537059a586ce3dab2b20f0 Reviewed-on: https://review.whamcloud.com/43611 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 socklnd: Change ksock_hello_msg to struct lnet_nid 'struct ksock_hello_msg' now stores 'struct lnet_nid', but it is converted to 'struct ksock_hello_msg_nid4' - the old format - for transmit, which is converted back on receive. This opens the way for a new version of the hello protocol which will use 16byte addresses. Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I22e86f9088f6001f203f24f93ef292fcf2a8e69f Reviewed-on: https://review.whamcloud.com/43610 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 socklnd: move lnet_hdr unpack into ->pro_unpack Converting the lnet_hdr from network-format to host-format is currently done in ksocknal_process_recv(). Move it to ->pro_unpack() so that a different protocol can send it in a different format. Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: Icc22f4b52c1391d382c28bad157795f5477f4d7c Reviewed-on: https://review.whamcloud.com/43609 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 lnet: separate lnet_hdr in msg from that in lnd. The lnet_hdr stored in an lnet_msg has fields which are sometimes in le byte order and sometimes in host byte order. The various lnds need all these fields to be in le byte order for transmission or reception over the network. To support larger (IPv6) NIDs, we will need the lnet_hdr in lnet_msg to store these NIDs, but the lnd will need both 4byte-addr and 16-byte lnds depending on protocol negotiation. This patch separates out the two to make the conversion easier to follow. 'struct lnet_hdr' is now used within common lnet code, and is not stored in network buffers. lnd_send will convert from 'struct lnet_hdr' to whatever is required in the network buffer. When lnet_parse() is called, the network buffer will be converted to a 'struct lnet_hdr' first, and that will be passed to lnet_parse(). The common fields of 'struct lnet_hdr' are always in host byte order. The command specific fields (now in 'union lnet_cmd_hdr') have not been changed and are sometimes host-byte-order and sometimes l3-byte-order. The new 'struct lnet_hdr_nid4' is used in network buffers. It is opaque - there are no subfields to access. Very few places in the lnd code want to access fields anyway. New functions lnet_hdr_to_nid4() and lnet_hdr_from_nid4() can convert between the lnet_hdr_nid4 to the internal lnet_hdr. 'struct _lnet_hdr_nid4' is provided to access fields inside 'struct lnet_hdr_nid4' when that is really needed. It is used by the to/from functions and a couple of other places. Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I55277a49538543376cf0404f749d3357a2950a7c Reviewed-on: https://review.whamcloud.com/43603 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Chris Horn <chris.horn@hpe.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 socklnd: prepare for new KSOCK_MSG type Various places in socklnd assume there are only two message type: KSOCK_MSG_NOOP and KSOCK_MSG_LNET. We will soon add another type to support a new lnet_hdr type with large addresses. So do some cleanup first: - get rid of ksock_lnet_msg - it doesn't add anything to lnet_hdr - separate out 'struct ksock_hdr'. We often want the size of this header, and instead request the offset of a field in ksock_msg. - introduce switch statements in a couple of places to handle the different types of ksock_msg. Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: Ibe484f76757c4100b8532cef659c3cc369b658ba Reviewed-on: https://review.whamcloud.com/43601 Reviewed-by: Chris Horn <chris.horn@hpe.com> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 lnet: introduce lnet_processid for ksock_peer_ni struct lnet_processid (without the '_') is like lnet_process_id, but contains a 'struct lnet_nid' rather than lnet_nid_t. So far it is only used for ksnp_id in struct ksock_peer_ni, and related functions. Test-Parameters: trivial Test-Parameters: serverversion=2.12 serverdistro=el7.9 testlist=runtests Test-Parameters: clientversion=2.12 testlist=runtests Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I1fea693b1c84ca4c3ac1821f55874ad11519a33b Reviewed-on: https://review.whamcloud.com/42104 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Chris Horn <chris.horn@hpe.com> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12678 lnet: use list_for_each_entry() Several loops use list_for_each(), then call list_entry() each time in the loop This complexity can be replaced with the use of list_for_each_entry(). Test-Parameters: trivial testlist=sanity-lnet Signed-off-by: James Simmons <jsimmons@infradead.org> Change-Id: Ib7968466c4fce5173b20cbaf6c878975ba522d43 Reviewed-on: https://review.whamcloud.com/43591 Reviewed-by: Chris Horn <chris.horn@hpe.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12678 lnet: use list_first_entry() in lnet/klnds subdirectory. Convert list_entry(foo->next .....) to list_first_entry(foo, ....) in 'lnet/klnds In several cases the call is combined with a list_empty() test and list_first_entry_or_null() is used Test-Parameters: trivial testlist=sanity-lnet Signed-off-by: Mr. NeilBrown <neilb@suse.de> Change-Id: I3b2b33c3c9284c02e44610614d64a1f84be300a4 Reviewed-on: https://review.whamcloud.com/43419 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10391 socklnd: use sockaddr instead of __u32 addresses. LNet/socklnd often uses __u32 to hold an ipv4 address. As we want to extend socklnd to work with IPv6 addresses too, this needs to change. This patch changes many __u32s to variants of 'struct sockaddr'. Library code from sunrpc is used for copying and comparing addresses rpc_copy_addr() and rpc_cmp_addr() and for extracting or setting the port: rpc_get_port() and rpc_set_port(). The "%pIS" printf format is used for printing a sockaddr (works for both IPv4 and IPv6), and "%pISp" for printing the address with the port. The __u32 is in host-byte-order, while addresses in sockaddr are always network-order, so htonl and ntohl are used as needed. When storing an address (e.g. in a structure), 'struct sockaddr_storage' is used. When passing an address to a function, 'struct sockaddr' is used. When an address is known to be IPv4 (i.e. when converting to or from __u32), 'struct sockaddr_in' is used. The following functions are changed to take a 'struct sockaddr*' argument: lnet_connect() lnet_connect_console_error() lnet_sock_getaddr() ksocknal_ip2iface() ksocknal_ip2index() ksocknal_create_route() ksocknal_connecting() ksocknal_close_peer_conns_locked() ksocknal_peer_del_interface_locked() The following structures have had fields changed to 'struct sockaddr_storage' struct ksock_interface: ksni_ipaddr -> ksni_addr struct ksock_conn ksnc_myipaddr -> ksnc_myaddr ksnc_ipaddr and ksnc_port -> ksnc_peeraddr struct ksock_route ksnr_myipaddr -> ksnr_myaddr ksnr_ipaddr and ksnr_port -> ksnr_addr Assorted strings have been joined onto a single line, and various indented have been changed from space to tabs. There should be no behaviour changes, though the structures mentioned above will now be a little larger. Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I42d12260185638407b5b611391fc69bfd9f91754 Reviewed-on: https://review.whamcloud.com/37704 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Reviewed-by: Aurelien Degremont <degremoa@amazon.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13004 socklnd: discard tx_iov. tx_iov always points to tx_hdr, so we can discard tx_iov, and just use &tx_hdr. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I9738b6f953184fdea859da7a9c4187227fa61143 Reviewed-on: https://review.whamcloud.com/37851 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-9679 various: use list_splice and list_splice_init The construct list_add(to, from); list_del(from); is equivalent to list_splice(from, to); providing 'to' has been initialized. Similarly with list_del_init and list_splice_init. There is no need to check if list_empty(from) first. Also looping over a list moving individiual entries to another list can more easily be done with list_splice. These changes improve code clarity. Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I710eb3bbd83c75e6c8f00b8d0a4c256ad28f9082 Reviewed-on: https://review.whamcloud.com/37457 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.super@gmail.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12678 lnet: socklnd: mark all ksock_proto struct 'const'. These structs are always read-only, so tell the compiler. Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: Icc7c3209135a2ab0d04a822b7053231fd2d9ff0c Reviewed-on: https://review.whamcloud.com/36894 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12678 lnet: remove locking protection ln_testprotocompat lnet_net_lock(LNET_LOCK_EX) is a heavy-weight lock that is not necessary here. The bits in this field are only set rarely - via an ioctl - and the pattern for reading and clearing them exactly matches test_and_clear_bit(). So change the field to "unsigned long" (so test_and_clear_bit() can be used), and use test_and_clear_bit(), discarding all other locking. Test-Parameters: trivial testlist=sanity-lnet Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: Ie420fcb3d547d9ec04025b921d5b24bd8f2fcce3 Reviewed-on: https://review.whamcloud.com/36856 Reviewed-by: Alexey Lyashkov <c17817@cray.com> Reviewed-by: Chris Horn <hornc@cray.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12678 lnet: use list_move where appropriate. There are several places in lustre where "list_del" (or occasionally "list_del_init") is followed by "list_add" or "list_add_tail" which moves the object to a different list. These can be combined into "list_move" or "list_move_tail". Test-Parameters: trivial testlist=sanity-lnet Change-Id: I481de128ea40928186f78a0a0cc26e89b43f1645 Signed-off-by: NeilBrown <neilb@suse.com> Reviewed-on: https://review.whamcloud.com/36339 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Shaun Tancheff <stancheff@cray.com> Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com> Reviewed-by: Amir Shehata <ashehata@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12923 lustre: Replace CLASSERT() with BUILD_BUG_ON() This patch replaces all CLASSERT() with kernel defined BUILD_BUG_ON() Test-Parameters: trivial Signed-off-by: Arshad Hussain <arshad.super@gmail.com> Change-Id: Ic21510ba4f1c99fa0ea6832d240d96ffc7622593 Reviewed-on: https://review.whamcloud.com/36725 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Shaun Tancheff <stancheff@cray.com> Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Neil Brown <neilb@suse.de>
LU-9679 modules: Use LIST_HEAD for declaring list_heads Rather than struct list_head foo = LIST_HEAD_INIT(foo); use LIST_HEAD(foo); This is shorter and more in-keeping with upstream style. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I36aa8c7e0763f3dfc88fe482cd28935184c1effa Reviewed-on: https://review.whamcloud.com/36669 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Ben Evans <bevans@cray.com> Reviewed-by: Shaun Tancheff <stancheff@cray.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11304 misc: update all url links to whamcloud Even old links could redirect to whamcloud automatically, We'd better update to use new whamcloud links to avoid any further confusions. Test-parameters: trivial Change-Id: Ida7161a062d822141bf0c1fdf20b2098a21ea9e7 Signed-off-by: Wang Shilong <wshilong@ddn.com> Reviewed-on: https://review.whamcloud.com/33094 Reviewed-by: Peter Jones <pjones@whamcloud.com> Tested-by: Jenkins Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <hpdd-maloo@intel.com>
LU-10308 misc: update Intel copyright messages for 2017 Update copyright messages for files updated in 2016, excluding trivial patches. Add trivial patches to updatecw.sh script exclude list. Revert some changes that were incorrectly attributed to the 2016 (d10200a80770f0029d1d665af954187b9ad883df) and 2015 (0754bc8f2623bea184111af216f7567608db35b6) copyright update patches themselves, since they were not in the exclude list when the subsequent script was run. Test-Parameters: trivial Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Change-Id: I82f21c30c4dac75792bb49fc139bee2ca51f5545 Reviewed-on: https://review.whamcloud.com/30341 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Jian Yu <jian.yu@intel.com> Reviewed-by: James Nunez <james.a.nunez@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-9480 lnet: add lnet_interfaces_max tunable Add an lnet_interfaces_max tunable value, that describes the maximum number of interfaces per node. This tunable is primarily useful for sanity checks prior to allocating memory. Allow lnet_interfaces_max to be set and get from the sysfs interface. Add LNET_INTERFACES_MIN, value 16, as the minimum value. Add LNET_INTERFACES_MAX_DEFAULT, value 200, as the default value. This value was chosen to ensure that the size of an LNet ping message with any associated LND overhead would fit in 4096 bytes. (The LNET_INTERFACES_MAX name was not reused to allow for the early detection of issues when merging code that uses it.) Rename LNET_NUM_INTERFACES to LNET_INTERFACES_NUM Test-Parameters: trivial Signed-off-by: Olaf Weber <olaf@sgi.com> Signed-off-by: Amir Shehata <amir.shehata@intel.com> Change-Id: I9bdc72cc688a414f7658fed93f84c9885c8342be Reviewed-on: https://review.whamcloud.com/25770