From 221fc0d10387414e27f9ed736f66ab233ef34eb3 Mon Sep 17 00:00:00 2001 From: Cory Spitz Date: Tue, 30 May 2017 17:18:37 -0500 Subject: [PATCH] LUDOC-235 ptllnd: Remove references to ptllnd LU-1422 (resolved for 2.6.0) removed the ptllnd from the source tree. References to the ptllnd should have been removed from the manual at that time. Signed-off-by: Cory Spitz Change-Id: I99d0d472c63993307eecc3306f8b7d75bd808647 Reviewed-on: https://review.whamcloud.com/27339 Tested-by: Jenkins Reviewed-by: Joseph Gmitter --- ConfigurationFilesModuleParameters.xml | 128 --------------------------------- ManagingSecurity.xml | 2 +- UnderstandingLustreNetworking.xml | 3 - 3 files changed, 1 insertion(+), 132 deletions(-) diff --git a/ConfigurationFilesModuleParameters.xml b/ConfigurationFilesModuleParameters.xml index e526884..56e94de 100644 --- a/ConfigurationFilesModuleParameters.xml +++ b/ConfigurationFilesModuleParameters.xml @@ -491,134 +491,6 @@ forwarding ("")
- <indexterm> - <primary>configuring</primary> - <secondary>portals</secondary> - </indexterm>Portals LND Linux (ptllnd) - The Portals LND Linux (ptllnd) can be used as a interface layer to communicate with Sandia Portals networking devices. This version is intended to work on Cray XT3 Linux nodes that use Cray Portals as a network transport. - Message Buffers - When ptllnd starts up, it allocates and posts sufficient message buffers to allow all expected peers (set by concurrent_peers) to send one unsolicited message. The first message that a peer actually sends is (so-called) "HELLO" message, used to negotiate how much additional buffering to setup (typically 8 messages). If 10000 peers actually exist, then enough buffers are posted for 80000 messages. - The maximum message size is set by the max_msg_size module parameter (default value is 512). This parameter sets the bulk transfer breakpoint. Below this breakpoint, payload data is sent in the message itself. Above this breakpoint, a buffer descriptor is sent and the receiver gets the actual payload. - The buffer size is set by the rxb_npages module parameter (default value is 1). The default conservatively avoids allocation problems due to kernel memory fragmentation. However, increasing this value to 2 is probably not risky. - The ptllnd also keeps an additional rxb_nspare buffers (default value is 8) posted to account for full buffers being handled. - Assuming a 4K page size with 10000 peers, 1258 buffers can be expected to be posted at startup, increasing to a maximum of 10008 as peers that are actually connected. By doubling rxb_npages halving max_msg_size, this number can be reduced by a factor of 4. - ME/MD Queue Length - The ptllnd uses a single portal set by the portal module parameter (default value of 9) for both message and bulk buffers. Message buffers are always attached with PTL_INS_AFTER and match anything sent with "message" matchbits. Bulk buffers are always attached with PTL_INS_BEFORE and match only specific matchbits for that particular bulk transfer. - This scheme assumes that the majority of ME/MDs posted are for "message" buffers, and that the overhead of searching through the preceding "bulk" buffers is acceptable. Since the number of "bulk" buffers posted at any time is also dependent on the bulk transfer breakpoint set by max_msg_size, this seems like an issue worth measuring at scale. - TX Descriptors - The ptllnd has a pool of so-called "tx descriptors", which it uses not only for outgoing messages, but also to hold state for bulk transfers requested by incoming messages. This pool should scale with the total number of peers. - To enable the building of the Portals LND (ptllnd.ko) configure with this option: - ./configure --with-portals=/path/to/portals/headers - - - - - - - - Variable - - - Description - - - - - - - ntx - (256) - - - Total number of messaging descriptors. - - - - - concurrent_peers - (1152) - - - Maximum number of concurrent peers. Peers that attempt to connect beyond the maximum are not allowed. - - - - - peer_hash_table_size - (101) - - - Number of hash table slots for the peers. This number should scale with concurrent_peers. The size of the peer hash table is set by the module parameter peer_hash_table_size which defaults to a value of 101. This number should be prime to ensure the peer hash table is populated evenly. It is advisable to increase this value to 1001 for ~10000 peers. - - - - - cksum - (0) - - - Set to non-zero to enable message (not RDMA) checksums for outgoing packets. Incoming packets are always check-summed if necessary, independent of this value. - - - - - timeout - (50) - - - Amount of time (in seconds) that a request can linger in a peers-active queue before the peer is considered dead. - - - - - portal - (9) - - - Portal ID to use for the ptllnd traffic. - - - - - rxb_npages - (64 * #cpus) - - - Number of pages in an RX buffer. - - - - - credits - (128) - - - Maximum total number of concurrent sends that are outstanding to a single peer at a given time. - - - - - peercredits - (8) - - - Maximum number of concurrent sends that are outstanding to a single peer at a given time. - - - - - max_msg_size - (512) - - - Maximum immediate message size. This MUST be the same on all nodes in a cluster. A peer that connects with a different max_msg_size value will be rejected. - - - - - -
-
<indexterm><primary>configuring</primary><secondary>MX LND</secondary></indexterm>MX LND MXLND supports a number of load-time parameters using Linux's module parameter system. The following variables are available: diff --git a/ManagingSecurity.xml b/ManagingSecurity.xml index 6035c7f..f83ca48 100644 --- a/ManagingSecurity.xml +++ b/ManagingSecurity.xml @@ -198,7 +198,7 @@ lctl get_param mdt.*.nosquash_nids <number> '-' <number> '/' <number> <net> :== <netname> | <netname><number> <netname> :== "lo" | "tcp" | "o2ib" | "cib" | "openib" | "iib" | - "vib" | "ra" | "elan" | "gm" | "mx" | "ptl" + "vib" | "ra" | "elan" | "gm" | "mx" <number> :== <nonnegative decimal> | <hexadecimal> For networks using numeric addresses (e.g. elan), the address range must be specified in the <numaddr_range> syntax. For networks using IP addresses, the address range must be in the <ipaddr_range>. For example, if elan is using numeric addresses, 1.2.3.4@elan is incorrect. diff --git a/UnderstandingLustreNetworking.xml b/UnderstandingLustreNetworking.xml index 4dd940c..27a65b1 100644 --- a/UnderstandingLustreNetworking.xml +++ b/UnderstandingLustreNetworking.xml @@ -109,9 +109,6 @@ TCP (any network carrying TCP traffic, including GigE, 10GigE, and IPoIB) - Cray: Seastar - - Myrinet: MX -- 1.8.3.1