<?xml version="1.0" encoding="UTF-8"?>
-<chapter version="5.0" xml:lang="en-US" xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink">
+<chapter version="5.0" xml:lang="en-US" xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" xml:id='settingupbonding'>
<info>
<title xml:id='settingupbonding.title'>Setting Up Network Interface Bonding</title>
</info>
+
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301676" xreflabel=""/>This chapter describes how to use multiple network interfaces in parallel to increase bandwidth and/or redundancy. Topics include:</para>
- <itemizedlist><listitem>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301677" xreflabel=""/><link xl:href="SettingUpBonding.html#50438258_95824">Network Interface Bonding Overview</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301970" xreflabel=""/><link xl:href="SettingUpBonding.html#50438258_29107">Requirements</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301979" xreflabel=""/><link xl:href="SettingUpBonding.html#50438258_25353">Bonding Module Parameters</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301983" xreflabel=""/><link xl:href="SettingUpBonding.html#50438258_26899">Setting Up Bonding</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301987" xreflabel=""/><link xl:href="SettingUpBonding.html#50438258_54769">Configuring Lustre with Bonding</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-<listitem>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301991" xreflabel=""/><link xl:href="SettingUpBonding.html#50438258_92244">Bonding References</link></para>
- </listitem>
-<listitem>
- <para> </para>
- </listitem>
-</itemizedlist>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438258_pgfId-1302006" xreflabel=""/>Using network interface bonding is optional.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
- <section remap="h2">
- <title><anchor xml:id="dbdoclet.50438258_pgfId-1301679" xreflabel=""/></title>
- <section remap="h2">
- <title>7.1 <anchor xml:id="dbdoclet.50438258_95824" xreflabel=""/>Network Interface <anchor xml:id="dbdoclet.50438258_marker-1301678" xreflabel=""/>Bonding Overview</title>
+
+ <itemizedlist>
+ <listitem>
+ <para>
+ <xref linkend='dbdoclet.50438258_95824'/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend='dbdoclet.50438258_29107'/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend='dbdoclet.50438258_25353'/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend='dbdoclet.50438258_26899'/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend='dbdoclet.50438258_54769'/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend='dbdoclet.50438258_92244'/>
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <note><para>Using network interface bonding is optional.</para></note>
+
+
+ <section xml:id="dbdoclet.50438258_95824">
+ <title>7.1 Network Interface Bonding Overview</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301680" xreflabel=""/>Bonding, also known as link aggregation, trunking and port trunking, is a method of aggregating multiple physical network links into a single logical link for increased bandwidth.</para>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301681" xreflabel=""/>Several different types of bonding are available in Linux. All these types are referred to as “modes,†and use the bonding kernel module.</para>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301682" xreflabel=""/>Modes 0 to 3 allow load balancing and fault tolerance by using multiple interfaces. Mode 4 aggregates a group of interfaces into a single virtual interface where all members of the group share the same speed and duplex settings. This mode is described under IEEE spec 802.3ad, and it is referred to as either “mode 4†or “802.3ad.â€</para>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301681" xreflabel=""/>Several different types of bonding are available in Linux. All these types are referred to as 'modes', and use the bonding kernel module.</para>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301682" xreflabel=""/>Modes 0 to 3 allow load balancing and fault tolerance by using multiple interfaces. Mode 4 aggregates a group of interfaces into a single virtual interface where all members of the group share the same speed and duplex settings. This mode is described under IEEE spec 802.3ad, and it is referred to as either 'mode 4' or '802.3ad.'</para>
</section>
- <section remap="h2">
- <title>7.2 <anchor xml:id="dbdoclet.50438258_29107" xreflabel=""/>Require<anchor xml:id="dbdoclet.50438258_marker-1301684" xreflabel=""/>ments</title>
+ <section xml:id="dbdoclet.50438258_29107">
+ <title>7.2 Requirements</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301686" xreflabel=""/>The most basic requirement for successful bonding is that both endpoints of the connection must be capable of bonding. In a normal case, the non-server endpoint is a switch. (Two systems connected via crossover cables can also use bonding.) Any switch used must explicitly handle 802.3ad Dynamic Link Aggregation.</para>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301687" xreflabel=""/>The kernel must also be configured with bonding. All supported Lustre kernels have bonding functionality. The network driver for the interfaces to be bonded must have the ethtool functionality to determine slave speed and duplex settings. All recent network drivers implement it.</para>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301688" xreflabel=""/>To verify that your interface works with ethtool, run:</para>
<anchor xml:id="dbdoclet.50438258_pgfId-1301735" xreflabel=""/> /sbin/ifenslave
</screen>
</section>
- <section remap="h2">
- <title>7.3 <anchor xml:id="dbdoclet.50438258_25353" xreflabel=""/>Bonding <anchor xml:id="dbdoclet.50438258_marker-1301747" xreflabel=""/>Module Parameters</title>
+ <section xml:id="dbdoclet.50438258_25353">
+ <title>7.3 Bonding Module Parameters</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301749" xreflabel=""/>Bonding module parameters control various aspects of bonding.</para>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301750" xreflabel=""/>Outgoing traffic is mapped across the slave interfaces according to the transmit hash policy. For Lustre, we recommend that you set the xmit_hash_policy option to the layer3+4 option for bonding. This policy uses upper layer protocol information if available to generate the hash. This allows traffic to a particular network peer to span multiple slaves, although a single connection does not span multiple slaves.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301751" xreflabel=""/>$ xmit_hash_policy=layer3+4
</screen>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301754" xreflabel=""/>For a busy network, increase the timeout.</para>
</section>
- <section remap="h2">
- <title>7.4 <anchor xml:id="dbdoclet.50438258_26899" xreflabel=""/><anchor xml:id="dbdoclet.50438258_26702" xreflabel=""/>Setting Up <anchor xml:id="dbdoclet.50438258_marker-1301757" xreflabel=""/>Bonding</title>
+ <section xml:id="dbdoclet.50438258_26899">
+ <title>7.4 Setting Up Bonding</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301759" xreflabel=""/>To set up bonding:</para>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301760" xreflabel=""/> 1. Create a virtual 'bond' interface by creating a configuration file in:</para>
+
+ <orderedlist>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301760" xreflabel=""/>Create a virtual 'bond' interface by creating a configuration file in:</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301761" xreflabel=""/>/etc/sysconfig/network-scripts/ # vi /etc/sysconfig/ network-scripts/ifcfg-\
bond0
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301762" xreflabel=""/> 2. Append the following lines to the file.</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301762" xreflabel=""/>Append the following lines to the file.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301763" xreflabel=""/>DEVICE=bond0
<anchor xml:id="dbdoclet.50438258_pgfId-1301764" xreflabel=""/>IPADDR=192.168.10.79 # Use the free IP Address of your network
<anchor xml:id="dbdoclet.50438258_pgfId-1301765" xreflabel=""/>NETWORK=192.168.10.0
<anchor xml:id="dbdoclet.50438258_pgfId-1301768" xreflabel=""/>BOOTPROTO=none
<anchor xml:id="dbdoclet.50438258_pgfId-1301769" xreflabel=""/>ONBOOT=yes
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301770" xreflabel=""/> 3. Attach one or more slave interfaces to the bond interface. Modify the eth0 and eth1 configuration files (using a VI text editor).</para>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301771" xreflabel=""/>a. Use the VI text editor to open the eth0 configuration file.</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301770" xreflabel=""/> Attach one or more slave interfaces to the bond interface. Modify the eth0 and eth1 configuration files (using a VI text editor).</para>
+ <orderedlist>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301771" xreflabel=""/>Use the VI text editor to open the eth0 configuration file.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301772" xreflabel=""/># vi /etc/sysconfig/network-scripts/ifcfg-eth0
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301773" xreflabel=""/>b. Modify/append the eth0 file as follows:</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301773" xreflabel=""/>Modify/append the eth0 file as follows:</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301774" xreflabel=""/>DEVICE=eth0
<anchor xml:id="dbdoclet.50438258_pgfId-1301775" xreflabel=""/>USERCTL=no
<anchor xml:id="dbdoclet.50438258_pgfId-1301776" xreflabel=""/>ONBOOT=yes
<anchor xml:id="dbdoclet.50438258_pgfId-1301778" xreflabel=""/>SLAVE=yes
<anchor xml:id="dbdoclet.50438258_pgfId-1301779" xreflabel=""/>BOOTPROTO=none
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301780" xreflabel=""/>c. Use the VI text editor to open the eth1 configuration file.</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301780" xreflabel=""/>Use the VI text editor to open the eth1 configuration file.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301781" xreflabel=""/># vi /etc/sysconfig/network-scripts/ifcfg-eth1
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301782" xreflabel=""/>d. Modify/append the eth1 file as follows:</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301782" xreflabel=""/>Modify/append the eth1 file as follows:</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301783" xreflabel=""/>DEVICE=eth1
<anchor xml:id="dbdoclet.50438258_pgfId-1301784" xreflabel=""/>USERCTL=no
<anchor xml:id="dbdoclet.50438258_pgfId-1301785" xreflabel=""/>ONBOOT=yes
<anchor xml:id="dbdoclet.50438258_pgfId-1301787" xreflabel=""/>SLAVE=yes
<anchor xml:id="dbdoclet.50438258_pgfId-1301788" xreflabel=""/>BOOTPROTO=none
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301789" xreflabel=""/> 4. Set up the bond interface and its options in /etc/modprobe.conf. Start the slave interfaces by your normal network method.</para>
+ </listitem>
+ </orderedlist>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301789" xreflabel=""/>Set up the bond interface and its options in /etc/modprobe.conf. Start the slave interfaces by your normal network method.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301790" xreflabel=""/># vi /etc/modprobe.conf
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301791" xreflabel=""/>a. Append the following lines to the file.</para>
+ <orderedlist>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301791" xreflabel=""/>Append the following lines to the file.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301792" xreflabel=""/>alias bond0 bonding
<anchor xml:id="dbdoclet.50438258_pgfId-1301793" xreflabel=""/>options bond0 mode=balance-alb miimon=100
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301794" xreflabel=""/>b. Load the bonding module.</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301794" xreflabel=""/>Load the bonding module.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301795" xreflabel=""/># modprobe bonding
<anchor xml:id="dbdoclet.50438258_pgfId-1301796" xreflabel=""/># ifconfig bond0 up
<anchor xml:id="dbdoclet.50438258_pgfId-1301797" xreflabel=""/># ifenslave bond0 eth0 eth1
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301798" xreflabel=""/> 5. Start/restart the slave interfaces (using your normal network method).</para>
- <informaltable frame="none">
- <tgroup cols="1">
- <colspec colname="c1" colwidth="100*"/>
- <tbody>
- <row>
- <entry><para><emphasis role="bold">Note -</emphasis><anchor xml:id="dbdoclet.50438258_pgfId-1301799" xreflabel=""/>You must modprobe the bonding module for each bonded interface. If you wish to create bond0 and bond1, two entries in modprobe.conf are required.</para></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>
+ </listitem>
+ </orderedlist>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301798" xreflabel=""/>Start/restart the slave interfaces (using your normal network method).</para>
+
+ <note>
+ <para>You must modprobe the bonding module for each bonded interface. If you wish to create bond0 and bond1, two entries in modprobe.conf are required.</para></note>
+
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301800" xreflabel=""/>The examples below are from RedHat systems. For setup use: /etc/sysconfig/networking-scripts/ifcfg-* The website referenced below includes detailed instructions for other configuration methods, instructions to use DHCP with bonding, and other setup details. We strongly recommend you use this website.</para>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1302123" xreflabel=""/><link xl:href="http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding">http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding</link></para>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301803" xreflabel=""/> 6. Check /proc/net/bonding to determine status on bonding. There should be a file there for each bond interface.</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301803" xreflabel=""/>Check /proc/net/bonding to determine status on bonding. There should be a file there for each bond interface.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301804" xreflabel=""/># cat /proc/net/bonding/bond0
<anchor xml:id="dbdoclet.50438258_pgfId-1301805" xreflabel=""/>Ethernet Channel Bonding Driver: v3.0.3 (March 23, 2006)
<anchor xml:id="dbdoclet.50438258_pgfId-1301806" xreflabel=""/>
<anchor xml:id="dbdoclet.50438258_pgfId-1301820" xreflabel=""/>Link Failure Count: 0
<anchor xml:id="dbdoclet.50438258_pgfId-1301821" xreflabel=""/>Permanent HW addr: 00:14:2a:7c:40:1d
</screen>
- <para><anchor xml:id="dbdoclet.50438258_pgfId-1301822" xreflabel=""/> 7. Use ethtool or ifconfig to check the interface state. ifconfig lists the first bonded interface as “bond0.â€</para>
+ </listitem>
+ <listitem>
+ <para><anchor xml:id="dbdoclet.50438258_pgfId-1301822" xreflabel=""/>Use ethtool or ifconfig to check the interface state. ifconfig lists the first bonded interface as 'bond0.'</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301823" xreflabel=""/>ifconfig
<anchor xml:id="dbdoclet.50438258_pgfId-1301824" xreflabel=""/>bond0 Link encap:Ethernet HWaddr 4C:00:10:AC:61:E0
<anchor xml:id="dbdoclet.50438258_pgfId-1301825" xreflabel=""/> inet addr:192.168.10.79 Bcast:192.168.10.255 \ Mask:255.255.255.0
<anchor xml:id="dbdoclet.50438258_pgfId-1301848" xreflabel=""/> RX bytes:152299 (148.7 KiB) TX bytes:64517 (63.0 KiB)
<anchor xml:id="dbdoclet.50438258_pgfId-1301849" xreflabel=""/> Interrupt:185 Base address:0x6000
</screen>
+ </listitem>
+ </orderedlist>
+
+
<section remap="h3">
<title><anchor xml:id="dbdoclet.50438258_pgfId-1301850" xreflabel=""/>7.4.1 Examples</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301851" xreflabel=""/>This is an example showing modprobe.conf entries for bonding Ethernet interfaces eth1 and eth2 to bond0:</para>
</screen>
</section>
</section>
- <section remap="h2">
- <title>7.5 <anchor xml:id="dbdoclet.50438258_54769" xreflabel=""/>Configuring <anchor xml:id="dbdoclet.50438258_marker-1301911" xreflabel=""/>Lustre with Bonding</title>
+ <section xml:id="dbdoclet.50438258_54769">
+ <title>7.5 Configuring Lustre with Bonding</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301913" xreflabel=""/>Lustre uses the IP address of the bonded interfaces and requires no special configuration. It treats the bonded interface as a regular TCP/IP interface. If needed, specify bond0 using the Lustre networks parameter in /etc/modprobe.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301914" xreflabel=""/>options lnet networks=tcp(bond0)
</screen>
</section>
- <section remap="h2">
- <title>7.6 <anchor xml:id="dbdoclet.50438258_92244" xreflabel=""/>Bonding <anchor xml:id="dbdoclet.50438258_marker-1301915" xreflabel=""/>References</title>
+ <section xml:id="dbdoclet.50438258_92244">
+ <title>7.6 Bonding References</title>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301917" xreflabel=""/>We recommend the following bonding references:</para>
<itemizedlist><listitem>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301918" xreflabel=""/> In the Linux kernel source tree, see documentation/networking/bonding.txt</para>
</listitem>
<listitem>
- <para> </para>
- </listitem>
-<listitem>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301920" xreflabel=""/><link xl:href="http://linux-ip.net/html/ether-bonding.html">http://linux-ip.net/html/ether-bonding.html</link></para>
</listitem>
<listitem>
- <para> </para>
- </listitem>
-<listitem>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301922" xreflabel=""/><link xl:href="http://www.sourceforge.net/projects/bonding">http://www.sourceforge.net/projects/bonding</link></para>
</listitem>
<listitem>
- <para> </para>
- </listitem>
-<listitem>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301923" xreflabel=""/> Linux Foundation bonding website: <link xl:href="http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding">http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding</link></para>
</listitem>
-<listitem>
- <para> </para>
- </listitem>
</itemizedlist>
<para><anchor xml:id="dbdoclet.50438258_pgfId-1301924" xreflabel=""/>This is the most extensive reference and we highly recommend it. This website includes explanations of more complicated setups, including the use of DHCP with bonding.</para>
<screen><anchor xml:id="dbdoclet.50438258_pgfId-1301673" xreflabel=""/>
</screen>
</section>
- </section>
</chapter>
git://git.whamcloud.com / doc / manual.git / commitdiff