Nathan Stratton’s Homepage

It’s taken me a while to get dracut PXE Boot working with bonded interfaces, so I wanted to take a moment and share.

My setup is as follows, 20 servers with dual gig ethernets connected to two Cisco 3750 switches connected togeter in a ring. The first ethernet, eth1 from each server are all connected to swich 1 with the 2nd ethernet, eth2 all connected to the 2nd switch. The ring configuration allows the switches to look like one larger switch, providing redundancy while still allowing for things like trunks spanning more then one switch.

Switch Configuration

The cisco 3750 is configured as follows:

interface Port-channel1
 description virt1
 switchport trunk encapsulation dot1q

interface GigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 1 mode passive

interface GigabitEthernet2/0/1
 switchport trunk encapsulation dot1q
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 1 mode passive

The above config first sets up a port-channel, a bonded interface and sets the encapsulation to dot1q, the standard that allows VLAN tagging. Two interfaces are then configured I set the speed, duplex, and spanning-tree portfast to help speed up port setup time. The ports are both configured to used standared lacp and are both made part of the port-channel interface with the channel-group 1 mode passive command. The mode passive is important it does not setup the ports into the trunk group until the other end (our server) brings up the LACP trunk. This allows the server to do standard PXE Boot with DHCP and TFTP on the standard interface rather then failing because it was in trunk mode.

Dracut Configuration

Dracut allows you to boot a server with as little as possible hard-coded into the initramfs. To make the image I typed:

dracut dracut.img 3.2.7-1.fc16.x86_64
dracut –add-drivers bonding -f dracut.img

The first line builds the image and the 2nd line adds bonding support to the image, note that the kernel name is important, you can pull that with uname -r. The Dracut configuration lives on the tftpserver in the pxelinux.cfg/default file. Mine looks like:

prompt 1
default Fedora-16_3.2.7-1.fc16.x86_64
timeout 10
serial 0 115200
console 0

label Fedora-16_3.2.7-1.fc16.x86_64
kernel vmlinuz-3.2.7-1.fc16.x86_64
append initrd=dracut.img root=10.10.0.4:/diskless/Fedora16_020303 console=ttyS0,115200 biosdevname=0 bond=bond0:eth0,eth1:mode=4 bridge=ovirtmgmt:bond0 ip=ovirtmgmt:dhcp

This file configures a serial console on the first serial port as a speed of 115,200, it passes to the tftpserver the kernel file with the dracut configuration. A breakdown of the dracut line is as follows:

initrd=dracut.img                                                                         This is the name of my dracut image.
root=10.10.0.4:/diskless/Fedora16_020303                          NFS IP and path for the root image.
console=ttys0,115200                                                                Sets the serial device and speed.
biosdevname=0                                                                           Keeps the old eth naming scheem.
bond=bond0:eth0,eth1:mode=4                                               Bonds eth0 and eth1 using mode4.
bridge=ovirtmgmt:bond0                                                            Creates bridge ovirtmgmt attached to bond0.
ip=ovirtmgmt:dhcp                                                                       Run DHCP on ovirtmgmt interface.

Now the problme….

So far we have a setup that will correctly DHCP and PXE Boot, the server will have access to Vlan 1, but not the other VLANs, this is because the switch LACP port is not yet running as a trunk. Cisco can do this automatically if there is a cisco on the other end via cisco proprietary protocol, but the Linux box does not support this. To get around this problem and still PXE Boot boot we have a script that adds “switchport mode trunk” to the interface Port-Channel. Once this is done you will be able to talk on all the VLANs you have setup. This is an ugly hack, but so far is the only way I have found to have a cisco work in this setup.

Be Sociable, Share!

•