Nathan Stratton’s Homepage

Dracut PXE Boot with bonded interfaces

by on Mar.07, 2012, under Software

It’s taken me a while to get dracut PXE Boot working with bonded interfaces, so I wanted to take a moment and share.

My setup is as follows, 20 servers with dual gig ethernets connected to two Cisco 3750 switches connected togeter in a ring. The first ethernet, eth1 from each server are all connected to swich 1 with the 2nd ethernet, eth2 all connected to the 2nd switch. The ring configuration allows the switches to look like one larger switch, providing redundancy while still allowing for things like trunks spanning more then one switch.

Switch Configuration

The cisco 3750 is configured as follows:

interface Port-channel1
 description virt1
 switchport trunk encapsulation dot1q

interface GigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 1 mode passive

interface GigabitEthernet2/0/1
 switchport trunk encapsulation dot1q
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 1 mode passive

The above config first sets up a port-channel, a bonded interface and sets the encapsulation to dot1q, the standard that allows VLAN tagging. Two interfaces are then configured I set the speed, duplex, and spanning-tree portfast to help speed up port setup time. The ports are both configured to used standared lacp and are both made part of the port-channel interface with the channel-group 1 mode passive command. The mode passive is important it does not setup the ports into the trunk group until the other end (our server) brings up the LACP trunk. This allows the server to do standard PXE Boot with DHCP and TFTP on the standard interface rather then failing because it was in trunk mode.

Dracut Configuration

Dracut allows you to boot a server with as little as possible hard-coded into the initramfs. To make the image I typed:

dracut dracut.img 3.2.7-1.fc16.x86_64
dracut –add-drivers bonding -f dracut.img

The first line builds the image and the 2nd line adds bonding support to the image, note that the kernel name is important, you can pull that with uname -r. The Dracut configuration lives on the tftpserver in the pxelinux.cfg/default file. Mine looks like:

prompt 1
default Fedora-16_3.2.7-1.fc16.x86_64
timeout 10
serial 0 115200
console 0

label Fedora-16_3.2.7-1.fc16.x86_64
kernel vmlinuz-3.2.7-1.fc16.x86_64
append initrd=dracut.img root=10.10.0.4:/diskless/Fedora16_020303 console=ttyS0,115200 biosdevname=0 bond=bond0:eth0,eth1:mode=4 bridge=ovirtmgmt:bond0 ip=ovirtmgmt:dhcp

This file configures a serial console on the first serial port as a speed of 115,200, it passes to the tftpserver the kernel file with the dracut configuration. A breakdown of the dracut line is as follows:

initrd=dracut.img                                                                         This is the name of my dracut image.
root=10.10.0.4:/diskless/Fedora16_020303                         
NFS IP and path for the root image.
console=ttys0,115200                                                                Sets the serial device and speed.
biosdevname=0                                                                           Keeps the old eth naming scheem.
bond=bond0:eth0,eth1:mode=4                                               Bonds eth0 and eth1 using mode4.
bridge=ovirtmgmt:bond0
                                                           Creates bridge ovirtmgmt attached to bond0.
ip=ovirtmgmt:dhcp                                                                       Run DHCP on ovirtmgmt interface.

Now the problme….

So far we have a setup that will correctly DHCP and PXE Boot, the server will have access to Vlan 1, but not the other VLANs, this is because the switch LACP port is not yet running as a trunk. Cisco can do this automatically if there is a cisco on the other end via cisco proprietary protocol, but the Linux box does not support this. To get around this problem and still PXE Boot boot we have a script that adds “switchport mode trunk” to the interface Port-Channel. Once this is done you will be able to talk on all the VLANs you have setup. This is an ugly hack, but so far is the only way I have found to have a cisco work in this setup.

Be Sociable, Share!

2 Comments for this entry

  • Chris C

    Hi Nathan,
    You are missing two options.

    First your Cisco ports need to be configure like this
    interface GigabitEthernet1/2/3
    switchport
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 10-100
    switchport mode trunk native vlan X
    switchport nonegotiate
    no ip address
    spanning-tree portfast trunk
    Where X is the vlan that your dhcp server lives. This allows untagged broadcast packets (dhcp)to be sent to vlan X and the dhcp server.

    Second your linux boot options have interfaces enslaved to a bond and bridge and dhcp is set on the network interface but there isn’t a vlan id specified. Add this to your boot options
    vlanid=X
    Where X is the vlan that your dhcp server lives

    Post an update to tell us if you got it working.

  • Chris C

    Correction:
    interface GigabitEthernet1/2/3
    switchport
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 10-100
    switchport mode trunk native vlan X
    switchport nonegotiate
    no ip address
    spanning-tree portfast trunk
    channel-group 1 mode on

1 Trackback or Pingback for this entry

Leave a Reply

ERROR: si-captcha.php plugin says GD image support not detected in PHP!

Contact your web host and ask them why GD image support is not enabled for PHP.

ERROR: si-captcha.php plugin says imagepng function not detected in PHP!

Contact your web host and ask them why imagepng function is not enabled for PHP.

Looking for something?

Use the form below to search the site:

Cool Links!

A few highly recommended links...