Layer 3 etherchannels

Submitted by rayc on Sat, 11/13/2021 - 06:44

In my last few articles, I've talked about etherchannels and what they are and how they work as far as being a layer 2 etherchannels goes. But what if we want to load balance across layer 3 interfaces and route traffic? Well, Cisco routers and switches (MLS switches at least) support both Layer 2 and Layer 3 etherchannels. 

The concept of a layer 3 etherchannel is the same. The router or switch still bundle the physical interfaces into a single logical port-channel interface, and traffic is load balanced across the interfaces the same way as with a layer 2 etherchannel. The only difference is how the port operates. Instead of being an access or trunk port operating at layer 2, the port is configured as a layer 3 routed port and an ip address is assigned. 

Why would you want a layer 3 etherchannel? Well the same reason you would want a layer 2 etherchannel. Not just do you effectively double your bandwidth, you also add redundancy. Think of a large scale network with thousands of users and your gateway router has multiple Gigabit interfaces which you have connected to a firewall or internal switch. Utilising a single Gigabit Ethernet like like that may not be enough bandwidth for your users. This is where a layer 3 etherchannel can come in (you could also use routing protocols and equal cost load balancing but were going to use layer 3 etherchannels). You can port-channel those interfaces into a single logical interface, doubling your available bandwidth.

Cisco switches and routers support three etherchannel protocols for both layer 2 and layer 3 etherchannels. 

  • On: This is an always on etherchannel protocol. There is no negotiation or health checking. 
  • LACP: This is an industry standard etherchannel protocol. This protocol dynamically establishes an etherchannels and monitors the health of the bundle. 
  • PAgP: This is a Cisco proprietary etherchannel protocol. This protocol dynamical establishes an etherchannel and monitors the health of the bundle. 

Configuring L3 etherchannels

To show how Layer 3 etherchannels work, I will be using the link between the two 3650 MLS switches SW2 and SW3. 

Layer 3 Etherchannel topology

Configuring a layer 3 etherchannel is similar to creating a layer 2 etherchannel. You still need to configure the channel protocol and the channel group to assign the physical interfaces as member interfaces of the port-channel but with a layer 3 etherchannels, there are a couple of extra steps involved. 

Before configure the channel protocol or group with a layer 3 etherchannel, it is best practice to first create the port-channel interface and then convert it to a layer 3 routed port. To create the port-channel interface use the global configuration command interface Port-channel <num> where <num> is the same as the channel-group number you are assigning to the physical member ports. To convert the port-channel interface to a layer 3 routed port, use the interface configuration subcommand no switchport

Configuring the port-channel interface as a routed port

You can verify that the port is a layer 3 routed port by using the show command show interface status. 

verifying that the port-channel interface is in routed mode

Once the port-channel is in routed mode, you can then assign the IP address. In this lab, I will use the subnet 10.1.99.0/30 with SW2 having the .1 IP address, and SW3 having the .2 IP address. To configuring the IP address, use the port-channel interface subconfiguration command ip address <ip> <mask>.

configuring Po2 with the IP address 10.1.99.1/30

Configuring layer 3 On etherchannel

Now that we have configured and confirmed out layer 3 interfaces, we can configure the protocol and channel-group. Because this first example uses to on etherchannel configuration, there is no need to configure a protocol so we can straight away configure the interfaces to be part of the etherchannel. The first thing we must do is convert the physical ports to routed ports, using the no switchport command, and then assign the ports to the etherchannel using the command channel-group <num> mode on where <num> refers to the previously configure port-channel interface number.

Configuring physical ports for On mode etherchannel

If we take a look at the output of the show etherchannel summary command we can see that the interface is now a layer 3 etherchannel are both up and bundled. Remember, there is no negotiation of etherchannels with on mode, the etherchannel is simply considered as up.

output of show etherchannel summary on SW2 without SW3 being configured

Let's go ahead and configure SW3 and confirm that the channel comes up on both ends, and the other side is reachable using ping. 

Configuring SW3 and confirming ping works

Configuring layer 3 LACP etherchannel

Configuring an LACP LAyer 3 etherchannel is the exact same process. First you should configure the port-channel interface, convert it to a routed port and assign the IP address which we did in the previous step. To configure the port for LACP, configure the channel-protocol as LACP, and configure the channel-group to be the same as the port-channel interface number. I'm not going to go over how to configure the port-channel again as I will just re-use the same port-channel configuration as with the On example. I've removed the On configuration commands, so now I will configure the ports as LACP using the command channel-protocol lacp to tell the switch to only allow LACP commands, and assign the ports to the etherchannel using the command channel-group <num> mode <active|passive>. For this, I will configure SW3 as passive, and SW2 as Active. 

SW2 LACP configuration without SW3 configured

You can see that the etherchannel is a Routed etherchannel and the interfaces are in the w, waiting to be aggregated state. Let's go ahead and configure SW3 now and confirm we have Layer 3 reachability. 

Configuring SW3 for L3 LACP etherchannel

Notice that the ports are in the w waiting for aggragation state as well. It actually took a few seconds for the etherchannel to become active and the ports were in a suspended stated at one stage. 

SW3 showing etherchannel port states

Now let's confirm we have connectivity across the L3 Etherchannel.

Confirming layer 3 connectivity over LACP etherchannel

Configuring layer 3 PAgP etherchannel

Now let's change our etherchannel bundle to PAgP. The concept is the same and so is the configuration. I've removed the LACP configuration from the switches but left the switchport configuration and the port-channel interface configuration as well. To configure PAgP let's first configure the switch to use PAgP as the protocol using the interface configuration command channel-protocol pagp, and then configure the physical interfaces to be members of the port-channel 5 interface using the command channel-group 5 mode desirable. SW3 will be configured as auto.

Configuring PAgP on SW2

From the output you can see that the ports automatically go into a suspended state due to PAgP not being configured on the remote switch (SW3). Let's configure SW3 and check the status of the etherchannel.

SW3 PAgP configuration

When I configured the port-channel at first the port state went to suspended. This was due to the port being in a suspended state on SW2 however, the switch will regularly check the remote end so once SW2 sends it's next PAgP packet the etherchannel will begin to negotiate.

Let's confirm we have Layer 3 connectivity by pinging SW3 from SW2.

confirm layer 3 connectivity with ping