CAM, TCAM and SDM

Submitted by rayc on Mon, 10/25/2021 - 09:10

The CAM Table

As mentioned in my previous post, switches are in their basic form smart hubs. I say smart because they are capable of learning information to aid in forwarding frames. Switches learn the MAC (Media Access Control) address of devices and store them in a table so that frames destined for that device, only get sent out the required port. These MAC addresses are stored in a fast memory called CAM (Content Accessible Memory). The CAM table is a fast lookup method because all of the information is stored ready for the router to use. The CAM table allows for one of 2 responses when doing a lookup, a 1 (True) or a 0 (False). This is why the CAM table is good for looking up exact matches such as MAC addresses.

As a frame is received on a port, the source MAC address, source VLAN, the Interface the frame was received and the time the frame was received, is all recorded in the CAM table. If a frame arrives at the switch from the same source MAC and interface, only the timestamp is updated. If the source MAC address is found on a different port, a new entry is created and the old one deleted. Switches are capable of storing a large number of MAC addresses inside the CAM table however it is not possible to store all information on some large networks. By default CAM entries are removed from the CAM table if no frames have been received from the specified source MAC in 300 seconds (5 Minutes). It may be necessary in some cases to change this time. This can be done using the global configuration command mac address-table aging-time <seconds>.

SWITCH3(config)#mac address-table aging-time ?
  <0-0>         Enter 0 to disable aging
  <10-1000000>  Aging time in seconds

SWITCH3(config)#mac address-table aging-time 500 

MAC addresses can also be configured statically on a port using the global configuration command mac address-table static <mac> vlan <id> interface <int>.

SWITCH3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SWITCH3(config)#$-table static 0123.1234.abcd vlan 20 interface g1/0/20      
SWITCH3(config)#

You can view the CAM table entries by using the command show mac address-table [dynamic|static]. (On some switches this command might be show mac-address-table)

SWITCH3#show mac address-table
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
 All    0100.0ccc.cccc    STATIC      CPU
 All    0100.0ccc.cccd    STATIC      CPU
 All    0180.c200.0000    STATIC      CPU
 All    0180.c200.0001    STATIC      CPU
 All    0180.c200.0002    STATIC      CPU
 All    0180.c200.0003    STATIC      CPU
 All    0180.c200.0004    STATIC      CPU
 All    0180.c200.0005    STATIC      CPU
 All    0180.c200.0006    STATIC      CPU
 All    0180.c200.0007    STATIC      CPU
 All    0180.c200.0008    STATIC      CPU
 All    0180.c200.0009    STATIC      CPU
 All    0180.c200.000a    STATIC      CPU
 All    0180.c200.000b    STATIC      CPU
 All    0180.c200.000c    STATIC      CPU
 All    0180.c200.000d    STATIC      CPU
 All    0180.c200.000e    STATIC      CPU
 All    0180.c200.000f    STATIC      CPU
 All    0180.c200.0010    STATIC      CPU
 All    0180.c200.0021    STATIC      CPU
 All    ffff.ffff.ffff    STATIC      CPU
   1    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  10    2c4f.52e6.aac6    STATIC      Vl10 
  10    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  20    0123.1234.abcd    STATIC      Gi1/0/20 
  20    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  20    f092.1c5a.b430    DYNAMIC     Gi1/0/46
  30    40a6.e88c.0483    DYNAMIC     Gi1/0/1
Total Mac Addresses for this criterion: 28

Here we can see our static entry that we just created and all of the dynamic entries that have been learned. There are various other show command options to use with the show mac address-table to view more detailed information about a CAM table entry. For example, to view only dynamic entries, use the keyword dynamic.

SWITCH3#show mac address-table dynamic
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
   1    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  10    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  20    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  20    f092.1c5a.b430    DYNAMIC     Gi1/0/46
  30    40a6.e88c.0483    DYNAMIC     Gi1/0/1
Total Mac Addresses for this criterion: 5

The same can be used for static entries. You can also view a specific entry by using the options address <mac>.

SWITCH3#show mac address-table addres  40a6.e88c.0483
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
   1    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  10    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  20    40a6.e88c.0483    DYNAMIC     Gi1/0/1
  30    40a6.e88c.0483    DYNAMIC     Gi1/0/1
Total Mac Addresses for this criterion: 4

TCAM (Ternary CAM)

TCAM or Ternary CAM is used by Multi Layer Switches (MLS) and modern Cisco Routers. MLS's are switches that operate at layer 3 of the OSI model and are capable of routing packets and other layer 3 functions like ACLs and QoS. MLS and modern Routers are capable of high speed data throughput because the packet switching function is done in hardware called ASICs (Application Specific Integrated Circuits) and Network Processing Units (NPU). These devices utilise a special kind of memory called TCAM. TCAM is called Ternary CAM as it allows for a third response (Ternary) of X (don't care). TCAM entries are stored in a special format of Value, Mask, Response (VMR). 

  • Value: Indicates fields to be searched such as IP address, Protoccol etc.
  • Mask: Infidcates the fields that are of interest and should be queried. 
  • Result: Indicates the action that should be take. 

 

There are 2 components to the TCAM operations. The Feature Manager, and the Switching Database Manager (SDM).

  • Feature Manager: Responsible for creating TCAM entries from the configured ACLs, QoS entries or Routes.
  • Switching Database Manager: Responsible for partitioning the TCAM tables to support various functions. 

 

For example, say you configure an ACL on a switch of the following

access-list 100 permit tcp host 192.168.199.14 10.41.0.0 0.0.255.255 eq telnet
access-list 100 permit ip any 192.168.100.0 0.0.0.255
access-list 100 deny udp any 192.168.5.0 0.0.0.255 gt 1024
access-list 100 deny udp any 192.168.199.0 0.0.0.255 range 1024 2047

The TCAM table would look like this

Example TCAM table

In this example, the 3 unique masks (255.255.255.255, 255.255.0.0 and 255.255.255.0) are placed into the TCAM table and all possible values are identified. Each mask entry can only have up to 8 pattern entries in the TCAM table. After that a new pattern entry is created with the same mask. When listing ports in the TCAM table, if the value is for a range of ports, the data is stored in the Logical Operation Unit (LOU) register and is referenced from there. One thing to note is that there are a limited number of entries available for both the TCAM and the LOU. If the LOU is full then the Feature Manager must break up the ACE's into multiple ACEs and use regular eq matching for the port. 

You can view the switches current TCAM utilisation using the command show platform tcam utilization on IOS platforms. 

SWITCH1#show platform tcam utilization 

CAM Utilization for ASIC# 0                      Max            Used
                                             Masks/Values    Masks/values

 Unicast mac addresses:                      32988/32988        20/20    
 IPv4 IGMP groups + multicast routes:         1072/1072          1/1     
 IPv4 unicast directly-connected routes:      2048/2048          0/0     
 IPv4 unicast indirectly-connected routes:    1024/1024         34/34    
 IPv6 Multicast groups:                       1072/1072         11/11    
 IPv6 unicast directly-connected routes:      2048/2048          0/0     
 IPv6 unicast indirectly-connected routes:    1024/1024          3/3     
 IPv4 policy based routing aces:               504/504          14/14    
 IPv4 qos aces:                                504/504          51/51    
 IPv4 security aces:                           600/600          76/76    
 IPv6 policy based routing aces:                20/20            8/8     
 IPv6 qos aces:                                500/500          44/44    
 IPv6 security aces:                           600/600          18/18    

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

And using the command show platform hardware fed switch 1 fwd-asic resource tcam utilization on IOS-XE.

SWITCH3#$rm hardware fed switch 1 fwd-asic resource tcam utilization         
CAM Utilization for ASIC  [0]
 Table                                              Max Values        Used Values
 --------------------------------------------------------------------------------
 Unicast MAC addresses                              32768/512          22/21  
 L3 Multicast entries                               4096/512           0/7   
 L2 Multicast entries                               4096/512           0/9   
 Directly or indirectly connected routes            16384/7168          7/20  
 QoS Access Control Entries                         2560                88
 Security Access Control Entries                    3072               125
 Netflow ACEs                                        768                15
 Policy Based Routing ACEs                          1024                 9
 Flow SPAN ACEs                                      512                 5
 Output Flow SPAN ACEs                               512                 8
 Control Plane Entries                               512               212
 Tunnels                                             256                17
 Lisp Instance Mapping Entries                       256                 3
 Input Security Associations                         256                 4
 Output Security Associations and Policies           256                 5
 SGT_DGT                                            4096/512           0/1   
 CLIENT_LE                                          4096/256           0/0   
 INPUT_GROUP_LE                                     6144                 0
 OUTPUT_GROUP_LE                                    6144                 0
 Macsec SPD                                          256                 2
CAM Utilization for ASIC  [1]
 Table                                              Max Values        Used Values
 --------------------------------------------------------------------------------
 Unicast MAC addresses                              32768/512          22/21  
 L3 Multicast entries                               4096/512           0/7   
 L2 Multicast entries                               4096/512           0/9   
 Directly or indirectly connected routes            16384/7168          7/20  
 QoS Access Control Entries                         2560                84
 Security Access Control Entries                    3072               125
 Netflow ACEs                                        768                15
 Policy Based Routing ACEs                          1024                 9
 Flow SPAN ACEs                                      512                 5
 Output Flow SPAN ACEs                               512                 8
 Control Plane Entries                               512               212
 Tunnels                                             256                17
 Lisp Instance Mapping Entries                       256                 3
 Input Security Associations                         256                 3
 Output Security Associations and Policies           256                 5
 SGT_DGT                                            4096/512           0/1   
 CLIENT_LE                                          4096/256           0/0   
 INPUT_GROUP_LE                                     6144                 0
 OUTPUT_GROUP_LE                                    6144                 0
 Macsec SPD                                          256                 2

 

Switching Database Manager

The Switching Database Manager or SDM as mentioned earlier is responsible for deciding how the TCAM memory is allocated to it's specific functions. Catalyst Switches support various preconfigured switching engines. The reasons behind wanting to alter how the CAM/TCAM table memory is allocated will depend on how the switch is used. If the Switch is a Layer 2 only switch, then you will want to increase the memory allocated to CAM and reduce the memory allocated to the FIB and vice versa for a layer 3 MLS. To view the switches current SDM template, use the command show sdm prefer.

show sdm prefer

To change the SDM templated that is used by the switch, use the global configuration command sdm prefer <template>

SWITCH3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SWITCH3(config)#sdm prefer ?
  advanced  Advanced Template
  vlan      VLAN Template

Different switch platforms will show different SDM template options. For example, the above output is from a 3650, while the below output is from a 2960X.

SWITCH1(config)#sdm prefer ?
  default          Default bias
  lanbase-default  Enhanced support for both IPv4 and IPv6 Routing
  lanbase-routing  Supports both IPv4 and IPv6 Static Routing

Once you specify the template that you want to use, the switch will need to be rebooted in order for the memory allocation to take effect.