Skip to content
Log in Sign up My account

3.3 Troubleshooting Control Plane Policing

Blueprint 3.3: Troubleshoot control plane policing (CoPP) (Telnet, SSH, HTTP(S), SNMP, EIGRP, OSPF, BGP)

  • What the control plane is and which packets reach it
  • Building CoPP with MQC: ACLs, class-maps, a policy-map with policers, the control-plane attachment
  • Classifying each listed protocol correctly: Telnet, SSH, HTTP, HTTPS, SNMP, EIGRP, OSPF, BGP
  • Reading show policy-map control-plane to find the class a protocol fell into
  • Fault symptoms: adjacencies flapping, slow or failed management sessions, SNMP timeouts

The verb is troubleshoot, and the blueprint lists the protocols you must be able to match. The usual exhibit is a CoPP ACL plus show policy-map control-plane output, and the fault is that one protocol is classified into the wrong class.

What CoPP protects

A router does two kinds of work. The data plane forwards transit packets in hardware or in the CEF fast path. The control plane runs on the route processor CPU: routing protocols, management sessions, ARP, ICMP replies, and anything else addressed to the router or needing CPU attention. The CPU is a far smaller resource than the forwarding path. A flood of packets aimed at the router, deliberate or accidental, can starve routing protocols of CPU time and bring the network down even though the forwarding path is idle.

CoPP is a QoS policy applied to the traffic going to the route processor. It lets you guarantee room for routing protocols, rate-limit management and ICMP, and drop traffic that should never reach the CPU.

graph LR
  IN["Packets arriving<br/>on all interfaces"] --> FWD{"Destined to the router<br/>or needs the CPU?"}
  FWD -->|"no, transit"| DP["Data plane forwarding<br/>CoPP not involved"]
  FWD -->|"yes, punted"| COPP["control-plane<br/>service-policy input"]
  COPP --> RTG["Routing class<br/>OSPF, EIGRP, BGP"]
  COPP --> MGMT["Management class<br/>SSH, Telnet, HTTP, SNMP"]
  COPP --> ICMP["ICMP class"]
  COPP --> DEF["class-default"]
  RTG --> CPU["Route processor"]
  MGMT --> CPU
  ICMP --> CPU
  DEF --> CPU

Packets that reach the control plane include:

  • packets addressed to any of the router's own IP addresses (SSH, SNMP queries, BGP, pings to the router);
  • routing protocol multicasts such as OSPF to 224.0.0.5 and 224.0.0.6, and EIGRP to 224.0.0.10;
  • packets that need CPU handling, such as those with an expiring TTL (traceroute) or IP options;
  • on many platforms, ARP and other non-IP control traffic, which then lands in class-default unless you classify it.

Building CoPP with MQC

CoPP uses the same Modular QoS CLI as interface QoS, in four steps.

  1. ACLs identify traffic. In this context a permit means "this packet belongs to the class" and a deny means "this packet does not belong to this class, try the next one". An ACL deny never drops anything by itself.
  2. Class-maps reference the ACLs.
  3. A policy-map gives each class an action, normally a policer with conform-action transmit and exceed-action drop, or a plain drop for unwanted traffic.
  4. The control-plane attachment applies the policy in the input direction.
ip access-list extended COPP-ROUTING
 10 permit ospf any any
 20 permit eigrp any any
 30 permit tcp any any eq bgp
 40 permit tcp any eq bgp any
ip access-list extended COPP-MGMT
 10 permit tcp 192.168.100.0 0.0.0.255 any eq 22
 20 permit tcp 192.168.100.0 0.0.0.255 any eq telnet
 30 permit tcp 192.168.100.0 0.0.0.255 any eq www
 40 permit tcp 192.168.100.0 0.0.0.255 any eq 443
 50 permit udp host 192.168.100.20 any eq snmp
 60 permit tcp host 192.168.100.50 eq tacacs any
 70 permit udp any eq ntp any
ip access-list extended COPP-ICMP
 10 permit icmp any any
ip access-list extended COPP-UNWANTED
 10 permit udp any any eq 1900
 20 permit tcp any any eq telnet
!
class-map match-any CM-ROUTING
 match access-group name COPP-ROUTING
class-map match-any CM-MGMT
 match access-group name COPP-MGMT
class-map match-any CM-ICMP
 match access-group name COPP-ICMP
class-map match-any CM-UNWANTED
 match access-group name COPP-UNWANTED
!
policy-map PM-COPP
 class CM-ROUTING
  police 1000000 conform-action transmit exceed-action transmit
 class CM-MGMT
  police 512000 conform-action transmit exceed-action drop
 class CM-ICMP
  police 64000 conform-action transmit exceed-action drop
 class CM-UNWANTED
  drop
 class class-default
  police 128000 conform-action transmit exceed-action drop
!
control-plane
 service-policy input PM-COPP

Notice two deliberate details. The routing class uses exceed-action transmit, so it is monitored rather than limited: you can see whether routing traffic ever exceeds the expected rate without risking an adjacency. And Telnet appears in both COPP-MGMT and COPP-UNWANTED. Because CM-MGMT comes first in the policy, Telnet from the management LAN is policed and allowed, while Telnet from anywhere else falls through to CM-UNWANTED and is dropped. Class order in the policy-map is evaluation order. First match wins.

Matching the blueprint protocols

Protocol Transport ACL match for traffic arriving at the router Common mistake
Telnet TCP 23 permit tcp <src> any eq telnet Port on the source side
SSH TCP 22 permit tcp <src> any eq 22 Port on the source side
HTTP TCP 80 permit tcp <src> any eq www Forgetting it if the web UI is used
HTTPS TCP 443 permit tcp <src> any eq 443 Matching only www
SNMP queries UDP 161 permit udp <nms> any eq snmp Using TCP, or matching 162 (traps go from the router, so they do not pass CoPP input)
EIGRP IP protocol 88 permit eigrp any any Matching TCP or UDP port 88
OSPF IP protocol 89 permit ospf any any Matching TCP or UDP port 89
BGP TCP 179 permit tcp any any eq bgp and permit tcp any eq bgp any Matching only one direction

The BGP pair needs explaining. A BGP session is one TCP connection, opened by whichever peer connects first. If the neighbor opened it, the packets arriving at the router have destination port 179. If this router opened it, the arriving packets come from port 179 to a random local port. Only one entry is needed for any given session, but you cannot predict which, so both are required. A CoPP policy with only eq bgp as destination often works until the session resets and reconnects the other way round, then the session lands in class-default.

In production you would also tighten any to the real neighbor addresses, for example permit tcp host 203.0.113.1 any eq bgp. That is good practice, but remember to update CoPP when you add a peer.

Exam trap

OSPF and EIGRP do not use TCP or UDP. permit tcp any any eq 89 looks plausible and matches nothing. OSPF then falls into class-default, where a low policer drops enough hellos to make the adjacency flap with "Dead timer expired". When an exhibit shows routing adjacencies flapping after CoPP was applied, check the routing ACL protocol keywords first.

Reading show policy-map control-plane

This one command answers most CoPP questions. For each class it shows how many packets matched, the offered and drop rates, and how many conformed or exceeded the policer.

R1# show policy-map control-plane
 Control Plane

  Service-policy input: PM-COPP

    Class-map: CM-ROUTING (match-any)
      51 packets, 4386 bytes
      5 minute offered rate 0000 bps, drop rate 0000 bps
      Match: access-group name COPP-ROUTING
        51 packets, 4386 bytes
        5 minute rate 0 bps
      police:
          cir 1000000 bps, bc 31250 bytes
        conformed 51 packets, 4386 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          transmit
        conformed 0000 bps, exceeded 0000 bps

    Class-map: CM-MGMT (match-any)
      0 packets, 0 bytes
      5 minute offered rate 0000 bps, drop rate 0000 bps
      Match: access-group name COPP-MGMT
        0 packets, 0 bytes
        5 minute rate 0 bps
      police:
          cir 512000 bps, bc 16000 bytes
        conformed 0 packets, 0 bytes; actions:
          transmit
        exceeded 0 packets, 0 bytes; actions:
          drop
        conformed 0000 bps, exceeded 0000 bps

    Class-map: class-default (match-any)
      38114 packets, 4471020 bytes
      5 minute offered rate 152000 bps, drop rate 24000 bps
      Match: any
      police:
          cir 128000 bps, bc 4000 bytes
        conformed 33004 packets, 3879210 bytes; actions:
          transmit
        exceeded 5110 packets, 591810 bytes; actions:
          drop
        conformed 128000 bps, exceeded 24000 bps

How to read it:

  1. A class that should be busy shows zero packets. CM-MGMT has never matched, yet administrators are using SSH. Its ACL is wrong.
  2. class-default is busy and dropping. The traffic that should be in CM-MGMT is landing here and sharing a small policer with everything else.
  3. CM-ROUTING has only 51 packets. On a router with several OSPF neighbors and a BGP session that has been up for hours, that is far too few. Some routing traffic is also falling into class-default.

Check the ACLs next: show access-lists COPP-MGMT and show access-lists COPP-ROUTING. Then look for the protocol-level evidence: show ip ospf neighbor (adjacency up times that keep resetting), show ip bgp summary (the Up/Down column), and the log for %OSPF-5-ADJCHG, %DUAL-5-NBRCHANGE or %BGP-5-ADJCHANGE messages.

Exam trap

In show policy-map control-plane, the counters under Class-map count packets that matched that class, and the exceeded ... drop counter shows how many of them the policer dropped. A class with zero matches is a classification problem (ACL or class order). A class with many matches and many exceeded drops is a rate problem. The fix is different, so decide which one you are looking at before you choose an answer.

Fault table

Symptom Likely cause Proof
OSPF or EIGRP adjacencies flap after CoPP is applied Routing protocol matched as TCP or UDP, so it is in class-default Routing class has few matches; class-default exceeds; ADJCHG or NBRCHANGE logs
BGP session drops with hold timer expired, often after a reset Only one BGP direction in the ACL show access-lists; the session works when the peer initiates and fails when the router initiates
SSH, Telnet or HTTPS to the router slow or timing out Port on the source side, or management policer too small Management class has zero matches (wrong ACL) or many exceeded drops (rate too low)
SNMP polls time out, SNMP walks fail part way SNMP not classified, or polled from an address the ACL does not list SNMP class zero matches; NMS address not in ACL
Telnet from the management LAN dropped although it is in the management ACL A drop class above the management class also matches Telnet Class order in show policy-map PM-COPP
No traffic matches any class, all in class-default Class-map is match-all with two match lines that one packet cannot both satisfy show class-map
CoPP has no effect at all Policy not attached, or attached under an interface instead of control-plane show policy-map control-plane returns nothing
ARP resolution fails intermittently Very low class-default policer catching ARP on a busy segment class-default exceeded drops; incomplete ARP entries

Exam trap

An ACL deny line in a CoPP ACL does not drop. It only removes the packet from that class. If a question asks you to block a protocol at the control plane, the answer is a class whose ACL permits that protocol, with a drop action (or a policer whose exceed and conform actions drop) in the policy-map.

Platform notes

On IOS and IOSv routers, CoPP is entirely user-defined as shown here. Catalyst 9000 switches ship with a system-defined CoPP policy named system-cpp-policy that is always active. On those switches you tune the rates of its predefined classes rather than building a policy from scratch. show policy-map control-plane still shows what is happening per class. The exam's troubleshooting logic, which is "find the class, then decide whether it is a classification or a rate problem", is the same on both.

A related feature, Management Plane Protection (control-plane host then management-interface Gi0/3 allow ssh snmp), restricts which interfaces accept management protocols at all. It complements CoPP rather than replacing it.

Lab: CoPP break/fix

Topology

graph LR
  MGMT["R4 Mgmt host<br/>192.168.100.10"] -- "192.168.100.0/24  .1 Gi0/2" --- R2
  R2 -- "Gi0/0 .2  10.1.2.0/30  .1 Gi0/0<br/>OSPF area 0" --- R1["R1<br/>Lo0 10.255.255.1"]
  R1 -- "Gi0/1 .1  10.1.3.0/30  .2 Gi0/0<br/>eBGP 65001 to 65003" --- R3["R3 AS 65003<br/>Lo0 10.255.255.3"]
  R1 -- "Gi0/2 .1  10.1.5.0/30  .2 Gi0/0<br/>EIGRP AS 10" --- R5["R5"]
Device Role
R1 Device under test. OSPF to R2, EIGRP to R5, eBGP to R3 (AS 65001 to AS 65003)
R2 OSPF neighbor and gateway for the management LAN
R3 eBGP peer, advertises its Loopback0
R4 IOSv acting as a management host: SSH, Telnet, HTTP client and SNMP poller
R5 EIGRP neighbor

Steps

  1. Build the routing so R4 can reach 10.255.255.1. Redistribute connected or advertise loopbacks as needed. On R1 enable SSH, ip http server, ip http secure-server, snmp-server community LabRO RO, and a local user with transport input ssh telnet on the VTY lines.
  2. Apply the PM-COPP policy shown above to R1, but change COPP-MGMT line 50 to permit udp 192.168.100.0 0.0.0.255 any eq snmp so R4 can poll. Remove line 20 from COPP-UNWANTED so the lab can test Telnet.
  3. Verify the baseline. All three routing adjacencies are up. From R4, ssh -l admin 10.255.255.1 and telnet 10.255.255.1 both work. In show policy-map control-plane, CM-ROUTING and CM-MGMT both have rising match counters and class-default has no exceeded drops. Save the baseline.
  4. Break 1: replace COPP-ROUTING line 10 with permit tcp any any eq 89, set the class-default policer to police 8000 conform-action transmit exceed-action drop, and remove class CM-ICMP from PM-COPP so pings to R1 also land in class-default. From R4 run ping 10.255.255.1 size 1400 repeat 100000 timeout 0 to load class-default. Expected: the OSPF adjacency to R2 flaps with "Dead timer expired", while EIGRP and BGP stay up. Fix with permit ospf any any and re-add class CM-ICMP with its policer.
  5. Break 2: keep the 8000 bps class-default policer. On R3 configure neighbor 10.1.3.1 transport connection-mode passive so that R1 always opens the BGP session. Remove COPP-ROUTING line 40 (permit tcp any eq bgp any), remove class CM-ICMP again and restart the ping flood from R4, then run clear ip bgp 10.1.3.2 on R1. Expected: R3's packets arrive from source port 179, miss CM-ROUTING, and compete with the flood in class-default. The session struggles to establish or drops with hold timer expiry. Restore line 40, the ICMP class and the class-default rate, and remove the passive setting on R3.
  6. Break 3: change COPP-MGMT line 10 to permit tcp 192.168.100.0 0.0.0.255 eq 22 any. Expected: SSH from R4 is slow or fails, CM-MGMT shows matches only for the other management protocols, and class-default counters rise as you SSH. Fix the port position.
  7. Break 4: move CM-UNWANTED above CM-MGMT in the policy (remove and re-add the classes in the new order), and put permit tcp any any eq telnet back into COPP-UNWANTED. Expected: SSH works, Telnet from R4 fails, and CM-UNWANTED drop counters rise. Fix the class order.

Expected verification output when fixed

R1# show policy-map control-plane | include Class-map|packets, |exceeded
    Class-map: CM-ROUTING (match-any)
      6121 packets, 512330 bytes
        exceeded 0 packets, 0 bytes; actions:
    Class-map: CM-MGMT (match-any)
      2214 packets, 160410 bytes
        exceeded 0 packets, 0 bytes; actions:
    Class-map: CM-ICMP (match-any)
      410 packets, 46740 bytes
        exceeded 0 packets, 0 bytes; actions:
    Class-map: CM-UNWANTED (match-any)
      0 packets, 0 bytes
    Class-map: class-default (match-any)
      188 packets, 15322 bytes
        exceeded 0 packets, 0 bytes; actions:

R1# show ip ospf neighbor
Neighbor ID     Pri   State           Dead Time   Address         Interface
10.255.255.2      1   FULL/DR         00:00:35    10.1.2.2        GigabitEthernet0/0

Quick check

A CoPP ACL for BGP contains only permit tcp any any eq bgp. Why might a BGP session still end up in class-default?

If the local router opened the TCP connection, the peer's packets arrive with source port 179 and a random destination port. They do not match eq bgp as a destination. You also need permit tcp any eq bgp any.

In show policy-map control-plane, the SSH class shows 0 packets and class-default shows exceeded drops whenever someone uses SSH. Is this a rate problem or a classification problem?

Classification. SSH is not matching its class at all. Check the ACL for a reversed port, a wrong source subnet, or an earlier class that captures SSH first.

Do SNMP traps sent by the router need to be permitted in the CoPP input policy?

No. Traps are sent by the router to the manager (UDP 162). CoPP in the input direction acts on traffic arriving at the control plane, such as SNMP queries to UDP 161.

References