Skip to content
Log in Sign up My account

2.2 MPLS Layer 3 VPN

Blueprint 2.2: Describe MPLS Layer 3 VPN

  • VRFs on the PE and the separation of customer routing tables
  • Route distinguisher (RD) and the VPNv4 address family
  • Route target (RT) import and export
  • MP-BGP between PEs and the two-label stack (transport label and VPN label)
  • PE-CE routing: static, eBGP, OSPF and EIGRP, and redistribution into MP-BGP

L3VPN is a describe objective, but the questions are detailed. You will see VRF definitions, show bgp vpnv4 unicast output and PE-CE configurations, and be asked why a site is missing routes. Almost every L3VPN fault falls into one of five buckets: RT import/export, VPNv4 session or address family, missing redistribution on the PE, the transport LSP to the remote PE, and PE-CE protocol details. This page is organised so you can walk those buckets in order.

The model

An MPLS L3VPN (RFC 4364) lets a provider carry many customers' routed traffic over one shared core while keeping every customer isolated, even when customers use the same private addresses.

graph LR
  CEA1["CE-A1"] -- "eBGP" --- PE1["PE1<br/>VRF CUST-A<br/>VRF CUST-B"]
  CEB1["CE-B1"] -- "OSPF" --- PE1
  PE1 -- "IGP + LDP" --- P["P"]
  P -- "IGP + LDP" --- PE2["PE2<br/>VRF CUST-A<br/>VRF CUST-B"]
  PE2 -- "eBGP" --- CEA2["CE-A2"]
  PE2 -- "OSPF" --- CEB2["CE-B2"]
  PE1 -. "MP-BGP VPNv4 Lo0 to Lo0" .- PE2
Device Knows Runs
CE Its own site and the routes the PE sends it Any routing protocol to the PE, or a static route. No MPLS, no VRF awareness.
PE One VRF per customer, plus the core IGP PE-CE protocol per VRF, MP-BGP VPNv4 to other PEs (or route reflectors), IGP and LDP toward the core
P Only core prefixes (PE loopbacks and links) IGP and LDP. No BGP, no VRFs, no customer routes.

The design splits the work into three planes that you troubleshoot separately:

  1. PE-CE: the customer route reaches the PE and lands in the correct VRF.
  2. PE-PE control plane: MP-BGP carries the route, tagged with RTs and a VPN label, to the other PEs, which import it into the right VRF.
  3. Transport: an LDP LSP from the ingress PE to the egress PE loopback carries the packet across the core.

VRF

A VRF (virtual routing and forwarding instance) is a separate routing table and CEF table on the PE. Interfaces are placed into a VRF, and routes learned on those interfaces only go into that VRF's table. The same technique without MPLS is VRF-Lite (objective 1.7); L3VPN adds the RD, RT and MP-BGP so the VRF can span many PEs.

vrf definition CUST-A
 rd 65000:1
 address-family ipv4
  route-target export 65000:1
  route-target import 65000:1
 exit-address-family
!
interface GigabitEthernet0/1
 vrf forwarding CUST-A
 ip address 10.1.5.1 255.255.255.252

route-target both 65000:1 is a shortcut for the import and export pair with the same value. The older ip vrf CUST-A syntax (with ip vrf forwarding on the interface) is IPv4 only and still accepted on IOS.

Exam trap

Applying vrf forwarding to an interface removes its IP address. IOS prints a message saying so. If an exhibit shows the VRF applied and the interface has no address, someone forgot to re-enter it. The same happens with ip vrf forwarding.

Route distinguisher

Two customers can both use 192.168.1.0/24. Inside BGP that would be one prefix, and BGP would pick one path and hide the other. The RD fixes this. It is a 64-bit value prepended to the 32-bit IPv4 prefix, producing a 96-bit VPNv4 prefix that is unique across the provider:

  • 65000:1:192.168.1.0/24 (customer A)
  • 65000:2:192.168.1.0/24 (customer B)
RD format Example Typical use
ASN:nn 65000:1 Most common
IP-address:nn 10.255.255.1:1 Unique RD per PE, useful for multihomed sites

The RD has one job: make the prefix unique. It does not decide which VRF imports the route. One RD is configured per VRF per PE. Different PEs may use the same or different RDs for the same customer.

Route target

The RT is a BGP extended community attached to each VPNv4 route. It is the import and export policy:

  • Export: when a PE sends a route from a VRF into MP-BGP, it attaches the VRF's export RTs.
  • Import: when a PE receives a VPNv4 route, it imports the route into every local VRF that has an import RT matching any RT on the route.

Because a route can carry several RTs and a VRF can import several, RTs build more than simple "any to any" VPNs:

Topology RT design
Full mesh (any to any) Every site exports and imports the same RT
Hub and spoke Spokes export RT-SPOKE and import RT-HUB; hub exports RT-HUB and imports RT-SPOKE
Shared services (extranet) The services VRF exports RT-SVC; each customer VRF also imports RT-SVC, and the services VRF imports each customer's RT 
graph TB
  subgraph "PE1"
    V1["VRF CUST-A<br/>RD 65000:1<br/>export 65000:1"]
  end
  subgraph "MP-BGP"
    R["VPNv4 route<br/>65000:1:192.168.1.0/24<br/>RT 65000:1, label 20<br/>next hop 10.255.255.1"]
  end
  subgraph "PE2"
    V2["VRF CUST-A<br/>import 65000:1"]
    V3["VRF CUST-B<br/>import 65000:2"]
  end
  V1 --> R
  R -- "RT matches, imported" --> V2
  R -. "no match, ignored" .-> V3

Exam trap

When a question shows two PEs with different RDs and asks why routes are missing, the RD is almost always a distractor. Compare the RTs: one PE's export must match the other PE's import. When the RTs match and routes are still missing, check that the VPNv4 neighbor is active and that send-community extended is present, because without extended communities the RTs never arrive.

MP-BGP and VPNv4

Normal BGP carries IPv4 unicast. Multiprotocol BGP (RFC 4760) adds address families. L3VPN uses the VPNv4 unicast family (AFI 1, SAFI 128). Each VPNv4 update carries:

  • the VPNv4 prefix (RD plus IPv4 prefix)
  • the RT extended communities
  • the VPN label the advertising PE assigned
  • the BGP next hop, which is the advertising PE's update source, normally its Loopback0

PEs peer with iBGP between loopbacks, either full mesh or through route reflectors. The VPNv4 next hop is always the egress PE's loopback, and that loopback must have an LDP label end to end, or the ingress PE cannot build the transport part of the label stack.

router bgp 65000
 bgp router-id 10.255.255.1
 no bgp default ipv4-unicast
 neighbor 10.255.255.3 remote-as 65000
 neighbor 10.255.255.3 update-source Loopback0
 !
 address-family vpnv4
  neighbor 10.255.255.3 activate
  neighbor 10.255.255.3 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf CUST-A
  ! PE-CE routing or redistribution for CUST-A goes here
 exit-address-family

IOS runs one BGP process. Each VRF is an address-family ipv4 vrf inside it. Routes in that address family are the VRF's BGP table; the PE converts them to VPNv4 routes (adding the RD, RTs and a label) when it advertises them to VPNv4 neighbors.

Exam trap

Summarizing the PE loopbacks in the core IGP breaks the VPN even though every route looks fine. The LSP must end at the egress PE. If a P router advertises only a summary such as 10.255.255.0/24, LDP binds a label to the summary, and the LSP ends at the router that originated it: the router before it pops the transport label, and the summarizing P router receives a packet with only the VPN label on top. That label means nothing to it, so the packet is dropped. PE loopbacks must be advertised as /32 host routes.

The two-label stack

graph LR
  CE1["CE-A1"] -- "IP" --> PE1["PE1<br/>push VPN 21<br/>push transport 17"]
  PE1 -- "17 | 21 | IP" --> P["P<br/>PHP: pop 17"]
  P -- "21 | IP" --> PE2["PE2<br/>pop 21, VRF CUST-A<br/>IP lookup"]
  PE2 -- "IP" --> CE2["CE-A2"]
Label Position Learned from Identifies Who acts on it
Transport Outer (top) LDP, for the egress PE loopback The LSP to the egress PE Every P router (swap), penultimate hop (pop)
VPN Inner (bottom, S bit set) MP-BGP, from the egress PE The VRF, or the prefix within the VRF, on the egress PE Only the egress PE

On IOS the default label allocation is per prefix: every VRF prefix gets its own VPN label. mpls label mode all-vrfs protocol bgp-vpnv4 per-vrf changes that to one label per VRF, which saves labels but forces an extra IP lookup on the egress PE.

Why two labels? The P routers have no VRF information, so they cannot use a VPN label. The egress PE has many VRFs, so it needs a label that says which one. If only the VPN label were used, P routers would not know how to forward it; if only the transport label were used, the egress PE would not know which VRF to use (and with PHP, it would receive a bare customer IP packet).

PE-CE routing

The CE learns nothing about MPLS. From the CE's point of view, the PE is a normal neighbor. On the PE, each PE-CE protocol runs inside the VRF, and its routes must be redistributed into the VRF's BGP address family (except when the PE-CE protocol is BGP itself). In the opposite direction, BGP routes from remote sites must be redistributed into the PE-CE protocol.

PE-CE protocol PE configuration Into MP-BGP Back to the CE
Static ip route vrf CUST-A 192.168.1.0 255.255.255.0 10.1.5.2 redistribute static under address-family ipv4 vrf CUST-A CE uses a static or default route
eBGP neighbor 10.1.5.2 remote-as 65001 under address-family ipv4 vrf CUST-A Automatic Automatic
OSPF router ospf 20 vrf CUST-B redistribute ospf 20 (internal routes by default) redistribute bgp 65000 subnets under the VRF OSPF process
EIGRP address-family ipv4 vrf CUST-C autonomous-system 100 under router eigrp redistribute eigrp 100 redistribute bgp 65000 metric ...

eBGP PE-CE details

Many enterprises use the same private AS at every site. Site 2 then receives routes whose AS path already contains its own AS and drops them (BGP loop prevention). Two fixes:

  • neighbor 10.3.6.2 as-override on the PE, in the VRF address family. The PE replaces the customer AS in the path with the provider AS before sending the route to the CE. The path the CE sees becomes 65000 65000.
  • neighbor 10.1.5.1 allowas-in on the CE. The CE accepts routes that contain its own AS.

as-override is the provider-side fix and the one you are more likely to see in a PE configuration.

OSPF PE-CE details

The MPLS VPN backbone acts as a "super backbone" above OSPF area 0. The PE carries the OSPF route type and a domain ID (derived from the OSPF process ID by default) in BGP extended communities. When the receiving PE redistributes the route back into OSPF:

Original route at site 1 Same domain ID on both PEs Different domain ID
Intra-area or inter-area (type 1, 2, 3) Arrives at site 2 as inter-area (O IA), type 3 Arrives as external type 5
External (type 5) Arrives as external Arrives as external

If the customer also has a backdoor link between the sites, the backdoor path is intra-area and always beats the inter-area route through the VPN. A sham link between the PEs makes the VPN path intra-area as well. The PE also sets the down bit on LSAs it sends to the CE, so that another PE at the same site does not redistribute those routes back into MP-BGP.

Exam trap

redistribute ospf 20 into BGP takes only OSPF internal routes by default. If a customer site has external routes (O E1 or O E2) that do not appear at the other site, the PE needs redistribute ospf 20 match internal external 1 external 2.

EIGRP PE-CE details

router eigrp 1
 address-family ipv4 vrf CUST-C autonomous-system 100
  network 10.1.9.0 0.0.0.3
  redistribute bgp 65000 metric 10000 100 255 1 1500
 exit-address-family
!
router bgp 65000
 address-family ipv4 vrf CUST-C
  redistribute eigrp 100

The PE copies the EIGRP metric components into BGP extended communities, so a route that started as EIGRP at one site keeps its metric at the other. A seed metric is still needed for routes that did not come from EIGRP, so configure one with redistribute bgp ... metric or default-metric.

Verification

Bucket Command Look for
PE-CE show ip vrf interfaces, show vrf Interface in the right VRF with an address
PE-CE show ip route vrf CUST-A, show ip bgp vpnv4 vrf CUST-A summary, show ip ospf 20 neighbor CE routes in the VRF table
Into BGP show bgp vpnv4 unicast vrf CUST-A Local routes with the right RD
PE-PE show bgp vpnv4 unicast all summary VPNv4 neighbor Established with prefixes received
PE-PE show bgp vpnv4 unicast all 192.168.2.0 RTs (Extended Community: RT:65000:1), label, next hop
Labels show bgp vpnv4 unicast all labels In label (local) and out label (remote VPN label)
Transport show mpls forwarding-table 10.255.255.3 32 A transport label to the remote PE loopback
End to end show ip cef vrf CUST-A 192.168.2.0 detail Both labels in the imposition stack
End to end ping vrf CUST-A 192.168.2.1, traceroute vrf CUST-A 192.168.2.1 Reachability from inside the VRF 
PE1# show bgp vpnv4 unicast all
BGP table version is 13, local router ID is 10.255.255.1
     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65000:1 (default for vrf CUST-A)
 *>   192.168.1.0      10.1.5.2                 0             0 65001 i
 *>i  192.168.2.0      10.255.255.3             0    100      0 65001 i
Route Distinguisher: 65000:2 (default for vrf CUST-B)
 *>   192.168.1.0      10.1.7.2                 2         32768 ?
 *>i  192.168.2.0      10.255.255.3             2    100      0 ?

The same prefixes appear twice under different RDs: overlapping customer addresses, kept apart. Remote routes have the remote PE's loopback as next hop. (Output trimmed to the customer LAN prefixes.)

PE1# show bgp vpnv4 unicast all labels
   Network          Next Hop      In label/Out label
Route Distinguisher: 65000:1 (CUST-A)
   192.168.1.0      10.1.5.2        20/nolabel
   192.168.2.0      10.255.255.3    nolabel/21

PE1 assigned label 20 for its own customer prefix and advertises it to PE2. For the remote prefix, PE2 assigned 21, which PE1 pushes as the inner label.

Exam trap

ping 192.168.2.1 from a PE fails even when the VPN works, because the ping uses the global routing table and the PE has no global route to a customer prefix. Always test from inside the VRF: ping vrf CUST-A 192.168.2.1.

Lab: two customers with overlapping addresses

Topology: seven IOSv routers. Reuse the LDP core idea from 2.1, shortened to PE1, P and PE2. Provider AS 65000. CUST-A uses eBGP and the same AS 65001 at both sites. CUST-B uses OSPF. Both customers use 192.168.1.0/24 at site 1 and 192.168.2.0/24 at site 2.

graph LR
  R5["R5 CE-A1 AS 65001<br/>Lo1 192.168.1.1/24"] -- "10.1.5.0/30" --- R1["R1 PE1"]
  R7["R7 CE-B1 OSPF<br/>Lo1 192.168.1.1/24"] -- "10.1.7.0/30" --- R1
  R1 -- "Gi0/0 10.1.2.0/30" --- R2["R2 P"]
  R2 -- "Gi0/1 10.2.3.0/30" --- R3["R3 PE2"]
  R3 -- "10.3.6.0/30" --- R6["R6 CE-A2 AS 65001<br/>Lo1 192.168.2.1/24"]
  R3 -- "10.3.8.0/30" --- R8["R8 CE-B2 OSPF<br/>Lo1 192.168.2.1/24"]
VRF RD RT both PE-CE
CUST-A 65000:1 65000:1 eBGP to AS 65001
CUST-B 65000:2 65000:2 OSPF process 20, area 0

Steps

  1. Build the core: OSPF process 1 and LDP on R1, R2 and R3 for loopbacks 10.255.255.1, .2 and .3 and the core links. Confirm show mpls ldp neighbor on R2 shows two peers.

  2. On R1 (and the mirror on R3), create both VRFs and put the customer interfaces in them:

    vrf definition CUST-A
 rd 65000:1
 address-family ipv4
  route-target both 65000:1
 exit-address-family
vrf definition CUST-B
 rd 65000:2
 address-family ipv4
  route-target both 65000:2
 exit-address-family
!
interface GigabitEthernet0/1
 vrf forwarding CUST-A
 ip address 10.1.5.1 255.255.255.252
interface GigabitEthernet0/2
 vrf forwarding CUST-B
 ip address 10.1.7.1 255.255.255.252

  3. Configure the PE-CE protocols and MP-BGP on R1:

    router ospf 20 vrf CUST-B
 network 10.1.7.1 0.0.0.0 area 0
 redistribute bgp 65000 subnets
!
router bgp 65000
 bgp router-id 10.255.255.1
 no bgp default ipv4-unicast
 neighbor 10.255.255.3 remote-as 65000
 neighbor 10.255.255.3 update-source Loopback0
 address-family vpnv4
  neighbor 10.255.255.3 activate
  neighbor 10.255.255.3 send-community extended
 exit-address-family
 address-family ipv4 vrf CUST-A
  neighbor 10.1.5.2 remote-as 65001
  neighbor 10.1.5.2 activate
  neighbor 10.1.5.2 as-override
 exit-address-family
 address-family ipv4 vrf CUST-B
  redistribute ospf 20
 exit-address-family

  4. On R5 and R6, run eBGP to the PE in AS 65001 and advertise Lo1 with a network statement. On R7 and R8, run OSPF area 0 on the PE link and on Lo1, and add ip ospf network point-to-point to Lo1 so it is advertised as a /24 rather than a /32.

  5. Verify on R1 with show bgp vpnv4 unicast all. You should see 192.168.1.0/24 and 192.168.2.0/24 twice, once under each RD.

  6. Verify on the CEs:

    R6# show ip bgp | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>   192.168.1.0      10.3.6.1                               0 65000 65000 i
 *>   192.168.2.0      0.0.0.0                  0         32768 i
R8# show ip route ospf | include 192.168
O IA     192.168.1.0/24 [110/3] via 10.3.8.1, 00:03:41, GigabitEthernet0/0

    R6 sees the AS path rewritten by as-override. R8 receives the CUST-B route as inter-area, because both PEs use OSPF process 20 and so share the domain ID.

  7. Break it. Remove as-override on R3 and clear the session (clear ip bgp vrf CUST-A *, or clear bgp vpnv4 unicast * soft for the VPNv4 side). R6 loses 192.168.1.0/24 because the AS path contains 65001.

  8. Break it again. Change R3's CUST-B export RT to 65000:22. R1 stops importing site 2's route into CUST-B, while CUST-A is unaffected. Confirm with show bgp vpnv4 unicast all 192.168.2.0 on R1 and read the RT.

Quick check

PE1 and PE2 use RD 65000:1 and 65000:11 for the same customer VRF, and both import and export RT 65000:100. Will the sites exchange routes?

Yes. The RD only makes prefixes unique and may differ per PE. Import is decided by the RT, and both PEs export and import 65000:100.

Which label does the P router use to forward a customer packet, and who assigned it?

The outer transport label, assigned through LDP by the P router's downstream neighbor for the egress PE's loopback. The P router never looks at the inner VPN label, which the egress PE assigned and advertised through MP-BGP.

A customer uses AS 65010 at all five sites with eBGP to the provider. Only routes from the local site are in each CE's table. What PE command fixes it?

neighbor <CE address> as-override in the VRF address family on each PE. The CEs drop routes that already contain AS 65010 in the path; as-override replaces it with the provider AS.

References