Skip to content
Log in Sign up My account

2.3 LISP and VXLAN

Blueprint 2.3: Describe network virtualization concepts

  • 2.3.a LISP
  • 2.3.b VXLAN

This is a describe objective. You will not configure LISP or VXLAN on ENCOR, but you must know the roles, the messages, the ports and what problem each one solves. Both protocols are the foundation of SD-Access (1.3): LISP is its control plane and VXLAN is its data plane. If you understand this page, half of SD-Access stops being vocabulary.

Overlays in one picture

Both technologies build an overlay on top of a routed underlay. The underlay only needs IP reachability between the tunnel endpoints' loopbacks. Endpoint addresses live only in the overlay.

graph TB
  subgraph OV ["Overlay: endpoint addresses, VNIs, EIDs"]
    E1["Host A<br/>10.1.1.10"] -.-> E2["Host B<br/>10.1.2.20"]
  end
  subgraph UN ["Underlay: routed IP between tunnel endpoints"]
    T1["Edge 1<br/>RLOC or VTEP 192.0.2.1"] --- S1["Core"] --- T2["Edge 2<br/>RLOC or VTEP 192.0.2.2"]
  end
  E1 --- T1
  E2 --- T2
LISP VXLAN
What it is Control plane (mapping system) plus its own data plane encapsulation Data plane encapsulation only
Overlay type Layer 3 (IP in UDP) Layer 2 (Ethernet in UDP)
UDP ports 4342 control, 4341 data 4789 data
Tunnel endpoint name xTR, identified by its RLOC VTEP
Segment identifier Instance ID (24-bit) VNI (24-bit)
Standard RFC 9300 and RFC 9301 (originally RFC 6830) RFC 7348
Role in SD-Access Control plane Data plane (VXLAN-GPO variant)

2.3.a LISP

The problem: one address, two meanings

In ordinary IP, an address says both who a device is and where it is attached. Move a device and its address must change, and the internet routing table has to carry every multihomed or provider-independent prefix. Locator/ID Separation Protocol (LISP) splits the two:

Namespace Meaning Example
EID (Endpoint Identifier) Who: the address of a host. Stays the same when the host moves 10.1.1.10
RLOC (Routing Locator) Where: the address of the LISP router the EID sits behind, reachable in the underlay 192.0.2.1

The underlay routes only RLOCs. A mapping system tracks which RLOC each EID prefix currently sits behind, much as DNS maps names to addresses.

Roles

Role Job
ITR (Ingress Tunnel Router) Receives packets from local hosts, looks up the destination EID in its map-cache, asks the mapping system on a miss, then encapsulates toward the destination RLOC
ETR (Egress Tunnel Router) Registers its local EID prefixes with the map server, answers Map-Requests, and decapsulates incoming LISP packets
xTR A router doing both ITR and ETR, which is the normal case at a site edge
MS (Map Server) Accepts Map-Register messages from ETRs and holds the EID-to-RLOC database
MR (Map Resolver) Accepts Map-Requests from ITRs and forwards them to the right place in the mapping system
MS/MR Both functions on one device, the usual deployment and what the SD-Access control plane node is
PITR (Proxy ITR) Attracts traffic from non-LISP sites and encapsulates it toward LISP sites
PETR (Proxy ETR) Receives traffic from LISP sites destined for non-LISP sites and forwards it natively
PxTR Both proxy functions, the gateway between LISP and non-LISP networks (the role an SD-Access border plays toward the outside world)

Control plane messages

sequenceDiagram
  participant H as Host A
  participant ITR as ITR (site 1)
  participant MR as Map Resolver
  participant MS as Map Server
  participant ETR as ETR (site 2)
  ETR->>MS: Map-Register: 10.1.2.0/24 is at RLOC 192.0.2.2
  MS-->>ETR: Map-Notify (if requested)
  H->>ITR: Packet to 10.1.2.20
  Note over ITR: Map-cache miss
  ITR->>MR: Map-Request for 10.1.2.20
  MR->>MS: Forward request
  MS->>ETR: Forward to the registering ETR
  ETR->>ITR: Map-Reply: 10.1.2.0/24, RLOC 192.0.2.2, priority 1, weight 100
  Note over ITR: Install in map-cache
  ITR->>ETR: Following packets encapsulated, UDP 4341
Message Sent by, to Purpose
Map-Register ETR to MS "These EID prefixes are behind my RLOC." Authenticated with a shared key and repeated periodically
Map-Notify MS to ETR Acknowledges a registration, and tells the old ETR when an EID has moved
Map-Request ITR to MR "Where is this EID?"
Map-Reply ETR (or MS acting as proxy) to ITR The mapping: EID prefix, RLOCs, priority, weight and TTL
Negative Map-Reply MR or MS to ITR "This destination is not a LISP EID", so forward natively or to a PETR
Solicit-Map-Request (SMR) ETR to ITR "Your cached mapping is stale, ask again", used when a host moves

Details worth memorising:

  • An ITR answers its first packets to a new destination with a cache miss. Those packets are dropped (or sent to a PETR) while the Map-Request is answered, which is why the first ping across a fresh LISP path usually shows a lost packet.
  • Map-cache entries have a TTL, 24 hours by default, and are refreshed by new requests or SMRs.
  • Priority and weight in the Map-Reply give the ITR ingress traffic engineering. Lower priority is preferred. Locators with the same priority share traffic in proportion to their weights. That gives a multihomed site control over how traffic enters it, something hard to do with BGP.
  • Control messages use UDP 4342. Encapsulated data uses UDP 4341.

Data plane

The ITR adds an outer IP header (source its RLOC, destination the remote RLOC), a UDP header (destination 4341) and an 8-byte LISP header in front of the original IP packet. That is 36 bytes for an IPv4 underlay. LISP can carry IPv4 in IPv6 and IPv6 in IPv4, which makes it useful for IPv6 transition too.

graph LR
  A["Outer IP<br/>RLOC to RLOC"] --> B["UDP<br/>dst 4341"] --> C["LISP header<br/>8 bytes, instance ID"] --> D["Original IP<br/>EID to EID"] --> E["Payload"]

Use cases

Use case How LISP helps
Multihoming and ingress traffic engineering Priority and weight per RLOC, no BGP needed at the site
Host and VM mobility The EID keeps its address when it moves, the new ETR registers it and the MS notifies the old one
IPv6 transition IPv6 EIDs over an IPv4 underlay or the reverse
Multitenancy The instance ID keeps tenants apart, each mapped to a VRF
SD-Access The control plane node is the MS/MR, fabric edges register endpoints, borders act as PxTRs

Exam trap

The Map Server receives registrations from ETRs. The Map Resolver receives requests from ITRs. The roles are easy to swap under time pressure. Remember that the Resolver handles map-Requests (it resolves questions), while the Server stores the registrations it serves answers from.

Exam trap

The underlay never carries EID prefixes. A question that says the core routers need routes to the endpoint subnets for LISP to work is wrong: the core only needs to route between RLOCs.

What LISP configuration looks like

You will not type this on ENCOR, but recognising it helps on "refer to the exhibit" items. This is the classic IOS syntax. Newer IOS XE releases use a service ipv4 and instance-id structure and may display the same configuration in that form.

router lisp
 database-mapping 192.168.1.0/24 10.1.3.1 priority 1 weight 100
 ipv4 itr map-resolver 10.255.255.3
 ipv4 itr
 ipv4 etr map-server 10.255.255.3 key LISP-KEY
 ipv4 etr

router lisp
 site SITE1
  authentication-key LISP-KEY
  eid-prefix 192.168.1.0/24
  exit
 site SITE2
  authentication-key LISP-KEY
  eid-prefix 192.168.2.0/24
  exit
 ipv4 map-server
 ipv4 map-resolver

show ip lisp                 ! roles enabled, MS and MR addresses
show ip lisp database        ! local EID prefixes and their RLOCs
show ip lisp map-cache       ! mappings this ITR has learned
show lisp site               ! on the MS: which sites have registered

2.3.b VXLAN

The problem: Layer 2 that scales

Data centres and campus fabrics want two things that VLANs cannot give together: Layer 2 adjacency anywhere (for VM mobility and for keeping a subnet across many switches), and a routed, loop-free underlay with ECMP. VLANs also stop at 4094 IDs. Virtual Extensible LAN (VXLAN) solves both by tunnelling Ethernet frames over IP.

Encapsulation

graph LR
  A["Outer Ethernet<br/>14 bytes"] --> B["Outer IPv4<br/>20 bytes, VTEP to VTEP"] --> C["Outer UDP<br/>8 bytes, dst 4789"] --> D["VXLAN header<br/>8 bytes, 24-bit VNI"] --> E["Original Ethernet frame"]
Field Detail
VXLAN Network Identifier (VNI) 24 bits, about 16 million segments, compared with 4094 VLANs
UDP destination port 4789 (IANA)
UDP source port Derived from a hash of the inner frame's headers, so different flows take different ECMP paths through the underlay
Overhead 50 bytes with an IPv4 underlay (54 if the outer frame carries an 802.1Q tag)
Underlay MTU Must be raised (commonly to 9000 or more, at least 1550) so that 1500-byte host frames fit

VTEPs and VNIs

A VXLAN Tunnel Endpoint (VTEP) encapsulates and decapsulates. It is usually a switch (hardware VTEP) but can be a hypervisor's virtual switch (software VTEP). Each VTEP has an IP address on a loopback, reachable through the underlay. On Cisco switches the VTEP is the NVE (Network Virtualization Edge) interface.

VNI type Maps to Used for
Layer 2 VNI A VLAN or bridge domain Bridging within a segment across the fabric
Layer 3 VNI A VRF Routing between segments of one tenant across the fabric (symmetric IRB)

How VTEPs learn where MAC addresses are

VXLAN itself defines only the encapsulation. Something must tell each VTEP which remote VTEP a destination MAC lives behind, and how to deliver broadcast, unknown unicast and multicast (BUM) traffic.

Control plane option How it works Where you see it
Flood and learn, multicast underlay Each VNI maps to a multicast group. BUM traffic goes to the group and VTEPs learn remote MACs from data traffic Early VXLAN (RFC 7348 baseline)
Flood and learn, ingress replication The source VTEP sends a unicast copy of each BUM frame to every VTEP in a configured list Underlays without multicast
BGP EVPN MP-BGP (L2VPN EVPN address family) advertises MAC and IP routes and VTEP membership. Learning becomes control-plane based Cisco data centre (Nexus) and Catalyst BGP EVPN campus
LISP Edges register endpoints with the MS/MR and query it, as in the LISP section above SD-Access

VXLAN in SD-Access

SD-Access uses VXLAN-GPO (Group Policy Option), which uses reserved bits in the VXLAN header to carry a 16-bit group policy ID. That field carries the Scalable Group Tag (SGT) across the fabric, so policy can be enforced at the egress edge based on the source group, not just the IP address. The VNI identifies the virtual network (VRF) or the Layer 2 segment.

Exam trap

VXLAN is not a control plane and does not use TCP. It is a stateless UDP encapsulation. If an option says VXLAN "distributes endpoint reachability" or "retransmits lost frames", it is wrong. The control plane is flood and learn, BGP EVPN or LISP.

Exam trap

The 50-byte VXLAN overhead lands on the underlay. If the fabric underlay is left at 1500 bytes, small pings work and full-size frames are dropped. The fix is a larger underlay MTU, not a smaller host MTU.

What VXLAN configuration looks like

Recognition only. A Catalyst 9000 BGP EVPN leaf maps a VLAN to a VNI and defines the NVE interface:

vlan configuration 10
 member evpn-instance 10 vni 10010
!
interface nve1
 no ip address
 source-interface Loopback1
 host-reachability protocol bgp
 member vni 10010 ingress-replication
!
show nve peers        ! remote VTEPs and their state
show nve vni          ! VNIs, their mode and state

LISP and VXLAN together

Question Answer
Which protocol finds where an endpoint is? LISP (in SD-Access), BGP EVPN (in EVPN fabrics)
Which protocol carries the user frame between switches? VXLAN
Why not LISP's own data plane in SD-Access? LISP's standard data plane (RFC 9300) is built to carry IP packets. VXLAN carries whole Ethernet frames, so SD-Access can offer Layer 2 services and carry the SGT in the GPO header
Port numbers to remember LISP control 4342, LISP data 4341, VXLAN 4789

Lab: watch LISP resolve an EID

Topology: three Catalyst 8000v routers. CORE is the MS/MR. xTR1 and xTR2 are site edges. Only RLOCs are in the underlay routing protocol; the EID loopbacks are not advertised anywhere.

graph LR
  E1["Lo10 EID<br/>192.168.1.1/24"] --- X1["xTR1"]
  X1 -- "Gi1 10.1.3.1/30" --- C["CORE MS/MR<br/>Lo0 10.255.255.3"]
  C -- "Gi2 10.2.3.2/30" --- X2["xTR2"]
  X2 --- E2["Lo10 EID<br/>192.168.2.1/24"]
  1. Address the links as shown (CORE uses 10.1.3.2 on Gi1 and xTR2 uses 10.2.3.1). Run OSPF area 0 on the three links and CORE's Lo0 only. Do not advertise Lo10 on either xTR.
  2. Confirm the underlay: ping 10.2.3.1 source 10.1.3.1 works from xTR1, and show ip route 192.168.2.0 on xTR1 returns nothing.
  3. Configure CORE as MS/MR with sites SITE1 (192.168.1.0/24) and SITE2 (192.168.2.0/24), as in the MS/MR tab above.
  4. Configure xTR1 with the xTR tab above. Configure xTR2 the same way with database-mapping 192.168.2.0/24 10.2.3.1 priority 1 weight 100.
  5. On CORE, check registrations. On xTR1, ping the remote EID and inspect the map-cache.

Expected verification (formats vary slightly by release):

CORE# show lisp site
Site Name      Last      Up    Who Last             Inst     EID Prefix
               Register        Registered           ID
SITE1          00:00:31  yes   10.1.3.1                      192.168.1.0/24
SITE2          00:00:29  yes   10.2.3.1                      192.168.2.0/24

xTR1# ping 192.168.2.1 source Loopback10
.!!!!

xTR1# show ip lisp map-cache
LISP IPv4 Mapping Cache for EID-table default (IID 0), 2 entries

0.0.0.0/0, uptime: 00:04:10, expires: never, via static send map-request
  Negative cache entry, action: send-map-request
192.168.2.0/24, uptime: 00:00:06, expires: 23:59:53, via map-reply, complete
  Locator   Uptime    State      Pri/Wgt
  10.2.3.1  00:00:06  up           1/100

The first ping packet is lost while the Map-Request is resolved. The map-cache then holds the remote EID prefix with its RLOC and a 24-hour TTL. The global routing table on xTR1 still has no route to 192.168.2.0/24, which is the whole point: reachability comes from the mapping system, not from routing.

Explore: change the key on xTR2 only and watch SITE2 stop registering on CORE (show lisp site shows no under Up). Then capture on the CORE links in CML and find UDP 4342 (Map-Register, Map-Request) and UDP 4341 (data).

Quick check

In SD-Access, which protocol provides the data plane and which provides the control plane?

VXLAN (the GPO variant) is the data plane. LISP is the control plane, with the control plane node acting as MS/MR.

Why does VXLAN use a hash of the inner headers for the outer UDP source port?

To give the underlay entropy for ECMP. Different inner flows produce different outer source ports, so underlay routers spread them across equal-cost paths instead of sending all VTEP-to-VTEP traffic down one link.

A multihomed LISP site has two RLOCs with priority 1, weights 75 and 25. How is incoming traffic distributed?

Both RLOCs have the same (best) priority, so ITRs share traffic between them in proportion to the weights: about 75 percent to the first RLOC and 25 percent to the second.

References