Skip to content
Log in Sign up My account

Cisco ACI vs VMware NSX

ACI and NSX are both described as "software-defined networking for the data centre", and they solve different problems. This article explains what each one controls, how each is built, how their policy models compare, where each fits, and how they run together in the same data centre, which is more common than either vendor's marketing suggests.

The short version

  • Cisco ACI is a fabric. It controls the physical Nexus 9000 leaf and spine switches (running in ACI mode) as one system, managed by an APIC cluster. Policy is applied to anything that attaches to the fabric: VMs, bare metal, containers.
  • VMware NSX is a virtualization overlay. It controls networking and security at the hypervisor (and in Edge nodes) and does not manage your physical switches. It needs only an IP underlay between the hosts.

ACI is network-centric with strong physical fabric control. NSX is workload-centric with strong per-VM security. Neither is a superset of the other.

ACI architecture

graph TB
    APIC["APIC cluster<br/>3 or more controllers"] --> S1["Spine 1"]
    APIC --> S2["Spine 2"]
    S1 --- L1["Leaf 1"]
    S1 --- L2["Leaf 2"]
    S1 --- L3["Border leaf"]
    S2 --- L1
    S2 --- L2
    S2 --- L3
    L1 --- H1["Hosts, VMs"]
    L2 --- H2["Bare metal"]
    L3 --- EXT["External network<br/>L3Out"]

Key points:

  • The fabric is a leaf-spine Clos. Every leaf connects to every spine. Leaf switches connect to endpoints and to the outside world, spines only to leaves.
  • The underlay is built automatically at fabric discovery using IS-IS, with VXLAN as the data plane between leaf VTEPs. Operators do not configure the underlay by hand.
  • Spines hold the endpoint location database, learned through COOP, and use MP-BGP with EVPN-style route distribution for external routes.
  • APIC is the policy controller. The fabric keeps forwarding if APIC is unreachable, but you cannot change policy.
  • All configuration is an object in the management information tree and is available through a REST API.

ACI policy model

ACI policy is whitelist by default for communication between endpoint groups.

Object Meaning
Tenant Administrative and policy container
VRF Layer 3 routing context (a "context")
Bridge domain (BD) Layer 2 forwarding domain with associated subnets
Endpoint group (EPG) Group of endpoints that share the same policy
Application profile Container that groups related EPGs
Contract Rule set between EPGs: a provider and a consumer, made of subjects and filters
L3Out Routed connection to outside networks, using static routes or a routing protocol

By default, endpoints in different EPGs cannot communicate unless a contract allows it. Service graphs and policy-based redirect insert firewalls and load balancers into the path defined by a contract.

NSX architecture

graph TB
    NM["NSX Manager cluster"] --> H1["ESXi host<br/>transport node"]
    NM --> H2["ESXi host<br/>transport node"]
    NM --> E1["Edge node"]
    H1 --- U["IP underlay"]
    H2 --- U
    E1 --- U
    E1 --- PHY["Physical network<br/>BGP or static"]

Key points:

  • NSX Manager is the management and control plane. Transport nodes (hypervisor hosts and Edge nodes) run the data plane.
  • The overlay uses Geneve encapsulation. The physical network only needs to carry IP between transport node tunnel endpoints, with an MTU large enough for the encapsulation.
  • Logical switching is done with segments. Routing uses Tier-0 and Tier-1 gateways: Tier-1 handles tenant or application routing, Tier-0 connects to the physical network, usually with BGP.
  • Edge nodes (VM or bare metal) run centralized services such as north-south routing, NAT, load balancing and gateway firewalling.
  • The distributed firewall (DFW) enforces rules at the virtual NIC of every VM, regardless of where it sits in the topology.

NSX policy model

NSX policy is built from groups and security policies:

  • Groups have dynamic membership based on tags, VM names, segments or IP sets.
  • Security policies contain firewall rules that refer to groups, services and context profiles (application ID).
  • Rules are evaluated in categories (Ethernet, Emergency, Infrastructure, Environment, Application), top to bottom. The default rule is configurable. Many deployments start with allow and tighten toward deny.

The advantage is that policy follows the workload by tag. A VM tagged web-prod gets the same rules wherever vMotion puts it, and no VLAN or IP change is needed.

Comparison

Area ACI NSX
Scope Physical fabric plus attached workloads Virtual networking and security on hypervisors
Underlay Managed by APIC, Nexus 9000 only Any IP network with adequate MTU
Encapsulation VXLAN Geneve
Policy unit EPG and contract Group and DFW rule
Default posture Deny between EPGs Configurable, often allow initially
Enforcement point Leaf switch hardware Hypervisor vNIC
Bare metal Native Requires agents or Edge/bridging
Multi-site Multi-Site / Nexus Dashboard Orchestrator Federation, multi-location
Skills Fabric and routing engineers Virtualization and security engineers
Licensing Check current Cisco terms Check current VMware by Broadcom terms

The licensing and packaging of both products have changed repeatedly. Verify the current terms and supported combinations before designing around either.

Where each fits

ACI fits when:

  • You are building or refreshing the physical data centre fabric on Nexus 9000 anyway.
  • You have a lot of bare metal, mixed hypervisors, or container platforms alongside VMs.
  • The network team wants a single operational model and strict segmentation enforced in the network.

NSX fits when:

  • The estate is predominantly vSphere and the virtualization team owns the platform.
  • You need fine-grained, per-VM microsegmentation that follows the VM and works on any underlay.
  • You need a network virtualization layer across data centres where you do not control the physical fabric.

Neither may be needed if the real requirement is a standards-based leaf-spine with VXLAN EVPN. A standalone EVPN fabric managed by a controller such as Nexus Dashboard Fabric Controller, or by Ansible and templates, is a legitimate third option with less lock-in and a smaller operational surface.

Coexistence

Running both is common. The usual model is NSX overlay on an ACI underlay.

graph LR
    VM["VM on NSX segment"] --> TEP["Host TEP<br/>Geneve"]
    TEP --> LEAF["ACI leaf<br/>sees IP packets in<br/>an infrastructure EPG"]
    LEAF --> SPINE["ACI VXLAN fabric"]
    SPINE --> BL["Border leaf L3Out"]
    BL --> EDGE["NSX Edge uplink<br/>BGP peering"]

How it works in practice:

  1. ACI sees the NSX host tunnel endpoint traffic as ordinary IP between hosts. The host TEP VLAN or subnet is an EPG with a contract that allows host-to-host traffic. The Geneve payload is opaque to the fabric.
  2. VM-level policy is done in NSX. ACI provides connectivity, performance and physical segmentation (management, storage, vMotion, TEP networks).
  3. NSX Edge nodes peer with ACI through an L3Out on a border leaf, typically over BGP. Use a dedicated L3Out per Tier-0 uplink VLAN, and plan which side advertises the overlay prefixes.
  4. Set the MTU on the fabric, host uplinks and Edge nodes to carry Geneve overhead on top of the guest MTU. MTU mismatches show up as large packets dropping while pings succeed.
  5. Troubleshooting becomes two-layered. A VM-to-VM drop can be a DFW rule, an ACI contract on the TEP network, an MTU issue, or a routing problem at the L3Out. Agree early who owns which layer.

Tip

Keep the design rule simple: ACI secures and connects the infrastructure, NSX secures the workloads. Avoid duplicating the same workload-level policy in both, since two policy engines that must agree will drift.

ACI also integrates directly with vCenter through the VMM domain, where ACI pushes EPGs to a distributed virtual switch as port groups. That is an alternative to NSX for VM networking and is the more common ACI-only design. You choose between ACI VMM integration or NSX segments for VM networking per cluster, not both at once for the same VM interface.

Operational considerations

  • Automation: both have good APIs. ACI has an official Ansible collection (cisco.aci) and a Terraform provider. NSX has a REST API with Ansible modules and a Terraform provider. Define everything as code from the start, because GUI-built policy in either system becomes hard to audit.
  • Failure domains: the APIC cluster and the NSX Manager cluster are control planes. Plan their placement, backup and upgrade order. Test restores.
  • Upgrades: ACI upgrades are fabric-wide with a defined order (controllers, then switches in groups). NSX upgrades are coordinated through the Manager with host upgrades tied to the vSphere lifecycle. Both need maintenance windows and rollback plans.
  • Visibility: ACI gives you endpoint tracking and contract hit data in the fabric. NSX gives you flow data and DFW rule statistics per VM. Neither replaces a proper flow collector and packet capture capability.
  • Skill model: decide who is on call for what before the first production workload moves.

Making the decision

Ask these questions in order:

  1. Who owns the physical network and the hypervisor platform, and are they the same team?
  2. What is the mix of bare metal, containers and VMs?
  3. Is the fabric being replaced anyway?
  4. What does policy granularity need to be: segment, application tier, or individual VM?
  5. What are the current licensing and support terms for each option?

If the answers point to a mixed estate with a fabric refresh, ACI (or a standalone EVPN fabric) is the base. If they point to an all-VMware estate with a strong microsegmentation requirement, NSX is the main tool. If both are true, plan for coexistence from day one.

For the VXLAN and BGP concepts underneath both, see the CCNP course home.

References