Logical overlay and encrypted tunnel construction

Two conversations on the same links

When BR-01 joins the fabric it starts two separate conversations over the same WAN links. One is with the controllers: BR-01 proves who it is, then receives routes and policy. The other is with other WAN Edges, such as HUB-01: BR-01 sends user traffic to them in encrypted tunnels. The controllers are only in the first conversation. Keeping them apart is what lets one central control system manage hundreds of sites without carrying their traffic.

The dashed blue paths carry control information to the Controller. The green path carries user packets between BR-01 and HUB-01, with BFD (Bidirectional Forwarding Detection) monitoring that tunnel. The diagram uses system IPs as device identities, not as WAN interface addresses. Manager and Validator connections are omitted here so the two paths remain clear.

The build order

1. Transport comes up. The WAN Edge has an address and a route on each transport, as in the transport-side lesson. Without it, nothing below can start.
2. Discovery through the Validator. The WAN Edge contacts the Validator (formerly vBond) first. The Validator authenticates it, tells it which Controller and Manager to use, and helps it discover its public address if NAT is present. This discovery connection is normally torn down after the required Controller and Manager connections form; some scenarios retain it.
3. Control connections. The WAN Edge opens its own secure connection to the Controller (formerly vSmart) and to the Manager (formerly vManage), using the certificates covered in later lessons. These run over DTLS or TLS; check your release for which is used and its ports.
4. OMP session. Over the Controller connection, the WAN Edge starts OMP (Overlay Management Protocol). It advertises its prefixes, its TLOCs and the keys other sites need, and receives the information permitted by control policy. It does not necessarily receive every site’s prefixes or TLOCs.
5. Tunnels between TLOCs. A TLOC (transport locator) identifies an endpoint by system IP, colour and encapsulation. An eligible remote TLOC, usable transport address/port, reachability and policy allow this example’s direct IPsec tunnel to form. Learning a TLOC alone does not guarantee a tunnel.
6. BFD inside each tunnel. When BFD comes up, the tunnel can carry traffic and the routes that depend on it are usable.

Steps 2 to 4 are the control plane. Steps 5 and 6 are the data plane. The fabric depends on all of them, but a fault in one produces different symptoms from a fault in another, which is why they are worth keeping separate.

What each protocol is responsible for

The key point for troubleshooting: OMP being up does not prove that a particular data tunnel is usable, and a tunnel being up does not mean the right routes are present. Each layer has its own show command.

How the data plane stays encrypted

The tunnels use IPsec. Rather than each pair of routers negotiating keys with each other, the usual model in the documentation is that each WAN Edge generates its keys and advertises them through OMP, so the encryption material arrives with the control information the router already receives. Which ciphers, authentication options and rekey intervals apply depends on your release and configuration, so check them in the configuration guide instead of assuming them.

Tunnels are also usually created to every TLOC the router is allowed to reach, not only when traffic appears, which is why BFD can measure them continuously. Designs that build some tunnels only on demand exist and belong to a later lesson.

Reading the overlay from the router

BR-01#show sdwan control connections
PEER    PEER      PEER         SITE  DOMAIN  PEER PRIV      STATE
TYPE    PROTOCOL  SYSTEM IP    ID    ID      IP
vsmart  dtls      10.255.0.11  1     1       198.51.100.31  up
vmanage dtls      10.255.0.10  1     0       198.51.100.30  up

BR-01#show sdwan omp peers
PEER         TYPE    DOMAIN ID  OVERLAY ID  SITE ID  STATE
10.255.0.11  vsmart  1          1           1        up

BR-01#show sdwan bfd sessions
SYSTEM IP    SITE  STATE  SOURCE COLOR  REMOTE COLOR  ENCAP
10.255.1.1   10    up     biz-internet  biz-internet  ipsec
10.255.1.2   20    up     biz-internet  biz-internet  ipsec

Learn more: OMP Peers and Exchanges · TLOC Components and Transport Reachability · Application-Aware Routing

Which layer to blame

Verify one failure and its recovery

Use an isolated, already onboarded lab with a permitted BR-01 to HUB-01 tunnel. Record control connections, OMP peer state, the expected service prefix and its BFD session. On IOS XE Catalyst SD-WAN 17.x, show sdwan omp routes checks advertised overlay routes; show ip route vrf 10 checks the installed service route when this lab’s service VPN is 10. Inspect the next hop and policy as well as whether a row exists.

If the Controller remains up while HUB-01’s BFD session goes down, compare the two WAN Edge transport endpoints, TLOC eligibility, NAT/firewall handling and IPsec state. Restore only the known lab fault, verify BFD returns, then check the service route and a permitted user flow in both directions. An absent session can also mean no eligible remote TLOC was learned, so do not assume the control policy is correct from connection state alone.

Control up, BFD up and the correct installed service route are separate pieces of evidence. Even all three need an end-to-end application test and return path. Do not bring down a shared production controller for this exercise.

Technical references: Cisco’s design guide and IOS XE 17.x BFD and high-availability guide. Syntax and retention depend on release; examples are authored, not device captures.

Mistakes when learning the overlay

• Believing that traffic is routed through the Controller. It distributes information only.
• Assuming every Validator connection must disappear immediately. Discovery is its main role, but connection retention depends on the control topology and configuration; it does not carry user traffic.
• Expecting a full mesh of OMP sessions between branches. OMP peers with the Controller, and tunnels between sites are a different mechanism.

Test yourself on the overlay