Course contents

Unit 3: Underlay, overlay and TLOCLesson 3.4 (4 of 7 in this unit)17 of 27 in the Cisco SD-WAN course

System IP, site ID and device identity

What the system IP and site ID each identify, why the system IP is not an interface address, and how to plan numbering without duplicates.

Intermediate · 9 min read

You'll learn to say what the system IP and site ID each identify, explain why the system IP is not an interface address, and plan a numbering scheme that avoids duplicates.

Technical definition

System IP and site ID are the two settings that identify a device in the SD-WAN fabric. The system IP is a 32-bit dotted address that names one device, in the style of a router ID. The site ID is a number that names a location, shared by the devices at that location.

Two identifiers with different jobs

Every device in the fabric carries two identifiers, and mixing them up is a common source of confusion. The system IP names the device. The site ID names the place the device is in. Interface addresses, which change with each provider, name neither of these. In the lab, BR-01 has system IP 10.255.2.1 and site ID 101, BR-02 has 10.255.2.2 and 102, HUB-01 has 10.255.1.1 and 10, and HUB-02 has 10.255.1.2 and 20. The site IDs are teaching examples, not a rule.

System IP and site ID
System IP and site IDThe system IP identifies the device. The site ID identifies the location.System IPUnique per deviceLike a router IDSite IDSame for all devicesat a siteand

On small screens, swipe sideways or focus the diagram and use the arrow keys.

They do different jobs.
DeviceSystem IPSite IDRole
Manager10.255.0.10exampleManagement and monitoring
CTRL-01 (Controller)10.255.0.11exampleControl plane
VAL-01 (Validator)10.255.0.12exampleDevice discovery and authentication
HUB-01 (WAN Edge)10.255.1.110Hub
HUB-02 (WAN Edge)10.255.1.220Hub
BR-01 (WAN Edge)10.255.2.1101Branch
BR-02 (WAN Edge)10.255.2.2102Branch

What the system IP is used for

  • Identity in OMP. Each route and TLOC an edge advertises is tied to its system IP as the originator, so the Controller and other edges know where it came from.
  • The first part of a TLOC. A TLOC is the system IP plus a colour and an encapsulation, as the TLOC lesson explains. One system IP, several TLOCs.
  • Router ID for fabric protocols. It is the device's ID in the control plane and in places such as BFD session lists, where it shows as the peer.
  • A stable name for monitoring. The Manager lists devices by it, so it should not change when a provider changes the WAN address.

Because it names the device and not an interface, it stays the same when a link is added, re-addressed or replaced. That stability is the reason for having it. It is not an address on any subnet of the underlay, so picking one that falls in a range already used by the transport networks is legal but confusing; many designs keep a block for system IPs, as the 10.255.0.0/16 range does in this course's lab.

System IP uses
System IP usesIt identifies the device in the control plane and in OMP.System IPOMPidentifies the routerTLOCpart of the identityManagementlogs and sessions

On small screens, swipe sideways or focus the diagram and use the arrow keys.

It is not used for forwarding user traffic.

What the site ID is used for

The site ID groups devices by location, and policy uses it. A control policy can say which sites see which routes, and a topology such as hub-and-spoke is written in terms of site lists. It is a number, so a clear scheme helps: one range for hubs, one for branches, one for data centres, for example. The ranges in this course are examples; check the limits and any reserved values for your release before choosing yours.

Site ID uses
Site ID usesIt groups devices and drives policy and loop checks.Site IDPolicymatch by siteLoop preventionroutes from the same siteDesignnumbering plan

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Two devices at one site share an ID.

Two properties matter for design. Devices at the same location share a site ID. And the Controller uses the site ID to avoid sending a router its own site's routes back, so giving two different locations the same ID can hide routes between them.

Where the identifiers are configured

Where identity is configured
Where identity is configuredBoth go in the system section of the device.System sectionsystem-ip, site-idTemplatepushed by ManagerCheckshow sdwan control

On small screens, swipe sideways or focus the diagram and use the arrow keys.

They come from the template or bootstrap.
system system-ip 10.255.2.1 site-id 101 organization-name LAB-EXAMPLE sp-organization-name LAB-EXAMPLE ! sdwan ...

Fragment of an IOS XE SD-WAN configuration for BR-01, shortened, with a placeholder organisation name. Most networks push these settings from a Manager template or configuration group, and the exact keywords can differ by release, so check yours.

Example output written for this lesson, simplified; real output has more fields and a different layout
BR-01#show sdwan control
personality        vedge
organization-name  LAB-EXAMPLE
system-ip          10.255.2.1
site-id            101
...
The router reports the identity it is using. If the Manager shows a different system IP or site ID from the one you intended, compare it with this output first. The lines marked with dots stand for fields left out here.
Example output written for this lesson, simplified; real output has more fields and a different layout
BR-01#show sdwan omp peers
PEER         TYPE    DOMAIN ID  OVERLAY ID  SITE ID  STATE
10.255.0.11  vsmart  1          1           1        up
The Controller is identified by its own system IP, and its site ID is listed. Everything in the fabric, controllers included, has both identifiers.

Verify identity and reachability separately

An up BFD session supports liveness of that tunnel path. Service-VPN applications require their own forwarding checks. A failed underlay ping to an unadvertised identity is not, by itself, a tunnel-failure test. If the same address is deliberately assigned to a management loopback in a service VPN, test it in that VPN and check its return route and policy.

Identity and reachability
Identity and reachabilityA router can have the right identity and no reachability.Identitysystem IP, site ID, orgReachabilitypath to the controllersand

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Check them separately.
show sdwan bfd sessions

Read-only IOS XE Catalyst SD-WAN check: match the intended remote system IP and local/remote colours before reading the session state. One up session does not describe every transport or application.

For a service that still fails, verify the actual destination prefix in the intended service VPN, the return path and the applied policy; test from the intended client or source. Keep the existing local-properties identity check separate from this forwarding check. The examples use IOS XE Catalyst SD-WAN command style; exact fields depend on the selected platform and release. See Cisco's SD-WAN design guide for system identity, service-VPN loopbacks and BFD path monitoring.

Planning checklist

  1. Reserve a block for system IPs and keep a list of every one assigned, including controllers.
  2. Pick a site ID scheme by role or region and write it down before the first device is onboarded.
  3. Give each location one site ID; give devices at that location the same one.
  4. Avoid reusing a system IP from a retired device until it is removed from the Manager.
  5. Check your release's guide for the allowed values before settling the scheme.

Learn more: TLOC Components and Transport ReachabilityOrganization Name and Certificate DomainOMP Peers and Exchanges

Mistakes with identity

  • Trying to ping the system IP across the underlay and concluding the router is down when it does not answer. It is an identity, not a transport address.
  • Reusing one system IP on a replacement router while the old entry is still in the Manager, which causes conflicts.
  • Giving each branch router in a redundant pair a different site ID. Policies written per site then treat one location as two.
  • Giving two sites the same ID by copying a template, so that routes between them are suppressed.

Test yourself on identity

Predict · scenario 1

BR-01 has two WAN interfaces and system IP 10.255.2.1. A colleague asks which of its interface addresses is the system IP. What is the right answer?

Predict · scenario 2

A branch has a redundant pair of WAN Edges. How should they be numbered?

Predict · scenario 3

An underlay ping to BR-01's system IP 10.255.2.1 fails. That identity was not advertised into the underlay, and the intended BFD session is up. What can you conclude?

FAQ

Does the system IP have to be reachable?
The underlay does not need a route to the system IP just because it is the fabric identity. A design can also assign that address to a loopback and advertise it in a service VPN for management. Whether a ping should succeed then depends on that VPN's routes, source address and policy. State which routing context you are testing.
Can two routers share a system IP?
No. The system IP identifies one device, and a duplicate breaks the identity that routes, TLOCs and control connections depend on. Plan the numbering so that every device, controllers included, is unique.
Do all routers at one site share a site ID?
Yes, that is its purpose. Two WAN Edges at the same branch, for example a redundant pair, normally use the same site ID, and each has its own system IP. Devices at different sites use different site IDs.