Course contents

Unit 13: Advanced scenariosLesson 13.2 (2 of 4 in this unit)23 of 25 in the Advanced Cisco SD-WAN course

Scenario: last-resort circuit for a single application

A branch whose cellular backup may carry one critical application and nothing else: the policy, the limits and the test.

Advanced · 12 min read

You'll learn to keep a metered last-resort circuit for the one application that must survive, choose between policy, segmentation and rate limiting, and test what really crosses the link.

Technical definition

Restricting a last-resort circuit means limiting which traffic may use the backup transport once it is active. A last-resort circuit decides when tunnels come up. It does not decide which flows use them, so the restriction has to come from policy, from the VPN layout or from rate limiting on the interface.

The tills must keep working, and nothing else needs to

Branches like BR-02 (site 102) have one internet line and a cellular LTE modem configured as a last-resort circuit. The LTE plan is metered. When the internet line fails, the business needs one thing: the point-of-sale terminals in VPN 40 (10.40.2.0/24) must reach the payment gateway behind the hub, here an example address in 10.20.40.0/24. Guest Wi-Fi, software updates and video calls should wait until the line is repaired.

The difficulty is that the last-resort circuit does not know about applications. It brings the LTE tunnels up when the other tunnel interfaces are gone. From then on the router has tunnels to the hub, and every flow that has a route to the hub can take them. A guest laptop starting a large download is treated the same as a till. The restriction is a separate piece of design.

Constraints that decide the design

  • The plan has a data cap or a high cost per megabyte. Check the contract: the numbers are not in this lesson.
  • LTE is slower and has higher latency than a wired line. A payment is small, but a rejected payment costs more than a slow one.
  • Carriers often put a mobile subscriber behind carrier-grade NAT (CGNAT) and drop idle mappings. A tunnel that sends little can lose its mapping, so keepalives matter. Ask the carrier what applies to your plan.
  • The terminals are not the only users of the path. Things that look unrelated need a route too, and they are listed below.

The traffic that is easy to forget

The traffic that is easy to forget
The traffic that is easy to forgetTraffic; Why the till fails without it; Where it goes. DNS for the gateway name: A terminal that cannot resolve the gateway name cannot pay, although the route exists If the resolver is in the data centre, it needs the path too. If it is on the internet, it is not reachable in this state NTP: Payment systems and TLS often reject a clock that has drifted Same question: where is the time source? Hub-initiated flows: Reverse connections, terminal management or polling from the data centre start from the hub, not the branch A branch outbound policy does not cover them. They arrive through the LTE tunnel too Management and control traffic: The edge keeps its control connections, and Manager-driven jobs such as software upgrades or template pushes can run Over LTE. Pause large jobs while a site is on its last resort Return traffic from the gateway: The gateway replies to 10.40.2.0/24 and the reply comes back over the LTE tunnel Needs the same permission in the inbound directionTrafficWhy the till fails without itWhere it goesDNS for the gatewaynameA terminal that cannot resolve thegateway name cannot pay, although theroute existsIf the resolver is in the datacentre, it needs the path too. If itis on the internet, it is notreachable in this stateNTPPayment systems and TLS often rejecta clock that has driftedSame question: where is the timesource?Hub-initiated flowsReverse connections, terminalmanagement or polling from the datacentre start from the hub, not thebranchA branch outbound policy does notcover them. They arrive through theLTE tunnel tooManagement and controltrafficThe edge keeps its controlconnections, and Manager-driven jobssuch as software upgrades or templatepushes can runOver LTE. Pause large jobs while asite is on its last resortReturn traffic from thegatewayThe gateway replies to 10.40.2.0/24and the reply comes back over the LTEtunnelNeeds the same permission in theinbound direction

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.

Decide for each row where the service lives. A design that permits the till and forgets DNS looks fine in the policy and fails in the shop.

Four ways to keep the link for the till

Four ways to keep the link for the till
Four ways to keep the link for the tillOption; How it limits traffic; Cost; Risk; Complexity. A. Data policy at the branch: Allows the till and keeps other flows off the LTE tunnels Low (no extra circuit) A wrong match blocks the till or lets the rest through. Hard to test in production Medium to high B. Only the till VPN on the fallback sites: Other VPNs do not exist at these sites, so nothing else can use the path Low Staff lose guest and other services at those sites on purpose. A site needing a second VPN later needs a redesign Low C. Shaping or policing on the LTE interface: Caps the rate, not the type of traffic Low A cap protects the bill but lets the wrong traffic compete with the till Low to medium D. Allow everything and size the plan: Nothing is restricted Highest data plan Surprise bill, or a plan cap that blocks the till as well LowestOptionHow it limitstrafficCostRiskComplexityA. Data policy atthe branchAllows the tilland keeps otherflows off the LTEtunnelsLow (no extracircuit)A wrong matchblocks the tillor lets the restthrough. Hard totest inproductionMedium to highB. Only the tillVPN on thefallback sitesOther VPNs do notexist at thesesites, so nothingelse can use thepathLowStaff lose guestand otherservices at thosesites on purpose.A site needing asecond VPN laterneeds a redesignLowC. Shaping orpolicing on theLTE interfaceCaps the rate,not the type oftrafficLowA cap protectsthe bill but letsthe wrong trafficcompete with thetillLow to mediumD. Alloweverything andsize the planNothing isrestrictedHighest data planSurprise bill, ora plan cap thatblocks the tillas wellLowest

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.

These options combine. A realistic design for many small sites is B for the sites that can accept it, and A plus C for the rest. D is acceptable when the plan is large and the outages are short, and it should be a decision someone has written down.

Option A: a data policy, described by its intent

A centralised data policy is built in SD-WAN Manager, held by the Controller and applied to a site list. It matches traffic going through the WAN Edge and acts on it. The intent here has two parts:

  1. Match the till traffic: VPN 40, source 10.40.2.0/24, destination the payment gateway prefix (or the application, if the router can identify it in your release). Allow it, with no restriction on colours.
  2. Match everything else that must stay off the LTE link and tie it to the colours it may use. If its allowed colour is not available, the traffic has no permitted path and is dropped. The data policy actions that pin traffic to local colours are the ones to look for. Some releases have an option that makes the choice strict instead of falling back to any other colour. The name and the exact effect depend on your release, so check the policy builder and the documentation.
policy lists vpn-list VPN-POS vpn 40 ! prefix-list POS-TERMINALS ip-prefix 10.40.2.0/24 ! prefix-list PAY-GATEWAY ip-prefix 10.20.40.0/24 ! site-list LTE-FALLBACK-SITES site-id 102 ! ! data-policy POS-ONLY-ON-LTE vpn-list VPN-POS sequence 10 match source-ip 10.40.2.0/24 destination-data-prefix-list PAY-GATEWAY ! action accept ! default-action accept ! ! ! apply-policy site-list LTE-FALLBACK-SITES data-policy POS-ONLY-ON-LTE from-service ! !

Outline of structure only, shortened and not tested on a device. It shows the part that allows the till traffic. The part that keeps other VPNs off the LTE colour (a policy on the other VPNs that names the colours they may use, with a strict or restrict option) is left out because its keywords differ by release. Verify every keyword in the policy builder of your release. from-service means traffic arriving from the branch LAN; traffic arriving from the hub through a tunnel is covered by a policy applied in the other direction.

The limit to remember is that a data policy matches traffic and picks an action, but it has no built-in notion of "the last-resort circuit is active now". The restriction works by saying which colours a flow may use. When the preferred colour is present, the flow uses it, and when only the LTE colour is left, the flow has either no permitted path or an LTE one, depending on the action. That is why the exact action matters more than the match, and why this option needs the most testing.

Option B: fewer VPNs at the fallback sites

If a site has no VPN 10 or VPN 30, no flow from those VPNs exists to take the LTE path. This is the cheapest restriction and the one least likely to surprise you, because it is configuration at one site instead of a policy decision at runtime. The price is that the site does not get those services, over any transport. It suits very small sites, such as a kiosk with a till and nothing else.

Watch for a default route in the service VPN. If guest or local internet traffic uses a local breakout (direct internet access with NAT) and the LTE interface becomes the only way out, that traffic can leave straight over LTE without any tunnel. This is a second path to the same bill, so check how the local breakout route is built before assuming it is off.

Option C: a rate cap on the interface

A policer drops or re-marks traffic above a rate. A shaper holds the excess in a queue and sends it later, within limits. For a slow link, shaping is normally kinder to TCP: it adds delay but loses fewer packets. Either one limits how much the link carries and not which application gets it, so combine a cap with a queue that gives the till traffic priority. The QoS policy is applied to the LTE interface, and its form depends on the platform and release.

What the router shows during the outage

Example output written for this lesson, simplified; real output has more fields and a different layout
BR-02#show sdwan bfd sessions
SYSTEM IP     SITE  STATE  SOURCE TLOC   REMOTE TLOC   SOURCE IP     DST PUBLIC IP   UPTIME
10.255.1.1    10    up     lte           biz-internet  100.64.0.20   198.51.100.20   0:00:04:12
10.255.1.2    20    up     lte           biz-internet  100.64.0.20   198.51.100.40   0:00:04:10
Only the LTE colour has tunnels, to both hubs. The biz-internet sessions that were here before the failure are gone. This is the moment the policy matters: any flow with a route to the hubs can use these two tunnels. The addresses are examples (100.64.0.0/10 is the shared space often used behind carrier NAT). Real column names and layout differ.
show sdwan policy from-vsmart

Shows the policy this router has received from the Controller. If the till policy is not listed here, the site list or the apply-policy direction is wrong. The command name varies a little between releases.

show sdwan policy service-path vpn 40 interface GigabitEthernet3 source-ip 10.40.2.15 dest-ip 10.20.40.10 protocol 6

Asks the router which path a till flow would take under the current policy. GigabitEthernet3 stands for the LAN-side interface in VPN 40; use your own. Run the same command with a guest address in its own VPN to see the difference. Keywords vary between releases.

Counters on the LTE interface (show interfaces) then show how many bytes cross it. Take a reading before the test and after, and compare with what the till traffic should be.

Test plan

Plan for a lab first and a maintenance window second. Notify the carrier-plan owner before a test that will use real data.

Test plan
Test planStep; Check; Expected result. 1. Normal state: BFD sessions and interface counters LTE tunnels down, LTE interface counters nearly still 2. Fail the internet line: Time until the LTE tunnels are up; till transaction to the gateway Transaction works. Record the switchover time 3. Try the other traffic: Guest browsing, a software download, a video call, ping from the guest VPN Blocked or unreachable, according to the design. Anything that works is a finding 4. Check DNS and NTP: Name resolution and time sync from a terminal Both work, or the design says how the terminal copes without them 5. Hub-initiated traffic: A connection from the data centre to a terminal As the design intends. If it works, it is using the plan 6. Measure data: Interface bytes over a set time with idle terminals Idle cost: BFD, control traffic and keepalives. Multiply by the length of a plausible outage 7. Restore the line: Time for traffic to leave LTE; LTE tunnels down again No flapping, and the LTE counters stop growingStepCheckExpected result1. Normal stateBFD sessions and interface countersLTE tunnels down, LTE interfacecounters nearly still2. Fail the internetlineTime until the LTE tunnels are up;till transaction to the gatewayTransaction works. Record theswitchover time3. Try the othertrafficGuest browsing, a software download,a video call, ping from the guest VPNBlocked or unreachable, according tothe design. Anything that works is afinding4. Check DNS and NTPName resolution and time sync from aterminalBoth work, or the design says how theterminal copes without them5. Hub-initiatedtrafficA connection from the data centre toa terminalAs the design intends. If it works,it is using the plan6. Measure dataInterface bytes over a set time withidle terminalsIdle cost: BFD, control traffic andkeepalives. Multiply by the length ofa plausible outage7. Restore the lineTime for traffic to leave LTE; LTEtunnels down againNo flapping, and the LTE countersstop growing

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.
Why measure data with the terminals idle?

Because tunnels over LTE cost data even when no user does anything: BFD probes, the control connection and keepalives that hold the carrier NAT mapping open. That idle cost sets the minimum price of every hour on the backup link. The till traffic comes on top of it, and is often the smaller part.

What this design cannot promise

  • A policy that blocks traffic cannot be proven in production without an outage. Lab results and one supervised test are the best available evidence, and they should be repeated after any policy or release change.
  • Carrier behaviour varies by plan and location. CGNAT, idle timeouts, throttling after a cap and radio coverage are outside the SD-WAN configuration.
  • The restriction is only as good as the list of what the till depends on. A forgotten dependency shows up as an application error that looks unrelated to the network.

Test yourself on the last-resort single application

Predict · scenario 1

BR-02's internet line fails and the LTE tunnels come up. Nothing else is configured. A guest starts a large download. What happens?

Predict · scenario 2

The data policy allows 10.40.2.0/24 to the payment gateway. In the outage the till cannot pay, but pings to the gateway work. What do you check first?

Predict · scenario 3

A site has only the till VPN and an LTE last-resort circuit. The cost group is worried about guest traffic using the plan. Which statement is accurate?

Predict · scenario 4

You want to protect the data plan from a runaway upload but keep the design simple. You add a policer at a low rate on the LTE interface. What is the main limit of that choice?

FAQ

Does the last-resort circuit limit traffic to important applications?
No. It only controls when the tunnels on that interface come up. Once they are up, any flow that has a route to the hub can use them, including guest browsing and software updates, unless a policy, the VPN layout or an interface policy stops it.
Is a policer or a shaper better on the LTE interface?
Shaping is usually the better tool for a link with a known, low rate, because it queues and delays packets instead of dropping the excess. A policer drops or re-marks. It protects the data plan but makes TCP back off by loss. Neither one chooses which application gets the bandwidth.
Can the policy be tested safely in production?
Not fully. The real test needs the internet line to fail, which is an outage. Do the first tests in a lab or in a maintenance window, and keep a rollback. A policy that looks right in the Manager can still be wrong in a corner you did not think of.