Course contents

Unit 7: Cloud and hybrid WANLesson 7.5 (4 of 4 in this unit)17 of 25 in the Advanced Cisco SD-WAN course

Cloud OnRamp with AWS: design options

Ways to connect an SD-WAN fabric to AWS (transit VPC, Transit Gateway, VPN and Direct Connect) and the trade-offs of each.

Advanced · 14 min read

You'll learn to compare the main ways to connect an SD-WAN fabric to AWS, ask the design questions that decide between them, and predict which routes exist where in a Transit Gateway design.

Technical definition

AWS connection design for an SD-WAN fabric is the choice of how branches and data centres reach workloads in AWS: through SD-WAN routers in a transit VPC, through an AWS Transit Gateway, through site-to-site VPN, through Direct Connect, or through a combination. Each choice trades scale, throughput, cost, effort and SD-WAN features against each other.

One cloud, five ways in

AWS gives several ways to attach a network, and Cisco's automation supports some of them. Amazon names and features change often, and so do the combinations Cisco supports, so this lesson describes the options and trade-offs and leaves exact support to the current documentation. VPC means virtual private cloud, a private network in an AWS account. A Transit Gateway is an AWS regional router that connects many VPCs and other networks.

The options

The options
The optionsOption; How it works; Where it fits. (a) Transit VPC with cloud routers: A pair of SD-WAN virtual routers in a central VPC joins the fabric. Spoke VPCs connect to them, commonly with IPsec tunnels When you want SD-WAN features on all cloud traffic and the number of spokes is modest. This is the older pattern (b) Transit Gateway as the hub: VPCs attach to a Transit Gateway. SD-WAN virtual routers connect to it, and exchange routes with it. Transit Gateway Connect attachments exist for this, using GRE, but check the supported options for your release Many VPCs and accounts, with AWS handling the VPC-to-VPC connections (c) Site-to-site VPN: An IPsec tunnel from a branch or hub straight to an AWS VPN endpoint Few VPCs, low to moderate traffic, simplest operation. No SD-WAN router in AWS (d) Direct Connect: A private circuit from a colocation site or provider to AWS. It can be an extra transport colour in the fabric or end at a hub Large or steady traffic, predictable latency, no use of the public internet (e) A combination: For example Direct Connect as the primary path and VPN over the internet as the backup Where an outage of the primary path is not acceptableOptionHow it worksWhere it fits(a) Transit VPC withcloud routersA pair of SD-WAN virtual routers in acentral VPC joins the fabric. SpokeVPCs connect to them, commonly withIPsec tunnelsWhen you want SD-WAN features on allcloud traffic and the number ofspokes is modest. This is the olderpattern(b) Transit Gateway asthe hubVPCs attach to a Transit Gateway.SD-WAN virtual routers connect to it,and exchange routes with it. TransitGateway Connect attachments exist forthis, using GRE, but check thesupported options for your releaseMany VPCs and accounts, with AWShandling the VPC-to-VPC connections(c) Site-to-site VPNAn IPsec tunnel from a branch or hubstraight to an AWS VPN endpointFew VPCs, low to moderate traffic,simplest operation. No SD-WAN routerin AWS(d) Direct ConnectA private circuit from a colocationsite or provider to AWS. It can be anextra transport colour in the fabricor end at a hubLarge or steady traffic, predictablelatency, no use of the publicinternet(e) A combinationFor example Direct Connect as theprimary path and VPN over theinternet as the backupWhere an outage of the primary pathis not acceptable

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.
Four ways to connect the fabric to one AWS network
Four ways to connect the fabric to one AWS networkBR-01 and HUB-01 can reach the AWS network 10.100.0.0/16 in four ways: a transit VPC with router instances, a Transit Gateway with SD-WAN routers attached, an IPsec site-to-site VPN from the edge, or a Direct Connect circuit. They differ in scale, throughput, effort, cost, segmentation and resilience.BR-01 and HUB-01Transit VPCrouters you run in a VPCTransit GatewayAWS-managed attachmentsSite-to-site VPNIPsec from the edge, low costDirect Connecta circuit with set capacityAWS 10.100.0.0/16

On small screens, swipe sideways or focus the diagram and use the arrow keys.

These are options, not a ranking. The right answer depends on the number of VPCs, the throughput you need and how much you want to operate.

Trade-offs compared

These ratings are relative and general, not measurements. Your design, instance sizes and AWS limits change them.

Trade-offs compared
Trade-offs comparedAspect; Transit VPC; Transit Gateway; Site-to-site VPN; Direct Connect. Scale in VPCs: Modest; each spoke needs its own connection High; attachments are managed by AWS Low High with the right gateways Throughput: Limited by router instance size Limited by the SD-WAN routers and the attachment type Limited per tunnel Set by the circuit Operational effort: High; you run routers and tunnels Medium; fewer tunnels, more AWS objects Low Medium; lead time and provider coordination Cost: Instances and data transfer Routers plus per-attachment and data charges Low fixed cost Port and circuit fees, often lower transfer charges Segmentation: Per VPN at the router Per VPN at the router, plus Transit Gateway route tables Per tunnel Per virtual interface Resilience: Pair of routers across zones Pair of routers; Transit Gateway is regional Two tunnels per connection by design Needs a second circuit for real redundancy Visibility: SD-WAN monitoring on cloud traffic SD-WAN monitoring up to the router; AWS tools beyond Tunnel state only Circuit state; SD-WAN view only if it ends on a fabric routerAspectTransit VPCTransit GatewaySite-to-site VPNDirect ConnectScale in VPCsModest; eachspoke needs itsown connectionHigh; attachmentsare managed byAWSLowHigh with theright gatewaysThroughputLimited by routerinstance sizeLimited by theSD-WAN routersand theattachment typeLimited pertunnelSet by thecircuitOperational effortHigh; you runrouters andtunnelsMedium; fewertunnels, more AWSobjectsLowMedium; lead timeand providercoordinationCostInstances anddata transferRouters plusper-attachmentand data chargesLow fixed costPort and circuitfees, often lowertransfer chargesSegmentationPer VPN at therouterPer VPN at therouter, plusTransit Gatewayroute tablesPer tunnelPer virtualinterfaceResiliencePair of routersacross zonesPair of routers;Transit Gatewayis regionalTwo tunnels perconnection bydesignNeeds a secondcircuit for realredundancyVisibilitySD-WAN monitoringon cloud trafficSD-WAN monitoringup to the router;AWS tools beyondTunnel state onlyCircuit state;SD-WAN view onlyif it ends on afabric router

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.

Questions to settle before choosing

  • How many VPCs and accounts? Five VPCs and fifty need different designs.
  • Do any prefixes overlap? Overlap between VPCs, or with branches and data centres, is a blocker for a simple design.
  • How much traffic, and between what? Branch to VPC, VPC to VPC, or both. VPC-to-VPC traffic does not need to cross the SD-WAN routers at all.
  • Who owns routing? If the cloud team runs the Transit Gateway route tables, agree who may change what.
  • What does compliance require? Traffic that must stay off the public internet points to Direct Connect or at least a documented encryption path. Inspection requirements add a firewall into the path.
  • How long can the connection be down? That sets the second path and the failover test.

A worked example: BR-01 and HUB-01 reach 10.100.0.0/16 through Transit Gateway

The VPC 10.100.0.0/16 is attached to a Transit Gateway. A pair of cloud routers connects to the Transit Gateway and is part of the fabric. The VPC is mapped to VPN 10. Site IDs and system IPs are lab examples. The cloud routers use the made-up system IPs 10.255.3.1 and 10.255.3.2.

A worked example: BR-01 and HUB-01 reach 10.100.0.0/16 through Tran...
A worked example: BR-01 and HUB-01 reach 10.100.0.0/16 through Tran...Where; Route to know; Learned from. VPC route table: Branch and data centre prefixes (for example 10.10.1.0/24, 10.20.1.0/24) via the Transit Gateway Configured or propagated by the cloud team or the automation Transit Gateway route table: 10.100.0.0/16 towards the VPC attachment. Branch prefixes towards the SD-WAN router attachments Propagation from attachments and from the routers Cloud routers: 10.100.0.0/16 from the Transit Gateway. Branch and data centre prefixes from OMP The Transit Gateway and the SD-WAN Controller HUB-01 and BR-01 (OMP table): 10.100.0.0/16 via the cloud routers' system IPs, in VPN 10 OMP, advertised by the cloud routersWhereRoute to knowLearned fromVPC route tableBranch and data centre prefixes (forexample 10.10.1.0/24, 10.20.1.0/24)via the Transit GatewayConfigured or propagated by the cloudteam or the automationTransit Gateway routetable10.100.0.0/16 towards the VPCattachment. Branch prefixes towardsthe SD-WAN router attachmentsPropagation from attachments and fromthe routersCloud routers10.100.0.0/16 from the TransitGateway. Branch and data centreprefixes from OMPThe Transit Gateway and the SD-WANControllerHUB-01 and BR-01 (OMPtable)10.100.0.0/16 via the cloud routers'system IPs, in VPN 10OMP, advertised by the cloud routers

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.
Example output written for this lesson, simplified; real output has more fields and a different layout
BR-01#show sdwan omp routes 10.100.0.0/16
VPN  PREFIX          FROM PEER    PATH ID  LABEL  STATUS  TLOC IP       COLOR         PREF
10   10.100.0.0/16   10.255.0.11  301      1002   C,I,R   10.255.3.1    biz-internet  -
10   10.100.0.0/16   10.255.0.11  302      1002   C,I,R   10.255.3.2    biz-internet  -
BR-01 learns the VPC prefix from the Controller (10.255.0.11), with both cloud routers as the next hop. The route exists for VPN 10 only. If a prefix is missing here, trace it back one table at a time: the Transit Gateway route table, then the router, then OMP. Each table in the list above is a separate place for the route to be lost.

In the other direction, a packet from a VPC host to 10.10.1.0/24 travels to the Transit Gateway, to a cloud router, through an SD-WAN tunnel to the branch or hub that advertises the prefix, and on to the destination. Return traffic needs the VPC route table to point the branch prefixes at the Transit Gateway. A missing return route is a frequent cause of one-way traffic.

Why does HUB-01 matter here, if BR-01 reaches the cloud routers directly?

Both can reach them, because the fabric is a full mesh by default and tunnels form between any two sites that have routes and policy permission. The hub matters when the design says it should: for example, a hub-and-spoke control policy that forces branch traffic via HUB-01 so that a data-centre firewall inspects it, or because the data centre also needs the VPC. Say which you intend, and check the OMP table for the next hop that is actually in use.

Failure and failover

  • One cloud router fails. The other keeps the routes. Expect a short interruption while tunnels and routes converge.
  • An availability zone fails. Place the two routers in different zones, and check that the Transit Gateway attachments are in both.
  • The Transit Gateway region has a problem. Everything behind it is unreachable. A second region or a second path is a separate design decision with its own cost.
  • Direct Connect fails. Traffic should move to the VPN or internet path. Test it, and check that the return path moves too.

A way to choose, not an answer

A way to choose, not an answer
A way to choose, not an answerIf the situation is; Lean towards. A few VPCs, modest traffic, a small team: Site-to-site VPN, and revisit when it grows Many VPCs and accounts, with SD-WAN features wanted for branch traffic: Transit Gateway with SD-WAN routers attached Fabric features wanted for every cloud flow, few spokes: Transit VPC with cloud routers Steady, high traffic or a no-internet rule: Direct Connect, with a second path An outage is costly: Two independent paths, testedIf the situation isLean towardsA few VPCs, modest traffic,a small teamSite-to-site VPN, and revisit when it growsMany VPCs and accounts,with SD-WAN features wantedfor branch trafficTransit Gateway with SD-WAN routers attachedFabric features wanted forevery cloud flow, fewspokesTransit VPC with cloud routersSteady, high traffic or ano-internet ruleDirect Connect, with a second pathAn outage is costlyTwo independent paths, tested

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Each item on the left, with what it means on the right.

See Cloud OnRamp for IaaS for what the automation builds, and its prerequisites. Check the current Cisco support matrix and AWS documentation before the choice is final.

Test yourself on AWS connection design

Predict · scenario 1

A company has forty VPCs in several AWS accounts and wants branch access to all of them. Which option scales best on the AWS side?

Predict · scenario 2

Branches reach a VPC at 10.100.0.0/16, but VPC hosts cannot answer them. BR-01's OMP table has the VPC route. What is the likely gap?

Predict · scenario 3

A team needs predictable bandwidth and does not want the traffic on the public internet. Which option best matches?

Predict · scenario 4

An engineer writes in the design that the Transit Gateway option supports a stated number of attachments and a stated throughput, quoting an article. What is the better practice?

FAQ

Is there one correct way to connect SD-WAN to AWS?
No. A few VPCs with modest traffic can use a simple VPN. Many VPCs across accounts usually favour a Transit Gateway design. A need for predictable bandwidth points to Direct Connect. The questions in this lesson help to choose, and many networks combine two options.
Do all these options give the same SD-WAN features inside AWS?
No. Where an SD-WAN router sits in the path, the fabric features (application-aware routing, policies, segmentation by VPN) can apply to that traffic. With a plain AWS VPN endpoint the SD-WAN side sees an IPsec tunnel and not much more.
Where do I find limits and prices?
In AWS's current documentation and price list, and in Cisco's support matrix for your release. Service limits, prices and supported attachment types change, so numbers in an article are a starting point at best. This lesson gives none.