Course contents

Module 6: Troubleshooting and PracticeLesson 6.2 (2 of 6 in this module)39 of 43 in the BGP course

Troubleshooting neighbour adjacency

The neighbour shows Active or Idle and nothing else. BGP session problems fall into a short list of causes, and the state, the logs and two commands usually point at the right one in a minute.

Advanced · 14 min read

You'll learn to map a neighbour state to its likely cause, check TCP 179 reachability and source address, and read a NOTIFICATION error code.

How a session forms, and where it breaks

A BGP session needs two things in order: a working TCP connection to port 179, and then a successful exchange of OPEN messages. Everything that can go wrong falls into one of those two stages, which is the most useful fact in troubleshooting. A router stuck before TCP completes shows Idle or Active. A router that completes TCP but fails the OPEN exchange cycles through OpenSent or OpenConfirm and logs a NOTIFICATION.

Two stages, two kinds of fault
Two stages, two kinds of faultTCP first, then BGP OPEN.IP reachabilityRoute to theneighbour addressTCP 179Connection opensin either directionOPENAS, ID, timers,capabilities agreeEstablishedUPDATEs flow

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Find which stage fails and the list of causes shrinks to half a dozen.

Read the state first

State and the usual cause
State and the usual causeWhat each state says about where to look.IdleNo attempt yet or cannot try:no route, shutdown, waiting to retryActiveTrying TCP, not completing:ACL on 179, wrong address, no return routeOpenSent / OpenConfirmTCP works, OPEN rejected or stuck:AS, router ID, MD5, timersEstablished, 0 prefixesSession fine, routes missing:policy, network statements, address family

On small screens, swipe sideways or focus the diagram and use the arrow keys.

The state names the stage. The next section turns each into commands.
Illustrative Cisco IOS XE output written for this lesson, not a router capture
R1#show ip bgp summary
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.0.2.2       4        64501       0       0        1    0    0 never    Active
10.255.0.2      4        64500       0       0        1    0    0 never    Idle
Both counters are zero and Up/Down says never: neither session has ever come up. The first is trying TCP and failing. The second is not trying, which usually means no route to 10.255.0.2.

Scenario 1: layer 1, 2 and 3 first

If the neighbour is on a directly connected link, test the link before anything BGP. A forgotten shutdown, a wrong mask, or an address typo will show Active and nothing in the BGP configuration is wrong.

ping 192.0.2.2 show ip interface brief show ip bgp neighbors 192.0.2.2 | include BGP state|remote AS|Local host

R1. Ping the neighbour, check both interfaces are up and up, and confirm that the neighbour address and AS number in the configuration are the ones you meant.

Scenario 2: peering with loopbacks

Loopback sessions fail for three separate reasons, and each needs its own fix. The neighbour address needs a route. eBGP packets have a TTL of 1 by default, so a loopback two hops away needs ebgp-multihop (or loopbacks on directly connected neighbours need disable-connected-check). And the source address needs to match, shown below.

Why a loopback session fails: the source address
Why a loopback session fails: the source addressR1 and R2 peer using loopbacks 10.255.0.1 and 10.255.0.2 over a link with addresses 192.0.2.1 and 192.0.2.2. Without update-source, R1 sends its TCP connection from the exit interface address 192.0.2.1. R2 only has a neighbour statement for 10.255.0.1, so it does not recognise the sender and refuses. With update-source Loopback0, R1 sends from 10.255.0.1 and the session forms.R1lo0 10.255.0.1R2lo0 10.255.0.2link 192.0.2.0/30.1.2Without update-sourceR1 sends from 192.0.2.1R2 expects 10.255.0.1: refusedWith update-source Loopback0R1 sends from 10.255.0.1R2 recognises it: session formsR1: neighbor 10.255.0.2 update-source Loopback0R2: neighbor 10.255.0.1 update-source Loopback0Both ends need it, and both loopbacks need a route.

On small screens, swipe sideways or focus the diagram and use the arrow keys.

BGP sends from the address of the interface that leaves towards the neighbour unless you say otherwise. Each side must configure the address the other side will actually use.
router bgp 64500 neighbor 10.255.0.2 remote-as 64500 neighbor 10.255.0.2 update-source Loopback0

R1, iBGP. Sends session traffic from Loopback0. R2 needs the matching statement towards 10.255.0.1. For eBGP over loopbacks also add ebgp-multihop 2.

Illustrative Cisco IOS XE output written for this lesson, not a router capture
R2#debug ip bgp
BGP: 10.255.0.1 open active, local address 192.0.2.2
BGP: 10.255.0.1 open failed: Connection refused by remote host, open active delayed 31810ms (35000ms max, 28% jitter)
The local address is the link interface, not the loopback, so the neighbour refuses. The delay in the message is the retry timer: BGP waits and tries again, with some random jitter, which is why an Active session flips back and forth. Turn the debug off with undebug all as soon as you have the line you need.

Scenario 3: TCP port 179 is blocked

An access list on an interface, or a firewall in the path, may drop TCP 179. The session stays Active, ping works, and the configuration looks right. Check which side opens the connection and whether either direction is allowed.

Both directions must be allowed
Both directions must be allowedEither router can open the connection.R1 opensRandom port toR2 port 179R2 opensRandom port toR1 port 179and

On small screens, swipe sideways or focus the diagram and use the arrow keys.

One router connects to port 179 on the other. Source ports are random, so filter on the destination port in each direction.
show tcp brief | include 179 show access-lists show ip bgp neighbors 192.0.2.2 | include TCP|Connections|Last reset

R1. Lists the BGP TCP connections, any ACLs that may match port 179, and how many connections the neighbour attempted and dropped.

Scenario 4: OPEN is rejected

When TCP works but the session still drops, the router logs a NOTIFICATION with a code and subcode. The code names the stage and the subcode the reason.

Scenario 4: OPEN is rejected
Scenario 4: OPEN is rejectedCode / subcode; Meaning; Usual cause. 2 / 2: Bad peer AS remote-as does not match the AS the neighbour sent in its OPEN 2 / 3: Bad BGP identifier Both routers use the same router ID 2 / 6: Unacceptable hold time Hold time below the minimum the neighbour accepts 4 / 0: Hold timer expired No keepalives arrived in time: congestion or a dead path 6 / 1 or 6 / 2: Cease Maximum prefixes reached, or an administrative shutdown or resetCode / subcodeMeaningUsual cause2 / 2Bad peer ASremote-as does not match the AS theneighbour sent in its OPEN2 / 3Bad BGP identifierBoth routers use the same router ID2 / 6Unacceptable hold timeHold time below the minimum theneighbour accepts4 / 0Hold timer expiredNo keepalives arrived in time:congestion or a dead path6 / 1 or 6 / 2CeaseMaximum prefixes reached, or anadministrative shutdown or reset

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Read each row across: the left box names the item, the others describe it.
Illustrative Cisco IOS XE output written for this lesson, not a router capture
R1#show logging | include BGP
%BGP-3-NOTIFICATION: sent to neighbor 192.0.2.2 2/2 (peer in wrong AS) 2 bytes FDE8
%BGP-5-ADJCHANGE: neighbor 192.0.2.2 Down BGP Notification sent
2/2 is code 2, subcode 2: the OPEN carried a different AS number from the one in R1's neighbor ... remote-as. The trailing bytes are the AS number that was sent, FDE8 in hexadecimal, which is 65000.

MD5 authentication mismatches do not produce a NOTIFICATION: the TCP segments are silently dropped, and the log shows %TCP-6-BADAUTH. That looks like Active, so check authentication on both ends whenever TCP and routing are fine and OPEN never appears.

A diagnosis order that works

Five checks, cheapest first
Five checks, cheapest firstStop at the first one that fails.ReachabilityPing from theright sourceConfigNeighbour IP,remote-asTCP 179ACLs, firewall,TTL, sourceOPENRouter ID, MD5,timersLogsNOTIFICATIONcode

On small screens, swipe sideways or focus the diagram and use the arrow keys.

Each one rules out a layer. Do not skip to the BGP configuration first.
ping 10.255.0.2 source Loopback0 show ip bgp neighbors 10.255.0.2 show ip route 10.255.0.2

R1. A ping sourced from the loopback proves the return path works for the address BGP will use. The neighbor detail shows the state, last reset reason and local and remote addresses.

Technical reference: RFC 4271, section 6 for the error codes, and the BGP sessions lesson for the state machine. Output above is illustrative.

Check your understanding

Predict · scenario 1

A neighbour shows Active with zero messages in both directions and ping works. What is a likely cause?

Predict · scenario 2

The log shows a NOTIFICATION 2/2. What is wrong?

Predict · scenario 3

iBGP between two loopbacks stays Active. Both loopbacks are reachable. What is the first thing to add?

FAQ

Does Active mean the session is working?
No. Active means the router is actively trying to open the TCP connection and has not succeeded. A working session shows a number of prefixes received, or Established in the detailed output.
Why does a session stay Idle?
The router has not tried yet, or cannot. Common causes are no route to the neighbour address, a neighbour that is administratively shut down, or a configuration that has just changed and is waiting to retry.
What does a NOTIFICATION message tell me?
It names the reason the neighbour closed the session, as an error code and subcode. Wrong peer AS, a bad BGP identifier and hold timer expired each have their own value.