A real-life situation
The R2–R3 link in the lab runs through a switch in a shared building. Anyone who plugs a laptop into the right VLAN can send OSPF hellos to R2 on that link, become a neighbour and advertise routes, and nothing in the earlier lessons stops it: the interface can't be passive, because R3 needs to be a neighbour. Authentication solves this. R2 only accepts OSPF packets that prove they know the shared key.
What OSPF authentication is
OSPFv2 offers three authentication types, chosen per interface (or per area):
| Type | What is sent | Key length | Use it? |
|---|---|---|---|
| 0 Null | Nothing | - | Default |
| 1 Plain text | The key itself, in every packet | Up to 8 characters | No: anyone capturing traffic sees it |
| 2 MD5 | Key ID + a hash of packet and key + a sequence number | Up to 16 characters | Common, widely supported |
| 2 HMAC-SHA (key chain) | Key ID + an HMAC-SHA hash + a sequence number | Key-chain key strings | Best, on IOS versions that support it |
The authentication type and key are part of the hello check, like the area and timers. If they don't match, the routers never become neighbours.
- Auth type 2, key ID 1, sequence 1739, hash of (packet + Lab-Key-23).Hello from R2 to R3.Hello
- R3 checksSame key ID 1 → hash the packet with its own key → the result matches → accept. A higher sequence number than last time also rules out replayed packets.
- HelloAuth type 2, key ID 1, sequence 902, hash of (packet + Lab-Key-23).Hello from R3 to R2.
- NeighboursBoth checks pass and the adjacency forms as normal. Every later DBD, LSR, LSU and LSAck carries a hash too.
- HelloFrom a laptop: no authentication, or a guessed key.Hello from R3 to R2.
- DroppedType or hash doesn't match: R2 ignores the packet. The laptop never becomes a neighbour.
Why it works this way
Routing is a trust relationship: a router believes what its neighbours advertise. Without authentication, being on the link is enough to be trusted. Cryptographic authentication ties trust to knowing the key, and because only a hash is sent, capturing traffic doesn't reveal the key. The sequence number stops an attacker from recording a valid packet and playing it back later.
How to configure it on Cisco IOS
MD5 on one interface
interface GigabitEthernet0/1
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 Lab-Key-23On R2 (and the same on R3's Gi0/0). The first line turns MD5 on for this interface; the second sets key ID 1. Both the key ID and the key must match the neighbour's.
MD5 for a whole area
router ospf 1
area 0 authentication message-digest
!
interface GigabitEthernet0/0
ip ospf message-digest-key 1 md5 Lab-Key-12
interface GigabitEthernet0/1
ip ospf message-digest-key 1 md5 Lab-Key-23area 0 authentication turns MD5 on for every interface in area 0. The keys are still set per interface, so each link can have its own. An interface-level ip ospf authentication command overrides the area setting.
HMAC-SHA with a key chain
key chain OSPF-KEYS
key 1
key-string Lab-Key-23-Long-Secret
cryptographic-algorithm hmac-sha-256
!
interface GigabitEthernet0/1
ip ospf authentication key-chain OSPF-KEYSSupported on newer IOS releases. Key chains allow send and accept lifetimes, so a new key can be added everywhere before it starts being used, and the old one removed later.
Plain text (for recognition only)
interface GigabitEthernet0/1
ip ospf authentication
ip ospf authentication-key Secret12Type 1. The key is sent in clear text; avoid it outside labs.
💡 service password-encryption hides the keys in the running configuration (shown as type 7), which stops people reading them over your shoulder. Type 7 is easily reversed, so it is not real protection for the configuration file.
How to verify it
R2#show ip ospf interface GigabitEthernet0/1 GigabitEthernet0/1 is up, line protocol is up Internet Address 10.0.23.1/30, Area 0, Attached via Network Statement Process ID 1, Router ID 2.2.2.2, Network Type POINT_TO_POINT, Cost: 10 ... Neighbor Count is 1, Adjacent neighbor count is 1 Adjacent with neighbor 3.3.3.3 Suppress hello for 0 neighbor(s) Cryptographic authentication enabled Youngest key id is 1
R2#show ip ospf | include Area|authentication Area BACKBONE(0) Area has message digest authentication
show ip ospf.What goes wrong and how to troubleshoot it
R2#debug ip ospf adj OSPF-1 ADJ Gi0/1: Rcv pkt from 10.0.23.2 : Mismatched Authentication Key - Message Digest Key 1
- Type mismatch: authentication on one end only, or MD5 on one end and plain text on the other.
- Key ID mismatch: key 1 on one router, key 2 on the other.
- Key mismatch: a typo, or trailing spaces copied into the key.
- Area setting missed on one router:
area 0 authentication message-digestconfigured on some routers only.
Change keys carefully: adding authentication to one end of a live link drops the adjacency until the other end matches. MD5 lets you configure a second key ID on both routers first, then remove the old one.
Common mistakes
- Setting the key but not turning authentication on (or the reverse).
- Expecting area-level authentication to set the keys too. Keys are always per interface.
- Using plain text outside a lab.
- Thinking authentication encrypts the LSAs.
Key takeaways
- Authentication stops unknown devices becoming neighbours on links where neighbours are expected.
- Types: 0 none, 1 plain text (avoid), 2 cryptographic (MD5 or HMAC-SHA).
- Enable per interface (
ip ospf authentication message-digest) or per area (area 0 authentication message-digest); keys are per interface. - Type, key ID and key must match, or the adjacency never forms.
Check yourself
R2 uses MD5 key 1 on Gi0/1. R3 has no authentication on Gi0/0. What happens?
With MD5 authentication, what crosses the link?
R2 has ip ospf message-digest-key 1 md5 Lab-Key-23; R3 has ip ospf message-digest-key 2 md5 Lab-Key-23. Both have MD5 enabled. Result?