Hands-On Lab: Configuring OSPF Authentication

This hands-on lab configures MD5 authentication on an OSPF link between two routers, verifying the neighbor relationship requires a matching key on both sides and observing the resulting failure when the keys are mismatched.

OSPF MD5 AuthenticationNeighbor SecurityAuthentication Key Mismatch

~3 min read · Updated Sep 23, 2026

Lab Objective

Configure MD5 authentication on an OSPF-enabled interface between two routers, verify the neighbor relationship forms successfully when keys match, then introduce a key mismatch and observe the resulting adjacency failure.

Lab Purpose

Without authentication, any device that can send properly formatted OSPF hello packets on a shared segment can potentially form an unauthorized neighbor relationship and inject false routing information. MD5 authentication ensures only routers sharing a common secret key can become neighbors.

Lab Topology

R1 ---- Gi0/0 ------------------ Gi0/0 ---- R2
     192.168.110.1/24         192.168.110.2/24

OSPF process 1, area 0, already forming a
healthy adjacency between R1 and R2

Task 1: Verify the Current Healthy Adjacency

Confirm R1 and R2 show a FULL neighbor relationship before adding authentication.

Task 2: Configure MD5 Authentication on R1

Enable OSPF MD5 authentication on R1's interface with a specific key.

Task 3: Configure Matching MD5 Authentication on R2

Enable the identical key on R2's corresponding interface.

Task 4: Verify the Adjacency Remains FULL

Confirm the neighbor relationship is unaffected, since both keys match.

Task 5: Introduce a Key Mismatch

Change R2's key to a different value and observe the resulting adjacency failure.

Task 6: Correct the Mismatch

Restore matching keys and confirm the adjacency recovers.

Solution and Verification

R1# show ip ospf neighbor

Neighbor ID     Pri   State           Address
2.2.2.2          1     FULL/BDR        192.168.110.2

R1(config)# interface gigabitethernet0/0
R1(config-if)# ip ospf message-digest-key 1 md5 OspfKey2026
R1(config-if)# ip ospf authentication message-digest

R2(config)# interface gigabitethernet0/0
R2(config-if)# ip ospf message-digest-key 1 md5 OspfKey2026
R2(config-if)# ip ospf authentication message-digest

R1# show ip ospf neighbor

Neighbor ID     Pri   State           Address
2.2.2.2          1     FULL/BDR        192.168.110.2
-- Adjacency remains FULL, since both sides
-- now share the identical key

R2(config)# interface gigabitethernet0/0
R2(config-if)# ip ospf message-digest-key 1 md5 DifferentKey999

R1# show ip ospf neighbor

-- (Neighbor entry eventually disappears
--  entirely once the dead timer expires)

R1# debug ip ospf adj

*OSPF: Rcv pkt from 192.168.110.2, Mismatch
Authentication Key - Message Digest Key ID 1

R2(config)# interface gigabitethernet0/0
R2(config-if)# ip ospf message-digest-key 1 md5 OspfKey2026

R1# show ip ospf neighbor

Neighbor ID     Pri   State           Address
2.2.2.2          1     FULL/BDR        192.168.110.2
-- Adjacency recovers once the keys match again

Key Takeaway

An MD5 key mismatch produces a distinct debug message explicitly naming the authentication failure, unlike the generic "up, line protocol down" symptom seen with clocking or encapsulation mismatches discussed earlier in this series — this specific error message immediately points toward checking authentication configuration on both sides rather than the broader troubleshooting checklist a vaguer symptom would require.

Written & researched by Dr. Shahin Siami

Related Articles

Hands-On Lab: Configuring UDLD

This hands-on lab configures UDLD in aggressive mode on a fiber link between two switches, simulating a unidirectional fiber failure and verifying UDLD detects the mismatch and shuts down the affected port before a Layer 2 loop can form.

Continue

Hands-On Lab: Configuring Loop Guard

This hands-on lab configures Loop Guard on a switch's non-designated ports to prevent a unidirectional link failure from causing a Layer 2 loop, simulating a one-way BPDU loss and verifying the affected port enters a loop-inconsistent blocking state rather than incorrectly transitioning to forwarding.

Continue

Hands-On Lab: Configuring BPDU Guard and BPDU Filter

This hands-on lab configures BPDU Guard globally for PortFast-enabled ports and demonstrates the distinct, riskier behavior of BPDU Filter, comparing how each responds when a switch is connected to an access port that should only ever see end-user devices.

Continue

Hands-On Lab: Configuring Root Guard

This hands-on lab configures Root Guard on switch ports facing access-layer switches to prevent an unauthorized or misconfigured switch from becoming the Spanning Tree root, verifying the port enters a root-inconsistent state when a superior BPDU is received.

Continue

Hands-On Lab: Comprehensive Troubleshooting Challenge

This hands-on lab presents a multi-layer connectivity failure across VLANs, trunking, routing, and NAT simultaneously, requiring systematic bottom-up troubleshooting to identify and correct three independent faults before full connectivity is restored.

Continue

Hands-On Lab: Configuring Enhanced Object Tracking with HSRP

This hands-on lab configures IP SLA-based object tracking on an HSRP-enabled router, automatically lowering its priority when its WAN uplink fails, forcing failover to the standby router without requiring a full interface shutdown.

Continue