"How to check udp port is open using IOS CLI only" or "Do you really know how traceroute works"?

Just want to share with you one interesting but very simple trick with udp traceroute which can be used to check if udp port is open or filtered. In practical case it can be very handy to use it if you think somebody in the middle filtering your ISAKM udp/500 or NAT-T udp/4500 ports for example and you can't use any third-party tools to check it (CCIE lab environment?)
Only few key things you had to remember: Cisco IOS using UDP traceroute and you can define initial destination port, moreover this port number is incremented with each subsequent packet, even if to the same hop with same TTL.
The technique is simple as said above: you need to send traceroute in such way so that intermediate hop router will receive packet with verifiable destination port number. If intermediate hop router is doing filtering and configured to allow to send ip unreachables ICMP messages, you will receive "!A" symbol in your traceroute output, meaning "Administratively filtered". This is the simplest case, but observed very rare since most of the routers configured with "no ip unreachables" to lower the load on CPU resources. Later I will also show how to check a port if "no ip unreachables" is configured on a verifiable router.

Anyway here is the example step by step, ip addresses is self-explained:

R1#traceroute 155.1.45.5 numeric probe 1 port 500
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 64 msec
  2 155.1.23.3 112 msec
  3 155.1.34.4 136 msec
  4 155.1.45.5 136 msec

All OK. At this step we know full path to the final router R5, and second router in the line (R2) was not filtering UDP/500. Lets decrement port number to 499, so next router R3 will receive UDP/500 packet:

R1#traceroute 155.1.45.5 numeric probe 1 port 499
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 60 msec
  2 155.1.23.3 124 msec
  3 155.1.34.4 108 msec
  4 155.1.45.5 148 msec

Once again all fine, R3 is not filtering,  we are decrementing destination port number again and testing R4:

R1#traceroute 155.1.45.5 numeric probe 1 port 498
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 60 msec
  2 155.1.23.3 88 msec
  3 155.1.34.4 !A

Gotcha! Next to the last router (R4) is filtering our ISAKMP packets! Here is how it looks in Wireshark:

Now let's try to do the same testing, but on R4 with "no ip unreachables" configured:

R1#traceroute 155.1.45.5 numeric probe 1 port 500
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 68 msec
  2 155.1.23.3 108 msec
  3 155.1.34.4 128 msec
  4 155.1.45.5 132 msec
R1#traceroute 155.1.45.5 numeric probe 1 port 499
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 52 msec
  2 155.1.23.3 100 msec
  3 155.1.34.4 104 msec
  4 155.1.45.5 140 msec
R1#traceroute 155.1.45.5 numeric probe 1 port 498
Type escape sequence to abort.Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 92 msec
  2 155.1.23.3 84 msec
  3  *
  4 155.1.45.5 136 msec
R1#traceroute 155.1.45.5 numeric probe 1 port 497
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 60 msec
  2 155.1.23.3 100 msec
  3 155.1.34.4 108 msec
  4  *
  5 155.1.45.5 128 msec
R1#traceroute 155.1.45.5 numeric probe 1 port 496
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 92 msec
  2 155.1.23.3 100 msec
  3 155.1.34.4 100 msec
  4 155.1.45.5 168 msec

As you can see, on the third step (bold text), exactly when R4 receives UDP/500 we can see symbol "*" which meaning timeout. On the fourth step here is also timeout, it is again because of R4 is filtering and UDP/500 packet can't reach R5. Here is how 3rd iteration looks like:

Please note that in our sandbox case filtering was made with simple extended access-list which was applied to inbound (ingress) interface to the "in" direction:

ip access-list extended DENY_IKE
 deny   udp any any eq isakmp
 permit ip any any
!
interface FastEthernet0/0
 ip address 155.1.34.4 255.255.255.0
 ip access-group DENY_IKE in

In case of this access-list applied to the outbound (egress) interface FastEthernet0/1 to the "out" direction, traceroute behavior will be different, but it still can bring us results and required information to detect intermediate hop router which performing filtering, behavior is just like in the fourth step/iteration in the previous example. 

R1#traceroute 155.1.45.5 numeric probe 1 port 496
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 104 msec
  2 155.1.23.3 100 msec
  3 155.1.34.4 140 msec
  4 155.1.45.5 128 msec
R1#traceroute 155.1.45.5 numeric probe 1 port 497
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 56 msec
  2 155.1.23.3 116 msec
  3 155.1.34.4 120 msec
  4  *
  5 155.1.45.5 156 msec
R1#traceroute 155.1.45.5 numeric probe 1 port 498
Type escape sequence to abort.
Tracing the route to 155.1.45.5
VRF info: (vrf in name/id, vrf out name/id)
  1 155.1.12.2 88 msec
  2 155.1.23.3 104 msec
  3 155.1.34.4 140 msec
  4 155.1.45.5 128 msec

Instead of conclusion I recommend for best results to make the same testing in both directions, obvious since some kind of filtering can be applied in outbound direction of ingress interface and you will not notice it until testing done from another side. Thank you for reading.