It was the IOS!

It was the IOS!!!!!
I had been using c3640-jk9o3s-mz.124-16.bin. Changed it to c3640-i-mz.122-46.bin
and that was it, return icmp echo-reply packets got natted back
hmmm, i thought they use the 12.4 version IOS in the exam lab - yikes!

*Mar 1 00:33:08.795: IP: tableid=0, s=10.1.1.3 (Ethernet0/0), d=201.114.37.5 (Serial1/0), routed via RIB
*Mar 1 00:33:08.795: NAT: i: icmp (10.1.1.3, 2) -> (201.114.37.5, 2) [2]
*Mar 1 00:33:08.799: NAT: s=10.1.1.3->204.15.86.3, d=201.114.37.5 [2]
*Mar 1 00:33:08.803: IP: s=204.15.86.3 (Ethernet0/0), d=201.114.37.5 (Serial1/0), g=199.100.35.253, len 100, forward
*Mar 1 00:33:08.803: ICMP type=8, code=0
*Mar 1 00:33:08.959: NAT: o: icmp (201.114.37.5, 2) -> (204.15.86.3, 2) [2]
*Mar 1 00:33:08.959: NAT: s=201.114.37.5->10.1.3.1, d=204.15.86.3 [2]
*Mar 1 00:33:08.963: NAT: s=10.1.3.1, d=204.15.86.3->10.1.1.3 [2]
*Mar 1 00:33:08.967: IP: tableid=0, s=10.1.3.1 (Serial1/0), d=10.1.1.3 (Ethernet0/0), routed via RIB
*Mar 1 00:33:08.967: IP: s=10.1.3.1 (Serial1/0), d=10.1.1.3 (Ethernet0/0), g=10.1.1.3, len 100, forward
*Mar 1 00:33:08.971: ICMP type=0, code=0

-a.c

Lost in translation - Help!!!

Matzalan#sh ip nat translations
1 Pro Inside global Inside local Outside local Outside global
2 --- --- --- 10.1.3.1 201.114.37.5
3 --- --- --- 10.1.3.2 201.114.37.1
4 icmp 204.15.87.1:39 10.1.1.3:39 201.114.37.5:39 201.114.37.5:39
5 --- 204.15.87.1 10.1.1.3 --- ---
6 icmp 204.15.87.2:9 10.1.2.2:9 201.114.37.5:9 201.114.37.5:9
7 --- 204.15.87.2 10.1.2.2 --- ---

Line 2 implies, 201.114.37.5 should get translated internally as 10.1.3.1.But if you look at line 4, you see this is not true. When the inside local IP initiates the icmp request, the response packet arrives as the OG and reaches the host with the source as 201.114.37.5
According to the book, the ICMP response (echo-reply) is supposed to have been translated. But this is not the case.

A#ping 201.114.37.5
*Mar 1 01:23:50.423: ICMP: echo reply rcvd, src 201.114.37.5, dst 10.1.1.3
*Mar 1 01:23:50.543: ICMP: echo reply rcvd, src 201.114.37.5, dst 10.1.1.3

Matzalan#
*Mar 1 18:01:00.127: IP: tableid=0, s=10.1.1.3 (Ethernet0/0), d=201.114.37.5 (Serial1/0), routed via RIB
*Mar 1 18:01:00.131: NAT: i: icmp (10.1.1.3, 56) -> (201.114.37.5, 56) [232]
*Mar 1 18:01:00.135: NAT: s=10.1.1.3->204.15.87.1, d=201.114.37.5 [232]
*Mar 1 18:01:00.139: IP: s=204.15.87.1 (Ethernet0/0), d=201.114.37.5 (Serial1/0), g=199.100.35.253, len 100, forward
*Mar 1 18:01:00.143: ICMP type=8, code=0
*Mar 1 18:01:00.203: NAT*: o: icmp (201.114.37.5, 56) -> (204.15.87.1, 56) [232]
*Mar 1 18:01:00.207: NAT*: s=201.114.37.5, d=204.15.87.1->10.1.1.3 [232]
*Mar 1 18:01:00.215: IP: tableid=0, s=201.114.37.5 (Serial1/0), d=10.1.1.3 (Ethernet0/0), routed via RIB
*Mar 1 18:01:00.219: IP: s=201.114.37.5 (Serial1/0), d=10.1.1.3 (Ethernet0/0), g=10.1.1.3, len 100, forward
*Mar 1 18:01:00.223: ICMP type=0, code=0

A#ping 10.1.3.1
*Mar 1 01:24:00.803: ICMP: echo reply rcvd, src 10.1.3.1, dst 10.1.1.3
*Mar 1 01:24:00.903: ICMP: echo reply rcvd, src 10.1.3.1, dst 10.1.1.3
*Mar 1 01:24:01.023: ICMP: echo reply rcvd, src 10.1.3.1, dst 10.1.1.3
*Mar 1 01:24:01.143: ICMP: echo reply rcvd, src 10.1.3.1, dst 10.1.1.3

Matzalan#
*Mar 1 18:07:07.083: IP: tableid=0, s=10.1.1.3 (Ethernet0/0), d=10.1.3.1 (Serial1/0), routed via RIB
*Mar 1 18:07:07.087: NAT: i: icmp (10.1.1.3, 61) -> (10.1.3.1, 61) [237]
*Mar 1 18:07:07.091: NAT: s=10.1.1.3->204.15.87.1, d=10.1.3.1 [237]
*Mar 1 18:07:07.091: NAT: s=204.15.87.1, d=10.1.3.1->201.114.37.5 [237]
*Mar 1 18:07:07.095: IP: s=204.15.87.1 (Ethernet0/0), d=201.114.37.5 (Serial1/0), g=199.100.35.253, len 100, forward
*Mar 1 18:07:07.099: ICMP type=8, code=0
*Mar 1 18:07:07.163: NAT*: o: icmp (201.114.37.5, 61) -> (204.15.87.1, 61) [237]
*Mar 1 18:07:07.163: NAT*: s=201.114.37.5->10.1.3.1, d=204.15.87.1 [237]
*Mar 1 18:07:07.167: NAT*: s=10.1.3.1, d=204.15.87.1->10.1.1.3 [237]
*Mar 1 18:07:07.175: IP: tableid=0, s=10.1.3.1 (Serial1/0), d=10.1.1.3 (Ethernet0/0), routed via RIB
*Mar 1 18:07:07.179: IP: s=10.1.3.1 (Serial1/0), d=10.1.1.3 (Ethernet0/0), g=10.1.1.3, len 100, forward
*Mar 1 18:07:07.179: ICMP type=0, code=0

Configs:
interface Serial1/0
ip address 199.100.35.254 255.255.255.252
ip nat outside
ip virtual-reassembly
no ip route-cache
serial restart-delay 0
interface Ethernet0/0
ip address 10.1.1.1 255.255.255.0
ip nat inside
ip virtual-reassembly
no ip route-cache
full-duplex
ip nat inside source static 10.1.1.3 204.15.87.1
ip nat inside source static 10.1.2.2 204.15.87.2
ip nat outside source static 201.114.37.5 10.1.3.1

ip route 0.0.0.0 0.0.0.0 Serial1/0
ip route 10.1.2.0 255.255.255.0 10.1.1.2

More observations:
Disabling inside NAT and defining a route to 10.1.1.0/24 (at the remote router) makes the icmp-echo reply packet to behave correctly.
A#ping 201.114.37.5 re 1
A#
*Mar 1 05:06:42.866: ICMP: echo reply rcvd, src 10.1.3.1, dst 10.1.1.3

Matzalan#
*Mar 1 03:59:20.607: IP: tableid=0, s=10.1.1.3 (Ethernet0/0), d=201.114.37.5 (Serial1/0), routed via RIB
*Mar 1 03:59:20.611: IP: s=10.1.1.3 (Ethernet0/0), d=201.114.37.5 (Serial1/0), g=201.114.37.5, len 100, forward
*Mar 1 03:59:20.615: ICMP type=8, code=0
*Mar 1 03:59:20.719: NAT: Processing out-2-in packet in after_routing2
*Mar 1 03:59:20.723: NAT: s=201.114.37.5->10.1.3.1, d=10.1.1.3 [36]

Interestingly, in this scenario, we can ping 201.114.37.5 but not telnet - this is because icmp does not check for the src/dest IP addressm but just a ICMP identifier which should match.

A#telnet 201.114.37.5
Trying 201.114.37.5 ...

A#telnet 10.1.3.1
Trying 10.1.3.1 ... Open

From all the above tests, I find that I can never get NAT to translate both ways...
IG IL OL OG
Either : 10.1.1.3:54164 10.1.1.3:54164 10.1.3.1:23 201.114.37.5:23
or : 204.15.87.1:21 10.1.1.3:21 201.114.37.5:21 201.114.37.5:21

How do i achive:
IG IL OL OG
204.15.87.1 10.1.1.3 10.1.3.1 201.114.37.5

The Translator

Progress. I started off with NAT from Jeff Doyle's Vol-II. I like long distance running and it's been my experience that the first mile is always the hardest. In that sense, I'm still within the first 100 meters of the race, but I'm just happy to see my feet move. I started with NAT mainly because the technology is relatively independent. The plan is to master each technology as I go and try and know every aspect.
Over the last 2 days, I got to reading up the introductory parts of NAT. Some things I thought that were worth recording on the blog were:
The NAT address types

1. Inside Local (IL): Internal private addresses, as they appears LOCALLY within the Enterprise
2. Inside Global (IG): Public IP addresses of translated Internal addresses, that appear GLOBALLY
3. Outside Local (OL): Public IP addresses - translated inside; as it appear LOCALLY within the Enterprise
4. Outside Global (OG): Public internet IP addresses as they appear GLOBALLY.

Commands:
sh ip nat translations
They show the dynamic and static translation table. Dynamic translations are caused by the many-to-1 and 1-to-many NAT translations. They typically timeout in 24 hours. This time can be specified by:
ip nat translation timeout <>

It's interesting that Doyle points out two CIDR issues alleviated by NAT.
1. ISP dependence - Kinda obvious
2. Multihomed ISP, address advertisement issues.
This is interesting because if you were an Enterprise and were multihomed to two ISPs and were given a more specific subnet from the ISP's IP address space, how would you possibly advertise it through the other ISP?
Even if you got ISP2 advertise ISP1's (more specific)subnet, ISP1 can no longer advertise the less specific subnet (because ISP2 is also broadcasting the Enterprise's subnet). This can cause the route to be lost, as some national ISPs filter out subnets that are higher than /19. Doyle suggests having the enterprise use RFC1918 addresses and NAT then to both ISPs at the edge.

The other interesting uses of NAT were:
Port address translation. Overloading a single IP to represent more than 1 address by translating the source port and not just the IP
TCP load balancing: NAT based server load balancing by translating one IG address to a farm of IL addresses (NAT supports only Round-Robin and has no way to determine server health)
Service based Translation: NAT can route traffic into the Enterprise based on the destination port. Therefore the same IG address can be mapped to different IL addresses based on the destination port.

As awesome as NAT is, it is not without limits. Some more interesting facts about how NAT works with certain types of traffic:

Checksum: When the IP is translated, the IP header get messed up. When the port is translated, the TCP header gets messed up. Cisco's implementation of NAT recalculates the headers and all is well

Fragmentation: For the instance of using virtual servers that are NATted to different IL's based on the destination port, what happens if say, a packet destined to the SMTP (port 25) arrives fragmented? The first fragment is the only one that has the details of the port. Thankfully, Cisco's implementation of NAT maintains state. But what if - given the nature of IP- the fragments arrive out of sequence? Well, then, there is no other option but to buffer the fragments until the first one arrives.

Encryption: NAT should always be done before encryption

Some protocols carry IP addresses in the data field. Cisco's implementation of NAT works as follows for :

(i) ICMP: Some ICMP messages include the IP Header of the packet that caused the message to be generated (3,4,5,11 & 12).
(ii) DNS: Rule of thumb is that the Master and slave must reside on the same side of the NAT (Zone transfer across a NAT is not supported). However, an IP address in a DNS query or response is NATted.
(iii) FTP: Everyone's favourite! In short, Cisco's NAT translates the IP address within the PORT or PASV command. (PORT is the command the host sends the server informing the server of the port to which it should connect for Active FTP and PASV is the command the host sends the server requesting it to open up a passive Data port to connect to). The most interesting fact for me was that during FTP, the IP could be transmitted as ASCII data rather than the binary. This implies, an IP address NAT within the data could result in a size change ( if 10.1.1.1 gets translated to 235.123.178.222!!). Cisco's NAT handles it pretty artfully. If the translated IP address is smaller than the original, NAT pads 0's. But if the size is higher, it gets a little more complex. This is because, the SEQ & ACK numbers are directly related to size of the data field. Cisco's NAT handles it by recalculatin these numbers.
(iv) SMTP: No problems here. Cisco's NAT makes the appropriate translations. Worth noting here is that IMAP and POP are strictly client-server protocols (unlike SMTP) and hence always use a hostname (never IP addresses).
(v) SNMP: There can be many IPs in the data fields for the different MIBS in different formats. NAT does not translate these. (this does not imply that an SNMP request cannot traverse a NAT, only the IPs within the SNMP data field are not translated)
(vi) Routing Protocols: No routing protocol packets should traverse a NAT boundary in which the advertised IPs change. Routing protocols are rich and complex just like SNMP fields.
(vii) Traceroute: Uses ICMP type 11 (Time Exceeded) which NAT handles as seen in the ICMP section or UDP, which also NAT handles.

I'll leave you with this:
Many critics, no defenders,
translators have but two regrets:
when we hit, no one remembers,
when we miss, no one forgets.
-Anonymous

- A.C

Do not go gentle into that good night

I am going to get my CCIE number. This blog is part of my study-tool. I passed the written late August 2008. Since then, I haven't done ANY studying. This first post is the first step. Here is my initial study plan for the lab. Broadly:

• Jeff Doyle - Volume II, Until end of February
• Purchase Narbik Kocharian's Advanced Tech Workbook - End of Feb
• Work through it using Dynamips
• Start IE's self-paced course sometime in May/June

I plan to devote around 20 minutes a day to this blog - during lunch, at work to record the previous day's progress/lack-off. I want to be realistic about my goals and will post a study routine, once I get into a groove. For now, I'm bracing myself and plunging in.
............. Welcome to my journey.
-Aspiring Candidate