Starting a new Network Engineer Job – Part 1

I searched around the internet for a few minutes to see what/how people do when they first start a new job as a network engineer. Now, I am not talking about, someone starting their first ever network engineer job but rather a person who has been a network engineer and is now transitioning into a another job at a different company or network.

I am currently experiencing this and wanted to see what other engineers have done to adapt to a new network to learn the new network infrastructure they are now faced with supporting. My current position has such a large network that it has become quite overwhelming but I, in my sick ways, am excited about such a great opportunity.

So, what do you do when you start a new job or have to learn a new network infrastructure?

Well, I have started off asking questions about the network to other engineers that work in my team and in other teams. However, I don’t want to ask too many questions as I face a personal dilema with the fact that I don’t want these co-workers to think anything negative of me.

As I ask a couple of questions here and there I then begin the search for documentation and in fact my first questions from other co-workers is, “where should I go to look at some network diagrams?”. Unfortunately, for me, the network infrastructure is so huge that there  are so many diagrams and so many versions scattered across the network. I think this situation is probably all too common as the demand and volume of an engineers work is so high that the documentation doesn’t get made. After all, if I know the network, why would I document it, right?

The next step is to just manually map it out. Get on router 1 then 2 then 3 then 100 and read through config after config after config… You then look at ip addresses and how particular devices obtain these static IP addresses. You then start to look at the routing table, doing traceroute’s and so on and so forth until you figure it out.

There is, unfortunately, no easy way to learn a new network except to hope that there is good documentation of processes and procedures and of the entire network.

So what I do is I get on a router. I then do a sh run and see what’s there… I then look at the routing table, I look at what routing protocols are being used. From my local workstation I will ping yahoo.com and then get on the router and do a traceroute out to the yahoo.com ip address (you can do google.com, cnn.com, etc…). The place on the internet you ping doesn’t matter because all we need/want is a public ip address.

So after I do the ping and it works to yahoo.com from a router I will then do a traceroute to the yahoo.com ip address. Unfortunately a lot of companies are very tight with ICMP (ping) and/or tracerotue. Meaning that the company’s infrastructure firewalls will block ICMP or traceroute traffic.

This is okay as we’ll just jump on each of the hops by attempting to first ssh and then telnet into those devices. As you visit each hop out to the internet you first do a sh ver to see what model it is. Then you do a show run and then put this in a visio diagram showing the model and the interfaces on the router/switch/etc…

I would not worry about routing protocols at the beginning as I would first start to see who is the routers neighbor. show cdp neighbor command works great for this… If CDP is enabled as it should be in a large environment on the core and distribution layer at least you can then start to diagram what is connected to what.

The next thing to do, especially if the configs seem old like, the router is not using an extended ACL is to clear ip access-list counter <access-list-name or number> and then do a sh ip access-list <name or number of list> to then see the hits. So the reason I do this is to see what ACL’s are being used and which one’s are not. I have found it, recently, all to common to see what seems like pages of ACL statements and after clearing the counters (AKA the hits) on the ACL I then notice that maybe 10 out of 40 are actually getting hits. I’m not saying that the one’s not getting hits are not valid or needed but rather you can see what’s going on.

Another option is to do a debug ip packet or some debug ip <something> to just see what’s going on. However, I’d be very very very careful with this as you can max out a CPU reallly fast with this. I’d have the un all ready to be pasted into the CLI window before you turn on debugging… I’d also do the debugging during lunch or some other non-peak time. I would also ask if this is okay, just to make sure, because if you bring a device down you may not have to learn the network any longer as you will now be unemployed.

Sh ip nat translations is good to do as well to see what is getting nat-d and where.

sh ip int brief is of course the first thing you should do every time you get on a router

Once you have this all put together in a diagram and you feel like you have touched more devices than your 11×17 visio diagram can hold I would then start to look at the routing protocols used and note them on the diagram.

Now unforutnately, if you work for a service provider of some sort then you could possibly be looking at 1000′s of tunnel interfaces, sub interfaces, ACL statements, etc… So just remember at first don’t get lost in the details as the goal is to just map out the companies network that you are responsible for or will be supporting.

As you go through the learning process, write down your questions about something immediately so that you can go ask the co-worker who knows the network a very good question. The answer you get will most likely stick because you had to do a lot of digging to come up with that question. Always always do as much research as possible before you ask anybody for help with something UNLESS it is an emergency like something is down or not working. Even then, give it a good go to at least gather the details so that when you ask a more experienced or knowledgeable co-worker he/she will be able to resolve the issue faster.

Afterwards  I would then start to see what are multiple routes, what are the gateway’s of last resort, what protocols are being used on which routers and go from there.

This method I’ve described is not fool proof but matter an idea and an action that I have performed myself.

Learning a new network, whether huge or small can be a daunting task without the support and education from co-workers but in time and with some patience and lots of caffeine you can do it.

Never get discouraged and enjoy the new network and the challenges it brings. As you go through this you will always learn something. I also google commands I see in the running-config to see what these, to me, odd commands are and then start to forumlate why they are there (the goal).

I know that this post wasn’t all that orderly. However, I just wanted to share my perspective on how I’m going about learning a new network infrastructure.

I hope this has been informative for you and I would like to thank you for readying.

GRE IPSEC Tunnel Setup

I am not going to go into the reasons why you would use this type of a connection as that is not my goal. My goal is to just show you how to set one of these up. Below is a screen shot from my GNS3 application that I’ve setup as an example here.

R2 is playing as the internet.

 So on both routers you will bash the following in

NOTE: so on R1 you’ll put in R2′s f0/1 ip address and vice versa when configuring R3 by replace this text with that ip address:  <other routers physical interface ip address>

!
crypto isakmp policy 1
 encryption 3des
 authentication pre-share
 group 2
!
crypto isakmp key cisco address <other routers physical interface ip address>
!
crypto ipsec transform-set MYSET esp-3des esp-sha-hmac
!
crypto ipsec profile MYPROFILE
 set transform-set MYSET

Then configure the physical interfaces on all routers and static routes on R1 and R2:

R1

interface FastEthernet0/0
 ip address 12.13.14.2 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 FastEthernet0/0

R2

interface FastEthernet0/0
 ip address 12.13.14.1 255.255.255.0
!
interface FastEthernet0/1
 ip address 15.16.17.1 255.255.255.0

R3

interface FastEthernet0/1
 ip address 15.16.17.2 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 FastEthernet0/1

 

Then configure the Tunnel interfaces on each router (pay attention to what IP addresses are being used here AND what is being referenced in the crypto change-o stuff above):

R1

interface Tunnel0
 ip address 10.1.1.1 255.255.255.252
 tunnel source FastEthernet0/0
 tunnel destination 15.16.17.2
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile MYPROFILE

R3

interface Tunnel0
 ip address 10.1.1.1 255.255.255.252
 tunnel source FastEthernet0/0
 tunnel destination 15.16.17.2
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile MYPROFILE

Then from R1 try to ping R3′s tunnel0 interface. Okay great. Then try traceroute to it. Then  traceroute to R3′s f0/1 interface ip address and not the tunnel address. Notice only 1 hop for the Tunnel interface and notice two hops for the f0/1 interface ip on R3. So great, that means you did everything right. Congrats!

 

OSPF :: Non-Broadcast Multi-Access

GNS3 ROUTE, #12 – OSPF Frame Relay

R1

Sh ip int brief
!
Router ospf 1
 Network 10.12.3.0 0.0.0.255 area 0
 Network 1.1.1.1 0.0.0.0 area 0
 Router-id 1.1.1.1
!
Sh ip ospf interface
Sh run int lo 0

 

R2

sh ip int brief
!
router ospf 1
 router-id 2.2.2.2
 network 10.12.3.0 0.0.0.255 area 0
 network 10.24.0.2 0.0.0.0 area 24
 network 2.2.2.2 0.0.0.0 area 0
!
sh ip ospf int
sh run int s0/0.1
sh run int s0/1.1

 

*Note* = Area 0 is Non-Broadcast. R1 is the hub router.

 

R2

int s0/0.1
 ip ospf priority 0

 

R3

sh ip int brief
!
int s0/0.1
 ip ospf priority 0
!
router ospf 1
 router-id 3.3.3.3
 network 10.12.3.0 0.0.0.255 area 0
 network 10.35.6.3 0.0.0.0 area 356
 network 3.3.3.3 0.0.0.0 area 0

 

R1

sh ip ospf neigh

*Note* = there are no neighbor forming

 

R1

neighbor 10.12.3.2 ?
!
neighbor 10.12.3.2 priority 0

*Note* = Priority of zero is the default

 

R1

neighbor 10.12.3.2

*Note* = good time to start a wirehsark capture between R1 and R2

 

R1

sh ip ospf neigh

*Note* = Neighbor is set as DRother and there is no BDR as expected.

 

R1

sh ip ospf int

R4

sh ip int br
!
router ospf 1
 router-id 4.4.4.4
 network 10.24.0.0 0.0.0.255 area 24
 network 4.4.4.4 0.0.0.0 area 24

*Note* = neighbor relationship just came up because this is a point to point connection

 

R4

sh ip ospf int
sh ip route

 

R3

sh run | s ospf
sh run int s0/1.1
!
int s0/1.1
 ip ospf network point-to-multipoint
!
sh ip ospf int

 

R5

sh ip int br
!
router ospf 1
 router-id 5.5.5.5
 network 5.5.5.5 0.0.0.0 area 356
 network 10.35.0.0 0.0.0.255 area 356

*Note* = Point to Point neighbors come up right away but no neighbor relationships come up.

 

R5

sh ip ospf neigh
sh ip ospf int
sh run int s0/0.1

*Note* = Router 3 is the hub of our network. Point to point is the default network type.

 

Why are neighbor relationships not forming?

 

Look at Hello Timers on R3 and R5. We have to adjust the hello timers.

 

R3

int s0/1.1
 ip ospf hello-interval 10

*Note* = The dead timer has automatically adjusted to 4x’s the hello timer.

 

R3

sh ip ospf neigh

 

R6

sh ip int br
!
router ospf 1
 router-id 6.6.6.6
 network 10.35..6 0.0.0.0 area 356
 network 6.6.6.6 0.0.0.0 area 356
!
sh ip ospf neigh
sh ip route

*Note* = at this point core OSPF is configured

 

Now ping the loop backs to confirm the connectivity.

 

R4

ping 5.5.5.5

*Note* = Ping fails, lets see why

no ip domain lookup
 exit
!
traceroute 5.5.5.5

*Note* = it dies at Router 2

 

R2

no ip domain lookup
 exit
!
traceroute 5.5.5.5

*Note* = Router 2 has no idea that it can connect directly to Router 3 but Router 3 cannot get to area 24 to area 356. So the problem is that router 2 needs to know how to get to area 356. There is no frame-relay map on router 3 to reach router 2

 

R2

int s0/0.1
 frame-relay map 10.12.3.3 201 broadcast

*Note* = now we still can’t ping it because router 3 needs a dlci map back to router 2.

 

R3

int s0/0.1
 frame-relay map 10.12.3.2 301 broadcast

 

R4

traceroute 5.5.5.5

*Note* = Cisco always drops the second ping of the final hop of the traceroute

 

R4

ping 5.5.5.5
!
ping 6.6.6.6

 

 

OSPF Network Types

  • Broadcast, Multi-Access Networks
    • Example: Ethernet
    • Single operation mode
    • DR/BDR election, 10s Hello’s, Dual Multicast Add
    • 1 mode of operation
    • Point-To-Point
      • Example: T1 CAS, ISDN BRI/PRI
      • Single Operation Mode
      • No DR/BDR, 10s Hello’s, Single multicast add
      • 1 mode of operaton
      • Non-Broadcast Multi-Access (NBMA) Networks
        • Example: Frame Relay, ATM, MPLS
        • Five Modes of Operation
          • Non-Broadcast (NBMA)
          • Point-To­-Multipoint
          • Point-To-Point
          • Broadcast (Cisco proprietary)
          • Point-to-Multipoint, Non-Broadcast (Cisco proprietary)

3 different styles of WAN networks?

1.)    Hub and Spoke

2.)    Partial Mesh

3.)    Full Mesh

 

Non-Broadcast Mode

  • Default mode for x.25, Frame-Relay, ATM
  • Acts like a LAN Environment
  • Neighbors statically configured
  • DR/BDR elected (must have full connectivity)
  • All routers will be on 1 subnet

Note: Sending a message to a specific host/router we use “sudo broadcasts”. Here is an example configuration statement that allows this to happen after statically configuring your neighbors: frame map ip x.x.x.x 402 broadcast ]

Note: The only router that can be the DR/BDR is the Hub router because the spokes will not be able to talk to ALL routers on the network like the hub.

 

Point-To-Multipoint Mode

  • Broadcast messages are sent out on 224.0.0.5
  • Fixes issues with NBMA networks
  • No DR/BDR elected
  • Requires Single subnet
  • Neighbors automatically form (Point-To-Multipoint Non-Broadcast Mode neighbors do not automatically form).

 

Point-To-Point Mode

  • Point-To-Point is a simple setup
  • No DR/BDR elections