Moving On…

Today is my penultimate day with my current company; I am moving on to bigger and (hopefully!) better things.

It’s a weird feeling leaving a job, especially one where you have made so many changes. There is only a small network here (~700 users, ~200 servers) but when I started it had been badly neglected for several years – There wasn’t really anyone with sufficient experience to manage a network, so things had just been left and fingers crossed that nothing would go wrong…

During my time here I have implemented many things that I hope will make the network faster, more stable and more resilient, as well as some procedural changes to help keep things going smoothly when I’m gone. Here is a snippet of some of the things I’ve done during my tenure here;

• Documentation/Diagrams – There were virtually no complete diagram of the network, or accompanying documentation, for maybe the first month of my time here, I was on my hands and knees in computer rooms chasing cables, or sat with a scrap of paper trying to map things out using CDP. Once I had a full diagram of the network, I realized just how bad things were.
• Redundant Links – were non-existence, the whole network was a mess of SPOF’s (Single Point of Failure)   To be fair to the original designers, you could see what they had planned, and in many cases implemented, but due to lack of accurate monitoring, they had one by one failed (GBIC’s/SFP failure, fibres that had been crushed etc…) This leads to….
• Monitoring – there was an installation of Solarwinds NPM, but it was not being fully utilized, only nodes were monitored, and then only by ICMP – no interface status, no SNMP, no traffic graphs etc… I pretty much started from scratch with monitoring. I installed NCM alongside NPM so that there was actually some configuration management as well as monitoring. All the major interfaces were monitored, alerts configured, configurations backed up, even syslog and SNMP traps setup and alerts generated.
• VLAN’s – No one seemed to know how to change the VLAN on a switchport, instead the desktop support guys had a diagram of which switchport’s were configured to which VLAN’s for each switch and just ran a cable, in some instances, this let to cables being stretched across multiple cabs (see my previous posts about recabling the Patching Room) I’ve now gone as far as giving the desktop guys privileges on the access switches to change VLAN’s themselves to speed up desk moves etc…
• Remote Authentication – all the equipment was accessed using local accounts, when someone started or left, you can imagine the trouble going to all the kit and changing it, and forget about regular password changes – so people just didn’t bother, there were accounts on there that were for people who had been left 4/5 years. I used windows NPS (Network Policy Server) as a Radius Server, and went to each device, removed all the local accounts (accept a backup local admin account) and configured them for AAA and Radius. Now people use their domain credentials, which are controlled by the AD Password Policies, and the local admin account has the password changed often (which is very easy to do thanks to NCM!!!)

There are many many other things I have done here to try to bring the network up to scratch, unfortunately there are still many many things that need to be done, but I have been unable to due to the people who sign the cheques being unwilling to spend the money. I can, however, at least say it is in a MUCH better state than when I started.

Before I wrote this post, I decided to have a last walk around the computer rooms and patching rooms and couldn’t help feel a little sad. This was my first job where I had my “own” network, I know every cable and every switch, and I just hope it won’t miss me too much! I believe they are looking for a replacement, as I think they realized they can’t have the situation again where things are left to rot. I just hope the new engineer who comes in looks after it and treats it well…   …anyway, Nexus – here I come!!!

Advertisement