On Python, Networks and the py-junos-eznc library
One of my recent forays into Increasing the Awesome has involved learning about NETCONF and the Python programming language. I was lucky enough to spend some time with Jeremy Schulman during my trip to Sunnyvale for the Juniper Ambassadors Summit, and he introduced me to the new py-junos-eznc Python library he has been working on. I had spent a little bit of time earlier in the year looking at the original Ruby library, and I was amazed at how much thought had been put into this new library – obviously Jeremy’s learned a lot on the way!
 An Impatient Start
Let me make a couple of things clear right from the outset:
- I am not a programmer! Yes I have written the odd script here and there in the deep dark past, but I am by no means a programmer. All of my scripts have been about automating some task I had to do. As long as it worked, I didn’t care how efficient or pretty it was – it did what I needed.
- I have no intention of becoming a full time programmer! I like being a network architect and I like building and playing with network toys. All I want is the ability to make my job easier, which leads me to my last point…
- I am lazy! I don’t like repetitive work. I would rather do something once or twice and move on. Computers are here to do the mundane stuff for us, so we can create more awesome. I would rather write scripts for other people to do it next time instead of bugging me about it.
So with the above three stipulations in hand, I started to learn Python. Now, when I say “started”, I literally mean a week ago. I already knew the basics of loops and conditionals etc, but I couldn’t read a lick of Python this time last week. I tried the various online tutorials such as over at Code Academy and Learn Python the Hardway, but I knew the only way I was going to get my head around Python was to jump in and just start coding the working I had on my plate.
A Looming Project
I have a project starting in the next couple of weeks that is going to entail configuring about 50 switches with mostly the same configuration, followed by customising port layouts etc. Now just time involved in unboxing all those switches, cabling them up in the lab, firmware upgrading them, prepping the config and doing burn in tests can take a couple of days.
I knew this would be a prime candidate for “automation”. I had better got off my butt and work out how to do this!
So I contacted Jeremy and we hopped onto a Google Hangout where he was able to unleash 2.5 months of “Python knowledge” on me. It was 11pm for me and 5am for him, and it was so much fun 🙂 Over the course of our conversation we discussed a couple of the core concepts and data structures structures in the Junos-EZ libraries, and Jeremy discussed an idea he had to make extending the existing code base even easier. When I woke up the next morning, he had already committed it to the Github repository.
The Basic Structures
There is a whole heck of a lot in the Junos-EZ libraries, but by only understanding the very basics of the following components, I was able to create some pretty awesome scripts – with only 6 days of Python experience and about 24 hours with the Junos-EZ libraries 🙂
Device
The Device object represents a particular network device. When you call the .open() method, the script establishes an NETCONF over SSH connection to the device. From here you can make any of the NETCONF RPC calls you want against the device.
Tables
Now one of the great things about this library is that Jeremy has gone to great lengths to ensure that the average user of these libraries don’t need to understand the deep internals of the code to be able to add functionality. Tables are a prime example of this.
A table essentially represents all the data collected for a certain RPC request, sorted and keyed on a particular set of values and presented to the user as a collection of native Python data structures. The end user only needs to describe the data sets in YAML format without knowing the Python objects below.
The Table I use in this example is called PhyPortTable, shown here:
PhyPortTable: rpc: get-interface-information args: interface_name: '[fgx]e*' args_key: interface_name item: physical-interface view: PhyPortView
Views
A view is applied to a table to create a custom combination of the data in the table. As such, a combination of table and view definitions can be created with just a handful of lines of code yet scale infinitely.
The above PhyPortTable example calls the PhyPortView filter as shown here:
PhyPortView: fields: oper : oper-status admin : admin-status mtu: { mtu : int } link_mode: link-mode speed: speed macaddr: current-physical-address flapped: interface-flapped
The Opportunistic Script
So there I was onsite working with a customer yesterday afternoon, and I was asked –
“Can you please get me a list of all of the ports on the following switches, and tell me the current operating status and when they last flapped?”.
This request required looking at 5 switches and about 300 ports. Now what was I to do? Within the last 24hrs I had learned enough to know I could probably whip up a piece of code that would generate the required output. And it turned out to be relatively simple.
The code I wrote is shown here:
from jnpr.junos.op.phyport import * from jnpr.junos import Device dev = Device( user='netconf-test', host='lab-switch', password='lab123' ) dev.open() ports = PhyPortTable(dev).get() print "Port,Status,Flapped" #Print Header for CSV for port in ports: print("%s,%s,%s" % (port.key, port.oper, port.flapped))
This is by no means the prettiest code on the planet, but remember our goal was to just remove the mundane so we could increase the awesome, not become a programmer.
- Lines 1 and 2 tell Python to load the relevant libraries.
- Lines 4 and 5 make the NETCONF connection to my lab switch.
- Line 7 is where “the magic” lies. This line requests that the collection of data be returned as defined by the “PhyPortTable” and assign it to a variable called ports.
- Line 9 prints a simple CSV style header to the console
- Lines 12 – 14 loop through the collected data set and print the values for “oper” and “flapped” (names as defined in the view) in CSV format.
The output of this script is shown here:
Port,Status,Flapped ge-0/0/0,down,2013-05-24 08:06:54 UTC (11w6d 01:21 ago) ge-0/0/1,down,2013-01-11 11:56:10 UTC (30w5d 21:32 ago) ge-0/0/2,down,2013-05-18 08:09:32 UTC (12w5d 01:18 ago) ge-0/0/3,down,2013-04-18 16:57:15 UTC (16w6d 16:30 ago) ge-0/0/4,down,Never ge-0/0/5,down,Never ge-0/0/6,down,Never ge-0/0/7,down,Never ge-0/0/8,down,Never ge-0/0/9,down,Never ge-0/0/10,down,Never ge-0/0/11,down,Never ge-0/0/12,down,2013-07-06 06:00:48 UTC (5w5d 03:27 ago) ge-0/0/13,down,2013-07-06 06:00:48 UTC (5w5d 03:27 ago) ge-0/0/14,down,Never ge-0/0/15,down,Never ge-0/0/16,down,Never ge-0/0/17,down,Never ge-0/0/18,down,Never ge-0/0/19,down,Never ge-0/0/20,down,Never ge-0/0/21,up,2013-08-02 08:20:46 UTC (1w6d 01:07 ago) ge-0/0/22,up,2012-09-09 14:10:50 UTC (48w3d 19:17 ago) ge-0/0/23,up,2012-09-09 14:10:51 UTC (48w3d 19:17 ago)
I was able to run this script across each of the switches I needed to query and provide an “Excel Version” for the customer who asked for the information.
Mop and Bucket
This post only covers an extremely simple example, but most of what we really want to stop doing manually every day is not much more complicated that the above example.
I am going write a follow up post in the next couple of days to discuss how within 24hrs of starting with the Junos-EZ library I was able to bring LLDP discovery functionality to the library with no real skill other than the ability to cut, paste and pray my way through it.
Anybody can work on this stuff, so get out there and start coding!