Saturday, September 8, 2018

Update to testing 10Gb links with iPerf3

In a previous blog, I discussed using an HP Z420 workstation for testing 10Gb link quality. You can find it here - Using iPerf3 to Test 2.5Gb/5Gb and 10Gb Links. 

The Z420 that I used has an Intel E5-1620 processor and 8GB of RAM. I purchased it on eBay for around $300. I just checked and a Z420 with an E5-2680 and 32GB of RAM is going for around $375. The extra RAM and faster processor would be nice.

I added a 10Gb PCIe card from eBay - 666172-001 10GB MELLANOX PCIe 10GBe ETHERNET NIC for $16.00. That allowed me to connect the Z420 to the customer's switch and run iPerf3. That worked great but I wanted a way to verify that the Z420 could actually run at a full 10Gb and be able to show the customer before I test.

So, I decided to buy another 10Gb adapter and use a VM to test from the Z420's Ubuntu 18.04 host to the guest. I purchased the exact same adapter because the Mellanox driver was already installed and I knew it worked well.

Using KVM and Virt-manager

I decided to use KVM as the hypervisor and Virt-manager as the manager instead of VMware Workstation for this application. There are a few reasons I decided to use KVM:
  • VMware workstation is a proprietary package that costs $199. Plus you have to keep maintenance on it or buy it again when the next version comes out. It's a great tool and I use it on my laptop, but I didn't want to spend the money on my server.
  • KVM is built into Linux and Virt-manager is a free, open-source tool. That means that KVM is automatically upgraded when the kernel is updated and Virt-manager will be updated by the Ubuntu package manager. 
  • I'm studying Software Defined Networking and Linux is a big part of that. For example, NetAPP has a KVM based version of their SAN controller and customers will need help installing and optimizing it. There are also a lot of Linux servers running KVM in data centers and as a network engineer I want to know how to configure them for network access.

I am going to write another blog on how to get KVM up and running. It was a great experience and I learned quite a bit in the process since I had to install Virt-manager, create the bridge, modify firewall rules, and troubleshoot a communications issues that ultimately was caused by my Docker install. How much more fun could you have on a Saturday afternoon?

The results

I had a 10Gb 16 port switch in my lab and a couple of short DAC cables so connecting the two 10Gb adapters to my network was easy enough. Once I had KVM installed and the bridges created, I grabbed my Perfsonar toolkit ISO and built a CentOS 7 Perfsonar VM. The process was almost identical to VMware Workstation.

Virt-manager makes it easy to clone VMs so before I started configuring the VM I cloned it using "virt-clone --original Perfsonar4-1 --name Perfsonar4-2 --auto-clone" from the terminal. The tool takes care of changing MAC addresses but you will need to change the hostname, ssh keys, etc.

Here is what Virt-manager looked like with the two VMs created:


Networking in Virt-manager is similar to VMware Workstation. Here is a screenshot of the bridges:


In the VM details, you select the Bridge to use. Virt-manager lists the Bridge name, BR0 in this case, and the physical interface on the host, ens3.

One thing I learned is that you need to use the virtio device model. Initially, I selected E1000 based on my experience with VMware ESXi and it took me a few minutes to figure out why I was getting 941Gbps when testing!



I was worried that the E5-1620 wouldn't have enough power to run the Z420 and the VM at 10Gb but it worked no problem. CPU utilization on the VM ran around 65% most of the time and maxed out at 77%.

I only gave the VM 2 vCPUs, if it had maxed out, I would have been able to add another one. Here is a screenshot of HTOP that I grabbed on the VM during the test:


Here is the output from iPerf3 on the Z420. Notice there were no retries (Retr) after the first second and the Congestion Window (Cwnd) was very consistent.



mhubbard@Z420:~$ iperf3 -c 192.168.10.187 -P4 -O2
Connecting to host 192.168.10.187, port 5201
[  5] local 192.168.10.185 port 47044 connected to 192.168.10.187 port 5201
[  7] local 192.168.10.185 port 47046 connected to 192.168.10.187 port 5201
[  9] local 192.168.10.185 port 47048 connected to 192.168.10.187 port 5201
[ 11] local 192.168.10.185 port 47050 connected to 192.168.10.187 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   310 MBytes  2.60 Gbits/sec   35    404 KBytes       (omitted)
[  7]   0.00-1.00   sec   267 MBytes  2.24 Gbits/sec   78    288 KBytes       (omitted)
[  9]   0.00-1.00   sec   236 MBytes  1.98 Gbits/sec   85    372 KBytes       (omitted)
[ 11]   0.00-1.00   sec   239 MBytes  2.00 Gbits/sec  111    349 KBytes       (omitted)
[SUM]   0.00-1.00   sec  1.03 GBytes  8.82 Gbits/sec  309             (omitted)
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec   281 MBytes  2.36 Gbits/sec    0    417 KBytes       (omitted)
[  7]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec    0    407 KBytes       (omitted)
[  9]   1.00-2.00   sec   279 MBytes  2.34 Gbits/sec    0    440 KBytes       (omitted)
[ 11]   1.00-2.00   sec   279 MBytes  2.34 Gbits/sec    0    450 KBytes       (omitted)
[SUM]   1.00-2.00   sec  1.09 GBytes  9.39 Gbits/sec    0             (omitted)
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   0.00-1.00   sec   279 MBytes  2.34 Gbits/sec    0    428 KBytes       
[  7]   0.00-1.00   sec   280 MBytes  2.35 Gbits/sec    0    407 KBytes       
[  9]   0.00-1.00   sec   279 MBytes  2.34 Gbits/sec    0    491 KBytes       
[ 11]   0.00-1.00   sec   280 MBytes  2.35 Gbits/sec    0    452 KBytes       
[SUM]   0.00-1.00   sec  1.09 GBytes  9.38 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec    0    428 KBytes       
[  7]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec    0    417 KBytes       
[  9]   1.00-2.00   sec   280 MBytes  2.35 Gbits/sec    0    509 KBytes       
[ 11]   1.00-2.00   sec   279 MBytes  2.34 Gbits/sec    0    475 KBytes       
[SUM]   1.00-2.00   sec  1.09 GBytes  9.39 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec   280 MBytes  2.35 Gbits/sec    0    437 KBytes       
[  7]   2.00-3.00   sec   280 MBytes  2.35 Gbits/sec    0    420 KBytes       
[  9]   2.00-3.00   sec   279 MBytes  2.34 Gbits/sec    0    533 KBytes       
[ 11]   2.00-3.00   sec   279 MBytes  2.34 Gbits/sec    0    498 KBytes       
[SUM]   2.00-3.00   sec  1.09 GBytes  9.37 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0    465 KBytes       
[  7]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0    427 KBytes       
[  9]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0    533 KBytes       
[ 11]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0    498 KBytes       
[SUM]   3.00-4.00   sec  1.09 GBytes  9.39 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec   280 MBytes  2.35 Gbits/sec    0    530 KBytes       
[  7]   4.00-5.00   sec   280 MBytes  2.35 Gbits/sec    0    430 KBytes       
[  9]   4.00-5.00   sec   281 MBytes  2.35 Gbits/sec    0    533 KBytes       
[ 11]   4.00-5.00   sec   280 MBytes  2.35 Gbits/sec    0    498 KBytes       
[SUM]   4.00-5.00   sec  1.09 GBytes  9.40 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec   280 MBytes  2.35 Gbits/sec    0    619 KBytes       
[  7]   5.00-6.00   sec   279 MBytes  2.34 Gbits/sec    0    438 KBytes       
[  9]   5.00-6.00   sec   280 MBytes  2.35 Gbits/sec    0    533 KBytes       
[ 11]   5.00-6.00   sec   279 MBytes  2.34 Gbits/sec    0    498 KBytes       
[SUM]   5.00-6.00   sec  1.09 GBytes  9.38 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.00   sec   280 MBytes  2.35 Gbits/sec    0    735 KBytes       
[  7]   6.00-7.00   sec   280 MBytes  2.35 Gbits/sec    0    445 KBytes       
[  9]   6.00-7.00   sec   279 MBytes  2.34 Gbits/sec    0    533 KBytes       
[ 11]   6.00-7.00   sec   279 MBytes  2.34 Gbits/sec    0    501 KBytes       
[SUM]   6.00-7.00   sec  1.09 GBytes  9.38 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.00-8.00   sec   280 MBytes  2.35 Gbits/sec    0    871 KBytes       
[  7]   7.00-8.00   sec   280 MBytes  2.35 Gbits/sec    0    448 KBytes       
[  9]   7.00-8.00   sec   281 MBytes  2.35 Gbits/sec    0    539 KBytes       
[ 11]   7.00-8.00   sec   279 MBytes  2.34 Gbits/sec    0    513 KBytes       
[SUM]   7.00-8.00   sec  1.09 GBytes  9.39 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec    0   1.05 MBytes       
[  7]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec    0    448 KBytes       
[  9]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec    0    550 KBytes       
[ 11]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec    0    523 KBytes       
[SUM]   8.00-9.00   sec  1.09 GBytes  9.39 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec   281 MBytes  2.36 Gbits/sec    0   1.25 MBytes       
[  7]   9.00-10.00  sec   281 MBytes  2.35 Gbits/sec    0    479 KBytes       
[  9]   9.00-10.00  sec   280 MBytes  2.35 Gbits/sec    0    576 KBytes       
[ 11]   9.00-10.00  sec   280 MBytes  2.35 Gbits/sec    0    546 KBytes       
[SUM]   9.00-10.00  sec  1.10 GBytes  9.41 Gbits/sec    0             
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.73 GBytes  2.35 Gbits/sec    0             sender
[  5]   0.00-10.03  sec  2.74 GBytes  2.35 Gbits/sec                  receiver
[  7]   0.00-10.00  sec  2.73 GBytes  2.35 Gbits/sec    0             sender
[  7]   0.00-10.03  sec  2.74 GBytes  2.35 Gbits/sec                  receiver
[  9]   0.00-10.00  sec  2.73 GBytes  2.35 Gbits/sec    0             sender
[  9]   0.00-10.03  sec  2.74 GBytes  2.35 Gbits/sec                  receiver
[ 11]   0.00-10.00  sec  2.73 GBytes  2.35 Gbits/sec    0             sender
[ 11]   0.00-10.03  sec  2.74 GBytes  2.35 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  10.9 GBytes  9.39 Gbits/sec    0             sender
[SUM]   0.00-10.03  sec  11.0 GBytes  9.39 Gbits/sec                  receiver


References

Convert code to HTML for Blogger
How to clone existing KVM virtual machine images on Linux
Install And Set Up KVM On Ubuntu 18.04 Bionic Beaver Linux
Create and Run Virtual Machines With virt-manager
Predictable Network Interface Names
libvirt Networking Handbook
KVM/Networking
Ubuntu 16.04 kvm bridges not working
bridge networking not working on ubuntu 16.04
Configure network interface as DHCP client on RHEL7 Linux
Open firewall port on CentOS 7
KVM - Create a virtual machine with 2 bridges interfaces
Posted by at 1 comment:
Labels: , , ,

Saturday, August 11, 2018

Using iPerf3 to Test 2.5Gb/5Gb and 10Gb Links

I am a big fan of the iPerf3 tool written by ESnet, a part of the US Department of Energy. Here is a definition of iPerf from their official github page:

"iperf is a tool for active measurements of the maximum achievable bandwidth on IP networks.  For each test it reports the measured throughput / bitrate, loss, and other parameters."

I have previously blogged about iPerf and how to use it on Windows, Mac OSX, IOS, Android and Linux. You can find that blog here -  Using iPerf3 to verify Link Quality

Introduction

IPerf can be used to test\verify any IP based link. Here are examples of what I have tested using iperf3:

  • Remote access VPNs - When a user complains that his home Internet connection is 60Mbps but using VPN back to the office is "slow" you can verify the connection with iPerf. A lot of business Internet connections are asymmetric, for example 60Mbps down and 5Mbps up. When the user connects to the office they are on the 5Mbps upload side, not the 60Mbps download side!
  • Site to site VPNs - If you are experiencing a slow connection on a site to site VPN it could be the Internet connection at either site, the firewall at either site or the protocol being used to transfer data. With iPerf you can determine the root cause. 
  • MPLS links - If you are having performance issues with an MPLS circuit the carrier will always say that their circuit is working correctly. An iPerf test will give you the data you need to push back. 
  • Wireless access points - Anytime I deploy a new AP I set my laptop up in the MDF, connect wirelessly to the AP with a second laptop and verify the bandwidth. I have found problems with fibre connections, structured CAT cabling and even the carriers NID using iPerf.
  • Data center to central office - Depending on the carrier you may be able to use iPerf between the data center and the central office. That was actually my introduction to iPerf years ago when I worked for a carrier services group.
  • Virtual machine to virtual machine - Find bottlenecks in your virtual infrastructure.

 

Testing 10Gb site to site links

A lot of customers are moving services to the data center and eliminating servers at remote sites. This presents a problem when you are asked to test the site to site link. 

It's easy enough spin up a CentOS box on the virtual infrastructure at the data center as an end point but what to do at the remote site? I purchased an HP z420 workstation off lease on ebay for under $300. I also purchased an HP (Mellanox) 10Gb fiber card off ebay for under $30. 

I installed Ubuntu on the Z420 and Mellanox had the correct driver on thier website. I purchased a single port card but with hindsight I should have purchased a dual port card. That would allow me to test from virtual machine to virtual machine over the 10Gb link (not the Z420's backplane) without needing two 10G capable Z420s.

I recently got to test new 10Gb links at a customer with four remote sites. The customer had HPE switches and luckily he had a 3m HP DAC cable so connecting the Z420 to the switches was easy. On ebay you can purchase 3m DAC cables for under $50. It's best to have a DAC cable made by the switch manufacturer to avoid compatibility issues. You can also find 10Gb optics for under $50 on ebay.

The Z420 worked great and I was able to verify that each site was performing correctly. But it was 110°F (43°C) outside and carrying the Z420, monitor and keyboard to each site wasn't ideal. What to do?

Test MultiGig, NBASE-T and 10Gb with a Laptop?

Laptops have started shipping with Thunderbolt 3 connections. Thunderbolt has a 40Gbps interface to 10Gb is well within its capability. A quick Google search turned up the following Thunderbolt 3 to 10Gb adapters:

 Sonnet Solo 10Gbase-T - This Thunderbolt 3 to 10Gb copper adapter also supports 2.5Gb/5Gb Ethernet so you can test the new MultiGig and NBASE-T switches. The webpage only shows Mac/Windows but the 10Gbe controller is an AQC-107S and there are Linux drivers for it. You have to build from source but there are detailed instructions in the readme. The cost is only $199 so it's within my budget!

 Sonnet Twin 10G SFP+ - This Thunderbolt 3 to 10Gb adapter has two standard SFP+ ports. It uses the Intel 82599 controller so there are Linux/Mac/Windows drivers. The cost is $499 so it's outside the budget for my personal toolkit but is reasonable for a company.

Now, I just need to buy a new laptop with a Thunderbolt 3 port! The 17" System76 Oryx Pro is the model on my short list! It has Thunderbolt 3, nvidia 1060 (or 1070) and two m.2 NVME slots.

References

iperf3: A TCP, UDP, and SCTP network bandwidth measurement tool 
perfSonar -  A bandwidth testing suite of tools. Available in ISO format in four different toolkits. You can build a complete distributed link quality system with web based dashboard using perfSONAR.
perfSONAR Project YouTube Channel 
perfSONAR Powered - Podcast on the Research Computing and Engineering (RCE) podcast network
Posted by at 1 comment:

Saturday, August 4, 2018

DNS Rebinding attacks

As we all know, DNS is used to translate Domain names into IP addresses. DNS uses UDP so it has had a long history of being abused by hackers for DoS. To make matters worse it doesn't have authentication or encryption so Man in the Middle (MiTM) attacks are possible.

Since DNS is used everytime you use the Internet it is hard to overstate the importance of a good DNS service. Companies like OpenDNS (Now Cisco Umbrella) and Quad 9 (www.quad9.net) have added security features like Malware detection and malicious site protection. These services are free for home use and paid for businesses.

Recently an old type of attack using DNS has become popular again - DNS Rebinding. Tripwire has a good explanation of what a DNS rebinding attack is - Practical Attacks with DNS Rebinding.

Armis.com gives this definition for DNS Rebinding. See the references for the link to Armis.com's DNS Rebinding Exposes Half a Billion Devices in the Enterprise. There is a link in the reference section to a youtube video on how it works.

**************************************************
DNS Rebinding Attacks Explained

DNS rebinding takes advantage of a nearly decade-old flaw in web browsers that allows a remote attacker to bypass a victim’s network firewall and use their web browser as a proxy to communicate directly with vulnerable devices on the local network. An example of a vulnerable device is one that is running an unauthenticated protocol like Universal Plug and Play (UPnP) or HTTP (used on unencrypted web servers). These protocols are commonly used to host administrative consoles (for routers, printers, IP cameras) or to allow easy access to the device’s services (for example, streaming video players), and are pervasive in businesses.
**************************************************

Preventing the Attack

There are several things you should do on your home network to prevent attacks:
  • Change default credentials - A lot of script based attacks work because the default credentials weren't changed.
  • Change the internal network IP address scheme - The scripts work by trying to log into common IP addresses used by network devices like 192.168.1.1 or 192.168.0.1 
  • Disable uPnP - Universal Plug and Play can be abused by attackers. If you are a gamer there are plenty of sites that will explain how to port forward once you turn off uPnP.
  • Update the firmware on you network devices - This is a MUST DO and is overlooked my most home users
  • Install DD-WRT on your SOHO router - There is a link in the reference section below.
  • Use OpenDNS 

Configuring OpenDNS to block rebinding attacks

I had been using Quad9 recently because it's fast, new and supports DNS over TLS along with DNSSec but decided to switch to OpenDNS because they offer rebinding filtering. Here is their explanation:

**************************************************
Block internal IP addresses

When enabled, DNS responses containing IP addresses listed in RFC1918 will be filtered out. This helps to prevent DNS Rebinding attacks. For example, if badstuff.attacker.com points to 192.168.1.1, this option would filter out that response.

The three blocks of IP addresses filtered in responses are:

10.0.0.0     - 10.255.255.255  (10/8)
172.16.0.0   - 172.31.255.255  (172.16/12)
192.168.0.0  - 192.168.255.255 (192.168/16)
**************************************************

To take advantage of this feature you need to create an OpenDNS account at https://login.opendns.com. Once you have an account, login and click on the Settings tab. At the bottom you will see a link "Keep your network's IP up-to-date with our free software." It says Mac and Windows but there is a Linux client also.

When you click the link it start the download. Once it finishes, run the program. It will ask you to log into OpenDNS. The updater will show the public IP address of your router.


Once you do that, go back to OpenDNS in the browser and click settings again. You should see the public IP address of your router listed under Add a network. Click Add This Network.



If you look at the updater now, you will see your public IP address listed. Back on the OpenDNS page click down arrow next to --Select a Network-- and select your network. On the dialog that opens, click on security and put a check in the box next to "Block Internal IP addresses"




Now, if a script tries to use an RFC1918 address to spoof a domain it will get filtered by OpenDNS. This isn't a silver bullet but just one more layer of defense.

Testing the Filter

Steve Gibson of Gibson Research wrote a DNS benchmark way back in 2010 that is free and works well for benchmarking DNS performance. He also created some DNS addresses for testing rebinding. There is a link to the original 2010 podcast (episode 260) and the July 24, 2018 update in the reference section.

To test if your DNS server filters RFC1918 addresses, open a terminal or cmd window and enter the following:

nslookup net4.rebindtest.com
nslookup net10.rebindtest.com
nslookup net127.rebindtest.com
nslookup net172.rebindtest.com
nslookup net192.rebindtest.com

Below is the output before I configured the OpenDNS filter. Notice that the address returned for net172.rebindtest.com is 172.16.0.1 which would allow a malicious script to bypass the Same Origin Policy of the browser.


nslookup net172.rebindtest.com
Server:  10.208.0.1
Address: 10.208.0.1#53

Non-authoritative answer:
Name: net172.rebindtest.com
Address: 172.16.0.1
Name: net172.rebindtest.com
Address: ::ffff:172.16.0.1

Below is the output after the filter was enabled. Notice that the address returned is 146.112.61.109.


nslookup net172.rebindtest.com
Server:  127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: net172.rebindtest.com
Address: 146.112.61.109
Name: net172.rebindtest.com
Address: ::ffff:146.112.61.109

What is 146.112.61.109 you ask? You could use nslookup to find out but I wanted to show the dig (DNS Information Groper) command. It's built into Linux/Mac and you can install it on Windows. Here is a blog I wrote on installing dig DNS Information Groper for Windows

146.112.61.109 is the address OpenDNS uses for hit-block.opendns.com so the filter is working!


dig -x 146.112.61.109

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> -x 146.112.61.109
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1651
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;109.61.112.146.in-addr.arpa.    IN    PTR

;; ANSWER SECTION:
109.61.112.146.in-addr.arpa. 3600 IN    PTR    hit-block.opendns.com.

;; Query time: 18 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Sat Aug 04 23:38:43 PDT 2018
;; MSG SIZE  rcvd: 91



References
Dear developers, beware of DNS Rebinding
How DNS Rebinding Attacks Impacts The Enterprise - youtube video
Half a billion smart devices vulnerable to decade-old DNS rebinding attacks 
DNS Rebinding Exposes Half a Billion Devices in the Enterprise
DNS Rebinding - Security Now podcast from 2010. Still applicable today.
Security Now 673 - Show notes
GRC DNS Benchmark - Windows only
DNS Information Groper for Windows - How to install dig on Windows
DD-WRT
Convert code samples into HTML for blogger
Posted by at No comments:
Labels: , , ,

Friday, July 27, 2018

Learning Python 3

Updated September 10th, 2022

Here are some of the resources I used to start learning python. With my newly learned  Python skills and some Google-Fu I have been able to automate several network device tasks. You can find most of my python scripts at Hubbard on Networking GitHub Repo.

You do NOT need to know anything about python to use the scripts. This is just like you don't have to know how to program to run "show interface g1/0/1". You simply download the script and execute it.

I found this tweet that has a curated list of python resources:


Why You Should Learn Python

A lot of the tasks that network engineers perform are repetitive, mind-numbing, and error-prone. With a little python skill, you can automate these tasks and spend the time you save on more productive tasks, like learning more python.

As networking moves away from the CLI and into software-defined networking (SDN) you will need to have some dev skills. Cisco has a three-part video on how the network engineer's job is changing. You will need to register with Cisco Devnet to watch it but you should have a Devnet account if you going down this path:
Lesson 1: The Network Engineer of Old

If you have never used Python at all, this blog is the place to get started!

First, to install python, head over to the Python download page - Download Python
Follow the instructions to install python on your OS of choice.

Tools

There isn't much needed to use Python other than python itself but like most things, life is easier with some tools.

Integrated Development Environment (IDE)

IDEs are tools that allow you to write and debug code. Once you start writing scripts that are more than a few lines long you will want to use an IDE.

Thonny - Python IDE for beginners. A free Integrated Development Environment. This tool is great for learning. It lets you step through a script and see exactly what is happening.

MU - a simple Python editor for beginner programmers. This one is really nice. It's cross-platform and has support for Linux, Mac, Windows, and Raspian. What separates it from the other tools listed here is the support for small devices like Adafruit and Micro:bit. Here is a screenshot of MU starting up:



Microsoft Visual Studio Code - A free open source IDE from Microsoft. VSCode has turned into an amazing development environment and can be used for any size project. It has an integrated debugger that looks just like the Powershell ISE tool. It supports almost every programming language, not just python. The open-source version of PowerShell is fully supported which is great. I can use one tool and create Python and PowerShell scripts. 

There is a Visual Studio Marketplace with thousands of plugins, similar to the Chrome store or Firefox store. Extensions for the Visual Studio family of products

I have more detail on my macOS blog. Once you have VScode installed it works the same on Windows, Mac, and Linux so the blog is useful regardless of what OS you use. My VSCode install blog

Code Editor

You can definitely use VSCode as your editor. It has about every feature that you could want. But I have been using Sublime text since before VSCode came out and old habits die hard! If you are on Ubuntu 22.04 you can install Gnome Text Editor (I know, really, really stupid name) and it works great for quick editing.

Sublime text - Sublime is a text editor that is optimized for programming. It has a huge community around it and thousands of plugins. It's $75.00 but the license lets you run it on as many machines as you own. I bought it and installed it on my Linux, Windows, and Mac laptops. It has a tabbed interface like notepad++ which I like. You can also split the screen and open two files side by side. This is useful because you can have your script on one side and the data file on the other. 

Realpython.com sells a great tutorial that walks you through installing Sublime text optimized for Python. It includes video and text for Mac, Linux, and Windows. This tutorial is well worth the cost and has improved my productivity in Sublime.

In the next section, I discuss revision control with Git. Sublime text has several plugins that integrate Git into your workflow.  One I really like is Git Gutter. It puts a + sign next to lines that have changed, then you can revert easily if the change didn’t work.
Your Shortcut to a Professional Python Development Setup

Revision Control

Git - A revision control system. It's useful once you start writing scripts large enough to have bugs or future enhancements. Even if you don't program in python it's worth installing git because of github.com and gitlab.com.

Both of these sites have thousands of python tools. If you have git installed on your computer all you have to do to use them is "git clone <repository>". For example, to install the ARP sorting tool I wrote, you simply go to ARP-Sort and click the "Code" button. It will display the URL for the project. Click the copy button, type “git clone” and paste the URL you copied into the terminal, then press enter:

git clone https://github.com/rikosintie/ARP-Sort.git

This downloads the project and unzips it into the folder ARP-Sort.




Someone has created an online game for learning Git - An open source game about learning Git!
Cisco DevNet GitHub Repo - Great resources on setting up your laptop with Python, Git, Postman, etc.
Git and GitHub for Beginners - Crash Course - A really good video on git.
 


Videos from Udemy.com 

Udemy has inexpensive video training for Python, Linux, and a lot of other applications. These are on sale all the time for $9.99

  • Python Network Programming - Part 1: Build 7 Python Apps
  • Complete Python Bootcamp: Go from zero to hero in Python 3
  • 100 Days of Code - The Complete Python Pro Bootcamp for 2021

Videos on Youtube

I find that I learn better when using books and websites but sometimes it's nice to watch a video.

Real python did a blog on the best Youtube channels for python
The Ultimate List of Python YouTube Channels

Subscribe to HackerSploit on Youtube and there is a complete series on Python. He uses 2.7 but it is still a good tutorial. They are really basic but I find Alexis entertaining.
Python For Ethical Hacking - #1 - Introduction & Python Modules

Telusko Python for Beginners
Python Tutorial - Python for Beginners [Full Course] - Mosh 
Learn Python - Full Course for Beginners [Tutorial] Free Code Camp
Writing Pythonic Code - A Tweet thread by @Bascodes. He is worth following.

Corey Shafer Videos

I recommend that you subscribe to this channel. Here are four of his videos that I found very useful

Requests Tutorial - The requests library is used to pull data down from webservers
JSON Tutorial - Java Script Object Notation is a common data format used by APIs
Sorting Tutorial - Sorting routines for lists and dictionaries
List Comprehensions - Working with list comprehensions

e-books from Amazon  

I love the Kindle app on my laptop, phone and iPad. If I get stuck in a long line I just open it up and do some studying! These were all under $5. You can go to Amazon, set filter to Kindle store and enter "python programming free book" and find a lot of free books.

  • Learn Python in One Day and Learn It Well Python for Beginners with Hands-on Project The only book you need to start coding in Python immediately By Jamie Chan
  • Python Tips and Tricks: Learn the Best Tips and Tricks to Get The Most out of Python NOW! Jones, Daniel
  • The Fundamentals Of Python Programming: A Complete Beginners Guide To Python Mastery.
  • Python Programming Tips and Tricks: The Ultimate Cheat Sheet for Python Programming. 20+ Tips and Tricks to Make Your Life Easier and More Efficient
  • Automate the Boring Stuff with Python: Practical Programming for Total Beginners


Websites

There are so many python websites it would be impossible to list them all, here are a few I have found very useful:

Getting Started

General Tips

Jinja Templates

Network Engineering

Debugging

Web Development


Style guides

Python is a structured language that uses white space as part of the structure. But you can still create ugly code. The official Python style guide is part of the Python Enhancement Proposals (PEPs) which are documents that provide guidance and spell out best practices for how Python code should be organized, packaged, released, deprecated, and so on. PEP8 is a popular standard for styling Python code and is extensively used in the developer community. 

Python Virtual Environments

As you start learning and creating more python tools you will want to start using virtual environments. These two blogs will teach you what you need to know. There are a lot more blogs available on the Internet.

Podcasts

You probably won't learn much coding from a podcast but these are very interesting to listen to. I find a lot of good links in the show notes. For example, I learned about Thonny, MU, and Visual Studio Code from podcasts.
Talk Python To Me - A good podcast that covers a lot of topics. There will be one called "Teaching Python to network engineers" in August, 2018!
Podcast.__init__ - A podcast about Python and the people who make it great. Hosted by Tobias Macey.
Python Bytes - Python Bytes podcast delivers headlines directly to your earbuds. If you want to stay up on the Python developer news but don’t have time to scour Reddit, Twitter, and other news sources, just subscribe and you’ll get the best picks delivered weekly.

Keywords In Python

There are 33 keywords that should never be used as a variable, function name, class, object, or as any other identifiers in your programs.

 false
 True
 finally
 class
 for
 continue
 none
 return
 lambda
 try
 is
 def
 from
 nonlocal
 while
 and
 not
 global
 del
 with
 as
 elif
 if
 or
yield
 break
 import
 except
 pass
 assert
 else
 raise
 in

Libraries

Pint - Pint is a Python package to define, operate and manipulate physical quantities

Posted by at 2 comments:
Labels: , , ,
Subscribe to: Posts (Atom)