Saturday, August 11, 2018

Using iPerf3 to Test 2.5Gb/5Gb and 10Gb Links

I am a big fan of the iPerf3 tool written by ESnet, a part of the US Department of Energy. Here is a definition of iPerf from their official github page:

"iperf is a tool for active measurements of the maximum achievable bandwidth on IP networks.  For each test it reports the measured throughput / bitrate, loss, and other parameters."

I have previously blogged about iPerf and how to use it on Windows, Mac OSX, IOS, Android and Linux. You can find that blog here -  Using iPerf3 to verify Link Quality


Introduction

IPerf can be used to test\verify any IP based link. Here are examples of what I have tested using iperf3:

  • Remote access VPNs - When a user complains that his home Internet connection is 60Mbps but using VPN back to the office is "slow" you can verify the connection with iPerf. A lot of business Internet connections are asymmetric, for example 60Mbps down and 5Mbps up. When the user connects to the office they are on the 5Mbps upload side, not the 60Mbps download side!
  • Site to site VPNs - If you are experiencing a slow connection on a site to site VPN it could be the Internet connection at either site, the firewall at either site or the protocol being used to transfer data. With iPerf you can determine the root cause. 
  • MPLS links - If you are having performance issues with an MPLS circuit the carrier will always say that their circuit is working correctly. An iPerf test will give you the data you need to push back. 
  • Wireless access points - Anytime I deploy a new AP I set my laptop up in the MDF, connect wirelessly to the AP with a second laptop and verify the bandwidth. I have found problems with fibre connections, structured CAT cabling and even the carriers NID using iPerf.
  • Data center to central office - Depending on the carrier you may be able to use iPerf between the data center and the central office. That was actually my introduction to iPerf years ago when I worked for a carrier services group.
  • Virtual machine to virtual machine - Find bottlenecks in your virtual infrastructure.

 

Testing 10Gb site to site links

A lot of customers are moving services to the data center and eliminating servers at remote sites. This presents a problem when you are asked to test the site to site link. 

It's easy enough spin up a CentOS box on the virtual infrastructure at the data center as an end point but what to do at the remote site? I purchased an HP z420 workstation off lease on ebay for under $300. I also purchased an HP (Mellanox) 10Gb fiber card off ebay for under $30. 

I installed Ubuntu on the Z420 and Mellanox had the correct driver on thier website. I purchased a single port card but with hindsight I should have purchased a dual port card. That would allow me to test from virtual machine to virtual machine over the 10Gb link (not the Z420's backplane) without needing two 10G capable Z420s.

I recently got to test new 10Gb links at a customer with four remote sites. The customer had HPE switches and luckily he had a 3m HP DAC cable so connecting the Z420 to the switches was easy. On ebay you can purchase 3m DAC cables for under $50. It's best to have a DAC cable made by the switch manufacturer to avoid compatibility issues. You can also find 10Gb optics for under $50 on ebay.

The Z420 worked great and I was able to verify that each site was performing correctly. But it was 110°F (43°C) outside and carrying the Z420, monitor and keyboard to each site wasn't ideal. What to do?


Test MultiGig, NBASE-T and 10Gb with a Laptop?

Laptops have started shipping with Thunderbolt 3 connections. Thunderbolt has a 40Gbps interface to 10Gb is well within its capability. A quick Google search turned up the following Thunderbolt 3 to 10Gb adapters:

Sonnet Solo 10Gbase-T - This Thunderbolt 3 to 10Gb copper adapter also supports 2.5Gb/5Gb Ethernet so you can test the new MultiGig and NBASE-T switches. The webpage only shows Mac/Windows but the 10Gbe controller is an AQC-107S and there are Linux drivers for it. You have to build from source but there are detailed instructions in the readme. The cost is only $199 so it's within my budget!

Sonnet Twin 10G SFP+ - This Thunderbolt 3 to 10Gb adapter has two standard SFP+ ports. It uses the Intel 82599 controller so there are Linux/Mac/Windows drivers. The cost is $499 so it's outside the budget for my personal toolkit but is reasonable for a company.

Now, I just need to buy a new laptop with a Thunderbolt 3 port! The 17" System76 Oryx Pro is the model on my short list! It has Thunderbolt 3, nvidia 1060 (or 1070) and two m.2 NVME slots.


References

iperf3: A TCP, UDP, and SCTP network bandwidth measurement tool 
perfSonar -  A bandwidth testing suite of tools. Available in ISO format in four different toolkits. You can build a complete distributed link quality system with web based dashboard using perfSONAR.
perfSONAR Project YouTube Channel 
perfSONAR Powered - Podcast on the Research Computing and Engineering (RCE) podcast network

Saturday, August 4, 2018

DNS Rebinding attacks

As we all know, DNS is used to translate Domain names into IP addresses. DNS uses UDP so it has had a long history of being abused by hackers for DoS. To make matters worse it doesn't have authentication or encryption so Man in the Middle (MiTM) attacks are possible.

Since DNS is used everytime you use the Internet it is hard to overstate the importance of a good DNS service. Companies like OpenDNS (Now Cisco Umbrella) and Quad 9 (www.quad9.net) have added security features like Malware detection and malicious site protection. These services are free for home use and paid for businesses.

Recently an old type of attack using DNS has become popular again - DNS Rebinding. Tripwire has a good explanation of what a DNS rebinding attack is - Practical Attacks with DNS Rebinding.

Armis.com gives this definition for DNS Rebinding. See the references for the link to Armis.com's DNS Rebinding Exposes Half a Billion Devices in the Enterprise. There is a link in the reference section to a youtube video on how it works.

**************************************************
DNS Rebinding Attacks Explained

DNS rebinding takes advantage of a nearly decade-old flaw in web browsers that allows a remote attacker to bypass a victim’s network firewall and use their web browser as a proxy to communicate directly with vulnerable devices on the local network. An example of a vulnerable device is one that is running an unauthenticated protocol like Universal Plug and Play (UPnP) or HTTP (used on unencrypted web servers). These protocols are commonly used to host administrative consoles (for routers, printers, IP cameras) or to allow easy access to the device’s services (for example, streaming video players), and are pervasive in businesses.
**************************************************

Preventing the Attack

There are several things you should do on your home network to prevent attacks:
  • Change default credentials - A lot of script based attacks work because the default credentials weren't changed.
  • Change the internal network IP address scheme - The scripts work by trying to log into common IP addresses used by network devices like 192.168.1.1 or 192.168.0.1 
  • Disable uPnP - Universal Plug and Play can be abused by attackers. If you are a gamer there are plenty of sites that will explain how to port forward once you turn off uPnP.
  • Update the firmware on you network devices - This is a MUST DO and is overlooked my most home users
  • Install DD-WRT on your SOHO router - There is a link in the reference section below.
  • Use OpenDNS 

Configuring OpenDNS to block rebinding attacks

I had been using Quad9 recently because it's fast, new and supports DNS over TLS along with DNSSec but decided to switch to OpenDNS because they offer rebinding filtering. Here is their explanation:

**************************************************
Block internal IP addresses

When enabled, DNS responses containing IP addresses listed in RFC1918 will be filtered out. This helps to prevent DNS Rebinding attacks. For example, if badstuff.attacker.com points to 192.168.1.1, this option would filter out that response.

The three blocks of IP addresses filtered in responses are:

10.0.0.0     - 10.255.255.255  (10/8)
172.16.0.0   - 172.31.255.255  (172.16/12)
192.168.0.0  - 192.168.255.255 (192.168/16)
**************************************************

To take advantage of this feature you need to create an OpenDNS account at https://login.opendns.com. Once you have an account, login and click on the Settings tab. At the bottom you will see a link "Keep your network's IP up-to-date with our free software." It says Mac and Windows but there is a Linux client also.

When you click the link it start the download. Once it finishes, run the program. It will ask you to log into OpenDNS. The updater will show the public IP address of your router.


Once you do that, go back to OpenDNS in the browser and click settings again. You should see the public IP address of your router listed under Add a network. Click Add This Network.



If you look at the updater now, you will see your public IP address listed. Back on the OpenDNS page click down arrow next to --Select a Network-- and select your network. On the dialog that opens, click on security and put a check in the box next to "Block Internal IP addresses"




Now, if a script tries to use an RFC1918 address to spoof a domain it will get filtered by OpenDNS. This isn't a silver bullet but just one more layer of defense.

Testing the Filter

Steve Gibson of Gibson Research wrote a DNS benchmark way back in 2010 that is free and works well for benchmarking DNS performance. He also created some DNS addresses for testing rebinding. There is a link to the original 2010 podcast (episode 260) and the July 24, 2018 update in the reference section.

To test if your DNS server filters RFC1918 addresses, open a terminal or cmd window and enter the following:

nslookup net4.rebindtest.com
nslookup net10.rebindtest.com
nslookup net127.rebindtest.com
nslookup net172.rebindtest.com
nslookup net192.rebindtest.com

Below is the output before I configured the OpenDNS filter. Notice that the address returned for net172.rebindtest.com is 172.16.0.1 which would allow a malicious script to bypass the Same Origin Policy of the browser.


nslookup net172.rebindtest.com
Server:  10.208.0.1
Address: 10.208.0.1#53

Non-authoritative answer:
Name: net172.rebindtest.com
Address: 172.16.0.1
Name: net172.rebindtest.com
Address: ::ffff:172.16.0.1

Below is the output after the filter was enabled. Notice that the address returned is 146.112.61.109.


nslookup net172.rebindtest.com
Server:  127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: net172.rebindtest.com
Address: 146.112.61.109
Name: net172.rebindtest.com
Address: ::ffff:146.112.61.109

What is 146.112.61.109 you ask? You could use nslookup to find out but I wanted to show the dig (DNS Information Groper) command. It's built into Linux/Mac and you can install it on Windows. Here is a blog I wrote on installing dig DNS Information Groper for Windows

146.112.61.109 is the address OpenDNS uses for hit-block.opendns.com so the filter is working!


dig -x 146.112.61.109

; <<>> DiG 9.11.3-1ubuntu1.1-Ubuntu <<>> -x 146.112.61.109
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1651
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;109.61.112.146.in-addr.arpa.    IN    PTR

;; ANSWER SECTION:
109.61.112.146.in-addr.arpa. 3600 IN    PTR    hit-block.opendns.com.

;; Query time: 18 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Sat Aug 04 23:38:43 PDT 2018
;; MSG SIZE  rcvd: 91



References
Dear developers, beware of DNS Rebinding
How DNS Rebinding Attacks Impacts The Enterprise - youtube video
Half a billion smart devices vulnerable to decade-old DNS rebinding attacks 
DNS Rebinding Exposes Half a Billion Devices in the Enterprise
DNS Rebinding - Security Now podcast from 2010. Still applicable today.
Security Now 673 - Show notes
GRC DNS Benchmark - Windows only
DNS Information Groper for Windows - How to install dig on Windows
DD-WRT
Convert code samples into HTML for blogger

Friday, July 27, 2018

Learning Python 3

Updated September 10th, 2022

Here are some of the resources I used to start learning python. With my newly learned  Python skills and some Google-Fu I have been able to automate several network device tasks. You can find most of my python scripts at Hubbard on Networking GitHub Repo.

You do NOT need to know anything about python to use the scripts. This is just like you don't have to know how to program to run "show interface g1/0/1". You simply download the script and execute it.

I found this tweet that has a curated list of python resources:


Why You Should Learn Python

A lot of the tasks that network engineers perform are repetitive, mind-numbing, and error-prone. With a little python skill, you can automate these tasks and spend the time you save on more productive tasks, like learning more python.

As networking moves away from the CLI and into software-defined networking (SDN) you will need to have some dev skills. Cisco has a three-part video on how the network engineer's job is changing. You will need to register with Cisco Devnet to watch it but you should have a Devnet account if you going down this path:
Lesson 1: The Network Engineer of Old

If you have never used Python at all, this blog is the place to get started!

First, to install python, head over to the Python download page - Download Python
Follow the instructions to install python on your OS of choice.

Tools

There isn't much needed to use Python other than python itself but like most things, life is easier with some tools.

Integrated Development Environment (IDE)

IDEs are tools that allow you to write and debug code. Once you start writing scripts that are more than a few lines long you will want to use an IDE.

Thonny - Python IDE for beginners. A free Integrated Development Environment. This tool is great for learning. It lets you step through a script and see exactly what is happening.

MU - a simple Python editor for beginner programmers. This one is really nice. It's cross-platform and has support for Linux, Mac, Windows, and Raspian. What separates it from the other tools listed here is the support for small devices like Adafruit and Micro:bit. Here is a screenshot of MU starting up:



Microsoft Visual Studio Code - A free open source IDE from Microsoft. VSCode has turned into an amazing development environment and can be used for any size project. It has an integrated debugger that looks just like the Powershell ISE tool. It supports almost every programming language, not just python. The open-source version of PowerShell is fully supported which is great. I can use one tool and create Python and PowerShell scripts. 

There is a Visual Studio Marketplace with thousands of plugins, similar to the Chrome store or Firefox store. Extensions for the Visual Studio family of products

I have more detail on my macOS blog. Once you have VScode installed it works the same on Windows, Mac, and Linux so the blog is useful regardless of what OS you use. My VSCode install blog

Code Editor

You can definitely use VSCode as your editor. It has about every feature that you could want. But I have been using Sublime text since before VSCode came out and old habits die hard! If you are on Ubuntu 22.04 you can install Gnome Text Editor (I know, really, really stupid name) and it works great for quick editing.

Sublime text - Sublime is a text editor that is optimized for programming. It has a huge community around it and thousands of plugins. It's $75.00 but the license lets you run it on as many machines as you own. I bought it and installed it on my Linux, Windows, and Mac laptops. It has a tabbed interface like notepad++ which I like. You can also split the screen and open two files side by side. This is useful because you can have your script on one side and the data file on the other. 

Realpython.com sells a great tutorial that walks you through installing Sublime text optimized for Python. It includes video and text for Mac, Linux, and Windows. This tutorial is well worth the cost and has improved my productivity in Sublime.

In the next section, I discuss revision control with Git. Sublime text has several plugins that integrate Git into your workflow.  One I really like is Git Gutter. It puts a + sign next to lines that have changed, then you can revert easily if the change didn’t work.
Your Shortcut to a Professional Python Development Setup

Revision Control

Git - A revision control system. It's useful once you start writing scripts large enough to have bugs or future enhancements. Even if you don't program in python it's worth installing git because of github.com and gitlab.com.

Both of these sites have thousands of python tools. If you have git installed on your computer all you have to do to use them is "git clone <repository>". For example, to install the ARP sorting tool I wrote, you simply go to ARP-Sort and click the "Code" button. It will display the URL for the project. Click the copy button, type “git clone” and paste the URL you copied into the terminal, then press enter:

git clone https://github.com/rikosintie/ARP-Sort.git

This downloads the project and unzips it into the folder ARP-Sort.




Someone has created an online game for learning Git - An open source game about learning Git!
Cisco DevNet GitHub Repo - Great resources on setting up your laptop with Python, Git, Postman, etc.
Git and GitHub for Beginners - Crash Course - A really good video on git.
 



Videos from Udemy.com 

Udemy has inexpensive video training for Python, Linux, and a lot of other applications. These are on sale all the time for $9.99

  • Python Network Programming - Part 1: Build 7 Python Apps
  • Complete Python Bootcamp: Go from zero to hero in Python 3
  • 100 Days of Code - The Complete Python Pro Bootcamp for 2021

Videos on Youtube

I find that I learn better when using books and websites but sometimes it's nice to watch a video.

Real python did a blog on the best Youtube channels for python
The Ultimate List of Python YouTube Channels

Subscribe to HackerSploit on Youtube and there is a complete series on Python. He uses 2.7 but it is still a good tutorial. They are really basic but I find Alexis entertaining.
Python For Ethical Hacking - #1 - Introduction & Python Modules

Telusko Python for Beginners
Python Tutorial - Python for Beginners [Full Course] - Mosh 
Learn Python - Full Course for Beginners [Tutorial] Free Code Camp
Writing Pythonic Code - A Tweet thread by @Bascodes. He is worth following.

Corey Shafer Videos

I recommend that you subscribe to this channel. Here are four of his videos that I found very useful

Requests Tutorial - The requests library is used to pull data down from webservers
JSON Tutorial - Java Script Object Notation is a common data format used by APIs
Sorting Tutorial - Sorting routines for lists and dictionaries
List Comprehensions - Working with list comprehensions

e-books from Amazon  

I love the Kindle app on my laptop, phone and iPad. If I get stuck in a long line I just open it up and do some studying! These were all under $5. You can go to Amazon, set filter to Kindle store and enter "python programming free book" and find a lot of free books.

  • Learn Python in One Day and Learn It Well Python for Beginners with Hands-on Project The only book you need to start coding in Python immediately By Jamie Chan
  • Python Tips and Tricks: Learn the Best Tips and Tricks to Get The Most out of Python NOW! Jones, Daniel
  • The Fundamentals Of Python Programming: A Complete Beginners Guide To Python Mastery.
  • Python Programming Tips and Tricks: The Ultimate Cheat Sheet for Python Programming. 20+ Tips and Tricks to Make Your Life Easier and More Efficient
  • Automate the Boring Stuff with Python: Practical Programming for Total Beginners


Websites

There are so many python websites it would be impossible to list them all, here are a few I have found very useful:

Getting Started

General Tips

Jinja Templates

Network Engineering

Debugging

Web Development


Style guides

Python is a structured language that uses white space as part of the structure. But you can still create ugly code. The official Python style guide is part of the Python Enhancement Proposals (PEPs) which are documents that provide guidance and spell out best practices for how Python code should be organized, packaged, released, deprecated, and so on. PEP8 is a popular standard for styling Python code and is extensively used in the developer community. 

Python Virtual Environments

As you start learning and creating more python tools you will want to start using virtual environments. These two blogs will teach you what you need to know. There are a lot more blogs available on the Internet.

Podcasts

You probably won't learn much coding from a podcast but these are very interesting to listen to. I find a lot of good links in the show notes. For example, I learned about Thonny, MU, and Visual Studio Code from podcasts.
Talk Python To Me - A good podcast that covers a lot of topics. There will be one called "Teaching Python to network engineers" in August, 2018!
Podcast.__init__ - A podcast about Python and the people who make it great. Hosted by Tobias Macey.
Python Bytes - Python Bytes podcast delivers headlines directly to your earbuds. If you want to stay up on the Python developer news but don’t have time to scour Reddit, Twitter, and other news sources, just subscribe and you’ll get the best picks delivered weekly.

Keywords In Python

There are 33 keywords that should never be used as a variable, function name, class, object, or as any other identifiers in your programs.

 false
 True
 finally
 class
 for
 continue
 none
 return
 lambda
 try
 is
 def
 from
 nonlocal
 while
 and
 not
 global
 del
 with
 as
 elif
 if
 or
yield
 break
 import
 except
 pass
 assert
 else
 raise
 in

Libraries

Pint - Pint is a Python package to define, operate and manipulate physical quantities

Wednesday, May 2, 2018

The tools on my Ubuntu 18.04 laptop

Update September 15, 2018: Ubuntu 18.04 has support for Snaps and Flatpaks. What are these? A new method for installing applications where everythimg is bundled into the package so you don’t have to deal with dependencies. I haven’t used Flatpaks yet but Snaps are great!  You go to https://snapcraft.io/store and find the app you want. Then you click a button, copy the link and paste it into a terminal.

So far I have used snaps for:
  • Brave - A new browser that is much more privacy respecting than Chrome
  • Corebird - A nice Twitter app
  • fKill - Fabulously kill processes
  • GHex - A graphical Hex editor from Canonical 
  • Hiri - A Microsoft Exchange/Office 365
  • Hollywood - Fill your console with Hollywood melodrama technobabble 
  • KeepasXC - My preferred Keepas client
  • Mailspring - A lretty good mail client for IMAP servers. I’m using it for gmail  
  • Mumble - An open source voice chat client that is popular in Linux circles
  • Notepadqq - A notepad++ like editor for Linux
  • Powershell - Microsofts Opensource version of Powershel
  • Skype
  • Slack- The official Slack client
  • Speedy Duplicate Finder - Blazingly fast duplicate finder for Windows, Mac and Linux
  • Telegram - Official desktop client 
  • Termius - A cross platform SSH client. I use it on IOS when I just need to make a few changes and don’t want to carry a laptop.
******************************************************************************

After the 2016 Southern California Linux Expo (SCALE15) I purchased a System76 Gazelle laptop. System76 sells laptop and desktops designed for Linux so there are no issues with drivers or hardware incompatibility.

I was thinking about buying a new Macbook to replace my aging Macbook Air but Apple dropped all ports except USB-C and that was just unacceptable for my work. Unlike the Macbook, the Gazelle has VGA, HDMI, three USB3-A, one USB2-A, Ethernet, and a full-size SD card slot.

Plus, System76 laptops are as open as you can get!  I purchased it with 8GB of RAM and a 250GB spinning disk. Not long after, I decided to go to 16GB of RAM and an m.2 NVME drive (The new Gazelle supports 32GB of RAM!). How hard was it? Removed a few screws, popped off the cover, added one 8GB stick, plugged in the NVME and reinstalled the cover.

I contacted System76 beforehand and they told me to buy the drive and RAM from Amazon because it would be less expensive than buying from them!!! That is great customer service.

I think the build quality is pretty good, I recently knocked it off the top of a 6-foot ladder. It was open at the time, the floor was concrete with a thin carpet. I didn't want to look at it. I was sure the LCD would be broken and there would be other damage. But it's been a week and I haven't noticed any damage, it's still working.

So, how hard was it switching from Windows/MAC to Linux?  Not much of a challenge actually. The Ubuntu 16.04 desktop uses the Super key (Windows key if you are on a Dell/Lenovo that you converted to Linux) to open up the search tool just like Windows. The file explorer, Nautilus, is very similar to Explorer in Window or finder in OSX.

What I am loving about Linux is no tracking of what I am doing, lots of free open source tools and updates happen almost daily. Updates seldom require a reboot and a typical update takes a few minutes, and if you do need to reboot there isn't a long delay with a message not to power off while the system is reconfigured.

I updated to Ubuntu 18.04 the week it came out. Normally I would do a nuke and pave because it’s so easy in Ubuntu.  Most of the configuration data for the apps is in your home folder so you back it up, nuke/pave and put the home folder back.  But I hadn’t been running 17.04 very long and just did an in place upgrade. It went off without any drama and now I’m using the Gnome desktop.

Ubuntu has decided to drop the Unity desktop and standardize on Gnome. I am loving 18.04, the Gnome desktop has been easy to transition to and 18.04 lets you setup the live update feature so kernel updates are automatic and you can postpone a reboot if one is needed. I installed the following Gnome extensions to replace functionality lost in the switch to Gnome:

Learning Linux
There are so many Linux sites with free tutorials and a lot of Kindle books for $.99 to $2.99. I had no problem finding answers to questions I had. Also, it seems that the old RTFM response om forums has gone away, I didn't get flamed anywhere! I purchased the Linux Professional Institute’s cert guide and took the Linux Essentials test. It’s not a system engineer level but I’m comfortable with the OS now.

I found the following podcasts for Linux and they have really helped my transition:
  • Linux Unplugged
  • The Ask Noah show
  • Ubuntu podcast
  • Linux in the ham shack
  • Late Night Linux
  • Linux Action News
  • Destination Linux

The network stack on Linux is much better than Windows. You can create multiple network profiles and use them as needed. For example, I have a profile named Eth-DHCP that uses DHCP and one called Eth-con1 that uses static addressing.

It's easy to change the static settings and then just click it to use it. Another small advantage is that the boxes for IP, Mask, Gateway allow pasting. You don't have to type one octet, tab, next octet, tab, etc. And the gateway can be /24 or whatever you need.



Not earth-shattering, but if you change networks a lot it's more convenient. Plus, I haven't had issues changing networks many times per day. The stack just seems to work. It is much easier to query networks status than it is on Windows.

The tools iwlist, nm-tool, nmcli and ip let you quickly see what channels your wifi card supports, what SSIDs are available, what interfaces are up, the IP addresses, etc. This blog isn't going to go into how to use them but I do have a blog on some uses for the tools here.

Finally, you can do monitor mode captures without a special adapter like a Riverbed AirPcap. If you do a lot of wireless troubleshooting this is a big advantage.

The Tools

The great thing about Linux is all of the built-in and free open source networking tools.

Tools from the Ubuntu software store

Network Tools (gnome)- Graphical tool for ping, ifconfig, netstat, etc.

simplenote - free tool similar to Evernote.

PERL auto connect - SSH/RDP client. Install PAC-VS from the Ubuntu software store. This tool is really nice, you have RDP, SSH, Telnet all in one interface.

KeepassXC - A cross platform password manager. Supports Yubikey and other forms of two factor authentication.

unetbootin - Tool for creating live USB drives

FSLint - is a utility to find and clean various forms of lint on a filesystem, especially duplicate files and broken symlinks.

Disk Usage Analyzer - Similar to WinDirStat on windows. It's built into Ubuntu.

brasero - CD/DVD burner. I know, but my Gazelle has a DVD burner and occasionally I need to burn an archive.

cheese webcam - I use this with my Ebay.com endoscope for looking into tight spaces. An interesting story, I bought the scope and it said Windows only. I figured I would use it with my Windows VM.

When it came in it had a small CD with all Chinese on it. I used ClamAV to scan it and ClamAV found a virus. I plugged the Endoscope into the Gazelle and did an lsusb (list USB). It listed the endoscope and the hardware manufacturer. I did a quick Google and found an AskUbuntu.com post saying it is supported by the built-in Cheese webcam! No windows needed.

FBReader - eBook reader.

bleachbit - You too can be like Hillary.

Meld - A cross-platform file compare utility. I like it a lot better than notepad++ with the compare plugin

ClamTK - Graphical frontend for ClamAV.

shutter - A screenshot tool with rectangles, lines, etc. I have used the Windows Snipping tool for years and really like it. Shutter is even better. It can be used like MS Paint.net

etcher - A live usb tool. This tool is highly recommended in the Raspberry Pi and SoC communities.

smemstat - snap package memory tool.

Gnome Hex Editor - Inspect and edit binary files.

Remmina - Remote Desktop tool built into Ubuntu. Works great.

Tools installed from the terminal

Arpscan - A great tool from Roy Hill. Allows you to find devices that don’t respond to ping and much more. I wrote a Python wrapper for Arpscan. It’s on my github at https://github.com/rikosintie

Gnome Sushi file preview - Allows you to select a file in the file manager (Nautilus) and just press the spacebar to preview it. Install instructions are here - How to Geek

Docker - https://www.docker.com/community-edition

Clam AV - I use this to scan windows files before inserting a flash drive or running something on windows. Install from the synaptic package manager.

Python 2.7 - A fresh install of 18.04 doesn’t include Python 2.7, 3.x is now the default. 

Microsoft Visual Studio Code - A great open source IDE. Supports Python, Powershell and just about every other programming language.

Microsoft Powershell Core - For those times when you need to pop a windows box but don’t want to fire up a Windows VM! Microsoft released PS Core as an open source project. I have to admit, it’s pretty convenient having Powershell native on Linux. 
https://blogs.msdn.microsoft.com/powershell/2018/01/10/powershell-core-6-0-generally-available-ga-and-supported/

FoxtrotGPS - I use this with my external USB GPS when I run wigle.net or want to track my trip on my laptop.

Sublime Text - $75.00 text editor. Has great support for Python. The site realpython.com sells a packege

sudo apt install minicom - Serial terminal emulator for use with USB to Serial adapters. It runs in the Linux terminal so it's very convenient.

sudo apt install smem - memory display tool

sudo apt-get install xsltproc - xml to html converter

sudo apt install traceroute - This is a legacy tool but I still like it.

sudo apt install gcc python-dev python-pip - Python installer and c compiler

sudo apt-get install xclip - copies files to the clipboard Ex. xclip -sel clip < ~/.ssh/id_rsa.pub

Hex editor - https://github.com/krpors/hx

Solaar - sudo apt install solaar - A tool for Logitech unify receivers.
https://github.com/pwr/Solaar

sudo apt install snmp - Allows you to run all the snmpget, snmpwalk, etc. tools

sudo apt install linssid - A tool similar to inSSIDer

sudo apt install aircrack-ng - Put wireless card into monitor mode.

sudo apt install apt-show-versions

https://www.insynchq.com/ - Google Drive sync tool.

Progress - Linux tool to show progress for cp, mv, dd, ... (formerly known as cv)
https://github.com/Xfennec/progress

sudo apt install apt-file - a software package that indexes the contents of packages in your available repositories and allows you to search for a particular file among all available packages.

sudo apt install cryptsetup - manage plain dm-crypt and LUKS encrypted volumes

sudo apt install procinfo - lsdev is installed with this.

sudo apt-get install network-manager-vpnc-gnome (Cisco VPN)

sudo apt install htop - an ncurses replacement for top. Has sorting capability.

sudo apt-get install joe - a super fast cli text editor. It can open and work with huge files. I use it for my password cracking dictionary files.

**************************************************
 IP Address calculator - Simple terminal program. Works great!
sudo apt install sipcalc

Example:
sipcalc 10.34.208.0/20
-[ipv4 : 10.34.208.0/20] - 0

[CIDR]
Host address - 10.34.208.0
Host address (decimal) - 170053632
Host address (hex) - A22D000
Network address - 10.34.208.0
Network mask - 255.255.240.0
Network mask (bits) - 20
Network mask (hex) - FFFFF000
Broadcast address - 10.34.223.255
Cisco wildcard - 0.0.15.255
Addresses in network - 4096
Network range - 10.34.208.0 - 10.34.223.255
Usable range - 10.34.208.1 - 10.34.223.254

*************************************************
Tool to locate files in the file system.
sudo apt install mlocate

run sudo updatedb after installing locate. It sets up the database and indexes all package files.

**************************************************
Moreutils - A small package with several useful utilities. See this page for more.
https://rentes.github.io/unix/utilities/2015/07/27/moreutils-package/#installing
sudo apt install moreutils

**************************************************
INXI is a great system information tool
https://github.com/smxi/inxi
sudo apt install inxi - inxi is a full featured CLI system information tool.
inxi -F -x -c13 -- all output with extra data
inxi -F -xx -c13 -- all output with extra, extra data
inxi -B -- Battery info
inxi -c13 -- use black output
inxi -C -- CPU information
inxi -D -- hard drive info
inxi -f -- all cpu flags
inxi -G -- graphics card info
inxi -i -- network interface info
inxi -I -- processes, uptime, memory, inxi version,
sudo inxi -m -c13 - list memory

********************************************************
LLDP client for Linux.
sudo apt-get install lldpd
sudo service lldpd restart
https://fnord.no/2016/04/28/lldp-on-linux/
Usage
lldpcli show neighbors

********************************************************
ssh-audit - A simple tool to pull down the crypto ciphers that an SSH server offers.
https://github.com/arthepsy/ssh-audit
usage: ssh-audit.py [-1246pbnvl]

-1, --ssh1 force ssh version 1 only
-2, --ssh2 force ssh version 2 only
-4, --ipv4 enable IPv4 (order of precedence)
-6, --ipv6 enable IPv6 (order of precedence)
-p, --port= port to connect
-b, --batch batch output
-n, --no-colors disable colors
-v, --verbose verbose output
-l, --level= minimum output level (info|warn|fail)

./ssh-audit.py 192.168.10.181

********************************************************
I-NEX - I-Nex is an application that gathers information for hardware components available on your system and displays it using an user interface similar to the popular Windows tool CPU-Z.

http://i-nex.linux.pl/install/

sudo add-apt-repository ppa:i-nex-development-team/daily
sudo add-apt-repository ppa:gambas-team/gambas3
sudo apt-get update
sudo apt-get install i-nex

********************************************************
sudo apt install sendip
SendIP is a command-line tool to send arbitrary IP packets.
http://www.ubuntugeek.com/sendip-tool-to-send-arbitrary-ip-packets.html

********************************************************
nmap scan report tool
https://blog.techorganic.com/2012/09/15/parsing-nmaps-output/
https://github.com/superkojiman/scanreport
https://github.com/superkojiman/onetwopunch

Example
cat targets.txt
192.168.81.171
192.168.81.182
192.168.81.143
192.168.81.119
192.168.81.190

nmap -sV -oG scan.txt -iL targets.txt
remove # from report
grep -v ^# scan.txt > report.txt

scanreport.sh -f report.txt
Host: 192.168.81.171 ()
22 open tcp ssh OpenSSH 5.3p1 Debian 3ubuntu4 (protocol 2.0)
80 open tcp http Apache httpd 2.2.14 ((Ubuntu))

Host: 192.168.81.182 ()
22 open tcp ssh OpenSSH 5.3p1 Debian 3ubuntu4 (protocol 2.0)
80 open tcp http Apache httpd 2.2.14 ((Ubuntu))
443 open tcp ssl|http Apache httpd 2.2.14 ((Ubuntu))

*************************************************************
Angry IP
http://angryip.org/download/#linux
dpkg -s ipscan_3.5.2_amd64.deb

**************************************************************
An extremely handy tool :: Incremental history searching
In terminal enter:
gedit ~/.inputrc
Then copy paste and save:
"\e[A": history-search-backward
"\e[B": history-search-forward
"\e[C": forward-char
"\e[D": backward-char

From now on, and many agree this is the most useful terminal tool, it saves you a
lot of writing/memorizing...

All you need to do to find a previous command is to enter say the first two or three letters and upward arrow will take you there quickly:

Say I want:

for f in *.mid ; do timidity "$f"; done
All I need to do is enter:

fo
And hit upward arrow command will soon appear.

*************************************************************************
Wireshark
On Ubuntu
sudo apt-get install wireshark libcap2-bin
sudo groupadd wireshark
sudo usermod -a -G wireshark $USER
sudo chgrp wireshark /usr/bin/dumpcap
sudo chmod 755 /usr/bin/dumpcap
sudo setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap
(step 6 to make the interfaces visible)

*************************************************************************
Go language
sudo apt install golang-go

gotop - Another tool like htop
https://github.com/cjbassi/gotop

/home/mhubbard/bin/gotop
└─> $ ./gotop
************************************************************************
sudo apt-get install libncurses5-dev libncursesw5-dev
sudo apt install ncdu - an ncurses disk file display tool

************************************************************************
nmap 7.70
https://nmap.org/download.html

bzip2 -cd nmap-7.70.tar.bz2 | tar xvf -
cd nmap-7.70
./configure
make
sudo passwd root
Enter a strong password for the root user
sudo passwd -u root
to unlock the account
su root
make install
exit - logout as root

disable root account
sudo passwd -l root

Install HD Moore's banner-plus script
https://github.com/hdm/scan-tools/blob/master/nse/banner-plus.nse

*********************************************************


References
8 Ways to Maintain a Clean, Lean Ubuntu Machine
How To Make Gedit More Programmer Friendly