Upgrade Cisco 2960 firmware using SCP instead of TFTP

I recently needed to upgrade a couple dozen Cisco 2960-x switches located at several different sites. Obviously, I didn't want to drive to each site and insert flash drives to do the upgrade but I also didn't want to use TFTP and risk a failed transfer causing a reboot issue.

The solution was to use Secure Copy or SCP. Unlike TFTP, SCP is TCP based so you get the advantages of a TCP - Reliability, Flow Control, error checking and error recovery. SCP adds encryption. While encryption isn't so important for a firmware transfer it is important when transferring configurations so that the passwords aren't sent in plain text.

The drawback to SCP is that you need to set up an SCP server and create a username/password combination. For Windows, the Solarwinds SFTP/SCP server is free and works well. It supports secure FTP and secure copy protocols. You can download the server here. While free, it does require that you enter a valid email address. Once you have it installed, click on configure and create a new user/password.

On Ubuntu, you can use the OpenSSH server from the Ubuntu repository. For a How To on configuring the server click here.

Once you have the SCP server installed, download the tar file from Cisco and save it to the SCP servers root directory. For the Solarwinds server, the default is c:\sftp-root. For Ubuntu, you will specify the path after the IP address.

I recommend you check the hash of the firmware file after you download it from Cisco. To do that, simply click on the file name on the Cisco download page. You will see the MD5 hash on the popup, right click, copy and paste it into an editor.

On windows
From a cmd window run "certutil -hashfile <filename> -MD5.

Linux
From a terminal windows run md5sum <filename>.

Compare the result to the hash saved in the text file. If the hashes match, proceed to the next step.

On the switch

You will need to set up SCP first. See my blog Using Secure Copy (SCP) to Transfer Files for a tutorial on setting up SCP. 

Once that is complete, log in and go to enable mode.

Enter the following (substitute the appropriate IP address, filename, and credentials). The switch puts the password: prompt on the screen but the password will be passed automatically. Since I used the /reload switch no further intervention is required - the firmware will be upgraded and the switch will reload. You can kick off several switches in parallel as the load on the SCP server isn't too heavy. Using an Ubuntu desktop VM with the defaults set in ESXi I have upgraded 10 switches at once with no problems.

archive download-sw /overwrite /reload scp://mhubbard:hU9*b2Sis@192.168.10.221/c2960x-universalk9-tar.152-2.E7.tar
Password:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Password:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
examining image...
extracting info (109 bytes)
extracting c2960x-universalk9-mz.152-2.E7/info (786 bytes)
extracting info (109 bytes)

Stacking Version Number: 1.56

System Type:             0x00000000
  Ios Image File Size:   0x0144DA00
  Total Image File Size: 0x019ACA00
  Minimum Dram required: 0x08000000
  Image Suffix:          universalk9-152-2.E7
  Image Directory:       c2960x-universalk9-mz.152-2.E7
  Image Name:            c2960x-universalk9-mz.152-2.E7.bin
  Image Feature:         IP|LAYER_2|SSH|3DES|MIN_DRAM_MEG=128
  FRU Module Version:    No FRU Version Specified

References

Using Secure Copy (SCP) to Transfer Files
2960-X Switch System Management Command Reference, Cisco IOS Release 15.0(2)EX
Configuring Secure Shell (SSH)
Security Configuration Guide
10 SCP Commands to Transfer Files/Folders in Linux
Example syntax for Secure Copy (scp)

Removing provisioned interfaces on the 6880-x switch

I recently had to configure a 6880-x that had three additional C6880-X-LE-16P10G modules.  After I finished I realized that the customer wanted the cards inserted into slots 2-4 and I had put them in 1-3.

I moved the module in slot 1 to slot 4 and waited for it to power up. When the cli message said the module was online I configured it and did a write mem. Then I did a "sh run | b interface" and found that the configuration for t1/1/1 was still in the configuration.

The solution is to enter module provision mode and then use the "no" form of the slot command. The first step is to find out what is in the slot.

Here is the command line:
show module provision switch 1 slot 1
Switch number:     1    Module:  1    Status: PROVISIONED
     Slot Type: (364)
  Port type: ( 60)  10GBASE Number: 16
  Port type: (113)  40GBASE Number: 4

The important information is the slot type (364) and port type (60)

Notice the Status: PROVISIONED. 

For a slot with a module inserted the output looks like this:

show module provision switch 1 slot 5

Switch number:     1    Module:  5    Status: ONLINE

    Slot Type: (363)  6880-X-LE 16P SFP+ Multi-Rate

    Port type: ( 60)  10GBASE Number: 16

    Port type: (113)  40GBASE Number: 4

Using the information from the "show module" command you can de-provision using the following:

(config)#module provision switch 1
(config-prov-switch)#no slot 1 slot-type 364 port-type 60 number 16

An interesting command I found while Googling for this:
show asic-version switch 1 slot 5

Module in switch 1, slot 5 has 9 type(s) of ASICs
           ASIC Name      Count      Version
         METRO_ARGOS          4      (4.0)
       METRO_KRYPTON          4      (4.0)                   
              MALFRA          2      (1.4)
                 SSA          4      (9.0)
        SANTA_MONICA          1      (1.0)
              RADIAN          2      (1.3)
          I_GIGATRON          2      (0.24)
     EGRESS_GIGATRON          2      (0.40)
                  G1          2      (0.25)

References

Online Insertion and Removal (OIR) of Modules in Cisco Catalyst Switches
BUG Fix -Fix show module provision switch display and card insertion event issue. (requires CCO)
VSS Command Refeerence - module provision

Upgrading a Cisco Nexus 7000 to 6.2(16) with one Supervisor.

The Cisco Nexus 7000 series switches have several features to make upgrading firmware safe and reliable, especially if you don't have a redundant supervisor. The release notes for each software version has a table that clearly shows what previous software versions can be updated to the chosen version.

The Nexus 7000 are Linux based switches and they use kickstart to automate the installation. This means you will be copying a new n7000-s2-kickstart.x.x.xx.bin file along with the n7000-s2-dk9.x.x.xx.bin firmware file. Note that the kickstart and firmware file must be the same version.

Before the upgrade

Make a Backup of the running configuration of all VDCs and key files

The Nexus switch has a couple USB slots that mount as usb1: and USB2:. I used a 1GB flash drive formatted with the FAT file system to back up the current configuration. While not a requirement for an upgrade I like to have a backup in case anything goes wrong. The Nexus can read larger USB drives but I keep several 1GB drives handy for this type of work.

The command that backs up the running configuration of all Virtual Device Contexts (VDC) at once to the USB stick is:

• copy running-config usb1:MY-N7K.txt vdc-all

Just to be safe I copied the vlan.dat and license files to the USB drive. The license files use a .lic file extension so they are easy to identify. Once the configurations were backed up I put the USB stick into my laptop and verified that the backup was good.

I also looked at the boot settings before and after the upgrade
show boot
Current Boot Variables:

sup-1
kickstart variable = bootflash:/n7000-s2-kickstart.6.2.8a.bin
system variable = bootflash:/n7000-s2-dk9-npe.6.2.8a.bin
No module boot variable set

Download the new kickstart and Nexus software files 
You will need a CCO account and a current contract to get the software. Once the software is downloaded verify that the files are valid using the MD5 hash on the download page. If you are not familiar with verifying hashes it is very easy.

On Linux
mhubbard@1S1K-SYS76:/media/mhubbard/783E-8CFE$ md5sum n7000-s2-dk9.6.2.16.bin
f6ad2c2ea750fb15fc455d670277340c  n7000-s2-dk9.6.2.16.bin

On Windows 7
certutil -hashfile C:\tftp-root\n7000-s2-dk9.6.2.16.bin md5

or

with powershell 4 or above and the community extensions installed.
$PSVersionTable.PSVersion
PS C:\Users\mhubbard> get-hash C:\tftp-root\n7000-s2-dk9.6.2.16.bin -algorithm MD5

If you need more information on how to verify hashes on Windows you can see my blog on Using iPerf3 to verify Link Quality. Scroll down to "Installing iPerf3 on Windows".

Copy the new files to from the USB to bootflash

• copy usb1: bootflash:

n7000-s2-dk9.6.2.16.bin

• copy usb1: bootflash:

n7000-s2-kickstart.6.2.16.bin

Run the installer

install all kickstart bootflash:n7000-s2-kickstart.6.2.16.bin system bootflash:n7000-s2-dk9.6.2.16.bin parallel

The parallel keyword allows all modules to be upgraded in parallel to save time.

A lot of feed back is given during the upgrade so you can see if everything is proceeding correctly.

***********************************************************

Installer will perform compatibility check first. Please wait.

Verifying image bootflash:/n7000-s2-kickstart.6.2.16.bin for boot variable "kickstart".
[####################] 100% -- SUCCESS

Verifying image bootflash:/n7000-s2-dk9-npe.6.2.16.bin for boot variable "system".
[####################] 100% -- SUCCESS

Verifying image type.
[####################] 100% -- SUCCESS

Extracting "system" version from image bootflash:/n7000-s2-dk9-npe.6.2.16.bin.
[####################] 100% -- SUCCESS

Extracting "kickstart" version from image bootflash:/n7000-s2-kickstart.6.2.16.bin.
[####################] 100% -- SUCCESS

Extracting "bios" version from image bootflash:/n7000-s2-dk9-npe.6.2.16.bin.
[####################] 100% -- SUCCESS

Extracting "lc1n7k" version from image bootflash:/n7000-s2-dk9-npe.6.2.16.bin.
[####################] 100% -- SUCCESS

Extracting "fexth" version from image bootflash:/n7000-s2-dk9-npe.6.2.16.bin.
[####################] 100% -- SUCCESS

Performing module support checks.
[####################] 100% -- SUCCESS

Notifying services about system upgrade.
[####################] 100% -- SUCCESS



Compatibility check is done:
Module  bootable          Impact  Install-type  Reason
------  --------  --------------  ------------  ------
     1       yes      disruptive         reset  Reset due to single supervisor
     3       yes      disruptive         reset  Reset due to single supervisor
     4       yes      disruptive         reset  Reset due to single supervisor
   101       yes      disruptive         reset  Reset due to single supervisor
   102       yes      disruptive         reset  Reset due to single supervisor
   103       yes      disruptive         reset  Reset due to single supervisor



Images will be upgraded according to following table:
Module       Image                  Running-Version(pri:alt)           New-Version  Upg-Required
------  ----------  ----------------------------------------  --------------------  ------------
     1      system                                   6.2(8a)               6.2(16)           yes
     1   kickstart                                   6.2(8a)               6.2(16)           yes
     1        bios   v2.12.0(05/29/2013):v2.12.0(05/29/2013)   v2.12.0(05/29/2013)            no
     3      lc1n7k                                   6.2(8a)               6.2(16)           yes
     3        bios       v2.0.32(12/16/13):v2.0.32(12/16/13)     v2.0.32(12/16/13)            no
     4      lc1n7k                                   6.2(8a)               6.2(16)           yes
     4        bios       v2.0.32(12/16/13):v2.0.32(12/16/13)     v2.0.32(12/16/13)            no
   101       fexth                                   6.2(8a)               6.2(16)           yes
   102       fexth                                   6.2(8a)               6.2(16)           yes
   103       fexth                                   6.2(8a)               6.2(16)           yes


Additional info for this installation:
--------------------------------------

Service "lacp" in vdc 1: LACP: Upgrade will be disruptive as 0 switch ports and 10 fex ports are not upgrade ready!!
      Issue the "show lacp issu-impact" cli for more details.



Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)?  [n] y

Install is in progress, please wait.

Performing runtime checks.
[####################] 100% -- SUCCESS

Setting boot variables.
[####################] 100% -- SUCCESS

Performing configuration copy.
[####################] 100% -- SUCCESS

Module 1:  Upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS

Module 3:  Upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS

Module 4:  Upgrading bios/loader/bootrom.
Warning: please do not remove or power off the module at this time.
[####################] 100% -- SUCCESS

Finishing the upgrade, switch will reboot in 10 seconds.

*****************************************************************

Upgrade any Fabric Extenders that connected

Once the switch reloaded it will notice that the Fabric extenders need to be upgraded and will automatically start the process. You can view the progress using:

show fex detail
FEX: 102 Description: < SAN Hosts - Backup Rack >   state: Image Download
  FEX version: 6.2(8a) [Switch version: 6.2(16)]

Once the Fabric Extenders have upgraded they will automatically reload.

You can use the following command to see a log of all upgrade steps:

show install all status

References

Cisco Nexus 7000 Series NX-OS Release Notes, Release 6.2

Download Software Nexus 7000 4-Slot Switch

Boot System and Boot kickstart commands

Cisco Nexus 7000 Series NX-OS Software Upgrade and Downgrade Guide, Release 6.x

N7K-SUP2/E: eUSB Flash Failure or Unable to Save Configuration

I recently had a customer that installed a new firewall and made several changes to their Cisco Nexus 7004 core switch. Once all the configuration changes were made they issued a wr to save the changes (wr is an alias. See the references for details). But they didn't get the usual complete message. Instead, they received "Configuration update aborted: request was aborted".

You never want to see that but especially not on a core switch that is responsible for routing 72 sites! I opened a TAC case with Cisco and was told to run these four  commands and send the output back to the TAC engineer:

show module
show version
show system internal raid (Hidden Command)
slot x show system internal raid ( x = standby sup )

Note: you can use slot x show system internal raid and replace x with the slot the supervisor is in regardless if it's the standby. For example, slot 1 show system internal raid gives the same output as show system internal raid with only one supervisor.

The key output was from the command:
MY-MDF-DC1# show system internal raid
Current RAID status info:
RAID data from CMOS = 0xa5 0xc3 < ----------- Both primary and alternate failed.

and from the show module command:

Mod  Online Diag Status
---  ------------------
1    Pass
3    Pass

4    Fail

TAC said this meant that both eUSB flash memory cards were failed. Since we didn't have a redundant supervisor the only way to recover was to reboot the switch. The "Failed" eUSB memory cards aren't failed as in they don't work but that they are full. The References section below has a link to the actual bug report (CSCus22805). It explains in detail how to recover if only one eUSB is failed or in you have a redundant supervisor.

The Problem

The customer had made several configuration changes and wasn't able to save the running configuration. Obviously, all changes would be lost during the reload.

The Solution

The Nexus switch has a couple USB slots and a command that backs up the running configuration of all Virtual Device Contexts (VDC) up to the USB stick:

copy running-config usb1:MY-N7K.txt vdc-all

Once the configurations were backed up I put the USB stick into my laptop and verified that the backup was good.

Since this switch has so many routes and some of the changes that were made were routing related I wanted to make sure all routes came up after the reboot. I saved the output from:

show ip route summary
Number of routes per mask-length:
  /0 : 1       /8 : 2       /16: 82      /23: 2       /24: 113
  /25: 2       /26: 1       /27: 5       /28: 1       /29: 2
  /30: 1       /32: 788

to a text file so that I could compare after the reboot.

I also saved the output from
show interface status | i connected
show cdp ne det | i Dev

These two commands gave me a quick summary of the interfaces that were up and the neighboring switches.

Finally, I copied the all the license files and vlan.dat file to a tftp server.

The Reload

The maintenance window arrived and I had a plan in place. All that was left now was to reload. I consoled in and entered reload. The switch came back up and I reran the four commands. Show module was all "pass" and the RAID report was 0xa5 0xf0. The 0xf0 meaning the eUSB memory was working correctly.

The Clean Up

I reran the "show ip route summary" command and was missing some routes. In addition, some interface configurations were missing. This was to be expected since the changes were lost.

I ran "copy running-config usb1:MY-N7K1.txt vdc-all" and inserted the USB stick into my laptop. I use a great file diff program called MELD. I put a link to it in the references. I opened both files in MELD and it instantly highlighted the differences between the current running configuration and the backup I made before the reboot. It was a simple task to add the changes back and all routes came up.

Comparing two files in MELD

References

N7K-SUP2/E: eUSB Flash Failure or Unable to Save Configuration CSCus22805
Meld - Open source file diff tool
Write Command On Nexus Switches - How to create an alias for copy run start

Cisco 6800 Instant Access (IA) switch trunk ports

The 6800 Instant access switch allows you to extend the core switch into access closets throughout the campus. I wrote a blog on configuring the 6880-x to work with the 6800IA here.

From the 6800IA Cisco Catalyst Instant Access FAQ:

Q. What is Cisco Catalyst ® Instant Access?
A. Instant Access is a solution that uses Cisco IOS ® Software to connect Cisco ® Catalyst 6800ia access switches to Cisco Catalyst 6500 or 6800 Series core switches. Once connected, the entire configuration works as a single extended switch with a single management domain. The solution is intended to simplify your campus network operations and management.

What does that mean? It means you can connect the 6800IA to a 6880 or 6500 series core switch and manage it from the core switch. But the 6800IA isn't a standalone switch, it's a Fabric Extender (FEX) and has some limitations that a standalone switch doesn't.

Again, from the FAQ:
Q. Why is the default configuration of Instant Access client host port configuration “switch trunk allowed vlan 1” and not “all”?

A. Each Instant Access host port can be configured in access or trunk mode (default is dynamic). If in trunk mode, there is a constraint as to how many VLANs can be trunked on each port.
Note: No more than 1,000 VLANs can be associated with a single FEX ID, divided by the number of Instant Access trunk ports.

To make sure that this constraint is followed, implementation requires specifying explicitly which VLANs will be trunked. We recommend no more than 20 VLANs per Instant Access trunk port 
(up to the total of 1000 per FEX), to limit the amount of BPDU processing.

On a standalone Cisco switch, by default, a trunk port passes all VLANs. The 6800IA by default only passes VLAN1!  Here is an example of a trunk port configured to work with an access point that needs vlans 1, 4, 201, 202, 203 and 204. If you don't explicitly allow a vlan it isn't passed (other than vlan1 of course).

It is very easy to forget this if you are replacing older switches with IAs and basically copying the configs! Obviously, if the port is only passing vlan1 and you need 4, 201, 202, 203 and 204 the SSIDs won't work correctly.

interface GigabitEthernet101/1/0/1
 description < Access Point >
 switchport
 switchport trunk allowed vlan 1,4,201,202,203,204
 switchport mode trunk
 logging event trunk-status

Enabling TLS 1.1/1.2 for RDP in Microsoft Server 2008R2/Windows 7 SP1

I updated the nmap3.py Python script to include RDP on option 1 "ssl-cert,ssl-enum-ciphers". You can find nmap3.py on my Github if you don't have it already.

I ran the script against my Windows 7/Server 2008R2 VMs and found that they were offering up RC4 and MD5 for RDP!

Example with defaults

nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995,3389 192.168.10.135

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995,3389 192.168.10.135

Starting Nmap 7.50 ( https://nmap.org ) at 2017-07-08 10:28 PDT
Nmap scan report for 192.168.10.135
Host is up (0.0052s latency).

PORT     STATE  SERVICE

443/tcp  closed https

465/tcp  closed smtps

993/tcp  closed imaps

995/tcp  closed pop3s

3389/tcp open   ms-wbt-server

| ssl-cert: Subject: commonName=WIN-L6HBT78G89G.pu.pri

| Issuer: commonName=WIN-L6HBT78G89G.pu.pri

| Public Key type: rsa

| Public Key bits: 2048

| Signature Algorithm: sha1WithRSAEncryption

| Not valid before: 2017-05-27T19:55:04

| Not valid after:  2017-11-26T19:55:04

| MD5:   6429 05ea b708 ffa1 fb56 cd62 8a7e 8acb

|_SHA-1: c160 c6b5 8d2c 0702 b86b fa8b c717 d25c 44a1 89df

| ssl-enum-ciphers: 

|   TLSv1.0: 

|     ciphers: 

|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp256r1) - A

|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp256r1) - A

|       TLS_DHE_RSA_WITH_AES_256_CBC_SHA (dh 1024) - A

|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 1024) - A

|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A

|       TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C

|       TLS_RSA_WITH_RC4_128_SHA (rsa 2048) - C

|       TLS_RSA_WITH_RC4_128_MD5 (rsa 2048) - C

|     compressors: 

|       NULL

|     cipher preference: server

|     warnings: 

|       64-bit block cipher 3DES vulnerable to SWEET32 attack

|       Broken cipher RC4 is deprecated by RFC 7465

|       Ciphersuite uses MD5 for message integrity

|       Key exchange (dh 1024) of lower strength than certificate key

|       Weak certificate signature: SHA1

|_  least strength: C

Nmap done: 1 IP address (1 host up) scanned in 5.74 seconds

Enabling FIPS

Then I enabled FIPS level security per this MS document Tip: Secure RDS (Remote Desktop Services) Connections with SSL. This is worse, as you lose the ECDHE and AES suites and get 3DES and SHA1! 

Example after enabling FIPS

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995,3389 192.168.97.135

Starting Nmap 7.50 ( https://nmap.org ) at 2017-07-08 13:50 PDT

Nmap scan report for 192.168.97.135

Host is up (0.00058s latency).

PORT     STATE  SERVICE

443/tcp  closed https

465/tcp  closed smtps

993/tcp  closed imaps

995/tcp  closed pop3s

3389/tcp open   ms-wbt-server

| ssl-cert: Subject: commonName=FFKN25S

| Issuer: commonName=FFKN25S

| Public Key type: rsa

| Public Key bits: 2048

| Signature Algorithm: sha1WithRSAEncryption

| Not valid before: 2017-07-07T19:43:57

| Not valid after:  2018-01-06T19:43:57

| MD5:   835d ab76 7752 5d8a 4a3e d5d9 5fc3 4248

|_SHA-1: ac32 dc26 ae9d 2308 405e 595f 0e9f 4102 f661 8341

| ssl-enum-ciphers: 

|   TLSv1.0: 

|     ciphers: 

|       TLS_RSA_WITH_3DES_EDE_CBC_SHA (rsa 2048) - C

|     compressors: 

|       NULL

|     cipher preference: indeterminate

|     cipher preference error: Too few ciphers supported

|     warnings: 

|       64-bit block cipher 3DES vulnerable to SWEET32 attack

|       Weak certificate signature: SHA1

|_  least strength: C

Nmap done: 1 IP address (1 host up) scanned in 6.97 seconds

Manually Disable RC4/3DES

Then I found this tool [IIS Crypto] and disabled RC4/Triple DES in the ciphers column. It's still using TLS1.0 but at least it's not offering up RC/3DES.

Example After Running IISCryto

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995,3389 192.168.97.135

Starting Nmap 7.50 ( https://nmap.org ) at 2017-07-08 14:35 PDT

Nmap scan report for 192.168.97.135

Host is up (0.00031s latency).

PORT     STATE  SERVICE

443/tcp  closed https

465/tcp  closed smtps

993/tcp  closed imaps

995/tcp  closed pop3s

3389/tcp open   ms-wbt-server

| ssl-cert: Subject: commonName=FFKN25S

| Issuer: commonName=FFKN25S

| Public Key type: rsa

| Public Key bits: 2048

| Signature Algorithm: sha1WithRSAEncryption

| Not valid before: 2017-07-07T19:43:57

| Not valid after:  2018-01-06T19:43:57

| MD5:   835d ab76 7752 5d8a 4a3e d5d9 5fc3 4248

|_SHA-1: ac32 dc26 ae9d 2308 405e 595f 0e9f 4102 f661 8341

| ssl-enum-ciphers: 

|   TLSv1.0: 

|     ciphers: 

|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp521r1) - A

|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp521r1) - A

|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A

|     compressors: 

|       NULL

|     cipher preference: server

|     warnings: 

|       Weak certificate signature: SHA1

|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 7.39 seconds

Enabling TLS 1.1/1.2

Finally, I found this article How do I disable TLS 1.0 without breaking RDP? which pointed me to this MS KB - KB3080079. That update enabled TLS1.1/1.2. It's really hard to believe that MS thinks a patch to drop RC4 and 3DES is "optional".

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995,3389 192.168.97.135

Starting Nmap 7.50 ( https://nmap.org ) at 2017-07-08 15:00 PDT

Nmap scan report for 192.168.97.135

Host is up (0.0021s latency).

PORT     STATE  SERVICE

443/tcp  closed https

465/tcp  closed smtps

993/tcp  closed imaps

995/tcp  closed pop3s

3389/tcp open   ms-wbt-server

| ssl-cert: Subject: commonName=FFKN25S

| Issuer: commonName=FFKN25S

| Public Key type: rsa

| Public Key bits: 2048

| Signature Algorithm: sha1WithRSAEncryption

| Not valid before: 2017-07-07T19:43:57

| Not valid after:  2018-01-06T19:43:57

| MD5:   835d ab76 7752 5d8a 4a3e d5d9 5fc3 4248

|_SHA-1: ac32 dc26 ae9d 2308 405e 595f 0e9f 4102 f661 8341

| ssl-enum-ciphers: 

|   TLSv1.0: 

|     ciphers: 

|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp521r1) - A

|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp521r1) - A

|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A

|     compressors: 

|       NULL

|     cipher preference: server

|     warnings: 

|       Weak certificate signature: SHA1

|   TLSv1.1: 

|     ciphers: 

|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp521r1) - A

|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp521r1) - A

|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A

|     compressors: 

|       NULL

|     cipher preference: server

|     warnings: 

|       Weak certificate signature: SHA1

|   TLSv1.2: 

|     ciphers: 

|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (secp521r1) - A

|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (secp521r1) - A

|       TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (secp521r1) - A

|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp521r1) - A

|       TLS_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A

|       TLS_RSA_WITH_AES_256_CBC_SHA256 (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_CBC_SHA256 (rsa 2048) - A

|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A

|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A

|     compressors: 

|       NULL

|     cipher preference: server

|     warnings: 

|       Weak certificate signature: SHA1

|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 7.34 seconds

Setting the RDP server to use TLS

Now that the RDP server is offering up TLS1.2 we need to make a couple changes to the server so that the client will connect using TLS instead of the RDP protocol.

Open gpedit.msc and navigate to "ComputerConfigurationn, Administrative Templates, Windows Components, Remote Desktop Services, Remote Desktop Session Host, Security"

Then double click on "Set Client Encryption level"

Require use of specific security layer for remote (RDP) Connections. Set it to Negotiate or SSL(TLS 1.0). This will negotiate the highest level that the client supports.

Connecting to the VM from a 2012 server I could see that TLS1.2 was used with either setting.

References

Cipher Suites in TLS/SSL (Schannel SSP)
Update adds new TLS cipher suites and changes cipher suite priorities in Windows 8.1 and Windows Server 2012 R2
Default cipher suites and their preferred order for every Windows Server version

Create Interface Descriptions from CDP Neighbor Output

Most customers want interface descriptions on core switch ports that are connected to edge switches and edge switch ports that are connected to access points or other critical devices. When you are doing a green field design this is usually pretty easy and you can include the descriptions in your spreadsheet data and they are automatically inserted when you run the template.

But there are times when you are in the field and need to create a lot of descriptions. Doing this manually is tedious, time-consuming and error-prone. To work around this I wrote a simple Python script to create the interface descriptions. Here are the instructions to use it.

Quick Steps

Download the script from my Github page 

Log onto the switch and run

sh cdp ne det | i  Dev|Interface

This will display the device name and interface for each neighbor

Copy the output and save it in a text file called interface.txt in the same folder as the script

Run the script. This will output the code needed on the screen.
Note: this isn't a double click to run script, you need to run the script from the terminal or command line. If you want to double click to run it add
input('Press <ENTER> to continue') to the very end. Make sure the statement isn't indented, it should start at column 1. 

Copy the code and paste it into the switches 

Detailed Steps

If you haven’t used Github before don’t worry. It’s simple to download the script, just click the “Clone or download” button once you are on the page. 

Click the “Download Zip” link. If you have Git installed you can also copy the link and use git clone to download the file.

Log onto the switch you want to update.

Type sh cdp ne det | i Dev|Interface to display the neighbors and their interfaces

Copy from the first Device ID to the last interface

Paste this into a text editor and save it as interface.txt in the same folder as the script. If you don’t want DNS suffixes included do a search and replace before saving the file.

Run the script using python3 interface.txt. This will display the code needed to update the descriptions. Copy from the screen and paste into the switch.

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ python3 interface.py 

Interface GigabitEthernet1/0/1

des < ISHS-IDFG-GDAT-G1 >

exit

Interface GigabitEthernet1/0/3

des < ISHS-IDFG-GDAT-G3 >

exit

Interface GigabitEthernet1/0/2

des < ISHS-IDFG-GDAT-G2 >

exit

Interface GigabitEthernet1/0/4

des < ISHS-IDFG-GDAT-G4 >

exit

Interface GigabitEthernet1/0/49

des < ISHS-6880X.sbc-district.local >

exit

That does it for this simple script, I hope you find it useful!

Remember - Automate or perish!

Using arp-scan to find an available IP address

If you are running Linux, as a VM or on hardware, there is a very useful tool called arp-scan written by Roy Hill. I have links in the reference section to several pages that have in depth articles on other ways to use arp-scan.

In this short blog I want to show you a simple way find an available IP address on a LAN. Most of this is taken from the Pentestmonkey blog in the reference section.

If you are connected to a LAN that doesn't have DHCP enabled you will need to manually assign one. But how can you be sure that the IP isn't actually in use? Running Angry IP or some other ping scanning tool won't list devices that have firewalls that block ping. The last thing you want to do is pick an IP address that conflicts with a critical piece of equipment.

The first step is to figure out what the LAN IP address scheme is if you don't know that. Fire up Wirehark and set the display filter to "arp". In a few minutes you should see an arp request from a device on the network. The IP address can be found in the info column. You won't know the mask  start with a /24. The arp-scan user guide has a section on determining the mask - Determining the interface netmask.

If you found this site on my Github page you should already have the python script I wrote. The script makes it easy to create the arp-scan commands. If you need to download the script click the GitHub link above. Save the arpscan.py file to a folder.

Once you run the script use the commands to find devices that didn't respond to ping. In the example below I have a Windows 10 VM with the firewall on and set to block all incoming packets. The IP address of the VM is 192.168.10.164. A ping fails and would make you think that 192.168.10.164 is available!

ping 192.168.10.164
PING 192.168.10.164 (192.168.10.164) 56(84) bytes of data.
^C
--- 192.168.10.164 ping statistics ---
116 packets transmitted, 0 received, 100% packet loss, time 115220ms

Now run arp-scan. In this case both 0.0.0.0 and 1.0.0.1 returned all the mac addresses on the network.
You can clearly see that 192.168.10.164 is in use and you can't assign it to your laptop.

sudo arp-scan -I eth0 --arpspa=1.0.0.1 192.168.10.0/24

Interface: eth0, datalink type: EN10MB (Ethernet)

Starting arp-scan 1.8.1 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)

192.168.10.13 64:52:99:69:fd:30 (Unknown)

192.168.10.20 c0:3f:d5:68:3c:cc (Unknown)

192.168.10.151 08:66:98:45:3f:86 (Unknown)

192.168.10.152 30:52:cb:20:3d:c1 (Unknown)

192.168.10.155 98:f1:70:7e:3f:6c (Unknown)

192.168.10.156 70:81:eb:55:23:a8 (Unknown)

192.168.10.157 84:b8:02:01:33:58 (Unknown)

192.168.10.158 04:db:56:ed:3d:58 (Unknown)

192.168.10.159 00:0c:29:99:43:b2 VMware, Inc.

192.168.10.161 00:0c:29:33:73:00 VMware, Inc.

192.168.10.164 00:0c:29:40:39:97 VMware, Inc.

192.168.10.165 00:1e:06:30:43:65 WIBRAIN

192.168.10.166 34:64:a9:03:33:f1 (Unknown)

192.168.10.167 b8:78:2e:08:23:05 (Unknown)

192.168.10.169 24:77:03:8f:f3:24 Intel Corporate

192.168.10.175 b8:8d:12:08:63:aa (Unknown)

192.168.10.221 00:0c:29:4c:a3:4e VMware, Inc.

192.168.10.239 10:1f:74:63:31:f8 (Unknown)

192.168.10.250 50:06:04:cb:83:40 (Unknown)

19 packets received by filter, 0 packets dropped by kernel

Ending arp-scan 1.8.1: 256 hosts scanned in 1.818 seconds (140.81 hosts/sec). 19 responded

127.0.0.1 returned three mac addresses.

sudo arp-scan -I eth0 --arpspa=127.0.0.1 192.168.10.0/24

[sudo] password for mhubbard: 

Interface: eth0, datalink type: EN10MB (Ethernet)

Starting arp-scan 1.8.1 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)

192.168.10.13 64:52:99:69:fd:20 (Unknown)

192.168.10.20 c0:3f:d5:68:0c:cc (Unknown)

192.168.10.167 b8:78:2e:08:28:05 (Unknown)

3 packets received by filter, 0 packets dropped by kernel

Ending arp-scan 1.8.1: 256 hosts scanned in 1.680 seconds (152.38 hosts/sec). 3 responded

References

arp-scan homepage

The science of safely finding an unused ip address

use arp-scan to find hidden devices in your network

Arp-scan User Guide

A Simple Python 3 Script for my Favorite nmap Scripts

There are a few nmap scripts I use all the time. I can't always remember the syntax so I wrote a simple Python 3 script to list them.

If an IP address is required you are prompted to enter one. You can put in any valid IP address or address range in nmap format. There is no error checking so if you put in an invalid address you will get an invalid script output.

If an SNMP community string is required you will be prompted to enter it. Again, there is no error checking so enter carefully.

Example Usage

Trouble shooting NTP on Cisco devices can be time consuming. The first step I take is to run the nmap script for ntp info. It quickly tells me if the ip address I'm pointing "ntp server" to is actually serving up NTP.

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ python3 nmap3.py

0 Download Cisco Configs using SNMP
1 Checking Server Cipher Suites using ports 443, 465, 993 and 995
2 Display SSH fingerprint (Host Keys)
3 Troubleshooting DHCP with the NMAP Broadcast-DHCP-Discover script
4 Troubleshooting DHCP with the NMAP DHCP-Discover script
5 Troubleshooting IPv6 DHCP with a broadcast discover
6 Brute Forcing SNMP with NMAP - Requires a text file of guesses in c: ftp-root\snmp-string.txt
7 BACNET - scripts from https://github.com/digitalbond/Redpoint#enip-enumeratense
8 DNS Broadcast Discover
9 Banner Grab using banner-plus from HD Moore
10 NTP Monlist - Pull down NTP server information
11 NTP INFO - Pull down general NTP information
12 DNS Brute - Uses nselib/data/dns-srv-names for list of SRV records to try, nselib/data/vhosts-full.lst for hosts
13 SMB - Various scripts for SMB servers
14 SNMP on Windows
15 Basic Script Scan the -vv option includes more detail
16 SQL nmap --script smb-os-discovery.nse -p445 192.168.10.221
17 - Check for SSH V1


3.4.3 (default, Nov 17 2016, 01:08:31)
[GCC 4.8.4]


Input a number to select 11
Enter the IP Address 192.168.10.221
nmap -sU -p 123 --script ntp-info 192.168.10.221
mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ sudo nmap -sU -p 123 --script ntp-info 192.168.10.221

Starting Nmap 7.40 ( https://nmap.org ) at 2017-03-19 18:23 PDT
Nmap scan report for 192.168.10.221
Host is up (0.0013s latency).
PORT    STATE SERVICE
123/udp open  ntp
| ntp-info:
|_  receive time stamp: 2017-03-20T01:23:57
MAC Address: 00:0C:29:4C:AA:4E (VMware)

Nmap done: 1 IP address (1 host up) scanned in 10.31 seconds

If you don't see a time stamp the server you are using won't work.

Environmental Monitoring Systems

Here is a more involved example. I do a lot of core switch and edge switch replacements. It's important to know if any Environmental Monitoring Systems (EMS) are in place. Choice 7 in the script will return each of the Digital Bond ISC nmap scripts. Luckily most of my customers have a dedicated EMS vlan so I just scan that vlan.

BUT, I have found EMS devices on GUEST networks (oops!) and on user LAN segments so you may need to scan a lot of networks.

Since there are a lot of different EMS vendors the script outputs 11 different nmap scripts.

mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$ python3 nmap3.py

0 Download Cisco Configs using SNMP
1 Checking Server Cipher Suites using ports 443, 465, 993 and 995
2 Display SSH fingerprint (Host Keys)
3 Troubleshooting DHCP with the NMAP Broadcast-DHCP-Discover script
4 Troubleshooting DHCP with the NMAP DHCP-Discover script
5 Troubleshooting IPv6 DHCP with a broadcast discover
6 Brute Forcing SNMP with NMAP - Requires a text file of guesses in c: ftp-root\snmp-string.txt
7 BACNET - scripts from https://github.com/digitalbond/Redpoint#enip-enumeratense
8 DNS Broadcast Discover
9 Banner Grab using banner-plus from HD Moore
10 NTP Monlist - Pull down NTP server information
11 NTP INFO - Pull down general NTP information
12 DNS Brute - Uses nselib/data/dns-srv-names for list of SRV records to try, nselib/data/vhosts-full.lst for hosts
13 SMB - Various scripts for SMB servers
14 SNMP on Windows
15 Basic Script Scan the -vv option includes more detail
16 SQL nmap --script smb-os-discovery.nse -p445 192.168.10.221
17 - Check for SSH V1


3.4.3 (default, Nov 17 2016, 01:08:31)
[GCC 4.8.4]


Input a number to select 7
Enter the IP Address 10.23.200.0/24
nmap -sU -p 47808 -n --script bacnet-info.nse 10.23.200.0/24
nmap -sU -p 47808 -n --script BACnet-discover-enumerate 10.23.200.0/24
nmap -sU -p 47808 -n --script BACnet-discover-enumerate --script-args full=yes 10.23.200.0/24
nmap -p 44818 --script enip-enumerate 10.23.200.0/24
nmap -p 1911 --script fox-info 10.23.200.0/24
nmap -p 502 --script modicon-info.nse -sV 10.23.200.0/24
nmap -p 9600 --script omrontcp-info 10.23.200.0/24
nmap -sU -p 9600 --script omronudp-info 10.23.200.0/24
nmap -p 1962 --script pcworx-info -sV 10.23.200.0/24
nmap -p 20547 --script proconos-info -sV 10.23.200.0/24
nmap -p 102 --script s7-enumerate -sV 10.23.200.0/24
sudo nmap -sU -p 47808 -n --script BACnet-discover-enumerate 10.23.200.0/24
sudo nmap -sU -p 47808 -n --script BACnet-discover-enumerate --script-args full=yes 10.23.200.0/24
sudo nmap -p 1911 --script fox-info 10.23.200.0/24
nmap -p 9600 --script omrontcp-info 10.23.200.0/24
sudo nmap -sU -p 9600 --script omronudp-info 10.23.200.0/24
nmap -p 1962 --script pcworx-info -sV 10.23.200.0/24
nmap -p 20547 --script proconos-info -sV 10.23.200.0/24
sudo nmap -p 102 --script s7-enumerate -sV 10.23.200.0/24
sudo nmap -p 44818 --script enip-enumerate 10.23.200.0/24
sudo nmap -p 502 --script modicon-info.nse -sV 10.23.200.0/24
mhubbard@1S1K-SYS76:~/Dropbox/Python/Scripts$

The Script

You can download the script on github at nmap-python

Southern California Linux Expo Scale 15x

I decided to go big this year, get a hotel and make the most of the Southern California Linux Expo or Scale 15x. This is the fifteenth Scale and the second year that it has been in Pasadena. The conference opened on Thursday March 2^nd and ran through Sunday March 5^th. Here is the home page for scale 15x if you want to review the show.

The talks are being uploaded to Youtube. It's odd but the title on youtube doesn't have the talk's name in it. You need to know what room the talk was in and what day it was given. I have a link to the daily schedules in the references below.

How did Scale 15x compare to Scale 14x?

Last year I only went on Saturday so I didn't get to do nearly as much. But my impression is that 15 was quite different. Last year the expo floor had several HAM radio and Maker booths along with a few local Linux Users Groups. In addition there were several booths with small projects dedicated to TOR and anonymity on the web. There was definitely a "Snowden" effect going on. HPE purchased 4 booths and had a huge group of products and people talking about them.

The Scale 15x expo floor seemed much more like a typical tech show. The only LUG I saw was the San Gabriel Valley LUG and I didn't see any Maker or Ham booths. But there were a lot more companies showing products. The expo floor was still excellent and well worth my time but it was a different feeling.

I wasn’t able to make the Thursday or Friday talks but I got there in time on Friday to pick up my badge and hit the expo floor. I purchased a System76 Gazelle last summer and love it. I went with the fastest i7 6700 processor, 16GB of RAM and a Samsung Evo 850 SSD. The laptop supports m.2 NVME drives but my wallet didn’t!

System76 was one of the first booths and I got to meet James from tech support and see the new Oryx Pro laptop. As always, I learned something new from James and loved the Oryx. It supports up to 64GB of RAM, two NVME drives and an NVIDIA 1060 or 1070 GPU. The GPU drives a beautiful 4k display. I definitely need to get an Oryx Pro!

I also spent some time at the Libre Office booth. A very nice guy demoed Draw for me. I obviously have Libre Office installed since I’m running Ubuntu but I hadn’t noticed the Draw application. It is well done and after a quick search he found several networking icon packs. I need to spend some time to see what all is available for Draw.

Gentoo had a booth and I spent a lot of time with them. I learned a lot about Gentoo (and Pentoo) that I think will be useful as I continue learning about Linux. Gentoo is a distro that does a minimal install and then you can add the packages you need. Sounds like a great distro for a couple older quad core Core Duo desktops that I have at home.

The big change though was the demographics of the crowd. There were still a lot of old gray beards but there were a lot of high school students. This wasn't just luck, the organizers made a push to get younger people involved.  

Speaking of Getting Involved

I attended Luis Hernandez's talk on “Open Source Role in Cyber Competitions” and it was great. Luis is working hard to get middle and high schools to start security classes. Here is a link to the description of his talk - Open Source Role in Cyber Competitions. The page has a PDF of his slides. If you have any spare time and want to help start a program at a local school please contact Luis.

Capture that Flag!

On Saturday,  the inaugural “Capture the Flag” event for the Cyber Patriot program was held. The first hour of the competition was hardening an Ubuntu system. The next section was based on the Facebook CTF and involved a lot of decrypting of coded messages.

Don't know anything about Linux?

That's no problem! There were full day classes on Saturday and Sunday for beginners. There was an installfest along with the training so you could bring your own older PC and get help installing Linux on it. See the references below for more information.

The Security Track   

Obviously this is the track I followed, who needs to know what Kubernetes is! Both Saturday and Sunday were filled with talks. I even scored a brand new Yubikey 4 at the “Hardening PGP keys with the Yubikey" talk. I have wanted a Yubikey for some time but just hadn’t bought one yet so I was very excited! I can’t wait to get my SSH keys setup and to start using it for 2 factor Authentication. Most of the cloud services that I use support the Yubikey for 2FA. Here is a link to the Security Track page.

Open Source Role in Cyber Competitions

As mentioned above, I sat in on a talk focused on getting IT security training set up in middle and high schools. The instructor, Luis hernandez, was very passionate and successful! He had students from North Hollywood HS that had successfully competed in national competition. Here is the summary from the talk - “The overall purpose is to show how Linux has helped prepare students for sysadmin roles in the real world as this is what they need to do during competition.” If you work in a school or have a desire to volunteer Luis would be the guy to contact!

The Web of Trust

I have had a PGP key pair for quite a while but never attended a key signing party before. Over thirty of us showed up after the expo on Saturday night and after a couple hours we each had verified everyone’s key fingerprint and two forms of ID. Now I have thirty signatures in my web of trust. It was a really cool thing to do and I met a lot of hard core Linux fans.

My HAM radio License

I have wanted to get my FCC HAM radio license for a couple years now but never took the time. I was a 2841 Field Radio Technician in the Corps and have a degree in electronics so I figured the technical part of the exam wouldn’t be too hard but the regulation part always scared me off.

At Scale the exam was being offered on both Saturday and Sunday but I hadn’t realized it so I hadn’t prepared. One of the guys at the Key signing party was member of the test staff. He encouraged me to download the guide from dc408 ham radio guide, study and take the exam on Sunday. It was already after 21:00 and it had been long day but I went back to the hotel and plowed through the guide. I passed the exam on Sunday!!

The Dark Arts of SSH

The last talk I attended was right after I passed my HAM license exam. I got there a bit early and there was a kid in the row ahead of me. He looked to be about 13 and was glued to his phone. I assumed that he was playing a game but it turns out he was studying for the Ham exam! Unfortunately it was too late to take the exam but he was preparing for next year!

This was a pretty good talk. I have been studying from Michael Lucas's great book “SSH Mastery” but I still picked up a few tips. I can't recommend SSH Mastery enough. It's $10 and Michael self publishes it.

What about Next Year?

Scale is a really good event. The ticket is $87.50 and the Howard Johnson hotel was only $125 per night so compared to a VMworld or Cisco Live it's practically free. Next year I hope to be able to take Friday off and go all day. This year I was too tired to go the the Friday after party, next year I'm making it is my goal!

References

• Scale 15x Homepage
• Security Track page
• Capture the Flag
• The Key Signing Party
• Beginner Linux Training at Scale
• Scale 15 Schedules