Sunday, April 11, 2021

Apple MacBook Air M1 for Network Engineers Part 4

If you have been following along with parts 1-3, you now have macOS tuned up and some basic applications installed. If you need to review previous blogs in this series:

In part 4 we will:

In part 5 we will start installing terminal and networking applications like arp-scan, lldpd, cdpr, etc.

Using USB Ethernet Adapters

Big Sur makes it easy to configure USB Ethernet adapters using the Network Preferences app. Being a network engineer it's common for me to use two different USB Ethernet adapters at the same time. 

  • Connect one of the adapters
  • Open Network Preferences
  • Click the + sign on the bottom left of the panel
  • The new service dialog will open
  • Click the drop-down beside "Interface:" and select the adapter
  • You can give it a descriptive name or just click "create"

  • Repeat if you need a second adapter.
  • Connect Ethernet cables to the adapter and switch.

If both networks have DHCP running the interfaces will just come up. 

If there is no DHCP server available, the interfaces will show "Not Connected". That took a little getting used to since normally you think having a link will make the interface go into connected status. 

Note: if you change an adapter from DHCP to Manual or Manual to DHCP you have to click Apply before the change goes into effect.



Notice that the second USB Ethernet adapter has a "2" appended to it. In this example, both adapters are identical Satechi USB-C adapters with 3 USB-A ports.


Adding a route

If you need to reach additional networks connected to one of the adapters you will probably need to add a route. It's easy to add a route on MacOS. 

Let's say one of our adapters is connected to a surveillance network 10.29.1.0/24 with a gateway of .253. But there is another network 10.29.2.0/24 that you also need to access. You just need to add a route to 10.29.2.0/24.

Open Terminal

Type:

sudo route add 10.29.2.0/24 10.29.1.253


To display the routing table

netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            192.168.10.254     UGSc           en0
default            10.29.1.253        UGScI          en9
10.29.1/24         link#22            UCS            en9      !
10.29.1.100/32     link#22            UCS            en9      !
10.29.1.253/32     link#22            UCS            en9      !
10.29.1.253        link#22            UHLWIir        en9      !
10.29.2/24         10.29.1.253        UGSc           en9
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0


To remove the route

sudo route delete 10.29.2.0/24 10.29.1.253

Note: The route won't be persistent. When you reboot it will be gone.


Add a second IP Subnet to a USB Ethernet adapter

This is a common requirement for a network engineer but it's not intuitively obvious how to do it on Big Sur!

For this example, I want to be able to send traffic on the 10.10.10.0/24 subnet.

Follow these steps:
  • Open the Network Preferences app
  • Click the "+" sign on the bottom left to add a new service
  • Next to "Interface:" pick the USB Ethernet adapter
  • Next to "Service Name:" enter a descriptive name.
  • Click Create



The new service will appear in the network preferences app. It will probably show as "Not Connected" because the default IP is set to DHCP. Change to "Manually" and enter the correct IP address, subnet and optionally a router IP. Click "Apply" to activate the change.

This is what the interface looks like in the terminal:


en9: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=6467<RXCSUM,TXCSUM,VLAN_MTU,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether 00:e0:4c:68:0a:ab
	inet6 fe80::d6:bc89:db67:96ee%en9 prefixlen 64 secured scopeid 0x16
	inet 10.253.7.100 netmask 0xfffffc00 broadcast 10.253.7.255
	inet 10.10.10.100 netmask 0xffffff00 broadcast 10.10.10.255
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect (1000baseT <full-duplex>)
	status: active



Adding Vlan tagging

Again, Big Sur makes this easy but not obvious! In the network preferences app, click the funny little icon that looks like a circle with 3 dots in it:


  • Click on "Manage Virtual Interfaces..."




  • Click the "+" sign
  • Click on "New Vlan..."




In this example, I need to add Vlan 100

  • Next to "Vlan Name:" enter a descriptive name
  • Next to "Tag:" enter 100
  • Make sure "Interface:" is the USB 10/100/100 LAN
  • Click Create



The new interface will show up in the network preferences app. 



One annoying trait is that the interface will be named Vlan0 on the system no matter what Vlan tag you assigned. Output from ifconfig:

vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=6063<RXCSUM,TXCSUM,TSO4,TSO6,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether 00:e0:4c:68:0a:ab
	inet6 fe80::27:216d:4729:6c0d%vlan0 prefixlen 64 secured scopeid 0x18
	inet 10.10.10.100 netmask 0xffffff00 broadcast 10.10.10.255
	nd6 options=201<PERFORMNUD,DAD>
	vlan: 100 parent interface: en9
	media: autoselect (1000baseT <full-duplex>)
	status: active



Here is the switch port configuration that the USB adapter is connected to:
interface 1
   name "Uplink"
   tagged vlan 86,100
   exit


Here is the Vlan 100 configuration:
vlan 100
   name "Management"
   tagged 1
   untagged 3-24
   ip address 10.10.10.254 255.255.255.0
   exit


Here are the ping results:
ping 10.10.10.254
PING 10.10.10.254 (10.10.10.254): 56 data bytes
64 bytes from 10.10.10.254: icmp_seq=0 ttl=255 time=2.447 ms
64 bytes from 10.10.10.254: icmp_seq=1 ttl=255 time=1.624 ms
^C
--- 10.10.10.254 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.624/2.035/2.447/0.412 ms


Using Wireshark to verify the Vlan tagging

Wireshark on macOS can capture traffic and show the Vlan ID but there is a caveat. Even though we created the Vlan0 service and set it to Vlan 100, you must select the parent interface in wireshark. In our case that is en9.



Notice that VLAN100: vlan0 and USB 10/100/1000: en9 show the same amount of traffic in the wireshark capture display.


Here is a snippet of traffic showing that it is indeed tagged on Vlan 100
Ethernet II, Src: HewlettP_fe:88:80 (98:f2:b3:fe:88:80), Dst: RealtekS_68:0a:ab (00:e0:4c:68:0a:ab)
    Destination: RealtekS_68:0a:ab (00:e0:4c:68:0a:ab)
        Address: RealtekS_68:0a:ab (00:e0:4c:68:0a:ab)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: HewlettP_fe:88:80 (98:f2:b3:fe:88:80)
        Address: HewlettP_fe:88:80 (98:f2:b3:fe:88:80)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 100
    000. .... .... .... = Priority: Best Effort (default) (0)
    ...0 .... .... .... = DEI: Ineligible
    .... 0000 0110 0100 = ID: 100
    Type: IPv4 (0x0800)


A word about IPv6

If you look back at the output of ifconfig for the vlan0 interface you will notice an IPv6 address was created but it ends in %vlan0.

inet6 fe80::27:216d:4729:6c0d%vlan0 prefixlen 64 secured scopeid 0x18


To ping that interface we would use:

ping6 -I vlan0 fe80::27:216d:4729:6c0d
PING6(56=40+8+8 bytes) fe80::27:216d:4729:6c0d%vlan0 --> fe80::27:216d:4729:6c0d
16 bytes from fe80::27:216d:4729:6c0d%vlan0, icmp_seq=0 hlim=64 time=0.158 ms
16 bytes from fe80::27:216d:4729:6c0d%vlan0, icmp_seq=1 hlim=64 time=0.262 ms
16 bytes from fe80::27:216d:4729:6c0d%vlan0, icmp_seq=2 hlim=64 time=0.228 ms
16 bytes from fe80::27:216d:4729:6c0d%vlan0, icmp_seq=3 hlim=64 time=0.340 ms


Using show management on the switch will list it's IPv6 address

Address    |                                             Address    
  Origin     | IPv6 Address/Prefix Length                  Status     
  ---------- + ------------------------------------------- -----------
  autoconfig | fe80::9af2:b3ff:fefe:8880/64                preferred  


Let's see if we can ping the switch on Vlan 100 using IPv6:

ping6 -I vlan0 fe80::9af2:b3ff:fefe:8880
PING6(56=40+8+8 bytes) fe80::27:216d:4729:6c0d%vlan0 --> fe80::9af2:b3ff:fefe:8880
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=0 hlim=64 time=3.237 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=1 hlim=64 time=1.498 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=2 hlim=64 time=1.853 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=3 hlim=64 time=1.865 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=4 hlim=64 time=1.374 ms
^C
--- fe80::9af2:b3ff:fefe:8880 ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.374/1.965/3.237/0.665 ms


You can also use this format of ping6. Notice the %vlan0 after the IPv6 address. If you were using interface en7 instead of vlan0, you would append en7 to the IPv6 address instead.

ping6 fe80::9af2:b3ff:fefe:8880%vlan0
PING6(56=40+8+8 bytes) fe80::27:216d:4729:6c0d%vlan0 --> fe80::9af2:b3ff:fefe:8880%vlan0
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=0 hlim=64 time=1.365 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=1 hlim=64 time=0.815 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=2 hlim=64 time=0.848 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=3 hlim=64 time=0.724 ms
16 bytes from fe80::9af2:b3ff:fefe:8880%vlan0, icmp_seq=4 hlim=64 time=0.771 ms
^C
--- fe80::9af2:b3ff:fefe:8880%vlan0 ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.724/0.905/1.365/0.234 ms





Excellent, now let's check the SSH port with nmap on the IPv6 Interface. 

nmap -e vlan0 -sV -p22 -6 fe80::9af2:b3ff:fefe:8880
Starting Nmap 7.91 ( https://nmap.org ) at 2021-04-18 14:43 PDT
Nmap scan report for fe80::9af2:b3ff:fefe:8880
Host is up (0.0018s latency).

PORT   STATE SERVICE VERSION
22/tcp open  ssh     Mocana NanoSSH 6.3 (protocol 2.0)

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 0.63 seconds



So how do we SSH using the IPv6 address?

From the mac
ssh vector@fe80::9af2:b3ff:fefe:8880%vlan0

vector@fe80::9af2:b3ff:fefe:8880's password:
HP J9727A 2920-24G-PoE+ Switch
Software revision WB.16.10.0010

That worked. Without IPv6 specific ACLs and with IPv6 autoconfig enabled, SSH is open to anyone that tries IPv6. However, if you noticed, the autoconfig address starts with fe80 which means it is link local so the attacker would have to be on the same Vlan as the interface.

What if I restrict management access to only subnet 10.10.100.0/24

So let's try it out. 
First I'll ssh to the IPv4 address to make sure that works:

ssh vector@10.10.10.254

vector@10.10.10.254's password:
HP J9727A 2920-24G-PoE+ Switch
Software revision WB.16.10.0010


That worked, so I will add an ip authorized-manager as 10.10.100.0/24 and try to ssh using the IPv4 address. Remember, our IPv4 address is 10.10.10.100, so it's not on the allow list.

HP-2920-24G-PoEP(config)# ip authorized-managers 10.10.100.0 255.255.255.0 access manager

from the MacBook
ssh vector@10.10.10.254
kex_exchange_identification: Connection closed by remote host


As you can see, ssh from the MacBook now fails.

But what if I try it from the IPv6 address?

ssh vector@fe80::9af2:b3ff:fefe:8880%vlan0
HP J9727A 2920-24G-PoE+ Switch
Software revision WB.16.10.0010

Your previous successful login (as manager) was on 2021-04-18 07:35:00
 from 10.10.10.100
There has been 1 unsuccessful login attempt since your previous login,
 most recently on 1990-01-01 07:36:16

Connection to fe80::9af2:b3ff:fefe:8880%vlan0 closed by remote host.
Connection to fe80::9af2:b3ff:fefe:8880%vlan0 closed.

Since IPv6 isn't disallowed we logged right in. You can see the failed attempt from 10.10.10.100 in the log.


Now how would you ssh to the MacBook from the Aruba 2930f?


If we look at "show management" again we see that Vlan 100 has IPv6 enabled and is set for autoconfig. So, just like on the MacBook, we use ssh <IPv6 Address> then append the Vlan id, %vlan100 in this case.

I didn't have an ssh server running on my MacBook so it didn't succeed but it tried.

  Interface Name  : Management          
  IPv6 Status     : Enabled 

  Address    |                                             Address    
  Origin     | IPv6 Address/Prefix Length                  Status     
  ---------- + ------------------------------------------- -----------
  autoconfig | fe80::9af2:b3ff:fefe:8880/64                preferred  


From the 2930f
ssh fe80::27:216d:4729:6c0d%vlan100
The SSH connection failed: Connection refused.


Looking at IPv6 routes

You still use the netstat -nr command. For IPv6 you scroll down past the IPv4 routes. I have a link in the reference section at the end of the blog that explains the flags. 

For our example, here is the IPv6 table:

netstat -nr
Routing tables

. 
. IPv4 detail removed for brevity
.

Internet6:
Destination                             Gateway                         Flags         Netif Expire
fe80::9af2:b3ff:fefe:8880%vlan0         98.f2.b3.fe.88.80               UHLWI         vlan0


Internet Connection Sharing

Sometimes you need to provide Internet access to a new network to complete your work. For example, recently I was installing a greenfield Ubiquiti network. It included a Cloud Key so it needed Internet access to register and be fully functional. Unfortunately, the Internet access hadn't been installed yet. 

I plugged in a second USB Ethernet adapter, connected wireless to my phone set to hotspot mode and used the "Sharing" System Preferences app to configure the sharing. 

  • Set "Share your connection from:" to wifi
  • Put put a check next to the USB Ethernet interface in "To computers using:"
  • Under "Service" put a check next to "Internet Sharing"    

Now connect the USB Ethernet to the device you need to share Internet with. I have found that it doesn't matter if the USB Ethernet is set to DHCP or manual. The device using the connection gets an address in the range 192.168.2.0/24




Combine Ethernet ports into a virtual port (LACP - Port Channel)

I haven't had a reason to bond two USB-C Ethernet adapters into an LACP bond for bandwidth on my laptop(!) but I have had a need to verify the configuration of a "Trunk" on an Aruba switch or a "Port-Channel" on a Cisco switch. 

It's very easy on Big Sur, again, just not so obvious. One caveat is that the bond has to use LACP. To get started:

In the network preferences app, click the funny little icon that looks like a circle with 3 dots in it:


Click on "Manage Virtual Interfaces..."



Click the "+" sign

Click on "New Link Aggregate..."



Enter a descriptive name for the bond and check the two USB Ethernet interfaces:



Click "Create"

You should see the new bond with the BSD Name "Bond0"


Click "Done"

You can now connect the Ethernet cables to the switch. As we found earlier, the Bond won't show "Connected" until it gets a DHCP assigned address or you manually assign a static IP address.

In this example, I set a static address since this Vlan didn't have a DHCP server. Here is what the interfaces look like in the terminal.


en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 50:ed:3c:22:be:32
	inet6 fe80::1465:e07c:8c73:4b87%en0 prefixlen 64 secured scopeid 0xa
	inet 192.168.10.143 netmask 0xffffff00 broadcast 192.168.10.255
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect
	status: active
en6: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=6467<RXCSUM,TXCSUM,VLAN_MTU,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether 00:e0:4c:68:0a:0d
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect (1000baseT <full-duplex>)
	status: active
en9: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=6467<RXCSUM,TXCSUM,VLAN_MTU,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether 00:e0:4c:68:0a:ab
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect (1000baseT <full-duplex>)
	status: active
bond0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=6067<RXCSUM,TXCSUM,VLAN_MTU,TSO4,TSO6,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether 00:e0:4c:68:0a:ab
	inet6 fe80::4b6:645c:b6da:611f%bond0 prefixlen 64 secured scopeid 0x19
	inet 10.112.254.20 netmask 0xffff0000 broadcast 10.112.254.255
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect (1000baseT <full-duplex>)
	status: active
	bond interfaces: en9 en6

I included en0, my wifi interface, for a reason. Look at the Ethernet address of en0. A "show lacp peer" lists en0's MAC as the "System ID" of the MacBook. I disabled wifi and disconnected/reconnected the Ethernet cables but the Aruba 5412r still showed the en0 MAC as the system ID.  


AHS-5412-MDF# sh lacp peer

LACP Peer Information.


System ID: 883a30-768a00


  Local  Local                          Port      Oper    LACP     Tx
  Port   Trunk  System ID         Port  Priority  Key     Mode     Timer
  ------ ------ ----------------- ----- --------- ------- -------- -----
  A8     Trk1   50ed3c-22be32     8     32768     1       Active   Slow
  B8     Trk1   50ed3c-22be32     22    32768     1       Active   Slow


But, doing a show mac-address trk1 lists the MAC addresses of interfaces en6/en9.


AHS-5412-MDF# sh mac-address trk1

 Status and Counters - Port Address Table - Trk1

  MAC Address       VLANs
  ----------------- ------------
  00e04c-680a0d     254
  00e04c-680aab     254


Here is the configuration of the trunk and Vlan 254 on the Aruba 5412r switch.

trunk A8,B8 trk1 lacp

show run vl 254

Running configuration:

Vlan 254
   name "Device Management"
   untagged Trk1
   ip address 10.112.254.254 255.255.255.0
   exit


I haven't covered installing the lldpd software yet, which will be in part 5, but here is what the lldp neighbor looks like on the MacBook:

[lldpcli] # sh ne
-------------------------------------------------------------------------------
LLDP neighbors:
-------------------------------------------------------------------------------
Interface:    en6, via: LLDP, RID: 4, Time: 0 day, 00:44:53
  Chassis:
    ChassisID:    mac 88:3a:30:76:8a:00
    SysName:      AHS-5412-MDF
    SysDescr:     HP J9851A Switch 5412Rzl2, revision KB.16.10.0012, ROM KB.16.01.0009 (/ws/swbuildm/rel_ajanta_qaoff/code/build/bom(swbuildm_rel_ajanta_qaoff_rel_ajanta))
    MgmtIP:       10.112.254.254
    Capability:   Bridge, on
    Capability:   Router, on
  Port:
    PortID:       local 8
    PortDescr:    A8
    TTL:          120
  Unknown TLVs:
    TLV:          OUI: 00,16,B9, SubType: 2, Len: 2 00,01
-------------------------------------------------------------------------------
Interface:    en9, via: LLDP, RID: 4, Time: 0 day, 00:45:00
  Chassis:
    ChassisID:    mac 88:3a:30:76:8a:00
    SysName:      AHS-5412-MDF
    SysDescr:     HP J9851A Switch 5412Rzl2, revision KB.16.10.0012, ROM KB.16.01.0009 (/ws/swbuildm/rel_ajanta_qaoff/code/build/bom(swbuildm_rel_ajanta_qaoff_rel_ajanta))
    MgmtIP:       10.112.254.254
    Capability:   Bridge, on
    Capability:   Router, on
  Port:
    PortID:       local 40
    PortDescr:    B8
    TTL:          120
  Unknown TLVs:
    TLV:          OUI: 00,16,B9, SubType: 2, Len: 2 00,01
-------------------------------------------------------------------------------
[lldpcli] #


As expected, it shows both interfaces, A8/B8, on the 5412. 


Finally, to show that it worked here is a ping to an access point connected to another switch:

ping 10.112.254.155
PING 10.112.254.155 (10.112.254.155): 56 data bytes
64 bytes from 10.112.254.155: icmp_seq=0 ttl=64 time=5.203 ms
64 bytes from 10.112.254.155: icmp_seq=1 ttl=64 time=1.693 ms
64 bytes from 10.112.254.155: icmp_seq=2 ttl=64 time=1.613 ms
64 bytes from 10.112.254.155: icmp_seq=3 ttl=64 time=1.752 ms
^C
--- 10.112.254.155 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.613/2.565/5.203/1.524 ms


Wireshark

I opened Wireshark and Bond0 appeared in the capture interface dialog. While capturing, this was the title:


Listing everything about the Network!

macOS includes a utility called "networksetup". It has over 100 options! I can't cover all of them here. To see a complete list:

  • Open a terminal
  • enter networksetup 1

The 1 is a invalid option and networksetup will dump every open out as a list. You can also enter networksetup with no options and it will dump a help screen.


Getting Started with networksetup

You can use the 

networksetup -listallnetworkservices

command to list all network services on your Mac. On my M1 I have connected several USB Ethernet adapters and created a Vlan interface. Here are what my services look like:

networksetup -listallnetworkservices
An asterisk (*) denotes that a network service is disabled.
Wi-Fi
iPhone USB
USB 10/100/1000 LAN 3
USB 10/100/1000 LAN
StarTech USBA2DPGB
USB 10/100/1000 LAN 2
Bluetooth PAN 2
Thunderbolt Ethernet Slot 1
Thunderbolt Bridge
VLAN-254
VPN (L2TP)


Now that you have a list of services, you can get information about them. In this case, my current wifi settings:

networksetup -getinfo Wi-Fi                                                                                                           [18:16:18]
DHCP Configuration
IP address: 10.0.23.119
Subnet mask: 255.255.252.0
Router: 10.0.20.1
Client ID:
IPv6 IP address: none
IPv6 Router: none
Wi-Fi ID: 50:ed:3c:22:be:32



That does it for Part 4. Be sure to check back soon for Part 5!


References