Wednesday, February 24, 2016

Cisco 6880-X with 6800IA Instant Access Fabric Extender

I was lucky enough to get a 6880-X-LE switch and 6800IA-FPD to configure this week!  In fact I got 10 of them to deploy to ten middle school sites.

References

Cisco has a lot of documentation for the the 6800IA. Here are the documents I used to configure mine:

Cisco Catalyst Instant Access FAQ
Cisco Catalyst Instant Access Solution White Paper
Instant Access (IA)
So you really want to manually configure the 6800ia? It’s possible

For the VSS configuration I found a great blog on the Ciscozine.com site
Cisco 6500 VSS configuration

What is a Cisco 6880-X?

According to Cisco the Catalyst 6880-X meets the unique needs of a midsize campus, with the ideal combination of:

  •     Advanced services
  •     Optimal port capacity
  •     10 Gigabit Ethernet density
  •     Compelling economics

These switches offer:

  • Five slots (one fixed and four extensible half-wide slots) on less than a 5-rack-unit (RU) form factor chassis
  • Support for up to eighty 10/1 Gigabit Ethernet or twenty 40 Gigabit Ethernet ports, and readiness for 100 Gigabit Ethernet
  • Full feature consistency with Cisco Catalyst 6500 Series Switches

Prerequisites

To use the 6800IA Instant Access FEX  you must be running at least IOS Software 15.1(2). I chose 15.1(2)SY6 since it's an MD release. The FEX connects to the 6880-x using 10Gbe links and both fiber and copper options are supported. For this deployment I am using 2m Twinax cables. 

The 6800IA FAQ says the following about cabling:
The Cisco Catalyst 6800ia has two 10 Gigabit Ethernet SFP+ uplink ports. The ports support Short-Reach (SR), Long-Reach (LR), Long-Reach Multimode (LRM), and Extended-Reach (ER) Small Form-Factor Pluggable Plus (SFP+) fiber transceivers and associated MMF or SMF fiber cables. They also support 1M, 3M, and 5M copper Twinax cables.

So if you need to place your 48 port FEX 10Km from  your data center Cisco has you covered! 

Preparing the 6880-X

The Parent as Cisco calls it must be put into Virtual Switching System (VSS) mode before it can be connected to the 6800IA. Even if you only have one 6880 you still have to put it into VSS mode. 

Here are the steps to put the 6880-X into VSS Mode:

(config)switch virtual domain 100
(config-vs-domain)#switch 1
Domain ID 100 config will take effect only
after the exec command 'switch convert mode virtual' is issued
exit

#switch convert mode virtual
This command will convert all interface names
to naming convention "interface-type switch-number/slot/port",
save the running config to startup-config and 
reload the switch.

NOTE: Make sure to configure one or more dual-active detection methods
once the conversion is complete and the switches have come up in VSS mode.

Do you want to proceed? [yes/no]: yes

The switch will reload at this point. 

When it restarts the Ten Gig ports will be renamed in the format T1/5/1.

Now that the switch is in VSS mode we can configure it for the 6800IA. FEX IDs can must be in the range of 101-199. In this example I am using 101.

#module provision create fex 101 type c6800IA-48FPD

Instant Access staggered initialization mode avoids any excessively high CPU utilization that might occur if multiple IA clients attempt to initialize simultaneously. In my deployment there is only one IA but I configured stagger in case the customer stacked a second IA in future. Cisco's recommended delay is 120.

conf t
fex stagger 120
exit

interface po101
switchport
switchport mode fex-fabric
fex associate 101
no shutdown
exit

interface range tenGigabitEthernet 1/5/15-16
switchport
switchport mode fex-fabric
channel-group 101 mode on
no shut
exit
fex 101
description < My 6800IA-48FPD >
end

Connecting the 6800IA
You can now connect the cables from the 6880 to the 6800IA. Once you do the 6880-X will verify the firmware version on the IA. In my case 15.1(2)SY6 was newer so the 6880-X started the upgrade process automatically. 

Once the upgrade/reload process completed I did a show interface to look at the interfaces on the  IA. In the output below I have configured the first port as an access port on vlan 5 with a voice port. Note the switchport trunk allowed vlan 1 command. That is a default and cannot be removed. It didn't affect operation on an access port. Here is the explanation from the FAQ:

Q. Why is the default configuration of Instant Access client host port configuration “switch trunk allowed vlan 1” and not “all”?

A. Each Instant Access host port can be configured in access or trunk mode (default is dynamic). If in trunk mode, there is a constraint as to how many VLANs can be trunked on each port.

Note: No more than 1,000 VLANs can be associated with a single FEX ID, divided by the number of Instant Access trunk ports. To make sure that this constraint is followed, implementation requires specifying explicitly which VLANs will be trunked. We recommend no more than 20 VLANs per Instant Access trunk port (up to the total of 1000 per FEX), to limit the amount of BPDU processing.

sh run | b 101/1/0/1
interface GigabitEthernet101/1/0/1
 switchport
 switchport access vlan 5
 switchport trunk allowed vlan 1
 switchport mode access
 switchport nonegotiate
 switchport voice vlan 253
 logging event link-status


Show Commands

Cisco provides several show commands for the FEX. Here is "show fex" with a ?:

sh fex 101 ?
  detail       FEX detailed information
  protocol     SDP/SRP protocol details
  transceiver  FEX transceiver information
  version      FEX version
  |            Output modifiers
  <cr>


show fex 101 detail
FEX: 101        Description: < My 6800IA-48FPD >     state: online
  FEX version: 15.0(2)EX9
  Extender Model: C6800IA-48FPD, Extender Serial: FCW196789PK
  FCP ready: yes
  Image Version Check: enforced
  Fabric Portchannel Ports: 2
  Fabric port for control traffic: Te1/5/15
  Fabric interface state:
    Po101      - Interface Up.
    Te1/5/15   - Interface Up.      state: bound
    Te1/5/16   - Interface Up.      state: bound

show fex 101 protocol | incl SDP
    8789 SDP pkts sent
    8785 SDP pkts received
    8768 SDP pkts sent
    8763 SDP pkts received

sh fex 101 transceiver
Module 1:
Fex Uplink: 1
SFP+ transceiver present
 Identifier        : SFP [0x03]
 Connector         : Copper Pigtail [0x21]
 Speed             : Unspecified [0x00]
 Product ID        : SFP-H10GB-CU3M
 Version ID        : V03
 Encoding          : Unspecified [0x00]
 BR, Nominal       : 103x100 MHz [0x67]
 Length(9u)        : Transceiver does not support single mode fibre,
                     or the length information must be determined from
                     the transceiver technology. [0x00]
 Length(50u)       : Transceiver does not support 50 micron multi-mode fibre,
                     or the length information must be determined from
                     the transceiver technology. [0x00]
 Length(62.5u)     : Transceiver does not support 62.5 micron multi-mode fibre,
                     or the length information must be determined from
                     the transceiver technology. [0x00]
 Vendor Name       : CISCO-TYCO
 Vendor OUI        : 0x0 0x40 0x20
 Vendor PN         : 1-2053783-2
 Vendor SN         : TED1726B08N
 Vendor rev        : R
 CC_BASE           : 0xF
 Date code         : 17062600
 CC_EXT            : 0xDB

sh interfaces fex
     Fabric    Fabric Port  FEX                  FEX
FEX  Port      State        Uplink    Model              Serial
---------------------------------------------------------------------
101  Te1/5/15  bound        Te1/0/1   C6800IA-48FPD      FCW196789PK
101  Te1/5/16  bound        Te1/0/2   C6800IA-48FPD      FCW196789PK