As one of the largest Cisco Partners in the Netherlands we do a lot of Cisco UCS implementations and as the first company in the Netherlands with the Cisco Advanced Data Center Architecture Specialization, where the place in the Netherlands for Cisco UCS troubleshooting. Last week a colleague was called to a troubleshoot a customer problem.

The customer was unable to create a 14th Virtual Network Interfaces on their Cisco UCS Virtual Interface Card and 13 interfaces is far from the maximum of 128 or 256 possible virtual interfaces per Cisco UCS VIC. Fortunately the solution appeared to be simple.

In a Cisco UCS environment all centralized intelligent occurs in the Fabric Interconnect. When using Cisco UCS Virtual Interface Cards (VICs) you can create Virtual Network Interfaces (VIFs) which can be presented to individual virtual machines. All of these virtual interfaces that are created show up in the Fabric Interconnects. They are called VIFs (Virtual Interfaces) and use VN-Tags.

The number of VIFs per blade is limited by the most restrictive item in the following list:

  • the network connectivity from chassis I/O Module (IOM) to Fabric Interconnect;
  • the Adapter VN-Tag namespace;
  • the OS/BIOS version.

 

For the 6100-series Fabric Interconnect has a namespace of 512 VN-Tags per ASIC, of which 8 are reserved.

On the Fabric Interconnect, the number of physical ports per ASIC is 4 which means (512-8)/4=126 per physical port. One VN-Tag per physical port is reserved for the Management Controller on the IOM which leaves 125.

With 8 blade servers in a Cisco UCS chassis, the number of VN-Tags per blade is 125/8 = 15 (rounded down).

So to determine the number of VIFs per blade server, the formula is: (15*n)-2

Where ‘n’ is the number of uplinks from the IOM to the Fabric Interconnect and two VN-Tags are reserved for the management of the adapter itself.

 

The number of available VIFs per blade server on a 6100-series Fabric Interconnect is:

  • with 1 uplink: 13 VIFs per blade server;
  • with 2 uplinks: 28 VIFs per blade server;
  • with 4 uplinks: 58 VIFs per blade server;
  • with 8 uplinks: 118 VIFs per blade server.

For the 6200-series Fabric Interconnect the implementation is the same but the numbers are much higher. Here there are 4096 VN-Tags per ASIC, of which 8 are reserved.

The number of physical ports per ASIC is 8 which means (4096-8)/8 = 511 per physical port. There is one reserved for the Management Controller on the IOM which leaves 510.

With 8 blade servers in a Cisco UCS chassis, the number of VN-Tags per port available for a blade is 510/8 = 63 (rounded down).

So to determine the number of VIFs per blade server, the formula is: (63*n)-2

Where ‘n’ is the number of uplinks from the IOM to the Fabric Interconnect and two VN-Tags are reserved for the management of the adapter itself.

The number of available VIFs per blade server on a 6200-series Fabric Interconnect is:

  • with 1 uplink: 61 VIFs per blade server;
  • with 2 uplinks: 124 VIFs per blade server;
  • with 4 uplinks: 250 VIFs per blade server;
  • with 8 uplinks: 502 VIFs per blade server.

Now this is the maximum the Fabric Interconnect can support based on the capacity of a single port. Of course the adapter itself will also have a limit. The combination will determine the useable number of VIFs.

Cisco VIC M81KR P81E VIC 1240 VIC 1280
# VIFs 128 128 256 256

 

M81KR/P81E VIC 1240 VIC 1280

 

 

The 6200/2200 series devices introduce some extra design options, including port-channeling capability. Because the VN-tags can be dynamically allocated, if a single link fails, the other links can make use of those VN-tags that are available at the ASIC level.

It’s important to note that to ensure the (63*n)-2 formula applies, you need to connect the links from a single IOM to the ports that belong to the same ASIC (ports 1-8, 9-16 etc…).

 

*Kudos to Michel for supplying this information.