Category Archives: HP C7000

ESXi 5.5 VMNIC Sequencing order on HP534FLB and HP554FLB Ethernet

Most of you might be aware but thought to share the KB  regarding HP554FLB and HP 534FLB difference in term of native FC and FCOE. HP 554 (Gen 8) supports native FC while HP 534FLB (Gen9) does not support native FC but Fceo. That is the reason when we run the following commands on 554FLB enabled blade we get nothing except prompt causing the VMNIC in sequence order i.e. VMNIC0,1,2,3,4,5,6

#esxcli fcoe nic list.

But if we checked the same against HP 534FLB , we get the allocated vmnicX ( X could be vmnic2, 3) for Fceo connection causing the VMNIC in non-sequence order i.e. VMNIC0,1,4,5,6,7

#esxcli fcoe nic list.


Data Transfer Bandwidth Calculation based on 10G link with 3ms latency and 50% of TCP windows size i.e 32KB

Data Transfer Bandwidth Calculation based on 10G link with 3ms latency and 50% of TCP windows size i.e 32KB

TCP Windows size calculation in bits = 32KB * 1024 ( bytes) * 8 (bits) = 262,144 bits

Network Latency = 3ms / 1000 (sec) = 0.003 second

Throughput = TCP windows size calculation in bits/ Network Latency in second

                                                 =   262,144 / 0.003

Bits/Second                                =   87381333.33333333 bits/second

Bytes/Second                             =   87381333.33333333 bits/second /8

                                                =   10922666.6666667 bytes/Second

KB/Second                                =   10922666.6666667 bytes/Second / 1024

                                               =   10666.66666666667 KB/Second

MB/Second                               =   10666.66666666667 KB/Second / 1024

                                               =   10.4166 MB/Second

MB/Hours                                 =   10.4166 MB/Second * 3600 seconds

                                               =   37500 MB/Hours

GB/Hours                                  =   37500 MB/Hours / 1024

                                               =   36.62 GB/Hours

Note – There could be more running instance at same time so I have reduced TCP windows size packet for the following reasons.

1-  To avoid packet buffering during data transfer

2-  Running Multiple instance at same time.

I have observed that how much latency you are getting the only factor deciding how much data will be transfer in said period.

Let me know if there is any comments???


Downgrade OA ( Firmware v 3.60) & VCM ( Firware v 3.70) to OA ( Firmware v 3.56) & VCM ( Firmware v 3.51)

Here are the steps I followed during OA & VCM firmware downgrade in order to sync other VCM domain running with same version 3.51 and managed by VCEM.

I would be covering only SUDO steps over here 🙂

  • Delete domain ( if any) from VCM module ( used https://vcmdomainip) – (That is very important to delete domain first if we are downgrading the firmware. That is not applicable if we are upgrading the same.)
  • Downgrade Active OA firmware ( Check Force downgrade) to 3.56. Once this is downgraded it will update standby OA automatically
  • You must have administrative credential before proceeding for VCM downgrade ( as user created during domain creation will be lost).
  • Install VCSU 1.7.x ( Version should be greater than 1.5 if we are working with 3.x VCM module firmware upgrade ) – this is UNIX style. However you can use HP DVD method but that looks very convincing and promising to me.
  • Run VCSU and enter ‘update’ command following with OA IP and their credential. It may take around 30 minutes to update the same.
  • Once done, check OA with their settings like EBAY addressing, Interconnect/Devices bay IP and verified your OA and VCM Modules firmware as v3.56 & v3.51
  • Create VC domain like SATYENDRA_A24R1 ( whatever naming standard you guys follow) and follow the wizard ( in last wizard, DO NOT click on network as that would be populated once we add domain to VCEM domain group).
  • Once domain is created, discover/add them on VCEM.
  • License the domain
  • Put the domain in maintenance once license is done and change the SAN PATH Login-redistribution setting from MANUAL to AUTOMATIC .
  • Exit the maintenance mode.
  • Create the servers profile as per requirement
  • Server is ready for deployment.

Few Points to take into consideration before architect HP c7000 Networking for Blades. – VCM (VCEM) firmware 3.x onwards

1 –  What could be VLAN Capacity mode i.e. expanded or Legacy.

Note -Virtual Connect release 3.30 provides an expanded VLAN capacity mode when using Shared Uplink Sets, this mode can be enabled through the Ethernet Settings tab or the VC CLI. The default configuration is “Legacy VLAN Capacity” mode. This scenario does require a change to this setting as Expanded VLAN capacity will be required. Once a domain has been set to Expanded VLAN Capacity, it cannot be reset back to Legacy mode. In order to go back to legacy mode, the Domain would need to be deleted and recreated. Expanded VLAN capacity will be greyed out, this is only supported with 10Gb based VC modules

If the VC domain is not in Expanded VLAN capacity mode, you will receive an error when attempting to create more that 128 VLANs in a SUS. Expended VLAN supports 1000 VLAN/SUS.There are 8 faceplate ports. Ports X1-X4 are SFP+ transceiver slots only; which can accept a 10Gb or 8Gb SFP+ transceiver. Ports X5-X8 are SFP and SFP+ capable, and do not support 8Gb SFP+ transceivers.

2 – The protocol personality of HP Virtual Connect Flex Fabric module uplink ports X1-X4 is determined by the type of SFP+ transceiver plugged in, i.e. – 4/8Gb FC or 10GbE SFP+. The remaining ports X5-X8 are fixed 1/10GbE protocol ports and will not accept FC SFP+ transceivers.

3 –  Enclosure EBAY addressing, OA and ILO IPs must be on same VLAN/Range in order to proper communicate.

4 – It’s up to you how you wants to configure VCM module Active/Active or Active/Passive. an Active/Standby configuration places the redundancy at the VC level, where Active/Active places it at the OS NIC teaming or bonding level.

5 – For supporting A/A or A/P you needs to configure LACP within the same Link Aggregation Group.

6 – Configure FAST MAC Cache Failover settings if you are architecting VCM modules on Active/Passive mode. Enabling Fast MAC Cache Failover forces Virtual Connect to transmit Ethernet packets on newly

active links, which enables the external Ethernet switches to identify the new connection (and update their MAC caches appropriately). This transmission sequence repeats a few times at the MAC refresh interval (five seconds is the recommended interval) and completes in about one minute. ( Not applicable in Active/Active Mode)

7 – VMware does provide several NIC teaming algorithms.. HP will support any of the algorithms except for IP Hash.  IP Hash requires switch assisted load balancing (802.3ad), which Virtual Connect does not support 802.3ad with server downlink ports. HP and VMware recommend using Originating Virtual Port ID with Standard vSwitch, and Physical NIC Load when using vDS and NetIOC,

Note – If configuring with Vsphere5, I would not recommend to implement NetIOC as we are segregating traffic based on different VLAN i.e Vmotion, Management, ILO for different purpose and we do have trunk enabled port configuration at switch level to handle these traffic based on VLAN Packet shaking mechanism

FlexFebric architectural limitation

Note – Each FlexNIC on a physical port can be assigned to a different vNet or remain unassigned, but multiple FlexNICs on a single port cannot be assigned to the same vNet. This is an architectural limitation and will not change (Adding the same vNet multiple times on same single port  looks as a constraint by architectural design & limitation of Flexfebric) means, same vlan like A96 and B96 ( A & B are SUS) cannot be used twice in same profile.