Monthly Archives: September 2010

VMware – Configure Software iSCSI Multipathing to VMware vSphere 4.x


iSCSI Software Initiator – In respective to VMware vSphere 4.x, the iSCSI software initiator code is re-written for better performance. Since VMware vSphere 4.x is running on linux kernel 2.6 and previously VMware ESX was designed to run on kernel 2.4 so a huge change on drivers list J

Subsequently most important factor is that Jumbo Frames (MTU – 9000) can be enabled on this software initiator. Jumbo Frames enabled you to send the larger packets to be transferred between VMware vSphere 4.x servers and SAN for enhancing performance and functionality.

One important thing, I would like to emphasized over here is that Jumbo Frames must be supported at the networking layer ( supported by Layer-3 Switch) – Please check it out or consult your Network Administrator – J

Note – I am not going to cover Jumbo Frames here.

Since all of you would be aware that iSCSI initiator works on TCP session. VMware vSphere ESX 4.x gives the opportunity for Multi-pathing I/O (MPIO) between the VMware vSphere ESX 4.x and SAN environment. This allow the multiple connections to be concurrently used to allow for better performance but keep in the mind that it works on TCP session to create a number of VMkernel Port , you wish to use for MPIO.

Make sure that the VMkernel only have a 1 nic active at a time & rest must be moved on “Unused Adapter

Here I would like to configure 4 VMkernel port with 2 nics –

Note – There can be maximum of 8 connections (iSCSI VMkernel) to a single volume for iSCSI MPIO while we are using only 4 VMkernel port.

I am going to illustrate with command line interface only. VC GUI will be covered sometime later.

1- Adding vSwitch2 Virtual Switch

esxcfg-vswitch -a vSwitch2

2 – Verify if Switch is configured

esxcfg-vswitch –l

It gives the listing of vSwitch2 output.

3 – Adding iSCSI VMkernel Ports for MPIO

Configuring iSCSI port does not required Service console & Vmotion but each VMkernel port should have its own IP address for TCP session.

It is highly recommend that IP address for each VMkernel port must have the same subnet as SAN storage.

Each environments can be differ and this is just a explanation with two nics and 4 VMkernel port i.e vmk0-iSCSI-1, vmk1-iSCSI-2, vmk2-iSCSI-3, vmk3-iSCSI-4


esxcfg-vswitch –A vmk0-iSCSI-1 vSwitch2

esxcfg-vmknic –a –i –n vmk0-iSCSI-1

esxcfg-vswitch –A vmk1-iSCSI-2 vSwitch2

esxcfg-vmknic –a –i –n vmk1-iSCSI-2

esxcfg-vswitch –A vmk2-iSCSI-3 vSwitch2

esxcfg-vmknic –a –i –n vmk2-iSCSI-3

esxcfg-vswitch –A vmk3-iSCSI-4 vSwitch2

esxcfg-vmknic –a –i –n vmk3-iSCSI-4


To verify the VMkernel configuration like IP address, please enter the following commands

esxcfg-vmknic –l

4 – Assign Network Adapter to vSwitch2

This nics are the uplink between Virtual Switch “vSwitch2” and Physical Network hardware to handle iSCSI network traffic.

To list all of the adapters in the system run the following commands-

esxcfg-nics –l

This will list all of the adapters in the system. Assign the NICs that are physically connected to the SAN infrastructures and to the vSwitch. The following command assumes that we are assigning vmnic1 & vmnic2 to the vSwitch2

esxcfg-vswitch –L vmnic1 vSwitch2
esxcfg-vswitch –L vmnic2 vSwitch2

Once assigned , run the following command to check the configuration of nics binds against VMKernel port .

esxcfg-vswitch -l

5 – Configuring VMkernel Ports to Physical Ethernet Adapter

From above steps there are 4 VMkernel ports and 2 NICs. This means that each NIC will have 2 VMkernel ports assigned to it. Both vmnic1 and vmnic2 are assigned to each VMkernel port. We need to move one vmnic from each VMkernel port to ‘‘Unused Adapter’ under

vSwitch2 Properties -> VMkernel Port like vmk0-iSCSI-1 properties -> NIC Teaming

so that each VMkernel port i.e vm0-iSCSI-1 has only one uplink.

– Removing the other vmnic for active only one uplink connection i.e moving the other nics on ‘Unused Adapter

esxcfg-vswitch –p vmk0-iSCSI-1 –N vmnic1 vSwitch2

esxcfg-vswitch –p vmk1-iSCSI-2 –N vmnic1 vSwitch2

esxcfg-vswitch –p vmk2-iSCSI-3 –N vmnic2 vSwitch2

esxcfg-vswitch –p vmk3-iSCSI-4 –N vmnic2 vSwitch2

(GUI-Method under VC)

Here are the nexus of VMkernel Port & N

configuration which can be configured under the

vSwitch2 Properties -> VMkernel Port like vmk0-iSCSI-1 properties -> NIC Teaming

|vmk0-iSCSI-1 | nic2 |
|vmk1-iSCSI-2 | nic2 |
|vmk2-iSCSI-3 | nic1 |
|vmk3-iSCSI-4 | nic1 |


1 – Here nic2 is an ‘Active Adapter’ for vmk0-iSCSI-1 & vmk1-iSCSI-2 and nic1 is an ‘Unused Adapter’.

2 – Here nic1 is an ‘Active Adapter’ for vmk2-iSCSI-3 & vmk3-iSCSI-4 and nic2 is an ‘Unused Adapter

Note – Please select ‘Override vSwitch failover order’ checkbox under this if working from GUI console.

To verify that this is done by the following command.
esxcfg-vsswitch –l

6- Configuring VMware iSCSI Software Initiator
I am not going to cover how to configure iSCSI Software Initiator because this is the same as it was previously configured with VMware ESX 3.5

7 – Mapping each VMkernel Port to iSCSI Software Initiator (V- imp)

This is possible with CLI command line.

Note – vmhba33 is iSCSI software Initiator HBA

esxcli swiscsi nic add –n vmk0 –d vmhba33

esxcli swiscsi nic add –n vmk1 –d vmhba33

esxcli swiscsi nic add –n vmk2 –d vmhba33

esxcli swiscsi nic add –n vmk3 –d vmhba33

To verify that all of the vmk## are bound properly to the vmhba33 run the following command:

esxcli swiscsi nic list –d vmhba33

8 – Configure Round Robin path selection for all iSCSI

Round Robin is provided to allow administrators to create policies that will split I/O to a particular LUN over multiple paths. All the path would be active in RR path selection like status – Active ( I/O)

The following method must be applied for each iSCSI datastore on VMware vSphere ESX 4.x

1. In vSphere Client, select the Storage view from the ESX host’s Configuration tab.
Select one of your iSCSI datastores and click the Properties link.
Click the Manage Paths button.
Select Round Robin (VMware) from the Path Selection menu and click Close.