vSphere 4.0 with Software iSCSI and 2 Paths

Posted: July 17, 2009 in Tips and Tricks, VMware

One thing about the ISCSI initiator in 3.x was even with 2x adapters bound to the vSwitch, the iSCSI adapter would only ever show a single path to the lun, and the only time it would use the secondary adapter is if the primary one failed.

Now something I noticed in the new vSphere iSCSI SAN configuration guide was now we are able to bind 2 network adapters via a single vSwitch with two port groups, or bind 2 x vSwitches with a single adapter and port group each to the built in software iSCSI initiator.

I started up my vSphere lab system and took some screenshots and thought id share a few of the key steps. Ill be honest I’m not great at sitting down and following documents step by step so of course I got an error which ill also explain and show what  was doing wrong.

Step 1.

So here you ll see Ive started off with only vSwitch0 with the service console attached.

single VMKern

Step 2.

Next I goto Add Networking and choose to add VMKernel and then add vmnic1 and vmnic2 to the new vSwitch.

two VMKern

 

Step3.

Now that I have both adapters bound to vSwitch1 I need to go and make a small change and set the secondry adapter for each port group as “unused”. If you dont do this then later on in the section where we add both port groups to the iSCSI Initiator you ll get the following error “Add Nic Failed in IMA”

The next step is for me to select the first port group which I called iSCSI_VMKernel1 and then select the NIC TEAMING tab. Then Select the override option and move the secondary adapter to unused, note for the next port group it will be the other way around.

overide_1

 

Step4.

So now Im going to go and change this for iSCSI_VMkernel2. Note this time its vmnic1 that is set to unused.

overide2

 

Step.5

So that’s the configuration of the vSwitch and port groups done. Now lets bind both vmk0 and vmk1 to the iSCSI Initiator using esxcli.

esxcli swiscsi nic add -n vmk0 -d vmhba33

esxcli swiscsi nic add -n vmk1 -d vmhba33

esxcli_add

 

 Step6.

Now I want to confirm that everything worked as expected.

esxcli swiscsi nic list -d vmhba33

esxcli_list

Yep thats all looking good.

Step7.

Now lets connect my vSphere server to the iSCSI Target.

iscsi_add

 

Step8.

Now  lets take a look after selecting YES to a rescan.

Great now we have two paths to the lun.

Now how much better is that then ESX 3.x, Oh and dont forget to go and change the path policy to “Round Robin” which makes both path active.

2paths

Advertisements
Comments
  1. Very nice article! Great info, thanks for sharing!

  2. Hersey says:

    Great post!

    Are you able to do the same thing using vDS? Just using the dvUplink ports instead of the physical nic on each host?

    Going to give it a try myself, but just curious if you had tried it.

    Thanks
    Hersey

    • Brian Norris says:

      Hi There, Ive been playing and im not having much luck getting it to work. Ill be sure to post if I do figure it out though.

      Regards
      Brian

  3. d_glynn says:

    Can you show how to enable jumbo frames?

    • Brian Norris says:

      Sure, Here you go.

      1 Use the vicfg-vswitch -m command in the VMware vSphere CLI to set the MTU size for the vSwitch.

      This command sets the MTU for all uplinks on that vSwitch. Set the MTU size to the largest MTU size among all the virtual network adapters connected to the vSwitch.

      2 Use the vicfg-vswitch -l command to display a list of vSwitches on the host and check that the configuration of the vSwitch is correct.

  4. Shawn says:

    I assume if you used two vSwitches with one pNIC each, and one iSCSI vmkernel each, that would work just as well?

    • Brian Norris says:

      Hi Shawn.
      Yes thats right. My understanding is that with a single Network you use just one vSwitch, and for environments where the switches are on different Network segments, you create 2 vSwithes, one for each segment.

      Regards
      Brian

  5. In term of performance, how much different after the changes?

    • Brian Norris says:

      Hi “Malaysia VMware”

      Thats a good question and something Ive been meaning to do, currently this was setup on the vSpehere systems running on my laptop under workstation 6.5.2 so Ive not really had a chance to put my physical systems through testing. Ill be sure to post when I get around to this.

      Regards
      Brian

  6. […] information on how to use software iSCSI with multiple paths in VMware vSphere 4. This site has a good write-up with screenshots, and Rich Brambley discusses it on his site as well. I haven’t had the opportunity to try […]

  7. […] paths. This is newly supported within vSphere, see this post on setting up multiple paths : https://goingvirtual.wordpress.com/2009/07/17/vsphere-4-0-with-software-iscsi-and-2-paths/ Categories: Uncategorized Comments (0) Trackbacks (0) Leave a comment […]

  8. […] am by Brian Norris By far one of the more popular posts Ive done over the last few months has been this post where I showed how the iSCSI initiator in vSphere 4 could be configured to provide multiple […]

  9. David says:

    Hi,

    I have followed this article to a T and everytime when I enter
    esxcli swiscsi nic add -n vmk0 -d vmhba33 I get “add nic failed in ima”. I see that it is related to the following: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1009450&sliceId=1&docTypeID=DT_KB_1_1&dialogID=50766520&stateId=1%200%2050768038 and the port in each is in un-used.

    I have tried removing the vswitch and re-create and still the same. I did have connectivity to the SAN prior to trying to enable multipathing. I have since removed all connections to the SAN and I still get the same error. I am running 4.0 update 1.

    any ideas??

  10. David says:

    a reboot fixed it

  11. Bob says:

    Great article,

    I played with these settings in our test enviroment. But now I want to remove 1 binding is this also possible??

    regards Bob

  12. Ian says:

    Hi Brian. Any luck getting iSCSI working with the dvSwitch yet?

    • Brian Norris says:

      Hi Ian, not yet… Ive been meaning to do this in my spare time… Im just downloading the latest version of the SIM and hope to get this done shortly.

      Thanks
      Brian

  13. […] vSphere 4 with Software iSCSI and 2 Paths […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s