Have you ever worked hard to get things set up the way you want only to find out later that all the work you just did got changed back to default.  This can be enough to make you scream.  For today’s discussion I am talking about load balancing your HBA’s on your ESX servers. You set up pathing to be balanced across your HBA’s but did you know that once you reboot your ESX server all the pathing work you just did becomes mute because it just got set back to the default. If you have 2 HBA’s you could add this script to rc.local so when the ESX server boots it will balance the LUN automatically.  This script has three different options show current config, show what the changes will be and to execute the change.  This could be good for logging information before the change

A quick update.  It was quickly mentioned to me on twitter that this issue may only be ESXi and not the full blown ESX.  Next time someone reboots an ESX host let me know.

PREVIOUS=”vmhba0″
echo “Starting HBA Balance…..”
case “$1″ in
  -c)
        # Show current config
        echo >> /root/PostInstall/PostInstall.log
        echo “****** CURRENT HBA/LUN CONFIG ******”
        echo
        for LUN in $(esxcfg-vmhbadevs | tail +2 | awk ‘{print $1}’)
        do
           esxcfg-mpath -q –lun=${LUN} | grep FC
           echo
        done
        ;;
  -p)
        # Show new config
        echo
        echo “****** PREVIEW NEW BALANCED HBA/LUN CONFIG ******”
        echo
        for LUN in $(esxcfg-vmhbadevs | tail +2 | awk ‘{print $1}’)
        do
           CURRENT=$(esxcfg-mpath -q –lun=${LUN} | grep FC | grep “preferred” | awk ‘{print $4}’ | awk -F “:” ‘{print $1}’)
           if [[ ${CURRENT} = ${PREVIOUS} ]]
           then
                 NEW=$(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v “preferred” | awk ‘{print $4}’ | awk -F “:” ‘{print $1}’)
                 if [[ ${NEW} = vmhba1 ]]
                 then
                    echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep ${NEW} | awk ‘{print ” ” $1, $2, $3, $4, $5}’) “active preferred”
                    echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v ${NEW} | awk ‘{print ” ” $1, $2, $3, $4, $5}’)
                 else
                    echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v ${NEW} | awk ‘{print ” ” $1, $2, $3, $4, $5}’)
                    echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep ${NEW} | awk ‘{print ” ” $1, $2, $3, $4, $5}’) “active preferred”
                 fi
                 PREVIOUS=${NEW}
                 echo
           else
              if [[ ${CURRENT} = vmhba1 ]]
              then
                  echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep ${CURRENT})
                  echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v ${CURRENT})
              else
                  echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v ${CURRENT})
                  echo $(esxcfg-mpath -q –lun=${LUN} | grep FC | grep ${CURRENT})
              fi
              PREVIOUS=${CURRENT}
              echo
           fi
        done
        ;;
  -e)
        # Execute new config
        for LUN in $(esxcfg-vmhbadevs | tail +2 | awk ‘{print $1}’)
        do
           CURRENT=$(esxcfg-mpath -q –lun=${LUN} | grep FC | grep “preferred” | awk ‘{print $4}’ | awk -F “:” ‘{print $1}’)
           if [[ ${CURRENT} = ${PREVIOUS} ]]
           then
              NEW=$(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v “preferred” | awk ‘{print $4}’ | awk -F “:” ‘{print $1}’)
              NEWPATH=$(esxcfg-mpath -q –lun=${LUN} | grep FC | grep -v “preferred” | awk ‘{print $4}’)
              esxcfg-mpath –lun=${LUN} –path=${NEWPATH} –preferred
              PREVIOUS=${NEW}
           else
              PREVIOUS=${CURRENT}
           fi
        done
 
        echo
        echo “****** NEW HBA CONFIG *****”
        echo
        for LUN in $(esxcfg-vmhbadevs | tail +2 | awk ‘{print $1}’)
        do
           esxcfg-mpath -q –lun=${LUN} | grep FC
           echo
        done
 
        echo
        echo “****** \”active\” flag will be moved after disk activity *****”
        echo
 
        ;;
 
  *)
        echo $”Usage: balancelun {-c (current config)| -p (preview new config) -e (implement new config)}”
        exit 1
esac

There are some powershell option to do this but ESX itself would not be able to automatically launch these scripts so you lose the automation of doing it automatically but powershell, I think, is the way of the future so here you go.

$Mypolicy = “rr”

Get-Datastore | %{(Get-View $_.ID).Info.Vmfs.Extent[0].DiskName} |%{

  $diskname = $_

  Get-VMHost | %{Get-View (Get-View $_.ID).configmanager.storageSystem} | %{

    $mpathpolicy = New-Object vmware.vim.HostMultipathInfoLogicalUnitPolicy

    $mpathpolicy.policy = $Mypolicy

    $_.SetMultipathLunPolicy($diskname,$mpathpolicy)

  }

}

Here is another really good powershell script to balance odd LUNS to one HBA and even LUNS to another HBA.  This script was written by Rob Mokkink rob@mokkinksystems.com

 #PURPOSE: SET ODD LUN’s PREFERRED PATH TO vmhba1 AND EVEN LUN’s PREFERRED PATH TO vmhba2
# AND SET THE POLICY TO FIXED
#CREATED BY: ROB MOKKINK rob@mokkinksystems.com

#FORCE TO LOAD VMWARE POWERSHELL PLUGIN
[Reflection.Assembly]::LoadWithPartialName(”vmware.vim”)

#ON ERROR CONTINUE
$erroractionpreference = “SilentlyContinue”
#VARIABLES
$policy = new-object VMware.Vim.HostMultipathInfoFixedLogicalUnitPolicy
$policy.policy = “fixed”
$UNEVEN = “vmhba1″
$EVEN = “vmhba2″
$VCHOST = “localhost”
#CONNECT TO THE SERVER
Get-viserver -Server $VCHOST

#GET ALL THE ESX HOST
$ALL_ESXHOSTS = get-vmhost | where { $_.State -eq “Connected” }

# LOOP THROUGH THE ARRAY
foreach ($i in $ALL_ESXHOSTS)
    {
     $ESXHOST = $i.name
     Get-VMhost $ESXHOST
     
     #ERROR HANDLING
     if (! $?)
     {
      write-host “ERROR CONNECTING TO HOST: $ESXHOST [ERROR]” -ForeGroundColor Red
     }
     
     else
     
     {
      write-host “CONNECTED TO HOST: $ESXHOST [OK]” -ForegroundColor Green
     } 
     
     
      #GET THE STORAGE PATHS
      $GET_HOST = Get-VMhost $ESXHOST
      $HOST_VIEW = Get-View $GET_HOST.id
      $STORAGESYSTEM = get-view $HOST_VIEW.ConfigManager.StorageSystem
       #LOOP THROUGH THE LUN’S ADD THEM TO THE ARRAY
       $ARRLUN = $STORAGESYSTEM.StorageDeviceInfo.MultipathInfo.lun | where { $_.Path.length -gt 1 }
       
        if ( $ARRLUN -eq $null )
         {
          
          write-host “ESXHOST: $ESXHOST HAS NO LUNS [ERROR]” -ForeGroundColor Red
         }
         
        else
         {
       
                #LOOP THROUGH LUN’s
                foreach ($LUN in $ARRLUN)
         
                 #CHECK IF THE LUN IS EVEN OR UNEVEN AND SET THE POLICY
                 { $CHECK_LUN = $LUN.id.split(’:')
                 $CHECK_LUNID = $CHECK_LUN[2]
         
                  if($CHECK_LUNID -band 1)
                   { Write-Host LUN: $LUN.id = “ODD LUN NUMBER” -ForegroundColor Blue
                    
                    #GET THE PROPER VMHBA
                    $GET_VMHBA = $LUN.Path | where {$_.Name -match $UNEVEN}
                    $policy.prefer = $GET_VMHBA.Name
                    
                    #SET THE NEW PATH + POLICY
                    $NEW = “NEW PATH FOR LUN:” + ” ” + $LUN.id + ” ” + “IS:” + ” ” + $policy.prefer
                    $storageSystem.SetMultipathLunPolicy($LUN.id, $policy)
                    
                     #ERROR HANDLING
                     if (! $?)
                     {
                     
                      write-host $NEW “[ERROR]” -ForegroundColor Red
                     }
                     
                     else
                     
                     { 
                     
                      write-host $NEW “[OK]” -ForegroundColor Green
                     
                      
                     }
                    
                    
                   }
                     
                  else
                   {  Write-Host LUN: $LUN.id = “EVEN LUN NUMBER” -ForegroundColor Blue
                    
                    #GET THE PROPER VMHBA
                    $GET_VMHBA = $LUN.Path | where {$_.Name -match $EVEN}
                    $policy.prefer = $GET_VMHBA.Name
                    
                    #SET THE NEW PATH + POLICY
                    $NEW = “NEW PATH FOR LUN:” + ” ” + $LUN.id + ” ” + “IS:” + ” ” + $policy.prefer
                    $storageSystem.SetMultipathLunPolicy($LUN.id, $policy)
                    
                     #ERROR HANDLING
                     if (! $?)
                     {
                     
                      write-host $NEW “[ERROR]” -ForegroundColor Red
                     }
                     
                     else
                     
                     { 
                      write-host $NEW “[OK]” -ForegroundColor Green
                     }
                   }
                 }
         }
         
         }

Happy Scripting!!!

administration, Automation, bash, ESX, powershell, Scripting, Steve Beaver, Virtual Center, virtual infrastructure, VMTN, vmware

This entry was posted on Tuesday, January 20th, 2009 at 12:03 pm and is filed under Virtual Tech. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.