Category: vSphere

vCenter services not starting after 6.7 Update 3f upgrade

After upgrading a 3 vCenter Enhanced Link Mode environment a customer experienced the following error:

Server is at a higher functional level (1) than partner (<partner vCenter server>)(0) and cannot perform at a lower level.

When checking the domain functional level of each of the three servers, they all state that they are at level 1. The other server are starting like normal.

One server is not starting the vmdir service and it is also that service that is reporting the error. Most of the other services on a vCenter is dependant on this the vmdir.

I do not have a solution as of now. I might have a reason for the issue, and I might have a workaround. VMware is currently trying to figure out how to fix this.

vSphere 7 – Return of the blue folders

Back in the day it was customary to divide your VMs into three “blue” folders. I was not uncommon to see the three folders named High Priority, Normal Priority and Low Priority, and then people would distribute the VMs into those folders. The reason they were called blue folder was because it was really resource pools created in a DRS cluster in the Hosts and Clusters view in vCenter. And they were not made to logically divide you VMs into “folders”.

Spoiler Alert
New DRS Feature – Scalable SHares

The problem with this approach is that unless you regularly reconfigured these resource pools you would get unexpected performance. I was not uncommon for an installation to have 5-10% of the VMs in the high priority “folder”, 80% in the normal priority “folder” and 5-10% in the low priority “folder”. The problem with this is that if you then select High, Normal and Low shares for these folders, you will not get the performance you would expect.

Powershell: Find largest VM disk

Sometimes you need to find the largest virtual disk. Lets say if you are sizing LUNs for datastores.

Here is a script that help you do that.

Requirement are powershell and the VMware.PowerCLI module.

Use it at your own risk.

Import-Module VMware.PowerCLI

Connect-VIServer <vCenter Name>


Function Get-LargestDisk {
  param(
    $Datastores=$null
  )
  $largest = 0

  if ($Datastores -eq $null) {
    Write-Host "Searching through all VMs."
    $vms = Get-VM
  } else {
    Write-Host "Searching through VMs on datastores: $Datastores"
    $vms = $Datastores | Get-VM
  }

  foreach ($vm in $vms) {
    $hdds = $vm | Get-HardDisk

    foreach ($hdd in $hdds) {
      $size = $hdd.CapacityGB

      if ($size -gt $largest) {
        Write-Host "Found a larger VM: $vm Size: $size GB"
        $largestVm = $vm
        $largest = [math]::Round($size)
      }
    }
  }
  Write-Host "Largest Disk: $largest GB Largest VM: $largestVm"
}

Get-LargestDisk -Datastore (Get-Datastore V7000*)

Powershell: Migrate Standard Portgroups

Hi, just wanted to share this piece of code with you. This short script creates a mirror of virtual portgroups from one vSphere ESXi host to another.

I only takes the name and vlan id into account, so all policies, nics and other settings are not migrated at this time, but it is easy to do. I however does not always want that.

You need VMware.PowerCLI module for it to work.

Use it at you own risk.

Import-Module VMware.PowerCLI

connect-viserver <vCenter Name>

$srcHost = Get-VMHost <Source Host Name>
$dstHost = Get-VMHost <Destination Host Name>

# Change the source and destination switch names if you need to
$srcSwitch = $srcHost | Get-VirtualSwitch -Name vSwitch0
$dstSwitch = $dstHost | Get-VirtualSwitch -Name vSwitch0

$srcPGs = $srcSwitch | Get-VirtualPortGroup
$dstSwitch = $dstHost | Get-VirtualSwitch -Name vSwitch1 -ErrorAction:SilentlyContinue
if ($dstSwitch -eq $null) {
  $dstSwitch = $dstHost | New-VirtualSwitch -Name $srcSwitch.Name
}

foreach ($srcPg in $srcPGs) {
  $pgName = $srcPg.name
  $pgVlan = $srcPg.VLanId

  $pgExists = $false
  foreach ($dstPg in $dstPGs) {
    if ($dstPg.Name -eq $pgName) {
      $pgExists = $true
    }
  }
  if (-not $pgExists) {
    #Write-Host "Creating portgroup $pgName with vlan $pgVlan"
    $dstSwitch | New-VirtualPortGroup -Name $pgName -VLanId $pgVlan
  }
}

ESXi 6.7 PSOD with qfle3 driver version above 1.0.69.1

Had a ESXi PSOD today. That does not happened that often, so I was quite surprised to find out that it was not a hardware related issue that was the root cause.

VMware did an analysis of the memory dump, and it turned out to be a faulty driver. That made sense since the PSOD often comes from drivers og agents when it is not a hardware issue.

The PSOD i got was the following:

#PF Exception 14 in World xxxxxxx:vmnicX-pollw IP xxxxxxxxxx addr xxxxxxxx

Error: cannot install the vcenter agent service. cannot upload agent after vCSA upgrade

I was just updating a vCenter server and some ESXi hosts, but after running the vCenter update I found vCenter full of HA Agent install fails. To stop this fail loop, I turned off VMware HA while figuring out what was wrong.

Error: cannot install the vcenter agent service. cannot upload agent after vCSA upgrade

Update Manager ELX_bootbank_elx-esx-libelxima.so driver conflict

Just provisioned the HPE ESXi 6.7 Update 3 custom OEM image onto some HP DL560 Gen10 servers.

After I updated the servers using update manager and the HPE vibsdepot I ran into problems. Turns out there is a conflict between the VMware provided driver and the HPE provided driver.

The result is that I cannot install all updates to satisfy compliance.

Checking the esxupdate.log file on the ESXi hosts I get the following error:

ValueError: VIBs ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03 and ELX_bootbank_elx-esx-libelxima.so_12.0.1108.0-03 have unequal values of the 'payloads' attribute: '[elx-esx-libelxi: 1602.936 KB]' != '[elx-esx-libelxi: 1493.833 KB]'

How to upgrade to VMware Tools 10.3.10

So VMware released a security advisory (VMSA-2019-0009) about vulnerable VMware Tools.

In short it is recommended to upgrade all VMware Tools to at least version 10.3.10 (Build 10346) which is the newest at the time of writing.

After patching ESXi to the latest build, you might realize that the VMware tools you are left with, after upgrading the tools, in a VM, is 10.3.5 (build 10341) Continue reading How to upgrade to VMware Tools 10.3.10

VMware vCenter Appliance Installer on Ubuntu Linux

I have found that, when working with VMware vSphere, there are many benefits of running Linux on your workstation.

Things like tight integration with SSH, ISO files, API’s, Python are all things that make your life easier not only on Linux, but also when working with VMware.

Continue reading VMware vCenter Appliance Installer on Ubuntu Linux

VMware PowerCLI on Powershell Core

I recently installed Powershell Core om Ubuntu 18.04, and after installing the PowerCLI module. I ran into an error.

The error is not an uncommon one, but on Windows the error message makes a lot more sense, so I just wanted to let you know what this error actually means.

The error you might get when you try to connect to your vCenter server using the connect-viserver is the following:

Connect-VIServer : 9/27/18 10:41:37 AM	Connect-VIServer		The SSL connection could not be established, see inner exception.	
At line:1 char:1
+ Connect-VIServer <servername>
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Connect-VIServer], ViError
+ FullyQualifiedErrorId : Client20_ConnectivityServiceImpl_Reconnect_SoapException,VMware.VimAutomation.ViCore.Cmdlets.Commands.ConnectVIServer

The only hint here is “The SSL connection could not be established…”

This actually means that you do not have a valid certificate. And if you want to connect to vCenter without a valid certificate, you have to allow this.

You can either change you vCenter certificate to a trusted one, which is the correct solutions or you can ignore invalid certificates, which circumvents all security, but makes it work right now.

Set-PowerCLIConfiguration -InvalidCertificateAction:ignore

Please comment if this was helpful.