vCenter Bind Request Failed Error 49 persists after Password Reset?

Sometimes you run into a vCenter issue where the situation is not just broken, but dangerously broken.

This is one of those cases.

If your vCenter is throwing vmdird authentication failures like the ones below, and the normal machine account password reset procedure does not fix it, your vCenter may already be in a very bad state:

err vmdird t@140245530842880: Bind Request Failed (x.x.x.x) error 49: Protocol version: 3, Bind DN: "cn=vcsa,ou=Domain Controllers,dc=vsphere,dc=local", Method: SASL
err vmdird t@140245530842880: SASLSessionStep: sasl error (-13)(SASL(-13): authentication failure: client evidence does not match what we calculated. Probably a password error)

Let me be very clear: this is a serious recovery situation.

Any remediation from this point is potentially destructive and provided as-is with no guarantee of success. You should assume that services may fail to return cleanly, additional repair steps may be required, certificate repair or endpoint re-registration may become necessary, and in the worst case full recovery may fail.

So do not treat this as a casual “run a command and move on” type of issue.

Continue reading vCenter Bind Request Failed Error 49 persists after Password Reset?

How To Easily Master vSphere Desired-State Cluster Configuration Files

Introduction

VMware By Broadcom has moved away from the old, clunky Host Profiles in favor of the modern Desired State Cluster Configuration. This “Desired-State” model is fantastic for consistency, but it introduces a new challenge: managing the massive JSON documents that define your cluster’s state.

When you need to scale a cluster, you’re often stuck manually editing host-specific overrides for IPs and hostnames inside a complex JSON structure. I built ClusterConfigForge to turn that manual grind into a streamlined, automated workflow.

The Challenge: Scaling Desired-State Configurations

In the Desired-State model, the entire configuration for a cluster is managed as a single document. While this is great for avoiding “configuration drift,” updating unique host details (like Management/vMotion IPs) for a 32-node cluster still requires tedious, repetitive data entry.

If you’re a consultant or a lead admin, you don’t want to spend your afternoon copy-pasting IPv4 addresses into a text editor. You want a tool that understands the structure and does the heavy lifting for you.

Continue reading How To Easily Master vSphere Desired-State Cluster Configuration Files

Cannot export ISO from vLCM cluster image

When you try to export an ISO file in VMware vCenter from a cluster using single cluster image with vLCM. You will get the following error:

A general system error occurred: Error occurred while exporting ESXi image and/or image document.

The error is accompanied with an error in the vmware-vum-server-#.log file in /var/log/vmware/vmware-updatemgr/vum-server catalog like the following:

2023-06-14T12:21:23.882Z error vmware-vum-server[09453] [Originator@6876 sub=VumVapi::Lib::Utils] [ExportTask 92] Failed to export cluster image from depot. errorCode: 99

In my case I was able to export it as a zip bundle and the corresponding json configuration file exported successfully as well.

The problem lies with vendor signatures, and vmware does not currently have a solution for this unfortunately except that it normally helps to remove the vendor packages attached to the cluster.

https://kb.vmware.com/s/article/91237

More information is available here: https://communities.vmware.com/t5/vCenter-Server-Discussions/Cannot-export-vLCM-image-if-you-use-a-custom-SSL-cert-Non/td-p/2881200/page/2

How To Make vRO Execute Python Code Blocks

vRealize Orchestrator (vRO) is a powerful automation platform that enables you to automate and orchestrate various IT processes, including the execution of Python scripts. In this article, we will show you how to set up vRO to run Python scripts and provide some tips along the way.

Prerequisites

Before you can run Python scripts in vRO, you need to have the following:

  • A working installation of vRealize Orchestrator (8.10.2+).
  • A vCloud Suite Advanced or Enterprise license. You cannot run Python scripts with the standard vCenter license for vRO

Setting up vRO to run Python scripts

To set up vRO to run Python scripts, follow these steps:

  • Open the vRealize Orchestrator client and log in with your administrator credentials. (https://<servername>/orchestration-ui)
  • In the main menu, go to the “Assets” and select “Environments” from the submenu.
  • Click the “New Environment” button to create a python3.7 environment.
  • In the “General” tab, enter a name and a description for the environment.
Continue reading How To Make vRO Execute Python Code Blocks

Reconnect ESXi host after reinstall

After a VMware ESXi host reinstall reconnecting your host is not as easy as changing the state of the host by using the Set-VMHost -State command in VMware PowerCLI. The reason is that your host has gotten a new certificate. So to do this scripted you need to retrieve the SSL thumbprint and specify that during the reconnect.

Continue reading Reconnect ESXi host after reinstall

Automating VMware Workstation LAB

I am often working with quite large test environments. Powering on ESXi hosts with nested VMs can be a pain when you need to get it running quickly.

Here are some of my tricks to automating VMware Workstation

Continue reading Automating VMware Workstation LAB

Nested or Native

Should you buy dedicated hardware or a OP workstation for you next testing environment. If you are not sharing it with others, this might be useful for you.

History

For many years now VMware Workstation has been my secret weapon an daily tool for just about everything in regards to customer remote connections, test environments as so on.

Recently I needed to do some advanced testing with NSX-V and NSX-T. This required a lot more power than what I normally use so I needed to upgrade my testing platform.

The consideration is always with these things. How much are you going to invest, and what are the benefits. For a long time I have been considering buying 4 Intel NUC PC’s for doing these tests, but the problem is that to get a real setup that is flexible you need to invest a lot. Also it is not very flexible as you need to maintain them, and reinstall them everytime you need to play with a newer or older version.

Continue reading Nested or Native

VMware HA Cluster: Set-Advanced HA Settings

Recently a adjustment was made to the vCenter Cluster HA Agent timeout Settings.

Ref: https://kb.vmware.com/s/article/2017778

Performing a Reconfigure for VMware HA operation on a primary node causes an unexpected virtual machine failover

This calles for at script, because there is no way we are going to do this by hand! Feel free to use or alter the script, just remember. It is all done at your own risk.

Continue reading VMware HA Cluster: Set-Advanced HA Settings

Devices that are only supported in 6.7

Back in 2017 VMware changed their policy on VMKlinux Inbox Drivers. This has now come into effect from vSphere 7.0 and onwards.

Unfortunately this means that some otherwise supported hardware platforms, now are having support issue. And you might see the following error in vCenter Skyline Health:

Devices that are only supported in 6.7 or earlier by a VMKlinux inbox driver. This support has been removed in 7.0

This is the case for many Lenovo SR650 servers if they are booting from a SD card, as many will be configured with the Lewisburg SATA AHCI Controller.

Continue reading Devices that are only supported in 6.7

Migrate Tags from one vCenter to another

When upgrading to vSphere 7 or any other version, you might choose to create a brand new vCenter instead of migrating the old one. But what about folder structure, tags, distributed switches and so on.

Here I will demonstrate how you can easily migrate your tags from one vCenter to another using VMware PowerCLI.

Continue reading Migrate Tags from one vCenter to another