vSAN – Downgrading NVMe driver in ESXi 6.7 Update 1

Recently ran into a HPE Proliant m510 server running vSAN, where vSAN complained that the controller driver for the NVMe disk where too new.

The health error said that the current driver nvme (1.2.2.17.-1vmw.670.1.28.10302608) was to new and the recommended driver was nvme (1.2.1.34-1vmw.670.0.08169922)

Downgrading is not always a breeze. When going to VMware compatibility guide, the NVMe disk is supported for vSAN 6.7 Update 1, and there are no download links to a specific driver, so how do you get the old driver?

After some investigation I found that the correct driver was delivered as a patch for 6.5, and in the 6.7 GA version.

Update:

You can follow the guide below or you should be able to install the new drivers directly with the following command as mentioned in a comment to this article.

# Run the following commands on the host.<br><br>esxcli software vib install -v https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/nvme/VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922.vib

esxcli software vib install -v https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/vmware-esx-esxcli-nvme-plugin/VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922.vib

#reboot

If the above links stops working you can follow the original guide:

As I did not want to download a 6.5 driver at install in my 6.7 Update 1 environment, I needed to figure out how to get the driver for 6.7.

The way to get the driver is to download the ESXi 6.7 GA image from VMware download repository in zip format. It you extract the image you will find the files you need.

VMware-ESXi-6.7.0-8169922-depot/vib20/nvme/VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922.vib
VMware-ESXi-6.7.0-8169922-depot/vib20/vmware-esx-esxcli-nvme-plugin/VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922.vib

I figure the nvme-plugin version goes along with the nvme driver version, but as always. EVERYTHING YOU READ HERE ARE PERFORMED AT YOUR OWN RISK!

Copy the two files to the ESXi hosts using scp or any method you find convenient. I placed the files in the /tmp directory on each host, and put them in maintenance and installed the vibs one by one. After each install I rebooted the host even though the vib claims it is not necessary.

#Before install you can check the current version by running the command.
esxcli software vib list | gep nvme

# Install the vibs by running the commands:

esxcli software vib install -v /tmp/VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922.vib

esxcli software vib install -v /tmp/VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922.vib

After the install, the version on my install looked like this:

We are done…

I hope that the information is useful, please feel free to leave any comment.

24 thoughts on “vSAN – Downgrading NVMe driver in ESXi 6.7 Update 1”

  1. Hi, this has been annoying me ever since 6.7 U1 came out that I decided to make it a bit easier to find these vibs and install them:

    (Replace h***s with the standard secure web prefix)

    1. Enable ESXi Shell and SSH access and log in to your host
    2. Run this:
    esxcli software vib install -v h***s://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/nvme/VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922.vib
    esxcli software vib install -v h***s://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/vmware-esx-esxcli-nvme-plugin/VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922.vib
    3. Reboot

    Hope this helps anyone that finds this post until the vib is fixed.

    1. Thanks!

      When you’re stuck with ESXi free license like me, offline downloads / bundle are not available for download on VMWare website. But these cli commands still work.

    2. Thank you for the direct links! This solved my problem, I wasn’t able to detect my sx8200 pro, but with these steps it is now detected and ready for use. Thanks again!

  2. I just did this with 6.7u2 that would not see my Asus pci-e NVME m.2 controller (ASUS HYPER M.2 X16 CARD V2 – interface adapter – M.2 Card – PCIe 3.0 x16 (p/n: HYPER M.2 X16 CARD V2)) and 1tb adata m.2 (ADATA XPG SX8200 Pro – solid state drive – 1TB), worked perfectly after a reboot, thanks.

  3. damn…thanks for that post and your comment steven. I had a similar problem getting my SX8200 Pro going on 6.7u2. After reading your comment I finally decided to go for the downgrade and, of course, it worked as well.
    let’s hope vmware opens up their driver for consumer-grade devices again…

    1. Hello Michael.

      I have an SX8200 Pro in a new 6.7u2 host build

      can you elaborate on what you did specifically for this drive/driver?

      1. You literally have to do what is listed in the very first comment to get it working. Connect to your ESXi host with Putty or my favorite Bitvise client and follow the instructions as listed. worked for me.

  4. Hmm, I followed the instructions and ended up with the following driver state as expected:

    nvme 1.2.2.27-1vmw.670.2.48.13006603 VMW VMwareCertified 2019-04-14
    vmware-esx-esxcli-nvme-plugin 1.2.0.32-0.0.8169922 VMware VMwareCertified 2019-07-29

    But my SX8200 Pro 256GB is still not recognized. Gigabyte H170N + 6.7.0 here.

    1. The drive is correctly detected at least:
      esxcli nvme device get -A vmhba2 spits out all the right device info.

  5. So using the two packages I got the controller to be detected and it shows two devices in the PCI Device list, however it only detects one of the XPG8200 Pro 2TB drives. Why would it only detect one?

    1. How does it look in the BIOS, it there a controller interface where you can confirm that the disks are detected?

  6. Whew! Thanks for the write up! Saved my day. Well at least hours of rolling back to 6.5 😉

  7. Interesting

    I followed this guide when I first installed 6.7_u3 and it worked great. (ADATA SX8200PNP)

    I upgraded to ESXi-6.7.0-20191204001-no-tools then added VMware_locker_tools-light_11.0.1.14773994-15160134.vib and rebooted and the NVMe was no longer detected

    I followed these instructions (esxcli software vib install -v https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/esx/vmw/vib20/vmware-esx-esxcli-nvme-plugin/VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922.vib)

    and I ended up with

    nvme 1.2.2.28-1vmw.670.3.73.14320388
    vmware-esx-esxcli-nvme-plugin 1.2.0.32-0.0.8169922

    My device is detected but offline
    [root@esxi1:~] esxcli nvme device list
    HBA Name Status Signature
    ——– ——- ———————
    vmhba0 Offline nvme Mgmt-nvme00010000

    Wonder if I need to downgrade the nvme vib as well?

  8. Yes! It worked for my configuration NUC8i7BEH2, Samsung M471A4G43MB1-CTD 2×32 GB DDR4, ADATA XPG SX8200 Pro NVME 1TB M.2 and Samsung 860 QVO 4TB SSD.

    The Samsung 860 QVO SSD was detected during the ESXi 6.70 U3 installation. (And the NVME M.2 not)

    After running these commands (+restart) the NVME M.2 SSD was found.

    Thank you 🙂

  9. Hi,

    It seems it doesn’t work with ESXi7. Applying both commands caused my E200-8D to lose network adapters. Any idea how to find a driver to make ESXi7 to see my XPG8200 Pro? Removed VIBs are below that caused problem for ESXi7.

    VIBs Installed: VMW_bootbank_nvme_1.2.1.34-1vmw.670.0.0.8169922
    VIBs Removed: VMware_bootbank_native-misc-drivers_7.0.0-1.0.15843807

    VIBs Installed: VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.32-0.0.8169922
    VIBs Removed: VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.37-1vmw.700.1.0.15843807

    1. Did you find any solution? I’ve stucked with same situation – when i replaced vib it lose network interfaces. In Native misc drivers there is all drivers for network cards too, but with it there is no method to install nvme driver from older version.

Leave a Reply

Your email address will not be published. Required fields are marked *