Allow only specific hosts to log to vRealize Log Insight

Today a college of mine was asked by a customer if it would be possible to only allow specific host to send logs to VMware vRealize Log Insight (vRLI).

And as it is running on a Linux platform iptables is built in, so I figured why not?

iptables is a in kernel firewall built in to almost any Linux distribution.

Why would you limit who can send logs to your vRLI. This is not something that I hear many customers ask for, but I can certainly understand why you would not want any host or user without permission to spam you logs. And even though the filtering in vRLI is very good, you could potentially run out of disk space, and miss log that you actually wanted. Also it would be possible for an attacker to disguise his whereabouts with generated false logs. This would not be a foolproof method to avoid this, as I can easily think of a couple of ways to accomplish this anyway. Continue reading Allow only specific hosts to log to vRealize Log Insight

VMs with multiple vNics could be a security risk

Often when I do health checks on vSphere environments I come across VMs that have multiple vNics. That can be a serious security risk if these vNics are connected to different security zones. A VM that is connected both to a DMZ and to a Administration network could allow a hacker easy access to more privileged networks. Sometimes this configuration is acceptable if the operating system is designed to handle it, if for instance we are dealing with a firewall.

I often find VMs that have a configuration where one of the network adapters is disconnected. Sometimes the second vNic was forgotten, and other times it is connected from vCenter when access to the secondary network is wanted for some kind of maintenance.

There is a settings on the virtual network adapter called “allowGuestControl”, and I was wondering if this setting could be a security issue. Could a hacker enable the disconnected network adapter from within the guest operating system, and thereby gain access to a privileged network? Continue reading VMs with multiple vNics could be a security risk

VMware NSX Active Directory Groups Missing

Today I was doing some micro segmentation at a customer site, and I was having an issue with new active directory groups not showing up when I wanted to add them to a Security Group.

I turned out that there were a lot of groups missing. I checked the synchronization  but there was no errors, and no pattern in what groups was missing and what groups were there. If you do have errors this article might be relevant for you: https://kb.vmware.com/s/article/2150678

I checked the release notes for recent updates, but it did not look like this was a known bug.

Solution:
I did find a workaround. I deleted the Domain from NSX, and set if up again. Now all groups was available. This is not a very good solution since all your existing setup in regards to Identity based rules in the distributed firewall, and security groups with AD Group members, needs to be redone. So be careful to document everything before you delete the domain.

Relevant NSX version: 6.3.2.5672532

Things to know about upgrading vCSA 6.0 to vCSA 6.5

Here is a list of things that you might want to do before you upgrade your vCenter from vCSA 6.0 to vCSA 6.5.

Postgres table owner

First check you postgres database. For some reason the table owner is often wrong. Thanks to black88mx6 There is a way to check this, and also to fix it. Remember that anything you see here is executed at your own risk. An important step to perform before upgrading is taking a backup/snapshot of your vCenter VM, and any related components, so that you will be able to recover from a failed upgrade. Continue reading Things to know about upgrading vCSA 6.0 to vCSA 6.5

Award – VMware Technical Ambassador of the year 2017

I was very surprised to receive the Technical Ambassador 2017 award from VMware at the Nordic Partner Briefing on Sunday evening.

For those who don’t know my history, I have been implementing and promoting VMware and virtualization since 2002, and it is not often that you receive awards in this industry unless you are working with sales, so i really appreciate this.

I owe a big thank you to Michael Eskildsen from VMware for nominating me, to Atea and my team leader Flemming Westervang for giving me the opportunity to work with the biggest customers and most interesting cases in the Nordic, and of course to VMware for constantly raising the bar.

vCenter Recent Tasks Descriptions and Names are broken after VCSA Upgrade

After upgrading VCSA from version 6.5 to 6.5 Update 1 you might experience a problem with Task Names and object Descriptions. Names are not “resolved” to its human understandable name, but instead they are written as a API object name.

For instance a host profile compliance check would normally be “Compliance check” but is instead written as “profile.ComplianceManager.check.label” or a VMotion is written as “Drm.ExecuteVMotionLRO.label”

Continue reading vCenter Recent Tasks Descriptions and Names are broken after VCSA Upgrade

ESXi 6.5 Update 1 PSOD on HPE 460c Gen9 after Ixgben driver update

Today I upgraded a customer to ESXi 6.5 Update 1, but unfortunately some of them ended up purple screening at reboot after they were updated.

Affected Servers so far

  • HPE BL460c Gen9
  • HPE DL360p Gen8 (Reported by anonymous user)
  • HPE DL380 Gen9 (Reported by Bernhard)
  • HPE DL380 Gen8 (Reported by Ralf)
  • HPE DL380p Gen9 (Reported by Victor)

PSOD Error

PSOD: #PF Exception 14 in world 68297:sfcb-intelcim IP 0x41801b704d8f addr 0x443919649c000

Continue reading ESXi 6.5 Update 1 PSOD on HPE 460c Gen9 after Ixgben driver update

Deep Dive into VMware vSAN Performance Benchmarks

VMworld 2017 Breakout Session Proposal Accepted!

I am VERY happy to announce that my application for a VMworld session at VMworld in Barcelona 2017 has been accepted. I will be sharing this session with my excellent coworker Karsten Drejer.

I can’t wait to tell you all about our finding and the awesome performance that we are seeing on VMware vSAN. I will be comparing these benchmark numbers to traditional storage types from known vendors.

Please support me by attending my session at VMworld 2017 in Barcelona. My session ID is #STO1117BE. You can find it here: https://my.vmworld.com/scripts/catalog/eucatalog.jsp?search=STO1117

WARNING: My session is very technical, so please be ware. I will however also have some graphs with pretty colors, so if you are not completely down with IOPS, Read Write latency, bits and bytes, come anyway. I will try hard explain my findings. Also Karsten will give some general knowledge about vSAN in the first part of the session.

VMworld 2017 in Barcelona is running from the 11-14th of September.

Update: Our sessions is scheduled at the 13th of September in Hall 8, Room 17.

VMware Auto Deploy stopped working with Parse Error

I ran into a problem with VMware vSphere 6.5 Auto Deploy suddenly stopped working.

When trying to change rules with New-DeployRule or Repair-DeployImageCache I got the following error:

Repair-DeployImageCache
System.Runtime.Serialization.SerializationException: Parse Error, no assembly associated with Xml key ImagefactoryPkgImageProfile
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessGetType(String value, String xmlKey, String& assemblyString)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessType(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessAttributes(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.StartChildren()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.ParseXml()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.Run()
at System.Runtime.Serialization.Formatters.Soap.ObjectReader.Deserialize(HeaderHandler handler, ISerParser serParser)
at System.Runtime.Serialization.Formatters.Soap.SoapFormatter.Deserialize(Stream serializationStream, HeaderHandler handler)
at VMware.DeployAutomation.Types.PxeProfile.get_ImageProfile()
System.Runtime.Serialization.SerializationException: Parse Error, no assembly associated with Xml key ImagefactoryPkgImageProfile
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessGetType(String value, String xmlKey, String& assemblyString)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessType(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessAttributes(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.StartChildren()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.ParseXml()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.Run()
at System.Runtime.Serialization.Formatters.Soap.ObjectReader.Deserialize(HeaderHandler handler, ISerParser serParser)
at System.Runtime.Serialization.Formatters.Soap.SoapFormatter.Deserialize(Stream serializationStream, HeaderHandler handler)
at VMware.DeployAutomation.Types.PxeProfile.get_ImageProfile()
System.Runtime.Serialization.SerializationException: Parse Error, no assembly associated with Xml key ImagefactoryPkgImageProfile
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessGetType(String value, String xmlKey, String& assemblyString)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessType(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessAttributes(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.StartChildren()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.ParseXml()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.Run()
at System.Runtime.Serialization.Formatters.Soap.ObjectReader.Deserialize(HeaderHandler handler, ISerParser serParser)
at System.Runtime.Serialization.Formatters.Soap.SoapFormatter.Deserialize(Stream serializationStream, HeaderHandler handler)
at VMware.DeployAutomation.Types.PxeProfile.get_ImageProfile()
System.Runtime.Serialization.SerializationException: Parse Error, no assembly associated with Xml key ImagefactoryPkgImageProfile
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessGetType(String value, String xmlKey, String& assemblyString)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessType(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.ProcessAttributes(ParseRecord pr, ParseRecord objectPr)
at System.Runtime.Serialization.Formatters.Soap.SoapHandler.StartChildren()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.ParseXml()
at System.Runtime.Serialization.Formatters.Soap.SoapParser.Run()
at System.Runtime.Serialization.Formatters.Soap.ObjectReader.Deserialize(HeaderHandler handler, ISerParser serParser)
at System.Runtime.Serialization.Formatters.Soap.SoapFormatter.Deserialize(Stream serializationStream, HeaderHandler handler)
at VMware.DeployAutomation.Types.PxeProfile.get_ImageProfile()

Continue reading VMware Auto Deploy stopped working with Parse Error

Migrate folder structure from old to new vSphere vCenter

Sometimes I find it easier to create a new vCenter server then migrate the old one, and it is a perfectly good solution in many cases.

But annoyingly there is a lot of manual work involved.

One problem is the VM’s and Templates folders. They do not follow the host, so you have to create the folder structure manually and move each VM into the correct folder. Well I am way to lazy to do that by hand, so it’s time to Automate! Continue reading Migrate folder structure from old to new vSphere vCenter