vSphere 6.5 Security Configuration Guide Available

vSphere 6.5 Security Configuration guide and Mike’s blog post about it.  In particular, he writes:

I’d like to take this opportunity to remind folks what the vSphere Hardening Guide and/or the vSphere Security Configuration Guide is and is not. It is not meant to be used as a “compliance” tool nor a set of boxes to check. It is not a set of mandates. Blanket application of ANY changes to a system should be carefully reviewed before being made.  It is a set of guidelines that attempts to explain risk and start a risk management conversation between IT and security and “guide” both teams into setting up the product in a secure fashion.

I’ve definitely seen this used as a “we must set all the settings to the hardening guide,” which just causes operations issues, thus generating workarounds, and certainly does not automatically improve your security stance.

Here’s the article:

Addressing the “Guest Escape” security vulnerability

VMware recently responded to the pwn2own security exploit against VMware Workstation.  Patches have been released for ESXi, Fusion, and Workstation.  VMware says they are not aware of the exploit being used in the wild.

Here’s VMware’s response:

The security advice boils down to following general best practices:

  • Depth in defense
  • Apply vendor patches

From the article:

As offensive security evolves, so does defensive security. With vSphere 6.5, VMware began deploying sandboxing technology around virtual machines to prevent a single arbitrary code execution from spreading across a host – a technique we have adapted from studying how web browsers and cell phones have evolved to defend against offensive security research. We are proactively disabling or removing legacy features – the loss in compatibility is increasingly outweighed by the reduction in attack surface. And we are investing deeply in ease-of-upgrade, recognizing that prompt security patches do little good if they cannot be deployed to production in time.


When VMware models threats, we consider three categories of actor. The “nation-state” actor has vast resources but generally employs them against limited targets; such an actor will find a way to breach security, whether by technical means or something simpler (money, ideology). The “professional” actor has more limited resources, and tends to look for the softest and most profitable target; defending against this actor amounts to staying on or ahead of the security research curve. And the “script kiddie” uses off-the-shelf resources and previously-known issues; defending here generally requires little more than staying up to date, and the biggest risk is installations which fail to deploy existing patches. The Pwn2Own competition shows that the difficulty of hypervisor attacks is moving from the “nation-state” category to the upper end of the “professional” category. This is a trend we have been expecting for a while, as security research tools become more powerful.

Be sure to prioritize security focus appropriately.  Using low entropy passwords or allowing direct access to the management network are probably greater risks to your infrastructure security than more difficult to exploit issues such as this (YMMV and IMHO).

Vester – open source, native PowerCLI Configuration Management for vSphere

Recently, I was able to play around a bit with Vester, an open source native PowerCLI configuration management tool for vSphere environments.

The basic idea: 1) Build a configuration file from existing objects (VM, Host, Cluster, etc.), 2) tweak if needed, 3) audit all the things for compliance, and 4) remediate (optional).

At various customer sites, I find myself providing a particular type of code over and over – hey, there’s a hot KB setting tweak we need to apply, so let’s audit current state, test the change, update all objects in your environment, and then leave the customer a script to continue this audit/remediate cycle.  I’d like a better way, following the principle of DRY IT.

While other configuration management tools exist (such as ansible’s VMware module), I find that many VMware admins have a reasonable comfort level with PowerCLI but less so with other CM tools.  Additionally, in my role as a consultant, I may not have scope to engage in a full-blown deployment of CM tools.

However, most of my customers already have a place to run PowerCLI scripts.  This makes Vester an easy, fast solution.

Vester would allow me to hand my customer a PowerCLI module and config script, with simple instructions to run “Invoke-Vester” and “Invoke-Vester -Remediate”. This seems an elegant way to address the above use case. Pretty neat!

Still, in my assessment, this is definitely not a replacement for enterprise “infrastructure as code” and configuration management tools, which provide the feature rich framework required for managing IT at scale.  Instead, it is an easy to understand tool that is accessible to the majority of vSphere admins.  As such, I think it fills a much-needed niche between manual administration and full-blown infrastructure automation.

P.S. I have updated my Resources page to include the beginners guide to “contributing to a github project” I found useful here.  Although I’ve worked with github and some git basics, I found the linked guide helpful to see how the pieces fit together.

vSphere 6.0 PSC Replication Ring Topology

In vSphere 6.0, the PSC introduces multi-master replication.  It is important to understand that the default replication topology is one-to-one, which may not be immediately intuitive.

Design Examples

For example, a site with three sites, each with a single PSC, connected via Enhanced Linked Mode, may follow this replication topology:

In this scenario, Site 1 and Site 3 do not replicate directly, only via proxy with site 2.

Here’s another topology that might “happen”, if you aren’t taking too much care about how you are deploying this:

In this scenario, Site 2 and Site 3 both replicate directly to Site 1, but not directly between Site 2 and Site 3.

In both of these scenarios, all three sites will converge on the same replication set.  However, a single site failure will prevent replication between two other sites.  This design concern becomes more critical in larger environments with more sites.

Recommendation – Use a Ring

For our larger customers we have used a ring topology to improve the design.  Using a ring topology is recommended in the VMware SDDC validated design.  Following our three site example, a ring topology would look like this:

So, how is the replication topology determined?  When you deploy each subsequent PSC after the first, during the install wizard or in the install script you must specify an existing PSC to connect for enhanced linked mode.  A replication relationship will be formed between the target PSC and the PSC you are currently deploying.

How To Deploy a Ring Topology

Ring is simple – it is just a straight line, with the last deployed PSC having an additional replication relationship to the first node in the ring.  Carefully planning the order of PSC deployments will allow you to build the “straight line” replication topology (see the first image).  Once that’s done, log on to the last PSC via SSH and add a new 2-way replication relationship to the first PSC:

For more details, see this incredibly helpful article: https://kb.vmware.com/kb/2127057