Restoring datastore performance graphs after upgrading to vCenter 5.0U1

My recent upgrade from ESXi and vCenter 4.1U3 to 5.0U1 went so smoothly, I should have known that not everything was as it should have been. My battle scars from decades of fighting buggy software (in other words, software) were tingling, and it wasn’t too long before I found the problem. All my datastores’ non-realtime performance data graphs—which I rely on for troubleshooting slow VMs—were blank. ‘No data available‘ was all I got. A call to the refreshingly excellent VMware support folk resulted in a pointer to this KB article. It’s a bit involved, especially if you don’t already have PowerCLI installed, so here’s a quick walk-through.

The reason for all this

The KB article says that

In vSphere 5.0 Update 1 and later, by default the Storage DRS and SIOC counters are active at the higher statistics collection level 3. This effectively disables non-realtime datastore performance charts. In order to re-enable these performance charts, you can choose to move those counters back to collection level 1.

but goes on to warn that

moving Storage DRS and SOIC counters to a lower collection level will negatively impact performance, because more statistics are persisted, requiring more I/O and database operations.

First, install PowerCLI…

…for which you will need a Windows box. That’s right: while the ordinary CLI has both Windows and Linux installers, PowerCLI needs .NET and PowerShell. Oh well, luckily I have a dogsbody Windows VM that I can use for such things.

Finding a download for PowerCLI isn’t easy either. Searching for ‘PowerCLI’ in ‘All Downloads’ on VMware’s download website gives no results (cheers, very helpful), at least at the time of writing. You have to go into vSphere downloads and click ‘Drivers & Tools’, then open up ‘Automation Tools and SDKs’; it’s the link at the bottom. If you can’t be arsed with all that, here’s a direct link to the PowerCLI download page. You’ll need to be registered on VMware’s website to download it.

When you run the downloaded installer, it’ll first install the VIX API. If you’ve never used Powershell for anything on this machine, you’ll then get a warning about the execution policy not being set to ‘RemoteSigned’.

By default, the policy is ‘Restricted‘, meaning that Powershell only works interactively. Setting the policy to ‘RemoteSigned‘ means that ‘downloaded scripts must be signed by a trusted publisher before they can be run‘. If you’re happy to make that change (the installer says it’ll be OK if you don’t, but I didn’t want any more trouble), you’ll need to run a Powershell session as administrator, and enter the command

Set-ExecutionPolicy RemoteSigned

If it throws an error about not being able to write to the registry, you’re not running as administrator: remember to start a command prompt by right-clicking and selecting ‘Run as Administrator‘.

After that, the installation should complete without incident.

Running VMware’s little hack

There’s no getting away from it: this is a little hack. The KB article asks that you download a file called LevelMappingUtility.zip (it’s near the bottom of the article under ‘Attachments’). This contains two files: a Powershell script, and a CSV file with some variable-value pairs in it. You need to extract both files into a directoryfolder (I used c:\tmp), start a PowerCLI session from the new entry in your start menu, and do something like this (and be patient, it takes ages to respond):

Import-Module c:\tmp\VMware.VimAutomation.PowerCliExtensions.CounterLevelMapping.psm1
Connect-VIServer <ip address of vCenter server>,<same ip address again>


At this point (or at least a minute or so after this point) a window will pop up asking you for your vCenter credentials. You might get a message whinging about certificate problems, then you’ll be prompted for the credentials again. Finally you’ll be asked if you want to use multiple default servers; I just said ‘no’ as I only have one vCenter instance.

If you’ve got this far, you can now run the command that actually makes the changes:

Import-Csv -path c:\tmp\sdrs-sioc-reenable.csv | Set-PxCounterLevelMapping -Server <ip address of vCenter server>,<same ip address again>

After a while, you should get a couple of dozen ‘Updating’ messages to show it all went well.

If that was a bit hard to follow, here’s a screenshot of the first couple of commands:

and the last one:

My graphs came back about half an hour after I finished this procedure. I’ve been monitoring my vCenter instance closely but I haven’t seen any evidence of performance problems. However, I don’t use Storage DRS, which is said to increase the risk of vCenter database overload. It’s probably worth keeping an eye on things for a few days after you make this change.