Wednesday, September 11, 2013

Using procmon to troubleshoot "Vsphere client stopped working" crash (System.NullReferenceException)

Hello,

Below steps taken to troubleshoot issue I encountered on Windows 7 x64 launching Vsphere client (v4.0).

I used Vsphere quite exstensively for past 2 years and today for whatever reason it started failing to launch crashing with NullReferenceException below


I installed some Microsoft patches previous day as well as increased UAC prompt from "Never Notify" level to one notch up. There were no other changes to system.


I uninstalled patches but this did not fix the issue. I reinstalled Vmware vsphere client and this did not fix it either.
Now where sysinternal tools comes to rescue.
First I downloaded DbgView.Exe (http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx) which allows to see debug output from system. Below is snapshot what was captured during crash.


[5368] [        :Error   :W: 1] 2013-09-11 10:45:49.767  A fatal unhandled error occurred in vi client 
[5368] [        :Error   :W: 1] 2013-09-11 10:45:49.786  System.NullReferenceException: Object reference not set to an instance of an object. 
[5368]    at VirtualInfrastructure.Utils.Log.Init(String logFileName, Nullable`1 traceFlags, Boolean clearLog, Boolean setAppDomainVars) 
[5368]    at VpxClient.Program.InitLogFile() 
[5368]    at VpxClient.Program.Main(String[] commandline) 

We have a stack trace from application which points to some sort File Access/Permission error.

Now comes second tool from SysInternals called procmon.exe (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) which listens to configured ETW events. For my case I needed just file events for knows process vpxclient.exe. So below is what was captured



There were 2100 events preceeding crash and you need to filter those to find a curlpit. In my case I made filter for Result Column to "Access Denied"


It's clear what the issue at this point. VpxClient trying to read some sort initialization file and get's access denied. At this point I remembered that I changed UAC prompt the day before but did not launch VpxClient with "run as Administrator" and hence the issue since previously I did not have to do that since it was set to "Never Notify" before.



Post a Comment