PowerDbg v5.1 – Using PowerShell to Control WinDbg

Roberto Alexis Farah, 25th February 2009
http://blogs.msdn.com/debuggingtoolbox/

Here I would like to introduce a minor version of PowerDbg 5.0[6] with a few new cmdlets. These new cmdlets are those that we use most of the time for .NET debugging.

POWERDBG FILES

WinDbg.PSM1 ß Contains cmdlets used to communicate with WinDbg.

Microsoft.PowerShell_Profile.PS1 ß Contains cmdlets that parse command output. Uses WinDbg.PSM1 under the hood.

INSTALLATION

WinDbg.PSM1

Goes to %\Windows\System32\WindowsPowerShell\v1.0\Modules\WinDbg

Note: PowerDbg assumes the folder c:\debuggers as the default installation folder. This is true for the default in­stallation of our private debugger version (Microsoft) but not for the public version. So, please, change this variable to reflect your installation:

param($cdbPath = "C:\debuggers\cdb.exe")

Microsoft.PowerShell_Profile.PS1

Goes to %\Documents\windowspowershell

In order to know the exact location, use this command from PowerShell:

$profile

REQUIREMENT

PowerShell v2.0

USAGE

First, make sure you can run scripts:

set-executionpolicy remotesigned

From the WinDbg window type:

.server tcp:port=10456,server=ServerName ß ServerName is your server name.

The command above enables a port for communication with the WinDbg instance as a server. You can use other port numbers.

From the PowerShell window you must initialize the communication:

Import-module WinDbg ß Importing our module WinDbg.PSM1

Connect-Windbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg instance.

Or:

Connect-Dbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg

Note: Don’t forget to load symbols and your extensions!

At this point you’re ready to use PowerDbg or PowerDbg scripts.

Example:

Analyze-PowerDbgThreads ß Cmdlet.

.\PowerDbgScriptExceptions ß Script.

NEW FOR POWERDBG v5.1

Load-PowerDbgSymbols <$symbolPath>

Load symbols.

Usage:

Load-PowerDbgSymbols “SRV*c:\PUBLICSYMBOLS*http://msdl.microsoft.com/download/symbols"

Parse-PowerDbgASPXPAGES

Maps the output from the !ASPXPages command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

For this version we consider the fields:

Key: HttpContext

Value: Timeout+Completed+Running+ThreadId+ReturnCode+Verb+RequestPath+QueryString

Parse-PowerDbgCLRSTACK

Maps the output from the !clrstack command or ~* e !clrstack and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Attention! The key is the thread number and the value is the call stack separated by $global:g_frameDelimiter.

Commas "," are replaced for ";" to avoid confusing with the comma used by the CSV file.

If you use this cmdlet to parse the output from ~* e !clrstack the threads not running managed code are au­tomatically ignored.

Parse-PowerDbgTHREADS

Maps the output from the !threads command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The following fields are extracted:

Thread Number- Key

ID+OSID+ThreadOBJ+State+GC+Context+Domain+Count+APT+Exception -Value

Parse-PowerDbgDSO

Maps the output from the !dso or ~* e !dso command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The Thread Number is the key and the stack is the value, like the way that Parse-PowerDbgK or Parse-PowerDbgCLRSTACK operate.

Attention! Commas are replaced by ";" and $global:g_FrameDelimiter is used to separate frames.

CMDLETS FROM POWERDBG

Send-PowerDbgCommand <$command>

This was the most complex cmdlet, but now it’s just a wrapper for Invoke-WinDbgCommand.

SendPowerDbgCommand sends commands to WinDbg.

Parse-PowerDbgDT [$useFieldNames]

Parses the output from the dt command and saves it into POWERDBG-PARSED.LOG using a CSV file format.

If $useFieldNames has a value, the cmdlet stores fields from struct/classes and values. Otherwise it stores offsets and values.

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Convert-PowerDbgCSVToHashTable

Converts the output from the Parse-PowerDbg* cmdlets to a Hash Table.

Send-PowerDbgDML <$hyperLinkDML> <$commandDML>

Creates a DML command and sends it to WinDbg.

DML stands for Debug Markup Language. Using DML you can create hyperlinks that execute commands.

Parse-PowerDbgNAME2EE

Maps the output from the !name2ee and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMD

Maps the output from !dumpmd command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMODULE

Maps the output from !DumpModule command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPLMI

Maps the output from !lmi command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Has-PowerDbgCOMMANDSUCCEEDED

Returns $true if the last command succeeded or $false if not.

Send-PowerDbgComment

Sends a comment, a string in bold, to the WinDbg window.

Parse-PowerDbgVERTARGET

Maps the output from vertarget command, either the Kernel Time or the User Time.

The output is saved into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgRUNAWAY

Maps the output of !runaway 1 or !runaway 2 and stores the results into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! If you need to know the top threads consuming CPU time, use Convert-PowerDbgRUNAWAYtoArray. The items will be in the same exact order of the original command.

Convert-PowerDbgRUNAWAYtoArray

Returns an array of two dimensions corresponding to the output of !runaway 1 or !runaway 2.

Parse-PowerDbgK

Maps the output of k command and its variations like kv, kbn, kpn, etc.

The output is saved into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn’t work with kPn. It also replaces “,” with “;” to avoid conflict with the CSV deli­miter.

Parse-PowerDbgSymbolsFromK

Maps just the symbols from k command and its variants, saving the content into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn’t work with kPn. It also replaces “,” with “;” to avoid conflict with the CSV deli­miter.

Parse-PowerDbgLM1M

Maps just the output from lm1m and stores it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Classify-PowerDbgThreads

Returns an array where the index is the thread number and the value is one of these values:

0 UNKNOWN_SYMBOL

1 WAITING_FOR_CRITICAL_SECTION

2 DOING_IO

3 WAITING

4 GC_THREAD

5 WAIT_UNTIL_GC_COMPLETE

6 SUSPEND_FOR_GC

7 WAIT_FOR_FINALIZE

8 TRYING_MANAGED_LOCK

9 DATA_FROM_WINSOCK

It’s very easy to add more symbols and constants to get a more granular analysis. Look at the source code for de­tails.

Analyze-PowerDbgThreads

Analyzes and shows what each thread is doing and its cor­responding CPU time, sorted by User Mode time.

This cmdlet is very useful for scenarios like hangs, high CPU, and crashes.

Attention! This command requires thread information if debugging a dump file.

Parse-PowerDbgPRINTEXCEPTION

Maps the output from !PrintException command and saves it into the CSV file POWERDBG-PARSED.LOG.

The following fields are considered while others are ignored:

Exception object:

Exception type;

Message:

InnerException:

HRESULT:

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDD-L1

Maps the output from dd <address> L1 or dd poi(<address>) L1 and saves the results into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgGCHANDLELEAKS

Maps the output from !GCHandleLeaks command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPOBJ

Maps the output from !DumpObj command and saves it into the CSV file POWERDBG-PARSED.LOG.

The assembly path and file name are saved using the key name ‘Assembly:’.

If the object is invalid the ‘Name:’ field will have the string “Invalid Object.” You may want to check this string to make sure you’ve got valid data.

The keys are the fields or Method Table, and values are the corresponding value.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This version maps the fields below “Fields:” using MethodTable as key and Value as value. The prob­lem with this approach is that the same MethodTable may appear more than once. If it happens, the last or most recent MethodTable and value will be considered.

Based on users’ feedback this approach may be changed in the near future.

POWERDBG SCRIPTS

PowerDbgScriptDumpDict.PS1

http://blogs.msdn.com/debuggingtoolbox/archive/2008/10/29/powershell-script-extracting-all-key-value-pairs-from-a-dictionary-object.aspx

Extracts the key/value pair from a Dictionary.

PowerDbgScriptExceptions.PS1

http://blogs.msdn.com/debuggingtoolbox/archive/2008/01/15/powershell-script-displaying-inner-and-hidden-exceptions.aspx

Displays the call stacks that have inner or hidden exceptions.

PowerDbgScriptGCHandleLeaksChart.PS1

http://blogs.msdn.com/debuggingtoolbox/archive/2008/08/22/powershell-script-chart-and-statistics-from-top-20-objects-leaking.aspx

It displays statistics and a chart from the top 20 objects leaking.

PowerDbgScriptHighCPU.PS1

http://blogs.msdn.com/debuggingtoolbox/archive/2007/12/17/powershell-script-isolating-the-threads-consuming-high-cpu.aspx

It displays all threads consuming high CPU using a specific time as a threshold.

PowerDbgScriptSaveModule.PS1

http://blogs.msdn.com/debuggingtoolbox/archive/2007/09/06/powershell-script-saving-a-module-from-a-net-method-call.aspx

It saves all modules that have a specific method. You pro­vide the method name, and it gives you the corresponding modules.

Download PowerDbg

http://www.codeplex.com/powerdbg

Example: [PowerShell Script] Statistics from .NET Applications

This script is the reason why PowerDbg v5.1 was created. I had to create some new cmdlets in order to create this script. By the way, thanks to my teammate Aaron Barth that gave the idea for this script!

This script collects information from all threads running managed code and gives the user statistics by threads like:

- CLR stack.

- Managed objects from the stack.

- ASP.NET page.

- What the thread is doing.

- Exceptions by threads.

- Threads running ASP.NET pages.

Contrary to what you may think this script is very simple and shows you how to use PowerDbg. It’s very easy to customize it or improve it. For example, you may want to display the ASP.NET pages by threads or queries/stored procedures by threads.

Screenshots:

ScriptASPX_1.jpg


ScriptASPX_2.jpg


ScriptASPX_3.jpg


ScriptASPX_4.jpg


Source code for PowerShellScriptASPXStatistics.ps1:

###########################################################################

# Script: PowerDbgScriptASPXStatistics

#

# Parameters: None.

#

# Purpose: Shows statistics from threads running ASP.NET pages.

#

# Attention! This script was not tested on Win64.

#

# Changes History:

#

# Roberto Alexis Farah

# All my functions are provided "AS IS" with no warranties, and confer no rights.

###########################################################################set-psdebug -strict

$ErrorActionPreference = "stop"

trap {"Error message: $_"}

write-Host "Scanning all threads and extracting the CLR stack..." -foreground Green -background Black

# First, let's scan all threads and identify those running managed code.

Send-PowerDbgCommand "~* e !clrstack"

Parse-PowerDbgCLRSTACK

# Get all the stacks running managed code.

$clrStack = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

# Sorts the keys by Thread Number and save them into an array.

$arrayOfThreads = $clrStack.keys | Sort-Object {[int] $_}

# Let's consider the situation where the dump has no thread running managed code.

if($arrayOfThreads.Count -eq 0)

{

write-Host "There are not threads running managed code!" -foreground Red -background Black

return

}

write-Host "Scanning all threads and extracting the managed objects..." -foreground Green -background Black

Send-PowerDbgCommand "~* e !dso"

Parse-PowerDbgDSO

$dso = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

write-Host "Collecting information about each thread..." -foreground Green -background Black

Send-PowerDbgCommand "!Threads"

Parse-PowerDbgTHREADS

$threads = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

write-Host "Collecting information from threads running ASP.NET..." -foreground Green -background Black

Send-PowerDbgCommand "!ASPXPages"

Parse-PowerDbgASPXPAGES

$aspxPages = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

write-Host "Scanning all threads and preparing statistics..." -foreground Green -background Black

# Scans all threads running managed code.

for($i = 0; $i -lt $arrayOfThreads.Length; $i++)

{

# Make sure the content is not null.

if($arrayOfThreads[$i] -eq "")

{

continue; # Invalid, get next element.

}

write-Progress -activity "Thread Statistics" -status "Thread number $arrayOfThreads[$i]" -percentComplete ($i / $arrayOfThreads.Count * 100)

write-Host "==============================================================" -foreground Green -background Black

write-Host "`nThread number: " -foreground Green -background Black -nonewline

write-Host $arrayOfThreads[$i] -foreground Red -background Black

write-Host "`nCLR stack:`n" -foreground Green -background Black

[string] $temp = $clrstack[$arrayOfThreads[$i]]

$temp = $temp.Replace($global:g_frameDelimiter, "`n")

$temp = $temp.Replace(";", ",")

write-Host $temp -foreground Red -background Black

write-Host "`nManaged objects from the stack:`n" -foreground Green -background Black

$temp = $dso[$arrayOfThreads[$i]]

$temp = $temp.Replace($global:g_frameDelimiter, "`n")

$temp = $temp.Replace(";", ",")

write-Host $temp -foreground Red -background Black

write-Host "`nThread Number ID OSID ThreadOBJ State GC Context Domain Count APT Exception`n" -foreground Green -background Black

write-Host " " $arrayOfThreads[$i] " " $threads[$arrayOfThreads[$i]] -foreground Red -background Black

$threadNum = $arrayOfThreads[$i]

# Change context to the current thread being analyzed.

Send-PowerDbgCommand "~ $threadNum s"

# Get exception.

Send-PowerDbgCommand "!PrintException"

Parse-PowerDbgPRINTEXCEPTION

$exception = $null

$exception = Convert-PowerDbgCSVToHashTable

# Makes sure there is an exception coming from that thread.

if($exception["Message:"] -ne $null)

{

write-Host "`nException object:" -foreground Green -background Black

write-Host $exception["Exception object:"] -foreground Red -background Black

write-Host "Exception type:" -foreground Green -background Black

write-Host $exception["Exception type:"] -foreground Red -background Black

write-Host "Message:" -foreground Green -background Black

write-Host $exception["Message:"] -foreground Red -background Black

write-Host "Inner Exception:" -foreground Green -background Black

write-Host $exception["InnerException:"] -foreground Red -background Black

write-Host "HRESULT:" -foreground Green -background Black

write-Host $exception["HResult:"] -foreground Red -background Black

}

# User must press any key to continue after 5 threads were displayed.

if((($i + 1) % 5) -eq 0)

{

write-Host "`n####### Press any key to see 5 more threads... #######"

$keyboard = $host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")

}

}

write-Host "Done!" -foreground Green -background Black

write-Host "`nASP.NET Pages:`n" -foreground Green -background Black

write-Host "HttpContext Timeout Completed Running ThreadId ReturnCode Verb RequestPath QueryString" -foreground Green -background Black

foreach($item in $aspxPages.keys)

{

write-Host $item " " $aspxPages[$item] -foreground Red -background Black

}

[6] PowerDbg is a PowerShell tool that automates debugging sessions. Using PowerDbg we can create PowerShell scripts that work like extensions. Here is the link to download the tool: http://blogs.msdn.com/debuggingtoolbox/archive/tags/PowerDbg+Library/def...