Accelerated Windows Memory Dump Analysis (2016) [Vostokov Services] [9781908043467] 9781908043467
1,127 178 17MB
English Pages [212]
Polecaj historie
![Accelerated macOS Core Dump Analysis: Training Course Transcript with LLDB Practice Exercises [3 ed.]
1912636751, 9781912636754](https://dokumen.pub/img/200x200/accelerated-macos-core-dump-analysis-training-course-transcript-with-lldb-practice-exercises-3nbsped-1912636751-9781912636754.jpg)
![Accelerated Linux Core Dump Analysis: Training Course Transcript with GDB and WinDbg Practice Exercises [3 ed.]
191263659X, 9781912636594](https://dokumen.pub/img/200x200/accelerated-linux-core-dump-analysis-training-course-transcript-with-gdb-and-windbg-practice-exercises-3nbsped-191263659x-9781912636594.jpg)
![Windows Forensic Analysis Toolkit: Advanced Analysis Techniques for Windows 7 [3° ed.]
1597497274, 9781597497275](https://dokumen.pub/img/200x200/windows-forensic-analysis-toolkit-advanced-analysis-techniques-for-windows-7-3nbsped-1597497274-9781597497275.jpg)
![Accelerated Windows API for Software Diagnostics: With Category Theory in View [1 ed.]
1912636638, 9781912636631](https://dokumen.pub/img/200x200/accelerated-windows-api-for-software-diagnostics-with-category-theory-in-view-1nbsped-1912636638-9781912636631.jpg)
![Accelerated Windows Memory Dump Analysis (2016) [Vostokov Services] [9781908043467]
9781908043467](https://dokumen.pub/img/200x200/accelerated-windows-memory-dump-analysis-2016-vostokov-services-9781908043467-9781908043467.jpg)
Table of contents :
Training Courses
Training Packs
Training Roadmap
About the Author
Accelerated Windows Memory Dump Analysis
Accelerated .NET Memory Dump Analysis
Accelerated Mac OS X Core Dump Analysis
Accelerated Linux Core Dump Analysis
Accelerated Windows Debugging 3
Accelerated Windows Malware Analysis with Memory Dumps
Practical Foundations of Windows Debugging, Disassembly, Reversing
Accelerated Disassembly, Reconstruction and Reversing
Accelerated Windows Software Trace Analysis
Advanced Windows Memory Dump Analysis with Data Structures
Citation preview
Pattern-Oriented
Sample Training Exercises Version 2.0
Facebook
LinkedIn
Twitter
Software Diagnostics Services
Training Courses
Accelerated Windows Memory Dump Analysis
Accelerated .NET Memory Dump Analysis
Accelerated Mac OS X Core Dump Analysis
Accelerated Linux Core Dump Analysis
Accelerated Windows Debugging3
Accelerated Windows Malware Analysis with Memory Dumps
Practical Foundations of Windows Debugging, Disassembling, Reversing
Accelerated Disassembly, Reconstruction and Reversing
Accelerated Windows Software Trace Analysis
Advanced Windows Memory Dump Analysis with Data Structures © 2018 Software Diagnostics Services
Training Packs
Pattern-Oriented Trace and Log Analysis
Pattern-Oriented Malware Analysis
Pattern-Oriented Unix Memory Dump Analysis
Pattern-Oriented Memory Dump Analysis
Pattern-Oriented Windows Crash Dump Analysis
Pattern-Oriented Windows Debugging
Pattern-Oriented Windows Memory Forensics
Pattern-Oriented Complete Windows Memory Dump Analysis
Complete Pattern-Oriented Software Diagnostics
© 2018 Software Diagnostics Services
Training Roadmap Intermediate
Intermediate
Crash and hang Windows diagnostics and analysis?
Live and source code Windows debugging via WinDbg?
Beginner/ Intermediate
Beginner/ Intermediate
Accelerated Windows Debugging3
Accelerated Windows Memory Dump Analysis Intermediate/ Advanced
Kernel space
Accelerated Windows Malware Analysis
User space Managed .NET space
Intermediate Advanced Windows Memory Dump Analysis
Accelerated .NET Memory Dump Analysis
Intermediate/ Advanced
Accelerated Disassembly, Reconstruction and Reversing
Process Monitor log and/or CDF/ETW trace analysis? Beginner/ Intermediate
Assembly language for Windows debugging? Intermediate/ Advanced
Beginner/ Intermediate
Accelerated Windows Software Trace Analysis
Practical Foundations of Windows Debugging, Disassembling, Reversing
Mac OX X and GDB/LLDB core dump analysis and diagnostics?
Linux and GDB core dump analysis and diagnostics?
Beginner/ Intermediate
Accelerated Mac OS X Core Dump Analysis
Beginner/ Intermediate
Accelerated Linux Core Dump Analysis
© 2018 Software Diagnostics Services
Dmitry Vostokov is an internationally recognized expert, speaker, educator, scientist, and author. He is the founder of pattern-oriented software diagnostics, forensics, and prognostics discipline and Software Diagnostics Institute (DA+TA: DumpAnalysis.org + TraceAnalysis.org). Vostokov has also authored more than 30 books on software diagnostics, forensics and problem-solving, memory dump analysis, debugging, software trace and log analysis, reverse engineering, and malware analysis. He has more than 20 years of experience in software architecture, design, development, and maintenance in a variety of industries including leadership, technical and people management roles. Dmitry also founded DiaThings, Logtellect, OpenTask Iterative and Incremental Publishing (OpenTask.com), Software Diagnostics Services (former Memory Dump Analysis Services) PatternDiagnostics.com and Software Prognostics. In his spare time, he presents various topics on Debugging.TV and explores Software Narratology, an applied science of software stories that he pioneered, and its further development as Narratology of Things and Diagnostics of Things (DoT). His current areas of interest are theoretical software diagnostics and its mathematical and computer science foundations, software diagnostics engineering and diagnostics-driven development.
6
Published by OpenTask, Republic of Ireland Copyright © 2016 by OpenTask Copyright © 2016 by Software Diagnostics Services Copyright © 2016 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover, and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalog record for this book is available from the British Library. ISBN-l3: 978-1-908043-46-7 (Paperback) Version 4, 2016
2
Contents About the Author ...........................................................................................................................................................7 Presentation Slides and Transcript .................................................................................................................................9 Practice Exercises .........................................................................................................................................................35 Exercise 0: Download, setup and verify your WinDbg installation ............................................................................40 Exercise P1: Analysis of a normal application process dump (32-bit notepad) .........................................................47 Exercise P2: Analysis of a normal application process dump (64-bit notepad) .........................................................72 Exercise P3: Analysis of a normal application process dump (64-bit Microsoft Edge) ..............................................84 Exercise P4: Analysis of an application process dump (64-bit ApplicationK, no symbols).......................................113 Exercise P5: Analysis of an application process dump (64-bit ApplicationK, with application symbols) .................126 Exercise P6: Analysis of application process dump (ApplicationL, 32-bit) ...............................................................131 Exercise P7: Analysis of an application process dump (ApplicationL, 64-bit) ..........................................................140 Exercise P8: Analysis of an application process dump (ApplicationM, 64-bit) ........................................................148 Exercise P9: Analysis of an application process dump (ApplicationN, 64-bit) .........................................................162 Exercise P10: Analysis of an application process dump (ApplicationO, 64-bit) .......................................................174 Exercise P11: Analysis of an application process dump (ApplicationP, 64-bit) .......................................................184 Exercise P12: Analysis of an application process dump (ApplicationR, 32-bit) .......................................................199 Exercise P13: Analysis of an application process dump (ApplicationA, 64-bit) .......................................................217 Exercise P14: Analysis of an application process dump (ApplicationS, 64-bit) ........................................................225 Exercise P15: Analysis of an application process dump (notepad, 32-bit) ..............................................................238 Exercise P16: Analysis of an application process dump (notepad, 64-bit) ..............................................................242 Exercise P17: Analysis of an application process dump (ApplicationQ, 32-bit) .......................................................249 Exercise K1: Analysis of a normal kernel dump (64-bit) ..........................................................................................262 Exercise K2: Analysis of a kernel dump with pool leak (64-bit) ...............................................................................308 Exercise K3: Analysis of a kernel dump with pool corruption (64-bit) ....................................................................326 Exercise K4: Analysis of a kernel dump with code corruption (64-bit) ....................................................................335 Exercise K5: Analysis of a kernel dump with hang I/O (64-bit) ...............................................................................359 Exercise C1: Analysis of a normal complete dump (64-bit) .....................................................................................379 Exercise C2: Analysis of a problem complete dump (64-bit) ...................................................................................400 Exercise C3: Analysis of a problem complete dump (64-bit) ...................................................................................424 Exercise C4: Analysis of a problem complete dump (64-bit) ...................................................................................441 Exercise A1: Analysis of a problem active dump (64-bit) ........................................................................................463 Legacy Exercises .........................................................................................................................................................485 Exercise Legacy.0 ....................................................................................................................................................487 3
Exercise Legacy.P1: Analysis of a normal application process dump (32-bit notepad) ...........................................492 Exercise Legacy.P2: Analysis of a normal application process dump (64-bit notepad) ...........................................513 Exercise Legacy.P3: Analysis of a normal application process dump (32-bit IE) ......................................................522 Exercise Legacy.P4: Analysis of an application process dump (32-bit ApplicationK, no symbols) ...........................537 Exercise Legacy.P5: Analysis of an application process dump (32-bit ApplicationK, with application symbols) .....547 Exercise Legacy.P6: Analysis of application process dump (ApplicationL, 32-bit) ...................................................551 Exercise Legacy.P7: Analysis of an application process dump (ApplicationL, 64-bit) ..............................................558 Exercise Legacy.P8: Analysis of an application process dump (ApplicationM, 32-bit) ............................................562 Exercise Legacy.P9: Analysis of an application process dump (ApplicationN, 64-bit) .............................................572 Exercise Legacy.P10: Analysis of an application process dump (ApplicationO, 64-bit) ...........................................580 Exercise Legacy.P11: Analysis of an application process dump (ApplicationP, 32-bit) ............................................586 Exercise Legacy.P13: Analysis of an application process dump (ApplicationA, 32-bit) ...........................................597 Exercise Legacy.P14: Analysis of an application process dump (ApplicationS, 32-bit) ............................................605 Exercise Legacy.P15: Analysis of an application process dump (notepad, 32-bit) ..................................................614 Exercise Legacy.P16: Analysis of an application process dump (notepad, 64-bit) ..................................................618 Exercise Legacy.P17: Analysis of an application process dump (ApplicationQ, 32-bit) ...........................................624 Exercise Legacy.K1: Analysis of a normal kernel dump (32-bit) ..............................................................................633 Exercise Legacy.K2: Analysis of a kernel dump with pool leak (32-bit) ...................................................................670 Exercise Legacy.K3: Analysis of a kernel dump with pool corruption (32-bit) .........................................................689 Exercise Legacy.K4: Analysis of a kernel dump with code corruption (32-bit) ........................................................701 Exercise Legacy.K5: Analysis of a kernel dump with hang I/O (32-bit) ....................................................................715 Exercise Legacy.C1: Analysis of a normal complete dump (32-bit) .........................................................................728 Exercise Legacy.C2: Analysis of a problem complete dump (32-bit) .......................................................................748 Application Source Code ............................................................................................................................................783 ApplicationA ...........................................................................................................................................................785 ApplicationB ...........................................................................................................................................................787 ApplicationC ...........................................................................................................................................................789 ApplicationE ...........................................................................................................................................................791 ApplicationK ...........................................................................................................................................................793 ApplicationL ............................................................................................................................................................794 ApplicationM ..........................................................................................................................................................795 ApplicationN ...........................................................................................................................................................796 ApplicationO ...........................................................................................................................................................797 ApplicationP ...........................................................................................................................................................798 ApplicationR ...........................................................................................................................................................799 4
ApplicationS............................................................................................................................................................800 ApplicationQ ...........................................................................................................................................................801 Selected Q&A .............................................................................................................................................................805 Minidump Analysis .....................................................................................................................................................849 Scripts and WinDbg Commands .............................................................................................................................849 Component Identification.......................................................................................................................................852 Raw Stack Data Analysis .........................................................................................................................................857 Symbols and Images ...............................................................................................................................................866 Wait Chain (Executive Resources) ..............................................................................................................................869
5
Exercise P1: Analysis of a normal application process dump (32-bit notepad) Goal: Learn how to see dump file type and version, get a stack trace, check its correctness, perform default analysis, list modules, check their version information, check process environment. Patterns: Manual Dump; Stack Trace; Not My Version; Environment Hint. 1.
Launch WinDbg from Windows Kits \ WinDbg (X64).
2.
Open \AWMDA-Dumps\x86\Processes\notepad.DMP.
3.
We get the dump file loaded:
47
4.
Open a log file to save all future output using .logopen command:
Note: You can type any comment by using the * command. 5.
Type the command .symfix c:\mss to set a path to download symbol files from Microsoft symbol file server:
48
6.
Type .reload command to download symbols if necessary:
7.
Type k command to verify the correctness of the stack trace:
49
8. Type version command to get OS version, system and process uptimes, the dump file timestamp and its type:
50
Note: This is the full output: 0:000> version Windows 10 Version 10240 MP (4 procs) Free x86 compatible Product: WinNt, suite: SingleUserTS Personal kernel32.dll version: 10.0.10240.16384 (th1.150709-1700) Machine Name: Debug session time: Sun May 1 16:07:18.000 2016 (UTC + 1:00) System Uptime: 1 days 2:47:47.329 Process Uptime: 0 days 0:00:31.000 Kernel time: 0 days 0:00:00.000 User time: 0 days 0:00:00.000 Full memory user mini dump: C:\AWMDA-Dumps\x86\Processes\notepad.DMP Microsoft (R) Windows Debugger Version 10.0.10586.567 AMD64 Copyright (c) Microsoft Corporation. All rights reserved. command line: '"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\windbg.exe" ' Debugger Process 0x2B54 dbgeng: image 10.0.10586.15, built Fri Nov 20 04:56:41 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\dbgeng.dll] dbghelp: image 10.0.10586.15, built Fri Nov 20 04:55:01 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\dbghelp.dll] DIA version: 40116 Extension DLL search Path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\arcade;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\pri;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\arcade;C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS
51
Client\;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowe rShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x86;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET Web Pages\v1.0\;C:\Program Files (x86)\Symantec\VIP Access Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowe rShell\v1.0\;C:\Program Files (x86)\Skype\Phone\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\ Extension DLL chain: dbghelp: image 10.0.10586.15, API 10.0.6, built Fri Nov 20 04:55:01 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\dbghelp.dll] ext: image 10.0.10586.15, API 1.0.0, built Fri Nov 20 04:55:08 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\ext.dll] exts: image 10.0.10586.15, API 1.0.0, built Fri Nov 20 04:54:07 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\exts.dll] uext: image 10.0.10586.15, API 1.0.0, built Fri Nov 20 04:54:02 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\uext.dll] ntsdexts: image 10.0.10586.15, API 1.0.0, built Fri Nov 20 05:28:14 2015 [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\ntsdexts.dll]
Note: Debug session time is when the dump was generated. Although the dump is called “mini dump” it is a full memory user dump with all process memory included. 9.
Type the default analysis command !analyze -v:
52
53
Note: This (or.reload command) may take some time initially as symbols are downloaded from the symbol server:
54
10.
Let’s now look at the output in more detail:
0:000> !analyze -v ******************************************************************************* * * * Exception Analysis * * * *******************************************************************************
DUMP_CLASS: 2 DUMP_QUALIFIER: 400 FAULTING_IP: +0 00000000 ??
???
EXCEPTION_RECORD: ExceptionAddress: ExceptionCode: ExceptionFlags: NumberParameters:
(.exr -1) 00000000 80000003 (Break instruction exception) 00000000 0
FAULTING_THREAD:
00003078
DEFAULT_BUCKET_ID: PROCESS_NAME:
STATUS_BREAKPOINT
notepad.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}
Breakpoint
A breakpoint has been reached.
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid EXCEPTION_CODE_STR:
80000003
WATSON_BKT_PROCSTAMP: WATSON_BKT_PROCVER:
55bebe90 10.0.10240.16425
PROCESS_VER_PRODUCT:
Microsoft® Windows® Operating System
WATSON_BKT_MODULE:
unknown
WATSON_BKT_MODVER:
0.0.0.0
WATSON_BKT_MODOFFSET: WATSON_BKT_MODSTAMP:
0 bbbbbbb4
BUILD_VERSION_STRING:
10.0.10240.16384 (th1.150709-1700)
MODLIST_WITH_TSCHKSUM_HASH: MODLIST_SHA1_HASH: NTGLOBALFLAG:
409dc00a3b07a0619d19699aaf2ad34995696fba
a2b8dbdc12e291e73566ab6765f5a7461a85a26b
400
APPLICATION_VERIFIER_FLAGS:
0
55
PRODUCT_TYPE:
1
SUITE_MASK:
784
DUMP_FLAGS:
8000c07
DUMP_TYPE: APP:
0
notepad.exe
ANALYSIS_SESSION_HOST:
TRAINING-PC
ANALYSIS_SESSION_TIME:
05-01-2016 19:08:54.0766
ANALYSIS_VERSION: 10.0.10586.567 amd64fre THREAD_ATTRIBUTES: OS_LOCALE: ENU PROBLEM_CLASSES:
Tid [0x0] Frame [0x00] String [STATUS_BREAKPOINT] Data Bucketing
BUGCHECK_STR:
STATUS_BREAKPOINT
LAST_CONTROL_TRANSFER: STACK_TEXT: 04ebf8e0 74d7325a 04ebf8fc 009e5eb6 04ebf93c 009f5b41 04ebf9d0 749e3744 04ebf9e4 773e9e54 04ebfa2c 773e9e1f 04ebfa3c 00000000
STACK_COMMAND:
from 74d7325a to 74d74d9c
04ebf920 04ebf920 009e0000 7e3da000 7e3da000 ffffffff 009f59f0
00000000 00000000 00000000 749e3720 1c64488a 7740d6d6 7e3da000
00000000 00000000 05134032 0b053f62 00000000 00000000 00000000
user32!NtUserGetMessage+0xc user32!GetMessageW+0x2a notepad!WinMain+0xe6 notepad!WinMainCRTStartup+0x151 kernel32!BaseThreadInitThunk+0x24 ntdll!__RtlUserThreadStart+0x2f ntdll!_RtlUserThreadStart+0x1b
~0s; .ecxr ; kb
THREAD_SHA1_HASH_MOD_FUNC:
938dec2050a1e4605831341df0b0049900cc489a
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: THREAD_SHA1_HASH_MOD:
77973f77be56c743a9806c895e818a3dc0c6b5f2
FOLLOWUP_IP: notepad!WinMain+e6 009e5eb6 85c0 FAULT_INSTR_CODE: SYMBOL_STACK_INDEX:
48302f2507a707f990bbcb69a94480fc874178b2
test
eax,eax
9075c085 2
56
SYMBOL_NAME:
notepad!WinMain+e6
FOLLOWUP_NAME:
MachineOwner
MODULE_NAME: notepad IMAGE_NAME:
notepad.exe
DEBUG_FLR_IMAGE_TIMESTAMP: BUCKET_ID:
55bebe90
STATUS_BREAKPOINT_notepad!WinMain+e6
PRIMARY_PROBLEM_CLASS: BUCKET_ID_OFFSET:
STATUS_BREAKPOINT_notepad!WinMain+e6
e6
BUCKET_ID_MODULE_STR:
notepad
BUCKET_ID_MODTIMEDATESTAMP: BUCKET_ID_MODCHECKSUM:
55bebe90
37c17
BUCKET_ID_MODVER_STR:
10.0.10240.16425
BUCKET_ID_PREFIX_STR:
STATUS_BREAKPOINT_
FAILURE_PROBLEM_CLASS:
STATUS_BREAKPOINT
FAILURE_EXCEPTION_CODE: FAILURE_IMAGE_NAME:
notepad.exe
FAILURE_FUNCTION_NAME:
WinMain
BUCKET_ID_FUNCTION_STR: FAILURE_SYMBOL_NAME: FAILURE_BUCKET_ID:
80000003
WinMain
notepad.exe!WinMain STATUS_BREAKPOINT_80000003_notepad.exe!WinMain
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/notepad.exe/10.0.10240.16425/55bebe90/unknown/0.0.0.0/bbbb bbb4/80000003/00000000.htm?Retriage=1 TARGET_TIME: OSBUILD:
2016-05-01T15:07:18.000Z
10240
OSSERVICEPACK:
16384
SERVICEPACK_NUMBER: 0 OS_REVISION: 0 OSPLATFORM_TYPE: OSNAME:
x86
Windows 10
OSEDITION:
Windows 10 WinNt SingleUserTS Personal
57
USER_LCID:
0
OSBUILD_TIMESTAMP:
2015-07-10 04:25:21
BUILDDATESTAMP_STR: BUILDLAB_STR:
150709-1700
th1
BUILDOSVER_STR:
10.0.10240.16384
ANALYSIS_SESSION_ELAPSED_TIME: 1e4 ANALYSIS_SOURCE:
UM
FAILURE_ID_HASH_STRING: FAILURE_ID_HASH: Followup: ---------
um:status_breakpoint_80000003_notepad.exe!winmain
{39352512-8c1c-b033-4491-409b6d85420b}
MachineOwner
Note: “Break instruction exception“ can be the sign of Manual Dump pattern but often WinDbg is not able to figure out an exception which may be on another thread or hidden. 11.
Now we check how many threads by using ~ command:
58
12.
Now we dump a stack trace using kc command (only modules and symbols):
59
13. Now we dump the stack trace of the current thread using k command (with symbols, return addresses, and function offsets):
60
0:000> k # ChildEBP 00 04ebf8e0 01 04ebf8fc 02 04ebf93c 03 04ebf9d0 04 04ebf9e4 05 04ebfa2c 06 04ebfa3c
RetAddr 74d7325a 009e5eb6 009f5b41 749e3744 773e9e54 773e9e1f 00000000
user32!NtUserGetMessage+0xc user32!GetMessageW+0x2a notepad!WinMain+0xe6 notepad!WinMainCRTStartup+0x151 kernel32!BaseThreadInitThunk+0x24 ntdll!__RtlUserThreadStart+0x2f ntdll!_RtlUserThreadStart+0x1b
Hint: How to check that the stack trace is correct. Use ub command (unassemble backwards) to check if there is a call instruction. We check that GetMessageW function was called from WinMain function: 0:000> k # ChildEBP 00 04ebf8e0 01 04ebf8fc 02 04ebf93c 03 04ebf9d0 04 04ebf9e4 05 04ebfa2c 06 04ebfa3c
RetAddr 74d7325a 009e5eb6 009f5b41 749e3744 773e9e54 773e9e1f 00000000
user32!NtUserGetMessage+0xc user32!GetMessageW+0x2a notepad!WinMain+0xe6 notepad!WinMainCRTStartup+0x151 kernel32!BaseThreadInitThunk+0x24 ntdll!__RtlUserThreadStart+0x2f ntdll!_RtlUserThreadStart+0x1b3
61
0:000> ub 009e5eb6 notepad!WinMain+0xd2: 009e5ea2 50 009e5ea3 ff15b8a19f00 009e5ea9 53 009e5eaa 53 009e5eab 53 009e5eac 8d45e4 009e5eaf 50 009e5eb0 ff15a8a19f00
push call push push push lea push call
eax dword ptr [notepad!_imp__DispatchMessageW (009fa1b8)] ebx ebx ebx eax,[ebp-1Ch] eax dword ptr [notepad!_imp__GetMessageW (009fa1a8)]
Then we check that NtUserGetMessage function was called from GetMessageW function: 0:000> k # ChildEBP 00 04ebf8e0 01 04ebf8fc 02 04ebf93c 03 04ebf9d0 04 04ebf9e4 05 04ebfa2c 06 04ebfa3c
RetAddr 74d7325a 009e5eb6 009f5b41 749e3744 773e9e54 773e9e1f 00000000
user32!NtUserGetMessage+0xc user32!GetMessageW+0x2a notepad!WinMain+0xe6 notepad!WinMainCRTStartup+0x151 kernel32!BaseThreadInitThunk+0x24 ntdll!__RtlUserThreadStart+0x2f ntdll!_RtlUserThreadStart+0x1b
0:000> ub 74d7325a user32!GetMessageW+0x15: 74d73245 0f85c7cc0100 74d7324b 56 74d7324c 8b7508 74d7324f 50 74d73250 52 74d73251 ff750c 74d73254 56 74d73255 e8361b0000
jne push mov push push push push call
user32!GetMessageW+0x1cce2 (74d8ff12) esi esi,dword ptr [ebp+8] eax edx dword ptr [ebp+0Ch] esi user32!NtUserGetMessage (74d74d90)
62
14.
Now we dump the stack trace using verbose kv command (includes the first possible function parameters):
Note: Remember the functions call each other from bottom to top. The topmost function is the last one that was called. ExceptionAddress or FAULTING_IP may point to the last one. We would come to this in the real exception process dumps later. Here in another example below I would like to point out that the top function call func1 has a return address already (to func2), and the function was being executed somewhere in its code at 0x20 offset: 63
0:000> k ChildEBP 0024f9a0 0024f9a4 [...] 0024fa9c 0024fadc
15.
RetAddr 772c199a ModuleA!func1+0x20 772c19cd ModuleA!func2+0x16 776fa9bd kernel32!BaseThreadInitThunk+0xe 00000000 ntdll!_RtlUserThreadStart+0x23
Now we check the list of loaded modules using lm command:
64
16. We can check verbose module information using lmv command or use lmv m to check an individual module (Not My Version pattern):
65
17.
Sometimes lmv command doesn’t show much and !lmi command might give extra information:
66
Note: We can also use lmt command variant if we are interested in timestamps only. 18. Sometimes Environment Hint pattern can give troubleshooting suggestions related to environment variables and DLL paths. !peb command (Process Environment Block): 0:000> !peb PEB at 7e3da000 InheritedAddressSpace: No ReadImageFileExecOptions: No BeingDebugged: No ImageBaseAddress: 009e0000 Ldr 77498b40 Ldr.Initialized: Yes Ldr.InInitializationOrderModuleList: 051337b0 . 0513adf8 Ldr.InLoadOrderModuleList: 05133880 . 0513ade8 Ldr.InMemoryOrderModuleList: 05133888 . 0513adf0 Base TimeStamp Module 9e0000 55bebe90 Aug 03 02:06:24 2015 C:\Windows\SysWOW64\notepad.exe 77390000 56ad9358 Jan 31 04:53:44 2016 C:\WINDOWS\SYSTEM32\ntdll.dll 749d0000 559f3b21 Jul 10 04:25:21 2015 C:\WINDOWS\SYSTEM32\KERNEL32.DLL 758a0000 56e8cf1c Mar 16 03:12:28 2016 C:\WINDOWS\SYSTEM32\KERNELBASE.dll 75770000 568b1dff Jan 05 01:35:59 2016 C:\WINDOWS\SYSTEM32\ADVAPI32.dll 75460000 559f3e0e Jul 10 04:37:50 2015 C:\WINDOWS\SYSTEM32\msvcrt.dll 75850000 559f3afd Jul 10 04:24:45 2015 C:\WINDOWS\SYSTEM32\sechost.dll 75b10000 55b992ea Jul 30 03:58:50 2015 C:\WINDOWS\SYSTEM32\RPCRT4.dll 74440000 559f3af4 Jul 10 04:24:36 2015 C:\WINDOWS\SYSTEM32\SspiCli.dll 74430000 559f3af8 Jul 10 04:24:40 2015 C:\WINDOWS\SYSTEM32\CRYPTBASE.dll 743d0000 559f3c0f Jul 10 04:29:19 2015 C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll 771f0000 568b1b15 Jan 05 01:23:33 2016 C:\WINDOWS\SYSTEM32\GDI32.dll 74d40000 56553339 Nov 25 04:04:09 2015 C:\WINDOWS\SYSTEM32\USER32.dll 75bc0000 56ad9664 Jan 31 05:06:44 2016 C:\WINDOWS\SYSTEM32\combase.dll 75530000 559f3b0b Jul 10 04:24:59 2015 C:\WINDOWS\SYSTEM32\OLEAUT32.dll
67
745d0000 5655342b Nov 25 04:08:11 2015 C:\WINDOWS\SYSTEM32\COMDLG32.dll 74cb0000 559f3d59 Jul 10 04:34:49 2015 C:\WINDOWS\SYSTEM32\shcore.dll 72b80000 559f3e45 Jul 10 04:38:45 2015 C:\WINDOWS\WinSxS\x86_microsoft.windows.commoncontrols_6595b64144ccf1df_6.0.10240.16384_none_3bccb1ff6bcd1849\COMCTL32.dll 75720000 559f3c42 Jul 10 04:30:10 2015 C:\WINDOWS\SYSTEM32\SHLWAPI.dll 75df0000 56e8d63b Mar 16 03:42:51 2016 C:\WINDOWS\SYSTEM32\SHELL32.dll 74f80000 55fa574f Sep 17 07:01:51 2015 C:\WINDOWS\SYSTEM32\windows.storage.dll 757f0000 559f3aff Jul 10 04:24:47 2015 C:\WINDOWS\SYSTEM32\kernel.appcore.dll 75800000 559f3aff Jul 10 04:24:47 2015 C:\WINDOWS\SYSTEM32\powrprof.dll 74690000 559f3af5 Jul 10 04:24:37 2015 C:\WINDOWS\SYSTEM32\profapi.dll 730d0000 559f3c05 Jul 10 04:29:09 2015 C:\Windows\SYSTEM32\WINSPOOL.DRV 73d90000 559f3c18 Jul 10 04:29:28 2015 C:\Windows\SYSTEM32\bcrypt.dll 756f0000 559f3b8d Jul 10 04:27:09 2015 C:\WINDOWS\SYSTEM32\IMM32.DLL 74850000 56ad94ab Jan 31 04:59:23 2016 C:\WINDOWS\SYSTEM32\MSCTF.dll 72dc0000 55af08da Jul 22 04:07:06 2015 C:\WINDOWS\system32\uxtheme.dll 10000000 4c31b72f Jul 05 11:42:55 2010 C:\Program Files (x86)\Samsung\Easy Settings\WinCRT.dll 71e70000 55a862ea Jul 17 03:05:30 2015 C:\WINDOWS\system32\dwmapi.dll 75a20000 56cc3889 Feb 23 10:46:33 2016 C:\WINDOWS\SYSTEM32\ole32.dll 755d0000 559f3cb0 Jul 10 04:32:00 2015 C:\WINDOWS\SYSTEM32\clbcatq.dll SubSystemData: 00000000 ProcessHeap: 05130000 ProcessParameters: 05131b98 CurrentDirectory: 'C:\Windows\SysWOW64\' WindowTitle: 'C:\Windows\SysWOW64\notepad.exe' ImageFile: 'C:\Windows\SysWOW64\notepad.exe' CommandLine: '"C:\Windows\SysWOW64\notepad.exe" ' DllPath: '< Name not readable >' Environment: 051305c8 =::=::\ ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\Training\AppData\Roaming asl.log=Destination=file CommonProgramFiles=C:\Program Files (x86)\Common Files CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files CommonProgramW6432=C:\Program Files\Common Files COMPUTERNAME=TRAINING-PC ComSpec=C:\WINDOWS\system32\cmd.exe FPS_BROWSER_APP_PROFILE_STRING=Internet Explorer FPS_BROWSER_USER_PROFILE_STRING=Default FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\Training LOCALAPPDATA=C:\Users\Training\AppData\Local LOGONSERVER=\\TRAINING-PC NUMBER_OF_PROCESSORS=4 OS=Windows_NT Path=C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowe rShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x86;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET Web Pages\v1.0\;C:\Program Files (x86)\Symantec\VIP Access Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowe
68
rShell\v1.0\;C:\Program Files (x86)\Skype\Phone\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\ PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=3a09 ProgramData=C:\ProgramData ProgramFiles=C:\Program Files (x86) ProgramFiles(x86)=C:\Program Files (x86) ProgramW6432=C:\Program Files PSModulePath=C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public SESSIONNAME=Console SystemDrive=C: SystemRoot=C:\WINDOWS TEMP=C:\Users\Training\AppData\Local\Temp TMP=C:\Users\Training\AppData\Local\Temp USERDOMAIN=TRAINING-PC USERDOMAIN_ROAMINGPROFILE=TRAINING-PC USERNAME=Training USERPROFILE=C:\Users\Training VS110COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\ VS140COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\ windir=C:\WINDOWS windows_tracing_flags=3 windows_tracing_logfile=C:\BVTBin\Tests\installpackage\csilogfile.log
19.
We close logging before exiting WinDbg:
69
Note: To avoid possible confusion and glitches we recommend exiting WinDbg after each exercise.
70
Published by OpenTask, Republic of Ireland Copyright © 2018 by OpenTask
Copyright © 2018 by Software Diagnostics Services Copyright © 2018 by Dmitry Vostokov
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. Product and company names mentioned in this book may be trademarks of their owners.
OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected].
A CIP catalog record for this book is available from the British Library. ISBN-l3: 978-1-908043-87-0 (Paperback) Revision 3.0 (August 2018)
2
Contents About the Author.............................................................................................................................................................. 5 Introduction ...................................................................................................................................................................... 7 Practice Exercises ........................................................................................................................................................... 23 Exercise 0: Download, setup and verify your WinDbg installation ............................................................................ 28 Exercise PN1: Analysis of an application process dump (ApplicationA, 64-bit) ......................................................... 37 Exercise PN2: Analysis of an application process dump (ApplicationA, 32-bit) ......................................................... 56 Exercise PN3: Analysis of an application process dump (LINQPadB, 64-bit).............................................................. 72 Exercise PN4: Analysis of an application process dump (LINQPadB, 32-bit).............................................................. 95 Exercise PN5: Analysis of an application process dump (LINQPadC, 64-bit)............................................................ 118 Exercise PN6: Analysis of an application process dump (LINQPadC, 32-bit)............................................................ 133 Exercise PN7: Analysis of an application process dump (ApplicationD, 64-bit) ....................................................... 152 Exercise PN8: Analysis of an application process dump (ApplicationD, 32-bit) ....................................................... 179 Exercise PN9: Analysis of an application process dump (LINQPadD, 64-bit) ........................................................... 194 Exercise PN10: Analysis of an application process dump (LINQPadD, 32-bit) ......................................................... 210 Exercise PN11: Analysis of an application process dump (LINQPadE, 64-bit) .......................................................... 227 Exercise PN12: Analysis of an application process dump (LINQPadE, 32-bit) .......................................................... 237 Legacy Exercises ........................................................................................................................................................... 253 Exercise Legacy.0: Download, setup and verify your WinDbg installation .............................................................. 255 Exercise Legacy.PN1: Analysis of an application process dump (ApplicationA, 32-bit, CLR2) ................................. 260 Exercise Legacy.PN2: Analysis of an application process dump (ApplicationA, 32-bit, CLR4) ................................. 270 Exercise Legacy.PN3: Analysis of an application process dump (LINQPadB, 64-bit, CLR4) ...................................... 284 Exercise Legacy.PN4: Analysis of an application process dump (LINQPadB, 32-bit, CLR2) ...................................... 306 Exercise Legacy.PN5: Analysis of an application process dump (LINQPadC, 64-bit, CLR4) ...................................... 324 Exercise Legacy.PN6: Analysis of an application process dump (LINQPadC, 32-bit, CLR4) ...................................... 344 Exercise Legacy.PN7: Analysis of an application process dump (LINQPadD, 32-bit, CLR4)...................................... 364 Exercise Legacy.PN8: Analysis of an application process dump (LINQPadE, 32-bit, CLR4) ...................................... 403 Application Source Code .............................................................................................................................................. 413 ApplicationA ............................................................................................................................................................. 415 LinqB ......................................................................................................................................................................... 416 LinqC ......................................................................................................................................................................... 417 ApplicationD ............................................................................................................................................................. 419 LinqD ......................................................................................................................................................................... 421 LinqE ......................................................................................................................................................................... 423 3
Selected Q&A................................................................................................................................................................ 425
4
Exercise PN1: Analysis of an application process dump (ApplicationA, 64-bit) Goal: Learn how to load the correct .NET SOS WinDbg extension and analyze managed space. Patterns: Stack Trace Collection; CLR Thread; Version-Specific Extension; Software Exception, Exception Stack Trace, Managed Code Exception; Managed Stack Trace. Commands: .logopen, .symfix, .reload, ~*k, .load, !pe, ~*e, lmv, .chain, .unload, !analyze -v, !CLRStack, .logclose 1.
Launch WinDbg from Windows Kits \ WinDbg (X64).
2.
Open \ANETMDA-Dumps\Processes\ApplicationA.DMP
3.
We get the dump file loaded:
Note: ApplicationA shows this dialog when launched:
37
When we click on a button it shows the following exception dialog:
At this point, we saved a process memory dump on a Windows 10 x64 system using Task Manager. 4.
Open a log file using .logopen command and load symbols (.symfix and .reload commands):
0:000> .logopen C:\ANETMDA-Dumps\Processes\ApplicationA.log Opened log file 'C:\ANETMDA-Dumps\Processes\ApplicationA.log' 0:000> .symfix c:\mss 0:000> .reload ............................................................ Loading unloaded module list . *** WARNING: Unable to verify checksum for System.Windows.Forms.ni.dll *** ERROR: Module load completed but symbols could not be loaded for System.Windows.Forms.ni.dll ************* Symbol Loading Error Summary ************** Module name Error System.Windows.Forms.n 0x80190194 - Not found (404). : SRV*c:\mss*https://msdl.microsoft.com/download/symbols You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded. You should also verify that your symbol search path (.sympath) is correct.
Note: The results may be slightly different on your system if you don’t have .NET Framework 4.0.30319 installed or you have a version different from 4.7.3120.0 that was on a virtual machine where all the dumps were saved.
38
5. Type ~*k command to verify the correctness of all stack traces (the command execution time may be longer for the first time because symbol files need to be downloaded from Microsoft symbol server):
39
0:000> ~*k .
# 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35
0 Id: 7f0.22e0 Suspend: 0 Teb: 00000000`00fcc000 Unfrozen Child-SP RetAddr Call Site 00000000`0113bbc8 00007ffc`d8b933f8 win32u!NtUserWaitMessage+0x14 00000000`0113bbd0 00007ffc`d8b2f452 System_Windows_Forms_ni+0x2d33f8 00000000`0113bc80 00007ffc`d8b2ebd2 System_Windows_Forms_ni+0x26f452 00000000`0113bd70 00007ffc`d8b2e9df System_Windows_Forms_ni+0x26ebd2 00000000`0113be10 00007ffc`d9226bfd System_Windows_Forms_ni+0x26e9df 00000000`0113be70 00007ffc`d91f72f3 System_Windows_Forms_ni+0x966bfd 00000000`0113bf70 00007ffc`d920494a System_Windows_Forms_ni+0x9372f3 00000000`0113bfe0 00007ffc`d8b1a413 System_Windows_Forms_ni+0x94494a 00000000`0113c010 00007ffc`ef378a6d System_Windows_Forms_ni+0x25a413 00000000`0113c060 00007ffc`ef378934 clr!ExceptionTracker::CallHandler+0xfd 00000000`0113c150 00007ffc`ef378848 clr!ExceptionTracker::CallCatchHandler+0x90 00000000`0113c1f0 00007ffd`1918ed6d clr!ProcessCLRException+0x31c 00000000`0113c2d0 00007ffd`190f7670 ntdll!RtlpExecuteHandlerForUnwind+0xd 00000000`0113c300 00007ffc`ef379550 ntdll!RtlUnwindEx+0x3a0 00000000`0113c9e0 00007ffc`ef37950b clr!ClrUnwindEx+0x40 00000000`0113cf00 00007ffd`1918eced clr!ProcessCLRException+0x2e9 00000000`0113cfe0 00007ffd`190f6c86 ntdll!RtlpExecuteHandlerForException+0xd 00000000`0113d010 00007ffd`190f52ca ntdll!RtlDispatchException+0x3c6 00000000`0113d710 00007ffd`15d8a388 ntdll!RtlRaiseException+0x31a 00000000`0113df70 00007ffc`ef2b1209 KERNELBASE!RaiseException+0x68 00000000`0113e050 00007ffc`ef2b123b clr!NakedThrowHelper2+0x9 00000000`0113e080 00007ffc`ef2b1245 clr!NakedThrowHelper_RspAligned+0x1e 00000000`0113e5a8 00007ffc`8fcb0829 clr!NakedThrowHelper_FixRsp+0x5 00000000`0113e5b0 00007ffc`d8b060b2 0x00007ffc`8fcb0829 00000000`0113e5f0 00007ffc`d8b094cc System_Windows_Forms_ni+0x2460b2 00000000`0113e630 00007ffc`d92579cc System_Windows_Forms_ni+0x2494cc 00000000`0113e680 00007ffc`d9204602 System_Windows_Forms_ni+0x9979cc 00000000`0113e740 00007ffc`d8b1aebb System_Windows_Forms_ni+0x944602 00000000`0113e7c0 00007ffc`d8b10234 System_Windows_Forms_ni+0x25aebb 00000000`0113e880 00007ffc`d8b10184 System_Windows_Forms_ni+0x250234 00000000`0113e900 00007ffc`d8b1a3c3 System_Windows_Forms_ni+0x250184 00000000`0113e930 00007ffc`d91911f1 System_Windows_Forms_ni+0x25a3c3 00000000`0113e9d0 00007ffc`ef2b221e System_Windows_Forms_ni+0x8d11f1 00000000`0113ea40 00007ffd`17646cc1 clr!UMThunkStub+0x6e 00000000`0113ead0 00007ffd`17646693 user32!UserCallWinProcCheckWow+0x2c1 00000000`0113ec60 00007ffc`d8b9a378 user32!DispatchMessageWorker+0x1c3 00000000`0113ecf0 00007ffc`d8b2f23e System_Windows_Forms_ni+0x2da378 00000000`0113edb0 00007ffc`d8b2ebd2 System_Windows_Forms_ni+0x26f23e 00000000`0113eea0 00007ffc`d8b2e9df System_Windows_Forms_ni+0x26ebd2 00000000`0113ef40 00007ffc`8fcb04d2 System_Windows_Forms_ni+0x26e9df 00000000`0113efa0 00007ffc`ef2b6bb3 0x00007ffc`8fcb04d2 00000000`0113efe0 00007ffc`ef2b6a70 clr!CallDescrWorkerInternal+0x83 00000000`0113f020 00007ffc`ef2b735d clr!CallDescrWorkerWithHandler+0x4e 00000000`0113f060 00007ffc`ef30ec1c clr!MethodDescCallSite::CallTargetWorker+0xf8 00000000`0113f160 00007ffc`ef30ee06 clr!RunMain+0x1e7 00000000`0113f340 00007ffc`ef30ecfb clr!Assembly::ExecuteMainMethod+0xb6 00000000`0113f630 00007ffc`ef30eaf4 clr!SystemDomain::ExecuteMainMethod+0x57c 00000000`0113fc40 00007ffc`ef30ea72 clr!ExecuteEXE+0x3f 00000000`0113fcb0 00007ffc`ef30ef34 clr!_CorExeMainInternal+0xb2 00000000`0113fd40 00007ffc`efca7b2d clr!CorExeMain+0x14 00000000`0113fd80 00007ffc`f52ba4cc mscoreei!CorExeMain+0x112 00000000`0113fde0 00007ffd`165c3034 mscoree!CorExeMain_Exported+0x6c 00000000`0113fe10 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`0113fe40 00000000`00000000 ntdll!RtlUserThreadStart+0x21
40
# 00 01 02 03
1 Id: 7f0.2038 Suspend: 0 Teb: 00000000`00fce000 Unfrozen Child-SP RetAddr Call Site 00000000`0133f858 00007ffd`1910f856 ntdll!NtWaitForWorkViaWorkerFactory+0x14 00000000`0133f860 00007ffd`165c3034 ntdll!TppWorkerThread+0x536 00000000`0133fb50 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`0133fb80 00000000`00000000 ntdll!RtlUserThreadStart+0x21
# 00 01 02 03
2 Id: 7f0.203c Suspend: 0 Teb: 00000000`00fd0000 Unfrozen Child-SP RetAddr Call Site 00000000`0154f538 00007ffd`1910f856 ntdll!NtWaitForWorkViaWorkerFactory+0x14 00000000`0154f540 00007ffd`165c3034 ntdll!TppWorkerThread+0x536 00000000`0154f830 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`0154f860 00000000`00000000 ntdll!RtlUserThreadStart+0x21
# 00 01 02 03 04 05 06
3 Id: 7f0.2040 Suspend: 0 Teb: 00000000`00fd2000 Unfrozen Child-SP RetAddr Call Site 00000000`02ebf438 00007ffd`15d96099 ntdll!NtWaitForMultipleObjects+0x14 00000000`02ebf440 00007ffc`ef346a42 KERNELBASE!WaitForMultipleObjectsEx+0xf9 00000000`02ebf740 00007ffc`ef34696d clr!DebuggerRCThread::MainLoop+0xce 00000000`02ebf800 00007ffc`ef346880 clr!DebuggerRCThread::ThreadProc+0xd2 00000000`02ebf850 00007ffd`165c3034 clr!DebuggerRCThread::ThreadProcStatic+0x41 00000000`02ebf8a0 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`02ebf8d0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
# 00 01 02 03 04 05 06 07 08 09 0a
4 Id: 7f0.2058 Suspend: 0 Teb: 00000000`00fd4000 Unfrozen Child-SP RetAddr Call Site 00000000`1b4af7b8 00007ffd`15d96099 ntdll!NtWaitForMultipleObjects+0x14 00000000`1b4af7c0 00007ffc`ef372a36 KERNELBASE!WaitForMultipleObjectsEx+0xf9 00000000`1b4afac0 00007ffc`ef443b84 clr!FinalizerThread::WaitForFinalizerEvent+0xb6 00000000`1b4afb00 00007ffc`ef2b7b21 clr!FinalizerThread::FinalizerThreadWorker+0x54 00000000`1b4afb40 00007ffc`ef2b7a90 clr!ManagedThreadBase_DispatchInner+0x39 00000000`1b4afb80 00007ffc`ef2b79cd clr!ManagedThreadBase_DispatchMiddle+0x6c 00000000`1b4afc80 00007ffc`ef3374fa clr!ManagedThreadBase_DispatchOuter+0x75 00000000`1b4afd10 00007ffc`ef362e8f clr!FinalizerThread::FinalizerThreadStart+0x10a 00000000`1b4afdb0 00007ffd`165c3034 clr!Thread::intermediateThreadProc+0x86 00000000`1b4afe70 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`1b4afea0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
# 00 01 02 03 04 05
5 Id: 7f0.2030 Suspend: 0 Teb: 00000000`00fd6000 Unfrozen Child-SP RetAddr Call Site 00000000`1bb5fa48 00007ffd`1765029d win32u!NtUserMsgWaitForMultipleObjectsEx+0x14 00000000`1bb5fa50 00007ffd`021f5cf3 user32!RealMsgWaitForMultipleObjectsEx+0x1d 00000000`1bb5fa90 00007ffd`021f5c6f GdiPlus!BackgroundThreadProc+0x63 00000000`1bb5fb00 00007ffd`165c3034 GdiPlus!DllRefCountSafeThreadThunk+0x1f 00000000`1bb5fb30 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`1bb5fb60 00000000`00000000 ntdll!RtlUserThreadStart+0x21
# 00 01 02 03
6 Id: 7f0.205c Suspend: 0 Teb: 00000000`00fd8000 Unfrozen Child-SP RetAddr Call Site 00000000`1dc7fb98 00007ffd`1910f856 ntdll!NtWaitForWorkViaWorkerFactory+0x14 00000000`1dc7fba0 00007ffd`165c3034 ntdll!TppWorkerThread+0x536 00000000`1dc7fe90 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`1dc7fec0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
# 00 01 02 03
7 Id: 7f0.2184 Suspend: 0 Teb: 00000000`00fda000 Unfrozen Child-SP RetAddr Call Site 00000000`1f9cfb98 00007ffd`1910f856 ntdll!NtWaitForWorkViaWorkerFactory+0x14 00000000`1f9cfba0 00007ffd`165c3034 ntdll!TppWorkerThread+0x536 00000000`1f9cfe90 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`1f9cfec0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
41
# 00 01 02 03
8 Id: 7f0.2098 Suspend: 0 Teb: 00000000`00fdc000 Unfrozen Child-SP RetAddr Call Site 00000000`1facf9f8 00007ffd`1910f856 ntdll!NtWaitForWorkViaWorkerFactory+0x14 00000000`1facfa00 00007ffd`165c3034 ntdll!TppWorkerThread+0x536 00000000`1facfcf0 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 00000000`1facfd20 00000000`00000000 ntdll!RtlUserThreadStart+0x21
9 Id: 7f0.113c Suspend: 0 Teb: 00000000`00fde000 Unfrozen # Child-SP RetAddr Call Site 00 00000000`1fbcf1b8 00007ffd`15d96099 ntdll!NtWaitForMultipleObjects+0x14 01 00000000`1fbcf1c0 00007ffd`17382ab7 KERNELBASE!WaitForMultipleObjectsEx+0xf9 02 00000000`1fbcf4c0 00007ffd`1737ce40 combase!WaitCoalesced+0xb3 [onecore\com\published\comutils\coalescedwait.cxx @ 72] 03 00000000`1fbcf750 00007ffd`1737ff11 combase!CROIDTable::WorkerThreadLoop+0x50 [onecore\com\combase\dcomrem\refcache.cxx @ 1650] 04 00000000`1fbcf7a0 00007ffd`173c75dc combase!CRpcThread::WorkerLoop+0x169 [onecore\com\combase\dcomrem\threads.cxx @ 269] 05 00000000`1fbcf800 00007ffd`165c3034 combase!CRpcThreadCache::RpcWorkerThreadEntry+0x7c [onecore\com\combase\dcomrem\threads.cxx @ 76] 06 00000000`1fbcf830 00007ffd`19161431 kernel32!BaseThreadInitThunk+0x14 07 00000000`1fbcf860 00000000`00000000 ntdll!RtlUserThreadStart+0x21
Note: We see that threads #0, #3, #4 have clr module on their stack traces (old version of .NET 2.x used mscorwks module as can be seen in exercise Legacy.PN1). We also see signs of software exception (in red) and exception stack trace #0 which has signs of managed code exception processing (in yellow). 6. Since .NET Framework version can be different on a machine where the dump file was saved we need to load the corresponding WinDbg SOS extension version. In the folder C:\ANETMDA-Dumps\Framework64\v4.0.30319 we have the correct version of .NET Framework copied from the machine the memory dump came from. So we load SOS WinDbg extension (.load command): 0:000> .load C:\ANETMDA-Dumps\Framework64\v4.0.30319\SOS
7.
We check if there is a .NET exception on the current thread 0:
0:000> !pe Exception object: 0000000002fa3cb0 Exception type: System.NullReferenceException Message: Object reference not set to an instance of an object. InnerException:
StackTrace (generated): SP IP Function 000000000113E5B0 00007FFC8FCB0829 ApplicationA!ApplicationA.Form1.button1_Click_1(System.Object, System.EventArgs)+0x39 000000000113E5F0 00007FFCD8B060B2 System_Windows_Forms_ni!System.Windows.Forms.Control.OnClick(System.EventArgs)+0x82 000000000113E630 00007FFCD8B094CC System_Windows_Forms_ni!System.Windows.Forms.Button.OnClick(System.EventArgs)+0xbc 000000000113E680 00007FFCD92579CC System_Windows_Forms_ni!System.Windows.Forms.Button.OnMouseUp(System.Windows.Forms.MouseEventAr gs)+0x14c 000000000113E740 00007FFCD9204602 System_Windows_Forms_ni!System.Windows.Forms.Control.WmMouseUp(System.Windows.Forms.Message ByRef, System.Windows.Forms.MouseButtons, Int32)+0x2d2
42
000000000113E7C0 00007FFCD8B1AEBB System_Windows_Forms_ni!System.Windows.Forms.Control.WndProc(System.Windows.Forms.Message ByRef)+0x97b 000000000113E880 00007FFCD8B10234 System_Windows_Forms_ni!System.Windows.Forms.ButtonBase.WndProc(System.Windows.Forms.Message ByRef)+0x84 000000000113E900 00007FFCD8B10184 System_Windows_Forms_ni!System.Windows.Forms.Button.WndProc(System.Windows.Forms.Message ByRef)+0x24 000000000113E930 00007FFCD8B1A3C3 System_Windows_Forms_ni!System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)+0xc3 StackTraceString: HResult: 80004003
Note: We also double check that no other threads have exceptions by executing !pe command for each thread using ~*e command: 0:000> ~*e !pe Exception object: 0000000002fa3cb0 Exception type: System.NullReferenceException Message: Object reference not set to an instance of an object. InnerException:
StackTrace (generated): SP IP Function 000000000113E5B0 00007FFC8FCB0829 ApplicationA!ApplicationA.Form1.button1_Click_1(System.Object, System.EventArgs)+0x39 000000000113E5F0 00007FFCD8B060B2 System_Windows_Forms_ni!System.Windows.Forms.Control.OnClick(System.EventArgs)+0x82 000000000113E630 00007FFCD8B094CC System_Windows_Forms_ni!System.Windows.Forms.Button.OnClick(System.EventArgs)+0xbc 000000000113E680 00007FFCD92579CC System_Windows_Forms_ni!System.Windows.Forms.Button.OnMouseUp(System.Windows.Forms.MouseEventAr gs)+0x14c 000000000113E740 00007FFCD9204602 System_Windows_Forms_ni!System.Windows.Forms.Control.WmMouseUp(System.Windows.Forms.Message ByRef, System.Windows.Forms.MouseButtons, Int32)+0x2d2 000000000113E7C0 00007FFCD8B1AEBB System_Windows_Forms_ni!System.Windows.Forms.Control.WndProc(System.Windows.Forms.Message ByRef)+0x97b 000000000113E880 00007FFCD8B10234 System_Windows_Forms_ni!System.Windows.Forms.ButtonBase.WndProc(System.Windows.Forms.Message ByRef)+0x84 000000000113E900 00007FFCD8B10184 System_Windows_Forms_ni!System.Windows.Forms.Button.WndProc(System.Windows.Forms.Message ByRef)+0x24 000000000113E930 00007FFCD8B1A3C3 System_Windows_Forms_ni!System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr)+0xc3 StackTraceString: HResult: 80004003 The current thread is unmanaged The current thread is unmanaged The current thread is unmanaged There is no current managed exception on this thread The current thread is unmanaged The current thread is unmanaged
43
The current thread is unmanaged The current thread is unmanaged The current thread is unmanaged
8.
We now check the version of .NET used when ApplicationA was running:
0:000> lmv m clr Browse full module list start end module name 00007ffc`ef2b0000 00007ffc`efc9c000 clr (pdb symbols) c:\mss\clr.pdb\89AF76D6C0C841F8884C33E9CD93C8FF2\clr.pdb Loaded symbol image file: clr.dll Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll Image name: clr.dll Browse all global symbols functions data Timestamp: Fri May 25 18:28:01 2018 (5B08B821) CheckSum: 009E96E0 ImageSize: 009EC000 File version: 4.7.3120.0 Product version: 4.0.30319.0 File flags: 8 (Mask 3F) Private File OS: 4 Unknown Win32 File type: 2.0 Dll File date: 00000000.00000000 Translations: 0409.04b0 Information from resource tables: CompanyName: Microsoft Corporation ProductName: Microsoft® .NET Framework InternalName: clr.dll OriginalFilename: clr.dll ProductVersion: 4.7.3120.0 FileVersion: 4.7.3120.0 built by: NET472REL1LAST PrivateBuild: DDBLD413 FileDescription: Microsoft .NET Runtime Common Language Runtime - WorkStation LegalCopyright: © Microsoft Corporation. All rights reserved. Comments: Flavor=Retail
44
Note: On my analysis system the version is slightly different:
It has a different .3131 version suffix. The version can also be checked by listing all loaded WinDbg extensions (sos.dll is used for .NET analysis): 0:000> .chain Extension DLL search Path: [...] Extension DLL chain: c:\mss\SOS_AMD64_AMD64_4.7.3120.00.dll\5B08B8219ec000\SOS_AMD64_AMD64_4.7.3120.00.dll: image 4.7.3120.0, API 1.0.0, built Fri May 25 18:20:07 2018 [path: c:\mss\SOS_AMD64_AMD64_4.7.3120.00.dll\5B08B8219ec000\SOS_AMD64_AMD64_4.7.3120.00.dll] C:\ANETMDA-Dumps\Framework64\v4.0.30319\SOS: image 4.7.3120.0, API 1.0.0, built Fri May 25 18:20:07 2018 [path: C:\ANETMDA-Dumps\Framework64\v4.0.30319\SOS.dll] dbghelp: image 10.0.17134.12, API 10.0.6, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\dbghelp.dll] ext: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\ext.dll] exts: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\exts.dll] uext: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\uext.dll] ntsdexts: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\ntsdexts.dll]
45
Note: We see two SOS extension DLLs loaded having the same timestamp but different paths. The top one was probably downloaded from Microsoft symbol server and loaded as the resut of !pe command. We can unload them one after another and check !pe command again (which shouldn’t be available): 0:000> .unload SOS_AMD64_AMD64_4.7.3120.00 Unloading c:\mss\SOS_AMD64_AMD64_4.7.3120.00.dll\5B08B8219ec000\SOS_AMD64_AMD64_4.7.3120.00.dll extension DLL 0:000> .chain Extension DLL search Path: [...] Extension DLL chain: C:\ANETMDA-Dumps\Framework64\v4.0.30319\SOS: image 4.7.3120.0, API 1.0.0, built Fri May 25 18:20:07 2018 [path: C:\ANETMDA-Dumps\Framework64\v4.0.30319\SOS.dll] dbghelp: image 10.0.17134.12, API 10.0.6, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\dbghelp.dll] ext: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\ext.dll] exts: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\exts.dll] uext: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\uext.dll] ntsdexts: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP\ntsdexts.dll] 0:000> .unload SOS Unloading C:\ANETMDA-Dumps\Framework64\v4.0.30319\SOS extension DLL 0:000> .chain Extension DLL search Path: [...] Extension DLL chain: dbghelp: image 10.0.17134.12, API 10.0.6, [path: C:\Program Files (x86)\Windows ext: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows exts: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows uext: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows ntsdexts: image 10.0.17134.12, API 1.0.0, [path: C:\Program Files (x86)\Windows
Kits\10\Debuggers\x64\dbghelp.dll] Kits\10\Debuggers\x64\winext\ext.dll] Kits\10\Debuggers\x64\WINXP\exts.dll] Kits\10\Debuggers\x64\winext\uext.dll] Kits\10\Debuggers\x64\WINXP\ntsdexts.dll]
0:000> !pe No export pe found
46
9.
Let’s see what !analyze -v command says:
0:000> !analyze -v ******************************************************************************* * * * Exception Analysis * * * ******************************************************************************* *** ERROR: Module load completed but symbols could not be loaded for mscorlib.ni.dll *** WARNING: Unable to verify checksum for ApplicationA.exe GetUrlPageData2 (WinHttp) failed: 12002. KEY_VALUES_STRING: 1 TIMELINE_ANALYSIS: 1 Timeline: Name: Time: Diff:
!analyze.Start
2018-07-27T23:53:37.297Z 1569881297 mSec
Timeline: Name: Time: Diff:
Dump.Current
2018-07-09T19:48:56.0Z 0 mSec
Timeline: Name: Time: Diff:
Process.Start
2018-07-09T19:48:20.0Z 36000 mSec
Timeline: Name: Time: Diff:
OS.Boot
2018-07-08T16:43:01.0Z 97555000 mSec
DUMP_CLASS: 2 DUMP_QUALIFIER: 400 FAULTING_IP: +0 00000000`00000000 ??
???
EXCEPTION_RECORD: 000000001e58d400 -- (.exr 0x1e58d400) ExceptionAddress: 00050001ffff0006 ExceptionCode: 00010000 ExceptionFlags: 00050003 NumberParameters: 131071 Parameter[0]: 0005000300010000 Parameter[1]: 000300010000ffff Parameter[2]: 00010000ffff0006 Parameter[3]: 0000ffff00050003 Parameter[4]: ffff000600030001 Parameter[5]: 0005000300010000 Parameter[6]: 000300010000ffff Parameter[7]: 00010000ffff0006
47
Parameter[8]: 0026ffff00050003 Parameter[9]: ffff003a0039002c Parameter[10]: 0005000300010000 Parameter[11]: 000300010000ffff Parameter[12]: 00010000ffff0006 Parameter[13]: 0000ffff00050003 Parameter[14]: ffff000600030001 FAULTING_THREAD:
000022e0
DEFAULT_BUCKET_ID: PROCESS_NAME:
BREAKPOINT_NOSOS
ApplicationA.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}
Breakpoint
A breakpoint has been reached.
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid EXCEPTION_CODE_STR:
80000003
WATSON_BKT_PROCSTAMP: WATSON_BKT_PROCVER:
5b43b8ae 1.0.0.0
PROCESS_VER_PRODUCT:
ApplicationA
WATSON_BKT_MODULE:
unknown
WATSON_BKT_MODVER:
0.0.0.0
WATSON_BKT_MODOFFSET: WATSON_BKT_MODSTAMP:
0 bbbbbbb4
BUILD_VERSION_STRING:
17134.1.amd64fre.rs4_release.180410-1804
MODLIST_WITH_TSCHKSUM_HASH: MODLIST_SHA1_HASH: NTGLOBALFLAG:
a035b8758813cf1c8d02cba3f73b17e1bf0cb64f
cfe07c3c7dceb6b7fc873c4345687f87357309a6
0
PROCESS_BAM_CURRENT_THROTTLED: 0 PROCESS_BAM_PREVIOUS_THROTTLED: 0 APPLICATION_VERIFIER_FLAGS: PRODUCT_TYPE:
1
SUITE_MASK:
784
DUMP_FLAGS:
8000c07
DUMP_TYPE:
0
3
MISSING_CLR_SYMBOL: 0 ANALYSIS_SESSION_HOST:
DESKTOP-IS6V2L0
48
ANALYSIS_SESSION_TIME:
07-28-2018 00:53:37.0297
ANALYSIS_VERSION: 10.0.17134.12 amd64fre MANAGED_CODE: 1 MANAGED_ENGINE_MODULE:
clr
CONTEXT: 0000000051661bf8 -- (.cxr 0x51661bf8) Unable to read context, HRESULT 0x80004002 THREAD_ATTRIBUTES: OS_LOCALE: ENI ADDITIONAL_DEBUG_TEXT:
SOS.DLL is not loaded for managed code. Analysis might be incomplete
PROBLEM_CLASSES: ID: Type: Class: Scope: Name: Data: PID: TID: Frame: ID: Type: Class: Scope: Name: Data: PID: TID: Frame:
[0n317] [@APPLICATION_FAULT_STRING] Primary DEFAULT_BUCKET_ID (Failure Bucket ID prefix) BUCKET_ID Omit Add String: [BREAKPOINT] [Unspecified] [Unspecified] [0] [0n247] [NOSOS] Addendum DEFAULT_BUCKET_ID (Failure Bucket ID prefix) BUCKET_ID Add Omit [Unspecified] [Unspecified] [0]
BUGCHECK_STR:
BREAKPOINT_NOSOS
PRIMARY_PROBLEM_CLASS:
BREAKPOINT
LAST_CONTROL_TRANSFER:
from 00007ffcd8b933f8 to 00007ffd16171204
STACK_TEXT: 00000000`0113bbc8 0000cb83`56ecf3c2 00000000`0113bbd0 00000000`00000000 00000000`0113bc80 00007ffc`d8b68996 00000000`0113bd70 00007ffc`d91f629c 00000000`0113be10 00000000`0113be60 00000000`0113be70 00000000`00000000
00007ffc`d8b933f8 : 00000000`02f87908 : win32u!NtUserWaitMessage+0x14 00007ffc`d8b2f452 : 00000000`02f87908 : System_Windows_Forms_ni+0x2d33f8 00007ffc`d8b2ebd2 : 00000000`02f920a8 : System_Windows_Forms_ni+0x26f452 00007ffc`d8b2e9df : 00000000`02f87908 : System_Windows_Forms_ni+0x26ebd2 00007ffc`d9226bfd : 00000000`01390e50 : System_Windows_Forms_ni+0x26e9df 00007ffc`d91f72f3 : 00000000`02faacb8 : System_Windows_Forms_ni+0x966bfd
49
00007ffc`d8b2f6d9 00000000`00000000 00000000`0113bce0 00000000`0113bcf0 00000000`00000001 0000cb83`56ecf3c2 00000000`00000004 00000000`02fd4070 00000000`0113bea0 00000000`00001000 00000000`00000000 00000000`02fd34d8
00000000`0113bf70 00000000`01390e50 00000000`0113bfe0 00007ffc`d88f7ea0 00000000`0113c010 00007ffc`d8b1a3ea 00000000`0113c060 00000000`0138ea68 00000000`0113c150 00000000`00000001 00000000`0113c1f0 00000000`0113c3c0 00000000`0113c2d0 00000000`0113ca10 00000000`0113c300 00000000`00000000 00000000`0113c9e0 00000000`00000000 00000000`0113cf00 00000000`0113d0c0 00000000`0113cfe0 00000000`00000000 00000000`0113d010 00000000`00000000 00000000`0113d710 00000000`02fa1028 00000000`0113df70 00000000`00000000 00000000`0113e050 00000000`00000000 00000000`0113e080 00000000`0113e730 00000000`0113e5a8 00000000`00000002 00000000`0113e5b0 00000000`00000000 00000000`0113e5f0 00000000`0113e730 00000000`0113e630 00000000`0113e730 00000000`0113e680 000000a2`00000103 00000000`0113e740 00000000`00000004 00000000`0113e7c0 00000000`00000000 00000000`0113e880 00000000`02f8a6e0 00000000`0113e900 00000000`0000000f 00000000`0113e930 00007ffd`19123f93 00000000`0113e9d0 00007ffd`17646b37 00000000`0113ea40 00000000`00000000 00000000`0113ead0 00000000`00000202 00000000`0113ec60 00007ffc`d8b2f6d9
00007ffc`d920494a : 00000000`02f87908 00000000`02fa3cb0 : System_Windows_Forms_ni+0x9372f3 00007ffc`d8b1a413 : 00000000`02f8a598 00000000`02fa3cb0 : System_Windows_Forms_ni+0x94494a 00007ffc`ef378a6d : 00000000`00000004 00000000`01390e50 : System_Windows_Forms_ni+0x25a413 00007ffc`ef378934 : 00000000`0138e9d0 00007ffc`d8b1a3ea : clr!ExceptionTracker::CallHandler+0xfd 00007ffc`ef378848 : 00000000`0113e930 00000000`0113ca10 : clr!ExceptionTracker::CallCatchHandler+0x90 00007ffd`1918ed6d : 00007ffc`d8bbac00 00000000`0113e930 : clr!ProcessCLRException+0x31c 00007ffd`190f7670 : 00000000`0113c400 00000000`0113e930 : ntdll!RtlpExecuteHandlerForUnwind+0xd 00007ffc`ef379550 : 00000000`0113d0c0 00000000`00000000 : ntdll!RtlUnwindEx+0x3a0 00007ffc`ef37950b : 00000000`00000000 00000000`0113d0c0 : clr!ClrUnwindEx+0x40 00007ffd`1918eced : 00007ffc`d8bbaca4 00000000`0113e930 : clr!ProcessCLRException+0x2e9 00007ffd`190f6c86 : 00000000`0113d110 00000000`0113d960 : ntdll!RtlpExecuteHandlerForException+0xd 00007ffd`190f52ca : 00000000`1e58d400 00000000`51661bf8 : ntdll!RtlDispatchException+0x3c6 00007ffd`15d8a388 : 00000000`00000000 00000000`02f8b8a0 : ntdll!RtlRaiseException+0x31a 00007ffc`ef2b1209 : 00000000`00000000 00000000`00000000 : KERNELBASE!RaiseException+0x68 00007ffc`ef2b123b : 00000000`00000000 00000000`00000000 : clr!NakedThrowHelper2+0x9 00007ffc`ef2b1245 : 00007ffc`8fcb0829 00000000`02f8a598 : clr!NakedThrowHelper_RspAligned+0x1e 00007ffc`8fcb0829 : 00000000`02f8a598 00000000`02f8b8a0 : clr!NakedThrowHelper_FixRsp+0x5 00007ffc`d8b060b2 : 00000000`02f27ee8 00000000`02f8a598 : 0x00007ffc`8fcb0829 00007ffc`d8b094cc : 00000000`02f27ee8 00000000`00000000 : System_Windows_Forms_ni+0x2460b2 00007ffc`d92579cc : 00000000`02f27ee8 00000000`00000155 : System_Windows_Forms_ni+0x2494cc 00007ffc`d9204602 : 00000000`02f8a598 00000000`02fa1028 : System_Windows_Forms_ni+0x9979cc 00007ffc`d8b1aebb : 00000000`02f8a598 00000000`0113e860 : System_Windows_Forms_ni+0x944602 00007ffc`d8b10234 : 00000000`00000000 00007ffd`13ac369f : System_Windows_Forms_ni+0x25aebb 00007ffc`d8b10184 : 00000000`02f8a598 00000000`00000000 : System_Windows_Forms_ni+0x250234 00007ffc`d8b1a3c3 : 00000000`00000000 00000000`00000000 : System_Windows_Forms_ni+0x250184 00007ffc`d91911f1 : 00000000`02f8a6e0 00000000`00000000 : System_Windows_Forms_ni+0x25a3c3 00007ffc`ef2b221e : 00000000`00000070 ffffffff`febd718f : System_Windows_Forms_ni+0x8d11f1 00007ffd`17646cc1 : 00000000`80006010 00000000`00000000 : clr!UMThunkStub+0x6e 00007ffd`17646693 : 00000000`0113ed00 00000000`1b990c2c : user32!UserCallWinProcCheckWow+0x2c1 00007ffc`d8b9a378 : 00000000`0113ee10 00000000`00000000 : user32!DispatchMessageWorker+0x1c3
50
00000000`01390e50 00000000`01390e50 00000000`01390e50 00000000`0113e930 00000000`0113c269 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000001 00000000`00000000 00000000`00000000 00000000`0113d780 00000000`02f8a598 00000000`00000000 00000000`00000000 00000000`02f8b8a0 00000000`0113e730 00000000`02fa1028 00000000`0113e678 00000000`0113e678 0000c9a6`c076a0d7 00000003`00000000 00000000`0000000f 00000000`00000000 00000103`00000001 00000000`00000202 ffffffff`febffe97 00000000`00000000 00000000`001c040c 00000000`0113eda0
00000000`0113ecf0 00000000`00000000 00000000`0113edb0 00000000`00000000 00000000`0113eea0 00000000`0113f210 00000000`0113ef40 00000000`01390e50 00000000`0113efa0 00007ffc`00000000 00000000`0113efe0 00007ffc`ef2c4570 00000000`0113f020 00000000`0113f2c8 00000000`0113f060 00000000`00000000 00000000`0113f160 00000000`00000000 00000000`0113f340 00000000`01384dd0 00000000`0113f630 00000000`00000000 00000000`0113fc40 00000000`00000000 00000000`0113fcb0 00000000`00000000 00000000`0113fd40 00000000`0113fd18 00000000`0113fd80 00000000`00000000 00000000`0113fde0 00000000`00000000 00000000`0113fe10 00000000`00000000 00000000`0113fe40 00000000`00000000
00007ffc`d8b2f23e : 00000000`02f87908 00000000`0113ee10 : System_Windows_Forms_ni+0x2da378 00007ffc`d8b2ebd2 : 00000000`02f920a8 00000000`00000001 : System_Windows_Forms_ni+0x26f23e 00007ffc`d8b2e9df : 00000000`02f87908 00000000`ffffffff : System_Windows_Forms_ni+0x26ebd2 00007ffc`8fcb04d2 : 00000000`02f87908 00000000`ffffffff : System_Windows_Forms_ni+0x26e9df 00007ffc`ef2b6bb3 : 00007ffc`ef2b72e9 00007ffc`8fba4118 : 0x00007ffc`8fcb04d2 00007ffc`ef2b6a70 : 00000000`00df3067 00007ffc`ef2b78b9 : clr!CallDescrWorkerInternal+0x83 00007ffc`ef2b735d : 00000000`00000000 00000000`0113f188 : clr!CallDescrWorkerWithHandler+0x4e 00007ffc`ef30ec1c : 00000000`0113f110 00000000`00000000 : clr!MethodDescCallSite::CallTargetWorker+0xf8 00007ffc`ef30ee06 : 00000000`00000000 00000000`00000001 : clr!RunMain+0x1e7 00007ffc`ef30ecfb : 00007ffc`ef394a40 00000000`01384dd0 : clr!Assembly::ExecuteMainMethod+0xb6 00007ffc`ef30eaf4 : 00000000`00000000 00000000`00df0000 : clr!SystemDomain::ExecuteMainMethod+0x57c 00007ffc`ef30ea72 : 00000000`00df0000 00007ffc`ef30ef20 : clr!ExecuteEXE+0x3f 00007ffc`ef30ef34 : ffffffff`ffffffff 00007ffc`ef30ef20 : clr!_CorExeMainInternal+0xb2 00007ffc`efca7b2d : 00000000`00000000 00007ffd`00000091 : clr!CorExeMain+0x14 00007ffc`f52ba4cc : 00000000`00000000 00007ffc`ef30ef20 : mscoreei!CorExeMain+0x112 00007ffd`165c3034 : 00007ffc`efca0000 00000000`00000000 : mscoree!CorExeMain_Exported+0x6c 00007ffd`19161431 : 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x14 00000000`00000000 : 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21
00000000`00000000 00000000`ffffffff 00000000`02f8ccd8 00000000`02f8ccd8 00000000`00000000 00000000`0113f390 00000000`0113f210 00000000`00000000 00000000`00000000 00007ffc`ef394a40 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
STACK_COMMAND: dt ntdll!LdrpLastDllInitializer BaseDllName ; dt ntdll!LdrpFailureData ; .cxr 0x51661bf8 ; kb THREAD_SHA1_HASH_MOD_FUNC:
887d086448f96d24f3b65f66fc60a3e4bdb1e4a7
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: THREAD_SHA1_HASH_MOD:
af8bef11d1bf76b3e133b20a1a20ebffc06a9385
FOLLOWUP_IP: win32u!NtUserWaitMessage+14 00007ffd`16171204 c3 FAULT_INSTR_CODE:
FOLLOWUP_NAME:
ret
c32ecdc3
SYMBOL_STACK_INDEX: SYMBOL_NAME:
fe2edf247b80cd0b68ce89d015c32bb1c5fd1220
0
win32u!NtUserWaitMessage+14 MachineOwner
MODULE_NAME: win32u
51
IMAGE_NAME:
win32u.dll
DEBUG_FLR_IMAGE_TIMESTAMP: BUCKET_ID:
0
BREAKPOINT_NOSOS_win32u!NtUserWaitMessage+14
FAILURE_EXCEPTION_CODE: FAILURE_IMAGE_NAME:
80000003
win32u.dll
BUCKET_ID_IMAGE_STR:
win32u.dll
FAILURE_MODULE_NAME:
win32u
BUCKET_ID_MODULE_STR:
win32u
FAILURE_FUNCTION_NAME: BUCKET_ID_FUNCTION_STR: BUCKET_ID_OFFSET:
NtUserWaitMessage NtUserWaitMessage
14
BUCKET_ID_MODTIMEDATESTAMP: BUCKET_ID_MODCHECKSUM:
0
27b98
BUCKET_ID_MODVER_STR:
10.0.17134.1
BUCKET_ID_PREFIX_STR:
BREAKPOINT_NOSOS_
FAILURE_PROBLEM_CLASS: FAILURE_SYMBOL_NAME: FAILURE_BUCKET_ID:
BREAKPOINT
win32u.dll!NtUserWaitMessage BREAKPOINT_NOSOS_80000003_win32u.dll!NtUserWaitMessage
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/ApplicationA.exe/1.0.0.0/5b43b8ae/unknown/0.0.0.0/bbbbbbb4 /80000003/00000000.htm?Retriage=1 TARGET_TIME: OSBUILD:
2018-07-09T19:48:56.000Z
17134
OSSERVICEPACK:
1
SERVICEPACK_NUMBER: 0 OS_REVISION: 0 OSPLATFORM_TYPE: OSNAME:
x64
Windows 10
OSEDITION:
Windows 10 WinNt SingleUserTS Personal
USER_LCID:
0
OSBUILD_TIMESTAMP:
2020-08-28 05:38:41
52
BUILDDATESTAMP_STR: BUILDLAB_STR:
180410-1804
rs4_release
BUILDOSVER_STR:
10.0.17134.1.amd64fre.rs4_release.180410-1804
ANALYSIS_SESSION_ELAPSED_TIME: ANALYSIS_SOURCE:
UM
FAILURE_ID_HASH_STRING: FAILURE_ID_HASH: Followup: ---------
70e3
um:breakpoint_nosos_80000003_win32u.dll!ntuserwaitmessage
{c13a261a-1261-0b6a-f27a-a40bf396360c}
MachineOwner
Note: We see normal manual dump breakpoint error (in blue) but no .NET diagnostics (in red). 10.
Finally, we get managed stack trace of the current thread:
0:000> !CLRStack OS Thread Id: 0x22e0 (0) Child SP IP Call Site 000000000113bbf8 00007ffd16171204 [InlinedCallFrame: 000000000113bbf8] System.Windows.Forms.UnsafeNativeMethods.WaitMessage() 000000000113bbf8 00007ffcd8b933f8 [InlinedCallFrame: 000000000113bbf8] System.Windows.Forms.UnsafeNativeMethods.WaitMessage() 000000000113bbd0 00007ffcd8b933f8 DomainBoundILStubClass.IL_STUB_PInvoke() 000000000113bc80 00007ffcd8b2f452 System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMso ComponentManager.FPushMessageLoop(IntPtr, Int32, Int32) 000000000113bd70 00007ffcd8b2ebd2 System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext) 000000000113be10 00007ffcd8b2e9df System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext) 000000000113be70 00007ffcd9226bfd System.Windows.Forms.Form.ShowDialog(System.Windows.Forms.IWin32Window) 000000000113bf70 00007ffcd91f72f3 System.Windows.Forms.Application+ThreadContext.OnThreadException(System.Exception) 000000000113bfe0 00007ffcd920494a System.Windows.Forms.Control.WndProcException(System.Exception) 000000000113c010 00007ffcd8b1a413 System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr) 000000000113e0b0 00007ffcef378a6d [FaultingExceptionFrame: 000000000113e0b0] 000000000113e5b0 00007ffc8fcb0829 ApplicationA.Form1.button1_Click_1(System.Object, System.EventArgs) 000000000113e5f0 00007ffcd8b060b2 System.Windows.Forms.Control.OnClick(System.EventArgs) 000000000113e630 00007ffcd8b094cc System.Windows.Forms.Button.OnClick(System.EventArgs) 000000000113e680 00007ffcd92579cc System.Windows.Forms.Button.OnMouseUp(System.Windows.Forms.MouseEventArgs) 000000000113e740 00007ffcd9204602 System.Windows.Forms.Control.WmMouseUp(System.Windows.Forms.Message ByRef, System.Windows.Forms.MouseButtons, Int32) 000000000113e7c0 00007ffcd8b1aebb System.Windows.Forms.Control.WndProc(System.Windows.Forms.Message ByRef)
53
000000000113e880 00007ffcd8b10234 System.Windows.Forms.ButtonBase.WndProc(System.Windows.Forms.Message ByRef) 000000000113e900 00007ffcd8b10184 System.Windows.Forms.Button.WndProc(System.Windows.Forms.Message ByRef) 000000000113e930 00007ffcd8b1a3c3 System.Windows.Forms.NativeWindow.Callback(IntPtr, Int32, IntPtr, IntPtr) 000000000113e9d0 00007ffcd91911f1 DomainBoundILStubClass.IL_STUB_ReversePInvoke(Int64, Int32, Int64, Int64) 000000000113ed20 00007ffcef2b221e [InlinedCallFrame: 000000000113ed20] System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG ByRef) 000000000113ed20 00007ffcd8b9a378 [InlinedCallFrame: 000000000113ed20] System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG ByRef) 000000000113ecf0 00007ffcd8b9a378 DomainBoundILStubClass.IL_STUB_PInvoke(MSG ByRef) 000000000113edb0 00007ffcd8b2f23e System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMso ComponentManager.FPushMessageLoop(IntPtr, Int32, Int32) 000000000113eea0 00007ffcd8b2ebd2 System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext) 000000000113ef40 00007ffcd8b2e9df System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext) 000000000113efa0 00007ffc8fcb04d2 ApplicationA.Program.Main() 000000000113f210 00007ffcef2b6bb3 [GCFrame: 000000000113f210]
11.
We close logging before exiting WinDbg:
0:000> .logclose Closing open log file C:\ANETMDA-Dumps\Processes\ApplicationA.log
Note: To avoid possible confusion and glitches we recommend exiting WinDbg after each exercise.
54
Published by OpenTask, Republic of Ireland Copyright © 2014 by OpenTask Copyright © 2014 by Software Diagnostics Services Copyright © 2014 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalogue record for this book is available from the British Library. ISBN-l3: 978-1-908043-71-9 (Paperback) 1st printing, 2014
2
Contents Presentation Slides and Transcript ................................................................................................................................... 5 Core Dump Collection..................................................................................................................................................... 25 Practice Exercises ........................................................................................................................................................... 31 Exercise 0 (GDB) ......................................................................................................................................................... 36 Exercise 0 (LLDB)......................................................................................................................................................... 39 Exercise A1 (GDB) ....................................................................................................................................................... 42 Exercise A1 (LLDB) ...................................................................................................................................................... 54 Exercise A2 (GDB) ....................................................................................................................................................... 66 Exercise A2 (LLDB) ...................................................................................................................................................... 74 Exercise A3 (GDB) ....................................................................................................................................................... 83 Exercise A3 (LLDB) ...................................................................................................................................................... 88 Exercise A4 (GDB) ....................................................................................................................................................... 94 Exercise A4 (LLDB) .................................................................................................................................................... 105 Exercise A5 (GDB) ..................................................................................................................................................... 115 Exercise A5 (LLDB) .................................................................................................................................................... 121 Exercise A6 (GDB) ..................................................................................................................................................... 129 Exercise A6 (LLDB) .................................................................................................................................................... 155 Exercise A7 (GDB) ..................................................................................................................................................... 176 Exercise A7 (LLDB) .................................................................................................................................................... 184 Exercise A8 (GDB) ..................................................................................................................................................... 192 Exercise A8 (LLDB) .................................................................................................................................................... 207 Exercise A9 (GDB) ..................................................................................................................................................... 222 Exercise A9 (LLDB) .................................................................................................................................................... 249 Exercise A10 (GDB) ................................................................................................................................................... 277 Exercise A10 (LLDB) .................................................................................................................................................. 290 Exercise A11 (GDB) ................................................................................................................................................... 305 Exercise A11 (LLDB) .................................................................................................................................................. 312 Exercise A12 (GDB) ................................................................................................................................................... 321 Exercise A12 (LLDB) .................................................................................................................................................. 344 App Source Code .......................................................................................................................................................... 353 App0 ......................................................................................................................................................................... 354 App1 ......................................................................................................................................................................... 355 App2 ......................................................................................................................................................................... 356 3
App3 ......................................................................................................................................................................... 358 App4 ......................................................................................................................................................................... 360 App5 ......................................................................................................................................................................... 362 App6 ......................................................................................................................................................................... 364 App7 ......................................................................................................................................................................... 366 App8 ......................................................................................................................................................................... 368 App9 ......................................................................................................................................................................... 370 App10 ....................................................................................................................................................................... 372 App11 ....................................................................................................................................................................... 374 Selected Patterns.......................................................................................................................................................... 377 NULL Pointer (data) .................................................................................................................................................. 378 Incomplete Stack Trace ............................................................................................................................................ 379 Stack Trace................................................................................................................................................................ 380 Multiple Exceptions .................................................................................................................................................. 381 Shared Buffer Overwrite........................................................................................................................................... 382 Incorrect Stack Trace ................................................................................................................................................ 386 NULL Pointer (code).................................................................................................................................................. 387 Spiking Thread .......................................................................................................................................................... 389 Dynamic Memory Corruption (process heap) .......................................................................................................... 391 Double Free (process heap)...................................................................................................................................... 392 Execution Residue .................................................................................................................................................... 393 Coincidental Symbolic Information .......................................................................................................................... 395 Stack Overflow (user mode) ..................................................................................................................................... 397 Divide by Zero (user mode) ...................................................................................................................................... 400 Local Buffer Overflow ............................................................................................................................................... 401 C++ Exception ........................................................................................................................................................... 403 Truncated Dump ....................................................................................................................................................... 404 Paratext .................................................................................................................................................................... 405
4
Exercise A1 (GDB) Goal: Learn how to list stack traces, disassemble functions, check their correctness, dump data, compare core dumps with diagnostic reports, get environment Patterns: Manual Dump, Stack Trace, Stack Trace Collection, Annotated Disassembly, Paratext, Not My Version, Environment Hint
1.
Load a core dump core.1394 and App1 executable:
$ gdb -c ~/Documents/AMCDA-Dumps/core.1394 -e ~/Documents/AMCDADumps/Apps/App1/Build/Products/Release/App1 GNU gdb 6.3.50-20050815 (Apple version gdb-1820) (Sat Jun 16 02:40:11 UTC 2012) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-apple-darwin". Reading symbols for shared libraries . done Reading symbols for shared libraries .......................... done #0 0x00007fff8a10ce42 in __semwait_signal ()
2.
List all threads:
(gdb) info threads 6 0x00007fff8a10ce42 5 0x00007fff8a10ce42 4 0x00007fff8a10ce42 3 0x00007fff8a10ce42 2 0x00007fff8a10ce42 * 1 0x00007fff8a10ce42
3.
in in in in in in
__semwait_signal __semwait_signal __semwait_signal __semwait_signal __semwait_signal __semwait_signal
() () () () () ()
Get all thread stack traces:
(gdb) thread apply all bt Thread 6 (core thread 5): #0 0x00007fff8a10ce42 in #1 0x00007fff84d6edea in #2 0x00007fff84d6ec2c in #3 0x00007fff84d6ec08 in #4 0x000000010390bbb2 in #5 0x000000010390bbc9 in #6 0x000000010390bbe1 in #7 0x00007fff84db88bf in #8 0x00007fff84dbbb75 in
__semwait_signal () nanosleep () sleep () sleep () bar_five () foo_five () thread_five () _pthread_start () thread_start ()
42
Thread 5 (core thread 4): #0 0x00007fff8a10ce42 in #1 0x00007fff84d6edea in #2 0x00007fff84d6ec2c in #3 0x00007fff84d6ec08 in #4 0x000000010390bb52 in #5 0x000000010390bb69 in #6 0x000000010390bb81 in #7 0x00007fff84db88bf in #8 0x00007fff84dbbb75 in
__semwait_signal () nanosleep () sleep () sleep () bar_four () foo_four () thread_four () _pthread_start () thread_start ()
Thread 4 (core thread 3): #0 0x00007fff8a10ce42 in #1 0x00007fff84d6edea in #2 0x00007fff84d6ec2c in #3 0x00007fff84d6ec08 in #4 0x000000010390baf2 in #5 0x000000010390bb09 in #6 0x000000010390bb21 in #7 0x00007fff84db88bf in #8 0x00007fff84dbbb75 in
__semwait_signal () nanosleep () sleep () sleep () bar_three () foo_three () thread_three () _pthread_start () thread_start ()
Thread 3 (core thread 2): #0 0x00007fff8a10ce42 in __semwait_signal () #1 0x00007fff84d6edea in nanosleep () #2 0x00007fff84d6ec2c in sleep () #3 0x00007fff84d6ec08 in sleep () #4 0x000000010390ba92 in bar_two () #5 0x000000010390baa9 in foo_two () #6 0x000000010390bac1 in thread_two () ---Type to continue, or q to quit--#7 0x00007fff84db88bf in _pthread_start () #8 0x00007fff84dbbb75 in thread_start () Thread 2 (core thread 1): #0 0x00007fff8a10ce42 in #1 0x00007fff84d6edea in #2 0x00007fff84d6ec2c in #3 0x00007fff84d6ec08 in #4 0x000000010390ba32 in #5 0x000000010390ba49 in #6 0x000000010390ba61 in #7 0x00007fff84db88bf in #8 0x00007fff84dbbb75 in
__semwait_signal () nanosleep () sleep () sleep () bar_one () foo_one () thread_one () _pthread_start () thread_start ()
Thread 1 (core thread 0): #0 0x00007fff8a10ce42 in #1 0x00007fff84d6edea in #2 0x00007fff84d6ec2c in #3 0x00007fff84d6ec08 in #4 0x000000010390bcc3 in
__semwait_signal () nanosleep () sleep () sleep () main ()
4.
Switch to the thread #3 and get its stack trace:
(gdb) thread 3 [Switching to thread 3 (core thread 2)] 0x00007fff8a10ce42 in __semwait_signal ()
43
(gdb) bt #0 0x00007fff8a10ce42 #1 0x00007fff84d6edea #2 0x00007fff84d6ec2c #3 0x00007fff84d6ec08 #4 0x000000010390ba92 #5 0x000000010390baa9 #6 0x000000010390bac1 #7 0x00007fff84db88bf #8 0x00007fff84dbbb75
5.
in in in in in in in in in
__semwait_signal () nanosleep () sleep () sleep () bar_two () foo_two () thread_two () _pthread_start () thread_start ()
Check that bar_two called sleep function:
(gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x000000010390ba80 : push %rbp 0x000000010390ba81 : mov %rsp,%rbp 0x000000010390ba84 : sub $0x10,%rsp 0x000000010390ba88 : mov $0xffffffff,%edi 0x000000010390ba8d : callq 0x10390bce0 0x000000010390ba92 : mov %eax,-0x4(%rbp) 0x000000010390ba95 : add $0x10,%rsp 0x000000010390ba99 : pop %rbp 0x000000010390ba9a : retq 0x000000010390ba9b : nopl 0x0(%rax,%rax,1) End of assembler dump.
6.
Compare with intel disassembly flavor:
(gdb) set disassembly-flavor intel (gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x000000010390ba80 : push rbp 0x000000010390ba81 : mov rbp,rsp 0x000000010390ba84 : sub rsp,0x10 0x000000010390ba88 : mov edi,0xffffffff 0x000000010390ba8d : call 0x10390bce0 0x000000010390ba92 : mov DWORD PTR [rbp-0x4],eax 0x000000010390ba95 : add rsp,0x10 0x000000010390ba99 : pop rbp 0x000000010390ba9a : ret 0x000000010390ba9b : nop DWORD PTR [rax+rax+0x0] End of assembler dump. (gdb) set disassembly-flavor att
44
7.
Follow bar_two to sleep function code:
(gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x000000010390ba80 : push %rbp 0x000000010390ba81 : mov %rsp,%rbp 0x000000010390ba84 : sub $0x10,%rsp 0x000000010390ba88 : mov $0xffffffff,%edi 0x000000010390ba8d : callq 0x10390bce0 0x000000010390ba92 : mov %eax,-0x4(%rbp) 0x000000010390ba95 : add $0x10,%rsp 0x000000010390ba99 : pop %rbp 0x000000010390ba9a : retq 0x000000010390ba9b : nopl 0x0(%rax,%rax,1) End of assembler dump. (gdb) disassemble dyld_stub_sleep Dump of assembler code for function dyld_stub_sleep: 0x000000010390bce0 : jmpq *0x362(%rip) End of assembler dump.
8. it:
# 0x10390c048
Dump the annotated value as a memory address interpreting its contents as a symbol and then disassemble
(gdb) x/a 0x10390c048 0x10390c048: 0x7fff84d6ebef
(gdb) disassemble 0x7fff84d6ebef Dump of assembler code for function sleep: 0x00007fff84d6ebef : push %rbp 0x00007fff84d6ebf0 : mov %rsp,%rbp 0x00007fff84d6ebf3 : push %rbx 0x00007fff84d6ebf4 : sub $0x28,%rsp 0x00007fff84d6ebf8 : test %edi,%edi 0x00007fff84d6ebfa : mov %edi,%ebx 0x00007fff84d6ebfc : jns 0x7fff84d6ec11 0x00007fff84d6ebfe : mov $0x7fffffff,%edi 0x00007fff84d6ec03 : callq 0x7fff84d6ebef 0x00007fff84d6ec08 : lea -0x7fffffff(%rbx,%rax,1),%eax 0x00007fff84d6ec0f : jmp 0x7fff84d6ec4f 0x00007fff84d6ec11 : mov %ebx,%eax 0x00007fff84d6ec13 : mov %rax,-0x18(%rbp) 0x00007fff84d6ec17 : movq $0x0,-0x10(%rbp) 0x00007fff84d6ec1f : lea -0x18(%rbp),%rdi 0x00007fff84d6ec23 : lea -0x28(%rbp),%rsi 0x00007fff84d6ec27 : callq 0x7fff84d6ed46 0x00007fff84d6ec2c : cmp $0xffffffffffffffff,%eax 0x00007fff84d6ec2f : je 0x7fff84d6ec37 0x00007fff84d6ec31 : xor %ebx,%ebx 0x00007fff84d6ec33 : mov %ebx,%eax 0x00007fff84d6ec35 : jmp 0x7fff84d6ec4f 0x00007fff84d6ec37 : callq 0x7fff84e0cc88 0x00007fff84d6ec3c : cmpl $0x4,(%rax) 0x00007fff84d6ec3f : jne 0x7fff84d6ec33 0x00007fff84d6ec41 : cmpq $0x0,-0x20(%rbp) 0x00007fff84d6ec46 : setne %al 0x00007fff84d6ec49 : movzbl %al,%eax 0x00007fff84d6ec4c : add -0x28(%rbp),%eax 0x00007fff84d6ec4f : add $0x28,%rsp
45
0x00007fff84d6ec53 : pop 0x00007fff84d6ec54 : pop 0x00007fff84d6ec55 : retq End of assembler dump.
9.
%rbx %rbp
Repeat the same with resolving DYLD trampoline stub command:
(gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x000000010390ba80 : push %rbp 0x000000010390ba81 : mov %rsp,%rbp 0x000000010390ba84 : sub $0x10,%rsp 0x000000010390ba88 : mov $0xffffffff,%edi 0x000000010390ba8d : callq 0x10390bce0 0x000000010390ba92 : mov %eax,-0x4(%rbp) 0x000000010390ba95 : add $0x10,%rsp 0x000000010390ba99 : pop %rbp 0x000000010390ba9a : retq 0x000000010390ba9b : nopl 0x0(%rax,%rax,1) End of assembler dump. (gdb) info trampoline 0x10390bce0 Function at 0x10390bce0 becomes 0x7fff84d6ebef becomes 0x0
10. Compare stack trace for thread #3 (core thread 2) and its module info with the diagnostic report App1_1394.crash: Process: Path: Identifier: Version: Code Type: Parent Process:
App1 [1394] /Users/USER/Documents/*/App1 App1 ??? (???) X86-64 (Native) bash [661]
Date/Time: OS Version: Report Version:
2012-07-24 00:20:26.078 +0100 Mac OS X 10.7.4 (11E53) 9
Crashed Thread:
0
Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib 0x00007fff8a10ce42 1 libsystem_c.dylib 0x00007fff84d6edea 2 libsystem_c.dylib 0x00007fff84d6ec2c 3 libsystem_c.dylib 0x00007fff84d6ec08 4 App1 0x000000010390bcc3 5 App1 0x000000010390ba14
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 main + 195 start + 52
Thread 1: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_one + 18 foo_one + 9 thread_one + 17 _pthread_start + 335 thread_start + 13
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390ba32 0x000000010390ba49 0x000000010390ba61 0x00007fff84db88bf 0x00007fff84dbbb75
46
Thread 2: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390ba92 0x000000010390baa9 0x000000010390bac1 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_two + 18 foo_two + 9 thread_two + 17 _pthread_start + 335 thread_start + 13
Thread 3: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390baf2 0x000000010390bb09 0x000000010390bb21 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_three + 18 foo_three + 9 thread_three + 17 _pthread_start + 335 thread_start + 13
Thread 4: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390bb52 0x000000010390bb69 0x000000010390bb81 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_four + 18 foo_four + 9 thread_four + 17 _pthread_start + 335 thread_start + 13
Thread 5: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390bbb2 0x000000010390bbc9 0x000000010390bbe1 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_five + 18 foo_five + 9 thread_five + 17 _pthread_start + 335 thread_start + 13
Thread 0 crashed with X86 Thread State (64-bit): rax: 0x0000000000000004 rbx: 0x00007fff6350aa08 rdi: 0x0000000000000c03 rsi: 0x0000000000000000 r8: 0x000000007fffffff r9: 0x0000000000000000 r12: 0x0000000000000000 r13: 0x0000000000000000 rip: 0x00007fff8a10ce42 rfl: 0x0000000000000247 Logical CPU: 0
rcx: rbp: r10: r14: cr2:
0x00007fff6350a9c8 0x00007fff6350a9f0 0x0000000000000001 0x00007fff6350aa18 0x0000000103d0b880
rdx: rsp: r11: r15:
0x0000000000000001 0x00007fff6350a9c8 0xffffff80002da8d0 0x0000000000000000
Binary Images: 0x10390b000 0x10390bfff +App1 (??? - ???) /Users/USER/Documents/*/App1 0x7fff6350b000 0x7fff6353fbaf dyld (195.6 - ???) /usr/lib/dyld 0x7fff849f2000 0x7fff84a0ffff libxpc.dylib (77.19.0 - compatibility 1.0.0) /usr/lib/system/libxpc.dylib 0x7fff84d68000 0x7fff84d69ff7 libsystem_blocks.dylib (53.0.0 - compatibility 1.0.0) /usr/lib/system/libsystem_blocks.dylib 0x7fff84d6a000 0x7fff84e47fef libsystem_c.dylib (763.13.0 - compatibility 1.0.0) /usr/lib/system/libsystem_c.dylib 0x7fff85022000 0x7fff85030fff libdispatch.dylib (187.9.0 - compatibility 1.0.0) /usr/lib/system/libdispatch.dylib 0x7fff855f0000 0x7fff855f1fff libunc.dylib (24.0.0 - compatibility 1.0.0) /usr/lib/system/libunc.dylib 0x7fff85ae3000 0x7fff85ae4ff7 libremovefile.dylib (21.1.0 - compatibility 1.0.0) /usr/lib/system/libremovefile.dylib 0x7fff89114000 0x7fff89118fff libmathCommon.A.dylib (2026.0.0 - compatibility 1.0.0) /usr/lib/system/libmathCommon.A.dylib 0x7fff89119000 0x7fff8911dfff libdyld.dylib (195.5.0 - compatibility 1.0.0) /usr/lib/system/libdyld.dylib 0x7fff89740000 0x7fff89741ff7 libsystem_sandbox.dylib (??? - ???) /usr/lib/system/libsystem_sandbox.dylib 0x7fff8a0ef000 0x7fff8a0f5fff libmacho.dylib (800.0.0 - compatibility 1.0.0) /usr/lib/system/libmacho.dylib
47
0x7fff8a0f6000 0x7fff8a116fff libsystem_kernel.dylib (1699.26.8 - compatibility 1.0.0) /usr/lib/system/libsystem_kernel.dylib 0x7fff8a2ac000 0x7fff8a2b4fff libsystem_dnssd.dylib (??? - ???) /usr/lib/system/libsystem_dnssd.dylib 0x7fff8ae26000 0x7fff8ae61fff libsystem_info.dylib (??? - ???) /usr/lib/system/libsystem_info.dylib 0x7fff8b248000 0x7fff8b24afff libquarantine.dylib (36.6.0 - compatibility 1.0.0) /usr/lib/system/libquarantine.dylib 0x7fff8b3b4000 0x7fff8b3b4fff libkeymgr.dylib (23.0.0 - compatibility 1.0.0) /usr/lib/system/libkeymgr.dylib 0x7fff8b3dd000 0x7fff8b3e2fff libcompiler_rt.dylib (6.0.0 - compatibility 1.0.0) /usr/lib/system/libcompiler_rt.dylib 0x7fff8bd1a000 0x7fff8bd1bfff libdnsinfo.dylib (395.11.0 - compatibility 1.0.0) /usr/lib/system/libdnsinfo.dylib 0x7fff8c528000 0x7fff8c52dff7 libsystem_network.dylib (??? - ???) /usr/lib/system/libsystem_network.dylib 0x7fff8cfa3000 0x7fff8cfadff7 liblaunch.dylib (392.38.0 - compatibility 1.0.0) /usr/lib/system/liblaunch.dylib 0x7fff8fe02000 0x7fff8fe09fff libcopyfile.dylib (85.1.0 - compatibility 1.0.0) /usr/lib/system/libcopyfile.dylib 0x7fff8fe4b000 0x7fff8fe8dff7 libcommonCrypto.dylib (55010.0.0 - compatibility 1.0.0) /usr/lib/system/libcommonCrypto.dylib 0x7fff90c0f000 0x7fff90c18ff7 libsystem_notify.dylib (80.1.0 - compatibility 1.0.0) /usr/lib/system/libsystem_notify.dylib 0x7fff91376000 0x7fff913a3fe7 libSystem.B.dylib (159.1.0 - compatibility 1.0.0) /usr/lib/libSystem.B.dylib 0x7fff91489000 0x7fff9148fff7 libunwind.dylib (30.0.0 - compatibility 1.0.0) /usr/lib/system/libunwind.dylib 0x7fff91a22000 0x7fff91a27fff libcache.dylib (47.0.0 - compatibility 1.0.0) /usr/lib/system/libcache.dylib External Modification Summary: Calls made by other processes targeting this process: task_for_pid: 2 thread_create: 0 thread_set_state: 0 Calls made by this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by all processes on this machine: task_for_pid: 2696 thread_create: 0 thread_set_state: 0 VM Region Summary: ReadOnly portion of Libraries: Total=50.2M resident=50.2M(100%) swapped_out_or_unallocated=0K(0%) Writable regions: Total=38.9M written=10.8M(28%) resident=42.6M(110%) swapped_out=0K(0%) unallocated=16777216.0T(45221404475392%) REGION TYPE =========== MALLOC Stack __DATA __LINKEDIT __TEXT shared memory =========== TOTAL
VIRTUAL ======= 1220K 66.6M 464K 47.7M 2484K 12K ======= 118.4M
48
11.
Get App1 data section from the output of vmmap_1394.log:
Virtual Memory Map of process 1394 (App1) Output report format: 2.2 -- 64-bit process ==== Non-writable regions for process 1394 __TEXT 000000010390b000-000000010390c000 [ Dumps/Apps/App1/Build/Products/Release/App1
4K] r-x/rwx SM=COW
/Users/DumpAnalysis/Documents/AMCDA-
4K] rw-/rwx SM=PRV
/Users/DumpAnalysis/Documents/AMCDA-
[...] ==== Writable regions for process 1394 __DATA 000000010390c000-000000010390d000 [ Dumps/Apps/App1/Build/Products/Release/App1 [...]
12.
Compare with the section information in the core dump:
(gdb) maintenance info sections Exec file: `/Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1', file type mach-o-le. 0x0000000000000000->0x0000000000000000 at 0x00000000: LC_SEGMENT.__PAGEZERO ALLOC LOAD CODE HAS_CONTENTS 0x0000000100000000->0x0000000100001000 at 0x00000000: LC_SEGMENT.__TEXT ALLOC LOAD CODE HAS_CONTENTS 0x00000001000009e0->0x0000000100000cd3 at 0x000009e0: LC_SEGMENT.__TEXT.__text ALLOC LOAD READONLY CODE HAS_CONTENTS 0x0000000100000cd4->0x0000000100000ce6 at 0x00000cd4: LC_SEGMENT.__TEXT.__stubs ALLOC LOAD CODE HAS_CONTENTS 0x0000000100000ce8->0x0000000100000d16 at 0x00000ce8: LC_SEGMENT.__TEXT.__stub_helper ALLOC LOAD CODE HAS_CONTENTS 0x0000000100000d16->0x0000000100000d66 at 0x00000d16: LC_SEGMENT.__TEXT.__unwind_info ALLOC LOAD CODE HAS_CONTENTS 0x0000000100000d68->0x0000000100001000 at 0x00000d68: LC_SEGMENT.__TEXT.__eh_frame ALLOC LOAD CODE HAS_CONTENTS 0x0000000100001000->0x0000000100002000 at 0x00001000: LC_SEGMENT.__DATA ALLOC LOAD CODE HAS_CONTENTS 0x0000000100001000->0x0000000100001028 at 0x00001000: LC_SEGMENT.__DATA.__program_vars ALLOC LOAD CODE HAS_CONTENTS 0x0000000100001028->0x0000000100001038 at 0x00001028: LC_SEGMENT.__DATA.__nl_symbol_ptr ALLOC LOAD CODE HAS_CONTENTS 0x0000000100001038->0x0000000100001050 at 0x00001038: LC_SEGMENT.__DATA.__la_symbol_ptr ALLOC LOAD CODE HAS_CONTENTS 0x0000000100001050->0x0000000100001070 at 0x00000000: LC_SEGMENT.__DATA.__common ALLOC 0x0000000100002000->0x00000001000023b0 at 0x00002000: LC_SEGMENT.__LINKEDIT ALLOC LOAD CODE HAS_CONTENTS 0x0000000000000000->0x00000000000001a0 at 0x000020d0: LC_SYMTAB.stabs HAS_CONTENTS 0x0000000000000000->0x0000000000000120 at 0x00002290: LC_SYMTAB.stabstr HAS_CONTENTS 0x0000000000000000->0x0000000000000100 at 0x000020d0: LC_DYSYMTAB.localstabs HAS_CONTENTS 0x0000000000000000->0x00000000000000a0 at 0x000021d0: LC_DYSYMTAB.nonlocalstabs HAS_CONTENTS 0x0000000000000000->0x0000000000000018 at 0x000004b0: LC_LOAD_DYLINKER HAS_CONTENTS 0x0000000000000000->0x00000000000000a8 at 0x00000500: LC_THREAD.x86_THREAD_STATE64.0 HAS_CONTENTS 0x0000000000000000->0x0000000000000030 at 0x000005b0: LC_LOAD_DYLIB HAS_CONTENTS Core file: `/Users/DumpAnalysis/Documents/AMCDA-Dumps/core.1394', file type mach-o-le. 0x000000010390b000->0x000000010390c000 at 0x00002000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010390c000->0x000000010390d000 at 0x00003000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010390d000->0x000000010390e000 at 0x00004000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010390e000->0x000000010390f000 at 0x00005000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010390f000->0x0000000103910000 at 0x00006000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103910000->0x0000000103911000 at 0x00007000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103911000->0x0000000103926000 at 0x00008000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103926000->0x0000000103927000 at 0x0001d000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103927000->0x0000000103928000 at 0x0001e000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103928000->0x000000010393d000 at 0x0001f000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010393d000->0x000000010393e000 at 0x00034000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010393e000->0x000000010393f000 at 0x00035000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x000000010393f000->0x0000000103940000 at 0x00036000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103940000->0x00000001039c2000 at 0x00037000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103a00000->0x0000000103b00000 at 0x000b9000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103b00000->0x0000000103b01000 at 0x001b9000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103b01000->0x0000000103b83000 at 0x001ba000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103b83000->0x0000000103b84000 at 0x0023c000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103b84000->0x0000000103c06000 at 0x0023d000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103c06000->0x0000000103c07000 at 0x002bf000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103c07000->0x0000000103c89000 at 0x002c0000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103c89000->0x0000000103c8a000 at 0x00342000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS 0x0000000103c8a000->0x0000000103d0c000 at 0x00343000: LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS
49
0x00007fff5f50b000->0x00007fff62d0b000 0x00007fff62d0b000->0x00007fff6350a000 0x00007fff6350a000->0x00007fff6350b000 0x00007fff6350b000->0x00007fff63540000 0x00007fff63540000->0x00007fff63542000 0x00007fff63542000->0x00007fff6357c000 0x00007fff6357c000->0x00007fff6358f000 0x00007fff749b8000->0x00007fff74a00000 0x00007fff74a00000->0x00007fff74c00000 0x00007fff74c00000->0x00007fff74e00000 0x00007fff74e00000->0x00007fff75000000 0x00007fff75000000->0x00007fff75200000 0x00007fff75200000->0x00007fff75400000 0x00007fff75400000->0x00007fff75600000 0x00007fff75600000->0x00007fff75800000 0x00007fff75800000->0x00007fff75a00000 0x00007fff75a00000->0x00007fff75c00000 0x00007fff75c00000->0x00007fff75e00000 0x00007fff75e00000->0x00007fff76200000 0x00007fff76200000->0x00007fff76400000 0x00007fff76400000->0x00007fff764ac000 0x00007fff849b8000->0x00007fff91a28000 0x00007fff91a28000->0x00007fff94b30000 0x00007fffffe00000->0x00007fffffe02000 0x0000000000000000->0x00000000000000b0 0x0000000000000000->0x0000000000000214 0x0000000000000000->0x0000000000000018 0x0000000000000000->0x00000000000000b0 0x0000000000000000->0x0000000000000214 0x0000000000000000->0x0000000000000018 0x0000000000000000->0x00000000000000b0 0x0000000000000000->0x0000000000000214 0x0000000000000000->0x0000000000000018 0x0000000000000000->0x00000000000000b0 0x0000000000000000->0x0000000000000214 0x0000000000000000->0x0000000000000018 0x0000000000000000->0x00000000000000b0 0x0000000000000000->0x0000000000000214 0x0000000000000000->0x0000000000000018 0x0000000000000000->0x00000000000000b0 0x0000000000000000->0x0000000000000214 0x0000000000000000->0x0000000000000018
13.
at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at at
0x003c5000: 0x03bc5000: 0x043c4000: 0x043c5000: 0x043fa000: 0x043fc000: 0x04436000: 0x04449000: 0x04491000: 0x04691000: 0x04891000: 0x04a91000: 0x04c91000: 0x04e91000: 0x05091000: 0x05291000: 0x05491000: 0x05691000: 0x05891000: 0x05c91000: 0x05e91000: 0x05f3d000: 0x12fad000: 0x160b5000: 0x00000d68: 0x00000e20: 0x0000103c: 0x00001064: 0x0000111c: 0x00001338: 0x00001360: 0x00001418: 0x00001634: 0x0000165c: 0x00001714: 0x00001930: 0x00001958: 0x00001a10: 0x00001c2c: 0x00001c54: 0x00001d0c: 0x00001f28:
LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_SEGMENT. ALLOC LOAD CODE HAS_CONTENTS LC_THREAD.x86_THREAD_STATE.0 HAS_CONTENTS LC_THREAD.x86_FLOAT_STATE.0 HAS_CONTENTS LC_THREAD.x86_EXCEPTION_STATE.0 HAS_CONTENTS LC_THREAD.x86_THREAD_STATE.1 HAS_CONTENTS LC_THREAD.x86_FLOAT_STATE.1 HAS_CONTENTS LC_THREAD.x86_EXCEPTION_STATE.1 HAS_CONTENTS LC_THREAD.x86_THREAD_STATE.2 HAS_CONTENTS LC_THREAD.x86_FLOAT_STATE.2 HAS_CONTENTS LC_THREAD.x86_EXCEPTION_STATE.2 HAS_CONTENTS LC_THREAD.x86_THREAD_STATE.3 HAS_CONTENTS LC_THREAD.x86_FLOAT_STATE.3 HAS_CONTENTS LC_THREAD.x86_EXCEPTION_STATE.3 HAS_CONTENTS LC_THREAD.x86_THREAD_STATE.4 HAS_CONTENTS LC_THREAD.x86_FLOAT_STATE.4 HAS_CONTENTS LC_THREAD.x86_EXCEPTION_STATE.4 HAS_CONTENTS LC_THREAD.x86_THREAD_STATE.5 HAS_CONTENTS LC_THREAD.x86_FLOAT_STATE.5 HAS_CONTENTS LC_THREAD.x86_EXCEPTION_STATE.5 HAS_CONTENTS
Dump data with possible symbolic information:
(gdb) x/512a 0x000000010390c000 0x10390c000: 0x10390b000 0x10390c050 0x10390c010: 0x10390c058 0x10390c060 0x10390c020: 0x10390c068 0x7fff8911a6a0 0x10390c030: 0x7fff63546d80 0x10390bcf8 0x10390c040: 0x7fff84dbab01 0x7fff84d6ebef 0x10390c050 : 0x1 0x7fff6350aaf0 0x10390c060 : 0x7fff6350ab00 0x7fff6350ac73 0x10390c070: 0x0 0x0 0x10390c080: 0x0 0x0 0x10390c090: 0x0 0x0 0x10390c0a0: 0x0 0x0 0x10390c0b0: 0x0 0x0 0x10390c0c0: 0x0 0x0 0x10390c0d0: 0x0 0x0 0x10390c0e0: 0x0 0x0 0x10390c0f0: 0x0 0x0 0x10390c100: 0x0 0x0 0x10390c110: 0x0 0x0 0x10390c120: 0x0 0x0 0x10390c130: 0x0 0x0 0x10390c140: 0x0 0x0
50
0x10390c150: 0x0 0x10390c160: 0x0 0x10390c170: 0x0 0x10390c180: 0x0 0x10390c190: 0x0 0x10390c1a0: 0x0 0x10390c1b0: 0x0 0x10390c1c0: 0x0 0x10390c1d0: 0x0 0x10390c1e0: 0x0 0x10390c1f0: 0x0 0x10390c200: 0x0 0x10390c210: 0x0 0x10390c220: 0x0 0x10390c230: 0x0 0x10390c240: 0x0 0x10390c250: 0x0 0x10390c260: 0x0 0x10390c270: 0x0 0x10390c280: 0x0 0x10390c290: 0x0 ---Type to Quit
14.
0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 continue, or q to quit---q
Dump the contents of memory pointed to by environ variable in null-terminated string format:
(gdb) x/100s 0x7fff6350ab00 [...] 0x7fff6350abd5: "" 0x7fff6350abd6: "" 0x7fff6350abd7: "" 0x7fff6350abd8: "/Users/DumpAnalysis/Documents/AMCDADumps/Apps/App1/Build/Products/Release/App1" 0x7fff6350ac28: "/Users/DumpAnalysis/Documents/AMCDADumps/Apps/App1/Build/Products/Release/App1" 0x7fff6350ac78: "TERM_PROGRAM=Apple_Terminal" 0x7fff6350ac94: "TERM=xterm-256color" 0x7fff6350aca8: "SHELL=/bin/bash" 0x7fff6350acb8: "TMPDIR=/var/folders/ww/rmtqfhl93yj4213dnl2rqy6w0000gn/T/" 0x7fff6350acf1: "Apple_PubSub_Socket_Render=/tmp/launch-mYEvtN/Render" 0x7fff6350ad26: "TERM_PROGRAM_VERSION=303.2" 0x7fff6350ad41: "TERM_SESSION_ID=2B039506-8384-4620-B354-120BE31AEA84" 0x7fff6350ad76: "USER=DumpAnalysis" 0x7fff6350ad88: "COMMAND_MODE=unix2003" 0x7fff6350ad9e: "SSH_AUTH_SOCK=/tmp/launch-9sm7dH/Listeners" 0x7fff6350adc9: "__CF_USER_TEXT_ENCODING=0x1F5:0:0" 0x7fff6350adeb: "Apple_Ubiquity_Message=/tmp/launch-tWsFs8/Apple_Ubiquity_Message" 0x7fff6350ae2c: "PATH=/Applications/Xcode.app/Contents/Developer/usr/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/lo cal/bin:/usr/X11/bin" 0x7fff6350ae9f: "PWD=/Users/DumpAnalysis" 0x7fff6350aeb7: "LANG=en_IE.UTF-8" ---Type to continue, or q to quit--0x7fff6350aec8: "SHLVL=1" 0x7fff6350aed0: "HOME=/Users/DumpAnalysis" 0x7fff6350aee9: "LOGNAME=DumpAnalysis" 0x7fff6350aefe: "DISPLAY=/tmp/launch-M8cgb1/org.x:0" 0x7fff6350af21: "SECURITYSESSIONID=186af"
51
0x7fff6350af39: "_=/Users/DumpAnalysis/Documents/AMCDADumps/Apps/App1/Build/Products/Release/App1" 0x7fff6350af8b: "OLDPWD=/usr/share/man/man1" 0x7fff6350afa6: "" 0x7fff6350afa7: "" 0x7fff6350afa8: "stack_guard=0x74843dc6068699c3" 0x7fff6350afc7: "malloc_entropy=0x7406669509034332,0x71e4e2253a6d22b0" 0x7fff6350affc: "" 0x7fff6350affd: ""
15.
Get the list of loaded modules:
(gdb) info sharedlibrary The DYLD shared library state has been initialized from the executable's shared library information. All symbols should be present, but the addresses of some symbols may move when the program is executed, as DYLD may relocate library load addresses if necessary. Requested State Current State Num Basename Type Address Reason | | Source | | | | | | | | 1 App1 - 0x10390b000 exec Y Y /Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1 at 0x10390b000 (offset 0x390b000) (objfile is) [memory object "/Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1" at 0x10390b000] 2 dyld - 0x7fff6350b000 dyld Y Y /usr/lib/dyld at 0x7fff6350b000 (offset 0x7fff6350b001) with prefix "__dyld_" (objfile is) [memory object "/usr/lib/dyld" at 0x7fff6350b000] 3 libSystem.B.dylib - 0x7fff91376000 dyld Y Y /usr/lib/libSystem.B.dylib at 0x7fff91376000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/libSystem.B.dylib" at 0x7fff91376000] 4 libcache.dylib - 0x7fff91a22000 dyld Y Y /usr/lib/system/libcache.dylib at 0x7fff91a22000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libcache.dylib" at 0x7fff91a22000] 5 libcommonCrypto.dylib - 0x7fff8fe4b000 dyld Y Y /usr/lib/system/libcommonCrypto.dylib at 0x7fff8fe4b000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libcommonCrypto.dylib" at 0x7fff8fe4b000] 6 libcompiler_rt.dylib - 0x7fff8b3dd000 dyld Y Y /usr/lib/system/libcompiler_rt.dylib at 0x7fff8b3dd000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libcompiler_rt.dylib" at 0x7fff8b3dd000] 7 libcopyfile.dylib - 0x7fff8fe02000 dyld Y Y /usr/lib/system/libcopyfile.dylib at 0x7fff8fe02000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libcopyfile.dylib" at 0x7fff8fe02000] 8 libdispatch.dylib - 0x7fff85022000 dyld Y Y /usr/lib/system/libdispatch.dylib at 0x7fff85022000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libdispatch.dylib" at 0x7fff85022000] 9 libdnsinfo.dylib - 0x7fff8bd1a000 dyld Y Y /usr/lib/system/libdnsinfo.dylib at 0x7fff8bd1a000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libdnsinfo.dylib" at 0x7fff8bd1a000] 10 libdyld.dylib - 0x7fff89119000 dyld Y Y /usr/lib/system/libdyld.dylib at 0x7fff89119000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libdyld.dylib" at 0x7fff89119000] 11 libkeymgr.dylib - 0x7fff8b3b4000 dyld Y Y /usr/lib/system/libkeymgr.dylib at 0x7fff8b3b4000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libkeymgr.dylib" at 0x7fff8b3b4000] 12 liblaunch.dylib - 0x7fff8cfa3000 dyld Y Y /usr/lib/system/liblaunch.dylib at 0x7fff8cfa3000 (offset 0x49b800 ---Type to continue, or q to quit--0) (objfile is) [memory object "/usr/lib/system/liblaunch.dylib" at 0x7fff8cfa3000] 13 libmacho.dylib - 0x7fff8a0ef000 dyld Y Y /usr/lib/system/libmacho.dylib at 0x7fff8a0ef000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libmacho.dylib" at 0x7fff8a0ef000] 14 libmathCommon.A.dylib - 0x7fff89114000 dyld Y Y /usr/lib/system/libmathCommon.A.dylib at 0x7fff89114000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libmathCommon.A.dylib" at 0x7fff89114000] 15 libquarantine.dylib - 0x7fff8b248000 dyld Y Y /usr/lib/system/libquarantine.dylib at 0x7fff8b248000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libquarantine.dylib" at 0x7fff8b248000] 16 libremovefile.dylib - 0x7fff85ae3000 dyld Y Y /usr/lib/system/libremovefile.dylib at 0x7fff85ae3000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libremovefile.dylib" at 0x7fff85ae3000] 17 libsystem_blocks.dylib - 0x7fff84d68000 dyld Y Y /usr/lib/system/libsystem_blocks.dylib at 0x7fff84d68000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_blocks.dylib" at 0x7fff84d68000] 18 libsystem_c.dylib - 0x7fff84d6a000 dyld Y Y /usr/lib/system/libsystem_c.dylib at 0x7fff84d6a000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_c.dylib" at 0x7fff84d6a000] 19 libsystem_dnssd.dylib - 0x7fff8a2ac000 dyld Y Y /usr/lib/system/libsystem_dnssd.dylib at 0x7fff8a2ac000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_dnssd.dylib" at 0x7fff8a2ac000] 20 libsystem_info.dylib - 0x7fff8ae26000 dyld Y Y /usr/lib/system/libsystem_info.dylib at 0x7fff8ae26000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_info.dylib" at 0x7fff8ae26000] 21 libsystem_kernel.dylib - 0x7fff8a0f6000 dyld Y Y /usr/lib/system/libsystem_kernel.dylib at 0x7fff8a0f6000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_kernel.dylib" at 0x7fff8a0f6000] 22 libsystem_network.dylib - 0x7fff8c528000 dyld Y Y /usr/lib/system/libsystem_network.dylib at 0x7fff8c528000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_network.dylib" at 0x7fff8c528000] 23 libsystem_notify.dylib - 0x7fff90c0f000 dyld Y Y /usr/lib/system/libsystem_notify.dylib at 0x7fff90c0f000 (offset 0x49b8000) ---Type to continue, or q to quit--(objfile is) [memory object "/usr/lib/system/libsystem_notify.dylib" at 0x7fff90c0f000] 24 libsystem_sandbox.dylib - 0x7fff89740000 dyld Y Y /usr/lib/system/libsystem_sandbox.dylib at 0x7fff89740000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libsystem_sandbox.dylib" at 0x7fff89740000] 25 libunc.dylib - 0x7fff855f0000 dyld Y Y /usr/lib/system/libunc.dylib at 0x7fff855f0000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libunc.dylib" at 0x7fff855f0000] 26 libunwind.dylib - 0x7fff91489000 dyld Y Y /usr/lib/system/libunwind.dylib at 0x7fff91489000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libunwind.dylib" at 0x7fff91489000] 27 libxpc.dylib - 0x7fff849f2000 dyld Y Y /usr/lib/system/libxpc.dylib at 0x7fff849f2000 (offset 0x49b8000) (objfile is) [memory object "/usr/lib/system/libxpc.dylib" at 0x7fff849f2000]
52
Exercise A1 (LLDB) Goal: Learn how to list stack traces, disassemble functions, check their correctness, dump data, compare core dumps with diagnostic reports, get environment Patterns: Manual Dump, Stack Trace, Stack Trace Collection, Annotated Disassembly, Paratext, Not My Version, Environment Hint
1.
Load a core dump core.1394 and App1 executable:
$ lldb -c ~/Documents/AMCDA-Dumps/core.1394 -f ~/Documents/AMCDADumps/Apps/App1/Build/Products/Release/App1 error: core.1394 is a corrupt mach-o file: load command 46 LC_SEGMENT_64 has a fileoff + filesize (0x160b7000) that extends beyond the end of the file (0x160b5000), the segment will be truncated Core file '/Users/DumpAnalysis/Documents/AMCDA-Dumps/core.1394' (x86_64) was loaded. Process 0 stopped * thread #1: tid = 0x0000, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop thread #2: tid = 0x0001, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop thread #3: tid = 0x0002, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop thread #4: tid = 0x0003, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop
54
thread #5: tid = 0x0004, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop thread #6: tid = 0x0005, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop (lldb)
Note: We see LLDB listed 6 threads with their TIDs numbered from 0. Also we have code disassembly starting from the next instruction that was to be executed if dump wasn’t saved. The nice feature is annotated disassembly that shows symbolic names for jump and call destinations. 2.
List all threads:
(lldb) thread list Process 0 stopped * thread #1: tid = 0x0000, 0x00007fff8a10ce42 stop reason = signal SIGSTOP thread #2: tid = 0x0001, 0x00007fff8a10ce42 stop reason = signal SIGSTOP thread #3: tid = 0x0002, 0x00007fff8a10ce42 stop reason = signal SIGSTOP thread #4: tid = 0x0003, 0x00007fff8a10ce42 stop reason = signal SIGSTOP thread #5: tid = 0x0004, 0x00007fff8a10ce42 stop reason = signal SIGSTOP thread #6: tid = 0x0005, 0x00007fff8a10ce42 stop reason = signal SIGSTOP
libsystem_kernel.dylib`__semwait_signal + 10, libsystem_kernel.dylib`__semwait_signal + 10, libsystem_kernel.dylib`__semwait_signal + 10, libsystem_kernel.dylib`__semwait_signal + 10, libsystem_kernel.dylib`__semwait_signal + 10, libsystem_kernel.dylib`__semwait_signal + 10,
Note: Compared to GDB here threads are listed according to increasing thread number order. 3.
Get all thread stack traces:
(lldb) thread backtrace all * thread #1: tid = 0x0000, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390bcc3 App1`main + 195 frame #5: 0x000000010390ba14 App1`start + 52
55
thread #2: tid = 0x0001, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390ba32 App1`bar_one + 18 frame #5: 0x000000010390ba49 App1`foo_one + 9 frame #6: 0x000000010390ba61 App1`thread_one + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13 thread #3: tid = 0x0002, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390ba92 App1`bar_two + 18 frame #5: 0x000000010390baa9 App1`foo_two + 9 frame #6: 0x000000010390bac1 App1`thread_two + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13 thread #4: tid = 0x0003, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390baf2 App1`bar_three + 18 frame #5: 0x000000010390bb09 App1`foo_three + 9 frame #6: 0x000000010390bb21 App1`thread_three + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13 thread #5: tid = 0x0004, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390bb52 App1`bar_four + 18 frame #5: 0x000000010390bb69 App1`foo_four + 9 frame #6: 0x000000010390bb81 App1`thread_four + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13 thread #6: tid = 0x0005, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390bbb2 App1`bar_five + 18 frame #5: 0x000000010390bbc9 App1`foo_five + 9 frame #6: 0x000000010390bbe1 App1`thread_five + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13
56
4.
Switch to the thread #3 and get its stack trace:
(lldb) thread select 3 * thread #3: tid = 0x0002, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal + 10: -> 0x7fff8a10ce42: jae 0x7fff8a10ce49 ; __semwait_signal + 17 0x7fff8a10ce44: jmpq 0x7fff8a10dffc ; cerror 0x7fff8a10ce49: ret 0x7fff8a10ce4a: nop (lldb) bt * thread #3: tid = 0x0002, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390ba92 App1`bar_two + 18 frame #5: 0x000000010390baa9 App1`foo_two + 9 frame #6: 0x000000010390bac1 App1`thread_two + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13
Note: We can also list any thread stack trace without switching to it: (lldb) thread backtrace 4 thread #4: tid = 0x0003, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390baf2 App1`bar_three + 18 frame #5: 0x000000010390bb09 App1`foo_three + 9 frame #6: 0x000000010390bb21 App1`thread_three + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13
5.
Check that bar_two called sleep function:
(lldb) di -n bar_two App1`bar_two: 0x10390ba80: pushq 0x10390ba81: movq 0x10390ba84: subq 0x10390ba88: movl 0x10390ba8d: callq 0x10390ba92: movl 0x10390ba95: addq 0x10390ba99: popq 0x10390ba9a: ret 0x10390ba9b: nopl
%rbp %rsp, %rbp $16, %rsp $4294967295, %edi 0x10390bce0 %eax, -4(%rbp) $16, %rsp %rbp
; symbol stub for: sleep
(%rax,%rax)
57
(lldb) bt * thread #3: tid = 0x0002, 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10, stop reason = signal SIGSTOP frame #0: 0x00007fff8a10ce42 libsystem_kernel.dylib`__semwait_signal + 10 frame #1: 0x00007fff84d6edea libsystem_c.dylib`nanosleep + 164 frame #2: 0x00007fff84d6ec2c libsystem_c.dylib`sleep + 61 frame #3: 0x00007fff84d6ec08 libsystem_c.dylib`sleep + 25 frame #4: 0x000000010390ba92 App1`bar_two + 18 frame #5: 0x000000010390baa9 App1`foo_two + 9 frame #6: 0x000000010390bac1 App1`thread_two + 17 frame #7: 0x00007fff84db88bf libsystem_c.dylib`_pthread_start + 335 frame #8: 0x00007fff84dbbb75 libsystem_c.dylib`thread_start + 13
6.
Compare with Intel disassembly flavor:
(lldb) settings set target.x86-disassembly-flavor intel (lldb) di -n bar_two App1`bar_two: 0x10390ba80: push 0x10390ba81: mov 0x10390ba84: sub 0x10390ba88: mov 0x10390ba8d: call 0x10390ba92: mov 0x10390ba95: add 0x10390ba99: pop 0x10390ba9a: ret 0x10390ba9b: nop
RBP RBP, RSP RSP, 16 EDI, 4294967295 0x10390bce0 DWORD PTR [RBP - 4], EAX RSP, 16 RBP
; symbol stub for: sleep
DWORD PTR [RAX + RAX]
(lldb) set disassembly-flavor att (lldb)
7.
Follow bar_two function to sleep function code:
(lldb) di -n bar_two App1`bar_two: 0x10390ba80: pushq 0x10390ba81: movq 0x10390ba84: subq 0x10390ba88: movl 0x10390ba8d: callq 0x10390ba92: movl 0x10390ba95: addq 0x10390ba99: popq 0x10390ba9a: ret 0x10390ba9b: nopl
%rbp %rsp, %rbp $16, %rsp $4294967295, %edi 0x10390bce0 %eax, -4(%rbp) $16, %rsp %rbp
; symbol stub for: sleep
(%rax,%rax)
(lldb) di -a 0x10390bce0 App1`symbol stub for: sleep: 0x10390bce0: jmpq *866(%rip)
; (void *)0x00007fff84d6ebef: sleep
58
8.
Disassemble the annotated value:
(lldb) di -a 0x00007fff84d6ebef libsystem_c.dylib`sleep: 0x7fff84d6ebef: pushq %rbp 0x7fff84d6ebf0: movq %rsp, %rbp 0x7fff84d6ebf3: pushq %rbx 0x7fff84d6ebf4: subq $40, %rsp 0x7fff84d6ebf8: testl %edi, %edi 0x7fff84d6ebfa: movl %edi, %ebx 0x7fff84d6ebfc: jns 0x7fff84d6ec11 ; sleep + 34 0x7fff84d6ebfe: movl $2147483647, %edi 0x7fff84d6ec03: callq 0x7fff84d6ebef ; sleep 0x7fff84d6ec08: leal -2147483647(%rbx,%rax), %eax 0x7fff84d6ec0f: jmp 0x7fff84d6ec4f ; sleep + 96 0x7fff84d6ec11: movl %ebx, %eax 0x7fff84d6ec13: movq %rax, -24(%rbp) 0x7fff84d6ec17: movq $0, -16(%rbp) 0x7fff84d6ec1f: leaq -24(%rbp), %rdi 0x7fff84d6ec23: leaq -40(%rbp), %rsi 0x7fff84d6ec27: callq 0x7fff84d6ed46 ; nanosleep 0x7fff84d6ec2c: cmpl $-1, %eax 0x7fff84d6ec2f: je 0x7fff84d6ec37 ; sleep + 72 0x7fff84d6ec31: xorl %ebx, %ebx 0x7fff84d6ec33: movl %ebx, %eax 0x7fff84d6ec35: jmp 0x7fff84d6ec4f ; sleep + 96 0x7fff84d6ec37: callq 0x7fff84e0cc88 ; __error 0x7fff84d6ec3c: cmpl $4, (%rax) 0x7fff84d6ec3f: jne 0x7fff84d6ec33 ; sleep + 68 0x7fff84d6ec41: cmpq $0, -32(%rbp) 0x7fff84d6ec46: setne %al 0x7fff84d6ec49: movzbl %al, %eax 0x7fff84d6ec4c: addl -40(%rbp), %eax 0x7fff84d6ec4f: addq $40, %rsp 0x7fff84d6ec53: popq %rbx 0x7fff84d6ec54: popq %rbp
9. Compare stack trace for thread #3 (core thread 2) and its module info with the diagnostic report App1_1394.crash: Process: Path: Identifier: Version: Code Type: Parent Process:
App1 [1394] /Users/USER/Documents/*/App1 App1 ??? (???) X86-64 (Native) bash [661]
Date/Time: OS Version: Report Version:
2012-07-24 00:20:26.078 +0100 Mac OS X 10.7.4 (11E53) 9
Crashed Thread:
0
Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_kernel.dylib 0x00007fff8a10ce42 1 libsystem_c.dylib 0x00007fff84d6edea 2 libsystem_c.dylib 0x00007fff84d6ec2c 3 libsystem_c.dylib 0x00007fff84d6ec08 4 App1 0x000000010390bcc3 5 App1 0x000000010390ba14
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 main + 195 start + 52
59
Thread 1: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390ba32 0x000000010390ba49 0x000000010390ba61 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_one + 18 foo_one + 9 thread_one + 17 _pthread_start + 335 thread_start + 13
Thread 2: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390ba92 0x000000010390baa9 0x000000010390bac1 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_two + 18 foo_two + 9 thread_two + 17 _pthread_start + 335 thread_start + 13
Thread 3: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390baf2 0x000000010390bb09 0x000000010390bb21 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_three + 18 foo_three + 9 thread_three + 17 _pthread_start + 335 thread_start + 13
Thread 4: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390bb52 0x000000010390bb69 0x000000010390bb81 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_four + 18 foo_four + 9 thread_four + 17 _pthread_start + 335 thread_start + 13
Thread 5: 0 libsystem_kernel.dylib 1 libsystem_c.dylib 2 libsystem_c.dylib 3 libsystem_c.dylib 4 App1 5 App1 6 App1 7 libsystem_c.dylib 8 libsystem_c.dylib
0x00007fff8a10ce42 0x00007fff84d6edea 0x00007fff84d6ec2c 0x00007fff84d6ec08 0x000000010390bbb2 0x000000010390bbc9 0x000000010390bbe1 0x00007fff84db88bf 0x00007fff84dbbb75
__semwait_signal + 10 nanosleep + 164 sleep + 61 sleep + 25 bar_five + 18 foo_five + 9 thread_five + 17 _pthread_start + 335 thread_start + 13
Thread 0 crashed with X86 Thread State (64-bit): rax: 0x0000000000000004 rbx: 0x00007fff6350aa08 rdi: 0x0000000000000c03 rsi: 0x0000000000000000 r8: 0x000000007fffffff r9: 0x0000000000000000 r12: 0x0000000000000000 r13: 0x0000000000000000 rip: 0x00007fff8a10ce42 rfl: 0x0000000000000247 Logical CPU: 0
rcx: rbp: r10: r14: cr2:
0x00007fff6350a9c8 0x00007fff6350a9f0 0x0000000000000001 0x00007fff6350aa18 0x0000000103d0b880
60
rdx: rsp: r11: r15:
0x0000000000000001 0x00007fff6350a9c8 0xffffff80002da8d0 0x0000000000000000
Binary Images: 0x10390b000 0x10390bfff +App1 (??? - ???) /Users/USER/Documents/*/App1 0x7fff6350b000 0x7fff6353fbaf dyld (195.6 - ???) /usr/lib/dyld 0x7fff849f2000 0x7fff84a0ffff libxpc.dylib (77.19.0 - compatibility 1.0.0) /usr/lib/system/libxpc.dylib 0x7fff84d68000 0x7fff84d69ff7 libsystem_blocks.dylib (53.0.0 - compatibility 1.0.0) /usr/lib/system/libsystem_blocks.dylib 0x7fff84d6a000 0x7fff84e47fef libsystem_c.dylib (763.13.0 - compatibility 1.0.0) /usr/lib/system/libsystem_c.dylib 0x7fff85022000 0x7fff85030fff libdispatch.dylib (187.9.0 - compatibility 1.0.0) /usr/lib/system/libdispatch.dylib 0x7fff855f0000 0x7fff855f1fff libunc.dylib (24.0.0 - compatibility 1.0.0) /usr/lib/system/libunc.dylib 0x7fff85ae3000 0x7fff85ae4ff7 libremovefile.dylib (21.1.0 - compatibility 1.0.0) /usr/lib/system/libremovefile.dylib 0x7fff89114000 0x7fff89118fff libmathCommon.A.dylib (2026.0.0 - compatibility 1.0.0) /usr/lib/system/libmathCommon.A.dylib 0x7fff89119000 0x7fff8911dfff libdyld.dylib (195.5.0 - compatibility 1.0.0) /usr/lib/system/libdyld.dylib 0x7fff89740000 0x7fff89741ff7 libsystem_sandbox.dylib (??? - ???) /usr/lib/system/libsystem_sandbox.dylib 0x7fff8a0ef000 0x7fff8a0f5fff libmacho.dylib (800.0.0 - compatibility 1.0.0) /usr/lib/system/libmacho.dylib 0x7fff8a0f6000 0x7fff8a116fff libsystem_kernel.dylib (1699.26.8 - compatibility 1.0.0) /usr/lib/system/libsystem_kernel.dylib 0x7fff8a2ac000 0x7fff8a2b4fff libsystem_dnssd.dylib (??? - ???) /usr/lib/system/libsystem_dnssd.dylib 0x7fff8ae26000 0x7fff8ae61fff libsystem_info.dylib (??? - ???) /usr/lib/system/libsystem_info.dylib 0x7fff8b248000 0x7fff8b24afff libquarantine.dylib (36.6.0 - compatibility 1.0.0) /usr/lib/system/libquarantine.dylib 0x7fff8b3b4000 0x7fff8b3b4fff libkeymgr.dylib (23.0.0 - compatibility 1.0.0) /usr/lib/system/libkeymgr.dylib 0x7fff8b3dd000 0x7fff8b3e2fff libcompiler_rt.dylib (6.0.0 - compatibility 1.0.0) /usr/lib/system/libcompiler_rt.dylib 0x7fff8bd1a000 0x7fff8bd1bfff libdnsinfo.dylib (395.11.0 - compatibility 1.0.0) /usr/lib/system/libdnsinfo.dylib 0x7fff8c528000 0x7fff8c52dff7 libsystem_network.dylib (??? - ???) /usr/lib/system/libsystem_network.dylib 0x7fff8cfa3000 0x7fff8cfadff7 liblaunch.dylib (392.38.0 - compatibility 1.0.0) /usr/lib/system/liblaunch.dylib 0x7fff8fe02000 0x7fff8fe09fff libcopyfile.dylib (85.1.0 - compatibility 1.0.0) /usr/lib/system/libcopyfile.dylib 0x7fff8fe4b000 0x7fff8fe8dff7 libcommonCrypto.dylib (55010.0.0 - compatibility 1.0.0) /usr/lib/system/libcommonCrypto.dylib 0x7fff90c0f000 0x7fff90c18ff7 libsystem_notify.dylib (80.1.0 - compatibility 1.0.0) /usr/lib/system/libsystem_notify.dylib 0x7fff91376000 0x7fff913a3fe7 libSystem.B.dylib (159.1.0 - compatibility 1.0.0) /usr/lib/libSystem.B.dylib 0x7fff91489000 0x7fff9148fff7 libunwind.dylib (30.0.0 - compatibility 1.0.0) /usr/lib/system/libunwind.dylib 0x7fff91a22000 0x7fff91a27fff libcache.dylib (47.0.0 - compatibility 1.0.0) /usr/lib/system/libcache.dylib External Modification Summary: Calls made by other processes targeting this process: task_for_pid: 2 thread_create: 0 thread_set_state: 0 Calls made by this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by all processes on this machine: task_for_pid: 2696 thread_create: 0 thread_set_state: 0
61
VM Region Summary: ReadOnly portion of Libraries: Total=50.2M resident=50.2M(100%) swapped_out_or_unallocated=0K(0%) Writable regions: Total=38.9M written=10.8M(28%) resident=42.6M(110%) swapped_out=0K(0%) unallocated=16777216.0T(45221404475392%) REGION TYPE =========== MALLOC Stack __DATA __LINKEDIT __TEXT shared memory =========== TOTAL
10.
VIRTUAL ======= 1220K 66.6M 464K 47.7M 2484K 12K ======= 118.4M
Get App1 data section from the output of vmmap_1394.log:
Virtual Memory Map of process 1394 (App1) Output report format: 2.2 -- 64-bit process ==== Non-writable regions for process 1394 __TEXT 000000010390b000-000000010390c000 [ Dumps/Apps/App1/Build/Products/Release/App1
4K] r-x/rwx SM=COW
/Users/DumpAnalysis/Documents/AMCDA-
4K] rw-/rwx SM=PRV
/Users/DumpAnalysis/Documents/AMCDA-
[...] ==== Writable regions for process 1394 __DATA 000000010390c000-000000010390d000 [ Dumps/Apps/App1/Build/Products/Release/App1 [...]
11.
Compare with the section information in the core dump:
(lldb) image dump sections App1 Sections for SectID ---------0x00000100 0x00000200 0x00000001 0x00000002 0x00000003 0x00000004 0x00000005 0x00000300 0x00000006 0x00000007 0x00000008 0x00000009 0x00000400
'/Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1' (x86_64): Type Load Address File Off. File Size Flags Section Name ---------------- --------------------------------------- ---------- ---------- ---------- ---------------------------container [0x0000000000000000-0x0000000100000000)* 0x00000000 0x00000000 0x00000000 App1.__PAGEZERO container [0x000000010390b000-0x000000010390c000) 0x00000000 0x00001000 0x00000000 App1.__TEXT code [0x000000010390b9e0-0x000000010390bcd3) 0x000009e0 0x000002f3 0x80000400 App1.__TEXT.__text code [0x000000010390bcd4-0x000000010390bce6) 0x00000cd4 0x00000012 0x80000408 App1.__TEXT.__stubs code [0x000000010390bce8-0x000000010390bd16) 0x00000ce8 0x0000002e 0x80000400 App1.__TEXT.__stub_helper code [0x000000010390bd16-0x000000010390bd66) 0x00000d16 0x00000050 0x00000000 App1.__TEXT.__unwind_info eh-frame [0x000000010390bd68-0x000000010390c000) 0x00000d68 0x00000298 0x00000000 App1.__TEXT.__eh_frame container [0x000000010390c000-0x000000010390d000) 0x00001000 0x00001000 0x00000000 App1.__DATA data [0x000000010390c000-0x000000010390c028) 0x00001000 0x00000028 0x00000000 App1.__DATA.__program_vars data-ptrs [0x000000010390c028-0x000000010390c038) 0x00001028 0x00000010 0x00000006 App1.__DATA.__nl_symbol_ptr data-ptrs [0x000000010390c038-0x000000010390c050) 0x00001038 0x00000018 0x00000007 App1.__DATA.__la_symbol_ptr zero-fill [0x000000010390c050-0x000000010390c070) 0x00000000 0x00000000 0x00000001 App1.__DATA.__common container [0x000000010390d000-0x000000010390d3b0) 0x00002000 0x000003b0 0x00000000 App1.__LINKEDIT
12.
Dump data with possible symbolic information:
(lldb) error: error: error:
x/512a 0x000000010390c000 Normally, 'memory read' will not read over 1024 bytes of data. Please use --force to override this restriction just once. or set target.max-memory-read-size if you will often need a larger limit.
62
(lldb) x/512a 0x000000010390c000 --force 0x10390c000: 0x000000010390b000 0x10390c008: 0x000000010390c050 App1`NXArgc 0x10390c010: 0x000000010390c058 App1`NXArgv 0x10390c018: 0x000000010390c060 App1`environ 0x10390c020: 0x000000010390c068 0x10390c028: 0x00007fff8911a6a0 libdyld.dylib`dyld_stub_binder 0x10390c030: 0x00007fff63546d80 dyld`initialPoolContent + 2128 0x10390c038: 0x000000010390bcf8 0x10390c040: 0x00007fff84dbab01 libsystem_c.dylib`pthread_create 0x10390c048: 0x00007fff84d6ebef libsystem_c.dylib`sleep 0x10390c050: 0x0000000000000001 0x10390c058: 0x00007fff6350aaf0 0x10390c060: 0x00007fff6350ab00 0x10390c068: 0x00007fff6350ac73 0x10390c070: 0x0000000000000000 0x10390c078: 0x0000000000000000 0x10390c080: 0x0000000000000000 0x10390c088: 0x0000000000000000 0x10390c090: 0x0000000000000000 [...]
13.
Dump the contents of memory pointed to by environ variable in null-terminated string format:
(lldb) x/100s 0x00007fff6350ab00 [...] 0x7fff6350abd5: "" 0x7fff6350abd6: "" 0x7fff6350abd7: "" 0x7fff6350abd8: "/Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1" 0x7fff6350ac28: "/Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1" 0x7fff6350ac78: "TERM_PROGRAM=Apple_Terminal" 0x7fff6350ac94: "TERM=xterm-256color" 0x7fff6350aca8: "SHELL=/bin/bash" 0x7fff6350acb8: "TMPDIR=/var/folders/ww/rmtqfhl93yj4213dnl2rqy6w0000gn/T/" 0x7fff6350acf1: "Apple_PubSub_Socket_Render=/tmp/launch-mYEvtN/Render" 0x7fff6350ad26: "TERM_PROGRAM_VERSION=303.2" 0x7fff6350ad41: "TERM_SESSION_ID=2B039506-8384-4620-B354-120BE31AEA84" 0x7fff6350ad76: "USER=DumpAnalysis" 0x7fff6350ad88: "COMMAND_MODE=unix2003" 0x7fff6350ad9e: "SSH_AUTH_SOCK=/tmp/launch-9sm7dH/Listeners" 0x7fff6350adc9: "__CF_USER_TEXT_ENCODING=0x1F5:0:0" 0x7fff6350adeb: "Apple_Ubiquity_Message=/tmp/launch-tWsFs8/Apple_Ubiquity_Message" 0x7fff6350ae2c: "PATH=/Applications/Xcode.app/Contents/Developer/usr/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin" 0x7fff6350ae9f: "PWD=/Users/DumpAnalysis" 0x7fff6350aeb7: "LANG=en_IE.UTF-8" ---Type to continue, or q to quit--0x7fff6350aec8: "SHLVL=1" 0x7fff6350aed0: "HOME=/Users/DumpAnalysis" 0x7fff6350aee9: "LOGNAME=DumpAnalysis" 0x7fff6350aefe: "DISPLAY=/tmp/launch-M8cgb1/org.x:0" 0x7fff6350af21: "SECURITYSESSIONID=186af" 0x7fff6350af39: "_=/Users/DumpAnalysis/Documents/AMCDA-Dumps/Apps/App1/Build/Products/Release/App1" 0x7fff6350af8b: "OLDPWD=/usr/share/man/man1" 0x7fff6350afa6: "" 0x7fff6350afa7: "" 0x7fff6350afa8: "stack_guard=0x74843dc6068699c3" 0x7fff6350afc7: "malloc_entropy=0x7406669509034332,0x71e4e2253a6d22b0" 0x7fff6350affc: "" 0x7fff6350affd: ""
63
14.
Get the list of loaded modules:
(lldb) image list [ 0] 5BC0342F-7E97-3A7D-8EA6-75A0468021EA 0x000000010390b000 Dumps/Apps/App1/Build/Products/Release/App1 [ 1] 7BEBB139-50BB-3112-947A-F4AA168F991C 0x00007fff91376000 [ 2] 1571C3AB-BCB2-38CD-B3B2-C5FC3F927C6A 0x00007fff91a22000 [ 3] BB770C22-8C57-365A-8716-4A3C36AE7BFB 0x00007fff8fe4b000 [ 4] 98ECD5F6-E85C-32A5-98CD-8911230CB66A 0x00007fff8b3dd000 [ 5] 0AB51EE2-E914-358C-AC19-47BC024BDAE7 0x00007fff8fe02000 [ 6] 1D5BE322-A9B9-3BCE-8FAC-076FB07CF54A 0x00007fff85022000 [ 7] 853BAAA5-270F-3FDC-B025-D448DB72E1C3 0x00007fff8bd1a000 [ 8] 380C3F44-0CA7-3514-8080-46D1C9DF4FCD 0x00007fff89119000 [ 9] 61EFED6A-A407-301E-B454-CD18314F0075 0x00007fff8b3b4000 [ 10] 6ECB7F19-B384-32C1-8652-2463C1CF4815 0x00007fff8cfa3000 [ 11] 165514D7-1BFA-38EF-A151-676DCD21FB64 0x00007fff8a0ef000 [ 12] FF83AFF7-42B2-306E-90AF-D539C51A4542 0x00007fff89114000 [ 13] 0EBF714B-4B69-3E1F-9A7D-6BBC2AACB310 0x00007fff8b248000 [ 14] 739E6C83-AA52-3C6C-A680-B37FE2888A04 0x00007fff85ae3000 [ 15] 8BCA214A-8992-34B2-A8B9-B74DEACA1869 0x00007fff84d68000 [ 16] 41B43515-2806-3FBC-ACF1-A16F35B7E290 0x00007fff84d6a000 [ 17] D9BB1F87-A42B-3CBC-9DC2-FC07FCEF0016 0x00007fff8a2ac000 [ 18] 35F90252-2AE1-32C5-8D34-782C614D9639 0x00007fff8ae26000 [ 19] 1DDC0B0F-DB2A-34D6-895D-E5B2B5618946 0x00007fff8a0f6000 [ 20] 5DE7024E-1D2D-34A2-80F4-08326331A75B 0x00007fff8c528000 [ 21] A4D651E3-D1C6-3934-AD49-7A104FD14596 0x00007fff90c0f000 [ 22] 96D38E74-F18F-3CCB-A20B-E8E3ADC4E166 0x00007fff89740000 [ 23] 337960EE-0A85-3DD0-A760-7134CF4C0AFF 0x00007fff855f0000 [ 24] 1E9C6C8C-CBE8-3F4B-A5B5-E03E3AB53231 0x00007fff91489000 [ 25] 9F57891B-D7EF-3050-BEDD-21E7C6668248 0x00007fff849f2000 [ 26] 0CD1B35B-A28F-32DA-B72E-452EAD609613 0x00007fff6350b000 (lldb)
/Users/DumpAnalysis/Documents/AMCDA/usr/lib/libSystem.B.dylib (0x00007fff91376000) /usr/lib/system/libcache.dylib (0x00007fff91a22000) /usr/lib/system/libcommonCrypto.dylib (0x00007fff8fe4b000) /usr/lib/system/libcompiler_rt.dylib (0x00007fff8b3dd000) /usr/lib/system/libcopyfile.dylib (0x00007fff8fe02000) /usr/lib/system/libdispatch.dylib (0x00007fff85022000) /usr/lib/system/libdnsinfo.dylib (0x00007fff8bd1a000) /usr/lib/system/libdyld.dylib (0x00007fff89119000) /usr/lib/system/libkeymgr.dylib (0x00007fff8b3b4000) /usr/lib/system/liblaunch.dylib (0x00007fff8cfa3000) /usr/lib/system/libmacho.dylib (0x00007fff8a0ef000) /usr/lib/system/libmathCommon.A.dylib (0x00007fff89114000) /usr/lib/system/libquarantine.dylib (0x00007fff8b248000) /usr/lib/system/libremovefile.dylib (0x00007fff85ae3000) /usr/lib/system/libsystem_blocks.dylib (0x00007fff84d68000) /usr/lib/system/libsystem_c.dylib (0x00007fff84d6a000) /usr/lib/system/libsystem_dnssd.dylib (0x00007fff8a2ac000) /usr/lib/system/libsystem_info.dylib (0x00007fff8ae26000) /usr/lib/system/libsystem_kernel.dylib (0x00007fff8a0f6000) /usr/lib/system/libsystem_network.dylib (0x00007fff8c528000) /usr/lib/system/libsystem_notify.dylib (0x00007fff90c0f000) /usr/lib/system/libsystem_sandbox.dylib (0x00007fff89740000) /usr/lib/system/libunc.dylib (0x00007fff855f0000) /usr/lib/system/libunwind.dylib (0x00007fff91489000) /usr/lib/system/libxpc.dylib (0x00007fff849f2000) /usr/lib/dyld (0x00007fff6350b000)
64
Published by OpenTask, Republic of Ireland Copyright © 2015 by OpenTask Copyright © 2015 by Software Diagnostics Services Copyright © 2015 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover, and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalog record for this book is available from the British Library. ISBN-l3: 978-1-908043-97-9 (Paperback) 1st printing, 2015
2
Contents Presentation Slides and Transcript .................................................................................................................................5 Core Dump Collection...................................................................................................................................................25 Practice Exercises .........................................................................................................................................................31 Exercise 0..................................................................................................................................................................36 Exercise A1 ...............................................................................................................................................................40 Exercise A2D .............................................................................................................................................................53 Exercise A2C .............................................................................................................................................................58 Exercise A3 ...............................................................................................................................................................62 Exercise A4 ...............................................................................................................................................................66 Exercise A5 ...............................................................................................................................................................72 Exercise A6 ...............................................................................................................................................................76 Exercise A7 ...............................................................................................................................................................93 Exercise A8 .............................................................................................................................................................102 Exercise A9 .............................................................................................................................................................117 Exercise A10 ...........................................................................................................................................................132 Exercise A11 ...........................................................................................................................................................149 Exercise A12 ...........................................................................................................................................................157 App Source Code ........................................................................................................................................................171 App0 .......................................................................................................................................................................173 App1 .......................................................................................................................................................................174 App2D .....................................................................................................................................................................175 App2C .....................................................................................................................................................................177 App3 .......................................................................................................................................................................179 App4 .......................................................................................................................................................................181 App5 .......................................................................................................................................................................183 App6 .......................................................................................................................................................................185 App7 .......................................................................................................................................................................187 App8 .......................................................................................................................................................................189 App9 .......................................................................................................................................................................191 App10 .....................................................................................................................................................................193 App11 / App12 .......................................................................................................................................................195 Selected Patterns .......................................................................................................................................................197 NULL Pointer (data) ................................................................................................................................................199 3
Incomplete Stack Trace ..........................................................................................................................................200 Stack Trace .............................................................................................................................................................201 NULL Pointer (code)................................................................................................................................................202 Spiking Thread ........................................................................................................................................................203 Dynamic Memory Corruption (process heap) .........................................................................................................204 Execution Residue ..................................................................................................................................................205 Coincidental Symbolic Information.........................................................................................................................207 Stack Overflow (user mode) ...................................................................................................................................208 Divide by Zero (user mode) ....................................................................................................................................209 Local Buffer Overflow .............................................................................................................................................210 C++ Exception .........................................................................................................................................................211 Paratext ..................................................................................................................................................................212 Active Thread .........................................................................................................................................................213 Lateral Damage.......................................................................................................................................................214 Critical Region .........................................................................................................................................................215
4
Exercise A1 Goal: Learn how to list stack traces, disassemble functions, check their correctness, dump data, get environment. Patterns: Manual Dump, Stack Trace, Stack Trace Collection, Annotated Disassembly, Paratext, Not My Version, Environment Hint.
1.
Load a core dump core.3308 and App1 executable:
training@debian64:~/ALCDA$ gdb -c ./App1/core.3308 -se ./App1/App1 GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /home/training/ALCDA/App1/App1...done. [New LWP 3309] [New LWP 3310] [New LWP 3311] [New LWP 3312] [New LWP 3313] [New LWP 3308] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/home/training/ALCDA/App1/App1'. #0 0x000000000042fdf1 in nanosleep ()
2.
List all threads:
(gdb) info threads Id Target Id 6 LWP 3308 5 LWP 3313 4 LWP 3312 3 LWP 3311 2 LWP 3310 * 1 LWP 3309
3.
Frame 0x000000000042fdf1 0x000000000042fdf1 0x000000000042fdf1 0x000000000042fdf1 0x000000000042fdf1 0x000000000042fdf1
in in in in in in
nanosleep nanosleep nanosleep nanosleep nanosleep nanosleep
Get all thread stack traces:
(gdb) thread apply all bt Thread 6 (LWP 3308): #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x00000000004006c1 in main ()
40
() () () () () ()
Thread 5 (LWP 3313): #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x00000000004005f2 in bar_five () #3 0x0000000000400602 in foo_five () #4 0x000000000040061a in thread_five () #5 0x00000000004015f0 in start_thread (arg=) at pthread_create.c:304 #6 0x00000000004324a9 in clone () #7 0x0000000000000000 in ?? () Thread 4 (LWP 3312): #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x00000000004005b5 in bar_four () #3 0x00000000004005c5 in foo_four () #4 0x00000000004005dd in thread_four () #5 0x00000000004015f0 in start_thread (arg=) ---Type to continue, or q to quit--at pthread_create.c:304 #6 0x00000000004324a9 in clone () #7 0x0000000000000000 in ?? () Thread 3 (LWP 3311): #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x0000000000400578 in bar_three () #3 0x0000000000400588 in foo_three () #4 0x00000000004005a0 in thread_three () #5 0x00000000004015f0 in start_thread (arg=) at pthread_create.c:304 #6 0x00000000004324a9 in clone () #7 0x0000000000000000 in ?? () Thread 2 (LWP 3310): #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x000000000040053b in bar_two () #3 0x000000000040054b in foo_two () #4 0x0000000000400563 in thread_two () #5 0x00000000004015f0 in start_thread (arg=) at pthread_create.c:304 #6 0x00000000004324a9 in clone () ---Type to continue, or q to quit--#7 0x0000000000000000 in ?? () Thread 1 (LWP 3309): #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x00000000004004fe in bar_one () #3 0x000000000040050e in foo_one () #4 0x0000000000400526 in thread_one () #5 0x00000000004015f0 in start_thread (arg=) at pthread_create.c:304 #6 0x00000000004324a9 in clone () #7 0x0000000000000000 in ?? ()
41
4.
Switch to the thread #2 and get its stack trace:
(gdb) thread 2 [Switching to thread 2 (LWP 3310)] #0 0x000000000042fdf1 in nanosleep () (gdb) bt #0 0x000000000042fdf1 in nanosleep () #1 0x000000000042fcc0 in sleep () #2 0x000000000040053b in bar_two () #3 0x000000000040054b in foo_two () #4 0x0000000000400563 in thread_two () #5 0x00000000004015f0 in start_thread (arg=) at pthread_create.c:304 #6 0x00000000004324a9 in clone () #7 0x0000000000000000 in ?? ()
5.
Check that bar_two called sleep function:
(gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x000000000040052d : push %rbp 0x000000000040052e : mov %rsp,%rbp 0x0000000000400531 : mov $0xffffffff,%edi 0x0000000000400536 : callq 0x42fbe0 0x000000000040053b : pop %rbp 0x000000000040053c : retq End of assembler dump.
We see that the address in the stack trace for bar_two function is the address to return to after calling sleep function. 6.
Compare with Intel disassembly flavor:
(gdb) set disassembly-flavor intel (gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x000000000040052d : push rbp 0x000000000040052e : mov rbp,rsp 0x0000000000400531 : mov edi,0xffffffff 0x0000000000400536 : call 0x42fbe0 0x000000000040053b : pop rbp 0x000000000040053c : ret End of assembler dump. (gdb) set disassembly-flavor att
42
7.
Get App1 data section from the output of pmap (pmap.3308):
3308: ./App1 0000000000400000 00000000006b6000 00000000006b8000 000000000227c000 00007f2257e66000 00007f2257e67000 00007f2258667000 00007f2258668000 00007f2258e68000 00007f2258e69000 00007f2259669000 00007f225966a000 00007f2259e6a000 00007f2259e6b000 00007ffc7d24d000 00007ffc7d299000 ffffffffff600000 total
8.
732K 8K 28K 140K 4K 8192K 4K 8192K 4K 8192K 4K 8192K 4K 8192K 132K 4K 4K 42028K
r-x-rw--rw--rw------rw------rw------rw------rw------rw--rw--r-x-r-x--
/home/training/ALCDA/App1/App1 /home/training/ALCDA/App1/App1 [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ anon ] [ stack ] [ anon ] [ anon ]
Compare with the section information in the core dump:
(gdb) maintenance info sections Exec file: `/home/training/ALCDA/App1/App1', file type elf64-x86-64. 0x00400158->0x00400178 at 0x00000158: .note.ABI-tag ALLOC LOAD READONLY DATA HAS_CONTENTS 0x00400178->0x0040019c at 0x00000178: .note.gnu.build-id ALLOC LOAD READONLY DATA HAS_CONTENTS 0x004001a0->0x004002d8 at 0x000001a0: .rela.plt ALLOC LOAD READONLY DATA HAS_CONTENTS 0x004002d8->0x004002e6 at 0x000002d8: .init ALLOC LOAD READONLY CODE HAS_CONTENTS 0x004002f0->0x004003c0 at 0x000002f0: .plt ALLOC LOAD READONLY CODE HAS_CONTENTS 0x004003c0->0x0048b1b8 at 0x000003c0: .text ALLOC LOAD READONLY CODE HAS_CONTENTS 0x0048b1c0->0x0048bd3e at 0x0008b1c0: __libc_freeres_fn ALLOC LOAD READONLY CODE HAS_CONTENTS 0x0048bd40->0x0048bda1 at 0x0008bd40: __libc_thread_freeres_fn ALLOC LOAD READONLY CODE HAS_CONTENTS 0x0048bda4->0x0048bdad at 0x0008bda4: .fini ALLOC LOAD READONLY CODE HAS_CONTENTS 0x0048bdc0->0x004a9d24 at 0x0008bdc0: .rodata ALLOC LOAD READONLY DATA HAS_CONTENTS 0x004a9d28->0x004a9d88 at 0x000a9d28: __libc_subfreeres ALLOC LOAD READONLY DATA HAS_CONTENTS ---Type to continue, or q to quit--0x004a9d88->0x004a9d90 at 0x000a9d88: __libc_atexit ALLOC LOAD READONLY DATA HAS_CONTENTS 0x004a9d90->0x004a9d98 at 0x000a9d90: __libc_thread_subfreeres ALLOC LOAD READONLY DATA HAS_CONTENTS 0x004a9d98->0x004b686c at 0x000a9d98: .eh_frame ALLOC LOAD READONLY DATA HAS_CONTENTS 0x004b686c->0x004b6986 at 0x000b686c: .gcc_except_table ALLOC LOAD READONLY DATA HAS_CONTENTS 0x006b6988->0x006b69b0 at 0x000b6988: .tdata ALLOC LOAD DATA HAS_CONTENTS 0x006b69b0->0x006b69e0 at 0x000b69b0: .tbss ALLOC 0x006b69b0->0x006b69c0 at 0x000b69b0: .init_array ALLOC LOAD DATA HAS_CONTENTS 0x006b69c0->0x006b69d0 at 0x000b69c0: .fini_array ALLOC LOAD DATA HAS_CONTENTS 0x006b69d0->0x006b69d8 at 0x000b69d0: .jcr ALLOC LOAD DATA HAS_CONTENTS 0x006b69e0->0x006b6a50 at 0x000b69e0: .data.rel.ro ALLOC LOAD DATA HAS_CONTENTS 0x006b6a50->0x006b6a60 at 0x000b6a50: .got ALLOC LOAD DATA HAS_CONTENTS 0x006b6a60->0x006b6ae0 at 0x000b6a60: .got.plt ALLOC LOAD DATA HAS_CONTENTS 0x006b6ae0->0x006b77f0 at 0x000b6ae0: .data ALLOC LOAD DATA HAS_CONTENTS 0x006b7800->0x006beb68 at 0x000b77f0: .bss ALLOC 0x006beb68->0x006beb98 at 0x000b77f0: __libc_freeres_ptrs ALLOC 0x00000000->0x00000038 at 0x000b77f0: .comment READONLY HAS_CONTENTS 0x00000000->0x00000390 at 0x000b7830: .debug_aranges READONLY HAS_CONTENTS ---Type to continue, or q to quit--0x00000000->0x00000ac3 at 0x000b7bc0: .debug_pubnames READONLY HAS_CONTENTS 0x00000000->0x00011440 at 0x000b8683: .debug_info READONLY HAS_CONTENTS 0x00000000->0x000021b1 at 0x000c9ac3: .debug_abbrev READONLY HAS_CONTENTS 0x00000000->0x00002ebc at 0x000cbc74: .debug_line READONLY HAS_CONTENTS 0x00000000->0x000038da at 0x000ceb30: .debug_str READONLY HAS_CONTENTS 0x00000000->0x0000878e at 0x000d240a: .debug_loc READONLY HAS_CONTENTS 0x00000000->0x00001280 at 0x000dab98: .debug_ranges READONLY HAS_CONTENTS
43
Core file: `/home/training/ALCDA/./App1/core.3308', file type elf64-x86-64. 0x00000000->0x00002aa8 at 0x00000318: note0 READONLY HAS_CONTENTS 0x00000000->0x000000d8 at 0x00000438: .reg/3309 HAS_CONTENTS 0x00000000->0x000000d8 at 0x00000438: .reg HAS_CONTENTS 0x00000000->0x00000200 at 0x0000052c: .reg2/3309 HAS_CONTENTS 0x00000000->0x00000200 at 0x0000052c: .reg2 HAS_CONTENTS 0x00000000->0x00000340 at 0x00000740: .reg-xstate/3309 HAS_CONTENTS 0x00000000->0x00000340 at 0x00000740: .reg-xstate HAS_CONTENTS 0x00000000->0x000000d8 at 0x00000b04: .reg/3310 HAS_CONTENTS 0x00000000->0x00000200 at 0x00000bf8: .reg2/3310 HAS_CONTENTS 0x00000000->0x00000340 at 0x00000e0c: .reg-xstate/3310 HAS_CONTENTS 0x00000000->0x000000d8 at 0x000011d0: .reg/3311 HAS_CONTENTS 0x00000000->0x00000200 at 0x000012c4: .reg2/3311 HAS_CONTENTS 0x00000000->0x00000340 at 0x000014d8: .reg-xstate/3311 HAS_CONTENTS 0x00000000->0x000000d8 at 0x0000189c: .reg/3312 HAS_CONTENTS 0x00000000->0x00000200 at 0x00001990: .reg2/3312 HAS_CONTENTS ---Type to continue, or q to quit--0x00000000->0x00000340 at 0x00001ba4: .reg-xstate/3312 HAS_CONTENTS 0x00000000->0x000000d8 at 0x00001f68: .reg/3313 HAS_CONTENTS 0x00000000->0x00000200 at 0x0000205c: .reg2/3313 HAS_CONTENTS 0x00000000->0x00000340 at 0x00002270: .reg-xstate/3313 HAS_CONTENTS 0x00000000->0x000000d8 at 0x00002634: .reg/3308 HAS_CONTENTS 0x00000000->0x00000200 at 0x00002728: .reg2/3308 HAS_CONTENTS 0x00000000->0x00000340 at 0x0000293c: .reg-xstate/3308 HAS_CONTENTS 0x00000000->0x00000130 at 0x00002c90: .auxv HAS_CONTENTS 0x00400000->0x00400000 at 0x00002dc0: load1 ALLOC READONLY CODE 0x006b6000->0x006b8000 at 0x00002dc0: load2 ALLOC LOAD HAS_CONTENTS 0x006b8000->0x006bf000 at 0x00004dc0: load3 ALLOC LOAD HAS_CONTENTS 0x0227c000->0x0229f000 at 0x0000bdc0: load4 ALLOC LOAD HAS_CONTENTS 0x7f2257e67000->0x7f2258667000 at 0x0002edc0: load5 ALLOC LOAD HAS_CONTENTS 0x7f2258668000->0x7f2258e68000 at 0x0082edc0: load6 ALLOC LOAD HAS_CONTENTS 0x7f2258e69000->0x7f2259669000 at 0x0102edc0: load7 ALLOC LOAD HAS_CONTENTS 0x7f225966a000->0x7f2259e6a000 at 0x0182edc0: load8 ALLOC LOAD HAS_CONTENTS 0x7f2259e6b000->0x7f225a66b000 at 0x0202edc0: load9 ALLOC LOAD HAS_CONTENTS 0x7ffc7d24d000->0x7ffc7d26e000 at 0x0282edc0: load10 ALLOC LOAD HAS_CONTENTS 0x7ffc7d299000->0x7ffc7d29a000 at 0x0284fdc0: load11 ALLOC LOAD READONLY CODE HAS_CONTENTS 0xffffffffff600000->0xffffffffff601000 at 0x02850dc0: load12 ALLOC LOAD READONLY CODE HAS_CONTENTS
9.
Dump data with possible symbolic information:
(gdb) x/512a 0x6b6000: 0x6b6010: 0x6b6020: 0x6b6030: 0x6b6040: 0x6b6050: 0x6b6060: 0x6b6070: 0x6b6080: 0x6b6090: 0x6b60a0: 0x6b60b0: 0x6b60c0: 0x6b60d0: 0x6b60e0: 0x6b60f0: 0x6b6100: 0x6b6110: 0x6b6120: 0x6b6130: 0x6b6140: 0x6b6150:
0x006b6000 0x0 0xc2740000001c 0x50fffd2880 0x80e0a7e100e4400 0x80e470b46 0xc29400000014 0x8fffd28b0 0x0 0xc2ac00000014 0x15fffd28a8 0x0 0xc2c400000014 0x8fffd28b0 0x0 0xc2dc00000014 0x8fffd28a8 0x0 0xc2f400000014 0x8fffd28a0 0x0 0xc30c0000001c 0x24fffd2898 0x80e0a5a300e4400 0xb42 0xc32c00000014 0x8fffd28a8 0x0 0xc34400000014 0x8fffd28a0 0x0 0xc35c0000002c 0x110fffd2898 0xe580283100e4100 0x44100e0ae4020580 0x44100e490b41080e 0x80e 0xc38c00000014 0x1fffd2978 0x0 0xc3a40000003c 0x166fffd2970 0xd430286100e4100 0x58d048e038f4a06 0x8150078347068c49
44
0x6b6160: 0x70c0a8702098008 0x20cc6a2020b4b08 0x6b6170: 0x8 0xc3e400000034 ---Type to continue, or q to quit--0x6b6180: 0xe6fffd2aa0 0xd430286100e4100 0x6b6190: 0x783088109805006 0x4e048e058d4f068c 0x6b61a0: 0x8070c0a5b02038f 0x8020cc655020b41 0x6b61b0: 0xc41c00000034 0xc1fffd2b58 0x6b61c0: 0xd430286100e4100 0x58d048e038f4a06 0x6b61d0: 0x8153078348068c45 0x20cc68f02098008 0x6b61e0: 0x8 0xc45400000034 0x6b61f0: 0xf1fffd2bf0 0xd430286100e4100 0x6b6200: 0x815e098007834806 0x8f048e058d068c08 0x6b6210: 0xb4508070c0a6103 0x8020cc69d02 0x6b6220: 0xc48c00000014 0x1afffd2cb8 0x6b6230: 0x0 0xc4a40000002c 0x6b6240: 0x99fffd2cc0 0xd430286100e4100 0x6b6250: 0x58d048e038f4606 0x730207834f068c4c 0x6b6260: 0x8070c 0xc4d400000014 0x6b6270: 0x46fffd2d30 0x0 0x6b6280: 0xc4ec00000014 0x1bfffd2d68 0x6b6290: 0x0 0xc5040000004c 0x6b62a0: 0xa3fffd2d70 0xe42028f100e4200 0x6b62b0: 0x48d200e45038e18 0x300e44058c280e45 0x6b62c0: 0x480783380e410686 0x41380e0a5202500e 0x6b62d0: 0x200e42280e41300e 0xe42100e42180e42 0x6b62e0: 0xb4908 0xc55400000044 0x6b62f0: 0xc8fffd2dd0 0xe46028f100e4200 ---Type to continue, or q to quit--0x6b6300: 0x48d200e42038e18 0x300e44058c280e45 0x6b6310: 0x470783380e410686 0xe41380ea202500e 0x6b6320: 0x42200e42280e4130 0x80e42100e42180e 0x6b6330: 0xc59c0000002c 0x67fffd2e58 0x6b6340: 0x80e0a7a100e4400 0xb47080e0a490b42 0x6b6350: 0xe460b47080e0a49 0x8 0x6b6360: 0xc5cc00000024 0x13cfffd2e98 0x6b6370: 0xe4b028c04834a00 0x80e0a7a02038640 0x6b6380: 0xb41 0xc5f400000034 0x6b6390: 0x109fffd2fb0 0xe480286100e4100 0x6b63a0: 0xa68300e44038318 0x80e41100e41180e 0x6b63b0: 0x41180e0a97020b49 0xb47080e41100e 0x6b63c0: 0xc62c00000024 0x6bfffd3088 0x6b63d0: 0x80e0a77100e4400 0xb49080e0a470b45 0x6b63e0: 0xb49080e0a47 0xc6540000004c 0x6b63f0: 0x178fffd30d0 0xe45028f100e4200 0x6b6400: 0x48d200e42038e18 0x300e41058c280e42 0x6b6410: 0x440783380e410686 0x380e0a015103700e 0x6b6420: 0xe42280e41300e41 0x42100e42180e4220 0x6b6430: 0xb41080e 0xc6a40000004c 0x6b6440: 0x157fffd3200 0xe49028f100e4200 0x6b6450: 0x48d200e42038e18 0x300e45058c280e48 0x6b6460: 0x4a0783380e410686 0x41380e012703700e 0x6b6470: 0x200e42280e41300e 0xe42100e42180e42 ---Type to continue, or q to quit--0x6b6480: 0x8 0xc6f400000024 0x6b6490: 0xb0fffd3310 0x8d4d058606834a00 0x6b64a0: 0x48c400e4c028e03 0x80e8c02 0x6b64b0: 0xc71c0000004c 0x194fffd3398 0x6b64c0: 0xe4a028f100e4200 0x48d200e45038e18 0x6b64d0: 0x300e41058c280e45 0x4a0783380e470686 0x6b64e0: 0x380e0a015403700e 0xe42280e41300e44
45
0x6b64f0: 0x42100e42180e4220 0xb47080e 0x6b6500: 0xc76c00000024 0x6bfffd34e8 0x6b6510: 0x80e0a77100e4400 0xb49080e0a470b45 0x6b6520: 0xb49080e0a47 0xc7940000004c 0x6b6530: 0x673fffd3530 0xe42028f100e4200 0x6b6540: 0x48d200e42038e18 0x300e41058c280e42 0x6b6550: 0x470783380e410686 0x380e0a7d0201900e 0x6b6560: 0xe42280e41300e44 0x42100e42180e4220 0x6b6570: 0xb45080e 0xc7e400000024 0x6b6580: 0xcffffd3b60 0x8c4d058606834a00 0x6b6590: 0x28e400e4c038d04 0x80eab02 0x6b65a0: 0xc80c0000004c 0x4b3fffd3c08 0x6b65b0: 0xe42028f100e4200 0x48d200e42038e18 0x6b65c0: 0x300e41058c280e42 0x470783380e410686 0x6b65d0: 0x380e0af20201a00e 0xe42280e41300e43 0x6b65e0: 0x42100e42180e4220 0xb41080e 0x6b65f0: 0xc85c00000014 0x8afffd4078 ---Type to continue, or q to quit--0x6b6600: 0x80e6c200e460200 0xc87400000014 0x6b6610: 0x9fffd40f0 0x0 0x6b6620: 0xc88c0000001c 0x26fffd40e8 0x6b6630: 0xa4a0283100e4100 0x80e510b45080e 0x6b6640: 0xc8ac0000001c 0x72fffd40f8 0x6b6650: 0xa7e0283100e5b00 0x80e4f0b45080e 0x6b6660: 0xc8cc00000014 0x9fffd4158 0x6b6670: 0x0 0xc8e40000001c 0x6b6680: 0x1afffd4150 0xe540283100e4100 0x6b6690: 0x8 0xc9040000003c 0x6b66a0: 0x113fffd4150 0xe44028c100e4200 0x6b66b0: 0x483200e44038618 0x100e41180e0ab902 0x6b66c0: 0xe0a560b4a080e42 0x47080e42100e4118 0x6b66d0: 0xb 0xc94400000014 0x6b66e0: 0x5fffd4230 0x0 0x6b66f0: 0xc95c00000014 0x25fffd4228 0x6b6700: 0x80e49100e5400 0xc97400000044 0x6b6710: 0x1f8fffd4240 0xe42028e100e4200 0x6b6720: 0x48c200e45038d18 0x300e440586280e41 0x6b6730: 0xacb02700e440683 0x200e41280e44300e 0x6b6740: 0xe42100e42180e42 0xb4108 0x6b6750: 0xc9bc0000002c 0x7cfffd43f8 0x6b6760: 0x80e0a76100e4400 0xb49080e0a570b46 0x6b6770: 0xe470b49080e0a47 0x8 ---Type to continue, or q to quit--0x6b6780: 0xc9ec00000024 0x13cfffd4448 0x6b6790: 0x5a020283100e4500 0xedb020b41080e0a 0x6b67a0: 0x8 0xca140000004c 0x6b67b0: 0x242fffd4560 0xe45028e100e6200 0x6b67c0: 0x48c200e45038d18 0x300e410586280e44 0x6b67d0: 0x7e0301800e440683 0x280ec341300e0a01 0x6b67e0: 0x180ecc42200ec641 0x80ece42100ecd42 0x6b67f0: 0xb45 0xca6400000034 0x6b6800: 0x1aafffd4760 0x43180e47100e4200 0x6b6810: 0x43200e42028f038e 0x300e41280e42048d 0x6b6820: 0x4501900e44380e41 0x58c06860783 0x6b6830: 0xca9c0000001c 0x87fffd48d8 0x6b6840: 0x8302864a600e4e00 0x3 0x6b6850: 0xcabc00000014 0x15fffd4948 0x6b6860: 0x0 0x901ffff00000000 0x6b6870: 0x601910070044c 0x5c01a41001ffff00 0x6b6880: 0x3c10502f30000 0x1ffff0000050481
46
0x6b6890: 0x1b10001b603670a 0x961201ffff000046 0x6b68a0: 0x309b6000004eb02 0x1b60a96000b82 0x6b68b0: 0x301b90c01ffff00 0x2ac02830003e5 0x6b68c0: 0x501c61101ffff00 0x8ae068b01fd0000 0x6b68d0: 0xffff00000508b400 0x9500018105660a01 0x6b68e0: 0x801ffff00000501 0x561004d053d 0x6b68f0: 0x1d301c11e01ffff 0xba20503f90000 ---Type to continue, or q to quit--0x6b6900: 0xa406cb0000050684 0x2a50990000b8a02 0x6b6910: 0x5720a01ffff0000 0x502950001d5 0x6b6920: 0x920301990b01ffff 0xff00000502ce0002 0x6b6930: 0x1f705600a01ff 0x1ffff00000502b3 0x6b6940: 0x850002c903028a0b 0xc01ffff00000503 0x6b6950: 0x970004db029601eb 0xa01ffff00000505 0x6b6960: 0x501ef0001b3056b 0x5650a01ffff0000 0x6b6970: 0x501e90001ad 0x1f705600a01ffff 0x6b6980: 0x502b300 0x6bdec0 0x6b6990: 0x6b7640 0x6b7640 0x6b69a0: 0x6b7660 0x6b7648 0x6b69b0 : 0x4004b0 0x42f4c0 0x6b69c0 : 0x400480 0x46fcc0 0x6b69d0 : 0x0 0x0 0x6b69e0 : 0x6b72c0 0x7ffc7d26c7e8 0x6b69f0 : 0x7ffc7d26c9b9 0x0 0x6b6a00 : 0x1000000 0x0 0x6b6a10 : 0xffffffffffffffff 0x0 0x6b6a20 : 0x6be130 0x1 0x6b6a30 : 0x0 0x0 0x6b6a40 : 0x227d190 0x0 ---Type to continue, or q to quit--0x6b6a50: 0x403c00 0x0 0x6b6a60 : 0x0 0x0 0x6b6a70 : 0x0 0x41ea40 0x6b6a80 : 0x41b040 0x426950 0x6b6a90 : 0x423f00 0x453760
0x6b6aa0 : 0x470340 0x425300 0x6b6ab0 : 0x421820 0x41da30
0x6b6ac0 : 0x41a080 0x47f710
0x6b6ad0 : 0x421810 0x418b50
0x6b6ae0 : 0x0 0x0 0x6b6af0 : 0x6 0x0 0x6b6b00 : 0x7f22586669c0 0x7f225a66a9c0 0x6b6b10 : 0x6b6b10 0x6b6b10 0x6b6b20 : 0xffffffffffffffff 0x800000 0x6b6b30 : 0x1160 0x48c997 0x6b6b40 : 0x48c9c9 0x6bc6e0 0x6b6b50: 0x0 0x0 0x6b6b60 : 0xfbad2088 0x0 ---Type to continue, or q to quit--0x6b6b70 : 0x0 0x0 0x6b6b80 : 0x0 0x0 0x6b6b90 : 0x0 0x0 0x6b6ba0 : 0x0 0x0 0x6b6bb0 : 0x0 0x0 0x6b6bc0 : 0x0 0x0 0x6b6bd0 : 0x0 0xffffffffffffffff
47
0x6b6be0 : 0x0 0x6bcb20 0x6b6bf0 : 0xffffffffffffffff 0x0 0x6b6c00 : 0x6b6e20 0x0 0x6b6c10 : 0x0 0x0 0x6b6c20 : 0x0 0x0 0x6b6c30 : 0x0 0x48d440 0x6b6c40 : 0xfbad2084 0x0 0x6b6c50 : 0x0 0x0 0x6b6c60 : 0x0 0x0 0x6b6c70 : 0x0 0x0 0x6b6c80 : 0x0 0x0 0x6b6c90 : 0x0 0x0 0x6b6ca0 : 0x0 0x6b6b60 0x6b6cb0 : 0x1 0xffffffffffffffff 0x6b6cc0 : 0x0 0x6bcb30 0x6b6cd0 : 0xffffffffffffffff 0x0 0x6b6ce0 : 0x6b6f80 0x0 ---Type to continue, or q to quit--0x6b6cf0 : 0x0 0x0 0x6b6d00 : 0x0 0x0 0x6b6d10 : 0x0 0x48d440 0x6b6d20 : 0xfbad2086 0x0 0x6b6d30 : 0x0 0x0 0x6b6d40 : 0x0 0x0 0x6b6d50 : 0x0 0x0 0x6b6d60 : 0x0 0x0 0x6b6d70 : 0x0 0x0 0x6b6d80 : 0x0 0x6b6c40 0x6b6d90 : 0x2 0xffffffffffffffff 0x6b6da0 : 0x0 0x6bcb40 0x6b6db0 : 0xffffffffffffffff 0x0 0x6b6dc0 : 0x6b70e0 0x0 0x6b6dd0 : 0x0 0x0 0x6b6de0 : 0x0 0x0 0x6b6df0 : 0x0 0x48d440 0x6b6e00 : 0x6b6d20 0x0 0x6b6e10: 0x0 0x0 0x6b6e20 : 0x0 0x0 0x6b6e30 : 0x0 0x0 0x6b6e40 : 0x0 0x0 0x6b6e50 : 0x0 0x0 0x6b6e60 : 0x0 0x0 ---Type to continue, or q to quit--0x6b6e70 : 0x0 0x0 0x6b6e80 : 0x0 0x0 0x6b6e90 : 0x0 0x0 0x6b6ea0 : 0x0 0x0 0x6b6eb0 : 0x0 0x0 0x6b6ec0 : 0x0 0x0 0x6b6ed0 : 0x0 0x0 0x6b6ee0 : 0x0 0x0 0x6b6ef0 : 0x0 0x0 0x6b6f00 : 0x0 0x0 0x6b6f10 : 0x0 0x0 0x6b6f20 : 0x0 0x0 0x6b6f30 : 0x0 0x0 0x6b6f40 : 0x0 0x0 0x6b6f50 : 0x0 0x0 0x6b6f60 : 0x48d1c0 0x0 0x6b6f70: 0x0 0x0
48
0x6b6f80 : 0x6b6f90 : 0x6b6fa0 : 0x6b6fb0 : 0x6b6fc0 : 0x6b6fd0 : 0x6b6fe0 : ---Type to continue, or 0x6b6ff0 :
0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 q to quit--0x0 0x0
The output is in the following format: address:
value1 value2
Because the size of each value is 8 bytes the next address is +16 bytes or +10hex. The addresses can have associated symbolic names: address :
value1
value2
For example, from the output above: 0x6b6af0 :
0x6
0x0
Each value may also have an associated symbolic value: address :
value1
value2
For example, from the output above: 0x6b69e0 :
0x6b72c0
0x7ffc7d26c7e8
10. Explore the contents of memory pointed to by __nptl_nthreads, _dl_argv, program_invocation_short_name and 0x7ffc7d26c7e8 addresses: (gdb) x/u 0x6b6af0 0x6b6af0 :
6
(gdb) x/u &__nptl_nthreads 0x6b6af0 :
6
(gdb) x/2a 0x6b69e0 0x6b69e0 :
0x6b72c0
0x7ffc7d26c7e8
(gdb) x/2a &_dl_argv 0x6b69e0 :
0x6b72c0
0x7ffc7d26c7e8
(gdb) x/a 0x6b72c0 0x6b72c0 :
0x7ffc7d26d9a9
(gdb) x/a &program_invocation_short_name 0x6b72c0 :
0x7ffc7d26d9a9
(gdb) x/s 0x7ffc7d26d9a9 0x7ffc7d26d9a9: "App1"
49
(gdb) x/10a 0x7ffc7d26c7e8 0x7ffc7d26c7e8: 0x0 0x1 0x7ffc7d26c7f8: 0x7ffc7d26d9a7 0x7ffc7d26c808: 0x7ffc7d26d9ae 0x7ffc7d26c818: 0x7ffc7d26d9c9 0x7ffc7d26c828: 0x7ffc7d26d9e7
0x0 0x7ffc7d26d9be 0x7ffc7d26d9d9 0x7ffc7d26df08
(gdb) x/10c 0x7ffc7d26d9a7 0x7ffc7d26d9a7: 46 '.' 47 '/' 65 'A' 112 'p' 0x7ffc7d26d9af: 72 'H' 69 'E'
112 'p'
49 '1' 0 '\000'
83 'S'
(gdb) x/s 0x7ffc7d26d9a7 0x7ffc7d26d9a7: "./App1" (gdb) x/5s 0x7ffc7d26d9a7 0x7ffc7d26d9a7: "./App1" 0x7ffc7d26d9ae: "SHELL=/bin/bash" 0x7ffc7d26d9be: "TERM=linux" 0x7ffc7d26d9c9: "HUSHLOGIN=FALSE" 0x7ffc7d26d9d9: "USER=training"
11.
Explore the contents of memory pointed to by environ variable address:
(gdb) x/a &environ 0x6bd4c8 : 0x7ffc7d26c808 (gdb) x/10a 0x7ffc7d26c808 0x7ffc7d26c808: 0x7ffc7d26d9ae 0x7ffc7d26c818: 0x7ffc7d26d9c9 0x7ffc7d26c828: 0x7ffc7d26d9e7 0x7ffc7d26c838: 0x7ffc7d26df20 0x7ffc7d26c848: 0x7ffc7d26df7c
0x7ffc7d26d9be 0x7ffc7d26d9d9 0x7ffc7d26df08 0x7ffc7d26df5e 0x7ffc7d26df8d
(gdb) x/4s 0x7ffc7d26d9ae 0x7ffc7d26d9ae: "SHELL=/bin/bash" 0x7ffc7d26d9be: "TERM=linux" 0x7ffc7d26d9c9: "HUSHLOGIN=FALSE" 0x7ffc7d26d9d9: "USER=training"
12.
Get the list of loaded modules:
(gdb) info sharedlibrary No shared libraries loaded at this time.
We don’t see any shared libraries because they were statically linked. We also created the version of a dynamically linked App1.shared executable. If we load its core dump we see the list of shared libraries: training@debian64:~/ALCDA$ gdb -c ./App1/core.5476 -se ./App1/App1.shared GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /home/training/ALCDA/App1/App1.shared...(no debugging symbols found)...done. [New LWP 5477]
50
[New [New [New [New [New
LWP LWP LWP LWP LWP
5478] 5479] 5480] 5481] 5476]
warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/home/training/ALCDA/App1/App1.shared'. #0 0x00007f25a013e48d in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) info sharedlibrary From To Syms Read Shared Object Library 0x00007f25a0423690 0x00007f25a042ece8 Yes (*) /lib/x86_64-linux-gnu/libpthread.so.0 0x00007f25a00b1b80 0x00007f25a01c9c2c Yes (*) /lib/x86_64-linux-gnu/libc.so.6 0x00007f25a063aaf0 0x00007f25a0652c83 Yes (*) /lib64/ld-linux-x86-64.so.2 (*): Shared library is missing debugging information.
13.
Disassemble bar_two function and follow the indirect sleep function call:
(gdb) disassemble bar_two Dump of assembler code for function bar_two: 0x00000000004005f9 : push %rbp 0x00000000004005fa : mov %rsp,%rbp 0x00000000004005fd : mov $0xffffffff,%edi 0x0000000000400602 : callq 0x4004a0 0x0000000000400607 : pop %rbp 0x0000000000400608 : retq End of assembler dump. (gdb) disassemble 0x4004a0 Dump of assembler code for function sleep@plt: 0x00000000004004a0 : jmpq *0x20090a(%rip) 0x00000000004004a6 : pushq $0x2 0x00000000004004ab : jmpq 0x400470 End of assembler dump.
14.
# 0x600db0
Dump the annotated value as a memory address interpreting its contents as a symbol:
(gdb) x/a 0x600db0 0x600db0 : 0x7f25a013e220
51
Published by OpenTask, Republic of Ireland Copyright © 2018 by OpenTask
Copyright © 2018 by Software Diagnostics Services Copyright © 2018 by Dmitry Vostokov
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. Product and company names mentioned in this book may be trademarks of their owners.
OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected].
A CIP catalog record for this book is available from the British Library. ISBN-l3: 978-1-908043-89-4 (Paperback) Revision 2.00 (September 2018)
2
Contents About the Author.............................................................................................................................................................. 5 Presentation Slides and Transcript ................................................................................................................................... 7 Practice Exercises ........................................................................................................................................................... 35 Exercise 0 .................................................................................................................................................................... 41 Exercise D1 ................................................................................................................................................................. 50 Exercise D2 ................................................................................................................................................................. 70 Exercise D3 ................................................................................................................................................................. 83 Exercise D4 ............................................................................................................................................................... 108 Exercise D5 ............................................................................................................................................................... 115 Exercise D6 ............................................................................................................................................................... 134 Exercise D7 ............................................................................................................................................................... 143 Exercise D8 ............................................................................................................................................................... 151 Exercise K0 ................................................................................................................................................................ 163 Exercise KD6 ............................................................................................................................................................. 180 Exercise KD9 ............................................................................................................................................................. 213 Exercise KD10 ........................................................................................................................................................... 232 Exercise MD11 .......................................................................................................................................................... 255 Appendix ....................................................................................................................................................................... 299 Complete Stack Traces from x64 System ................................................................................................................. 301
3
Exercise D1 Goal: Learn how code generation parameters can influence process execution behavior. Elementary Diagnostics Patterns: Crash. Memory Analysis Patterns: Exception Stack Trace. Debugging Implementation Patterns: Scope, Variable Value, Type Structure, Code Breakpoint. 1.
Launch WinDbg from Windows Kits \ WinDbg (X64).
2.
Open \AWD3\AppD1A\x64\Release\AppD1A.exe executable:
50
3.
You get the executable file loaded and ready for a debugging session:
4.
Open a log file:
0:000> .logopen C:\AWD3\D1A.log Opened log file 'C:\AWD3\D1A.log'
5.
Set up a link to Microsoft symbol server and reload symbols:
0:000> .symfix c:\mss 0:000> .reload Reloading current modules ...........
51
6.
lm command lists module information:
0:000> lm start end module name 00007ff6`01800000 00007ff6`0181b000 AppD1A (deferred) 00007ffd`0a1f0000 00007ffd`0a27b000 apphelp (deferred) 00007ffd`0c770000 00007ffd`0c790000 win32u (deferred) 00007ffd`0c7e0000 00007ffd`0c8da000 ucrtbase (deferred) 00007ffd`0cbb0000 00007ffd`0cc4f000 msvcp_win (deferred) 00007ffd`0cc50000 00007ffd`0cec3000 KERNELBASE (deferred) 00007ffd`0d690000 00007ffd`0d822000 gdi32full (deferred) 00007ffd`0d900000 00007ffd`0d9b2000 KERNEL32 (deferred) 00007ffd`0d9c0000 00007ffd`0db50000 USER32 (deferred) 00007ffd`0e9e0000 00007ffd`0ea08000 GDI32 (deferred) 00007ffd`103a0000 00007ffd`10581000 ntdll (pdb symbols) c:\mss\ntdll.pdb\EA3C05F9EA540B02C1971816AF7CC8D21\ntdll.pdb
7. We continue process execution using g command and ignore any first chance exceptions until we come to a second chance exception: 0:000> g ModLoad: 00007ffd`101c0000 00007ffd`101ed000 C:\WINDOWS\System32\IMM32.DLL (4f80.707c): Access violation - code c0000005 (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. USER32!StringDuplicateW+0x20: 00007ffd`0d9c5cbc 66392c41 cmp word ptr [rcx+rax*2],bp ds:01816bb0`00000000=???? 0:000> g (4f80.707c): Access violation - code c0000005 (!!! second chance !!!) USER32!StringDuplicateW+0x20: 00007ffd`0d9c5cbc 66392c41 cmp word ptr [rcx+rax*2],bp ds:01816bb0`00000000=????
8.
We see that a crash happened in USER32 module with the following CPU state:
0:000> r rax=0000000000000000 rbx=0000005d794ff9d0 rcx=01816bb000000000 rdx=01816bb000000000 rsi=0000005d794ff960 rdi=01816bb000000000 rip=00007ffd0d9c5cbc rsp=0000005d794ff860 rbp=0000000000000000 r8=0000005d794ff9d0 r9=0000000000000000 r10=0000019011140000 r11=0000000000000000 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl zr ac po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010254 USER32!StringDuplicateW+0x20: 00007ffd`0d9c5cbc 66392c41 cmp word ptr [rcx+rax*2],bp ds:01816bb0`00000000=????
52
9.
The default analysis command also gives us a source code:
0:000> !analyze -v ******************************************************************************* * * * Exception Analysis * * * ******************************************************************************* *** WARNING: Unable to verify checksum for AppD1A.exe KEY_VALUES_STRING: 1 TIMELINE_ANALYSIS: 1 Timeline: Name: Time: Diff:
!analyze.Start
2018-09-12T11:47:03.53Z 946 mSec
Timeline: Name: Time: Diff:
Dump.Current
2018-09-12T11:47:04.0Z 0 mSec
Timeline: Name: Time: Diff:
Process.Start
2018-09-12T11:27:00.0Z 1204000 mSec
Timeline: Name: Time: Diff:
OS.Boot
2018-09-06T17:44:34.0Z 496950000 mSec
DUMP_CLASS: 2 DUMP_QUALIFIER: 0 FAULTING_IP: USER32!StringDuplicateW+20 00007ffd`0d9c5cbc 66392c41
cmp
word ptr [rcx+rax*2],bp
EXCEPTION_RECORD: (.exr -1) ExceptionAddress: 00007ffd0d9c5cbc (USER32!StringDuplicateW+0x0000000000000020) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000 NumberParameters: 2 Parameter[0]: 0000000000000000 Parameter[1]: ffffffffffffffff Attempt to read from address ffffffffffffffff FAULTING_THREAD: DEFAULT_BUCKET_ID: PROCESS_NAME:
0000707c INVALID_POINTER_READ
AppD1A.exe
53
FOLLOWUP_IP: AppD1A!MyRegisterClass+8d [c:\awd3\appd1a\appd1a\appd1a.cpp @ 84] 00007ff6`0180116d 4883c478 add rsp,78h READ_ADDRESS:
ffffffffffffffff
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s. EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s. EXCEPTION_CODE_STR:
c0000005
EXCEPTION_PARAMETER1:
0000000000000000
EXCEPTION_PARAMETER2:
ffffffffffffffff
WATSON_BKT_PROCSTAMP:
5b94d979
WATSON_BKT_MODULE:
USER32.dll
WATSON_BKT_MODSTAMP:
fd9a9c22
WATSON_BKT_MODOFFSET: WATSON_BKT_MODVER:
5cbc
10.0.17134.1
MODULE_VER_PRODUCT:
Microsoft® Windows® Operating System
BUILD_VERSION_STRING:
17134.1.amd64fre.rs4_release.180410-1804
MODLIST_WITH_TSCHKSUM_HASH: MODLIST_SHA1_HASH: NTGLOBALFLAG:
c517e1747eba893f351ec565e72502936e283027
f6d6417e5a956d590c2325ca86fc187e87a812ad
70
PROCESS_BAM_CURRENT_THROTTLED: 0 PROCESS_BAM_PREVIOUS_THROTTLED: 0 APPLICATION_VERIFIER_FLAGS: PRODUCT_TYPE: SUITE_MASK: DUMP_TYPE:
0
1
272 fe
ANALYSIS_SESSION_HOST:
DESKTOP-IS6V2L0
ANALYSIS_SESSION_TIME:
09-12-2018 12:47:03.0053
ANALYSIS_VERSION: 10.0.17134.12 amd64fre THREAD_ATTRIBUTES: OS_LOCALE: ENG PROBLEM_CLASSES:
54
ID: Type: Class: Scope: Name: Data: PID: TID: Frame:
[0n309] [@ACCESS_VIOLATION] Addendum BUCKET_ID Omit Omit [Unspecified] [0x707c] [0] : USER32!StringDuplicateW
ID: Type: Class: Scope:
[0n281] [INVALID_POINTER_READ] Primary DEFAULT_BUCKET_ID (Failure Bucket ID prefix) BUCKET_ID Add Omit [Unspecified] [0x707c] [0] : USER32!StringDuplicateW
Name: Data: PID: TID: Frame:
BUGCHECK_STR:
APPLICATION_FAULT_INVALID_POINTER_READ
PRIMARY_PROBLEM_CLASS:
APPLICATION_FAULT
LAST_CONTROL_TRANSFER:
from 00007ffd0d9c5475 to 00007ffd0d9c5cbc
STACK_TEXT: 0000005d`794ff860 00007ff6`01800000 0000005d`794ff890 00007ffd`0d9c4e40 0000005d`794ff8e0 00000000`00000000 0000005d`794ffc40 00000000`00000000 0000005d`794ffcd0 00000000`00000000 0000005d`794ffd50 00000000`0000000a 0000005d`794ffdb0 00000000`00000000 0000005d`794ffdf0 00000000`00000000 0000005d`794ffe20 00000000`00000000 STACK_COMMAND:
00007ffd`0d9c5475 : 0000005d`794ff9d0 : USER32!StringDuplicateW+0x20 00007ffd`0d9c4c52 : 0000005d`794ffc70 : USER32!InitClsMenuNameW+0x75 00007ffd`0d9c46ff : 00000000`00000006 : USER32!RegisterClassExWOWW+0x116 00007ff6`0180116d : 00000000`00000000 : USER32!RegisterClassW+0x6f 00007ff6`0180105c : 00007ff6`01800000 : AppD1A!MyRegisterClass+0x8d 00007ff6`0180166e : 00007ff6`01800000 : AppD1A!wWinMain+0x5c 00007ffd`0d913034 : 00000000`00000000 : AppD1A!__scrt_common_main_seh+0x106 00007ffd`10411431 : 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x14 00000000`00000000 : 00000000`00000000 : ntdll!RtlUserThreadStart+0x21
01816bb0`00000000 0000005d`794ff960 0000005d`794ff9e0 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000190`10d72aee 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
~0s ; .cxr ; kb
THREAD_SHA1_HASH_MOD_FUNC:
a981f01cd8fc185e8c4ffb6f2411e0ae6f8e3a0e
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: THREAD_SHA1_HASH_MOD: FAULT_INSTR_CODE:
ee1d72c7551cebfa6cb33a5bba1435f33ab539d3
363898a2e705fbd38e6a7fe68b9fe8bfa6feab5a
78c48348
FAULTING_SOURCE_LINE:
c:\awd3\appd1a\appd1a\appd1a.cpp
55
FAULTING_SOURCE_FILE:
c:\awd3\appd1a\appd1a\appd1a.cpp
FAULTING_SOURCE_LINE_NUMBER:
84
FAULTING_SOURCE_CODE: 80: wc.lpszMenuName = MAKEINTRESOURCE(IDC_APPD1A); 81: wc.lpszClassName = szWindowClass; 82: 83: return RegisterClass(&wc); > 84: } 85: 86: // 87: // FUNCTION: InitInstance(HINSTANCE, int) 88: // 89: // PURPOSE: Saves instance handle and creates main window SYMBOL_STACK_INDEX: SYMBOL_NAME:
4
appd1a!MyRegisterClass+8d
FOLLOWUP_NAME:
MachineOwner
MODULE_NAME: AppD1A IMAGE_NAME:
AppD1A.exe
DEBUG_FLR_IMAGE_TIMESTAMP: FAILURE_BUCKET_ID: BUCKET_ID:
5b94d979
INVALID_POINTER_READ_c0000005_AppD1A.exe!MyRegisterClass
APPLICATION_FAULT_INVALID_POINTER_READ_appd1a!MyRegisterClass+8d
FAILURE_EXCEPTION_CODE: FAILURE_IMAGE_NAME:
c0000005
AppD1A.exe
BUCKET_ID_IMAGE_STR:
AppD1A.exe
FAILURE_MODULE_NAME:
AppD1A
BUCKET_ID_MODULE_STR: FAILURE_FUNCTION_NAME:
AppD1A MyRegisterClass
BUCKET_ID_FUNCTION_STR: BUCKET_ID_OFFSET:
MyRegisterClass
8d
BUCKET_ID_MODTIMEDATESTAMP: BUCKET_ID_MODCHECKSUM:
5b94d979
0
BUCKET_ID_MODVER_STR:
0.0.0.0
BUCKET_ID_PREFIX_STR:
APPLICATION_FAULT_INVALID_POINTER_READ_
FAILURE_PROBLEM_CLASS:
APPLICATION_FAULT
56
FAILURE_SYMBOL_NAME: TARGET_TIME: OSBUILD:
AppD1A.exe!MyRegisterClass
2018-09-12T11:47:13.000Z
17134
OSSERVICEPACK:
1
SERVICEPACK_NUMBER: 0 OS_REVISION: 0 OSPLATFORM_TYPE: OSNAME:
x64
Windows 10
OSEDITION:
Windows 10 WinNt SingleUserTS
USER_LCID:
0
OSBUILD_TIMESTAMP:
2020-08-28 05:38:41
BUILDDATESTAMP_STR: BUILDLAB_STR:
180410-1804
rs4_release
BUILDOSVER_STR:
10.0.17134.1.amd64fre.rs4_release.180410-1804
ANALYSIS_SESSION_ELAPSED_TIME: ANALYSIS_SOURCE:
UM
FAILURE_ID_HASH_STRING: FAILURE_ID_HASH: Followup: ---------
10.
29fd
um:invalid_pointer_read_c0000005_appd1a.exe!myregisterclass
{0e59b433-475d-53b5-9229-de642189649b}
MachineOwner
We get a stack trace with frame numbers using kn command (k command also shows them by default):
0:000> kn # Child-SP RetAddr Call Site 00 0000005d`794ff860 00007ffd`0d9c5475 USER32!StringDuplicateW+0x20 01 0000005d`794ff890 00007ffd`0d9c4c52 USER32!InitClsMenuNameW+0x75 02 0000005d`794ff8e0 00007ffd`0d9c46ff USER32!RegisterClassExWOWW+0x116 03 0000005d`794ffc40 00007ff6`0180116d USER32!RegisterClassW+0x6f 04 0000005d`794ffcd0 00007ff6`0180105c AppD1A!MyRegisterClass+0x8d [c:\awd3\appd1a\appd1a\appd1a.cpp @ 84] 05 0000005d`794ffd50 00007ff6`0180166e AppD1A!wWinMain+0x5c [c:\awd3\appd1a\appd1a\appd1a.cpp @ 41] 06 (Inline Function) --------`-------- AppD1A!invoke_main+0x21 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 118] 07 0000005d`794ffdb0 00007ffd`0d913034 AppD1A!__scrt_common_main_seh+0x106 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 08 0000005d`794ffdf0 00007ffd`10411431 KERNEL32!BaseThreadInitThunk+0x14 09 0000005d`794ffe20 00000000`00000000 ntdll!RtlUserThreadStart+0x21
57
11.
Now we can set the frame we want to investigate (from where RegisterClassW was called):
0:000> .frame 4 04 0000005d`794ffcd0 00007ff6`0180105c AppD1A!MyRegisterClass+0x8d [c:\awd3\appd1a\appd1a\appd1a.cpp @ 84]
Note: You see a source code window immediately to the left of the command window:
58
12.
Go to View \ Options menu and check that “Evaluate on hover” is checked:
59
13.
If we select the source code window and hover a mouse pointer over wc variable we get structure variables:
We can also dump this variable using type information: 0:000> dt wc Local var @ 0x5d794ffcf0 Type tagWNDCLASSW +0x000 style : 3 +0x004 lpfnWndProc : 0x00007ff6`01801240 +0x00c cbClsExtra : 0n0 +0x010 cbWndExtra : 0n0 +0x014 hInstance : 0x00007ff6`01800000 +0x01c hIcon : 0x00000000`01730ecf +0x024 hCursor : 0x00000000`00010003 +0x02c hbrBackground : 0x00000000`00000006 +0x034 lpszMenuName : 0x00000000`0000006d 0x00000000`0000006d ---" +0x03c lpszClassName : 0x00007ff6`01816bb0
14.
int64
AppD1A!WndProc+0
HINSTANCE__ HICON__ HICON__ HBRUSH__ "--- memory read error at address "APPD1A"
We can also list all other local variables and parameters for the current frame:
0:000> dv /i /V prv param 0000005d`794ffd50 @rsp+0x0080 prv local 0000005d`794ffcf0 @rsp+0x0020
hInstance = 0x00007ff6`01800000 wc = struct tagWNDCLASSW
Note: Since all structure members seem to be valid let’s compare it with another application that doesn’t crash. 60
15. Launch another instance of WinDbg from Windows Kits \ WinDbg (X64) and open \AWD3\AppD1B\x64\Release\AppD1B.exe executable. We get the following output: Microsoft (R) Windows Debugger Version 10.0.17134.12 AMD64 Copyright (c) Microsoft Corporation. All rights reserved. CommandLine: C:\AWD3\AppD1B\x64\Release\AppD1B.exe Symbol search path is: srv* Executable search path is: ModLoad: 00007ff6`9d280000 00007ff6`9d29b000 AppD1B.exe ModLoad: 00007ffd`103a0000 00007ffd`10581000 ntdll.dll ModLoad: 00007ffd`0d900000 00007ffd`0d9b2000 C:\WINDOWS\System32\KERNEL32.DLL ModLoad: 00007ffd`0cc50000 00007ffd`0cec3000 C:\WINDOWS\System32\KERNELBASE.dll ModLoad: 00007ffd`0a1f0000 00007ffd`0a27b000 C:\WINDOWS\SYSTEM32\apphelp.dll ModLoad: 00007ffd`0d9c0000 00007ffd`0db50000 C:\WINDOWS\System32\USER32.dll ModLoad: 00007ffd`0c770000 00007ffd`0c790000 C:\WINDOWS\System32\win32u.dll ModLoad: 00007ffd`0e9e0000 00007ffd`0ea08000 C:\WINDOWS\System32\GDI32.dll ModLoad: 00007ffd`0d690000 00007ffd`0d822000 C:\WINDOWS\System32\gdi32full.dll ModLoad: 00007ffd`0cbb0000 00007ffd`0cc4f000 C:\WINDOWS\System32\msvcp_win.dll ModLoad: 00007ffd`0c7e0000 00007ffd`0c8da000 C:\WINDOWS\System32\ucrtbase.dll (8c34.8834): Break instruction exception - code 80000003 (first chance) ntdll!LdrpDoDebuggerBreak+0x30: 00007ffd`1046cd9c cc int 3
16.
We open a new log file, fix and reload symbols:
0:000> .logopen C:\AWD3\D1B.log Opened log file 'C:\AWD3\D1B.log' 0:000> .symfix c:\mss 0:000> .reload Reloading current modules ...........
61
17.
If we run it via g command, we don’t get any exceptions:
62
18.
So we choose Debug \ Break menu option and then Debug \ Restart. We get the following output:
0:000> g ModLoad: 00007ffd`101c0000 00007ffd`101ed000 C:\WINDOWS\System32\IMM32.DLL ModLoad: 00007ffd`0a390000 00007ffd`0a428000 C:\WINDOWS\system32\uxtheme.dll ModLoad: 00007ffd`0ea30000 00007ffd`0eace000 C:\WINDOWS\System32\msvcrt.dll ModLoad: 00007ffd`0dd60000 00007ffd`0e083000 C:\WINDOWS\System32\combase.dll ModLoad: 00007ffd`0e680000 00007ffd`0e7a4000 C:\WINDOWS\System32\RPCRT4.dll ModLoad: 00007ffd`0cad0000 00007ffd`0cb4a000 C:\WINDOWS\System32\bcryptPrimitives.dll ModLoad: 00007ffd`101f0000 00007ffd`10365000 C:\WINDOWS\System32\MSCTF.dll ModLoad: 00007ffd`0dd00000 00007ffd`0dd5b000 C:\WINDOWS\System32\sechost.dll ModLoad: 00007ffd`0d830000 00007ffd`0d8f2000 C:\WINDOWS\System32\OLEAUT32.dll ModLoad: 00007ffd`0aa90000 00007ffd`0aab9000 C:\WINDOWS\system32\dwmapi.dll ModLoad: 00007ffd`0c6d0000 00007ffd`0c6e1000 C:\WINDOWS\System32\kernel.appcore.dll ModLoad: 00007ffd`00880000 00007ffd`008eb000 C:\WINDOWS\system32\Oleacc.dll (8c34.6b98): Break instruction exception - code 80000003 (first chance) ntdll!DbgBreakPoint: 00007ffd`1043d880 cc int 3 0:001> .restart /f NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\atlmfc.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\concurrency.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\cpp_rest.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\stl.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Data.Json.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Devices.Geolocation.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Devices.Sensors.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\Windows.Media.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\windows.natvis' NatVis script unloaded from 'C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\Visualizers\winrt.natvis' CommandLine: C:\AWD3\AppD1B\x64\Release\AppD1B.exe ************* Path validation summary ************** Response Time (ms) Location Deferred srv* ************* Path validation summary ************** Response Time (ms) Location Deferred srv* Symbol search path is: srv* Executable search path is: srv* ModLoad: 00007ff6`9d280000 00007ff6`9d29b000 AppD1B.exe ModLoad: 00007ffd`103a0000 00007ffd`10581000 ntdll.dll ModLoad: 00007ffd`0d900000 00007ffd`0d9b2000 C:\WINDOWS\System32\KERNEL32.DLL ModLoad: 00007ffd`0cc50000 00007ffd`0cec3000 C:\WINDOWS\System32\KERNELBASE.dll ModLoad: 00007ffd`0d9c0000 00007ffd`0db50000 C:\WINDOWS\System32\USER32.dll ModLoad: 00007ffd`0c770000 00007ffd`0c790000 C:\WINDOWS\System32\win32u.dll ModLoad: 00007ffd`0e9e0000 00007ffd`0ea08000 C:\WINDOWS\System32\GDI32.dll ModLoad: 00007ffd`0d690000 00007ffd`0d822000 C:\WINDOWS\System32\gdi32full.dll ModLoad: 00007ffd`0cbb0000 00007ffd`0cc4f000 C:\WINDOWS\System32\msvcp_win.dll
63
ModLoad: 00007ffd`0c7e0000 00007ffd`0c8da000 C:\WINDOWS\System32\ucrtbase.dll (7628.9044): Break instruction exception - code 80000003 (first chance) ntdll!LdrpDoDebuggerBreak+0x30: 00007ffd`1046cd9c cc int 3
19. Since we want to compare the same behavior of RegisterClassW function we need to put a breakpoint to break in when this function is about to be executed. Then we would see WNDCLASS structure passed to it. We set a pattern matching breakpoint using bm command: 0:000> bm *!RegisterClassW *** WARNING: Unable to verify checksum for AppD1B.exe 1: 00007ffd`0cd40330 @!"KERNELBASE!RegisterClassW" 2: 00007ffd`0d9c4690 @!"USER32!RegisterClassW"
20.
Indeed we a hit immediately:
0:000> g ModLoad: 00007ffd`101c0000 00007ffd`101ed000 Breakpoint 2 hit USER32!RegisterClassW: 00007ffd`0d9c4690 4053 push rbx
C:\WINDOWS\System32\IMM32.DLL
We get an identical stack trace prior to RegisterClassW when we compare with the previously running instance of AppD1A.exe: 0:000> k ; AppD1B # Child-SP RetAddr Call Site 00 00000075`4e9bf808 00007ff6`9d28116d USER32!RegisterClassW 01 00000075`4e9bf810 00007ff6`9d28105c AppD1B!MyRegisterClass+0x8d [c:\awd3\appd1b\appd1b\appd1b.cpp @ 84] 02 00000075`4e9bf890 00007ff6`9d28166e AppD1B!wWinMain+0x5c [c:\awd3\appd1b\appd1b\appd1b.cpp @ 41] 03 (Inline Function) --------`-------- AppD1B!invoke_main+0x21 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 118] 04 00000075`4e9bf900 00007ffd`0d913034 AppD1B!__scrt_common_main_seh+0x106 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 05 00000075`4e9bf940 00007ffd`10411431 KERNEL32!BaseThreadInitThunk+0x14 06 00000075`4e9bf970 00000000`00000000 ntdll!RtlUserThreadStart+0x21 0:000> k ; AppD1A # Child-SP RetAddr Call Site 00 0000005d`794ff860 00007ffd`0d9c5475 USER32!StringDuplicateW+0x20 01 0000005d`794ff890 00007ffd`0d9c4c52 USER32!InitClsMenuNameW+0x75 02 0000005d`794ff8e0 00007ffd`0d9c46ff USER32!RegisterClassExWOWW+0x116 03 0000005d`794ffc40 00007ff6`0180116d USER32!RegisterClassW+0x6f 04 0000005d`794ffcd0 00007ff6`0180105c AppD1A!MyRegisterClass+0x8d [c:\awd3\appd1a\appd1a\appd1a.cpp @ 84] 05 0000005d`794ffd50 00007ff6`0180166e AppD1A!wWinMain+0x5c [c:\awd3\appd1a\appd1a\appd1a.cpp @ 41] 06 (Inline Function) --------`-------- AppD1A!invoke_main+0x21 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 118] 07 0000005d`794ffdb0 00007ffd`0d913034 AppD1A!__scrt_common_main_seh+0x106 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 08 0000005d`794ffdf0 00007ffd`10411431 KERNEL32!BaseThreadInitThunk+0x14 09 0000005d`794ffe20 00000000`00000000 ntdll!RtlUserThreadStart+0x21
64
21. We choose frame 1 which called RegisterClassW and immediately get access to wc variable (we also note that function MyRegisterClass source code is identical to AppD1A): 0:000> kn # Child-SP RetAddr Call Site 00 00000075`4e9bf808 00007ff6`9d28116d USER32!RegisterClassW 01 00000075`4e9bf810 00007ff6`9d28105c AppD1B!MyRegisterClass+0x8d [c:\awd3\appd1b\appd1b\appd1b.cpp @ 84] 02 00000075`4e9bf890 00007ff6`9d28166e AppD1B!wWinMain+0x5c [c:\awd3\appd1b\appd1b\appd1b.cpp @ 41] 03 (Inline Function) --------`-------- AppD1B!invoke_main+0x21 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 118] 04 00000075`4e9bf900 00007ffd`0d913034 AppD1B!__scrt_common_main_seh+0x106 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 05 00000075`4e9bf940 00007ffd`10411431 KERNEL32!BaseThreadInitThunk+0x14 06 00000075`4e9bf970 00000000`00000000 ntdll!RtlUserThreadStart+0x21 0:000> .frame 1 01 00000075`4e9bf810 00007ff6`9d28105c AppD1B!MyRegisterClass+0x8d [c:\awd3\appd1b\appd1b\appd1b.cpp @ 84] 0:000> dt wc ; AppD1B Local var @ 0x754e9bf830 Type tagWNDCLASSW +0x000 style : 3 +0x008 lpfnWndProc : 0x00007ff6`9d281240 +0x010 cbClsExtra : 0n0 +0x014 cbWndExtra : 0n0 +0x018 hInstance : 0x00007ff6`9d280000 +0x020 hIcon : 0x00000000`04602229 +0x028 hCursor : 0x00000000`00010003 +0x030 hbrBackground : 0x00000000`00000006 +0x038 lpszMenuName : 0x00000000`0000006d 0x00000000`0000006d ---" +0x040 lpszClassName : 0x00007ff6`9d296bb0
22.
AppD1B!WndProc+0
HINSTANCE__ HICON__ HICON__ HBRUSH__ "--- memory read error at address "APPD1B"
But if we look at AppD1A structure variant we see its members have different offsets:
0:000> dt wc ; AppD1A Local var @ 0x5d794ffcf0 Type tagWNDCLASSW +0x000 style : 3 +0x004 lpfnWndProc : 0x00007ff6`01801240 +0x00c cbClsExtra : 0n0 +0x010 cbWndExtra : 0n0 +0x014 hInstance : 0x00007ff6`01800000 +0x01c hIcon : 0x00000000`01730ecf +0x024 hCursor : 0x00000000`00010003 +0x02c hbrBackground : 0x00000000`00000006 +0x034 lpszMenuName : 0x00000000`0000006d 0x00000000`0000006d ---" +0x03c lpszClassName : 0x00007ff6`01816bb0
23.
int64
We close logs in both WinDbg instances:
0:000> .logclose ; AppD1A Closing open log file C:\AWD3\D1A.log 0:000> .logclose ; AppD1B Closing open log file C:\AWD3\D1B.log
65
int64
AppD1A!WndProc+0
HINSTANCE__ HICON__ HICON__ HBRUSH__ "--- memory read error at address "APPD1A"
Note: To avoid possible confusion and glitches, we recommend exiting WinDbg after each exercise. 24. The problem was partially fixed without changing alignment by using a different bigger structure WNDCLASSEX and RegisterClassExW Win32 API function. We open \AWD3\AppD1C\x64\Release\AppD1C.exe in another WinDbg instance: Microsoft (R) Windows Debugger Version 10.0.17134.12 AMD64 Copyright (c) Microsoft Corporation. All rights reserved. CommandLine: C:\AWD3\AppD1C\x64\Release\AppD1C.exe Symbol search path is: srv* Executable search path is: ModLoad: 00007ff7`f84f0000 00007ff7`f850b000 AppD1C.exe ModLoad: 00007ffd`103a0000 00007ffd`10581000 ntdll.dll ModLoad: 00007ffd`0d900000 00007ffd`0d9b2000 C:\WINDOWS\System32\KERNEL32.DLL ModLoad: 00007ffd`0cc50000 00007ffd`0cec3000 C:\WINDOWS\System32\KERNELBASE.dll ModLoad: 00007ffd`0a1f0000 00007ffd`0a27b000 C:\WINDOWS\SYSTEM32\apphelp.dll ModLoad: 00007ffd`0d9c0000 00007ffd`0db50000 C:\WINDOWS\System32\USER32.dll ModLoad: 00007ffd`0c770000 00007ffd`0c790000 C:\WINDOWS\System32\win32u.dll ModLoad: 00007ffd`0e9e0000 00007ffd`0ea08000 C:\WINDOWS\System32\GDI32.dll ModLoad: 00007ffd`0d690000 00007ffd`0d822000 C:\WINDOWS\System32\gdi32full.dll ModLoad: 00007ffd`0cbb0000 00007ffd`0cc4f000 C:\WINDOWS\System32\msvcp_win.dll ModLoad: 00007ffd`0c7e0000 00007ffd`0c8da000 C:\WINDOWS\System32\ucrtbase.dll (dec.331c): Break instruction exception - code 80000003 (first chance) ntdll!LdrpDoDebuggerBreak+0x30: 00007ffd`1046cd9c cc int 3 0:000> .symfix c:\mss 0:000> .reload Reloading current modules .......... 0:000> bm *!RegisterClassExW *** WARNING: Unable to verify checksum for AppD1C.exe 1: 00007ffd`0cd40330 @!"KERNELBASE!RegisterClassExW" 2: 00007ffd`0d9c4660 @!"USER32!RegisterClassExW" 0:000> g ModLoad: 00007ffd`101c0000 00007ffd`101ed000 C:\WINDOWS\System32\IMM32.DLL Breakpoint 2 hit USER32!RegisterClassExW: 00007ffd`0d9c4660 4883ec38 sub rsp,38h 0:000> kn # Child-SP RetAddr Call Site 00 000000a4`e30ff858 00007ff7`f84f118a USER32!RegisterClassExW 01 000000a4`e30ff860 00007ff7`f84f105c AppD1C!MyRegisterClass+0xaa [c:\awd3\appd1c\appd1c\appd1c.cpp @ 84] 02 000000a4`e30ff8e0 00007ff7`f84f167e AppD1C!wWinMain+0x5c [c:\awd3\appd1c\appd1c\appd1c.cpp @ 38] 03 (Inline Function) --------`-------- AppD1C!invoke_main+0x21 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 118] 04 000000a4`e30ff940 00007ffd`0d913034 AppD1C!__scrt_common_main_seh+0x106 [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288] 05 000000a4`e30ff980 00007ffd`10411431 KERNEL32!BaseThreadInitThunk+0x14 06 000000a4`e30ff9b0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
66
0:000> .frame 1 01 000000a4`e30ff860 00007ff7`f84f105c AppD1C!MyRegisterClass+0xaa [c:\awd3\appd1c\appd1c\appd1c.cpp @ 84] 0:000> dv /i /V prv param 000000a4`e30ff8e0 @rsp+0x0080 prv local 000000a4`e30ff880 @rsp+0x0020
hInstance = 0x00007ff7`f84f0000 wcex = struct tagWNDCLASSEXW
Note: Adding a new extra member in the new structure shifts the remaining members and set the same layout as in AppD1B: 0:000> dt wcex ; AppD1C Local var @ 0xa4e30ff880 Type tagWNDCLASSEXW +0x000 cbSize : 0x50 +0x004 style : 3 +0x008 lpfnWndProc : 0x00007ff7`f84f1250 +0x010 cbClsExtra : 0n0 +0x014 cbWndExtra : 0n0 +0x018 hInstance : 0x00007ff7`f84f0000 +0x020 hIcon : 0x00000000`14a4261d +0x028 hCursor : 0x00000000`00010003 +0x030 hbrBackground : 0x00000000`00000006 +0x038 lpszMenuName : 0x00000000`0000006d 0x00000000`0000006d ---" +0x040 lpszClassName : 0x00007ff7`f8506bb0 +0x048 hIconSm : 0x00000000`00bf1e45 0:000> dt wc ; AppD1B Local var @ 0x754e9bf830 Type tagWNDCLASSW +0x000 style : 3 +0x008 lpfnWndProc : 0x00007ff6`9d281240 +0x010 cbClsExtra : 0n0 +0x014 cbWndExtra : 0n0 +0x018 hInstance : 0x00007ff6`9d280000 +0x020 hIcon : 0x00000000`04602229 +0x028 hCursor : 0x00000000`00010003 +0x030 hbrBackground : 0x00000000`00000006 +0x038 lpszMenuName : 0x00000000`0000006d 0x00000000`0000006d ---" +0x040 lpszClassName : 0x00007ff6`9d296bb0
int64
AppD1C!WndProc+0
HINSTANCE__ HICON__ HICON__ HBRUSH__ "--- memory read error at address "APPD1C" HICON__
int64
AppD1B!WndProc+0
HINSTANCE__ HICON__ HICON__ HBRUSH__ "--- memory read error at address "APPD1B"
Note: AppD1A wasn’t working because of structure member alignment. This models an old Windows 3.x project that was ported to x64. It had the minimum alignment in the past to reduce memory consumption:
67
AppD1B was working because the alignment was changed to default. AppD1C still used the same 1-byte alignment but because the bigger structure shifted members of the substructure it didn’t crash.
68
Published by OpenTask, Republic of Ireland Copyright © 2017 by OpenTask Copyright © 2017 by Software Diagnostics Services Copyright © 2017 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover, and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalog record for this book is available from the British Library. ISBN-l3: 978-1-908043-86-3 (Paperback) Revision 2.02 (October 2017)
2
Contents About the Author.............................................................................................................................................................. 5 Introduction ...................................................................................................................................................................... 7 Practice Exercises ........................................................................................................................................................... 17 Exercise 0: Download, setup and verify your WinDbg installation ............................................................................ 22 Exercise M1A .............................................................................................................................................................. 35 Exercise M1B .............................................................................................................................................................. 48 Exercise M2................................................................................................................................................................. 60 Exercise M3................................................................................................................................................................. 77 Exercise M4............................................................................................................................................................... 130 Exercise M5............................................................................................................................................................... 186 Exercise M6............................................................................................................................................................... 210 Selected Q&A................................................................................................................................................................ 232 Appendix ....................................................................................................................................................................... 235 Malware Analysis Patterns ....................................................................................................................................... 237 Deviant Module .................................................................................................................................................... 237 Deviant Token....................................................................................................................................................... 244 Driver Device Collection ....................................................................................................................................... 245 Execution Residue ................................................................................................................................................ 246 Fake Module ......................................................................................................................................................... 270 Hidden Module ..................................................................................................................................................... 274 Hidden Process ..................................................................................................................................................... 276 Hooksware ............................................................................................................................................................ 278 Namespace ........................................................................................................................................................... 279 No Component Symbols ....................................................................................................................................... 280 Out-of-Module Pointer ......................................................................................................................................... 283 Packed Code ......................................................................................................................................................... 284 Patched Code........................................................................................................................................................ 287 Pre-Obfuscation Residue ...................................................................................................................................... 288 Raw Pointer .......................................................................................................................................................... 289 RIP Stack Trace ..................................................................................................................................................... 290 Self-Diagnosis (Kernel Mode) ............................................................................................................................... 292 Stack Trace Collection .......................................................................................................................................... 293 Stack Trace Collection (I/O Requests) .................................................................................................................. 301 3
String Hint ............................................................................................................................................................. 305 Unknown Module ................................................................................................................................................. 307 Raw Stack Dump of All Threads (Kernel Space) ........................................................................................................ 310 Complete Stack Traces from x64 System ................................................................................................................. 311
4
Exercise M1A Goal: Look at module headers and version information before load. Patterns: Unknown Module. 1.
Launch WinDbg from Windows Kits \ WinDbg (X64) or Windows Kits \ WinDbg (X86).
2.
Open \AWMA-Dumps\Executables\M1.exe
35
3.
You get the EXE file loaded:
4.
Symbols are not necessary for our exercise.
5.
Open a log file:
0:000> .logopen C:\AWMA-Dumps\M1A.log Opened log file 'C:\AWMA-Dumps\M1A.log'
36
6.
lmv command lists module information:
0:000> lmv start end module name 00000001`40000000 00000001`40018000 M1 C (no symbols) Loaded symbol image file: M1.exe Mapped memory image file: C:\AWMA-Dumps\Executables\M1.exe Image path: C:\AWMA-Dumps\Executables\M1.exe Image name: M1.exe Timestamp: Mon Jan 28 15:24:45 2013 (5106983D) CheckSum: 00000000 ImageSize: 00018000 Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
Note module default load address. 7.
!lmi command gives a bit more information:
0:000> !lmi 00000001`40000000 Loaded Module Info: [00000001`40000000] Module: M1 Base Address: 0000000140000000 Image Name: M1.exe Machine Type: 34404 (X64) Time Stamp: 5106983d Mon Jan 28 15:24:45 2013 Size: 18000 CheckSum: 0 Characteristics: 22 Debug Data Dirs: Type Size VA Pointer CODEVIEW 3b, e370, cb70 RSDS - GUID: {3F1487A5-A6DC-4351-AD23-76FC12BB9482} Age: 1, Pdb: C:\Work\AWMA\M1\x64\Release\M1.pdb ?? 10, e3ac, cbac [Data not mapped] Image Type: FILE - Image read successfully from debugger. M1.exe Symbol Type: NONE - PDB not found from image path. Load Report: no symbols loaded
Note a reference to a PDB file. If left by a developer it might give some clues as we in other exercises. 8.
We dump the first kilobyte:
0:000> dc 00000001`40000000 00000001`40000000 00905a4d 00000001`40000010 000000b8 00000001`40000020 00000000 00000001`40000030 00000000 00000001`40000040 0eba1f0e 00000001`40000050 70207369 00000001`40000060 65622074 00000001`40000070 65646f6d 00000001`40000080 cb8e1818 00000001`40000090 982fbfad 00000001`400000a0 982dbfad 00000001`400000b0 98590ea0 00000001`400000c0 9829befe 00000001`400000d0 982cbefe 00000001`400000e0 00000000 00000001`400000f0 5106983d
L100 00000003 00000000 00000000 00000000 cd09b400 72676f72 6e757220 0a0d0d2e 98e0795c 98e0794e 98e0795b 98e07959 98e0795d 98e0795d 00000000 00000000
00000004 00000040 00000000 00000000 4c01b821 63206d61 206e6920 00000024 98e0795c 982ebfad 98e1795c 9833befe 9877795c 68636952 00004550 00000000
0000ffff 00000000 00000000 000000e8 685421cd 6f6e6e61 20534f44 00000000 98e0795c 98e07908 98e07903 98e0795e 98e0795d 98e0795c 00068664 002200f0
37
MZ.............. ........@....... ................ ................ ........!..L.!Th is program canno t be run in DOS mode....$....... ....\y..\y..\y.. ../.Ny.......y.. ..-.[y..\y...y.. ..Y.Yy....3.^y.. ..).]y..\yw.]y.. ..,.]y..Rich\y.. ........PE..d... =..Q..........".
00000001`40000100 00000001`40000110 00000001`40000120 00000001`40000130 00000001`40000140 00000001`40000150 00000001`40000160 00000001`40000170 00000001`40000180 00000001`40000190 00000001`400001a0 00000001`400001b0 00000001`400001c0 00000001`400001d0 00000001`400001e0 00000001`400001f0 00000001`40000200 00000001`40000210 00000001`40000220 00000001`40000230 00000001`40000240 00000001`40000250 00000001`40000260 00000001`40000270 00000001`40000280 00000001`40000290 00000001`400002a0 00000001`400002b0 00000001`400002c0 00000001`400002d0 00000001`400002e0 00000001`400002f0 00000001`40000300 00000001`40000310 00000001`40000320 00000001`40000330 00000001`40000340 00000001`40000350 00000001`40000360 00000001`40000370 00000001`40000380 00000001`40000390 00000001`400003a0 00000001`400003b0 00000001`400003c0 00000001`400003d0 00000001`400003e0 00000001`400003f0
000b020b 000016a8 00001000 00000006 00000000 00001000 00001000 00000000 00015000 00000000 00009320 00000000 0000e300 00009000 00000000 7865742e 00007400 00000000 00006366 00000000 7461642e 00001400 00000000 0000078c 00000000 7273722e 00001e00 00000000 00000c52 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00007400 00001000 00000200 00000000 81600002 00000000 00000000 00000000 00001d68 00000000 00000038 00000000 00000070 000002a0 00000000 00000074 00000400 60000020 00009000 00000000 00000061 0000dc00 c0000040 00014000 00000000 00000063 0000f800 40000040 00017000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
0000d200 40000000 00000006 00018000 00100000 00100000 00000000 0000eaa4 00014000 00017000 00000000 00000000 00000000 00000000 00000000 0000731b 00000000 6164722e 00006400 00000000 00003900 00000000 6164702e 00000800 00000000 00001d68 00000000 6c65722e 00000e00 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000001 00000000 00000400 00000000 00000000 00000010 0000003c 0000078c 00000530 00000000 00000000 00000000 00000000 00000000 00001000 00000000 00006174 00007800 40000040 00010000 00000000 00006174 0000f000 40000040 00015000 00000000 0000636f 00011600 42000040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
38
.....t.......... ...........@.... ................ ................ ......`......... ................ ................ ............ !dh 00000001`40000000 File Type: EXECUTABLE IMAGE FILE HEADER VALUES 8664 machine (X64) 6 number of sections 5106983D time date stamp Mon Jan 28 15:24:45 2013 0 0 F0 22
file pointer to symbol table number of symbols size of optional header characteristics Executable App can handle >2gb addresses
OPTIONAL 20B 11.00 7400 D200 0 16A8 1000
HEADER VALUES magic # linker version size of code size of initialized data size of uninitialized data address of entry point base of code ----- new ----0000000140000000 image base 1000 section alignment 200 file alignment 2 subsystem (Windows GUI) 6.00 operating system version 0.00 image version 6.00 subsystem version 18000 size of image 400 size of headers 0 checksum 0000000000100000 size of stack reserve 0000000000001000 size of stack commit 0000000000100000 size of heap reserve 0000000000001000 size of heap commit 8160 DLL characteristics High entropy VA supported Dynamic base NX compatible Terminal server aware 0 [ 0] address [size] of Export Directory EAA4 [ 3C] address [size] of Import Directory 15000 [ 1D68] address [size] of Resource Directory 14000 [ 78C] address [size] of Exception Directory 0 [ 0] address [size] of Security Directory 17000 [ 530] address [size] of Base Relocation Directory 9320 [ 38] address [size] of Debug Directory 0 [ 0] address [size] of Description Directory 0 [ 0] address [size] of Special Directory 0 [ 0] address [size] of Thread Storage Directory E300 [ 70] address [size] of Load Configuration Directory 0 [ 0] address [size] of Bound Import Directory 9000 [ 2A0] address [size] of Import Address Table Directory 0 [ 0] address [size] of Delay Import Directory 0 [ 0] address [size] of COR20 Header Directory
39
0 [
0] address [size] of Reserved Directory
SECTION HEADER #1 .text name 731B virtual size 1000 virtual address 7400 size of raw data 400 file pointer to raw data 0 file pointer to relocation table 0 file pointer to line numbers 0 number of relocations 0 number of line numbers 60000020 flags Code (no align specified) Execute Read SECTION HEADER #2 .rdata name 6366 virtual size 9000 virtual address 6400 size of raw data 7800 file pointer to raw data 0 file pointer to relocation table 0 file pointer to line numbers 0 number of relocations 0 number of line numbers 40000040 flags Initialized Data (no align specified) Read Only Debug Directories(2) Type Size Address Pointer cv 3b e370 cb70 C:\Work\AWMA\M1\x64\Release\M1.pdb ( 12) 10 e3ac cbac
Format: RSDS, guid, 1,
SECTION HEADER #3 .data name 3900 virtual size 10000 virtual address 1400 size of raw data DC00 file pointer to raw data 0 file pointer to relocation table 0 file pointer to line numbers 0 number of relocations 0 number of line numbers C0000040 flags Initialized Data (no align specified) Read Write
40
SECTION HEADER #4 .pdata name 78C virtual size 14000 virtual address 800 size of raw data F000 file pointer to raw data 0 file pointer to relocation table 0 file pointer to line numbers 0 number of relocations 0 number of line numbers 40000040 flags Initialized Data (no align specified) Read Only SECTION HEADER #5 .rsrc name 1D68 virtual size 15000 virtual address 1E00 size of raw data F800 file pointer to raw data 0 file pointer to relocation table 0 file pointer to line numbers 0 number of relocations 0 number of line numbers 40000040 flags Initialized Data (no align specified) Read Only SECTION HEADER #6 .reloc name C52 virtual size 17000 virtual address E00 size of raw data 11600 file pointer to raw data 0 file pointer to relocation table 0 file pointer to line numbers 0 number of relocations 0 number of line numbers 42000040 flags Initialized Data Discardable (no align specified) Read Only
Note Import Directory, Import Address Table Directory, and code .text section. 10.
Let’s look at Import Address Table Directory before dynamic linking takes place:
0:000> dps 00000001`40000000+9000 00000001`40009000 ????????`???????? 00000001`40009008 ????????`???????? 00000001`40009010 ????????`???????? 00000001`40009018 ????????`???????? 00000001`40009020 ????????`???????? 00000001`40009028 ????????`???????? 00000001`40009030 ????????`???????? 00000001`40009038 ????????`????????
41
00000001`40009040 00000001`40009048 00000001`40009050 00000001`40009058 00000001`40009060 00000001`40009068 00000001`40009070 00000001`40009078
????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`????????
We see it is inaccessible or not present. However, Import Directory is available, and we can dump its contents using the module image address, relative offset, and size (in bytes). It is an array of structures each of 5 double words (4 bytes per double word). This is why we use dd command and divide the size by 4: 0:000> dd 00000001`40000000+EAA4 L3C/4 00000001`4000eaa4 0000eae0 00000000 00000000 0000ed90 00000001`4000eab4 00009000 0000ece0 00000000 00000000 00000001`4000eac4 0000eed8 00009200 00000000 00000000 00000001`4000ead4 00000000 00000000 00000000
The first double word in each structure is a relative offset to a relative offset to an array of names such as function names, and the fourth double word is a relative offset to an import DLL name: 0:000> da 00000001`40000000+0000ed90 00000001`4000ed90 "KERNEL32.dll" 0:000> da 00000001`40000000+0000eed8 00000001`4000eed8 "USER32.dll"
We now examine function names to be imported from KERNEL32.dll: 0:000> dc 00000001`40000000+0000eae0 00000001`4000eae0 00000000`0000ed80 00000001`4000eaf0 00000000`0000f33a 00000001`4000eb00 00000000`0000f316 00000001`4000eb10 00000000`0000f2f4 00000001`4000eb20 00000000`0000f2d0 00000001`4000eb30 00000000`0000f2b2 00000001`4000eb40 00000000`0000f28e 00000001`4000eb50 00000000`0000eee4
00000000`0000f34a 00000000`0000f326 00000000`0000f304 00000000`0000f2e0 00000000`0000f2c4 00000000`0000f29c 00000000`0000f282 00000000`0000eef6
0:000> dc 00000001`40000000+00000000`0000ed80 00000001`4000ed80 6f4c03c6 694c6461 72617262 00000001`4000ed90 4e52454b 32334c45 6c6c642e 00000001`4000eda0 64616f4c 69727453 0057676e 00000001`4000edb0 63416461 656c6563 6f746172 00000001`4000edc0 65470175 73654d74 65676173 00000001`4000edd0 6e617254 74616c73 63634165 00000001`4000ede0 726f7461 03430057 6e617254 00000001`4000edf0 73654d65 65676173 00b60000 00000001`4000ee00 68637461 7373654d 57656761 00000001`4000ee10 64616f4c 6e6f6349 02240057 00000001`4000ee20 73727543 0057726f 6552028a 00000001`4000ee30 6c437265 45737361 00005778 00000001`4000ee40 65746165 646e6957 7845776f 00000001`4000ee50 776f6853 646e6957 0000776f 00000001`4000ee60 65746164 646e6957 0000776f 00000001`4000ee70 676f6c61 50786f42 6d617261 00000001`4000ee80 74736544 57796f72 6f646e69 00000001`4000ee90 57666544 6f646e69 6f725077
L100 00005779 02330000 6f4c021e 00577372 03410057 72656c65 74616c73 70736944 02260000 64616f4c 74736967 72430071 03240057 7055035b 694400b3 00ad0057 00a10077 00005763
42
..LoadLibraryW.. KERNEL32.dll..3. LoadStringW...Lo adAcceleratorsW. u.GetMessageW.A. TranslateAcceler atorW.C.Translat eMessage....Disp atchMessageW..&. LoadIconW.$.Load CursorW...Regist erClassExW..q.Cr eateWindowExW.$. ShowWindow..[.Up dateWindow....Di alogBoxParamW... DestroyWindow... DefWindowProcW..
00000001`4000eea0 00000001`4000eeb0 00000001`4000eec0 00000001`4000eed0 00000001`4000eee0 00000001`4000eef0 00000001`4000ef00 00000001`4000ef10 00000001`4000ef20 00000001`4000ef30 00000001`4000ef40 00000001`4000ef50 00000001`4000ef60 00000001`4000ef70 00000001`4000ef80 00000001`4000ef90 00000001`4000efa0 00000001`4000efb0 00000001`4000efc0 00000001`4000efd0 00000001`4000efe0 00000001`4000eff0 00000001`4000f000 00000001`4000f010 00000001`4000f020 00000001`4000f030 00000001`4000f040 00000001`4000f050 00000001`4000f060 00000001`4000f070 00000001`4000f080 00000001`4000f090 00000001`4000f0a0 00000001`4000f0b0 00000001`4000f0c0 00000001`4000f0d0 00000001`4000f0e0 00000001`4000f0f0 00000001`4000f100 00000001`4000f110 00000001`4000f120 00000001`4000f130 00000001`4000f140 00000001`4000f150 00000001`4000f160 00000001`4000f170
6542000e 50646e45 74697551 61694464 00006c6c 656e694c 72507265 7365636f 6e657365 726f7272 726f7272 6854746e 65646f63 65646f63 72507469 6c75646f 654702bc 03ef0000 43656469 646e6148 00656c69 4e656c69 65636f72 6c694674 696c6169 6f697463 011f0074 65536c61 74726174 50797265 65746e75 7250746e 73795374 6d695465 656d6e6f 724601bd 72745374 75747061 4c6c7452 746e456e 6c617574 646e6168 746c6946 6c646e61 65746c69 7250746e
506e6967 746e6961 7373654d 00676f6c 654701e9 03860057 6e657365 46726f73 02700074 05250000 022e0000 64616572 6e696f50 6e696f50 7365636f 6e614865 6f725074 746c754d 00726168 0000656c 65470283 57656d61 65487373 70795465 7243657a 646e416e 656c6544 6f697463 6e497075 6f667265 022a0072 7365636f 546d6574 02470065 7453746e 6e456565 73676e69 6f436572 756b6f6f 00007972 69776e55 4564656c 00007265 78456465 02290072 7365636f
746e6961 02720000 00656761 52455355 6d6f4374 65447349 038b0074 75746165 4c746547 4c746553 43746547 00006449 00726574 00726574 02860073 45656c64 64644163 74794269 654702e4 72570601 646f4d74 02c10000 00007061 036f0065 63697469 6e697053 72436574 02de006e 00576f66 6e616d72 43746547 00644973 41656d69 45746547 676e6972 6f726976 04bb0057 7865746e 6e754670 745204c9 0000646e 70656378 6553055f 74706563 43746547 057e0073
00ea0000 74736f50 6e4500e8 642e3233 646e616d 67677562 72507349 72506572 45747361 45747361 65727275 6e450140 65440118 78450173 4d746547 00005778 73736572 576f5465 64745374 46657469 46656c75 50746547 6547025e 74696e49 65536c61 6e756f43 63697469 53746547 7551043f 6f436563 65727275 654702fb 6c694673 7269766e 00005773 6e656d6e 436c7452 04c20074 6f697463 7269566c 6e5505a0 6e6f6974 686e5574 466e6f69 65727275 6d726554
43
..BeginPaint.... EndPaint..r.Post QuitMessage...En dDialog.USER32.d ll....GetCommand LineW...IsDebugg erPresent...IsPr ocessorFeaturePr esent.p.GetLastE rror..%.SetLastE rror....GetCurre [email protected] codePointer...De codePointer.s.Ex itProcess...GetM oduleHandleExW.. ..GetProcAddress ....MultiByteToW ideChar...GetStd Handle....WriteF ile...GetModuleF ileNameW....GetP rocessHeap..^.Ge tFileType.o.Init ializeCriticalSe ctionAndSpinCoun t...DeleteCritic alSection...GetS tartupInfoW.?.Qu eryPerformanceCo unter.*.GetCurre ntProcessId...Ge tSystemTimeAsFil eTime.G.GetEnvir onmentStringsW.. ..FreeEnvironmen tStringsW...RtlC aptureContext... RtlLookupFunctio nEntry....RtlVir tualUnwind....Un handledException Filter.._.SetUnh andledExceptionF ilter.).GetCurre ntProcess.~.Term
We can also get offsets by using -i or -a options for !dh command: 0:000> !dh -i 00000001`40000000 _IMAGE_IMPORT_DESCRIPTOR 000000014000eaa4 KERNEL32.dll 0000000140009000 Import Address Table 000000014000EAE0 Import Name Table 0 time date stamp 0 Index of first forwarder reference
_IMAGE_IMPORT_DESCRIPTOR 000000014000eab8 USER32.dll 0000000140009200 Import Address Table 000000014000ECE0 Import Name Table 0 time date stamp 0 Index of first forwarder reference
11.
Close the log file:
0:000> .logclose Closing open log file C:\AWMA-Dumps\M1A.log
To avoid possible confusion and glitches, we recommend exiting WinDbg after each exercise.
44
Windows Debugging, Disassembling, Reversing Practical Foundations: Training Course
Dmitry Vostokov Software Diagnostics Services
2
Published by OpenTask, Republic of Ireland Copyright © 2009 by Dmitry Vostokov Copyright © 2015 by Software Diagnostics Services All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover and you must impose the same condition on any acquirer. OpenTask books are available through booksellers and distributors worldwide. For further information or comments send requests to: [email protected] Product and company names mentioned in this book may be trademarks of their owners. A CIP catalog record for this book is available from the British Library. ISBN-13: 978-1-908043-94-8 First printing, 2015 Revision 2.0
Contents 3
Summary of Contents Contents........................................................................................................................................................................................5 Preface to the New Edition ................................................................................................................................................ 15 Combined Preface from Previous Editions ................................................................................................................. 17 About the Author ................................................................................................................................................................... 19 Chapter x86.1: Memory, Registers, and Simple Arithmetic ................................................................................. 21 Chapter x86.2: Debug and Release Binaries ............................................................................................................... 35 Chapter x86.3: Number Representations .................................................................................................................... 50 Chapter x86.4: Pointers ...................................................................................................................................................... 57 Chapter x86.5: Bytes, Words, and Double Words .................................................................................................... 73 Chapter x86.6: Pointers to Memory ............................................................................................................................... 78 Chapter x86.7: Logical Instructions and EIP ........................................................................................................... 100 Chapter x86.8: Reconstructing a Program with Pointers .................................................................................. 108 Chapter x86.9: Memory and Stacks............................................................................................................................. 116 Chapter x86.10: Frame Pointer and Local Variables ........................................................................................... 136 Chapter x86.11: Function Parameters ....................................................................................................................... 151 Chapter x86.12: More Instructions ............................................................................................................................. 165 Chapter x86.13: Function Pointer Parameters....................................................................................................... 176 Chapter x86.14: Summary of Code Disassembly Patterns ................................................................................ 182 Chapter x64.1: Memory, Registers, and Simple Arithmetic .............................................................................. 187 Chapter x64.2: Debug and Release Binaries ............................................................................................................ 202 Chapter x64.3: Number Representations ................................................................................................................. 217 Chapter x64.4: Pointers ................................................................................................................................................... 224 Chapter x64.5: Bytes, Words, and Double Words ................................................................................................. 242 Chapter x64.6: Pointers to Memory ............................................................................................................................ 248 Chapter x64.7: Logical Instructions and EIP ........................................................................................................... 271
4
Chapter x64.8: Reconstructing a Program with Pointers .................................................................................. 279 Chapter x64.9: Memory and Stacks............................................................................................................................. 288 Chapter x64.10: Local Variables ................................................................................................................................... 308 Chapter x64.11: Function Parameters ....................................................................................................................... 320 Chapter x64.12: More Instructions ............................................................................................................................. 330 Chapter x64.13: Function Pointer Parameters....................................................................................................... 341 Chapter x64.14: Summary of Code Disassembly Patterns ................................................................................ 345
Contents 5
Contents
Contents........................................................................................................................................................................................5 Preface to the New Edition ................................................................................................................................................ 15 Combined Preface from Previous Editions ................................................................................................................. 17 About the Author ................................................................................................................................................................... 19 Chapter x86.1: Memory, Registers, and Simple Arithmetic ................................................................................. 21 Memory and Registers inside an Idealized Computer ...................................................................................... 21 Memory and Registers inside Intel 32-bit PC ....................................................................................................... 22 “Arithmetic” Project: Memory Layout and Registers ........................................................................................ 23 “Arithmetic” Project: A Computer Program .......................................................................................................... 24 “Arithmetic” Project: Assigning Numbers to Memory Locations ................................................................. 25 Assigning Numbers to Registers ................................................................................................................................ 27 “Arithmetic” Project: Adding Numbers to Memory Cells................................................................................. 28 Incrementing/Decrementing Numbers in Memory and Registers .............................................................. 30 Multiplying Numbers ...................................................................................................................................................... 32 Multiplication and Registers ........................................................................................................................................ 34 Chapter x86.2: Debug and Release Binaries ............................................................................................................... 35 “Arithmetic” Project: C/C++ Program...................................................................................................................... 35 Downloading and Configuring WinDbg Debugger ............................................................................................. 36 WinDbg Disassembly Output – Debug Executable ............................................................................................. 38 WinDbg Disassembly Output – Release Executable........................................................................................... 49 Chapter x86.3: Number Representations .................................................................................................................... 50 Numbers and Their Representations ....................................................................................................................... 50 Decimal Representation (Base Ten) ......................................................................................................................... 51 Ternary Representation (Base Three)..................................................................................................................... 52
6
Binary Representation (Base Two) .......................................................................................................................... 53 Hexadecimal Representation (Base Sixteen) ........................................................................................................ 54 Why Hexadecimals are used? ...................................................................................................................................... 55 Chapter x86.4: Pointers ...................................................................................................................................................... 57 A Definition ......................................................................................................................................................................... 57 “Pointers” Project: Memory Layout and Registers ............................................................................................. 58 “Pointers” Project: Calculations.................................................................................................................................. 59 Using Pointers to Assign Numbers to Memory Cells ......................................................................................... 60 Adding Numbers Using Pointers ................................................................................................................................ 66 Multiplying Numbers Using Pointers ....................................................................................................................... 69 Chapter x86.5: Bytes, Words, and Double Words .................................................................................................... 73 Using Hexadecimal Numbers ...................................................................................................................................... 73 Byte Granularity................................................................................................................................................................ 74 Bit Granularity ................................................................................................................................................................... 75 Memory Layout ................................................................................................................................................................. 76 Chapter x86.6: Pointers to Memory ............................................................................................................................... 78 Pointers Revisited ............................................................................................................................................................ 78 Addressing Types ............................................................................................................................................................. 79 Registers Revisited .......................................................................................................................................................... 85 NULL Pointers.................................................................................................................................................................... 86 Invalid Pointers ................................................................................................................................................................. 87 Variables as Pointers ...................................................................................................................................................... 88 Pointer Initialization ....................................................................................................................................................... 89 Note: Initialized and Uninitialized Data .................................................................................................................. 90 More Pseudo Notation.................................................................................................................................................... 91 “MemoryPointers” Project: Memory Layout ......................................................................................................... 92
Contents 7
Chapter x86.7: Logical Instructions and EIP ........................................................................................................... 100 Instruction Format........................................................................................................................................................ 100 Logical Shift Instructions ........................................................................................................................................... 101 Logical Operations ........................................................................................................................................................ 102 Zeroing Memory or Registers................................................................................................................................... 103 Instruction Pointer ....................................................................................................................................................... 104 Note: Code Section ........................................................................................................................................................ 105 Chapter x86.8: Reconstructing a Program with Pointers .................................................................................. 108 Example of Disassembly Output: No Optimization ......................................................................................... 108 Reconstructing C/C++ Code: Part 1 ....................................................................................................................... 111 Reconstructing C/C++ Code: Part 2 ....................................................................................................................... 112 Reconstructing C/C++ Code: Part 3 ....................................................................................................................... 113 Reconstructing C/C++ Code: C/C++ program ................................................................................................... 114 Example of Disassembly Output: Optimized Program................................................................................... 115 Chapter x86.9: Memory and Stacks............................................................................................................................. 116 Stack: A Definition......................................................................................................................................................... 116 Stack Implementation in Memory .......................................................................................................................... 117 Things to Remember .................................................................................................................................................... 119 PUSH Instruction ........................................................................................................................................................... 120 POP instruction .............................................................................................................................................................. 121 Register Review ............................................................................................................................................................. 122 Application Memory Simplified ............................................................................................................................... 123 Stack Overflow................................................................................................................................................................ 124 Jumps .................................................................................................................................................................................. 126 Calls ..................................................................................................................................................................................... 128 Call Stack ........................................................................................................................................................................... 130
8
Exploring Stack in WinDbg ........................................................................................................................................ 132 Chapter x86.10: Frame Pointer and Local Variables ........................................................................................... 136 Stack Usage ...................................................................................................................................................................... 136 Register Review ............................................................................................................................................................. 137 Addressing Array Elements ...................................................................................................................................... 138 Stack Structure (No Function Parameters) ........................................................................................................ 139 Raw Stack (No Local Variables and Function Parameters) ......................................................................... 140 Function Prolog .............................................................................................................................................................. 141 Function Epilog .............................................................................................................................................................. 142 “Local Variables” Project ............................................................................................................................................ 143 Disassembly of Optimized Executable (Release Configuration) ................................................................ 148 Advanced Topic: FPO ................................................................................................................................................... 149 Chapter x86.11: Function Parameters ....................................................................................................................... 151 “FunctionParameters” Project ................................................................................................................................. 151 Stack Structure ............................................................................................................................................................... 152 Stack Structure with FPO ........................................................................................................................................... 154 Function Prolog and Epilog ....................................................................................................................................... 156 Project Disassembled Code with Comments...................................................................................................... 157 Release Build with FPO Enabled ............................................................................................................................. 162 Cdecl Calling Convention............................................................................................................................................ 163 Parameter Mismatch Problem ................................................................................................................................. 164 Chapter x86.12: More Instructions ............................................................................................................................. 165 CPU Flags Register ........................................................................................................................................................ 165 The Fastest Way to Fill Memory.............................................................................................................................. 166 Testing for 0..................................................................................................................................................................... 168 TEST - Logical Compare .............................................................................................................................................. 169
Contents 9
CMP – Compare Two Operands ............................................................................................................................... 170 TEST or CMP? .................................................................................................................................................................. 171 Conditional Jumps ......................................................................................................................................................... 172 The Structure of Registers ......................................................................................................................................... 173 Function Return Value ................................................................................................................................................ 174 Using Byte Registers .................................................................................................................................................... 175 Chapter x86.13: Function Pointer Parameters....................................................................................................... 176 “FunctionPointerParameters” Project .................................................................................................................. 176 Commented Disassembly ........................................................................................................................................... 177 Dynamic Addressing of Local Variables ............................................................................................................... 180 Chapter x86.14: Summary of Code Disassembly Patterns ................................................................................ 182 Function Prolog / Epilog ............................................................................................................................................ 182 Passing Parameters ...................................................................................................................................................... 183 LEA (Load Effective Address) .................................................................................................................................. 184 Accessing Parameters and Local Variables ........................................................................................................ 185 Chapter x64.1: Memory, Registers, and Simple Arithmetic .............................................................................. 187 Memory and Registers inside an Idealized Computer ................................................................................... 187 Memory and Registers inside Intel 64-bit PC .................................................................................................... 188 “Arithmetic” Project: Memory Layout and Registers ..................................................................................... 189 “Arithmetic” Project: A Computer Program ....................................................................................................... 190 “Arithmetic” Project: Assigning Numbers to Memory Locations .............................................................. 191 Assigning Numbers to Registers ............................................................................................................................. 193 “Arithmetic” Project: Adding Numbers to Memory Cells.............................................................................. 194 Incrementing/Decrementing Numbers in Memory and Registers ........................................................... 197 Multiplying Numbers ................................................................................................................................................... 200 Chapter x64.2: Debug and Release Binaries ............................................................................................................ 202
10
“Arithmetic” Project: C/C++ Program................................................................................................................... 202 Downloading and Configuring WinDbg Debugger .......................................................................................... 203 WinDbg Disassembly Output – Debug Executable .......................................................................................... 205 WinDbg Disassembly Output – Release Executable........................................................................................ 216 Chapter x64.3: Number Representations ................................................................................................................. 217 Numbers and Their Representations .................................................................................................................... 217 Decimal Representation (Base Ten) ...................................................................................................................... 218 Ternary Representation (Base Three).................................................................................................................. 219 Binary Representation (Base Two) ....................................................................................................................... 220 Hexadecimal Representation (Base Sixteen) ..................................................................................................... 221 Why Hexadecimals are used? ................................................................................................................................... 222 Chapter x64.4: Pointers ................................................................................................................................................... 224 A Definition ...................................................................................................................................................................... 224 “Pointers” Project: Memory Layout and Registers .......................................................................................... 225 “Pointers” Project: Calculations............................................................................................................................... 226 Using Pointers to Assign Numbers to Memory Cells ...................................................................................... 227 Adding Numbers Using Pointers ............................................................................................................................. 234 Multiplying Numbers Using Pointers .................................................................................................................... 238 Chapter x64.5: Bytes, Words, and Double Words ................................................................................................. 242 Using Hexadecimal Numbers ................................................................................................................................... 242 Byte Granularity............................................................................................................................................................. 243 Bit Granularity ................................................................................................................................................................ 244 Memory Layout .............................................................................................................................................................. 246 Chapter x64.6: Pointers to Memory ............................................................................................................................ 248 Pointers Revisited ......................................................................................................................................................... 248 Addressing Types .......................................................................................................................................................... 249
Contents 11
Registers Revisited ....................................................................................................................................................... 255 NULL Pointers................................................................................................................................................................. 256 Invalid Pointers .............................................................................................................................................................. 257 Variables as Pointers ................................................................................................................................................... 258 Pointer Initialization .................................................................................................................................................... 259 Note: Initialized and Uninitialized Data ............................................................................................................... 260 More Pseudo Notation................................................................................................................................................. 261 “MemoryPointers” Project: Memory Layout ...................................................................................................... 262 Chapter x64.7: Logical Instructions and EIP ........................................................................................................... 271 Instruction Format........................................................................................................................................................ 271 Logical Shift Instructions ........................................................................................................................................... 272 Logical Operations ........................................................................................................................................................ 273 Zeroing Memory or Registers................................................................................................................................... 274 Instruction Pointer ....................................................................................................................................................... 275 Note: Code Section ........................................................................................................................................................ 277 Chapter x64.8: Reconstructing a Program with Pointers .................................................................................. 279 Example of Disassembly Output: No Optimization ......................................................................................... 279 Reconstructing C/C++ Code: Part 1 ....................................................................................................................... 282 Reconstructing C/C++ Code: Part 2 ....................................................................................................................... 284 Reconstructing C/C++ Code: Part 3 ....................................................................................................................... 285 Reconstructing C/C++ Code: C/C++ program ................................................................................................... 286 Example of Disassembly Output: Optimized Program................................................................................... 287 Chapter x64.9: Memory and Stacks............................................................................................................................. 288 Stack: A Definition......................................................................................................................................................... 288 Stack Implementation in Memory .......................................................................................................................... 289 Things to Remember .................................................................................................................................................... 291
12
PUSH Instruction ........................................................................................................................................................... 292 POP instruction .............................................................................................................................................................. 293 Register Review ............................................................................................................................................................. 294 Application Memory Simplified ............................................................................................................................... 295 Stack Overflow................................................................................................................................................................ 296 Jumps .................................................................................................................................................................................. 298 Calls ..................................................................................................................................................................................... 300 Call Stack ........................................................................................................................................................................... 302 Exploring Stack in WinDbg ........................................................................................................................................ 304 Chapter x64.10: Local Variables ................................................................................................................................... 308 Stack Usage ...................................................................................................................................................................... 308 Addressing Array Elements ...................................................................................................................................... 309 Stack Structure (No Function Parameters) ........................................................................................................ 310 Function Prolog .............................................................................................................................................................. 311 Function Epilog .............................................................................................................................................................. 312 “Local Variables” Project ............................................................................................................................................ 313 Disassembly of Optimized Executable (Release Configuration) ................................................................ 319 Chapter x64.11: Function Parameters ....................................................................................................................... 320 “FunctionParameters” Project ................................................................................................................................. 320 Stack Structure ............................................................................................................................................................... 321 Function Prolog and Epilog ....................................................................................................................................... 323 Project Disassembled Code with Comments...................................................................................................... 325 Parameter Mismatch Problem ................................................................................................................................. 329 Chapter x64.12: More Instructions ............................................................................................................................. 330 CPU Flags Register ........................................................................................................................................................ 330 The Fastest Way to Fill Memory.............................................................................................................................. 331
Contents 13
Testing for 0..................................................................................................................................................................... 333 TEST - Logical Compare .............................................................................................................................................. 334 CMP – Compare Two Operands ............................................................................................................................... 335 TEST or CMP? .................................................................................................................................................................. 336 Conditional Jumps ......................................................................................................................................................... 337 The Structure of Registers ......................................................................................................................................... 338 Function Return Value ................................................................................................................................................ 339 Using Byte Registers .................................................................................................................................................... 340 Chapter x64.13: Function Pointer Parameters....................................................................................................... 341 “FunctionPointerParameters” Project .................................................................................................................. 341 Commented Disassembly ........................................................................................................................................... 342 Chapter x64.14: Summary of Code Disassembly Patterns ................................................................................ 345 Function Prolog / Epilog ............................................................................................................................................ 345 Parameters and Local Variables .............................................................................................................................. 347 LEA (Load Effective Address) .................................................................................................................................. 349 Accessing Parameters and Local Variables ........................................................................................................ 350
Published by OpenTask, Republic of Ireland Copyright © 2013 by OpenTask Copyright © 2013 by Software Diagnostics Services Copyright © 2013 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalogue record for this book is available from the British Library. ISBN-l3: 978-1-908043-67-2 (Paperback)
2
Contents Presentation Slides and Transcript ................................................................................................................................... 5 Practice Exercises ........................................................................................................................................................... 29 Exercise 0 .................................................................................................................................................................... 34 Exercise R1 .................................................................................................................................................................. 41 Exercise R2 .................................................................................................................................................................. 56 Exercise R3 .................................................................................................................................................................. 73 Exercise R4 .................................................................................................................................................................. 83 Exercise R5 .................................................................................................................................................................. 90 Exercise R6 ................................................................................................................................................................ 101 Memory Cell Diagrams ................................................................................................................................................. 127 MCD-R1..................................................................................................................................................................... 129 MCD-R2..................................................................................................................................................................... 131 MCD-R3..................................................................................................................................................................... 134 MCD-R5..................................................................................................................................................................... 138 MCD-R6..................................................................................................................................................................... 144 Source Code .................................................................................................................................................................. 147 DataTypes.cpp .......................................................................................................................................................... 149 Separate.cpp ............................................................................................................................................................. 154 CPPx64.cpp ............................................................................................................................................................... 155 Selected Q&A................................................................................................................................................................ 161
3
Exercise R1 Goal: Review x64 assembly fundamentals; learn how to reconstruct stack trace manually. ADDR Patterns: Universal Pointer, Symbolic Pointer S2, Interpreted Pointer S3, Context Pyramid Memory Cell Diagrams: Register, Pointer, Stack Frame 1.
Launch WinDbg from Windows Kits \ Debugging Tools for Windows (X64)
2.
Choose File \ Open Crash Dump… menu option and load \ADDR\MemoryDumps\notepad.dmp.
3.
You get the following output:
Microsoft (R) Windows Debugger Version 6.3.9600.16384 AMD64 Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\ADDR\MemoryDumps\notepad.dmp] User Mini Dump File with Full Memory: Only application data is available Symbol search path is: *** Invalid *** **************************************************************************** * Symbol loading may be unreliable without a symbol search path. * * Use .symfix to have the debugger choose a symbol path. * * After setting your symbol path, use .reload to refresh symbol locations. * **************************************************************************** Executable search path is: Windows 7 Version 7601 (Service Pack 1) MP (4 procs) Free x64 Product: WinNt, suite: SingleUserTS Personal Machine Name: Debug session time: Wed Oct 9 20:25:46.000 2013 (UTC + 0:00) System Uptime: 2 days 23:35:31.218 Process Uptime: 0 days 0:00:53.000 ............................ *** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll ************* Symbol Loading Error Summary ************** Module name Error ntdll The system cannot find the file specified You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded. You should also verify that your symbol search path (.sympath) is correct. *** ERROR: Symbol file could not be found. Defaulted to export symbols for user32.dll user32!SfmDxSetSwapChainStats+0x1a: 00000000`77619e6a c3 ret
4.
Set up a link to Microsoft symbol server and reload symbol files:
0:000> .symfix c:\mss 0:000> .reload ............................
41
5.
We get this stack trace:
0:000> k Child-SP 00000000`000efdc8 00000000`000efdd0 00000000`000efe00 00000000`000efe80 00000000`000eff40 00000000`000eff70
6.
RetAddr 00000000`77619e9e 00000000`ff131064 00000000`ff13133c 00000000`7771652d 00000000`7784c541 00000000`00000000
Call Site user32!ZwUserGetMessage+0xa user32!GetMessageW+0x34 notepad!WinMain+0x182 notepad!DisplayNonGenuineDlgWorker+0x2da kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d
Let’s check the main CPU registers:
0:000> r rax=00000000000206c0 rbx=00000000000efe40 rcx=000000000d0111c6 rdx=000000000000003b rsi=0000000000000001 rdi=0000000000000000 rip=0000000077619e6a rsp=00000000000efdc8 rbp=00000000ff130000 r8=0000000000000000 r9=ffffffffffffffff r10=0000000000000000 r11=0000000004f424c8 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl zr na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b user32!ZwUserGetMessage+0xa: 00000000`77619e6a c3 ret
efl=00000246
Note: The register parts and naming are illustrated in MCD-R1.xlsx A section. 7. The current instruction registers (registers that are used and affected by the current instruction or semantically tied to it) can be checked by r. command: 0:000> r. At return instr, rax = 206c0
8.
Any register value or its named parts can be checked with ? command:
0:000> ? r11 Evaluate expression: 83109064 = 00000000`04f424c8 0:000> ? r11d Evaluate expression: 83109064 = 00000000`04f424c8 0:000> ? r11w Evaluate expression: 9416 = 00000000`000024c8 0:000> ? r11b Evaluate expression: 200 = 00000000`000000c8
9. Individual parts can also be interpreted using typed r command (here we format them as signed values, see WinDbg help for all other format types): 0:000> r r9 r9=ffffffffffffffff 0:000> r r9:iq r9=-1
42
0:000> r r9:id r9=-1 -1 0:000> r r9:iw r9=65535 65535 65535 65535 0:000> r r9:ib r9=255 255 255 255 255 255 255 255
10. Any registry value can be interpreted as a pointer to memory cells, a memory address (Universal Pointer pattern vs. a pointer that was originally designed to be such). However, memory contents at that address may be inaccessible or unknown as in the case of RCX and RDI below. 0:000> dp rcx 00000000`0d0111c6 00000000`0d0111d6 00000000`0d0111e6 00000000`0d0111f6 00000000`0d011206 00000000`0d011216 00000000`0d011226 00000000`0d011236
????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`????????
????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`????????
Note: The following output for R11 is illustrated in MCD-R1.xlsx B section. 0:000> dp r11 00000000`04f424c8 00000000`04f424d8 00000000`04f424e8 00000000`04f424f8 00000000`04f42508 00000000`04f42518 00000000`04f42528 00000000`04f42538
80000710`00020002 00000000`ff130000 fffff900`c06f2760 fffff900`c06b3ef0 00000000`00000000 000002b9`0000054a 000002b7`00000537 fffff900`c06f23d0
50200104`00000a00 00000000`00000000 00000000`00000000 00000000`00000000 000000a3`000000ea 000000a5`000000ec 000007fe`fc00975c 00000000`00000000
0:000> dp rax 00000000`000206c0 00000000`000206d0 00000000`000206e0 00000000`000206f0 00000000`00020700 00000000`00020710 00000000`00020720 00000000`00020730
00260002`00000000 0009002e`002e002e 00000000`004f002b 00430009`00650076 00040000`00000053 00730041`00260020 00000000`00000000 00650053`00200065
006e0065`0070004f 006c0072`00740043 00610053`00260003 002b006c`00720074 00650076`00610053 0000002e`002e002e 00670061`00500005 00700075`00260074
0:000> dp rbx 00000000`000efe40 00000000`000efe50 00000000`000efe60 00000000`000efe70 00000000`000efe80 00000000`000efe90 00000000`000efea0 00000000`000efeb0
00000000`0005096e 00000000`00000001 000002f8`0f5c7a0f 00000000`ff13cab0 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
00000000`00000113 00000000`00000000 00000000`00000375 00000000`ff13133c 00000000`00000000 00000000`01985022 00000000`01985022 00000000`ff13cab0
43
0:000> dp rdi 00000000`00000000 00000000`00000010 00000000`00000020 00000000`00000030 00000000`00000040 00000000`00000050 00000000`00000060 00000000`00000070
11.
????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`????????
????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`???????? ????????`????????
We can also specify a range or limit to just one value and use finer granularity for memory dumping:
0:000> dp rax L1 00000000`000206c0
00260002`00000000
Note: The similar output for R11 as below is illustrated in MCD-R1.xlsx C section. 0:000> dd rax 00000000`000206c0 00000000`000206d0 00000000`000206e0 00000000`000206f0 00000000`00020700 00000000`00020710 00000000`00020720 00000000`00020730
00000000 002e002e 004f002b 00650076 00000053 00260020 00000000 00200065
00260002 0009002e 00000000 00430009 00040000 00730041 00000000 00650053
0070004f 00740043 00260003 00720074 00610053 002e002e 00500005 00260074
006e0065 006c0072 00610053 002b006c 00650076 0000002e 00670061 00700075
Note: Visible 00xx00yy pattern in the output of dp command: UNICODE string fragments, an example of Regular Data memory analysis pattern. 0:000> dw rax 00000000`000206c0 00000000`000206d0 00000000`000206e0 00000000`000206f0 00000000`00020700 00000000`00020710 00000000`00020720 00000000`00020730
0000 002e 002b 0076 0053 0020 0000 0065
0:000> db rax 00000000`000206c0 00000000`000206d0 00000000`000206e0 00000000`000206f0 00000000`00020700 00000000`00020710 00000000`00020720 00000000`00020730
00 2e 2b 76 53 20 00 65
0000 002e 004f 0065 0000 0026 0000 0020
00 00 00 00 00 00 00 00
00 2e 4f 65 00 26 00 20
0002 002e 0000 0009 0000 0041 0000 0053
00 00 00 00 00 00 00 00
02 2e 00 09 00 41 00 53
0026 0009 0000 0043 0004 0073 0000 0065
00 00 00 00 00 00 00 00
004f 0043 0003 0074 0053 002e 0005 0074
26 09 00 43 04 73 00 65
0070 0074 0026 0072 0061 002e 0050 0026
00-4f 00-43 00-03 00-74 00-53 00-2e 00-05 00-74
00 00 00 00 00 00 00 00
0065 0072 0053 006c 0076 002e 0061 0075
70 74 26 72 61 2e 50 26
006e 006c 0061 002b 0065 0000 0067 0070
00 00 00 00 00 00 00 00
65 72 53 6c 76 2e 61 75
00 00 00 00 00 00 00 00
6e 6c 61 2b 65 00 67 70
00 00 00 00 00 00 00 00
......&.O.p.e.n. ........C.t.r.l. +.O.......&.S.a. v.e...C.t.r.l.+. S.......S.a.v.e. .&.A.s......... ..........P.a.g. e. .S.e.t.&.u.p.
Note: You may have noticed a slight delay when dumping memory pointed by registers. The faster equivalent approach is to use @ prefix, for example: @rax:
44
0:000> dp @rax 00000000`000206c0 00000000`000206d0 00000000`000206e0 00000000`000206f0 00000000`00020700 00000000`00020710 00000000`00020720 00000000`00020730
00260002`00000000 0009002e`002e002e 00000000`004f002b 00430009`00650076 00040000`00000053 00730041`00260020 00000000`00000000 00650053`00200065
006e0065`0070004f 006c0072`00740043 00610053`00260003 002b006c`00720074 00650076`00610053 0000002e`002e002e 00670061`00500005 00700075`00260074
12. Notice a difference between a value and its organization in memory stemmed from the little-endian organization of Intel x86-x64 platform (least significant parts are located at lower addresses): 0:000> dp @rbp L1 00000000`ff130000
00000003`00905a4d
0:000> dd @rbp L2 00000000`ff130000
00905a4d 00000003
Note: The similar double word output for R11 is illustrated in MCD-R1.xlsx C section. 0:000> dp @rbp L1 00000000`ff130000
00000003`00905a4d
0:000> dw @rbp L4 00000000`ff130000
5a4d 0090 0003 0000
0:000> dp @rbp L1 00000000`ff130000
00000003`00905a4d
0:000> db @rbp L8 00000000`ff130000
4d 5a 90 00 03 00 00 00
MZ......
13. Every value can be associated with a symbolic value from PDB symbols files or from the binary (exported symbols) if available. We call this Symbolic Pointer or S2: 0:000> dps r11 00000000`04f424c8 00000000`04f424d0 00000000`04f424d8 (notepad+0x0) 00000000`04f424e0 00000000`04f424e8 00000000`04f424f0 00000000`04f424f8 00000000`04f42500 00000000`04f42508 00000000`04f42510 00000000`04f42518 00000000`04f42520 00000000`04f42528 00000000`04f42530 00000000`04f42538 00000000`04f42540
80000710`00020002 50200104`00000a00 00000000`ff130000 notepad!CFileDialogEvents_QueryInterface 00000000`00000000 fffff900`c06f2760 00000000`00000000 fffff900`c06b3ef0 00000000`00000000 00000000`00000000 000000a3`000000ea 000002b9`0000054a 000000a5`000000ec 000002b7`00000537 000007fe`fc00975c comctl32!Edit_WndProc fffff900`c06f23d0 00000000`00000000
0:000> ln 000007fe`fc00975c (000007fe`fc00975c) comctl32!Edit_WndProc comctl32!Edit_CalcChangeBlocks Exact matches:
|
45
(000007fe`fc00a650)
comctl32!Edit_WndProc () 0:000> dt 000007fe`fc00975c Edit_WndProc Symbol not found.
Note: The address 00000000`04f42530 that points to 000007fe`fc00975c doesn’t have an associated symbol: 0:000> dt 00000000`04f42530 Symbol not found at address 0000000004f42530.
Note: The next instruction pointer address contained in RIP should have an associated symbol of the current function in our example, because we have symbols for user32.dll: 0:000> ? @rip Evaluate expression: 2002886250 = 00000000`77619e6a 0:000> dt @rip ZwUserGetMessage Symbol not found. 0:000> r rax=00000000000206c0 rbx=00000000000efe40 rcx=000000000d0111c6 rdx=000000000000003b rsi=0000000000000001 rdi=0000000000000000 rip=0000000077619e6a rsp=00000000000efdc8 rbp=00000000ff130000 r8=0000000000000000 r9=ffffffffffffffff r10=0000000000000000 r11=0000000004f424c8 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl zr na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b user32!ZwUserGetMessage+0xa: 00000000`77619e6a c3 ret
efl=00000246
14. Now we come to the next pointer level after its value and its symbol: its interpretation. We call it an Interpreted Pointer, S3. Such interpretation is implemented either via typed structures (dt command) or via various WinDbg extension commands (! Commands) that format information for us. In our example we would like to check memory pointed to by the value of RBX register. We suspect it might be MSG structure related to get message loop: typedef struct tagMSG { HWND hwnd; UINT message; WPARAM wParam; LPARAM lParam; DWORD time; POINT pt; } MSG; 0:000> dp @rbx 00000000`000efe40 00000000`000efe50 00000000`000efe60 00000000`000efe70 00000000`000efe80 00000000`000efe90 00000000`000efea0 00000000`000efeb0
00000000`0005096e 00000000`00000001 000002f8`0f5c7a0f 00000000`ff13cab0 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000
00000000`00000113 00000000`00000000 00000000`00000375 00000000`ff13133c 00000000`00000000 00000000`01985022 00000000`01985022 00000000`ff13cab0
46
Note: The raw structure makes sense for WM_TIMER message (0x113) where wParam is a time ID (1) and usually a callback function (lParam) is NULL (0x0). Also mouse pointer data makes sense. Unfortunately, MSG structure is not available in symbol files available for notepad memory dump. However, we can load a different unrelated module with better symbol files, for example, CPUx64.exe from C:\ADDR\MemoryDumps\ExtraSymbols which was compiled as Windows application with full symbols and so should have structures necessary for thread message loop processing. 15.
We add an additional symbol file path:
0:000> .sympath+ C:\ADDR\MemoryDumps\ExtraSymbols Symbol search path is: srv*;C:\ADDR\MemoryDumps\ExtraSymbols Expanded Symbol search path is: SRV*c:\mss*http://msdl.microsoft.com/download/symbols;c:\addr\memorydumps\extrasymbols
We need to find an address to “load” CPUx64 module with its symbols. We choose a committed address 02000000 from the output of !address command: 0:000> !address Mapping Mapping Mapping Mapping Mapping Mapping Mapping Mapping Mapping
file section regions... module regions... PEB regions... TEB and stack regions... heap regions... page heap regions... other regions... stack trace database regions... activation context regions...
BaseAddress EndAddress+1 RegionSize Type State Protect Usage ----------------------------------------------------------------------------------------------------------------------[…] 0`01ffe000 0`01fff000 0`00001000 MEM_PRIVATE MEM_RESERVE PageHeap [PageHeap: 18f1000; NormalHeap: 2920000] 0`01fff000 0`02000000 0`00001000 MEM_PRIVATE MEM_COMMIT PAGE_NOACCESS PageHeap [PageHeap: 18f1000; NormalHeap: 2920000] 0`02000000 0`02001000 0`00001000 MEM_PRIVATE MEM_RESERVE PageHeap [PageHeap: 18f1000; NormalHeap: 2920000] 0`02001000 0`02002000 0`00001000 MEM_PRIVATE MEM_COMMIT PAGE_NOACCESS PageHeap [PageHeap: 18f1000; NormalHeap: 2920000] 0`02002000 0`02003000 0`00001000 MEM_PRIVATE MEM_RESERVE PageHeap [PageHeap: 18f1000; NormalHeap: 2920000] 0`02003000 0`02004000 0`00001000 MEM_PRIVATE MEM_COMMIT PAGE_NOACCESS PageHeap [PageHeap: 18f1000; NormalHeap: 2920000] […] 0:000> .reload /f C:\ADDR\MemoryDumps\ExtraSymbols\CPUx64=02000000 0:000> lm m CPU* start end 00000000`02000000 00000000`02000000
module name CPUx64 (private pdb symbols)
47
c:\addr\memorydumps\extrasymbols\CPUx64.pdb
16.
Now we are able to use MSG structure:
0:000> dt MSG CPUx64!MSG +0x000 hwnd +0x008 message +0x010 wParam +0x018 lParam +0x020 time +0x024 pt
: : : : : :
0:000> dt -r MSG CPUx64!MSG +0x000 hwnd +0x000 unused +0x008 message +0x010 wParam +0x018 lParam +0x020 time +0x024 pt +0x000 x +0x004 y
: Ptr64 HWND__ : Int4B : Uint4B : Uint8B : Int8B : Uint4B : tagPOINT : Int4B : Int4B
0:000> dt -r MSG @rbx CPUx64!MSG +0x000 hwnd +0x000 unused +0x008 message +0x010 wParam +0x018 lParam +0x020 time +0x024 pt +0x000 x +0x004 y
: 0x00000000`0005096e HWND__ : 0n0 : 0x113 : 1 : 0n0 : 0xf5c7a0f : tagPOINT : 0n760 : 0n885
Ptr64 HWND__ Uint4B Uint8B Int8B Uint4B tagPOINT
17. When we have an exception such as a breakpoint or access violation the values of the thread CPU registers are saved in the so called exception context structure and valid for the currently executing function and its next instruction pointed to by RIP register (the topmost frame). In other situations such as a manual memory dump we can only be sure about some registers such as RIP and RSP: 0:000> k Child-SP 00000000`000efdc8 00000000`000efdd0 00000000`000efe00 00000000`000efe80 00000000`000eff40 00000000`000eff70
RetAddr 00000000`77619e9e 00000000`ff131064 00000000`ff13133c 00000000`7771652d 00000000`7784c541 00000000`00000000
Call Site user32!ZwUserGetMessage+0xa user32!GetMessageW+0x34 notepad!WinMain+0x182 notepad!DisplayNonGenuineDlgWorker+0x2da kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d
48
0:000> r rax=00000000000206c0 rbx=00000000000efe40 rcx=000000000d0111c6 rdx=000000000000003b rsi=0000000000000001 rdi=0000000000000000 rip=0000000077619e6a rsp=00000000000efdc8 rbp=00000000ff130000 r8=0000000000000000 r9=ffffffffffffffff r10=0000000000000000 r11=0000000004f424c8 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl zr na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b user32!ZwUserGetMessage+0xa: 00000000`77619e6a c3 ret
efl=00000246
18. In any situation when we move down to the next frame, for example, to GetMessageW+0x34 (which points to the next instruction after ZwUserGetMessage was called), we don’t have its CPU registers values saved previously (r command gives values only for the topmost frame 0): 0:000> k Child-SP 00000000`000efdc8 00000000`000efdd0 00000000`000efe00 00000000`000efe80 00000000`000eff40 00000000`000eff70
RetAddr 00000000`77619e9e 00000000`ff131064 00000000`ff13133c 00000000`7771652d 00000000`7784c541 00000000`00000000
Call Site user32!ZwUserGetMessage+0xa user32!GetMessageW+0x34 notepad!WinMain+0x182 notepad!DisplayNonGenuineDlgWorker+0x2da kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d
0:000> ub 00000000`77619e9e user32!GetMessageW+0xc: 00000000`77619e80 b90000feff 00000000`77619e85 410bc1 00000000`77619e88 458bd1 00000000`77619e8b 85c1 00000000`77619e8d 0f85968d0100 00000000`77619e93 458bca 00000000`77619e96 488bcb 00000000`77619e99 e8c2ffffff
mov or mov test jne mov mov call
ecx,0FFFE0000h eax,r9d r10d,r9d ecx,eax user32!GetMessageW+0x1b (00000000`77632c29) r9d,r10d rcx,rbx user32!ZwUserGetMessage (00000000`77619e60)
0:000> u 00000000`77619e9e user32!GetMessageW+0x34: 00000000`77619e9e 817b0802010000 00000000`77619ea5 448bd0 00000000`77619ea8 0f844e480000 00000000`77619eae 817b08cc000000 00000000`77619eb5 0f8441480000 00000000`77619ebb 418bc2 00000000`77619ebe 4883c420 00000000`77619ec2 5b
cmp mov je cmp je mov add pop
dword ptr [rbx+8],102h r10d,eax user32!GetMessageW+0x49 (00000000`7761e6fc) dword ptr [rbx+8],0CCh user32!GetMessageW+0x49 (00000000`7761e6fc) eax,r10d rsp,20h rbx
0:000> kn # Child-SP 00 00000000`000efdc8 01 00000000`000efdd0 02 00000000`000efe00 03 00000000`000efe80 04 00000000`000eff40 05 00000000`000eff70
RetAddr 00000000`77619e9e 00000000`ff131064 00000000`ff13133c 00000000`7771652d 00000000`7784c541 00000000`00000000
Call Site user32!ZwUserGetMessage+0xa user32!GetMessageW+0x34 notepad!WinMain+0x182 notepad!DisplayNonGenuineDlgWorker+0x2da kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d
0:000> .frame 1 01 00000000`000efdd0 00000000`ff131064 user32!GetMessageW+0x34
49
0:000> r rax=00000000000206c0 rbx=00000000000efe40 rcx=000000000d0111c6 rdx=000000000000003b rsi=0000000000000001 rdi=0000000000000000 rip=0000000077619e6a rsp=00000000000efdc8 rbp=00000000ff130000 r8=0000000000000000 r9=ffffffffffffffff r10=0000000000000000 r11=0000000004f424c8 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl zr na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b user32!ZwUserGetMessage+0xa: 00000000`77619e6a c3 ret
efl=00000246
19. But some CPU registers can be recovered such as RIP (saved address when using call instruction) and RSP (the stack pointer value that was before saving that RIP address). Other register values can be recovered too if they were not used in called frames or were saved in temporary memory cells (such as on stack). Let’s recover some registers for the first few frames. 0:000> r rax=00000000000206c0 rbx=00000000000efe40 rcx=000000000d0111c6 rdx=000000000000003b rsi=0000000000000001 rdi=0000000000000000 rip=0000000077619e6a rsp=00000000000efdc8 rbp=00000000ff130000 r8=0000000000000000 r9=ffffffffffffffff r10=0000000000000000 r11=0000000004f424c8 r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000 iopl=0 nv up ei pl zr na po nc cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b user32!ZwUserGetMessage+0xa: 00000000`77619e6a c3 ret
efl=00000246
Let’s disassemble the current function: 0:000> uf user32!ZwUserGetMessage user32!ZwUserGetMessage: 00000000`77619e60 4c8bd1 00000000`77619e63 b806100000 00000000`77619e68 0f05 00000000`77619e6a c3
mov r10,rcx mov eax,1006h syscall ret
It is a very short function we see it overwrites R10 and EAX. Note that EAX value also don’t correspond to what we see in the output of r command: 0:000> r @eax eax=206c0
We see that RSP is not used inside ZwUserGetMessage function and its value should point to the return address of the caller, GetMessageW function during execution of call instruction: 0:000> dp @rsp 00000000`000efdc8 00000000`000efdd8 00000000`000efde8 00000000`000efdf8 00000000`000efe08 00000000`000efe18 00000000`000efe28 00000000`000efe38
00000000`77619e9e 00000000`00000000 00000000`00000000 00000000`ff131064 00000000`01b20455 00000000`00000001 000007fe`00000000 00000000`00000000
00000000`00000000 00000000`00000000 00000000`01b20455 00000000`01950048 000007fe`ff552164 00000000`0000193c 00000000`00000000 00000000`0005096e
50
0:000> ub 00000000`77619e9e user32!GetMessageW+0xc: 00000000`77619e80 b90000feff 00000000`77619e85 410bc1 00000000`77619e88 458bd1 00000000`77619e8b 85c1 00000000`77619e8d 0f85968d0100 00000000`77619e93 458bca 00000000`77619e96 488bcb 00000000`77619e99 e8c2ffffff
mov or mov test jne mov mov call
ecx,0FFFE0000h eax,r9d r10d,r9d ecx,eax user32!GetMessageW+0x1b (00000000`77632c29) r9d,r10d rcx,rbx user32!ZwUserGetMessage (00000000`77619e60)
0:000> u 00000000`77619e9e user32!GetMessageW+0x34: 00000000`77619e9e 817b0802010000 00000000`77619ea5 448bd0 00000000`77619ea8 0f844e480000 00000000`77619eae 817b08cc000000 00000000`77619eb5 0f8441480000 00000000`77619ebb 418bc2 00000000`77619ebe 4883c420 00000000`77619ec2 5b
cmp mov je cmp je mov add pop
dword ptr [rbx+8],102h r10d,eax user32!GetMessageW+0x49 (00000000`7761e6fc) dword ptr [rbx+8],0CCh user32!GetMessageW+0x49 (00000000`7761e6fc) eax,r10d rsp,20h rbx
This is RIP value but RSP should be the value before call instruction was executed. When a return value is saved RSP is decremented by 8 so the value of RSP before call should be the value of RSP pointing to the saved return address + 8: 0:000> ? @rsp + 8 Evaluate expression: 982480 = 00000000`000efdd0 0:000> k Child-SP 00000000`000efdc8 00000000`000efdd0 00000000`000efe00 00000000`000efe80 00000000`000eff40 00000000`000eff70
RetAddr 00000000`77619e9e 00000000`ff131064 00000000`ff13133c 00000000`7771652d 00000000`7784c541 00000000`00000000
Call Site user32!ZwUserGetMessage+0xa user32!GetMessageW+0x34 notepad!WinMain+0x182 notepad!DisplayNonGenuineDlgWorker+0x2da kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d
Let’s now find out RIP and RSP for the next frame (the caller of GetMessageW function). To find out RSP we need see how it was used in the callee, GetMessageW function before the callee called ZwUserGetMessage. We disassemble GetMessageW function: 0:000> uf user32!GetMessageW user32!GetMessageW: 00000000`77619e74 fff3 00000000`77619e76 4883ec20 00000000`77619e7a 418bc0 00000000`77619e7d 488bd9 00000000`77619e80 b90000feff 00000000`77619e85 410bc1 00000000`77619e88 458bd1 00000000`77619e8b 85c1 00000000`77619e8d 0f85968d0100
push sub mov mov mov or mov test jne
rbx rsp,20h eax,r8d rbx,rcx ecx,0FFFE0000h eax,r9d r10d,r9d ecx,eax user32!GetMessageW+0x1b (00000000`77632c29)
user32!GetMessageW+0x29: 00000000`77619e93 458bca 00000000`77619e96 488bcb 00000000`77619e99 e8c2ffffff 00000000`77619e9e 817b0802010000
mov mov call cmp
r9d,r10d rcx,rbx user32!ZwUserGetMessage (00000000`77619e60) dword ptr [rbx+8],102h
51
00000000`77619ea5 448bd0 00000000`77619ea8 0f844e480000
mov je
r10d,eax user32!GetMessageW+0x49 (00000000`7761e6fc)
user32!GetMessageW+0x40: 00000000`77619eae 817b08cc000000 00000000`77619eb5 0f8441480000
cmp je
dword ptr [rbx+8],0CCh user32!GetMessageW+0x49 (00000000`7761e6fc)
user32!GetMessageW+0x51: 00000000`77619ebb 418bc2 00000000`77619ebe 4883c420 00000000`77619ec2 5b 00000000`77619ec3 c3
mov add pop ret
eax,r10d rsp,20h rbx
user32!GetMessageW+0x49: 00000000`7761e6fc 48816310ffff0000 and 00000000`7761e704 e9b2b7ffff jmp
qword ptr [rbx+10h],0FFFFh user32!GetMessageW+0x51 (00000000`77619ebb)
user32!GetMessageW+0x1b: 00000000`77632c29 4183f9ff 00000000`77632c2d 750d
cmp jne
r9d,0FFFFFFFFh user32!GetMessageW+0x5a (00000000`77632c3c)
user32!GetMessageW+0x21: 00000000`77632c2f 4485c1 00000000`77632c32 7508
test jne
ecx,r8d user32!GetMessageW+0x5a (00000000`77632c3c)
user32!GetMessageW+0x26: 00000000`77632c34 4533d2 00000000`77632c37 e95772feff
xor jmp
r10d,r10d user32!GetMessageW+0x29 (00000000`77619e93)
mov call
ecx,57h qword ptr [user32!_imp_RtlSetLastWin32Error
xor jmp
r10d,r10d user32!GetMessageW+0x51 (00000000`77619ebb)
user32!GetMessageW+0x5a: 00000000`77632c3c b957000000 00000000`77632c41 ff1561f60400 (00000000`776822a8)] 00000000`77632c47 4533d2 00000000`77632c4a e96c72feff
We see that stack pointer was decremented by 0x20 (sub instruction) and also by 8 (push instruction) and so we add these values to RSP we found out previously for ZwUserGetMessage call, 00000000`000efdd0: 0:000> dps 00000000`000efdd0 + 20 + 8 00000000`000efdf8 00000000`ff131064 notepad!WinMain+0x182 00000000`000efe00 00000000`01950048 00000000`000efe08 00000000`01b20455 00000000`000efe10 000007fe`ff552164 msctf!UIWndProc 00000000`000efe18 00000000`00000001 00000000`000efe20 00000000`0000193c 00000000`000efe28 000007fe`00000000 00000000`000efe30 00000000`00000000 00000000`000efe38 00000000`00000000 00000000`000efe40 00000000`0005096e 00000000`000efe48 00000000`00000113 00000000`000efe50 00000000`00000001 00000000`000efe58 00000000`00000000 00000000`000efe60 000002f8`0f5c7a0f 00000000`000efe68 00000000`00000375 00000000`000efe70 00000000`ff13cab0 notepad!_xi_z
52
We see that GetMessageW was called from WinMain function: 0:000> ub 00000000`ff131064 notepad!WinMain+0xf5: 00000000`ff131046 ff1544b40000 (00000000`ff13c490)] 00000000`ff13104c 488bd8 00000000`ff13104f eb00 00000000`ff131051 488d4c2440 00000000`ff131056 4533c9 00000000`ff131059 4533c0 00000000`ff13105c 33d2 00000000`ff13105e ff1524b40000 (00000000`ff13c488)]
call
qword ptr [notepad!_imp_SetWinEventHook
mov jmp lea xor xor xor call
rbx,rax notepad!WinMain+0x16f (00000000`ff131051) rcx,[rsp+40h] r9d,r9d r8d,r8d edx,edx qword ptr [notepad!_imp_GetMessageW
The value of RSP before call should be adjusted by 8 due to saved return address: 0:000> ? 00000000`000efdf8 + 8 Evaluate expression: 982528 = 00000000`000efe00 0:000> k Child-SP 00000000`000efdc8 00000000`000efdd0 00000000`000efe00 00000000`000efe80 00000000`000eff40 00000000`000eff70
RetAddr 00000000`77619e9e 00000000`ff131064 00000000`ff13133c 00000000`7771652d 00000000`7784c541 00000000`00000000
Call Site user32!ZwUserGetMessage+0xa user32!GetMessageW+0x34 notepad!WinMain+0x182 notepad!DisplayNonGenuineDlgWorker+0x2da kernel32!BaseThreadInitThunk+0xd ntdll!RtlUserThreadStart+0x1d
And so on we are able to reconstruct the stack trace like a debugger. Note that we are able to correctly disassemble functions using uf command because function boundaries are saved in PDB symbol files or the start of the function is available from image file as an exported function. If such information is not available we would most likely have a truncated stack trace. 20. Other registers and memory values are reused and overwritten when we move down the frames so less and less information can be recovered. We call this ADDR pattern (Inverse) Context Pyramid. 21. We also introduce special Stack Frame memory cell diagrams. The case of stack frame for GetMessageW function before calling ZwUserGetMessage is illustrated in MCD-R1.xlsx section D. 22.
To avoid possible confusion and glitches we recommend exiting WinDbg after each exercise.
53
MCD-R1 A. Main Registers RAX RAX
EAX
RAX
EAX
AX
RAX
EAX
AH | AL
RSI RSI
ESI
RSI
ESI
RSI
ESI
SI | SIL
R8 R8
R8D
R8
R8D
R8
R8D
R8W |R8B
129
B. Universal Pointer We use a similar color for the value it points to
R11
C. Pointing to a double word
R11
D. Stack Frame RSP 8 10 18 20 28 30 38 40 48 50
130
Published by OpenTask, Republic of Ireland Copyright © 2013 by OpenTask Copyright © 2013 by Software Diagnostics Services Copyright © 2013 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover, and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalogue record for this book is available from the British Library. ISBN-l3: 978-1-908043-42-9 (Paperback) Revision 2 (February 2016)
2
Contents Presentation Slides and Transcript .................................................................................................................................5 Practice Exercises .......................................................................................................................................................111 App Source Code ........................................................................................................................................................125
3
7
Published by OpenTask, Republic of Ireland Copyright © 2017 by OpenTask Copyright © 2017 by Software Diagnostics Services Copyright © 2017 by Dmitry Vostokov All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior written permission of the publisher. You must not circulate this book in any other binding or cover, and you must impose the same condition on any acquirer. Product and company names mentioned in this book may be trademarks of their owners. OpenTask books and magazines are available through booksellers and distributors worldwide. For further information or comments send requests to [email protected]. A CIP catalog record for this book is available from the British Library. ISBN-l3: 978-1-908043-84-9 (Paperback) Version 3, 2017 Revision 3.00 (June 2017)
2
Contents About the Author ...........................................................................................................................................................5 Presentation Slides and Transcript .................................................................................................................................7 Practice Exercises .........................................................................................................................................................13 Exercise 0: Download, setup and verify your WinDbg installation ............................................................................18 Exercise C1: Stack Trace Collection (64-bit) ..............................................................................................................25 Exercise C2: Memory Search (64-bit) ........................................................................................................................66 Exercise C3: Linked Lists (64-bit) ...............................................................................................................................80 Exercise C4A: WinDbg Built-in Scripting (64-bit) .....................................................................................................133 Exercise C4B: WinDbg JavaScript Scripting (64-bit) ................................................................................................151 Exercise C5: Registry (64-bit) ..................................................................................................................................167 Exercise C6: Module Variables (64-bit) ...................................................................................................................176 Exercise C7: System Objects (64-bit) ......................................................................................................................181 Exercise C8: Network (64-bit) .................................................................................................................................191 Exercise C9: Device Drivers (64-bit) ........................................................................................................................205 Exercise C10: Storage and File System (64-bit) .......................................................................................................221 Exercise C11: Window Messaging (64-bit) ..............................................................................................................226 Legacy Exercises .........................................................................................................................................................239 Exercise Legacy.0: Download, setup and verify your WinDbg installation ..............................................................241 Exercise Legacy.C1: Stack Trace Collection (64-bit) ................................................................................................246 Exercise Legacy.C2: Memory Search (64-bit) ..........................................................................................................271 Exercise Legacy.C3: Linked Lists (64-bit) .................................................................................................................282 Exercise Legacy.C4: Scripting (64-bit) .....................................................................................................................311 Exercise Legacy.C5: Registry (64-bit) ......................................................................................................................328 Exercise Legacy.C6: Module Variables (64-bit) .......................................................................................................336 Exercise Legacy.C7: System Objects (64-bit) ...........................................................................................................340 Exercise Legacy.C8: Network (64-bit) .....................................................................................................................346 Exercise Legacy.C9: Device Drivers (64-bit) ............................................................................................................354 Selected Q&A .............................................................................................................................................................365
3
Exercise C1: Stack Trace Collection (64-bit) Goal: Learn how to get stack traces related to sessions, processes, and threads; diagnose different thread types; get stack traces from WOW64 processes. Patterns: Stack Trace Collection (unmanaged space); Passive Thread; Coupled Processes (weak); Coupled Processes (strong); Wait Chain (ALPC); Virtualized Process; Truncated Stack Trace. 1.
Launch WinDbg from Windows Kits \ WinDbg (X64).
2.
Open \AdvMDA-Dumps\x64\MEMORY-Normal.DMP
3.
We get the dump file loaded:
Microsoft (R) Windows Debugger Version 10.0.15063.137 AMD64 Copyright (c) Microsoft Corporation. All rights reserved. Loading Dump File [F:\AdvWMDA-Dumps\x64\MEMORY-Normal.DMP] Kernel Bitmap Dump File: Full address space is available Symbol search path is: srv* Executable search path is: Windows 10 Kernel Version 10586 MP (4 procs) Free x64 Product: WinNt, suite: TerminalServer SingleUserTS Personal Built by: 10586.103.amd64fre.th2_release.160126-1819 Machine Name: Kernel base = 0xfffff801`4868a000 PsLoadedModuleList = 0xfffff801`48968cf0 Debug session time: Thu May 19 00:13:25.654 2016 (UTC + 1:00) System Uptime: 0 days 0:02:48.462 Loading Kernel Symbols ............................................................... ................................................................ ...................................... Loading User Symbols ........................................... Loading unloaded module list ............. ******************************************************************************* * * * Bugcheck Analysis * * * ******************************************************************************* Use !analyze -v to get detailed debugging information. BugCheck D1, {ffffc000dd71a800, 2, 0, fffff801c17a1385} *** ERROR: Module load completed but symbols could not be loaded for myfault.sys *** ERROR: Module load completed but symbols could not be loaded for NotMyfault.exe Probably caused by : myfault.sys ( myfault+1385 ) Followup: -----------
MachineOwner
Note: Probably caused by myfault.sys. We used NotMyFault tool from Windows Internals: http://technet.microsoft.com/en-us/sysinternals/bb963901 http://download.sysinternals.com/files/NotMyFault.zip 25
4.
We open a log file, set up symbols and reload them:
3: kd> .logopen F:\AdvWMDA-Dumps\x64\C1.log Opened log file 'F:\AdvWMDA-Dumps\x64\C1.log' 3: kd> .symfix c:\mss 3: kd> .reload Loading Kernel Symbols ............................................................... ................................................................ ...................................... Loading User Symbols ........................................... Loading unloaded module list .............
5.
We list running sessions:
3: kd> !session Sessions on machine: 2 Valid Sessions: 0 1 Current Session 1
6.
We check the current process:
3: kd> !process PROCESS ffffe000ec09a080 SessionId: 1 Cid: 1594 Peb: 00379000 ParentCid: 0c64 DirBase: 3cfce000 ObjectTable: ffffc000dd91c2c0 HandleCount: Image: NotMyfault.exe VadRoot ffffe000eb3fb4b0 Vads 92 Clone 0 Private 473. Modified 6. Locked 0. DeviceMap ffffc000db0667a0 Token ffffc000dd9d5a90 ElapsedTime 00:00:05.488 UserTime 00:00:00.000 KernelTime 00:00:00.000 QuotaPoolUsage[PagedPool] 224896 QuotaPoolUsage[NonPagedPool] 12632 Working Set Sizes (now,min,max) (3220, 50, 345) (12880KB, 200KB, 1380KB) PeakWorkingSetSize 3149 VirtualSize 115 Mb PeakVirtualSize 115 Mb PageFaultCount 3323 MemoryPriority FOREGROUND BasePriority 8 CommitCharge 539 Job ffffe000ec07ead0 THREAD ffffe000ecab7080 Cid 1594.08cc RUNNING on processor 3 THREAD ffffe000ec360080 Cid 1594.1538 (WrQueue) UserMode Alertable ffffe000ebdf0200 QueueObject
Teb: 000000000037a000 Win32Thread: ffffe000ebbfee30
THREAD ffffe000ec16e080 Cid 1594.1540 (WrQueue) UserMode Alertable ffffe000ebdf0200 QueueObject
Teb: 000000000037e000 Win32Thread: 0000000000000000 WAIT:
THREAD ffffe000ec97c840 Cid 1594.1544 (WrQueue) UserMode Alertable ffffe000ebdf0200 QueueObject
Teb: 0000000000380000 Win32Thread: 0000000000000000 WAIT:
Teb: 000000000037c000 Win32Thread: 0000000000000000 WAIT:
26
THREAD ffffe000ec41f040 Cid 1594.154c (WrQueue) UserMode Alertable ffffe000ec00bb40 QueueObject
Teb: 0000000000382000 Win32Thread: 0000000000000000 WAIT:
THREAD ffffe000ec43a080 Cid 1594.0614 (WrQueue) UserMode Alertable ffffe000ec00bb40 QueueObject
Teb: 0000000000384000 Win32Thread: 0000000000000000 WAIT:
THREAD ffffe000ec474080 Cid 1594.17b0 Teb: 0000000000386000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ec658730 SynchronizationTimer THREAD ffffe000ec475080 Cid 1594.17ac Teb: 0000000000388000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Alertable ffffe000ec08c9c0 SynchronizationEvent ffffe000ec675c50 SynchronizationTimer
7.
We set the current session 0 and examine its implicit process:
3: kd> !session -s 0 Sessions on machine: 2 Implicit process is now ffffe000`eb239080 Using session 0 3: kd> !process ffffe000`eb239080 3f PROCESS ffffe000eb239080 SessionId: 0 Cid: 0180 Peb: 61467f1000 ParentCid: 0174 DirBase: 04466000 ObjectTable: ffffc000daca8040 HandleCount: Image: csrss.exe VadRoot ffffe000eb14ac00 Vads 90 Clone 0 Private 216. Modified 444. Locked 0. DeviceMap ffffc000da21a760 Token ffffc000dacb0060 ElapsedTime 00:02:39.778 UserTime 00:00:00.000 KernelTime 00:00:00.093 QuotaPoolUsage[PagedPool] 149960 QuotaPoolUsage[NonPagedPool] 12696 Working Set Sizes (now,min,max) (314, 50, 345) (1256KB, 200KB, 1380KB) PeakWorkingSetSize 985 VirtualSize 2097199 Mb PeakVirtualSize 2097200 Mb PageFaultCount 2633 MemoryPriority BACKGROUND BasePriority 13 CommitCharge 323 PEB at 00000061467f1000 InheritedAddressSpace: No ReadImageFileExecOptions: No BeingDebugged: No ImageBaseAddress: 00007ff71e540000 Ldr 00007ff8ed365200 Ldr.Initialized: Yes Ldr.InInitializationOrderModuleList: 0000022132102ee0 . 000002213211b4e0 Ldr.InLoadOrderModuleList: 0000022132103050 . 000002213211b4c0 Ldr.InMemoryOrderModuleList: 0000022132103060 . 000002213211b4d0 Base TimeStamp Module 7ff71e540000 5632d16d Oct 30 02:09:49 2015 C:\Windows\system32\csrss.exe 7ff8ed220000 56a8483f Jan 27 04:31:59 2016 C:\Windows\SYSTEM32\ntdll.dll 7ff8e9820000 5632d16f Oct 30 02:09:51 2015 C:\Windows\system32\CSRSRV.dll 7ff8e9800000 5632d166 Oct 30 02:09:42 2015 C:\Windows\system32\basesrv.DLL 7ff8e97c0000 5632d722 Oct 30 02:34:10 2015 C:\Windows\system32\winsrv.DLL 7ff8eb3e0000 565423d2 Nov 24 08:46:10 2015 C:\Windows\system32\USER32.dll 7ff8e9c80000 56a8489c Jan 27 04:33:32 2016 C:\Windows\system32\kernelbase.dll 7ff8eb0b0000 5632d5aa Oct 30 02:27:54 2015 C:\Windows\system32\kernel32.dll 7ff8ed090000 568b2035 Jan 05 01:45:25 2016 C:\Windows\system32\GDI32.dll 7ff8e97b0000 5632d888 Oct 30 02:40:08 2015 C:\Windows\system32\sxssrv.DLL
27
7ff8e9670000 5632d5f0 Oct 30 02:29:04 2015 C:\Windows\system32\sxs.dll 7ff8ea890000 5632d515 Oct 30 02:25:25 2015 C:\Windows\system32\RPCRT4.dll 7ff8e9bb0000 5632d756 Oct 30 02:35:02 2015 C:\Windows\system32\bcryptPrimitives.dll SubSystemData: 0000000000000000 ProcessHeap: 00000221320e0000 ProcessParameters: 0000022132102550 CurrentDirectory: 'C:\Windows\system32\' WindowTitle: '< Name not readable >' ImageFile: 'C:\Windows\system32\csrss.exe' CommandLine: '%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16' DllPath: '< Name not readable >' Environment: 0000022132102080 ComSpec=C:\Windows\system32\cmd.exe NUMBER_OF_PROCESSORS=4 OS=Windows_NT Path=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\ PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=3a09 PSModulePath=%ProgramFiles%\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules SystemDrive=C: SystemRoot=C:\Windows TEMP=C:\Windows\TEMP TMP=C:\Windows\TEMP USERNAME=SYSTEM windir=C:\Windows THREAD ffffe000eb23f080 Cid 0180.0190 Teb: 00000061467f8000 Win32Thread: ffffe000eb75e260 WAIT: (WrLpcReceive) UserMode Non-Alertable ffffe000eb23f6b8 Semaphore Limit 0x1 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 10750 Ticks: 31 (0:00:00:00.484) Context Switch Count 467 IdealProcessor: 3 UserTime 00:00:00.015 KernelTime 00:00:00.000 Win32 Start Address CSRSRV!CsrApiRequestThread (0x00007ff8e9825380) Stack Init ffffd00024a14c90 Current ffffd00024a14410 Base ffffd00024a15000 Limit ffffd00024a0f000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 *** ERROR: Module load completed but symbols could not be loaded for myfault.sys Child-SP RetAddr Call Site ffffd000`24a14450 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`24a14590 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`24a14640 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`24a146d0 fffff801`487585e6 nt!KeWaitForSingleObject+0x375 ffffd000`24a14790 fffff801`48a7a4aa nt!AlpcpWaitForSingleObject+0x3e ffffd000`24a147d0 fffff801`48a77b32 nt!AlpcpReceiveMessagePort+0x45a ffffd000`24a14860 fffff801`48a776b3 nt!AlpcpReceiveMessage+0x322 ffffd000`24a149d0 fffff801`487d6ca3 nt!NtAlpcSendWaitReceivePort+0x103 ffffd000`24a14a90 00007ff8`ed2c6174 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`24a14b00) 00000061`465df7d8 00007ff8`e9825602 ntdll!NtAlpcSendWaitReceivePort+0x14 00000061`465df7e0 00007ff8`ed27c585 CSRSRV!CsrApiRequestThread+0x282 00000061`465dfc70 00000000`00000000 ntdll!RtlUserThreadStart+0x45
28
THREAD ffffe000eb74d080 Cid 0180.01b0 Teb: 00000061467fc000 Win32Thread: ffffe000ebf95c60 WAIT: (WrLpcReply) UserMode Non-Alertable ffffe000eb74d6b8 Semaphore Limit 0x1 Waiting for reply to ALPC Message ffffc000dae5fb30 : queued at port ffffe000eb73a800 : owned by process ffffe000eb83e840 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 2203 Ticks: 8578 (0:00:02:14.031) Context Switch Count 7 IdealProcessor: 1 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address winsrv!TerminalServerRequestThread (0x00007ff8e97c1320) Stack Init ffffd000250bcc90 Current ffffd000250bc3f0 Base ffffd000250bd000 Limit ffffd000250b7000 Call 0000000000000000 Priority 15 BasePriority 15 PriorityDecrement 0 IoPriority 2 PagePriority 5 Kernel stack not resident. THREAD ffffe000eb74e080 Cid 0180.01b4 Teb: 00000061467fe000 Win32Thread: ffffe000eb971090 WAIT: (UserRequest) UserMode Alertable ffffe000eb245c00 SynchronizationEvent ffffe000eb245d00 SynchronizationEvent ffffe000eb245c80 SynchronizationEvent ffffe000eb245b80 SynchronizationEvent Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 845 Ticks: 9936 (0:00:02:35.250) Context Switch Count 2 IdealProcessor: 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address winsrv!NotificationThread (0x00007ff8e97c2150) Stack Init ffffd0002531ac90 Current ffffd00025319f80 Base ffffd0002531b000 Limit ffffd00025315000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Kernel stack not resident. THREAD ffffe000eb752840 Cid 0180.01b8 Teb: 0000006146600000 Win32Thread: ffffe000eb76fa90 WAIT: (WrQueue) UserMode Alertable ffffe000eb23cac0 QueueObject Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 2196 Ticks: 8585 (0:00:02:14.140) Context Switch Count 40 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address ntdll!TppWorkerThread (0x00007ff8ed24b290) Stack Init ffffd000251a7c90 Current ffffd000251a73e0 Base ffffd000251a8000 Limit ffffd000251a2000 Call 0000000000000000 Priority 15 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Kernel stack not resident. THREAD ffffe000eb75a080 Cid 0180.01bc Teb: 0000006146602000 Win32Thread: 0000000000000000 WAIT: (WrLpcReceive) UserMode Non-Alertable ffffe000eb75a6b8 Semaphore Limit 0x1 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 570 Ticks: 10211 (0:00:02:39.546) Context Switch Count 3 IdealProcessor: 0 UserTime 00:00:00.000 KernelTime 00:00:00.000
29
Win32 Start Address CSRSRV!CsrSbApiRequestThread (0x00007ff8e9824ed0) Stack Init ffffd00025331c90 Current ffffd00025331490 Base ffffd00025332000 Limit ffffd0002532c000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Kernel stack not resident. THREAD ffffe000eb78a080 Cid 0180.01fc Teb: 0000006146604000 Win32Thread: ffffe000eb7df420 WAIT: (WrLpcReceive) UserMode Non-Alertable ffffe000eb78a6b8 Semaphore Limit 0x1 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 10750 Ticks: 31 (0:00:00:00.484) Context Switch Count 515 IdealProcessor: 1 UserTime 00:00:00.046 KernelTime 00:00:00.093 Win32 Start Address CSRSRV!CsrApiRequestThread (0x00007ff8e9825380) Stack Init ffffd00024ac6c90 Current ffffd00024ac6410 Base ffffd00024ac7000 Limit ffffd00024ac1000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`24ac6450 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`24ac6590 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`24ac6640 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`24ac66d0 fffff801`487585e6 nt!KeWaitForSingleObject+0x375 ffffd000`24ac6790 fffff801`48a7a4aa nt!AlpcpWaitForSingleObject+0x3e ffffd000`24ac67d0 fffff801`48a77b32 nt!AlpcpReceiveMessagePort+0x45a ffffd000`24ac6860 fffff801`48a776b3 nt!AlpcpReceiveMessage+0x322 ffffd000`24ac69d0 fffff801`487d6ca3 nt!NtAlpcSendWaitReceivePort+0x103 ffffd000`24ac6a90 00007ff8`ed2c6174 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`24ac6b00) 00000061`4697f5d8 00007ff8`e9825602 ntdll!NtAlpcSendWaitReceivePort+0x14 00000061`4697f5e0 00007ff8`ed27c585 CSRSRV!CsrApiRequestThread+0x282 00000061`4697fa70 00000000`00000000 ntdll!RtlUserThreadStart+0x45 THREAD ffffe000eb7cf080 Cid 0180.023c Teb: 0000006146606000 Win32Thread: ffffe000eb22a7c0 WAIT: (WrUserRequest) KernelMode Alertable ffffe000eb7b2610 SynchronizationEvent ffffe000eb7c5870 NotificationTimer ffffe000eb7b5af0 SynchronizationTimer fffff80148965dc0 NotificationEvent ffffe000eb737fe0 SynchronizationEvent ffffe000eb737f60 SynchronizationEvent ffffe000eb73cab0 SynchronizationEvent ffffe000eb737ba0 SynchronizationEvent ffffe000eb737aa0 SynchronizationEvent ffffe000eb737a20 SynchronizationEvent ffffe000eb7379a0 SynchronizationEvent ffffe000eb737800 SynchronizationTimer ffffe000eb737660 SynchronizationTimer ffffe000eb7375e0 SynchronizationEvent ffffe000eb737560 SynchronizationEvent ffffe000eb7374e0 SynchronizationEvent Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 10718 Ticks: 63 (0:00:00:00.984) Context Switch Count 17 IdealProcessor: 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address winsrv!StartCreateSystemThreads (0x00007ff8e97c5680) Stack Init ffffd00024f7ac90 Current ffffd00024f7a5e0 Base ffffd00024f7b000 Limit ffffd00024f75000 Call 0000000000000000 Priority 16 BasePriority 16 PriorityDecrement 0 IoPriority 2 PagePriority 5
30
Child-SP ffffd000`24f7a620 ffffd000`24f7a760 ffffd000`24f7a810 ffffd000`24f7a8a0 ffffd000`24f7a960 ffffd000`24f7aa90 ffffd000`24f7aad0 ffffd000`24f7ab00 ffffd000`24f7ab00) 00000061`469bfed8
RetAddr fffff801`487003ea fffff801`486ffe79 fffff801`48701a1e fffff961`7f61947a fffff961`7f9f3010 fffff961`7f62a83d fffff801`487d6ca3 00007ff8`e97c7274
Call Site nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForMultipleObjects+0x24e win32kfull!RawInputThread+0x9aa win32kbase!xxxCreateSystemThreads+0x70 win32kfull!NtUserCallNoParam+0x2d nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @
00000000`00000000 winsrv!NtUserCallNoParam+0x14
THREAD ffffe000eb7d0080 Cid 0180.0240 Teb: 0000006146608000 Win32Thread: ffffe000eb7b3260 WAIT: (WrUserRequest) UserMode Non-Alertable ffffe000eb70b360 SynchronizationEvent ffffe000eae0f1e0 SynchronizationEvent ffffe000eb226570 SynchronizationEvent ffffe000eb7387e0 SynchronizationEvent ffffe000eb738760 SynchronizationEvent ffffe000eb7385c0 SynchronizationTimer ffffe000eb7340d0 SynchronizationEvent ffffe000eb7383a0 SynchronizationEvent ffffe000eb737060 SynchronizationEvent ffffe000eb7ce340 SynchronizationEvent Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 2110 Ticks: 8671 (0:00:02:15.484) Context Switch Count 31 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 00:00:00.015 Win32 Start Address winsrv!StartCreateSystemThreads (0x00007ff8e97c5680) Stack Init ffffd00025006c90 Current ffffd00025006550 Base ffffd00025007000 Limit ffffd00025001000 Call 0000000000000000 Priority 15 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Kernel stack not resident. THREAD ffffe000eb8d0080 Cid 0180.03a0 Teb: 000000614660a000 Win32Thread: ffffe000eabce820 WAIT: (WrUserRequest) UserMode Non-Alertable ffffe000eb8cd640 SynchronizationEvent Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 819 Ticks: 9962 (0:00:02:35.656) Context Switch Count 4 IdealProcessor: 0 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address winsrv!StartCreateSystemThreads (0x00007ff8e97c5680) Stack Init ffffd000250e2c90 Current ffffd000250e2550 Base ffffd000250e3000 Limit ffffd000250dd000 Call 0000000000000000 Priority 15 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Kernel stack not resident. THREAD ffffe000eb94a3c0 Cid 0180.040c Teb: 000000614660c000 Win32Thread: ffffe000eb7db0e0 WAIT: (WrLpcReceive) UserMode Non-Alertable ffffe000eb94a9f8 Semaphore Limit 0x1 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 10750 Ticks: 31 (0:00:00:00.484) Context Switch Count 384 IdealProcessor: 1 UserTime 00:00:00.015 KernelTime 00:00:00.093 Win32 Start Address CSRSRV!CsrApiRequestThread (0x00007ff8e9825380)
31
Stack Init ffffd0002538bc90 Current ffffd0002538b410 Base ffffd0002538c000 Limit ffffd00025386000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`2538b450 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`2538b590 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`2538b640 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`2538b6d0 fffff801`487585e6 nt!KeWaitForSingleObject+0x375 ffffd000`2538b790 fffff801`48a7a4aa nt!AlpcpWaitForSingleObject+0x3e ffffd000`2538b7d0 fffff801`48a77b32 nt!AlpcpReceiveMessagePort+0x45a ffffd000`2538b860 fffff801`48a776b3 nt!AlpcpReceiveMessage+0x322 ffffd000`2538b9d0 fffff801`487d6ca3 nt!NtAlpcSendWaitReceivePort+0x103 ffffd000`2538ba90 00007ff8`ed2c6174 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`2538bb00) 00000061`46a7f358 00007ff8`e9825602 ntdll!NtAlpcSendWaitReceivePort+0x14 00000061`46a7f360 00007ff8`ed27c585 CSRSRV!CsrApiRequestThread+0x282 00000061`46a7f7f0 00000000`00000000 ntdll!RtlUserThreadStart+0x45 THREAD ffffe000eba97080 Cid 0180.0788 Teb: 000000614660e000 Win32Thread: ffffe000eba78c50 WAIT: (WrLpcReceive) UserMode Non-Alertable ffffe000eba976b8 Semaphore Limit 0x1 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb239080 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 10750 Ticks: 31 (0:00:00:00.484) Context Switch Count 311 IdealProcessor: 2 UserTime 00:00:00.078 KernelTime 00:00:00.046 Win32 Start Address CSRSRV!CsrApiRequestThread (0x00007ff8e9825380) Stack Init ffffd00025b99c90 Current ffffd00025b99410 Base ffffd00025b9a000 Limit ffffd00025b94000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`25b99450 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`25b99590 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`25b99640 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`25b996d0 fffff801`487585e6 nt!KeWaitForSingleObject+0x375 ffffd000`25b99790 fffff801`48a7a4aa nt!AlpcpWaitForSingleObject+0x3e ffffd000`25b997d0 fffff801`48a77b32 nt!AlpcpReceiveMessagePort+0x45a ffffd000`25b99860 fffff801`48a776b3 nt!AlpcpReceiveMessage+0x322 ffffd000`25b999d0 fffff801`487d6ca3 nt!NtAlpcSendWaitReceivePort+0x103 ffffd000`25b99a90 00007ff8`ed2c6174 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`25b99b00) 00000061`46abf998 00007ff8`e9825602 ntdll!NtAlpcSendWaitReceivePort+0x14 00000061`46abf9a0 00007ff8`ed27c585 CSRSRV!CsrApiRequestThread+0x282 00000061`46abfe30 00000000`00000000 ntdll!RtlUserThreadStart+0x45
Note: We see that the current process has changed. We specified 3f flags to have the process context changed to that of csrss.exe during the execution of !process command. We also notice passive threads waiting for ALPC notification, for example, ffffe000eb23f080 (weakly coupled processes) and ffffe000eb74d080 thread waiting for ALPC request reply from svchost.exe process (strongly coupled processes): 3: kd> !alpc /m ffffc000dae5fb30 Message ffffc000dae5fb30 MessageID : CallbackID : SequenceNumber : Type : DataLength : TotalLength : Canceled : Release :
0x0068 (104) 0x0267 (615) 0x00000003 (3) LPC_REQUEST 0x4048 (16456) 0x4070 (16496) No No
32
ReplyWaitReply Continuation OwnerPort WaitingThread QueueType QueuePort QueuePortOwnerProcess ServerThread QuotaCharged CancelQueuePort CancelSequencePort CancelSequenceNumber ClientContext ServerContext PortContext CancelPortContext SecurityData View HandleData
: : : : : : : : : : : : : : : : : : :
No Yes ffffe000eb884610 [ALPC_CLIENT_COMMUNICATION_PORT] ffffe000eb74d080 ALPC_MSGQUEUE_PENDING ffffe000eb73a800 [ALPC_CONNECTION_PORT] ffffe000eb83e840 (svchost.exe) ffffe000ebda8300 Yes 0000000000000000 0000000000000000 0x00000000 (0) 0000000000000000 0000000000000000 000001eaa7f10bd0 0000000000000000 0000000000000000 0000000000000000 0000000000000000
3: kd> !thread ffffe000ebda8300 3f THREAD ffffe000ebda8300 Cid 02b4.0c24 Teb: 000000016cc14000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable ffffe000eb840d40 QueueObject Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb83e840 Image: svchost.exe Attached Process N/A Image: N/A Wait Start TickCount 10745 Ticks: 36 (0:00:00:00.562) Context Switch Count 1832 IdealProcessor: 0 UserTime 00:00:00.046 KernelTime 00:00:00.046 Win32 Start Address ntdll!TppWorkerThread (0x00007ff8ed24b290) Stack Init ffffd00026ca8c90 Current ffffd00026ca83e0 Base ffffd00026ca9000 Limit ffffd00026ca3000 Call 0000000000000000 Priority 13 BasePriority 8 PriorityDecrement 80 IoPriority 2 PagePriority 5 Child-SP ffffd000`26ca8420 ffffd000`26ca8560 ffffd000`26ca8610 ffffd000`26ca86a0 ffffd000`26ca8740 ffffd000`26ca8850 ffffd000`26ca8a90 00000001`6e67f898 00000001`6e67f8a0 00000001`6e67fcb0 00000001`6e67fce0
RetAddr fffff801`487003ea fffff801`486ffe79 fffff801`487025ea fffff801`487021ba fffff801`48702e6b fffff801`487d6ca3 00007ff8`ed2c8794 00007ff8`ed24b528 00007ff8`eb0c8102 00007ff8`ed27c574 00000000`00000000
Call Site nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeRemoveQueueEx+0x22a nt!IoRemoveIoCompletion+0x8a nt!NtWaitForWorkViaWorkerFactory+0x30b nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`26ca8b00) ntdll!NtWaitForWorkViaWorkerFactory+0x14 ntdll!TppWorkerThread+0x298 KERNEL32!BaseThreadInitThunk+0x22 ntdll!RtlUserThreadStart+0x34
Note: ALPC wait chains in csrss.exe are normal and expected. We can get the list of ALPC receiver threads and threads waiting for reply using Microsoft MEX Debugging Extension: https://www.microsoft.com/en-us/download/details.aspx?id=53304 After downloading, extracting, and unzipping, we copy \x64\mex.dll to WinDbg installation folder (For example,
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64). 3: kd> .load mex Mex External 3.0.0.7172 Loaded!
33
3: kd> !help Mex currently has 255 extensions available. Or browse by category:
Please specify a keyword to search.
All PowerShell[6] SystemCenter[3] Networking[12] Process[5] Mex[2] Kernel[27] DotNet[32] Decompile[15] Utility[40] Thread[27] Binaries[6] General[22] 3: kd> !mex.help -all [...] 3: kd> !mex.wrlpcreceive Process PID ===================== === System 4 csrss.exe 180 csrss.exe 180 csrss.exe 180 csrss.exe 180 csrss.exe 180 csrss.exe 1d0 csrss.exe 1d0 csrss.exe 1d0 csrss.exe 1d0 csrss.exe 1d0 csrss.exe 1d0 csrss.exe 1d0 lsass.exe 25c svchost.exe (netsvcs) 388 svchost.exe 484 taskhostw.exe ac4 Count: 17
Thread Id CSwitches User Kernel State Time Reason Wait Function ================ ==== ========= ==== ====== ======= ========= ============ ======================================== ffffe000e9cf1040 114 46 0 0 Waiting 35s.703 WrLpcReceive nt!AlpcpSignalAndWait+0x1d9 ffffe000eb23f080 190 467 16ms 0 Waiting 484ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eb75a080 1bc 3 0 0 Waiting 2m:39.546 WrLpcReceive Kernel stack not resident ffffe000eb78a080 1fc 515 47ms 94ms Waiting 484ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eb94a3c0 40c 384 16ms 94ms Waiting 484ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eba97080 788 311 78ms 47ms Waiting 484ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eb76a080 1e0 365 47ms 125ms Waiting 578ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eb7a2080 218 3 0 0 Waiting 2m:39.453 WrLpcReceive Kernel stack not resident ffffe000eb7c5080 230 374 31ms 109ms Waiting 578ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eb8863c0 328 2 0 0 Waiting 2m:32.312 WrLpcReceive Kernel stack not resident ffffe000eb8a7080 35c 336 47ms 94ms Waiting 578ms WrLpcReceive CSRSRV!CsrApiRequestThread+0 x282 ffffe000ebfda840 123c 184 31ms 16ms Waiting 578ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000ebfd8840 1240 173 16ms 31ms Waiting 281ms WrLpcReceive CSRSRV!CsrApiRequestThread+0x282 ffffe000eb7f8080 26c 2 0 0 Waiting 2m:12.750 WrLpcReceive nt!AlpcpReceiveMessagePort+0x45a ffffe000eb9e9340 4cc 178 0 16ms Waiting 578ms WrLpcReceive themeservice!CAPIConnection::Listen+0x8b ffffe000eba3a780 538 182 0 16ms Waiting 49s.203 WrLpcReceive nt!AlpcpReceiveMessagePort+0x45a ffffe000ebd86080 974 371 16ms 31ms Waiting 62ms WrLpcReceive MSCTF!CCtfServerPort::ServerLoop+0x18a
0: kd> !mex.wrlpcreply Process PID Thread Id CSwitches User Kernel State Time Wait Function ============================ === ================ ==== ========= ==== ====== ======= ========= ============================================================ ========================= csrss.exe 180 ffffe000eb74d080 1b0 7 0 0 Waiting 2m:14.031 (0n692) Kernel stack not resident csrss.exe 1d0 ffffe000eb79e080 20c 276 0 0 Waiting 2m:14.031 (0n692) Kernel stack not resident svchost.exe (netsvcs) 388 ffffe000ebe2e080 cb8 3 0 0 Waiting 2m:12.750 svchost.exe 3f4 ffffe000e9054600 b54 2 0 0 Waiting 2m:12.734 svchost.exe (NetworkService) 4dc ffffe000e9278540 b78 2 0 0 Waiting 2m:12.734 explorer.exe c64 ffffe000ec19d080 1764 10 0 0 Waiting 1m:16.484
Reason
Waiting On
========== WrLpcReply Thread: ffffe000ebda8300 in svchost.exe (DcomLaunch) WrLpcReply Thread: ffffe000ebda8300 in svchost.exe (DcomLaunch) WrLpcReply WrLpcReply WrLpcReply WrLpcReply
Thread: Thread: Thread: Message
ffffe000ebc143c0 in svchost.exe (0n1012) ffffe000eb8863c0 in csrss.exe (0n464) ffffe000eb8ed040 in svchost.exe (0n1012) queued to ShellExpe rienceHost.exe (0n3484)
Note: MEX command changed the current CPU from 3 to 0. 8.
Now we list processes and threads from the session 1:
0: kd> !sprocess 1 3f Dumping Session 1 _MM_SESSION_SPACE ffffd000251ac000 _MMSESSION ffffd000251acb40 PROCESS ffffe000eb21d840 SessionId: 1 Cid: 01d0 Peb: 27d00b2000 ParentCid: 01c0 DirBase: 2685f000 ObjectTable: ffffc000dad6fac0 HandleCount: Image: csrss.exe VadRoot ffffe000eb79ed60 Vads 80 Clone 0 Private 212. Modified 2761. Locked 0. DeviceMap ffffc000da21a760 Token ffffc000dad84b30 ElapsedTime 00:02:39.544 UserTime 00:00:00.000 KernelTime 00:00:00.078 QuotaPoolUsage[PagedPool] 148992 QuotaPoolUsage[NonPagedPool] 16200 Working Set Sizes (now,min,max) (548, 50, 345) (2192KB, 200KB, 1380KB) PeakWorkingSetSize 2499 VirtualSize 2097199 Mb PeakVirtualSize 2097208 Mb PageFaultCount 6214 MemoryPriority BACKGROUND BasePriority 13 CommitCharge 344 PEB at 00000027d00b2000
34
InheritedAddressSpace: No ReadImageFileExecOptions: No BeingDebugged: No ImageBaseAddress: 00007ff71e540000 Ldr 00007ff8ed365200 Ldr.Initialized: Yes Ldr.InInitializationOrderModuleList: 0000018bfb602ee0 . 0000018bfb626870 Ldr.InLoadOrderModuleList: 0000018bfb603050 . 0000018bfb626850 Ldr.InMemoryOrderModuleList: 0000018bfb603060 . 0000018bfb626860 Base TimeStamp Module 7ff71e540000 5632d16d Oct 30 02:09:49 2015 C:\Windows\system32\csrss.exe 7ff8ed220000 56a8483f Jan 27 04:31:59 2016 C:\Windows\SYSTEM32\ntdll.dll 7ff8e9820000 5632d16f Oct 30 02:09:51 2015 C:\Windows\system32\CSRSRV.dll 7ff8e9800000 5632d166 Oct 30 02:09:42 2015 C:\Windows\system32\basesrv.DLL 7ff8e97c0000 5632d722 Oct 30 02:34:10 2015 C:\Windows\system32\winsrv.DLL 7ff8eb3e0000 565423d2 Nov 24 08:46:10 2015 C:\Windows\system32\USER32.dll 7ff8e9c80000 56a8489c Jan 27 04:33:32 2016 C:\Windows\system32\kernelbase.dll 7ff8eb0b0000 5632d5aa Oct 30 02:27:54 2015 C:\Windows\system32\kernel32.dll 7ff8ed090000 568b2035 Jan 05 01:45:25 2016 C:\Windows\system32\GDI32.dll 7ff8e97b0000 5632d888 Oct 30 02:40:08 2015 C:\Windows\system32\sxssrv.DLL 7ff8e9670000 5632d5f0 Oct 30 02:29:04 2015 C:\Windows\system32\sxs.dll 7ff8ea890000 5632d515 Oct 30 02:25:25 2015 C:\Windows\system32\RPCRT4.dll 7ff8e9bb0000 5632d756 Oct 30 02:35:02 2015 C:\Windows\system32\bcryptPrimitives.dll SubSystemData: 0000000000000000 ProcessHeap: 0000018bfb500000 ProcessParameters: 0000018bfb602550 CurrentDirectory: 'C:\Windows\system32\' WindowTitle: '< Name not readable >' ImageFile: 'C:\Windows\system32\csrss.exe' CommandLine: '%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16' DllPath: '< Name not readable >' Environment: 0000018bfb602080 ComSpec=C:\Windows\system32\cmd.exe NUMBER_OF_PROCESSORS=4 OS=Windows_NT Path=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\ PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 58 Stepping 9, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=3a09 PSModulePath=%ProgramFiles%\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules SystemDrive=C: SystemRoot=C:\Windows TEMP=C:\Windows\TEMP TMP=C:\Windows\TEMP USERNAME=SYSTEM windir=C:\Windows THREAD ffffe000eb76a080 Cid 01d0.01e0 Teb: 00000027d00b9000 Win32Thread: ffffe000eaec6ef0 WAIT: (WrLpcReceive) UserMode Non-Alertable ffffe000eb76a6b8 Semaphore Limit 0x1 Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000eb21d840 Image: csrss.exe Attached Process N/A Image: N/A Wait Start TickCount 10744 Ticks: 37 (0:00:00:00.578) Context Switch Count 365 IdealProcessor: 0 UserTime 00:00:00.046 KernelTime 00:00:00.125 Win32 Start Address CSRSRV!CsrApiRequestThread (0x00007ff8e9825380) Stack Init ffffd00025576c90 Current ffffd00025576410
35
Base ffffd00025577000 Limit ffffd00025571000 Call 0000000000000000 Priority 14 BasePriority 13 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP ffffd000`25576450 ffffd000`25576590 ffffd000`25576640 ffffd000`255766d0 ffffd000`25576790 ffffd000`255767d0 ffffd000`25576860 ffffd000`255769d0 ffffd000`25576a90 ffffd000`25576b00) 00000027`d027f858 00000027`d027f860 00000027`d027fcf0
RetAddr fffff801`487003ea fffff801`486ffe79 fffff801`486ffae5 fffff801`487585e6 fffff801`48a7a4aa fffff801`48a77b32 fffff801`48a776b3 fffff801`487d6ca3 00007ff8`ed2c6174
Call Site nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 nt!AlpcpWaitForSingleObject+0x3e nt!AlpcpReceiveMessagePort+0x45a nt!AlpcpReceiveMessage+0x322 nt!NtAlpcSendWaitReceivePort+0x103 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @
00007ff8`e9825602 ntdll!NtAlpcSendWaitReceivePort+0x14 00007ff8`ed27c585 CSRSRV!CsrApiRequestThread+0x282 00000000`00000000 ntdll!RtlUserThreadStart+0x45
[...] PROCESS ffffe000ec373080 SessionId: 1 Cid: 12b0 Peb: 00516000 ParentCid: 0c64 DirBase: 27369000 ObjectTable: ffffc000dd3bc840 HandleCount: Image: OneDrive.exe VadRoot ffffe000ec3e5b00 Vads 168 Clone 0 Private 918. Modified 1753. Locked 0. DeviceMap ffffc000db94eec0 Token ffffc000dccf26c0 ElapsedTime 00:02:08.766 UserTime 00:00:00.000 KernelTime 00:00:00.031 QuotaPoolUsage[PagedPool] 255608 QuotaPoolUsage[NonPagedPool] 23256 Working Set Sizes (now,min,max) (713, 50, 345) (2852KB, 200KB, 1380KB) PeakWorkingSetSize 4842 VirtualSize 134 Mb PeakVirtualSize 139 Mb PageFaultCount 6191 MemoryPriority BACKGROUND BasePriority 8 CommitCharge 1235 PEB at 0000000000516000 error 1 InitTypeRead( nt!_PEB at 0000000000516000)... THREAD ffffe000ec367080 Cid 12b0.12b4 Teb: 0000000000518000 Win32Thread: ffffe000ec2d44a0 WAIT: (WrUserRequest) UserMode Non-Alertable ffffe000ec5941e0 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 7705 Ticks: 3076 (0:00:00:48.062) Context Switch Count 215 IdealProcessor: 3 UserTime 00:00:00.062 KernelTime 00:00:00.078 Win32 Start Address 0x000000000037e2c6 Stack Init ffffd00027df8c90 Current ffffd00027df8480 Base ffffd00027df9000 Limit ffffd00027df3000 Call 0000000000000000 Priority 10 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`27df84c0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`27df8600 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`27df86b0 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`27df8740 fffff961`7f6de5c5 nt!KeWaitForSingleObject+0x375 ffffd000`27df8800 fffff961`7f6de1c8 win32kfull!xxxRealSleepThread+0x355 ffffd000`27df88f0 fffff961`7f6dcd9d win32kfull!xxxSleepThread2+0x98
36
ffffd000`27df8940 ffffd000`27df8a70 ffffd000`27df8b00 ffffd000`27df8b00) 00000000`0008e398
fffff961`7f6dc1e0 win32kfull!xxxRealInternalGetMessage+0xb4d fffff801`487d6ca3 win32kfull!NtUserGetMessage+0x90 00000000`6c393824 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ 00000000`00000000 0x6c393824
THREAD ffffe000ec5ef080 Cid 12b0.12c4 Teb: 0000000000524000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ec4c2350 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2556 Ticks: 8225 (0:00:02:08.515) Context Switch Count 1 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000777be7f0 Stack Init ffffd000270bcc90 Current ffffd000270bc710 Base ffffd000270bd000 Limit ffffd000270b7000 Call 0000000000000000 Priority 10 BasePriority 10 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`270bc750 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`270bc890 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`270bc940 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`270bc9d0 fffff801`48a9f032 nt!KeWaitForSingleObject+0x375 ffffd000`270bca90 fffff801`487d6ca3 nt!NtWaitForSingleObject+0xf2 ffffd000`270bcb00 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`270bcb00) 00000000`00aef018 00000000`00000000 0x6c4021bc THREAD ffffe000ec487840 Cid 12b0.12c8 Teb: 0000000000527000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable ffffe000ec59a8c0 QueueObject Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2571 Ticks: 8210 (0:00:02:08.281) Context Switch Count 37 IdealProcessor: 0 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x000000007777c6d0 Stack Init ffffd000274cfc90 Current ffffd000274cf3e0 Base ffffd000274d0000 Limit ffffd000274ca000 Call 0000000000000000 Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`274cf420 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`274cf560 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`274cf610 fffff801`487025ea nt!KiCommitThreadWait+0x149 ffffd000`274cf6a0 fffff801`487021ba nt!KeRemoveQueueEx+0x22a ffffd000`274cf740 fffff801`48702e6b nt!IoRemoveIoCompletion+0x8a ffffd000`274cf850 fffff801`487d6ca3 nt!NtWaitForWorkViaWorkerFactory+0x30b ffffd000`274cfa90 00007ff8`ed2c8794 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`274cfb00) 00000000`00b3e628 00000000`00000000 0x00007ff8`ed2c8794 THREAD ffffe000ec5de080 Cid 12b0.12cc Teb: 000000000052a000 Win32Thread: ffffe000ec26aba0 WAIT: (WrQueue) UserMode Alertable ffffe000ec59a8c0 QueueObject Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 10254 Ticks: 527 (0:00:00:08.234) Context Switch Count 76 IdealProcessor: 1 UserTime 00:00:00.031
37
KernelTime 00:00:00.015 Win32 Start Address 0x000000007777c6d0 Stack Init ffffd000273a2c90 Current ffffd000273a23e0 Base ffffd000273a3000 Limit ffffd0002739d000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`273a2420 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`273a2560 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`273a2610 fffff801`487025ea nt!KiCommitThreadWait+0x149 ffffd000`273a26a0 fffff801`487021ba nt!KeRemoveQueueEx+0x22a ffffd000`273a2740 fffff801`48702e6b nt!IoRemoveIoCompletion+0x8a ffffd000`273a2850 fffff801`487d6ca3 nt!NtWaitForWorkViaWorkerFactory+0x30b ffffd000`273a2a90 00007ff8`ed2c8794 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`273a2b00) 00000000`00b7ea38 00000000`6c33686e 0x00007ff8`ed2c8794 00000000`00b7ea40 00000000`6c403500 0x6c33686e 00000000`00b7ea48 00000000`00000001 0x6c403500 00000000`00b7ea50 00002326`00000100 0x1 00000000`00b7ea58 00000000`0298fbc8 0x00002326`00000100 00000000`00b7ea60 00000000`00b7eaa0 0x298fbc8 00000000`00b7ea68 00000000`0298faa4 0xb7eaa0 00000000`00b7ea70 00000000`00b7ea40 0x298faa4 00000000`00b7ea78 00000000`6c334185 0xb7ea40 00000000`00b7ea80 00000000`00b7eaa0 0x6c334185 00000000`00b7ea88 00000000`0000003c 0xb7eaa0 00000000`00b7ea90 00000000`007422d8 0x3c 00000000`00b7ea98 00000000`0298fdd8 0x7422d8 00000000`00b7eaa0 00000000`00000000 0x298fdd8 THREAD ffffe000ec5cf080 Cid 12b0.12d4 Teb: 0000000000530000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Non-Alertable ffffe000ec2ce080 QueueObject Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2563 Ticks: 8218 (0:00:02:08.406) Context Switch Count 1 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x0000000071304ccc Stack Init ffffd0002556bc90 Current ffffd0002556b560 Base ffffd0002556c000 Limit ffffd00025566000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`2556b5a0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`2556b6e0 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`2556b790 fffff801`487025ea nt!KiCommitThreadWait+0x149 ffffd000`2556b820 fffff801`487021ba nt!KeRemoveQueueEx+0x22a ffffd000`2556b8c0 fffff801`48af2964 nt!IoRemoveIoCompletion+0x8a ffffd000`2556b9d0 fffff801`487d6ca3 nt!NtRemoveIoCompletion+0x134 ffffd000`2556ba90 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`2556bb00) 00000000`035dec78 00000000`00000000 0x6c4021bc THREAD ffffe000ec5eb080 Cid 12b0.12d8 Teb: 0000000000533000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Non-Alertable ffffe000ec2ce080 QueueObject Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2563 Ticks: 8218 (0:00:02:08.406) Context Switch Count 1 IdealProcessor: 0 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x0000000071304ccc
38
Stack Init ffffd000268c7c90 Current ffffd000268c7560 Base ffffd000268c8000 Limit ffffd000268c2000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`268c75a0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`268c76e0 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`268c7790 fffff801`487025ea nt!KiCommitThreadWait+0x149 ffffd000`268c7820 fffff801`487021ba nt!KeRemoveQueueEx+0x22a ffffd000`268c78c0 fffff801`48af2964 nt!IoRemoveIoCompletion+0x8a ffffd000`268c79d0 fffff801`487d6ca3 nt!NtRemoveIoCompletion+0x134 ffffd000`268c7a90 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`268c7b00) 00000000`0371ebe8 00000000`00000000 0x6c4021bc THREAD ffffe000ec441840 Cid 12b0.12dc Teb: 0000000000536000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ec361580 NotificationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2563 Ticks: 8218 (0:00:02:08.406) Context Switch Count 1 IdealProcessor: 1 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000719bbfb4 Stack Init ffffd000279e6c90 Current ffffd000279e6710 Base ffffd000279e7000 Limit ffffd000279e1000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`279e6750 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`279e6890 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`279e6940 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`279e69d0 fffff801`48a9f032 nt!KeWaitForSingleObject+0x375 ffffd000`279e6a90 fffff801`487d6ca3 nt!NtWaitForSingleObject+0xf2 ffffd000`279e6b00 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`279e6b00) 00000000`0385edf8 00000000`00000000 0x6c4021bc THREAD ffffe000ec5c2840 Cid 12b0.12e0 Teb: 0000000000539000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ea90f760 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 10254 Ticks: 527 (0:00:00:08.234) Context Switch Count 8 IdealProcessor: 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000719bbfb4 Stack Init ffffd000255a9c90 Current ffffd000255a9710 Base ffffd000255aa000 Limit ffffd000255a4000 Call 0000000000000000 Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`255a9750 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`255a9890 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`255a9940 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`255a99d0 fffff801`48a9f032 nt!KeWaitForSingleObject+0x375 ffffd000`255a9a90 fffff801`487d6ca3 nt!NtWaitForSingleObject+0xf2 ffffd000`255a9b00 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`255a9b00) 00000000`0399ed08 00000000`6c402021 0x6c4021bc 00000000`0399ed10 00000023`777c854c 0x6c402021 00000000`0399ed18 00000000`00000023 0x00000023`777c854c 00000000`0399ed20 00000000`ee556126 0x23 00000000`0399ed28 00000000`03a9f37c 0xee556126
39
00000000`0399ed30 00000000`0399ed38 00000000`0399ed40 00000000`0399ed48
00000000`0399ed58 00000000`006e1b88 00000000`0000006e 00000000`00000000
0x3a9f37c 0x399ed58 0x6e1b88 0x6e
THREAD ffffe000ec5ab080 Cid 12b0.12e4 Teb: 000000000053c000 Win32Thread: ffffe000ec268970 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ec4b2480 NotificationEvent ffffe000eb793be0 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2572 Ticks: 8209 (0:00:02:08.265) Context Switch Count 5 IdealProcessor: 3 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000719bbfb4 Stack Init ffffd0002763cc90 Current ffffd0002763bf80 Base ffffd0002763d000 Limit ffffd00027637000 Call 0000000000000000 Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`2763bfc0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`2763c100 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`2763c1b0 fffff801`48701a1e nt!KiCommitThreadWait+0x149 ffffd000`2763c240 fffff801`48a9c21d nt!KeWaitForMultipleObjects+0x24e ffffd000`2763c300 fffff801`48af40a7 nt!ObWaitForMultipleObjects+0x2bd ffffd000`2763c810 fffff801`487d6ca3 nt!NtWaitForMultipleObjects32+0xf7 ffffd000`2763ca90 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`2763cb00) 00000000`03adec48 00000000`00000000 0x6c4021bc THREAD ffffe000ec59b840 Cid 12b0.12e8 Teb: 000000000053f000 Win32Thread: ffffe000ec15fb00 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ebc4afa0 NotificationEvent ffffe000ec2e4db0 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2572 Ticks: 8209 (0:00:02:08.265) Context Switch Count 20 IdealProcessor: 0 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000719bbfb4 Stack Init ffffd00027908c90 Current ffffd00027907f80 Base ffffd00027909000 Limit ffffd00027903000 Call 0000000000000000 Priority 8 BasePriority 6 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`27907fc0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`27908100 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`279081b0 fffff801`48701a1e nt!KiCommitThreadWait+0x149 ffffd000`27908240 fffff801`48a9c21d nt!KeWaitForMultipleObjects+0x24e ffffd000`27908300 fffff801`48af40a7 nt!ObWaitForMultipleObjects+0x2bd ffffd000`27908810 fffff801`487d6ca3 nt!NtWaitForMultipleObjects32+0xf7 ffffd000`27908a90 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`27908b00) 00000000`03c1ee68 00000000`00000000 0x6c4021bc THREAD ffffe000ec59e840 Cid 12b0.12ec Teb: 0000000000542000 Win32Thread: ffffe000ec2385a0 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ec282e20 NotificationEvent ffffe000eba64ba0 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A
40
Wait Start TickCount 2568 Ticks: 8213 (0:00:02:08.328) Context Switch Count 12 IdealProcessor: 1 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000719bbfb4 Stack Init ffffd00027422c90 Current ffffd00027421f80 Base ffffd00027423000 Limit ffffd0002741d000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`27421fc0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`27422100 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`274221b0 fffff801`48701a1e nt!KiCommitThreadWait+0x149 ffffd000`27422240 fffff801`48a9c21d nt!KeWaitForMultipleObjects+0x24e ffffd000`27422300 fffff801`48af40a7 nt!ObWaitForMultipleObjects+0x2bd ffffd000`27422810 fffff801`487d6ca3 nt!NtWaitForMultipleObjects32+0xf7 ffffd000`27422a90 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`27422b00) 00000000`03d5f0a8 00000000`00000000 0x6c4021bc THREAD ffffe000ec58b040 Cid 12b0.12f0 Teb: 0000000000545000 Win32Thread: 0000000000000000 WAIT: (WrQueue) UserMode Alertable ffffe000ec59a8c0 QueueObject Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 2569 Ticks: 8212 (0:00:02:08.312) Context Switch Count 3 IdealProcessor: 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x000000007777c6d0 Stack Init ffffd000276fbc90 Current ffffd000276fb3e0 Base ffffd000276fc000 Limit ffffd000276f6000 Call 0000000000000000 Priority 8 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`276fb420 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`276fb560 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`276fb610 fffff801`487025ea nt!KiCommitThreadWait+0x149 ffffd000`276fb6a0 fffff801`487021ba nt!KeRemoveQueueEx+0x22a ffffd000`276fb740 fffff801`48702e6b nt!IoRemoveIoCompletion+0x8a ffffd000`276fb850 fffff801`487d6ca3 nt!NtWaitForWorkViaWorkerFactory+0x30b ffffd000`276fba90 00007ff8`ed2c8794 nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`276fbb00) 00000000`03e9e5c8 00000000`00000000 0x00007ff8`ed2c8794 THREAD ffffe000ec585840 Cid 12b0.12f4 Teb: 0000000000548000 Win32Thread: ffffe000ebb31ec0 WAIT: (DelayExecution) UserMode Non-Alertable ffffffffffffffff NotificationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 10251 Ticks: 530 (0:00:00:08.281) Context Switch Count 27 IdealProcessor: 3 UserTime 00:00:00.015 KernelTime 00:00:00.000 Win32 Start Address 0x000000007777c6d0 Stack Init ffffd00027d5ac90 Current ffffd00027d5a790 Base ffffd00027d5b000 Limit ffffd00027d55000 Call 0000000000000000 Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`27d5a7d0 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`27d5a910 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`27d5a9c0 fffff801`486944ac nt!KiCommitThreadWait+0x149 ffffd000`27d5aa50 fffff801`48a5101c nt!KeDelayExecutionThread+0x28c ffffd000`27d5aad0 fffff801`487d6ca3 nt!NtDelayExecution+0x5c
41
ffffd000`27d5ab00 ffffd000`27d5ab00) 00000000`040decc8 00000000`040decd0 00000000`040decd8 00000000`040dece0 00000000`040dece8 00000000`040decf0 00000000`040decf8 00000000`040ded00 00000000`040ded08 00000000`040ded10 00000000`040ded18 00000000`040ded20 00000000`040ded28 00000000`040ded30 00000000`040ded38 00000000`040ded40
00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ 00000000`6c40209d 00000023`777c6ccc 00000000`00000023 00000000`007aee30 00000000`041df314 00000000`040ded20 00000000`000120bf 00000000`041dc5a8 00000000`00000018 00000000`00784900 00000000`00000004 00000000`041dc518 00000000`041df300 00000000`041dc518 00000000`040dede0 00000000`00000000
0x6c4021bc 0x6c40209d 0x00000023`777c6ccc 0x23 0x7aee30 0x41df314 0x40ded20 0x120bf 0x41dc5a8 0x18 0x784900 0x4 0x41dc518 0x41df300 0x41dc518 0x40dede0
[...]
Note: Incorrect and truncated stack traces with the presence of 32-bit return addresses may point to a virtualized WOW64 process in case wow64* module information was paged out. Please see exercise Legacy.C1 for iexplore.exe example. We can double check the process bitness by using MEX extention tasklist command: 0: kd> !tasklist -s 1 PID Address ============= ================ 0x1d0 0n464 ffffe000eb21d840 0x21c 0n540 ffffe000eb7a52c0 0x354 0n852 ffffe000eb8a3080 0xb0c 0n2828 ffffe000eb601080 0xac4 0n2756 ffffe000ebd4e840 0xc5c 0n3164 ffffe000ebdca840 0xc64 0n3172 ffffe000ebdc7840 0xca4 0n3236 ffffe000ebdcd840 0xd9c 0n3484 ffffe000ebf0a840 0xe58 0n3672 ffffe000ebf00840 0xfbc 0n4028 ffffe000ec252080 0xff0 0n4080 ffffe000ec121080 0x1228 0n4648 ffffe000eb6c3080 0x12b0 0n4784 ffffe000ec373080 0x1050 0n4176 ffffe000ec24a080 0x10f8 0n4344 ffffe000ec491080 0x1208 0n4616 ffffe000ec220080 0x1354 0n4948 ffffe000ec62c840 0x1378 0n4984 ffffe000ec6a8640 0x105c 0n4188 ffffe000ec77d840 0x1430 0n5168 ffffe000ec88d840 0x14c0 0n5312 ffffe000ec944840 0x2c4 0n708 ffffe000ec156840 0x3a8 0n936 ffffe000eca66840 0x1594 0n5524 ffffe000ec09a080 ============= ================ PID Address
Name ============================= csrss.exe winlogon.exe dwm.exe sihost.exe taskhostw.exe RuntimeBroker.exe explorer.exe SkypeHost.exe*32 ShellExperienceHost.exe SearchUI.exe TabTip.exe TabTip32.exe*32 vmtoolsd.exe OneDrive.exe*32 ApplicationFrameHost.exe MicrosoftEdge.exe browser_broker.exe MicrosoftEdgeCP.exe SearchProtocolHost.exe MicrosoftEdgeCP.exe MicrosoftEdgeCP.exe MicrosoftEdgeCP.exe notepad.exe svchost.exe(UnistackSvcGroup) NotMyfault.exe ============================= Name
Ses === 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 === Ses
Warning! Zombie process(es) detected (not displayed). Count: 2 [zombie report]
42
Note: For the complete list or tasklist command options, please use -? parameter. 9. Suppose, we are interested in the last OneDrive.exe thread ffffe000ec585840 (here we need /w switch): 0: kd> .load wow64exts 0: kd> .thread /w ffffe000ec585840 Implicit thread is now ffffe000`ec585840 WARNING: WOW context retrieval requires switching to the thread's process context. Use .process /p ffffe000`ec121080 to switch back. Implicit process is now ffffe000`ec373080 x86 context set 0: kd:x86> k *** Stack trace for last set context - .thread/.cxr resets it # ChildEBP RetAddr WARNING: Frame IP not in any known module. Following frames may be wrong. 00 041df7cc 748ea56f 0x777c6f3c 01 041df7dc 70700d58 0x748ea56f 02 041df804 7070124a 0x70700d58 03 041df848 7777933a 0x7070124a 04 041df8b4 7777929a 0x7777933a 05 041df8d4 7777cd32 0x7777929a 06 041dfa8c 75f538f4 0x7777cd32 07 041dfaa0 777b5e13 0x75f538f4 08 041dfae8 777b5dde 0x777b5e13 09 041dfaf8 00000000 0x777b5dde 0: kd:x86> .reload Loading Kernel Symbols ............................................................... ................................................................ ...................................... Loading User Symbols PEB is paged out (Peb.Ldr = 00000000`00516018). Type ".hh dbgerr001" for details Loading unloaded module list .............Unable to enumerate user-mode unloaded modules, NTSTATUS 0xC0000147 Loading Wow64 Symbols ......................Unable to read NT module Base Name string at 00000000`006bb938 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006d6d2c - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006c1460 - NTSTATUS 0xC0000147 ....Unable to read NT module Base Name string at 00000000`006c1850 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006c13d0 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006c1a90 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006da118 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`00707204 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e6518 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006ddd30 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`00707bbc - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`00707d54 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006ddc58 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e6568 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`006ddca0 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd118 - NTSTATUS 0xC0000147 ...Unable to read NT module Base Name string at 00000000`0070b68e - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`0070c17c - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd598 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd2c8 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`0070c680 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`006dd238 - NTSTATUS 0xC0000147
43
.Unable to read NT module Base Name string at 00000000`006df948 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd820 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd550 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd280 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`0070fa74 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e6248 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd478 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006df3c8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e5f78 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd1a8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd5e0 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e5ed8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd6b8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd628 - NTSTATUS 0xC0000147 .. .Unable to read NT module Base Name string at 00000000`0070b574 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`0072ee18 - NTSTATUS 0xC0000147 ..Unable to read NT module Base Name string at 00000000`0074bb98 - NTSTATUS 0xC0000147 ....Unable to read NT module Base Name string at 00000000`00755830 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. ........ ************* Symbol Loading Error Summary ************** Module name Error SharedUserData No error - symbol load deferred You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded. You should also verify that your symbol search path (.sympath) is correct. 0: kd:x86> k *** Stack trace for last set context - .thread/.cxr resets it # ChildEBP RetAddr 00 041df764 748ea619 ntdll!NtDelayExecution+0xc 01 041df7cc 748ea56f KERNELBASE!SleepEx+0x99 02 041df7dc 70700d58 KERNELBASE!Sleep+0xf 03 041df804 7070124a WINHTTP!SafeTerminateDll+0xa8 04 041df848 7777933a WINHTTP!FailFastThreadpoolWaitCallback+0x2a 05 041df8b4 7777929a ntdll!TppExecuteWaitCallback+0x7a 06 041df8d4 7777cd32 ntdll!TppWaitCompletion+0x8a 07 041dfa8c 75f538f4 ntdll!TppWorkerThread+0x662 08 041dfaa0 777b5e13 KERNEL32!BaseThreadInitThunk+0x24 09 041dfae8 777b5dde ntdll!__RtlUserThreadStart+0x2f 0a 041dfaf8 00000000 ntdll!_RtlUserThreadStart+0x1b
Note: To switch back to our native processor architecture we use .effmach or !sw commands: 0: kd:x86> .effmach AMD64 Effective machine: x64 (AMD64) 0: kd> .thread /w ffffe000ec585840 Implicit thread is now ffffe000`ec585840 x86 context set 0: kd:x86> !sw Switched to Host mode
10. Another way to list all stack traces is to use !for_each_thread command where we can customize stack trace output: 0: kd> !for_each_thread ".thread /r /p @#Thread; kv" Implicit thread is now ffffe000`e9058600 Implicit process is now ffffe000`e9040700 Loading User Symbols
44
************* Symbol Loading Error Summary ************** Module name Error SharedUserData No error - symbol load deferred You can troubleshoot most symbol related issues by turning on symbol loading loaded. You should also verify that your symbol search path (.sympath) is correct. *** Stack trace for last set context - .thread/.cxr resets it # Child-SP RetAddr : Args to Child 00 ffffd000`23db2890 fffff801`487003ea : 08488b2f`00000000 00000000`00000001 01 ffffd000`23db29d0 fffff801`486ffe79 : 458b0448`8bf4458b e58bfc45`8b088908 02 ffffd000`23db2a80 fffff801`486ffae5 : 0173850f`c085c10b 24fe835c`ee830000 03 ffffd000`23db2b10 fffff801`487ba48e : fffff801`48964140 8b66fc45`00000000 04 ffffd000`23db2bd0 fffff801`4876d5a5 : 3b0247b7`0f7a7504 404b88b9`4575f445 05 ffffd000`23db2c10 fffff801`487d1626 : ffffd000`28840180 ffffe000`e9058600 06 ffffd000`23db2c60 00000000`00000000 : ffffd000`23db3000 ffffd000`23dad000 Implicit thread is now ffffe000`e90cb040 Implicit process is now ffffe000`e9040700 Loading User Symbols
diagnostics (!sym noisy) and repeating the command that caused symbols to be
f67501ee`83c80301 00000000`00000000 b70f0000`0167820f 75c33b66`02c18300 66108b66`06478d00 fffff801`4876d564 00000000`00000000
72c13bc1`0301b70f 53f8458d`0cec83ec 00000000`00000000 d118ee83`fc4d2b00 66267508`558b113b e8502847`8d5213eb 00000000`00000000
: : : : : : : :
Call Site nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject +0x375 nt!PopIrpWorkerControl+0x22 nt!PspSystemThreadStartup+0x41 nt!KiStartSystemThread+0x16
[...] Implicit thread is now ffffe000`ec475080 Implicit process is now ffffe000`ec09a080 Loading User Symbols ........................................... ************* Symbol Loading Error Summary ************** Module name Error SharedUserData No error - symbol load deferred msrpc The system cannot find the file specified vmci The system cannot find the file specified vsock The system cannot find the file specified vmhgfs Symbol loading cancelled vmmemctl The system cannot find the file specified myfault The system cannot find the file specified You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded. You should also verify that your symbol search path (.sympath) is correct. *** Stack trace for last set context - .thread/.cxr resets it # Child-SP RetAddr : Args to Child : Call Site 00 ffffd000`257fefc0 fffff801`487003ea : ffffe000`00000000 00000000`00000001 00000000`00000000 fffff801`00000000 : nt!KiSwapContext+0x76 01 ffffd000`257ff100 fffff801`486ffe79 : ffffc000`dd8e2ef0 fffff801`488ba2a1 00000000`00000000 fffff801`486fb0ae : nt!KiSwapThread+0x15a 02 ffffd000`257ff1b0 fffff801`48701a1e : ffffd000`00000000 ffffc000`dd9992f4 00000000`00000008 00000000`00000002 : nt!KiCommitThreadWait+0x149 03 ffffd000`257ff240 fffff801`48a9c21d : ffffd000`00000002 ffffd000`257ff3c0 00000000`00000000 ffffd000`00000006 : nt!KeWaitForMultipleObjects+0x24e 04 ffffd000`257ff300 fffff801`48af1c26 : fffff801`4899eb01 00000000`00000000 00000000`00000000 ffffd000`257ffad8 : nt!ObWaitForMultipleObjects+0x2bd 05 ffffd000`257ff810 fffff801`487d6ca3 : 00000000`00000000 ffffd000`00000000 ffffe000`ec475080 00000000`03a5f8f8 : nt!NtWaitForMultipleObject s+0xf6 06 ffffd000`257ffa90 00007ff8`ed2c5bd4 : 00007ff8`e9cc3b2f 00007ff8`ecf68210 00000000`00000002 00000000`10000010 : nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`257ffb00) 07 00000000`03a5f8d8 00007ff8`e9cc3b2f : 00007ff8`ecf68210 00000000`00000002 00000000`10000010 00000000`00000000 : ntdll!NtWaitForMultipleObjects+0x14 08 00000000`03a5f8e0 00007ff8`eab1727f : 00000000`00000001 00007ff8`eac21148 00000000`00000001 00000000`0062d800 : KERNELBASE!WaitForMultipleObjectsEx+0xef 09 00000000`03a5fbe0 00007ff8`eab170e7 : 00000000`0062d800 00000000`00000000 00000000`00644f40 00000000`000017ac : combase!WaitCoalesced+0xb3 [d:\th\com\published\comutils\coalescedwait.cxx @ 72] 0a 00000000`03a5fe70 00007ff8`eab27c4c : 00000000`ffffffff 00000000`0062d800 00000000`00644f40 00000000`00000000 : combase!CRpcThread::WorkerLoop+0x11f [d:\th\com\combase\dcomrem\threads.cxx @ 321] 0b 00000000`03a5fee0 00007ff8`eb0c8102 : 00007ff8`eab27bd0 00000000`00000000 00000000`00000000 00000000`00000000 : combase!CRpcThreadCache::RpcWorkerThreadEntry+0x7c [d:\th\com\combase\dcomrem\threads.cxx @ 76] 0c 00000000`03a5ff10 00007ff8`ed27c574 : 00007ff8`eb0c80e0 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x22 0d 00000000`03a5ff40 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x34
Note: We can use this script to list all processes and threads including 32-bit stack traces when it is possible: 0: kd> !for_each_thread "!thread @#Thread 1f;.thread /w @#Thread; .reload; kb 256; .effmach AMD64" !thread @#Thread 1f;.thread /w @#Thread; .reload; kb 256; .effmach AMD64 Setting context for owner process... .process /p /r ffffe000e9040700 THREAD ffffe000e9058600 Cid 0004.000c Teb: 0000000000000000 Win32Thread: 0000000000000000 WAIT: (Executive) KernelMode Non-Alertable fffff80148964140 SynchronizationEvent Not impersonating DeviceMap ffffc000da21a760 Owning Process ffffe000e9040700 Image: System Attached Process N/A Image: N/A Wait Start TickCount 19 Ticks: 10762 (0:00:02:48.156) Context Switch Count 1 IdealProcessor: 0 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address nt!PopIrpWorkerControl (0xfffff801487ba46c) Stack Init ffffd00023db2c90 Current ffffd00023db2850 Base ffffd00023db3000 Limit ffffd00023dad000 Call 0000000000000000 Priority 15 BasePriority 13 PriorityDecrement 32 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`23db2890 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`23db29d0 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`23db2a80 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`23db2b10 fffff801`487ba48e nt!KeWaitForSingleObject+0x375
45
ffffd000`23db2bd0 fffff801`4876d5a5 nt!PopIrpWorkerControl+0x22 ffffd000`23db2c10 fffff801`487d1626 nt!PspSystemThreadStartup+0x41 ffffd000`23db2c60 00000000`00000000 nt!KiStartSystemThread+0x16 .process /p /r 0 Implicit thread is now ffffe000`e9058600 The context is partially valid. Only x86 user-mode context is available. x86 context set Loading Kernel Symbols ............................................................... ................................................................ ...................................... Loading User Symbols Loading unloaded module list .............Unable to enumerate user-mode unloaded modules, NTSTATUS 0xC0000147 ************* Symbol Loading Error Summary ************** Module name Error SharedUserData No error - symbol load deferred You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded. You should also verify that your symbol search path (.sympath) is correct. # ChildEBP RetAddr Args to Child WARNING: Frame IP not in any known module. Following frames may be wrong. 00 00000000 00000000 00000000 00000000 00000000 0x487d0f16 Effective machine: x64 (AMD64) !thread @#Thread 1f;.thread /w @#Thread; .reload; kb 256; .effmach AMD64 Setting context for owner process... .process /p /r ffffe000e9040700 [...] THREAD ffffe000ec5c2840 Cid 12b0.12e0 Teb: 0000000000539000 Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable ffffe000ea90f760 SynchronizationEvent Not impersonating DeviceMap ffffc000db94eec0 Owning Process ffffe000ec373080 Image: OneDrive.exe Attached Process N/A Image: N/A Wait Start TickCount 10254 Ticks: 527 (0:00:00:08.234) Context Switch Count 8 IdealProcessor: 2 UserTime 00:00:00.000 KernelTime 00:00:00.000 Win32 Start Address 0x00000000719bbfb4 Stack Init ffffd000255a9c90 Current ffffd000255a9710 Base ffffd000255aa000 Limit ffffd000255a4000 Call 0000000000000000 Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5 Child-SP RetAddr Call Site ffffd000`255a9750 fffff801`487003ea nt!KiSwapContext+0x76 ffffd000`255a9890 fffff801`486ffe79 nt!KiSwapThread+0x15a ffffd000`255a9940 fffff801`486ffae5 nt!KiCommitThreadWait+0x149 ffffd000`255a99d0 fffff801`48a9f032 nt!KeWaitForSingleObject+0x375 ffffd000`255a9a90 fffff801`487d6ca3 nt!NtWaitForSingleObject+0xf2 ffffd000`255a9b00 00000000`6c4021bc nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffffd000`255a9b00) 00000000`0399ed08 00000000`6c402021 0x6c4021bc 00000000`0399ed10 00000023`777c854c 0x6c402021 00000000`0399ed18 00000000`00000023 0x00000023`777c854c 00000000`0399ed20 00000000`ee556126 0x23 00000000`0399ed28 00000000`03a9f37c 0xee556126 00000000`0399ed30 00000000`0399ed58 0x3a9f37c 00000000`0399ed38 00000000`006e1b88 0x399ed58 00000000`0399ed40 00000000`0000006e 0x6e1b88 00000000`0399ed48 00000000`00000000 0x6e .process /p /r 0 Implicit thread is now ffffe000`ec5c2840 WARNING: WOW context retrieval requires switching to the thread's process context. Use .process /p ffffe000`e9040700 to switch back. Implicit process is now ffffe000`ec373080 x86 context set Loading Kernel Symbols ...............................................................
46
................................................................ ...................................... Loading User Symbols PEB is paged out (Peb.Ldr = 00000000`00516018). Type ".hh dbgerr001" for details Loading unloaded module list .............Unable to enumerate user-mode unloaded modules, NTSTATUS 0xC0000147 Loading Wow64 Symbols ......................Unable to read NT module Base Name string at 00000000`006bb938 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006d6d2c - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006c1460 - NTSTATUS 0xC0000147 ....Unable to read NT module Base Name string at 00000000`006c1850 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006c13d0 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006c1a90 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006da118 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`00707204 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e6518 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006ddd30 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`00707bbc - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`00707d54 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006ddc58 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e6568 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`006ddca0 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd118 - NTSTATUS 0xC0000147 ...Unable to read NT module Base Name string at 00000000`0070b68e - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`0070c17c - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd598 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd2c8 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`0070c680 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`006dd238 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006df948 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd820 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd550 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd280 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`0070fa74 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e6248 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd478 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006df3c8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e5f78 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd1a8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd5e0 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006e5ed8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd6b8 - NTSTATUS 0xC0000147 .Unable to read NT module Base Name string at 00000000`006dd628 - NTSTATUS 0xC0000147 .. .Unable to read NT module Base Name string at 00000000`0070b574 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. .Unable to read NT module Base Name string at 00000000`0072ee18 - NTSTATUS 0xC0000147 ..Unable to read NT module Base Name string at 00000000`0074bb98 - NTSTATUS 0xC0000147 ....Unable to read NT module Base Name string at 00000000`00755830 - NTSTATUS 0xC0000147 Missing image name, possible paged-out or corrupt data. ........ ************* Symbol Loading Error Summary ************** Module name Error SharedUserData No error - symbol load deferred You can troubleshoot most symbol related issues by turning on symbol loading diagnostics (!sym noisy) and repeating the command that caused symbols to be loaded. You should also verify that your symbol search path (.sympath) is correct. # ChildEBP RetAddr Args to Child 00 03a9f824 748de111 00000434 00000000 00000000 ntdll!NtWaitForSingleObject+0xc 01 03a9f898 719fcba5 00000434 ffffffff 00000000 KERNELBASE!WaitForSingleObjectEx+0x91 02 03a9f8ac 719fb506 86add51e 007665f0 007665ec MSVCR120!Concurrency::details::ExternalContextBase::Block+0x37 [f:\dd\vctools\crt\crtw32\concrt\externalcontextbase.cpp @ 145] 03 03a9f918 7193ea79 00780c94 03fa25a8 007665e0 MSVCR120!Concurrency::details::_Condition_variable::wait+0xab [f:\dd\vctools\crt\crtw32\concrt\event.cpp @ 595] 04 03a9f94c 7193eb58 007665ec 007665f0 00000000 MSVCP120!do_wait+0x42 [f:\dd\vctools\crt\crtw32\stdcpp\thr\cond.c @ 56] *** ERROR: Symbol file could not be found. Defaulted to export symbols for SyncEngine.DLL 05 03a9f960 712fdaa8 007665ec 007665f0 6038d983 MSVCP120!_Cnd_wait+0x10 [f:\dd\vctools\crt\crtw32\stdcpp\thr\cond.c @ 81] WARNING: Stack unwind information not available. Following frames may be wrong.
47
06 03a9f994 71144017 00007530 00007530 007664c8 SyncEngine!QoS::ScenarioQosWrapper::GetApiId+0x1d999d 07 03a9f9ec 71145105 00007530 71375228 00007530 SyncEngine!QoS::ScenarioQosWrapper::GetApiId+0x1ff0c 08 03a9fa0c 711769ec 00007530 719bbfb4 71077444 SyncEngine!QoS::ScenarioQosWrapper::GetApiId+0x20ffa 09 03a9fa34 7117669b 03a9fa5b 6038da7b 719bbfb4 SyncEngine!QoS::ScenarioQosWrapper::GetApiId+0x528e1 0a 03a9fa6c 719bc01d 00000000 86add702 719bbfb4 SyncEngine!QoS::ScenarioQosWrapper::GetApiId+0x52590 0b 03a9faa4 719bc001 719bbfb4 03a9fac4 75f538f4 MSVCR120!_callthreadstartex+0x1b [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 376] 0c 03a9fab0 75f538f4 00786200 75f538d0 63db2846 MSVCR120!_threadstartex+0x7c [f:\dd\vctools\crt\crtw32\startup\threadex.c @ 354] 0d 03a9fac4 777b5e13 00786200 4f093fe7 00000000 KERNEL32!BaseThreadInitThunk+0x24 0e 03a9fb0c 777b5dde ffffffff 777db7e8 00000000 ntdll!__RtlUserThreadStart+0x2f 0f 03a9fb1c 00000000 719bbfb4 00786200 00000000 ntdll!_RtlUserThreadStart+0x1b Effective machine: x64 (AMD64) !thread @#Thread 1f;.thread /w @#Thread; .reload; kb 256; .effmach AMD64 Setting context for owner process... .process /p /r ffffe000ec373080 [...]
11.
Yet another way is to use !stacks command (the default version omits paged out stacks):
0: kd> !stacks Proc.Thread .Thread
Ticks
ThreadState Blocker [fffff80148a1ca40 Idle] 0.000000 fffff80148a1d740 ffffd5e3 RUNNING hal!HalProcessorIdle+0xf 0.000000 ffffd00023f1abc0 0000000 RUNNING hal!HalProcessorIdle+0xf 0.000000 ffffd00023f99bc0 0000000 RUNNING hal!HalProcessorIdle+0xf 0.000000 ffffd0002884cbc0 0000000 RUNNING nt!KiIdleLoop+0x11d [ffffe000e9040700 System] 4.000018 ffffe000e90f0040 ffffffed Blocked nt!PopFxEmergencyWorker+0x29 4.000020 ffffe000e90fa040 fffff7ca Blocked nt!KeRemovePriQueue+0x1b7 4.000024 ffffe000e90f6040 ffffd602 Blocked nt!ExpWorkerFactoryManagerThread+0x28 4.00003c ffffe000e90f7040 ffffd604 Blocked nt!MiModifiedPageWriter+0x3c2 4.000048 ffffe000e9150040 ffffd5f5 Blocked nt!MiZeroPageThread+0x752 4.000050 ffffe000e9172040 ffffd661 Blocked nt!CcQueueLazyWriteScanThread+0x96 4.000054 ffffe000e9176040 ffffe1f7 Blocked nt!CcAsyncReadWorker+0x231 4.000058 ffffe000e9175040 ffffffe8 Blocked nt!CcAsyncReadWorker+0x231 4.00005c ffffe000e9174040 ffffffe8 Blocked nt!CcAsyncReadWorker+0x231 4.000068 ffffe000e9177040 fffff106 Blocked nt!KeRemovePriQueue+0x1b7 4.000070 ffffe000e92c5040 fffffc46 Blocked nt!EtwpLogger+0xcb 4.000074 ffffe000e92e6040 ffffd657 Blocked nt!EtwpLogger+0xcb 4.000078 ffffe000e92ef040 ffffd707 Blocked nt!EtwpLogger+0xcb 4.00007c ffffe000e92f0040 ffffd759 Blocked nt!EtwpLogger+0xcb 4.000080 ffffe000e9306240 ffffd897 Blocked nt!EtwpLogger+0xcb 4.000084 ffffe000e9327840 ffffd6d6 Blocked nt!EtwpLogger+0xcb 4.000088 ffffe000e93d6040 ffffffe2 Blocked nt!EtwpLogger+0xcb 4.00008c ffffe000e93da440 ffffe310 Blocked nt!EtwpLogger+0xcb 4.000090 ffffe000e93dd040 ffffe552 Blocked nt!EtwpLogger+0xcb 4.000094 ffffe000e93fe040 ffffe23f Blocked nt!EtwpLogger+0xcb 4.000098 ffffe000e93ff040 ffffffe2 Blocked nt!EtwpLogger+0xcb 4.00009c ffffe000e9ca3500 ffffe1ed Blocked +0xffffe000e9d885c9 4.0000a0 ffffe000e9072040 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.0000a4 ffffe000e905a040 ffffffe1 Blocked nt!IopPassiveInterruptRealtimeWorker+0x16 4.0000a8 ffffe000e9cb0040 ffffffe1 Blocked nt!IopPassiveInterruptRealtimeWorker+0x16 4.0000ac ffffe000e92a2040 ffffffe1 Blocked nt!IopPassiveInterruptRealtimeWorker+0x16 4.0000b0 ffffe000e92a1040 ffffffe1 Blocked nt!IopPassiveInterruptRealtimeWorker+0x16 4.0000b8 ffffe000e9d17040 fffffd16 Blocked ACPI!ACPIWorkerThread+0x74 4.0000bc ffffe000ea92b040 fffff613 Blocked nt!KeRemovePriQueue+0x1b7 4.0000c0 ffffe000ea90e040 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.0000c4 ffffe000ea939040 ffffff66 Blocked pci!RootPmeEventDispatcher+0x8b 4.0000c8 ffffe000ea92e040 ffffff66 Blocked ACPI!PciRootBusBiosMethodDispatcherOnResume+0x51 *** ERROR: Module load completed but symbols could not be loaded for vmci.sys 4.0000cc ffffe000ea9d5040 ffffd6e9 Blocked vmci+0x8110 *** ERROR: Module load completed but symbols could not be loaded for vsock.sys 4.0000d0 ffffe000ea9d7380 fffff5ef Blocked vsock+0x4387 4.0000d4 ffffe000eaafb840 ffffd602 Blocked WdFilter!MpAsyncpWorkerThread+0x13d 4.0000d8 ffffe000eab33340 ffffde7d Blocked ndis!ndisThreadPoolTimerHandler+0x1f 4.0000e0 ffffe000eab36840 fffffe63 Blocked ndis!ndisReceiveWorkerThread+0xb7
48
4.0000e4 ffffe000eab37040 fffffe63 Blocked ndis!ndisReceiveWorkerThread+0xb7 4.0000e8 ffffe000eab38040 fffffe63 Blocked ndis!ndisReceiveWorkerThread+0xb7 4.0000ec ffffe000eab39040 fffffe63 Blocked ndis!ndisReceiveWorkerThread+0xb7 4.000108 ffffe000e9ca7040 fffffe50 Blocked watchdog!SMgrGdiCalloutThread+0x43 *** ERROR: Module load completed but symbols could not be loaded for vmhgfs.sys 4.00010c ffffe000e9ca8040 fffffe4c Blocked vmhgfs+0xd394 4.000110 ffffe000e9cc1040 fffffe4c Blocked vmhgfs+0xd394 4.000114 ffffe000e9cf1040 ffffded0 Blocked nt!AlpcpSignalAndWait+0x1d9 4.00012c ffffe000e9cbe840 fffffe43 Blocked dxgkrnl!DpiPowerArbiterThread+0x67 4.00014c ffffe000eae13080 ffffd705 Blocked nt!CmpLazyWriteWorker+0x3a 4.000150 ffffe000eae14080 ffffd8b8 Blocked nt!CmpLazyWriteWorker+0x3a 4.00015c ffffe000eae5e5c0 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.000160 ffffe000eae62040 ffffd5ff Blocked nt!KeRemovePriQueue+0x1b7 4.000164 ffffe000eae664c0 fffff106 Blocked nt!KeRemovePriQueue+0x1b7 4.000168 ffffe000eae68040 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.00016c ffffe000eae6a040 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.00019c ffffe000e92ea080 ffffd5e5 Blocked dxgmms1!VidSchiWaitForSchedulerEvents+0x1cc 4.0001a0 ffffe000e9cee080 fffff9ce Blocked dxgkrnl!BLTQUEUE::BltQueueWorker+0x1ae 4.0001a4 ffffe000eb1db080 ffffd6db Blocked BasicRender!WARPKMADAPTER::RunGPU+0x14d 4.0001a8 ffffe000eb1f8840 ffffd73b Blocked dxgmms2!VidSchiWaitForSchedulerEvents+0x21e 4.0001ac ffffe000eb70b840 ffffd795 Blocked dxgmms2!VIDMM_WORKER_THREAD::Run+0x117 *** WARNING: Unable to verify timestamp for msrpc.sys *** ERROR: Module load completed but symbols could not be loaded for msrpc.sys 4.000204 ffffe000eb78e080 ffffd608 Blocked nt!IoRemoveIoCompletion+0x8a 4.000294 ffffe000eb934840 ffffd6d6 Blocked luafv!SynchronousFsControl+0x175 4.0002cc ffffe000eb93c040 fffffd75 Blocked storqosflt!SqosJobDispatcherThreadRoutine+0x51 4.0005f0 ffffe000e9097040 fffffd4c Blocked HTTP!UlpScavengerThread+0xfc 4.000674 ffffe000eb65c840 fffffb1a Blocked mpsdrv!NseQueryExportTable+0x124 4.0006bc ffffe000eb69b080 fffffd2f Blocked nt!EtwpLogger+0xcb *** ERROR: Module load completed but symbols could not be loaded for vmmemctl.sys 4.0006d4 ffffe000eb6d5840 ffffd622 Blocked vmmemctl+0x22ea 4.000714 ffffe000eb6fd040 fffffd25 Blocked Ndu!NduTokenComputeTokensWorkerRoutine+0x7a 4.000728 ffffe000eba63040 ffffd5e8 Blocked mmcss!CiSchedulerThreadFunction+0x5a7 4.0007d4 ffffe000ebae3280 ffffd606 Blocked nt!SmKmStoreHelperWorker+0x46 4.0007d8 ffffe000ebae9840 ffffd606 Blocked nt!SmKmStoreHelperWorker+0x46 4.0007dc ffffe000ebb05080 ffffd607 Blocked nt!SMKM_STORE::SmStReadThread+0xaa 4.0007e0 ffffe000ebb07840 ffffd5f4 Blocked nt!SMKM_STORE::SmStWorker+0xe1 4.0007e4 ffffe000ebb08080 ffffee7c Blocked nt!KeRemovePriQueue+0x1b7 4.0007e8 ffffe000ebb09080 ffffd607 Blocked nt!MiStoreEvictThread+0xfa 4.00044c ffffe000ebb66840 fffffcff Blocked srv2!RfspThreadPoolNodeManagerRun+0x7a 4.000340 ffffe000ebb67040 fffffcff Blocked srv2!RfspThreadPoolNodeWorkerProcessWorkItems+0xc1 4.000494 ffffe000ebb68040 fffffcff Blocked srv2!RfspThreadPoolNodeManagerRun+0x7a 4.0004b0 ffffe000ebb69040 fffffcff Blocked srv2!RfspThreadPoolNodeWorkerProcessWorkItems+0xc1 4.0004d0 ffffe000ebb6a040 fffffcff Blocked srv2!RfspThreadPoolNodeManagerRun+0x7a 4.0004fc ffffe000ebb6b040 fffffcff Blocked srv2!RfspThreadPoolNodeWorkerProcessWorkItems+0xc1 4.000814 ffffe000ebbe4080 fffffcf2 Blocked nt!EtwpLogger+0xcb 4.00092c ffffe000eb4d5080 ffffd724 Blocked nt!EtwpLogger+0xcb 4.0009cc ffffe000eb588200 fffffc74 Blocked nt!EtwpLogger+0xcb 4.000b18 ffffe000ebc1d080 fffff686 Blocked nt!EtwpLogger+0xcb 4.000c94 ffffe000ebdfb840 ffffd6d6 Blocked nt!SmKmStoreHelperWorker+0x46 4.000c98 ffffe000ebdfc080 ffffd6d6 Blocked nt!SmKmStoreHelperWorker+0x46 4.000c9c ffffe000ebdff080 ffffe320 Blocked nt!SMKM_STORE::SmStReadThread+0xaa 4.000ca0 ffffe000ebe1c840 ffffd606 Blocked nt!SMKM_STORE::SmStWorker+0xe1 4.000d74 ffffe000ebecc300 fffff106 Blocked nt!KeRemovePriQueue+0x1b7 4.000d78 ffffe000ebece840 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.000d7c ffffe000ebed8040 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.000d80 ffffe000ebed9040 fffff106 Blocked nt!KeRemovePriQueue+0x1b7 4.000d84 ffffe000ebede300 fffff106 Blocked nt!KeRemovePriQueue+0x1b7 4.000d88 ffffe000ebee2040 fffff106 Blocked nt!KeRemovePriQueue+0x1b7 4.000d8c ffffe000ebf0e840 ffffd8dc Blocked nt!SmKmStoreHelperWorker+0x46 4.000d90 ffffe000ebf11080 ffffdce3 Blocked nt!SmKmStoreHelperWorker+0x46 4.000d94 ffffe000ebf2d080 ffffd8de Blocked nt!SMKM_STORE::SmStReadThread+0xaa 4.000d98 ffffe000ebf30840 ffffd8c4 Blocked nt!SMKM_STORE::SmStWorker+0xe1 4.000da8 ffffe000eae4f840 ffffd732 Blocked nt!KeRemovePriQueue+0x1b7 4.000dac ffffe000ebf46840 ffffdec0 Blocked nt!KeRemovePriQueue+0x1b7 4.000db0 ffffe000ebf0f840 ffffdebf Blocked nt!KeRemovePriQueue+0x1b7 4.000db4 ffffe000ebee5040 fffff106 Blocked nt!KeRemovePriQueue+0x1b7
49
4.000e48 4.000e4c 4.000e50 4.000e54 4.000e64 4.000e68 4.000e6c 4.000e70 4.000e74 4.000e78 4.000e7c 4.000e80 4.000e84 4.000e88 4.000e8c 4.000e90 4.000e94 4.000e98 4.000e9c 4.0013a4 4.0013a8 4.0013ac 4.0013b0 4.001138 4.001140 4.001100 4.0010fc 4.001344 4.001348 4.00134c 4.001350 4.000ef4 4.00109c 4.0010a0 4.001098 4.001420 4.001424 4.001428 4.00142c 4.0014b0 4.0014b4 4.0014b8 4.0014bc 4.001574 4.001028
ffffe000ebf57840 ffffe000ebeee080 ffffe000ebd34080 ffffe000ebefc080 ffffe000ebf7e040 ffffe000ebcec240 ffffe000ebc53040 ffffe000ebc7c040 ffffe000ebcf0840 ffffe000ec115040 ffffe000ec122040 ffffe000ec123040 ffffe000ec124040 ffffe000ec125840 ffffe000ec126040 ffffe000ec128040 ffffe000ec12a040 ffffe000ec12b840 ffffe000ec135840 ffffe000ebe2f040 ffffe000eb682840 ffffe000eb8ab300 ffffe000eb8f4040 ffffe000ec4e3080 ffffe000ebfde080 ffffe000ec4bd080 ffffe000ec1eb840 ffffe000ec62a080 ffffe000ec6d3080 ffffe000ec6cf080 ffffe000ec5da180 ffffe000ec7cd840 ffffe000ec410840 ffffe000ec48c840 ffffe000ec7ea840 ffffe000ec88f840 ffffe000ec872080 ffffe000ec3ca840 ffffe000ec8ac840 ffffe000ec95d840 ffffe000ec941840 ffffe000ec95b080 ffffe000ec97a840 ffffe000eb8b4040 ffffe000ec647840
ffffd605 ffffd605 ffffd8de ffffd604 fffff106 fffff613 fffff106 ffffd732 fffff7c9 fffff943 ffffdec0 ffffd5f4 ffffdebe ffffdec0 fffff613 ffffdebe fffff7c9 fffff106 fffff106 ffffd657 ffffd8d8 ffffd8d8 ffffd8d8 ffffd7b4 ffffd7b4 ffffe31e ffffd63e ffffde28 ffffde28 ffffd8dd ffffd782 ffffd6db ffffd6db ffffd8dd ffffd6db ffffd6dc ffffd6dc ffffd8dd ffffd6d7 ffffd793 ffffd6d9 ffffd8dd ffffd6d8 ffffee7c ffffdee0
Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked Blocked
nt!SmKmStoreHelperWorker+0x46 nt!SmKmStoreHelperWorker+0x46 nt!SMKM_STORE::SmStReadThread+0xaa nt!SMKM_STORE::SmStWorker+0xe1 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!KeRemovePriQueue+0x1b7 nt!SmKmStoreHelperWorker+0x46 nt!SmKmStoreHelperWorker+0x46 nt!SMKM_STORE::SmStReadThread+0xaa nt!SMKM_STORE::SmStWorker+0xe1 nt!SmKmStoreHelperWorker+0x46 nt!SmKmStoreHelperWorker+0x46 nt!SMKM_STORE::SmStReadThread+0xaa nt!SMKM_STORE::SmStWorker+0xe1 nt!SmKmStoreHelperWorker+0x46 nt!SmKmStoreHelperWorker+0x46 nt!SMKM_STORE::SmStReadThread+0xaa nt!SMKM_STORE::SmStWorker+0xe1 nt!SmKmStoreHelperWorker+0x46 nt!SmKmStoreHelperWorker+0x46 nt!SMKM_STORE::SmStReadThread+0xaa nt!SMKM_STORE::SmStWorker+0xe1 nt!SmKmStoreHelperWorker+0x46 nt!SmKmStoreHelperWorker+0x46 nt!SMKM_STORE::SmStReadThread+0xaa nt!SMKM_STORE::SmStWorker+0xe1 nt!KeRemovePriQueue+0x1b7 nt!EtwpLogger+0xcb
[ffffe000ead78840 smss.exe] 180.000190 180.0001fc 180.00023c 180.00040c 180.000788
[ffffe000eb239080 csrss.exe] ffffe000eb23f080 ffffd602 Blocked nt!AlpcpWaitForSingleObject+0x3e ffffe000eb78a080 ffffd602 Blocked nt!AlpcpWaitForSingleObject+0x3e ffffe000eb7cf080 ffffd622 Blocked win32kfull!RawInputThread+0x9aa ffffe000eb94a3c0 ffffd602 Blocked nt!AlpcpWaitForSingleObject+0x3e ffffe000eba97080 ffffd602 Blocked nt!AlpcpWaitForSingleObject+0x3e
1c8.000208
[ffffe000eb764840 wininit.exe] ffffe000eb78f080 ffffee8a Blocked nt!IoRemoveIoCompletion+0x8a
[...]
1594.0008cc 1594.001538 1594.001540 1594.001544 1594.00154c
[ffffe000ec09a080 NotMyfault.exe] ffffe000ecab7080 ffffd5e3 RUNNING nt!KeBugCheckEx ffffe000ec360080 ffffd705 Blocked nt!IoRemoveIoCompletion+0x8a ffffe000ec16e080 ffffd705 Blocked nt!IoRemoveIoCompletion+0x8a ffffe000ec97c840 ffffd704 Blocked nt!IoRemoveIoCompletion+0x8a ffffe000ec41f040 ffffd704 Blocked nt!IoRemoveIoCompletion+0x8a
50
1594.000614 1594.0017b0 1594.0017ac
ffffe000ec43a080 ffffd704 Blocked ffffe000ec474080 ffffd704 Blocked ffffe000ec475080 ffffd704 Blocked
nt!IoRemoveIoCompletion+0x8a nt!ObWaitForMultipleObjects+0x2bd nt!ObWaitForMultipleObjects+0x2bd
Threads Processed: 1185
12.
Let’s now check processes that were waiting for user input:
0: kd> !stacks 2 NtUserGetMessage Proc.Thread .Thread Ticks ThreadState Blocker [fffff80148a1ca40 Idle] [ffffe000e9040700 System] [ffffe000ead78840 smss.exe] [ffffe000eb239080 csrss.exe] [ffffe000eb764840 wininit.exe] [ffffe000eb21d840 csrss.exe] [ffffe000eb7a52c0 winlogon.exe] [ffffe000eb7e7080 services.exe] [ffffe000eb7f4080 lsass.exe] [ffffe000eb83e840 svchost.exe] [ffffe000eb84e080 svchost.exe] 354.000358
[ffffe000eb8a3080 dwm.exe] ffffe000eb8a6080 fffff713 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000eb8c06c0 svchost.exe] [ffffe000eb8c1400 svchost.exe] [ffffe000eb8fa780 svchost.exe] [ffffe000eb93f840 svchost.exe] [ffffe000eb9426c0 vmacthlp.exe] [ffffe000eb958840 WUDFHost.exe] [ffffe000eb95b840 svchost.exe]
484.000574
[ffffe000eb9a2840 svchost.exe] ffffe000eba4f080 fffff712 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98
51
win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000eb9f4080 svchost.exe] [ffffe000e90d6840 spoolsv.exe] [ffffe000e90dd840 svchost.exe] [ffffe000eb6f5080 svchost.exe] [ffffe000eba89840 svchost.exe] [ffffe000eba61080 vmtoolsd.exe] 774.0003e4
[ffffe000eba8f840 MsMpEng.exe] ffffe000eb667080 ffffde3c Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000eba8e740 VGAuthService.]
8bc.000908
8c4.0008c8
938.000968
[ffffe000eb4a6840 dllhost.exe] ffffe000eb4cb080 fffff711 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000eb4a2840 WmiPrvSE.exe] ffffe000eb4ac080 fffff711 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000eb561080 dllhost.exe] ffffe000eb575080 fffff711 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14
52
9ac.0009c8
[ffffe000e92795c0 msdtc.exe] ffffe000eb46e6c0 fffff711 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000e926d840 NisSrv.exe] [ffffe000ebd12840 VSSVC.exe]
b0c.000a78
ac4.000af8
[ffffe000eb601080 sihost.exe] ffffe000ebd6e080 fffff711 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ebd4e840 taskhostw.exe] ffffe000ebd85080 ffffd608 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ebdb36c0 userinit.exe]
c5c.0011d8
c5c.001780
c64.000ce4
[ffffe000ebdca840 RuntimeBroker.] ffffe000ec6be840 fffff0f6 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000eb956080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ebdc7840 explorer.exe] ffffe000ebe4d080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a
53
c64.000f64
ffffe000ec313080 ffffdb03
c64.000f68
ffffe000ec304080 ffffd867
c64.00088c
ffffe000ec209840 ffffeaf5
c64.0010a4
ffffe000ec305740 fffff17d
c64.001070
ffffe000ec2d9080 ffffd86c
c64.00106c
ffffe000eba82840 ffffd7b4
nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13
54
c64.001124
c64.000718
USER32!NtUserGetMessage+0x14 ffffe000ec593080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForMultipleObjects+0x24e win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000ecaa45c0 ffffd7b4 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ebdcd840 SkypeHost.exe] [ffffe000eb4195c0 SearchIndexer.]
d9c.000ea4
e58.000f50
e58.000f54
fbc.000fc0
[ffffe000ebf0a840 ShellExperienc] ffffe000ec129840 fffff710 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ebf00840 SearchUI.exe] ffffe000ec32b080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000ec31c840 fffff710 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec252080 TabTip.exe] ffffe000ec263840 fffff710 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98
55
fbc.000a30
win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000ec227840 ffffd5e3 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec121080 TabTip32.exe] [ffffe000ebd73840 svchost.exe]
10d8.0010dc
10d8.001630
[ffffe000ec2b4840 WmiPrvSE.exe] ffffe000ec2b6080 fffff710 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000eb9a8440 ffffd8cb Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000eb6c3080 vmtoolsd.exe]
12b0.0012b4
1050.000b5c
[ffffe000ec373080 OneDrive.exe] ffffe000ec367080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 +0x6c393824 [ffffe000ec24a080 ApplicationFra] ffffe000ec634080 ffffd735 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForMultipleObjects+0x24e win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14
56
[ffffe000ec491080 MicrosoftEdge.] [ffffe000ec220080 browser_broker] 1354.000cec
[ffffe000ec62c840 MicrosoftEdgeC] ffffe000ec5bd080 fffff0d1 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec6a8640 SearchProtocol] [ffffe000ec6c4080 SearchFilterHo]
105c.00101c
1430.00169c
1430.0016a4
14c0.0014f0
14c0.001504
[ffffe000ec77d840 MicrosoftEdgeC] ffffe000ec84d080 fffff044 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec88d840 MicrosoftEdgeC] ffffe000ebdbd080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000ebe76080 ffffeb33 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec944840 MicrosoftEdgeC] ffffe000ec99d080 ffffd5e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000ec9cf080 ffffe1e7 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a
57
14c0.001508
2c4.0002bc
nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 ffffe000ec9d0080 ffffeff5 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec156840 notepad.exe] ffffe000ec563080 ffffd842 Blocked nt!KiSwapContext+0x76 nt!KiSwapThread+0x15a nt!KiCommitThreadWait+0x149 nt!KeWaitForSingleObject+0x375 win32kfull!xxxRealSleepThread+0x355 win32kfull!xxxSleepThread2+0x98 win32kfull!xxxRealInternalGetMessage+0xb4d win32kfull!NtUserGetMessage+0x90 nt!KiSystemServiceCopyEnd+0x13 USER32!NtUserGetMessage+0x14 [ffffe000ec9aa540 audiodg.exe] [ffffe000eca66840 svchost.exe] [ffffe000ec089080 WmiApSrv.exe] [ffffe000ec8db080 TabTip.exe] [ffffe000ec09a080 NotMyfault.exe]
Threads Processed: 1185
13.
We can also list processes grouped by session id:
0: kd> !sprocess -4 Total sessions : 2 Session 0 _MM_SESSION_SPACE ffffd000250f3000 _MMSESSION ffffd000250f3b40 PROCESS ffffe000eb239080 SessionId: 0 Cid: 0180 Peb: 61467f1000 ParentCid: 0174 DirBase: 04466000 ObjectTable: ffffc000daca8040 HandleCount: Image: csrss.exe PROCESS ffffe000eb764840 SessionId: 0 Cid: 01c8 Peb: 8e26a86000 ParentCid: 0174 DirBase: 26eac000 ObjectTable: ffffc000dad79e80 HandleCount: Image: wininit.exe
58
PROCESS ffffe000eb7e7080 SessionId: 0 Cid: 0250 Peb: 721f3eb000 ParentCid: 01c8 DirBase: 03be0000 ObjectTable: ffffc000e2351040 HandleCount: Image: services.exe PROCESS ffffe000eb7f4080 SessionId: 0 Cid: 025c Peb: ea96f64000 ParentCid: 01c8 DirBase: 2f4fd000 ObjectTable: ffffc000e236b340 HandleCount: Image: lsass.exe PROCESS ffffe000eb83e840 SessionId: 0 Cid: 02b4 Peb: 16cdd6000 ParentCid: 0250 DirBase: 364d2000 ObjectTable: ffffc000dae60440 HandleCount: Image: svchost.exe PROCESS ffffe000eb84e080 SessionId: 0 Cid: 02ec Peb: baa1131000 ParentCid: 0250 DirBase: 3636b000 ObjectTable: ffffc000daeb6d80 HandleCount: Image: svchost.exe PROCESS ffffe000eb8c06c0 SessionId: 0 Cid: 0388 Peb: a6e152b000 ParentCid: 0250 DirBase: 35129000 ObjectTable: ffffc000daf83e80 HandleCount: Image: svchost.exe PROCESS ffffe000eb8c1400 SessionId: 0 Cid: 0390 Peb: 5f9c0e8000 ParentCid: 0250 DirBase: 34cb3000 ObjectTable: ffffc000daf8bdc0 HandleCount: Image: svchost.exe PROCESS ffffe000eb8fa780 SessionId: 0 Cid: 03f4 Peb: a47f49000 ParentCid: 0250 DirBase: 343d8000 ObjectTable: ffffc000db068040 HandleCount: Image: svchost.exe PROCESS ffffe000eb93f840 SessionId: 0 Cid: 0318 Peb: 60c92fc000 ParentCid: 0250 DirBase: 34e26000 ObjectTable: ffffc000db0ac840 HandleCount: Image: svchost.exe PROCESS ffffe000eb9426c0 SessionId: 0 Cid: 03d4 Peb: 002dd000 ParentCid: 0250 DirBase: 266ac000 ObjectTable: ffffc000db13ae80 HandleCount: Image: vmacthlp.exe PROCESS ffffe000eb958840 SessionId: 0 Cid: 0420 Peb: 8c3d62b000 ParentCid: 0390 DirBase: 2e32f000 ObjectTable: ffffc000db0c7d80 HandleCount: Image: WUDFHost.exe PROCESS ffffe000eb95b840 SessionId: 0 Cid: 0428 Peb: 110cd9c000 ParentCid: 0250 DirBase: 26779000 ObjectTable: ffffc000db0c5700 HandleCount: Image: svchost.exe PROCESS ffffe000eb9a2840 SessionId: 0 Cid: 0484 Peb: b5f9f4a000 ParentCid: 0250 DirBase: 2dbc4000 ObjectTable: ffffc000db0ff580 HandleCount:
59
Image: svchost.exe PROCESS ffffe000eb9f4080 SessionId: 0 Cid: 04dc Peb: 781ee21000 ParentCid: 0250 DirBase: 2c612000 ObjectTable: ffffc000db1c56c0 HandleCount: Image: svchost.exe PROCESS ffffe000e90d6840 SessionId: 0 Cid: 05f8 Peb: 003fa000 ParentCid: 0250 DirBase: 2a489000 ObjectTable: ffffc000db3648c0 HandleCount: Image: spoolsv.exe PROCESS ffffe000e90dd840 SessionId: 0 Cid: 0634 Peb: 7ed35ba000 ParentCid: 0250 DirBase: 2a3d4000 ObjectTable: ffffc000db1dde80 HandleCount: Image: svchost.exe PROCESS ffffe000eb6f5080 SessionId: 0 Cid: 0708 Peb: dd28a8a000 ParentCid: 0250 DirBase: 22a1d000 ObjectTable: ffffc000db53b040 HandleCount: Image: svchost.exe PROCESS ffffe000eba89840 SessionId: 0 Cid: 0748 Peb: 78deb31000 ParentCid: 0250 DirBase: 22373000 ObjectTable: ffffc000db55a940 HandleCount: Image: svchost.exe PROCESS ffffe000eba61080 SessionId: 0 Cid: 0754 Peb: 00392000 ParentCid: 0250 DirBase: 21e8b000 ObjectTable: ffffc000db55d780 HandleCount: Image: vmtoolsd.exe PROCESS ffffe000eba8f840 SessionId: 0 Cid: 0774 Peb: dc8dd81000 ParentCid: 0250 DirBase: 21fd9000 ObjectTable: ffffc000db568580 HandleCount: Image: MsMpEng.exe PROCESS ffffe000eba8e740 SessionId: 0 Cid: 077c Peb: 00233000 ParentCid: 0250 DirBase: 21a1e000 ObjectTable: ffffc000db572e80 HandleCount: Image: VGAuthService.exe PROCESS ffffe000eb4a6840 SessionId: 0 Cid: 08bc Peb: f1d1a3b000 ParentCid: 0250 DirBase: 05f4b000 ObjectTable: ffffc000daf36680 HandleCount: Image: dllhost.exe PROCESS ffffe000eb4a2840 SessionId: 0 Cid: 08c4 Peb: 35a123000 ParentCid: 02b4 DirBase: 01314000 ObjectTable: ffffc000db2b3240 HandleCount: Image: WmiPrvSE.exe PROCESS ffffe000eb561080 SessionId: 0 Cid: 0938 Peb: b01aa10000 ParentCid: 0250 DirBase: 055a4000 ObjectTable: ffffc000db9d8d80 HandleCount: Image: dllhost.exe
60
PROCESS ffffe000e92795c0 SessionId: 0 Cid: 09ac Peb: 6db75d4000 ParentCid: 0250 DirBase: 08fb8000 ObjectTable: ffffc000dba8e8c0 HandleCount: Image: msdtc.exe PROCESS ffffe000e926d840 SessionId: 0 Cid: 0a8c Peb: f58d62c000 ParentCid: 0250 DirBase: 166d8000 ObjectTable: ffffc000db7a9480 HandleCount: Image: NisSrv.exe PROCESS ffffe000ebd12840 SessionId: 0 Cid: 0bd8 Peb: bbb79eb000 ParentCid: 0250 DirBase: 00a69000 ObjectTable: ffffc000db7cc540 HandleCount: Image: VSSVC.exe PROCESS ffffe000eb4195c0 SessionId: 0 Cid: 0d6c Peb: abc12fc000 ParentCid: 0250 DirBase: 0cb17000 ObjectTable: ffffc000dc22be80 HandleCount: Image: SearchIndexer.exe PROCESS ffffe000ebd73840 SessionId: 0 Cid: 1090 Peb: 87553cb000 ParentCid: 0250 DirBase: 382e7000 ObjectTable: ffffc000dc62f8c0 HandleCount: Image: svchost.exe PROCESS ffffe000ec2b4840 SessionId: 0 Cid: 10d8 Peb: 2a11a5c000 ParentCid: 02b4 DirBase: 1f9af000 ObjectTable: ffffc000dc714340 HandleCount: Image: WmiPrvSE.exe PROCESS ffffe000ec6c4080 SessionId: 0 Cid: 1364 Peb: 4bddb6a000 ParentCid: 0d6c DirBase: 21a68000 ObjectTable: ffffc000dd698c40 HandleCount: Image: SearchFilterHost.exe PROCESS ffffe000ec9aa540 SessionId: 0 Cid: 0bd4 Peb: 9076394000 ParentCid: 0318 DirBase: 36e80000 ObjectTable: ffffc000dd7f4e80 HandleCount: Image: audiodg.exe PROCESS ffffe000ec089080 SessionId: 0 Cid: 0be0 Peb: cd7c56f000 ParentCid: 0250 DirBase: 17c24000 ObjectTable: ffffc000dd89e6c0 HandleCount: Image: WmiApSrv.exe
Session 1 _MM_SESSION_SPACE ffffd000251ac000 _MMSESSION ffffd000251acb40 PROCESS ffffe000eb21d840 SessionId: 1 Cid: 01d0 Peb: 27d00b2000 ParentCid: 01c0 DirBase: 2685f000 ObjectTable: ffffc000dad6fac0 HandleCount: Image: csrss.exe PROCESS ffffe000eb7a52c0 SessionId: 1 Cid: 021c Peb: 9668399000 ParentCid: 01c0 DirBase: 01165000 ObjectTable: ffffc000dad8fe80 HandleCount: Image: winlogon.exe
61
PROCESS ffffe000eb8a3080 SessionId: 1 Cid: 0354 Peb: e4fdc6b000 ParentCid: 021c DirBase: 3483c000 ObjectTable: ffffc000daf1e3c0 HandleCount: Image: dwm.exe PROCESS ffffe000eb601080 SessionId: 1 Cid: 0b0c Peb: d8ab44a000 ParentCid: 0388 DirBase: 3424e000 ObjectTable: ffffc000dbe8fb00 HandleCount: Image: sihost.exe PROCESS ffffe000ebd4e840 SessionId: 1 Cid: 0ac4 Peb: 4b7b3c1000 ParentCid: 0388 DirBase: 0a8da000 ObjectTable: ffffc000dbeaad00 HandleCount: Image: taskhostw.exe PROCESS ffffe000ebdb36c0 SessionId: 1 Cid: 0c40 Peb: f248fb000 ParentCid: 021c DirBase: 08820000 ObjectTable: 00000000 HandleCount: 0. Image: userinit.exe PROCESS ffffe000ebdc7840 SessionId: 1 Cid: 0c64 Peb: 0036a000 ParentCid: 0c40 DirBase: 08c52000 ObjectTable: ffffc000dbf4c880 HandleCount: Image: explorer.exe PROCESS ffffe000ebdca840 SessionId: 1 Cid: 0c5c Peb: 10ca437000 ParentCid: 02b4 DirBase: 08cad000 ObjectTable: ffffc000dbf687c0 HandleCount: Image: RuntimeBroker.exe PROCESS ffffe000ebdcd840 SessionId: 1 Cid: 0ca4 Peb: 0032e000 ParentCid: 02b4 DirBase: 0a510000 ObjectTable: ffffc000dbc48d80 HandleCount: Image: SkypeHost.exe PROCESS ffffe000ebf0a840 SessionId: 1 Cid: 0d9c Peb: 30f35e8000 ParentCid: 02b4 DeepFreeze DirBase: 0f0b5000 ObjectTable: ffffc000dc267840 HandleCount: Image: ShellExperienceHost.exe PROCESS ffffe000ebf00840 SessionId: 1 Cid: 0e58 Peb: fc6f501000 ParentCid: 02b4 DeepFreeze DirBase: 02e3a000 ObjectTable: ffffc000dc32f880 HandleCount: Image: SearchUI.exe PROCESS ffffe000ec252080 SessionId: 1 Cid: 0fbc Peb: d460f56000 ParentCid: 0390 DirBase: 16d28000 ObjectTable: ffffc000dc556880 HandleCount: Image: TabTip.exe PROCESS ffffe000ec121080 SessionId: 1 Cid: 0ff0 Peb: 04490000 ParentCid: 0fbc DirBase: 17c60000 ObjectTable: ffffc000dc583200 HandleCount: Image: TabTip32.exe
62
PROCESS ffffe000eb6c3080 SessionId: 1 Cid: 1228 Peb: 00307000 ParentCid: 0c64 DirBase: 28ac7000 ObjectTable: ffffc000dca7b440 HandleCount: Image: vmtoolsd.exe PROCESS ffffe000ec373080 SessionId: 1 Cid: 12b0 Peb: 00516000 ParentCid: 0c64 DirBase: 27369000 ObjectTable: ffffc000dd3bc840 HandleCount: Image: OneDrive.exe PROCESS ffffe000ec24a080 SessionId: 1 Cid: 1050 Peb: c4977ce000 ParentCid: 02b4 DirBase: 21d8b000 ObjectTable: ffffc000dd286c80 HandleCount: Image: ApplicationFrameHost.exe PROCESS ffffe000ec491080 SessionId: 1 Cid: 10f8 Peb: 9023376000 ParentCid: 02b4 DirBase: 13390000 ObjectTable: ffffc000dd546040 HandleCount: Image: MicrosoftEdge.exe PROCESS ffffe000ec220080 SessionId: 1 Cid: 1208 Peb: 6db6d66000 ParentCid: 02b4 DirBase: 1d195000 ObjectTable: ffffc000dd5ab6c0 HandleCount: Image: browser_broker.exe PROCESS ffffe000ec62c840 SessionId: 1 Cid: 1354 Peb: b3574a5000 ParentCid: 0c5c DeepFreeze DirBase: 1d545000 ObjectTable: ffffc000dd602bc0 HandleCount: Image: MicrosoftEdgeCP.exe PROCESS ffffe000ec6a8640 SessionId: 1 Cid: 1378 Peb: da2c2ac000 ParentCid: 0d6c DirBase: 17e34000 ObjectTable: ffffc000dd68b480 HandleCount: Image: SearchProtocolHost.exe PROCESS ffffe000ec77d840 SessionId: 1 Cid: 105c Peb: 35b6605000 ParentCid: 0c5c DeepFreeze DirBase: 2baf9000 ObjectTable: ffffc000dd680200 HandleCount: Image: MicrosoftEdgeCP.exe PROCESS ffffe000ec88d840 SessionId: 1 Cid: 1430 Peb: 9b37cc6000 ParentCid: 0c5c DirBase: 39846000 ObjectTable: ffffc000dca0b040 HandleCount: Image: MicrosoftEdgeCP.exe PROCESS ffffe000ec944840 SessionId: 1 Cid: 14c0 Peb: 9af0d18000 ParentCid: 0c5c DirBase: 05df7000 ObjectTable: ffffc000dc647440 HandleCount: Image: MicrosoftEdgeCP.exe PROCESS ffffe000ec156840 SessionId: 1 Cid: 02c4 Peb: f3e927f000 ParentCid: 0c64 DirBase: 04d22000 ObjectTable: ffffc000dd7a2d40 HandleCount: Image: notepad.exe
63
PROCESS ffffe000eca66840 SessionId: 1 Cid: 03a8 Peb: 6389def000 ParentCid: 0250 DirBase: 388ee000 ObjectTable: ffffc000db8fde80 HandleCount: Image: svchost.exe PROCESS ffffe000ec8db080 SessionId: 1 Cid: 05ac Peb: b6c6ea8000 ParentCid: 0390 DirBase: 3a507000 ObjectTable: 00000000 HandleCount: 0. Image: TabTip.exe PROCESS ffffe000ec09a080 SessionId: 1 Cid: 1594 Peb: 00379000 ParentCid: 0c64 DirBase: 3cfce000 ObjectTable: ffffc000dd91c2c0 HandleCount: Image: NotMyfault.exe
14.
We close logging before exiting WinDbg:
0: kd> .logclose Closing open log file F:\AdvWMDA-Dumps\x64\C1.log
Note: To avoid possible confusion and glitches we recommend exiting WinDbg after each exercise.
64