Mastering Malware Analysis 9781789610789
2,370 531 25MB
English Pages 868 Year 2019
Table of contents :
Title Page......Page 2
Copyright and Credits......Page 3
Mastering Malware Analysis......Page 4
About Packt......Page 5
Why subscribe?......Page 6
Contributors......Page 7
About the authors......Page 8
About the reviewers......Page 9
Packt is searching for authors like you......Page 11
Preface......Page 26
Who this book is for......Page 27
What this book covers......Page 28
To get the most out of this book......Page 31
Download the example code files......Page 32
Download the color images......Page 33
Conventions used......Page 34
Get in touch......Page 35
Reviews......Page 36
Section 1: Fundamental Theory......Page 37
A Crash Course in CISC/RISC and Programming Basics......Page 38
Basic concepts......Page 40
Registers......Page 41
Memory......Page 42
Virtual memory......Page 43
Stack......Page 45
Branches, loops, and conditions......Page 46
Exceptions, interrupts, and communicating with other devices......Page 47
Assembly languages......Page 48
CISC versus RISC......Page 49
Types of instructions......Page 50
Becoming familiar with x86 (IA-32 and x64)......Page 51
Registers......Page 52
Special registers......Page 54
The instruction structure......Page 55
opcode......Page 56
dest......Page 57
src......Page 58
The instruction set......Page 59
Data manipulation instructions......Page 60
Data transfer instructions......Page 62
Flow control instructions......Page 64
Arguments, local variables, and calling conventions (in x86 and x64)......Page 65
stdcall......Page 66
Arguments......Page 67
Local variables......Page 69
cdecl......Page 71
fastcall......Page 72
thiscall......Page 73
The x64 calling convention......Page 74
Exploring ARM assembly......Page 75
Basics......Page 78
Instruction sets......Page 82
Basics of MIPS......Page 85
Basics......Page 87
The instruction set......Page 89
Diving deep into PowerPC......Page 92
Basics......Page 93
The instruction set......Page 95
Covering the SuperH assembly......Page 97
Basics......Page 98
The instruction set......Page 99
Working with SPARC......Page 101
Basics......Page 102
The instruction set......Page 104
From assembly to high-level programming languages......Page 105
Arithmetic statements......Page 106
If conditions......Page 108
While loop conditions......Page 111
Summary......Page 112
Section 2: Diving Deep into Windows Malware......Page 113
Basic Static and Dynamic Analysis for x86/x64......Page 114
Working with the PE header structure......Page 115
Why PE?......Page 116
Exploring PE structure......Page 117
MZ header......Page 118
PE header......Page 119
File header......Page 120
Optional header......Page 121
Data directory......Page 123
Section table......Page 124
PE+ (x64 PE)......Page 125
PE analysis tools......Page 126
Static and dynamic linking......Page 129
Static linking......Page 130
Dynamic linking......Page 131
Dynamic link libraries......Page 132
Application programming interface......Page 133
Dynamic API loading......Page 134
Using PE header information for static analysis......Page 135
How to use PE header for incident handling......Page 136
How to use a PE header for threat intelligence......Page 138
PE loading and process creation......Page 141
Basic terminology......Page 142
What's process?......Page 143
Virtual memory to physical memory mapping......Page 145
Threads......Page 147
Important data structures: TIB, TEB, and PEB......Page 149
Process loading step by step......Page 150
PE file loading step by step......Page 151
WOW64 processes......Page 154
Dynamic analysis with OllyDbg/immunity debugger......Page 156
Debugging tools......Page 157
How to analyze a sample with OllyDbg......Page 161
Types of breakpoints......Page 167
Step into/step over breakpoint......Page 168
INT3 breakpoint......Page 169
Memory breakpoints......Page 170
Hardware breakpoints......Page 171
Modifying the program execution......Page 172
Patching—modifying the program's assembly instructions......Page 173
Change EFlags......Page 174
Modifying the instruction pointer value......Page 175
Changing the program data......Page 176
Debugging malicious services......Page 177
What is service?......Page 178
Attaching to the service......Page 180
Summary......Page 182
Unpacking, Decryption, and Deobfuscation......Page 183
Exploring packers......Page 184
Exploring packing and encrypting tools......Page 186
Identifying a packed sample......Page 188
Technique 1 – checking PE tool static signatures......Page 189
Technique 2 – evaluating PE section names......Page 190
Technique 3 – using stub execution signs......Page 191
Technique 4 – detecting a small import table......Page 192
Automatically unpacking packed samples......Page 193
Technique 1 – the official unpacking process......Page 194
Technique 2 – using OllyScript with OllyDbg......Page 195
Technique 3 – using generic unpackers......Page 196
Technique 4 – emulation......Page 199
Technique 5 – memory dumps......Page 200
Manual unpacking using OllyDbg......Page 201
Technique 6 – memory breakpoint on execution......Page 202
Step 1 – setting the breakpoints......Page 203
Step 2 – turning on Data Execution Prevention......Page 204
Step 3 – preventing any further attempts to change memory permissions......Page 207
Step 4 – executing and getting the OEP......Page 208
Technique 7 – call stack backtracing......Page 210
Step 1 – setting the breakpoints......Page 213
Step 2 – following the call stack......Page 214
Step 3 – reaching the OEP......Page 215
Technique 8 – monitoring memory allocated spaces for unpacked code......Page 216
Technique 9 – in-place unpacking......Page 219
Technique 10 – stack restoration based......Page 220
Dumping the unpacked sample and fixing the import table......Page 221
Dumping the process......Page 222
Fixing the import table......Page 224
Identifying different encryption algorithms and functions......Page 228
Types of encryption algorithms......Page 229
Basic encryption algorithms......Page 231
How to identify encryption functions......Page 232
String search detection techniques for simple algorithms......Page 234
The basics of X-RAYING......Page 235
Simple static encryption......Page 236
Other encryption algorithms......Page 237
X-RAYING tools for malware analysis and detection......Page 238
Identifying the RC4 encryption algorithm......Page 240
The RC4 encryption algorithm......Page 241
Key-scheduling algorithm......Page 242
Pseudo-random generation algorithm......Page 243
Identifying RC4 algorithms in a malware sample......Page 244
Standard symmetric and asymmetric encryption algorithms......Page 246
Extracting information from Windows cryptography APIs......Page 247
Step 1 – initializing and connecting to the cryptographic service provider (CSP)......Page 248
Step 2 – preparing the key......Page 249
Step 3 – encrypting or decrypting the data......Page 251
Step 4 – freeing the memory......Page 252
Cryptography API next generation (CNG)......Page 253
Applications of encryption in modern malware – Vawtrak banking Trojan......Page 254
String and API name encryption......Page 255
Network communication encryption......Page 261
Using IDA for decryption and unpacking......Page 264
IDA tips and tricks......Page 265
Static analysis......Page 266
Dynamic analysis......Page 271
Classic and new syntax of IDA scripts......Page 274
Dynamic string decryption......Page 278
Dynamic WinAPIs resolution......Page 279
Summary......Page 280
Inspecting Process Injection and API Hooking......Page 281
Understanding process injection......Page 282
What's process injection?......Page 283
Why process injection?......Page 284
DLL injection......Page 285
Windows-supported DLL injection......Page 286
A simple DLL injection technique......Page 290
Working with process injection......Page 292
Getting the list of running processes......Page 293
Code injection......Page 295
Advanced code injection-reflective DLL injection......Page 297
Stuxnet secret technique-process hollowing......Page 299
Dynamic analysis of code injection......Page 302
Technique 1—debug it where it is......Page 303
Technique 2—attach to the targeted process......Page 305
Technique 3—dealing with process hollowing......Page 306
Memory forensics techniques for process injection......Page 308
Technique 1—detecting code injection and reflective DLL injection ......Page 309
Technique 2—detecting process hollowing......Page 312
Technique 3—detecting process hollowing using the HollowFind plugin......Page 314
Understanding API hooking......Page 316
Why API hooking?......Page 317
Working with API hooking......Page 318
Inline API hooking......Page 319
Inline API hooking with trampoline......Page 320
Inline API hooking with a length disassembler......Page 322
Detecting API hooking using memory forensics......Page 324
Exploring IAT hooking......Page 325
Summary......Page 327
Bypassing Anti-Reverse Engineering Techniques......Page 328
Exploring debugger detection......Page 329
Direct check for debugger presence......Page 330
Detecting a debugger through an environment change......Page 331
Detecting a debugger using parent processes......Page 332
Handling debugger breakpoints evasion......Page 334
Detecting software breakpoints (INT3)......Page 335
Detecting single-stepping breakpoints (trap flag)......Page 338
Detecting a trap flag using the SS register......Page 339
Detecting single-stepping using timing techniques......Page 341
Evading hardware breakpoints......Page 343
What is structured exception handling?......Page 344
Detecting and removing hardware breakpoints......Page 347
Memory breakpoints......Page 348
Escaping the debugger......Page 349
Process injection......Page 350
TLS callbacks......Page 351
Windows events callbacks......Page 353
Obfuscation and anti-disassemblers......Page 354
Encryption......Page 355
Junk code insertion......Page 356
Code transportation......Page 358
Dynamic API calling with checksum......Page 360
Proxy functions and proxy argument stacking......Page 361
Detecting and evading behavioral analysis tools......Page 363
Finding the tool process......Page 364
Searching for the tool window......Page 367
Detecting sandboxes and virtual machines......Page 369
Different output between virtual machines and real machines......Page 370
Detecting virtualization processes and services......Page 371
Detecting virtualization through registry keys......Page 372
Detecting virtual machines using PowerShell......Page 373
Detecting sandboxes by using default settings......Page 374
Other techniques......Page 375
Summary......Page 376
Understanding Kernel-Mode Rootkits......Page 377
Kernel mode versus user mode......Page 378
Protection rings......Page 379
Windows internals......Page 381
The infrastructure of Windows......Page 382
The execution path from user mode to kernel mode......Page 386
Rootkits and device drivers......Page 389
What is a rootkit?......Page 390
Types of rootkits......Page 391
What is a device driver?......Page 392
Hooking mechanisms......Page 393
SSDT hooking......Page 396
Hooking the SYSENTER entry function......Page 397
Modifying SSDT in an x86 environment......Page 399
Modifying SSDT in an x64 environment......Page 402
Hooking SSDT functions......Page 404
IRP hooking......Page 405
Devices and major functions......Page 406
Attaching to a device......Page 408
Modifying the IRP response and setting a completion routine......Page 409
DKOM......Page 410
The kernel objects—EPROCESS and ETHREAD......Page 411
How do rootkits perform an object manipulation attack?......Page 413
Process injection in kernel mode......Page 416
Executing the inject code using APC queuing......Page 421
KPP in x64 systems (PatchGuard)......Page 424
Bypassing driver signature enforcement......Page 425
Bypassing PatchGuard—the Turla example......Page 426
Bypassing PatchGuard—GhostHook......Page 427
Disabling PatchGuard using the Command Prompt......Page 428
Static and dynamic analysis in kernel mode......Page 429
Static analysis......Page 430
Tools......Page 431
Tips and tricks......Page 432
Dynamic and behavioral analysis......Page 433
Tools......Page 434
Monitors......Page 436
Rootkit detectors......Page 437
Setting up a testing environment......Page 438
Setting up the debugger......Page 441
Stopping at the driver's entrypoint......Page 445
Loading the driver......Page 449
Restoring the debugging state......Page 450
Summary......Page 451
Section 3: Examining Cross-Platform Malware......Page 452
Handling Exploits and Shellcode......Page 453
Getting familiar with vulnerabilities and exploits......Page 454
Types of vulnerabilities......Page 455
Stack overflow vulnerability......Page 456
Heap overflow vulnerabilities......Page 458
The use-after-free vulnerability......Page 459
Logical vulnerabilities......Page 460
Types of exploits......Page 461
Cracking the shellcode......Page 463
What's shellcode?......Page 464
Linux shellcode in x86-64......Page 465
Getting the absolute address......Page 466
Null-free shellcode......Page 467
Local shell shellcode......Page 468
Reverse shell shellcode......Page 470
Linux shellcode for ARM......Page 473
Null-free shellcode......Page 474
Windows shellcode......Page 475
Getting the Kernel32.dll ImageBase......Page 476
Getting the required APIs from Kernel32.dll......Page 477
The download and execute shellcode......Page 480
Static and dynamic analysis of exploits......Page 481
Analysis workflow......Page 482
Shellcode analysis......Page 484
Exploring bypasses for exploit mitigation technologies......Page 485
Data execution prevention (DEP/NX)......Page 486
Return-oriented programming......Page 487
Address space layout randomization......Page 489
DEP and partial ASLR......Page 490
DEP and full ASLR – partial ROP and chaining multiple vulnerabilities......Page 491
DEP and full ASLR – heap spray technique......Page 493
Other mitigation technologies......Page 494
Analyzing Microsoft Office exploits......Page 495
File structures......Page 496
Compound file binary format......Page 497
Rich text format......Page 499
Office open XML format......Page 500
Static and dynamic analysis of MS Office exploits......Page 501
Static analysis......Page 502
Dynamic analysis......Page 504
Studying malicious PDFs......Page 505
File structure......Page 506
Static and dynamic analysis of PDF files......Page 509
Static analysis......Page 510
Dynamic analysis......Page 513
Summary......Page 514
Reversing Bytecode Languages: .NET, Java, and More......Page 515
Exploring the theory of bytecode languages......Page 516
Object-oriented programming......Page 517
Inheritance......Page 518
Polymorphism......Page 519
.NET explained......Page 520
.NET file structure......Page 521
.NET COR20 header......Page 522
Metadata streams......Page 524
How to identify a .NET application from PE characteristics......Page 525
The CIL language instruction set......Page 527
Pushing into stack instructions......Page 528
Pulling out a value from the stack......Page 530
Mathematical and logical operations......Page 531
Branching instructions......Page 532
CIL language to higher-level languages......Page 533
Local variable assignments......Page 534
Local variable assignment with a method return value......Page 535
Basic branching statements......Page 536
Loops statements......Page 537
.NET malware analysis......Page 538
.NET analysis tools......Page 539
Static and dynamic analysis (with Dnspy)......Page 540
.NET static analysis......Page 541
.NET dynamic analysis......Page 543
Patching a .NET sample......Page 544
Dealing with obfuscation......Page 545
Obfuscated names for classes, methods, and others......Page 546
Encrypted strings inside the binary......Page 548
The sample is obfuscated using an obfuscator......Page 550
The essentials of Visual Basic......Page 552
File structure......Page 553
P-code versus native code......Page 557
Common p-code instructions......Page 560
Dissecting Visual Basic samples......Page 562
Static analysis......Page 563
P-code......Page 564
Native code......Page 566
Dynamic analysis......Page 569
P-code......Page 570
Native code......Page 571
The internals of Java samples......Page 572
File structure......Page 573
JVM instructions......Page 575
Static analysis......Page 576
Dynamic analysis......Page 579
Dealing with anti-reverse engineering solutions......Page 580
Python—script language internals......Page 581
File structure......Page 582
Bytecode instructions......Page 584
Analyzing compiled Python......Page 586
Static analysis......Page 587
Dynamic analysis......Page 589
Summary......Page 590
Scripts and Macros: Reversing, Deobfuscation, and Debugging......Page 591
Classic shell script languages......Page 592
Windows batch scripting......Page 593
Bash......Page 596
VBScript explained......Page 598
Basic syntax......Page 599
Static and dynamic analysis......Page 603
Deobfuscation......Page 606
Those evil macros inside documents......Page 607
Basic syntax......Page 608
Static and dynamic analysis......Page 610
Besides macros......Page 611
The power of PowerShell......Page 613
Basic syntax......Page 614
Static and dynamic analysis......Page 618
Handling JavaScript......Page 621
Basic syntax......Page 622
Static and dynamic analysis......Page 624
Anti-reverse engineering tricks......Page 630
Behind C&C—even malware has its own backend......Page 631
Things to focus on......Page 632
Static and dynamic analysis......Page 633
Other script languages......Page 634
Where to start from......Page 635
Questions to answer......Page 636
Summary......Page 637
Section 4: Looking into IoT and Other Platforms......Page 638
Dissecting Linux and IoT Malware......Page 639
Explaining ELF files ......Page 640
ELF structure......Page 641
System calls......Page 643
Filesystem......Page 644
Network......Page 645
Process management......Page 646
Other......Page 647
Syscalls in assembly......Page 648
Common anti-reverse engineering tricks......Page 652
Exploring common behavioral patterns......Page 653
Initial delivery and lateral movement......Page 654
Persistence......Page 656
Privilege escalation......Page 658
Interaction with the command and control server......Page 659
Attacking stage......Page 661
Static and dynamic analysis of x86 (32- and 64-bit) samples......Page 662
Static analysis......Page 663
File type detectors......Page 664
Data carving......Page 665
Disassemblers......Page 666
Actual tools......Page 667
Engines......Page 673
How to choose......Page 674
Dynamic analysis......Page 675
Tracers......Page 676
Network monitors......Page 677
Debuggers......Page 678
Binary emulators......Page 679
Radare2 cheat sheet......Page 680
Anti-reverse engineering techniques......Page 684
Learning Mirai, its clones, and more......Page 685
High-level functionality......Page 686
Propagation......Page 687
Weaponry......Page 688
Self-defense......Page 689
Later derivatives......Page 690
Other widespread families......Page 692
Static and dynamic analysis of RISC samples......Page 694
ARM......Page 697
MIPS......Page 699
PowerPC......Page 700
SuperH......Page 702
SPARC......Page 703
Handling other architectures......Page 704
What to start from......Page 705
Summary......Page 706
Introduction to macOS and iOS Threats......Page 707
Understanding the role of the security model......Page 708
macOS......Page 709
Security policies......Page 710
Filesystem hierarchy and encryption......Page 711
Directory structure......Page 712
Encryption......Page 713
Apps protection......Page 714
Gatekeeper......Page 715
App sandbox......Page 716
Other technologies......Page 718
iOS......Page 719
System security......Page 720
Data encryption and password management......Page 722
Apps' security......Page 725
File formats and APIs......Page 727
Mach-O......Page 728
Thin......Page 729
Fat......Page 732
Application bundles (.app)......Page 734
Info.plist......Page 735
macOS......Page 736
iOS......Page 737
Installer packages (.pkg)......Page 738
Apple disk images (.dmg)......Page 739
iOS app store packages (.ipa)......Page 740
APIs......Page 741
Static and dynamic analyses of macOS and iOS samples......Page 744
Static analysis......Page 745
Retrieving samples......Page 746
Disassemblers and decompilers......Page 747
Auxiliary tools and libraries......Page 749
Dynamic and behavioral analysis......Page 750
macOS......Page 751
Debuggers......Page 752
Monitoring and dynamic instrumentation......Page 754
Network analysis......Page 756
iOS......Page 757
Installers and loaders......Page 758
Debuggers......Page 760
Dumping and decryption......Page 761
Monitors and in-memory patching......Page 762
Network analysis......Page 763
Attack stages......Page 764
Jailbreaks on demand......Page 765
Penetration......Page 767
Deployment and persistence......Page 769
macOS......Page 770
iOS......Page 772
Action phase......Page 773
macOS......Page 774
iOS......Page 777
Other attack techniques......Page 779
macOS......Page 780
iOS......Page 781
Advanced techniques......Page 783
Anti-reverse-engineering (RE) tricks......Page 784
Misusing dynamic data exchange (DDE)......Page 785
User hiding......Page 786
Use of AppleScript......Page 787
API hijacking......Page 788
Rootkits for Mac—do they exist?......Page 789
Analysis workflow......Page 790
Summary......Page 792
Analyzing Android Malware Samples......Page 793
(Ab)using Android internals ......Page 794
File hierarchy......Page 795
Android security model......Page 798
Process management......Page 799
Filesystem......Page 800
App permissions......Page 801
Security services......Page 803
Console......Page 804
To root or not to root?......Page 807
Understanding Dalvik and ART ......Page 810
Dalvik VM (DVM)......Page 811
Android runtime (ART)......Page 812
APIs......Page 817
File formats......Page 819
DEX......Page 820
ODEX......Page 823
OAT......Page 824
VDEX......Page 825
ART......Page 826
ELF......Page 827
APK......Page 828
Bytecode set......Page 831
Malware behavior patterns......Page 837
Attack stages......Page 838
Penetration......Page 839
Deployment......Page 840
Action phase......Page 841
Advanced techniques—investment pays off......Page 844
Patching system libraries......Page 845
Keylogging......Page 846
Self-defense......Page 847
Rootkits—get it covered......Page 848
Static and dynamic analysis of threats......Page 849
Static analysis......Page 850
Disassembling and data extraction......Page 851
Decompiling......Page 854
Dynamic analysis......Page 857
Android debug bridge......Page 858
Emulators......Page 860
Behavioral analysis and tracing......Page 861
Debuggers......Page 862
Analysis workflow......Page 863
Summary......Page 865
Other Books You May Enjoy......Page 866
Leave a review - let other readers know what you think......Page 868
![SEC501.5: Malware [A12_02 ed.]](https://dokumen.pub/img/200x200/sec5015-malware-a1202nbsped.jpg)
