Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

HyperDbg: Reinventing Hardware-Assisted Debugging

Version 1 : Received: 30 May 2022 / Approved: 31 May 2022 / Online: 31 May 2022 (09:41:16 CEST)

How to cite: Karvandi, M.S.; Gholamrezaei, M.H.; Khalaj Monfared, S.; Medi, S.; Abbassi, B.; Amini, A.; Mortazavi, R.; Gogin, S.; Rahmati, D.; Schwarz, M. HyperDbg: Reinventing Hardware-Assisted Debugging. Preprints 2022, 2022050416 (doi: 10.20944/preprints202205.0416.v1). Karvandi, M.S.; Gholamrezaei, M.H.; Khalaj Monfared, S.; Medi, S.; Abbassi, B.; Amini, A.; Mortazavi, R.; Gogin, S.; Rahmati, D.; Schwarz, M. HyperDbg: Reinventing Hardware-Assisted Debugging. Preprints 2022, 2022050416 (doi: 10.20944/preprints202205.0416.v1).

Abstract

Software analysis, debugging, and reverse engineering have a crucial impact in today’s software industry. Efficient and stealthy debuggers are especially relevant for malware analysis. However, existing debugging platforms fail to address a transparent, effective, and high-performance low-level debugger due to their detectable fingerprints, complexity, and implementation restrictions. In this paper, we present HyperDbg, a new hypervisor-assisted debugger for high-performance and stealthy debugging of user and kernel applications. To accomplish this, HyperDbg relies on state-of-the-art hardware features available in today’s CPUs, such as VT-x and extended page tables. In contrast to other widely used existing debuggers, we design HyperDbg using a custom hypervisor, making it independent of OS functionality or API. We propose hardware-based instruction-level emulation and OS-level API hooking via extended page tables to increase the stealthiness. Our results of the dynamic analysis of 10,853 malware samples show that HyperDbg’s stealthiness allows debugging on average 22% and 26% more samples than WinDbg and x64dbg, respectively. Moreover, in contrast to existing debuggers, HyperDbg is not detected by any of the 13 tested packers and protectors. We improve the performance over other debuggers by deploying a VMX-compatible script engine, eliminating unnecessary context switches. Our experiment on three concrete debugging scenarios shows that compared to WinDbg as the only kernel debugger, HyperDbg performs step-in, conditional breaks, and syscall recording, 2.98x, 1319x, and 2018x faster, respectively. We finally show real-world applications, such as a 0-day analysis, structure reconstruction for reverse engineering, software performance analysis, and code-coverage analysis.

Keywords

Hypervisor; Debugging; Kernel-debugger; Fuzzing

Subject

MATHEMATICS & COMPUTER SCIENCE, Information Technology & Data Management

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.