HyperEvade: Countering Anti-Debugging Techniques and Enhancing Transparency in Nested Virtualization using HyperDbg

By: Björn Ruytenberg, Mohammad Sina Karvandi

Abstract

Modern malware increasingly employs sophisticated anti-debugging and anti-virtualization techniques to evade analysis. HyperDbg, an open-source hypervisor-level debugger, introduces advanced mechanisms to mitigate both its own hypervisor footprints and those of the underlying nested virtualization stack. In this paper, we demonstrate the capabilities of adding a transparency layer on top of the HyperDbg debugger to detect, mitigate, and bypass common and advanced anti-debugging methods, significantly raising the bar for malware attempting to detect analysis environments.

Keywords

Anti-Debugging, Anti-Virtualization, Nested Virtualization, Debugging Malware, Binary Analysis

Cite as:

Björn Ruytenberg, Mohammad Sina Karvandi, “HyperEvade: Countering Anti-Debugging Techniques and Enhancing Transparency in Nested Virtualization using HyperDbg”, Journal of Object Technology, Volume 25, no. 1 ( 2026), pp. 1:1-3, doi:10.5381/jot.2026.25.1.a8.

DOI | BiBTeX | Tweet this | Post to CiteULike | Share on LinkedIn