Hardware Hypervisor and Dataflow Tracking for Enhanced System Control and Monitoring, 10-R9848

Printer Friendly Version

Principal Investigators
Galen Rasche
Mark Brooks
Tam Do
Joe Loomis
Michael LeMay
Steve Cook
Ben Abbott

Inclusive Dates:  08/20/08 – 01/31/10

Background - The increased capabilities and flexibility of computer platforms and applications inherently lead to security vulnerabilities. Operating system (OS) security issues have helped to drive development of virtual machines (VM) and virtual machine monitors (VMM). VM technology limits the effects that a compromised OS can have on overall system security. Additionally, the VMM provides complete control over the OS, improving system monitoring and malware prevention. However, software VMMs are still vulnerable to attack, and their security is difficult to verify. Further, VMMs cannot understand dataflow semantics in individual processes, limiting the effectiveness of system monitoring. A hardware hypervisor would be more resistant to attack and amenable to security analysis than software-only hypervisors, providing a path forward for integration with mechanisms for monitoring and controlling the security state of applications, such as dynamic dataflow analysis (DDFA) and process coloring (PC). Enhancing the security in the layer below the VMM and increasing visibility into application behavior will significantly improve the end-to-end security of computer systems over the current state of the art.

Approach - The goal of this project is to leverage emerging research in component-level technologies, address known shortcomings and evaluate methods for combining the technologies to improve security at the system level. This goal will be pursued via two primary objectives. The first objective is to create a hardware-based, extensible, provably secure root-of-trust that will leverage current VMM research while addressing inherent vulnerabilities. The second objective is to leverage current research in DDFA and PC and investigate providing a secure channel between the hardware hypervisor/VMM system and running applications to close the semantic gap between the VMM and individual applications. If feasible, the combination of these component technologies will implement security at a system level that none of the approaches can achieve individually. An output of the project will be a demonstration scenario that will illustrate the power and effectiveness of our approach. This demo, and the associated publications, will be used as a tool for developing external interest in the associated technologies.

Accomplishments - The research team implemented a secure hardware hypervisor system with several important characteristics. It is largely transparent to the host system, is isolated and independent of the host, has out-of-band reporting mechanisms to avoid interference from malicious activity by the host, and has unrestricted access to system resources to maximize visibility and control of the host platform. These capabilities were achieved by modifying an open-source processor design to provide monitoring and control interfaces to an external coprocessor. Working together, these hardware components implement the hypervisor functionality. Additionally, the hardware hypervisor team completed initial integration of the hardware hypervisor into a policy-driven security enforcement system. This system demonstrates the combination of a true hardware root-of-trust and automated security policy enforcement. The hardware root-of-trust capability will significantly enhance automated enforcement of security policies and provide increased resistance to malicious attacks. The research team has also completed the initial DDFA/PC integration and has successfully demonstrated operation of a combined DDFA/PC system. The team has completed the initial integration of the low-level virtual machine (LLVM), and has added modifications to the parser and instrumentation engine to enable support for insertion of new function calls for functions that have been annotated within the policy language file. Programming hooks have also been added to the system to enable a full demonstration of DDFA/PC capabilities.

2010 Program Home